Today’s technology news feeds are full of articles claiming software testing is dead. According to these stories, companies are going Agile and testing will be done either by developers or testing robots. On the flip side, there are just as many stories that report epic software failures and their astronomical resulting financial losses. And what’s the first thing everyone says when they hear about a failure? They should have done more testing!
Clearly in the digital economy quality and testing are more important than ever. To keep pace with change, companies are adopting Agile and testing is becoming the responsibility of everyone. The result of this trend is that more testing is being done in development. But does this really negate the need for Quality Assurance teams and testers? I think the answer boils down to three key factors:
What are you testing?
One of the big drivers for shifting testing from QA to development is the adoption of Agile. But consider the origin of Agile. It’s a methodology that originated to help developers create code faster, in parallel. Each developer is assigned a single story for delivery in a two-week sprint. But what happens when there is no development, for example, in the case of an SAP transport? Who writes the test script when there is no story or code to test against? Without a testing team, this responsibility falls back to the business. As a result, business users are pulled away from more productive work to document existing processes and run manual/exploratory system tests. When they find a defect they start the tedious process of documenting, taking screen shots and submitting to the supporting IT team. All of this detracts from business users’ primary jobs – and reduces the value they bring to the company.
What types of tests need to be run?
For a single feature or application, unit, component, and functional tests may be run by development. But what happens when the new feature or update is part of a larger ecosystem that stretches across multiple applications in the cloud and on premise? Who is responsible for building regression test libraries and running end-to-end tests to ensure downstream systems aren’t impacted by change? Documenting these complex processes alone can take weeks. Effective testing also needs input from multiple groups who may not have visibility into the entire process. And then there’s the growing demand to run these tests more frequently – monthly, weekly or even daily. Who is responsible for maintaining automation and ensuring cross-functional documentation is kept up to date when tests are updated?
What are the risks?
Many industries operate under strict legal regulations. System changes must be carefully documented and meet audit requirements. In a presentation given by Southwest Airlines on SOX compliance, they discussed how they have 45 automated controls within SAP, comprising 58 different documents. Each quarter, these controls have to be validated by capturing the document and comparing it to expected results. Work this significant and detailed clearly requires a centralized team. In addition to legal risks there are the risks of a catastrophic failure, including major loss in revenue and negative public perception. The Starbucks register malfunction which caused 60% of stores to close in the US and Canada is just one of many examples that has taken place in the last few years.
The New Role of QA – Make Sure the Business Works. Not Just the Software.
It is true that as more organizations adopt Agile methodologies for the deployment of custom applications, more unit/component/functional testing will continue to “shift-left” to developers. But this doesn’t eliminate the need for centralized change management and QA teams. Application landscapes are becoming more complex — not less. It’s not about any single piece of software working. Today, it’s about the business working. Organizations are increasingly turning to packaged applications like SAP, Salesforce.com and Workday to manage underlying business processes. This allows development teams to focus their efforts on custom applications that touch the customer directly. Change management for applications like SAP requires a higher level of orchestration and management. This simply can’t be achieved in an ad-hoc fashion, or using Agile scrum methodology. The adoption of Scaled Agile Framework (SAFe) is one way the role of QA teams is quickly evolving. Whatever the approach, one thing is for sure – as long as organizations continue to rely on complicated networks of software to get business done, QA has a vital and important role to play!