Still waiting for test data?

Oh… the irony of the digital age for testers. All that data... but still no data for testing when you need it. Why does test data continue to be one of the top challenges for testers across the globe?

“It doesn’t matter how effective a test process is or how much effort you spend on test automation if you don’t feed testing the right data.”

 Forrester Research, Agile Test Data Management: The New Must-Have

With all the hype, it’s no wonder that more and more vendors are jumping on the test data creation and management bandwagon. Vendors claim they’ve got the solution for all your test data problems. If only it were true! Unfortunately, when it comes to TDM for testing end‑to‑end business processes, there is no easy fix. Simplistic vertical/functional TDM solutions fall short when it comes to meeting the complex data needs of testing end-to-end business processes that traverse multiple applications and have numerous data dependencies.

Not all Approaches to Test Data are Equal

Today, there are many ways you can generate test data, and there is no shortage of test data management tools on the market. But not all approaches to test data are equal, and not all test data tools are test data management solutions. In the White Paper “Enterprise Test Data Management Practicum vs. Marketing Hype”, I take a look at the pros and cons of the following approaches to Test Data Management which I’ve summarized in the table below:

Approach Pros Cons
Production Data Replication
  • Easy access to data
  • Still a manual process
  • No coverage for negative test cases or new systems
  • Risk of sensitive data being exposed
Data Masking vs. Synthetic Data

 
  • Prevents sensitive data from being exposed
  • Once models are built data can be auto-generated quickly
  • Mapping dependencies and correlating data across multiple systems of record is hard
Behavior-Driven Development (BDD)
  • Simple approach to mapping test data to test cases
  • Manual process
  • Prone to human error
  • Does not create data in system under tests
Pairwise-Based Matching
  • Good for single application functional tests
  • Challenging for end-to-end tests which require data to map-to and exist-in back-end supporting systems
Automated Self-Sufficient Test Cases

 
  • Automated approach to generating test data for complex end-to-end tests
  • Requires tests to be created to generate test data

With so many different tools and methods in the market, how do you choose the right one? By now, you’ve probably realized there is no simple answer to this, especially when looking for a solution to test complex end-to-end business processes and systems like SAP that involve a mix of both web-based, traditional Windows-based UIs, and mainframes.

What should you look for in a TDM solution? For that answer you’ll need to read the white paper!