Let's use the Marick Test Matrix to answer the question, Is software quality assurance good enough? Because the answer is no. But there's a path forward.
TABLE OF CONTENTS
So, Now What?
In part 1 of this series, we asked, "Is software quality assurance better than it was 20 years ago?" The answer is yes. Then, we analyzed whether it was good enough. I ended Part I by saying, “We can address all three points – what testing is, why it’s valuable, and why it’s so complex – by returning to Brian Marick’s four quadrants model.”
The Four Testing Quadrants
The advantage of this model is that it allows you to go over all the aspects to build a comprehensive strategy for testing software.
At the top of the matrix, you have Business Facing, which refers to tests that deal with the business domain. Any business expert will understand these tests. At the matrix’s bottom, you have Technology Facing. Per the name, the focus here is on the technology or the verification of the technical implementation. Support Programming is at the left side of the matrix. This side includes the efforts that support the development of the specific product. On the right side, you have Critique Product. These are the tests trying to break things (the critique). Check my blog post here for more details on Marick’s quadrants.
This matrix forms four distinct quadrants. We must work with our stakeholders to define what quality means for us as an organization. That takes you to Agile testing. You need to decide what you are going to do for each quadrant. Before Agile testing, most organizations only focused on quadrants 3 and 4.
Have we turned too far left?
The impact of Agile testing brings a new question: Have the QA changes, shifting responsibility to software engineers and product managers (quadrants 1 and 2), helped push QA towards outsourcing?
The trend could happen anyway due to the focus on the right side of the testing quadrants in many QA organizations. This observation is the second issue I see with the modern view of software testing. We lived in quadrants 3 and 4 for a long time, then Agile testing came, and we de-emphasized 3 and 4 and focused our attention on 1 and 2.
Refocusing on supporting programming was a good step forward. It was essential. Still, many experienced QA people noticed and fought the transition. For some, it is the normal reaction to change. Others because they knew you must critique the product. Like all ideas, Agile testing got co-opted and forced to mutate into something else. Suddenly, most teams only focus on the first two quadrants. I worked with an Agile consulting organization that insisted you only need engineers doing TDD. The unit tests will cover everything—the evil mutation of the testing pyramid.
“The future is already here. It’s just not evenly distributed yet.” (Maybe William Gibson)
This kind of post has a persistent flaw, which we discussed in Part 1 of this blog. Using the experience at Uber or similar Silicon Valley organizations paints an ideal world. Less than 20% of the teams live in that idyllic company. I have been lucky to work on companies focused on continuous improvement and building a quality culture, like Ultimate Software. However, most companies do not have this culture, let alone the interest in implementing the changes necessary to achieve it. In those companies, the future has yet to arrive.
QA Continues to Stumble in a Dark Room
Yes, QA has improved in the last 20 years of Agile development, but we forgot about the value within the right side of the matrix. We need to continue to critique the product. We need to do more than just support programming to build great software. We need better odds than 42% to achieve a successful software project.
At Testaify, we recognize that many improvement opportunities exist in software testing. That is why we are building an AI software testing platform focused on continuously critiquing the product. We want to help you Shift Left while we take care of the right side. We want to accelerate the most time-consuming testing. We want the future to be evenly distributed. Sorry, William Gibson.
About the Author
Testaify founder and COO Rafael E. Santos is a Stevie Award winner whose decades-long career includes strategic technology and product leadership roles. Rafael's goal for Testaify is to deliver comprehensive testing through Testaify's AI-first platform, which will change testing forever. Before Testaify, Rafael held executive positions at organizations like Ultimate Software and Trimble eBuilder.
Take the Next Step
Join the waitlist to be among the first to know when you can bring Testaify into your testing process.
