Testing is broken.

TABLE OF CONTENTS

• The cast (solution) is on the wrong bone (problem).
  • The issue isn't test speed; it's quality.
  • Testers are overwhelmed.

The cast (solution) is on the wrong bone (problem).

I love testing, but at the same time, I know testing is broken. Throughout software development history, testing has always been a bloated mess that needs to be stabilized and fixed.

The problem is no one cared to build the proper foundation. Unlike software engineering, no university or professional programs grew to support a testing career. Whatever knowledge testers acquire is typically learned by using the software, on the job, or through department-developed training programs. 

Back in the day at Ultimate Software, some of our best testers were customers that we hired! They were system thinkers who knew the product well. When we released software with bugs, they complained because they cared about what we were doing. Eventually, we hired them to test our products before release. These amazing people weren’t developers. They were payroll, benefits, and HR administrators embarking on a second career. It was the same story at another company I worked for: our customer support team members became our testers. 

Since the coursework doesn’t exist, it’s fair to say that exceptionally few people are adequately educated on software testing. As a result, the percentage of people who can test very well is limited. For a long time, the industry didn’t care. Eventually, test automation vendors saw an opportunity: sell testing speed to companies struggling with testing! Here is the magic bullet! But testing faster doesn’t matter if you aren’t testing the right things. You’re putting a cast on the wrong bone. 

The issue isn't test speed; it's quality.

The core problem isn’t how fast you’re running tests; it’s writing tests that find the defects affecting product quality. Testing is a risk management exercise. Right now, it’s impossible to test everything. You have to choose: What are the most important things to test? The average tester knows one or two testing methodologies. An excellent tester knows a few more. Limited knowledge combined with limited time cannot result in high product quality. 

Here’s an example: One test methodology is boundary value analysis. Many software defects are hidden here. Let’s pretend a city has a specific sales tax for items above $900, so the boundary is $900. I asked one of our best testers to create the test cases for this tax rule. She was the company’s best, most respected tester, and she got the boundary value test cases wrong because you have to remember you are dealing with currency. You need to consider decimal places. Was she ever educated on software testing? Of course not; she was a payroll administrator. If the first test is poorly designed, how can future tests be any better?

With each new set of variables, testing requirements inflate exponentially. Software testing is genuinely a bloated monster. We can’t solve testing at a societal level because there are no degrees you can earn to be a software tester. There’s a limited career trajectory. And no matter how you slice it, testing is not sexy. It’s something everyone needs, but only a few truly understand. How often does a clueless boss ask the testing team to test everything? Are you serious? Do you know what you are even asking?

Testers are overwhelmed.

Meanwhile, testers are doing their best but are overwhelmed. There’s never enough time to do the testing they want and need before release. Then, they have the “privilege” of taking the fall for undiscovered defects that irritate users. Executives tend to forget that the people testing did not write the code.

Until now, the software development industry has thrown people at testing, hoping it will improve, like outsourced testing, which eats up billions of dollars of value. We’ve also thrown tools at testing, hoping it will improve, like UI test automation and test case management. These tools consume millions of dollars and give testers and developers an incredible amount of busy work to maintain and revise test scripts. These tools are the biggest scam in software development, but that is a subject for another day.

Let’s get back to the core. We want you to stop praying for quality. Let’s heal testing once and for all by addressing the source of the problem. Why? Because testing can be fixed. Let’s go back and focus on testing and, in doing so, change software testing forever. Testaify is that fix.