Exploratory testing is learning, designing, and executing tests simultaneously. AI can enhance it by automating the process and improving test coverage.
TABLE OF CONTENTS
I first heard the term exploratory testing at the Agile conference, probably in 2005 or later. I remember sitting at a table with Brian Marick, who explained agile testing to us (Ultimate Software attendees).
I remember initially being curious about exploratory testing, but I found the term quirky. Before explaining why I reacted this way, let’s understand where the term comes from and discuss some criticisms.
There is general agreement that the first time the term appeared in print was in the “Testing Computer Software” book by Cem Kaner, Jack Falk, and Hung Quoc Nguyen.
Finding the term doesn’t take long in the book's second edition. On page 6, you can see the following two sentences:
The reason you need to conduct exploratory testing, according to this book, is stated in this sentence: No matter how many test cases or how many types you’ve created, you will run out of formally planned tests. In other words, you will never be able to execute all the tests you want, but after you finish your planned tests, you should keep going and continue testing (exploratory testing).
In 2003, Cem Kaner and James Bach collaborated on a paper titled “Exploratory Testing Explained.” Bach's definition of exploratory testing is the most widely used: Exploratory testing is simultaneous learning, test design, and test execution.
If you know Kaner and Bach, then you know exploratory testing comes from the Context-Driven school of testing. I covered the different schools here if you want to familiarize yourself with them.
While the agile testing school is most famous for its embrace of test automation with practices like Continuous Integration (CI), Test-Driven Development (TDD), and Behavior-Driven Development (BDD), it also includes exploratory testing.
One of the most comprehensive exploratory testing books is Elisabeth Hendrickson’s “Explore It!” Elisabeth Hendrickson is a well-known agile thought leader who is obsessed with testing. In her book, she modifies Bach’s definition of exploratory testing to simultaneously designing and executing tests to learn about the system, using your insights from the last experiment to inform the next.
I used "practice" to describe exploratory testing, but others use other terms, like "style," to refer to it. In the agile world, practice is usually more common, as agile software development is a set of principles, practices, and rituals teams follow. Also, exploratory testing falls within a specific quadrant in Marick’s test matrix. Here is Marick’s matrix for reference:
Exploratory testing falls in quadrant 3, the business-facing critique of the product. As you saw from the introduction of the concept, exploratory testing is manual testing with no defined test plan. It requires a product you can interact with, so it falls within the third quadrant.
While theoretically, you can conduct some fourth-quadrant tests in an exploratory fashion; this practice is mainly for functional testing. Explanations of exploratory testing include references to free jazz (in case you wonder why I think it is quirky).
Generally, it requires the following:
Many agile teams conduct exploratory testing sessions as part of their agile testing process. Whether they do it well or not is a subject for another day. The practice has detractors. In Part 2 of this series, we’ll review some of the criticisms.