The Death of Agile Through Repackaging of Ideas

Warning: this blog post is a little bit of a rant.

Usually, I write blogs about software testing and the emerging category of autonomous testing. This blog post is a little different. I will talk about testing, but in the context of a concerning trend I see in the software industry. I called this trend the repackaging of ideas.

Agile Origins

These repackaging events are at the core of the death of the agile movement. Many years ago, agile software development represented a significant rethinking of the software development process. On September 22nd, 2001 edition, The Economist magazine published an article titled Agility Counts. The Economist started the article with the following paragraph: In the latest of our series on managing innovation, we look at agile programming. This is the culmination of many faddish ideas for producing software more efficiently. But behind it lies a healthy emphasis on the virtues of teamwork in a business plagued with prima donnas.

The Economist wrote: 17 leading software gurus holed up in a Utah ski resort last February to produce a “Manifesto for Agile Software Development.” Portentous as it may sound, the manifesto represents the distillation of several successful team-oriented techniques, and should inspire innovation groups outside the confines of software development.

Here is the link to the manifesto. The following is directly from the manifesto:

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

The Economist magazine highlights some of the critical aspects of agile in these two sentences: All the manifesto's signatories believe in rapid and frequent releases of partial solutions, so the customer can react along the way to a final product. Indeed, a key feature of being agile is to integrate the customer into team discussions.

To release frequently, you have to work small. You can no longer develop the application in a year. Agile development prefers shorter iterations. Scrum, the most popular agile framework, prefers two weeks. You must rethink many aspects of developing software to release even a partial product every two weeks. The first one is that you need cross-functional teams. Remember, you have to go over the entire development process in weeks.

However, as The Economist magazine states, in some ways, the software world is a latecomer to the notion of a team-based innovation that is both extremely responsive and highly adaptive. Similar agile concepts have been around in manufacturing for years. Ironically, nothing in recent times has come as close to demanding total agility as the blistering pace of developing products for the Internet.

The Economist indirectly refers to the influence of lean manufacturing and explains why this change was needed in software development. The arrival of the Internet forced us to rethink how we develop software. It also made observing and interacting with customers more accessible, enabling one of the agile values regarding customer collaboration.

Agile Impacts

Agile had a tremendous impact on software development. Extreme Programming (XP) brought a set of engineering practices that many non-XP teams follow today. Scrum brought project management practices that many teams use today.

Here are some examples of the early impacts of agile in the world of software development:

• Two-week sprints (instance of a timed-box iteration)
• Small releases
• Continuous Integration (CI)
• 15-minute daily standup meeting
• Test-driven development
• Bi-weekly sprint demo
• Sustainable pace
• User stories
• Acceptance test-driven development using FIT
• Cross-functional teams

Even more important than many of these principles, values, and practices is that agile permits us to innovate and challenge established views, even if they come from agile frameworks.

Specific agile ideas never caught on, or we found them to be bad ideas and abandoned them. Others were replaced with better options.

Some examples are:

• System metaphor - a practice that never caught on
• Estimation using story points and planning poker - bad ideas many teams abandoned
• Acceptance test-driven development (ATDD) replaced by behavior-driven development (BDD)
• Kanban re-thought agile from first principles, going back to Lean thinking
• Kanban removes the need for timed-box iterations
• Measuring lead time, cycle time, and throughput instead of velocity and tracking burn-down charts
• Using Montecarlo simulations to forecast release dates
• New ideas regarding the design of new systems like Domain Driven Design (DDD)
• Expanding CI to CI/CD (Continuous Delivery/Deployment)
• The emergence of Lean UX and JTBD in software development

In the first decade of agile development, innovations came from many different sources. Attending the Agile conference was great during those days. At the same time, the number of companies and people trying to make a name for themselves inside the agile movement kept increasing. People always followed the money, and consultancies were not far behind.

To differentiate yourself, you need to develop something “unique.”

Here is when the repackaging started.

DevOps - The First Repackage

I am about to commit blasphemy in the eyes of many people in the industry. I promise it is not the last one. In the late 2000s, a group started discussing their companies' organizational issues. Someone noticed we are agile in software development but not anywhere else. The first prominent place was the IT and Operations teams supporting the production environments. The interaction between development and operations became the first apparent misalignment.

As such, the name DevOps came up. We need to bring Ops into the Agile picture. Some even suggested we were going beyond Agile with DevOps. These people did not listen closely during the talks about having cross-functional teams.

If you use DevOps to describe the evolution we were already experiencing with the emergence of cloud computing and CI/CD, nothing is particularly problematic. DevOps is a packaging of Agile/Lean with CI/CD pipelines on the cloud. It does bring to the attention of those not there yet that you need to extend your value stream (Lean concept).

Instead, DevOps became something to sell. You must change your organization and people’s titles because they are now DevOps engineers. If you genuinely understand DevOps, you know the idea that you have someone with that title makes no sense whatsoever.

The only new item within DevOps is Infrastructure as Code (IaC). While automating configuration management is not new, cloud computing platforms like AWS made it an essential component and transformed it into something much more complicated.

If you want proof of the repackaging hypothesis, you must examine a theme's many variations. You have DevOps, DevSecOps, ProdOps, etc., each with its vendors. These are all obvious to anyone who pays attention to the principles of Agile and Lean. But we need some new buzzword to sell you.

SAFe - The Mother of all Repackagings

When historians examine the beginning of the agile movement's decline, they will point to the emergence of the Scaled Agile Framework (SAFe). While I still try to understand where the “e” in the acronym comes from, you must admire the shameless marketing. Who doesn’t want to be “SAFe”?

SAFe is the most shameful event in the history of agile software development. It claims to resolve the problem of scaling Agile. As someone who scaled Agile at Ultimate Software, I can tell you that SAFe is a total sack of manure. You do not need it.

SAFe is a collection of ideas stolen from others without permission and put together without understanding. Its few original ideas are simply terrible. Please do not take that train. It takes you nowhere.

I do not want to spend too much time discussing this monstrosity. You should only know that if your company uses or implements SAFe, it is because your technology executive team is not very good. You should find another job in a better place. Also, you should sell the stock of any company using SAFe. 

The best way to scale agile is to try not to scale it. I know it sounds counterintuitive, but you want to keep it simple. Instead of implementing this bureaucracy, try creating your framework based on the principles of Lean and Agile. Everything you need is already there. You might need to create some new meetings, but let them be the ones that work for your context. Start small and only add something when it hurts. The meeting will reduce the pain until you can break that dependency.

The best path to scaling is to remove dependencies. Focus primarily on finding and eliminating dependencies until every business unit is as independent as possible. Believe me, there is no need for trains. Read Prateek Singh’s book. It will help you figure out how to scale.

Quality Engineering - The Testing Repackage

First, an established quality engineering practice has existed in the manufacturing context for a long time. Thanks to Lean’s influence, the ideas of people like Demming regarding quality have been part of Agile from the beginning, even if some of the original signatories did not know it. This section is not about quality engineering, as understood in manufacturing.

This section is about repackaging what we already have in Agile. I have spent much time looking at quality engineering blog posts to see if they can tell me what it is. The fascinating thing is that when they try to explain it, they end up with a list like the following:

• Embracing Shift Left
• Continuous Integration
• Test-driven development (TDD)
• Agile test automation strategies 
• Agile metrics

That is a list of quality engineering practices. I am serious. Is there anything new here? Shift left is suggested in the agile testing quadrants defined by Brian Marick. Continuous integration is an Extreme programming (XP) practice as it is TDD. The paragraph about test automation strategies does not tell you anything about strategy, but I will be generous and assume they meant the agile testing pyramid or agile testing trophy. Finally, what agile metrics are they referring to? The article mentions MTTR (Mean Time To Recovery). I guess this person read some articles about DORA metrics, and now he thinks MTTR came from Agile. It did not. It is a lot older than Agile.

Now, a different blog post has this list to tell you what a quality engineer does:

• Collaborating with clients, customers, and the business to understand their product and service quality criteria
• An attitude, culture, and approach of checking at every step, from ideation to production operation
• Not only developing great testing skills, but also understanding and implementing quality skills and practices
• Defining and implementing quality management approaches within teams and across teams of teams

I am not joking. These are some of the lists defining quality engineering or the quality engineer role.

Why do we need a quality engineer role? Well, we do not need it. If you read that list carefully, you will see that many people in QA used to do this job, and many people in management will do it, too. I did this job as a director of quality assurance and later as a vice president of software engineering.

At least the title Quality Engineer is not new, so using it is not a big deal. Just be careful of any vendors or consultants trying to sell you the same car with a new paint job for a second time.

Who is responsible for all these repackaging efforts?

The first answer you will get from people is the vendors. Some consultant or software company is trying to sell you something, and they devise a term to successfully market this new “offering” to your clueless executive team. That sounds simple, but it is not entirely true.

Vendors cannot sell you something that does not answer a need. Agile software development is revolutionary and interacts with many companies' existing organizational structures. If you are building cross-functional teams, do you need managers for each practice? Do you need a product management team, a UX team, a software engineering team, a quality assurance team, or an operations team? Do you need leaders for each one?

Many new companies avoid having all these roles and teams. Instead, they have engineering teams that handle the work independently. Many new teams no longer have dedicated testers or operations staff, which is a scary proposition for many technology leaders.

Confronted with this new reality, technology leaders jump on the buzzword bandwagon and tell their CEO they need to hire DevOps engineers for this and Quality Engineers for that. And do not forget the SREs (Site Reliability Engineers); we need SREs too. And do not get me started on SAFe RTEs. Do you need these roles? No, this is how crappy management hides their lack of capability.

In the end, remember: Agile is dead! Long live Agile! Go back to the source of the Agile Manifesto and apply Lean thinking, and you can resolve your problems without buying buzzword-driven pre-packaged solutions.