It hasn’t been uncommon in the software engineering world to hear proclamations of the death of Agile from people with a vested interest in the methodology, claiming that unless we show greater belief, it will be lost, and in cases where it fails, it merely hasn’t been implemented properly - reminiscent of other claims following the “no true Scotsman” fallacy like “real communism has never been tried.”

Despite its presence in software catastrophes varying from miscarriages of justice to aviation software disasters, the reality is Agile has held a position as somewhat of a professional religion of software engineering.

This situation has profoundly changed over recent months. Since I worked on research 6 months ago showing that Agile software projects had a 268% higher failure rate, a plethora of research has come out corroborating this work.

Like communism or faith healing, the fantasies of grandeur that universally good, but unrealistic, solutions can solve all our problems will always appeal to some notwithstanding the evidence - but what has now fundamentally changed is that there exists informed consent as to whether to follow Agile or do something different. ‘

(No doubt, the international news associated with the CrowdStrike botched software update certainly helped hammer home the points I was trying to make.)

Notwithstanding the repugnant extent some in the Agile community went to attempt to silence me from talking about Agile, it’s important to be magnanimous in victory so I won’t seek to relitigate the arguments here - instead, I feel it’s worth discussing recent developments and the road ahead, 6 months on.

Google’s DORA Research Turns on Agile

J.L. Partners, who I commissioned to conduct the Agile failure rate research, have had a phenomenal past few months. Unlike many others who conduct research with limited accountability, they regularly test their models by predicting elections. When it came to the UK elections, they found themselves producing amongst the most accurate predictions. Things are even more remarkable when it comes to the US election as Politico reports:

… J.L. Partners may well end up being among the most accurate in their final pre-election predictions. The firm’s final model was just one of two to forecast a Trump victory and it also projected him winning the Electoral College 287-251 — the highest projected Trump-winning margin of any pollster. It was also one of very few pollsters to predict Trump would win the popular vote.

Additionally, research from RAND Corporation initially conducted for the US Department of Defense has found that AI development and agile don't mix well. Furthermore, Synodus (a global technology provider supplying (Fortune 500 companies like BOC Aviation, KPMG, and Unilever) is reporting that by moving away from Agile, they are finding they’re delivering projects 2-3x faster and reporting a leading airline saved 63% of their development costs.

This is despite the fact that speed and cost are traditionally metrics associated with Agile and LEAN, whilst the Impact Engineering approach focuses on Impact as a primary metric.

Finally, as Colm Campbell has written in the article “P-Hacking with Dinosaurs,” the claims put forward by the “Agile Mafia” in response to the Agile failure rate study have been comprehensively disproven, leaving only pathetic personal attacks.

However, perhaps one of the most surprising areas of supporting research comes from Google’s DORA team. DORA itself finds its roots in the DevOps movement, which itself a creature of Agile. Their annual State of DevOps Report for 2024 hasn’t received any particularly widespread coverage this year given the airtime that has gone to Agile research, but it is interesting to see they are turning their backs on Agile too.

Now, I should note that I've been critical of the DORA team’s methodology (since I changed my mind on Agile, DevOps, etc.) as the primary metrics they use to measure things are focused on speed of delivery (which we know isn't what users want), but the fact they're seeing this even using their approach points to something deeper going on here.

On page 64 of the State of DevOps Report 2024, they claim:

‘The Agile manifesto advocates for “working software over comprehensive documentation”. We continue to find, however, that quality documentation is a key component of working software.’

Furthermore, on page 82, they appear to turn their back on DevOps itself:

'We are committed to the fundamentals principles that have always been a part of the DevOps movement: culture, collaboration, automation, learning, and using technology to achieve business goals. Our community and research benefit from the perspectives of diverse roles, including people who might not associate with the "DevOps" label. You should expect to see the term "DevOps" moving out of the spotlight.'

I still have issues with their research methodology; the endpoints used to measure success are focused on speed, and furthermore, they continue to advocate for transformational leadership despite the psychological research in this area questioning such approaches.

At a human level, it seems remarkably crude to focus so heavily on attempting to transform organizations and people without their informed consent whilst also failing to recognize how there is often no better source of learning than our own mistakes (to the extent we have the informed consent to make them), a key criticism I have of work like The Pheonix Project and The Unicorn Project.

However, it is remarkable nevertheless to see increasing corroboration for Impact Engineering against traditional Agile and DevOps metrics.

Scrum, TDD, and Design Patterns

When The Register interviewed me a few months ago in the article “Study backer: Catastrophic takes on Agile overemphasize new features”, I made no secret of the three factors I had issues with in modern approaches to Agile, DevOps, and digital transformation:

1. Approaches of Agile which deprecate requirements, despite disasters including the Post Office scandal and Boeing 737 Max 8 crashes.

2. Implementations of DevOps which prioritize speed of delivering new features and recovering from issues as opposed to preventing problems in the first place - despite research showing the public rank getting the latest features as quickly as possible as the least important thing for them when using computer systems, with data security, data accuracy, and no serious bugs being the most important factors.

3. Attempts at bottom-up organizational transformation without obtaining informed consent, neglecting the importance of allowing people to learn from their own mistakes they have the informed consent to make - with such programs sold as sure-fire successes overwhelmingly ending in stressful misery and failure.

As I began to research software disasters and killer computers (as reported in Forbes back in April), these were the three factors which I began to have increasing moral challenges with. I had no real issue with different software development approaches or project management techniques as I didn’t see the same link to the disaster case studies I was investigating. I didn’t feel the need to express my personal views as an engineer with respect to these other matters.

Nevertheless, it is interesting to see the blast radius has widened beyond these initial three areas to other approaches advocated by the Agile Manifesto co-authors such as Scrum, Test-Driven Development (TDD), Object-Oriented Programming (OOP), and Design Patterns (e.g., see Soumen Sarkar's LinkedIn post on Design Patterns).

I have little to add in this area. However, I do wonder if the Agile co-authors hadn’t fought so hard against its death as a religion if the software engineering community would have been more kind to their other ideas. The question is now moot given Agile’s religious status is rigor mortis and everything seems up for grabs: OOP, design patterns, TDD, etc.

Towards Loss Aversion Awareness

Since having spent a relatively considerable proportion of the past few months being front page news of various tech publications talking about Agile and being very widely discussed amongst the community generally, it would perhaps be unsurprising to have some bizarre encounters. For the most part though, they’ve been exceedingly pleasant encounters of people coming up to me in public and randomly engaging in discussion on Agile.

However, there was one encounter which left a stark impression on me. A few weeks ago, I was going to the UK Parliament to address policymakers on various tech scandals I was working on investigating. I’ve never been much of a fan of holding power myself, and I’ve always found it quizzical that in the private bars in Parliament, there have to be signs reminding individuals not to abuse their power when intoxicated.

However, giving this talk, I was not only well dressed and more presentable than usual, but I was also in front of a sympathetic audience, flanked by a Member of Parliament and a friend who happens to be a King’s Counsel (a senior lawyer granted that distinguishment by the monarch for exceptional advocacy).

Once I gave my talk and answered a number of questions, there was someone in the audience who was a visitor who began talking about Agile (the talk hadn’t been about Agile). In contrast to my three-piece suit, he was dressed in a ketchup-stained shirt and (without going into details) was neither presentable nor politically connected. He began “Can I give you some feedback?”

When he started talking about Agile, I made eye contact with him quizzically, and he looked down and stopped talking, almost as if he realized he was no longer online and actually facing someone with his words. Before I could respond, a CEO friend interjected to address the point for me.

At that moment, I realized I didn’t like power to an extent I have not realized so far.

Unfortunately, for this individual, it seemed he had already been a victim of those selling surefire transformation solutions apparently leading to difficulties in his own life. In these circumstances why are these transformation methodologies still advocated?

More broadly, why is Agile unable to operate in the real world? With communism, at least we can attribute the failure to greed, so what is the psychology of Agile failure? The answer, in my eyes, comes down to a psychological factor known as loss aversion. By popular demand, I recently wrote a pre-print paper on the Impact Engineering work and it focussed on the role loss aversion awareness can play in mitigating project success.

The paper, “So Am I Dr. Frankenstein? Or Were You a Monster the Whole Time?”: Mitigating Software Project Failure With Loss-Aversion-Aware Development Methodologies, concludes as follows:

Case studies have shown that software disasters can snowball into catastrophes when problems are not addressed and are instead covered-up. Psychological factors, a key being loss aversion, can incentivise organisations to go down this path despite the disastrous consequences.

Yet, software methodologies have historically disregarded this psychological factor, instead assuming that humans will want to address problems at the earliest opportunity. Previous research of varying methodologies have found the psychological safety to raise and discuss issues to be key to improved software delivery. However, it is no doubt optimistic to believe such significant cultural and psychological change can be achieved in all organisations. As we find, even between the UK and USA, psychological safety remains the biggest difference in engineering practice between the two countries.

In this paper, we focus on a different way forward. Instead of attempting to achieve a cultural change that is at best challenging, we instead focus on a more pragmatic approach: limit the triggers of loss aversion from the outset through robust requirements and then aim for psychological safety where it is needed. In our research, the sole factor we found that would increase the success rate of a project more so than psychological safety was having clear requirements from the outset.

Against our hypothesis, the results that merely having clear requirements, even when they change late in development or are not aligned with the real-world, seems to have a significant role in software project success. This suggests that having a gate before the start of a project to discuss and address problems, when loss aversion is at its lowest, allows for the greatest improvement in success rates.

The song Dr. Frankenstein by RedHook contains the refrain: “So am I Dr. Frankenstein? Or were you a monster the whole time?” For catastrophic software projects, this paper contends that software projects become disasters when humans fail to address smaller scale technical problems. Existing software engineering methodologies fail to address that humans are not machines and psychological factors impede the ability to address problems. Having identified this anachronism in existing methodologies, this paper presents how we can act upon it by using requirements before the start of a project to provide the opportunity to address problems when loss aversion is at its lowest.