I’ll come clean, that title is a bit like ‘Free Beer Here!’ or ‘Build the Wall!’ It’s intended to draw you in and provoke a response, without necessarily having any basis in rationality.

I’m certainly not about to tear into Agile and start advocating a return to the dark days of interminable waterfall projects and questionable customer value.

Agile is here to stay, at least until we hand it all over to the machines themselves and they can come up with their own methodology. That’s a good thing.

But I do think we sometimes take Agile too far.

In this post, I argue that seeing Agile as a panacea risks us stretching it into use cases where it’s not appropriate.

We risk applying it to situations which are too complex, poorly understood, badly architected or fault intolerant for it to succeed.

What got me thinking about this was an ex-colleague’s LinkedIn post sharing Fred Brooks’ great quote from ‘The Mythical Man Month‘ – ‘Nine people can’t make a baby in a month’.

It prompted me to come up with a much less pithy corollary:

‘Agile can’t make 0.025% of a baby in a one week sprint.’

There are some requirements which Agile simply doesn’t fit, and you just have to follow a process which delivers the whole thing at the end.

When I first started learning about Agile all those years ago, I noticed that everything I read pretty much slated the waterfall methodologies it replaced – waterfall was painfully slow, it didn’t deliver customer value, it was bad at managing change, etc. etc.

All true, of course, and a necessary part of making radical change stick.

If you want to revolutionise something, you need to start by attacking the status quo.

If you let people start thinking about the good points of what you’ve got now, you end up like the guy in ‘Life of Brian’, asking what the Romans ever did for us and instantly regretting it.

That’s why you’re unlikely to hear an electioneering politician say ‘Actually, what the other lot did with the economy was great, we’re having that!’, or a barnstorming new CEO talking about their company’s legacy in glowing terms.

The trouble is that sometimes the old way isn’t inherently wrong, it just focuses on the challenges which were important back when it was still the new way, like complexity and risk.

Other challenges, like time to market, get less focus, and when these get too onerous, a new approach is needed.

But if the new way ignores the old challenges, they can still come back and bite you. When they do, you may be able to modify the new way to accommodate them, or at worst you may have to come up with a new ‘new’ way, start explaining why your old ‘new’ way was such rubbish, and begin the whole cycle again.

This is an appalling oversimplification of what Karl Marx called dialectic and a nameless philosopher called ‘what goes around, comes around.’

Agile replaced the inertia and IT introspection of waterfall methods with a new and essential focus on customer value, speed to market, and evolving what you’re building as you build it.

Waterfall was much more about managing complexity and risk, and understanding exactly what it is you are building before you lay the first brick or code the first line.

It’s sometimes forgotten that waterfall methodologies were necessary for their time because there was relatively little IT, and stuff like systems architecture and re-use were in their infancy. Worse, our system development tools were inherently slow and error-intolerant.

All of this made a risk averse approach pretty much essential.

When I started in IT in the early 1980s, we coded in pencil on coding sheets, booking slots on a green screen terminal to enter our code and submit it to compile. The results, which invariably punished us for missing a critical semicolon, would come back from the line printer the next day.

Luckily we didn’t do daily scrums, as our product owner might have got tired of conversations on  ‘My pencil broke again, can we get a sharpener?’ and ‘Let’s celebrate success, I only got two errors on yesterday’s compile!’

On top of this, we often had to build interfaces with applications which were ‘legacy’ even in those days, many of which were poorly documented and not designed to play nicely with other systems.

And we tended to be writing the heavy lifting code for core business functions like payroll and engineering, because SAP was still something you got from a tree and ERP was just the noise you made after necking a Fanta.

There was a lot of inherent complexity in these environments, in the sense that they had a lot of moving parts and interfaces.

There was also a lot of accidental complexity (euphemism for ‘mess’) as the underlying business processes weren’t formalised or well understood, and the IT systems we had to interface with were often not well documented or architected.

And because we were dealing with core business functions, getting it wrong could have a major impact on costs, revenue or safety.

In this environment, a waterfall approach was more or less essential to minimise the risk of costly errors and rework.

It reflected the inertia in the environment, it didn’t create it. Its inability to deal effectively with business change was as much the result of the methodology as the  inability to turn a supertanker in a swimming pool is the fault of the steering.

Over time, we grew the amount of application and data collateral and we got much better at abstracting the layers of the stack, from data and business applications up to the service and interface layers.

Meanwhile, the demand for ever more flexible, faster to market IT mushroomed and the need for Agile or something like it became unavoidable.

All that accumulated collateral enabled the new methodology to focus single-mindedly on the most important thing, to deliver customer value as rapidly as possible.

But while we’ve removed a load of inherent complexity, mess and risk, they haven’t disappeared, and I believe Agile has limitations in dealing with them.

There are things, like babies and supertankers, which are too complex, mysterious, tightly integrated or risky to be delivered incrementally.

If you design, plan and understand them effectively, you can more easily deliver incremental benefits, but you need that solid product in place first.

The old adage that outsourcing a mess just gives you a bigger mess applies equally to Agile. Apply Agile to a mess and you just deliver a bigger mess more quickly.

Agile is great at creating value quickly when the environment is well understood and architected, complexity is low or already taken care of, and risk is low. When complexity and risk are defining factors, not so much.

I guess Agile proponents are rolling their eyes by now and protesting that Agile was never intended as a panacea for all projects.

Yet that’s what I recall the early pro-Agile, anti-Waterfall propaganda seeming to claim.

I certainly remember an exchange with Tom Gilb, one of the fathers of Agile, stating to me that you could deliver anything with an Agile-style incremental approach, and pretty much arguing that anyone who said you couldn’t was just too stupid to have understood the approach.

I believe some of that propaganda may have stuck, to the extent that we tend to think any opportunity or problem, no matter how complex, can always be tackled most effectively by rapidly delivering a minimum product which demonstrates value and achievement to the customer, then building on it.

So we end up with a wall being touted as an acceptable Minimum Usable Product for an immigration policy, and a Withdrawal Agreement which no-one can agree as the MUP for the entire relationship between the UK and the EU.

Back in the world of IT, and changing tack to consider fault tolerance, I’m seeing some worrying examples of ‘bugs on the backlog’ making it into customer releases.

This makes me wonder if our application of Agile always considers how context can determine the impact of non-critical faults

I went into ‘Words with Friends’ after a recent new release and a load of old, completed games suddenly started showing as active again. For me, that’s a minor irritation which I think no more about – it’ll be in the backlog and it’ll get sorted sometime.

I recently went onto an airline website and it gave me the wrong decode for an obscure IATA airport acronym, it’s still only a minor irritation, but it’s an airline, so I started thinking like that old TQM pundit who said ‘If the airline’s tray-tables are dirty, you start to worry about their maintenance standards.’ Context is important.

We’ve already seen the growth of DevOps, a positive sign of constructive synthesis starting to happen in the Waterfall/Agile dialectic. It represents a recognition that stuff like operability still needs to be front and centre, even in a customer value led methodology.

I hope and trust we’ll see Agile itself continue to evolve so that we can maintain its advantages without being tripped up by its pitfalls.

In the meantime, I think it’s well worth taking time at concept and inception, in collaboration with the Product Owner, to consider things like the complexity of the environment, both inherent and accidental, and the end user’s fault tolerance posture.

Facing up to these elements early on should help things run more smoothly in the sprints.

And, worst case, if the complexity or risk is too great for an Agile project to be viable, I can recommend an independent consultant with lots of experience of waterfall projects.