cumulativehypotheses

Jump to the TL;DR if you want.

What does #NoEstiamtes mean? It’s surprisingly difficult to tell. Depending on whom you ask It might mean, as the name suggests, No Estimates! or it might mean Estimate All The Things—just don’t call it that! or it might mean something in between those, or it might mean something which has nothing to do with estimates at all, or it might be about questions not answers and it might be about just “starting a debate”¹. A term that can mean so many things runs the risk of meaning nothing, or of just being the latest shiny buzzword to signal that you get it (not like those other silly folks, stuck in their ways).

When it comes to #NoEstimates what I’ve found is that the most concrete statement of what it might mean that anyone can point to is Vasco Duatre’s self–published book No Estimates: How to measure project success without estimating. (I’ll refer to it as “NE” in what follows, whereas the NoEstimates movement at large will be #NE)

It’s a commendably brief book, and not so expensive. It’s also clearly a labour of love and I do respect that. The urge to share really cool ideas is a strong and respectable one. There’s a bit of a “business novel” style of story running through it, linking some tutorial style material, and this story tells the initially very sad tale of Carmen, a well-meaning but inexperienced project manager—even within 10 pages of the end of the book she still thinks that a Gantt chart is going to be of any use to her—and her profoundly idiotic and bullying boss, both of whom seem to work at a rather desperate and very old–fashioned outsource software development house, named Carlsson and Associates (I’ll call them “CA”).

Quotes?

The word “budget” occurs ten times in NE, variously in the contexts of: the difficulty of not exceeding one, the unreasonableness of demands made relative to them, the further unreasonableness of demanding that people conform to budgets that they neither determined nor can control, and so on. And those are all difficult and unreasonable things. However, a company like CA, which is taking part in a competitive bid in Carmen’s story is going to have to produce a proposal, containing a quote, a proposed budget for the work on the “Big Fish” government contract.

NE often focusses on the difference between an estimate, a commitment, and a forecast. Those are different things, but NE seems to want the distinction to hinge on whether or not you have data (that would be a “forecast”) or whether you’re just guessing (that’s an “estimate”). I’d like to suggest that amongst people who know what they are doing the distinction is much less clear-cut, and much of what NE calls forecasting looks a great deal like estimation to me.

But NE doesn’t seem to mention quotes (other than as in “what somebody said once”). Throughout the book there’s no indication that I can find of how exactly an #NE “practitioner” is supposed to produce a quote for a piece of work—which will be required at some point by anyone who isn’t working for an in–house team and needs to win a contract.

Update: I’ve been asked how quotes fit into an agile world. In my experience, if you are a supplier of development effort to clients then the quote is what gets you permission to start spending money. It’s not really “the budget”—although it might be described that way—it’s a starting point for an on-going conversation about value. Again, in my experience, a £60,000 proof–of–concept or a £120,000 Discovery activity can, though the establishment of a reputation for steady delivery of value, grow into a multi-million pound endeavour spanning several years without anyone ever deciding that this is what it should end up being. But sometimes you really do need to talk about the years and the millions, and if you can’t: no sale!

In the story Carmen sets about the estimation task (to produce the undeclared quote) in the worst way possible: she tries to construct a Work Breakdown Structure², estimate the effort for the leaf nodes, and then roll that up into an estimate for the whole thing, which is madness.  CA get the gig—after somehow having sight of their competitor’s bid, which suggests that the client is pretty sloppy. It also suggests that CA did a very common thing and priced their bid “to win”, that is, by producing a very low quote. It’s important to realise that the estimated effort (time/team size/cost, whatever…) to complete a piece of work is only one input to a quote. By quoting a price to win the work CA are following in the footsteps of many a supplier who has low balled an alleged “fixed–price” for a piece of work comfortable in the knowledge that he client will want to change their mind about the scope and can then be charged for change control for very, very long time—which is where the unscrupulous supplier³ makes their profit. CA don’t seem to be even that smart, and Carmen’s boss seems to think that CA can somehow price to win with a fixed price and a fixed scope and then deliver against both. Carmen’s project is pre–doomed. Which can be a good thing. So long as everyone recognises that you have no chance of delivering, whatever you do, then it doesn’t matter what you do and all sorts of options which were previously unavailable can become plausible, because what the hell!

Now, Carmen’s boss is an idiot but weirdly, on page 62, he suddenly asks a smart question, albeit in a stupid way and for the wrong reason:

“Carmen, we have a review of the project with the Client next week. How are things going, what kind of progress can we show them?” Asked her boss.

“Good Morning sir. Yes, we do have the requirements delivery and the Earned Value Management reports that I showed you just yesterday.”

“That is great Carmen, but I was asking if we can show them working software. You know, to make a good impression.”

Tuns out that there is no way to demonstrate any useful intermediate state of the implementation of the Big Fish system. Carmen’s project has become even more doomed than it was before CA won the gig. Although CA seem highly clueless, unfortunately Carmen’s situation is not so fictional as one might hope. But…and this I think speaks to the core of why #NE is so disappointing to so many people, CA have allowed their client to make them do stupid things and then CA have piled stupidity upon stupidity in how they respond to that. Competent suppliers just don’t behave the way that CA does, not these days.

Although all too plausible the scenario in the story is also a sort of pastiche of what too many mainstream project looked like more than ten years ago. I certainly saw projects like this when I started working in the industry in the early 90s. But these days, not so much…in between times, something changed.

Government

BigFish is a government project and as NE explains, government projects are notoriously very expensive, very late, and often deliver almost nothing of any value. The astonishingly terrible UK project to build a new IT system for the NHS is cited. But, here’s the thing, that project came to a long slow, shuddering halt, finally stopping all together in 2013—and even governments can learn. Since 2011 new build projects in HM Government departments⁴ are run with oversight from the Government Digital Service, who know what they are doing. All GDS projects are iterative, incremental and evolutionary. Spending departments simply are not allowed to sign up for the kind of catastrophic deal with the Usual Suspects that lead to those horror story government IT projects of the lore.

This was meant to be chapter–by–chapter review of NE, but my eyes started to glaze over—which I release is a poor trait in a book reviewer, but the reason why they did is interesting. Back to the story:

Carmen’s Big Fish project gets into exactly the sort of trouble that you’d expect, being driven by guesswork and wishful thinking, and she ends up appealing to the local #NE guru, Herman. In the charming illustrations by Ángel Medinilla this Herman is depicted as a portly, bearded, balding fellow. I certainly applaud the principle that portly, bearded, balding men are the fount of all wisdom. Anyway, Herman gives Carmen various items of good, commonplace and uncontroversial advice and between them they get the project back on track.

Now, through the first half of NE I’d been thinking: so far so unsurprising, when do we get to the new thing? And when Herman entered the story I though: great! here comes the punchline. But it just doesn’t.

Errata to NE

Perhaps these can be addressed in later version of the book. They are found in the pdf of version 1.0

p16 J. B. Rainsberger has made many fine contributions to the state of the art, but did not introduce the concept of distinguishing essential from accidental complexity in 2013 (although I’m happy to believe that he spoke about it that year). This distinction was introduced by Fred Brooks in his famous paper No Silver Bullet[pdf] — Essence and Accidents of Software Engineering. The distinction was part of the folklore of the industry when I started programming for money in the early 1990s, a long time before I met J.B.

p51 incorrectly characterises Set Based Concurrent Engineering[pdf] as the process of starting to build the production line for a product before you’ve finished developing it. It isn’t. Or rather, doing that is just (one part of) “Concurrent Engineering”. The “Set” is of alternative design choices and they are all developed (concurrently) to a surprisingly high level of refinement and each eliminated through a tournament until one remains which then goes into production. This SBCE process is followed in part to allow for the decision to go to production to be made as late as possible. Reinertsen, in his The Principles of Product Development Flow criticises this approach as too often delaying the decision too long, beyond the point where the economic return on further delay starts to decline.

p64 wrongly states that RUP⁵ is a linear process model. It’s not. Or rather, it’s not supposed to be. Philippe Kruchten, who was the brains of the operation, built RUP to be very flexible and highly configurable and the first thing any RUP project was supposed to do was tailor the process within some very broad parameters by creating a “Development Case”. The non–negotiable bits of a RUP-derived process were meant to be [emphasis added]:

1. Develop iteratively, with [technical] risk as the primary iteration driver
2. Manage requirements
3. Employ a component-based architecture
4. Model software visually
5. Continuously verify quality
6. Control changes

It’s important to note that in Kruchten’s idea of what a RUP project should look like, the implementation, testing and deployment to production of code happens in every iteration of every phase of the project. However, what a lot of people (every RUP project I ever saw, in the UK or the USA, certainly) did was to carry on doing whatever linear, phased process they were doing before but rename bits of it using RUP terminology. Thus, the requirements gathering phase was renamed “Inception” and so on, and this worked about as well as you’d expect: very, very badly. And so the reputation of RUP was destroyed.

The aspect of RUP—when done right—that most lean/agile folks would object to most these days is the scheduling of work by risk rather than by value: we believe that agile technical practices tame technical risk for us, whatever order we develop features in. They’d probably not to keen on visual modelling (it is a mistake not to use visual modelling) nor on controlling changes (we embrace change, don’t we?) .

TL;DR

I think it was the great philosopher Robert Anton Wilson who said that the secret of leadership is to find some people who are going somewhere and get in front of them. I feel as if #NE, certainly as described in NE, might be doing something very much like that. Which isn’t a bad thing, necessarily, so much as it is disingenuous. Maybe that makes the #NE folks sound too cynical—which I don’t think they are. But there’s a huge gulf between the sort of pre–doomed idiocy of the way CA run their project to begin with in the story and what competent suppliers working with the  current good practice of iterative, incremental, evolutionary development (the only way that has ever worked in the general case, currently known as “Agile”) do today. And the gap⁶ between that and what #NE recommends and what NE very well explains is very small to non–existent.

At least this book is the first place I have seen all of those current good practices collected together with a semi-coherent story about how to use them all together on the same project. That’s a very useful artefact to have. But I might wish that the continual identification these good practices as being an approach distinct from the leading edge of mainstream lean/agile practice (which it is not) were dropped. The book would be greatly improved thereby and would, specifically, look a lot less like snake-oil salesmanship—which I don’t believe it is, but it looks like it, especially with all the charlatan hard–sell techniques you have to get past on the site to buy the thing.

Punchline

So what is the substantive content of #NE (as revealed in NE)?

There is one specific practice, illustrated very well in the book, which may be unfamiliar to many people doing mainstream Agile: slicing stories until they are all about the same size⁷, at which point “velocity” becomes a count of stories completed, not the sum of estimates of stories completed. Note that this isn’t a new, nor particularly radical idea, merely unfamiliar to many.

If you’ve drunk too much of the Scrum kool-aid (enough the for the effects to become irreversible) then you will hold fast to the dictum that “Work may be of varying size, or estimated effort” [Scrum Guide, v1, p 9] however, what might have slipped you mind is that the Scrum Guide says only this about how Sprint Planning works:

The input to this meeting is the Product Backlog, the latest product Increment, projected capacity of the Development Team during the Sprint, and past performance of the Development Team.

This allows for a great deal of latitude in how that goal is achieved—and the #NE proposition, as explained in NE would seem to fit that fine, if you were so minded. My experience with Certified ScrumMasters and Professional Scrum Masters⁸, however, is that the actual courses they do lead them to have a fetishistic determination to estimate, and as the Scrum Guide says, “estimate” [9 occurrences] and “re-estimate” [2] stories, and even “[make] more precise estimates […] based on the greater clarity and increased detail [available on items at the top of the backlog]” I’ll admit that the obsession that Scrum seems to have with estimating and re-estimating has struck me as odd, ever since I myself became a Certified ScrumMaster back in the 2000s. But is doing estimation the root of all evil? No.

Who is this for, again?

So, NE and #NE take a specific view on this specific issue: don’t estimate stories, slice them. And this is pretty much the only difference I can see between what #NE recommends and what any of the Agile teams that I think of as “getting it” do—and since many of them do slicing, often there’s no difference. Now, the detail material in NE explains with great subtlety and much appeal to thought experiments with probability distributions and what-not how not doing estimation is a waste–eliminating optimisation for  your process—although they do not demonstrate the effort of doing the slicing is actually less than the effort of doing the estimation, nor indeed that slicing is somehow value–adding and therefore not waste. But, Carmen’s story is one of utter foolish disregard for intelligence in project management brought under control by an Agile process which just so happens to use slicing instead of estimation—and the story also just so happens to leave out how you’d do the activities (such as providing a quote) that really do need estimates. This leaves me at a loss as to who NE (and #NE) is for: is it a subtle optimisation for people who are basically doing everything pretty much right? Is it a wakeup call for those in the lengthy tail of very, very late adopters of Agile processes? I don’t know, and I can’t tell.

With some brutal editing to strip out all the propaganda, NE would actually be very useful both as a thing to use to introduce current good practice in Agile to newbies, and as an aide memoire for current practitioners. But it has this incessant drumbeat insistence that the techniques presented are New! and Different! and Radical! when they simply are not, which I think makes it little use for either group.

I do strongly suspect that if v2.0 of NE had, instead of he story of Carmen and the chaos at CA, a protagonist working at a company that was operating current good practice in Agile development, then the switch to #NE then the differences, and the story, would be much less compelling—but maybe more useful.

1 When was the last time you heard anyone say that they “just want to start a debate” and anything remotely enlightening happened?

2 WBSs for software development are almost never valid. I have seen valid ones, but only cases where a team is in almost a manufacturing mode, grinding out another instantiation of a very well-known product with only marginal changes from a bunch of other instantiations of it. This is dull, low risk work and therefore low margin, and most of it is done by low–cost development shops in Farawayvia (or, as it may be, Distantistan). Anyone doing any remotely interesting software development work simply will not be able to construct a valid—never mind useful—WBS and should not even bother trying.

3 I often refer to these jokers as “the Usual Suspects”. You know who they are.

4 Full disclosure: my employer is a supplier to more than one department of HM Government, where we run projects as mandated by GDS and it works so well that we’e started to use the same DABL framework on private sector projects.

5 RUP is the process that will not lie down dead. Amongst those people who don’t seem to be comfortable running a development project without a vast and incomprehensible wall chart to follow, parts of the re-animated corpse of RUP are currently lurching around in two flavours: SAFe and SEMAT.

6 Theres this diagram in NE which could have been copy-pasted out of one of my own project proposals—I don’t suggest plagiarism, nor any sort of influence either way, it’s just a nice illustration of how NE doesn’t contain much of anything new, and of how #NE doesn’t contain much of anything that many people aren’t just doing anyway. It’s the one on p116 of the PDF, where Herman explains how to explain to a client what of their backlog they will, might, and won’t get—as best we know.

Illustrative Sketch of a structured backlog

You and I might imagine that constructing such a diagram might involve estimation…that’s certainly how I do mine. In fact, many of the techniques that Herman uses are estimation techniques, even though he insists otherwise, without really explaining why not. I think that this sort of thing is what leads Alister Cockburn to conclude that #NE is a “bait–and–switch”, they spend far too much time explaining how they estimate stuff.

7 Yes, doing that would appear to require that you estimate and re-estimate the size of a story to see if it needs to be sliced down any further—I guess you have to just not call it estimation…

8 “ScrumMaster” or “Scrum Master”? What are the semiotics of that interposed whitespace? Or is it simply a matter of not infringing intellectual property rights? A “ScrumMaster” was, originally, someone who had mastery of doing Scrum. A “Scrum Master” seems more like the master–of–the–Scrum…