#NoEstimates - An Alternative Means Of Risk Management

Introduction

A continuing theme of counter-arguments posed at the #NoEstimates ideas is that development cost estimates are required in order both to manage risk and to derive value.

This blog post intends to give further insights into how risk can be effectively managed, and how we might determine the value of our initiatives, without the need for making up front and deterministic development cost estimates.

Risk

“Risk is the probability of an unfavourable impact to the project” — Glen Alleman (@galleman).

From the risk angle, the argument goes along the lines that the built-in “risk management” in Agile approaches is not aligned with conventional definitions of risk management in software development.

I’ll go along with this. Agile (and #NoEstimates) does not take the conventional approach to software risk management, which sees project success as “on time, on budget” and thus requires an up front estimate of total scope, cost and duration.

Agile/#NoEstimates offers an alternative way to manage risk on projects (and, no, I’m not taking about Agile Estimation, the spin-off brand of traditional estimation promoted by Mike Cohn). I’ll explain more about this later.

Value

The argument regarding value is that estimated cost is required to determine value, given that value is related both to the timing of when things are released and how much it costs to develop the things that will (potentially) generate value. That the worth of something to someone can only be evaluated if we know how much that things costs.

Again I agree to an extent, but there are two key sticking points for me here. One is that we only know how much software development costs after the fact. People say “we need to estimate because we need to know the cost”. Estimating, however accurately we think it is being done, does not allow us to know the cost.

Before the event we can only estimate what will be done and how much it will cost. In addition, the further out we are estimating cost and value, the riskier (and potentially costlier) our estimates become.

By estimating, rather than fixing, cost we have no greater insight into the value, which is also estimated. Essentially we are increasing our risk by estimating both cost and value rather than just value, which is what #NoEstimates promotes. More on this later.

The other sticking point is that value is often highly subjective and personal. I know how valuable a particular brand new Ferrari is, partly because I know how much it costs. That said, if you gave me two different Ferraris to test drive and didn’t tell me how much they cost, I would tell you which one I prefer. Which one was more valuable to me. This has nothing to do with the cost. The one I prefer might be significantly cheaper, but its value to me is higher because it’s more fun to drive and I prefer the look of it.

The same applies with software. There is so much to consider when we try and measure value. Aside from the empirical measure of monetary returns, we have to consider the needs of the customers, the stakeholders and our corporate strategy (to name but a few), not to mention the fact that all of these things change over time.

Agile is about delivering value early, not trying to predict how to maximise value over a given timeframe or a product’s lifecycle. It is the early delivery of value that allows us to tune and adjust our course for maximum longer term benefit.

This is why it is an alternative, and completely viable, approach and should be considered as such.

Agile Risk Management

The key aspects of Agile that help us manage risk effectively are:

• Iteration
• Continuous selection of highest value work (i.e. making decisions)
• Fixed, cross-functional teams with 100% focus on current project
• Early and frequent delivery of end-to-end working software increments and
• Empirical measures of progress toward goals

With Waterfall projects, the need for conventional risk management is clear. We have no way of measuring progress from day one in terms of working software because we are carrying out requirements analysis, specification and design phases before we write a line of code. People are often working on multiple projects and so we must allocate a percentage of their time to the project at hand.

The only way to measure percentage progress toward project completion is to have a breakdown of the SDLC phases and tasks within each, estimated in days/weeks, and tick them off as we go along. If we don’t complete all the necessary tasks for a given phase in the estimated timeframes, we are “off track” and we need to take corrective action.

With a phased delivery approach, the only way to manage risk is to have an estimate of the total scope, cost and duration of the project.

But if we are working in an Agile way, we are not taking a phased approach to project delivery. We are delivering full end-to-end working solutions in an iterative manner, early and frequently. We are working in fixed, cross-functional teams so teams costs are known and consistent.

This approach allows us to manage risk and measure progress toward project completion (meeting of stakeholder goals within a given budget) from the get-go.

Progress

If we are truly iterating by delivering vertical slices through the system, after our first iteration we will be able to measure progress toward the project goals. We will have delivered a working, albeit perhaps low quality, solution to the problem. We may even have actually met the project goals.

Either way, we can inspect what we have done and decide if we are on the right track. If we are, we can iterate over our solution, improving quality in the desired areas and incrementing new features. If we are not, or we see a better way of solving the problem, we can throw away what we’ve done and start again. We may even decide to scale up our efforts and add more teams, if there is emergent estimated value in doing so.

Given in Agile we are delivering end-to-end working software from the get-go, we are not burdened with the problems we faced in our Waterfall projects for measuring progress. We have the ability to empirically measure progress because we are delivering “done” functionality, as opposed to hitting pre-determined “milestones” which are not based on what we have actually delivered in terms of a working product.

In Waterfall, so long as we are hitting our milestones then the project status is “green”. For software product development projects, this means that we are deferring our risk management until we actually start writing code. We don’t know that the scope of what we want to build is achievable, and we can’t reduce scope until we actually realise it’s too much (well into the development phase, deep into the project).

In Agile we can manage scope right from the beginning, because we are continually focusing on building the most valuable thin, vertical slices which represent iterations over an end-to-end solution to the problem. We can empirically measure how much we got done and how much is left to do. We can regularly take proactive decisions to cut scope or switch to an alternative approach to improve our chances of delivering a successful outcome. What should we do next for maximum value and maximum impact in meeting our goals? What should we not do? What is the simplest approach for our next iteration?

This is risk management.

These kinds of conversations enable us to focus on doing the simplest thing, for maximum impact, given the budget that we have available. To not wait 9 months to deliver a solution but to deliver a solution in 1 month, then make it better.

Most “Agile” projects are not managing risk

If we decide up front in a project inception on the requirements (product backlog) and solution we will be sticking to, and estimate it will take, say, 9 months, all we will do is incrementally build the solution, usually in horizontal slices, components or modules.

After each “iteration” we will not have a holistic view of what we’re building.

This is a very common approach by “Agile” teams. In this situation we are deferring the management of risk until we actually have a system that can meet (some of) the needs of the project stakeholders, usually late in the game when the deadline is getting close.

This is not risk management. If we work in this way we cannot work with #NoEstimates.

How do we estimate value without estimating development cost?

OK, so assuming we have the capability and will to deliver vertical slices through a solution early and rapidly, and we have a fixed cross-functional team, 100% committed to the project at hand, we can focus on the potential value of the ideas we want to build while controlling cost using small “drips”.

When we use ROI to decide whether a project is worth pursuing, or which of 2 or more potentially valuable projects we should choose given limited people and resources, we base the “investment” measure on the estimated cost, of which the development costs are part, and the “return” is the value we expect to generate, measured on the same scale as the investment (usually money).

There is a flaw with this approach.

6 months, 2 years, it’s all the same!

Let’s say we estimate a project will take 6 months of development time, costing $500k. We expect that when the product is complete it will generate $2m in revenue. The timing of when that revenue gets generated is key. Will we get anything at all before the product is built in its entirety? Will there be a few months of marketing required after all the features are done before we will start seeing the cash rolling in?

The implication of the timing of value generation is that the actual ROI of what we’re building in a 6-month project might still be negative after 6 months of development time, even if we get everything done that we originally wanted done (and estimated).

Now compare that to, say, a project with an estimated duration of 2 years. After 6 months, the ROI of the two projects will be identical. Our net loss in both cases is $500k, so our ROI is -100%; we have spent half a million bucks with nothing (yet) to show for it.

So, given the erratic, inconsistent and numerous ways we can measure value in software, is the traditional ROI approach an ideal decision making model in this domain?

Agile is about early delivery of value, not trying to predict maximum value

The upshot of this is that the less risky approach to generating a positive “ROI” is to work on options that will potentially generate value early, i.e. with relatively small and simple effort. Put simply, if we prioritise initiatives by virtue of which ones we expect to generate value early rather than how much value they will generate over the product’s lifecycle then we do not need to batch these initiatives up into “projects” and estimate how long the project will take.

This can easily be reverse engineered. If our starting point is a “project”, with a list of requirements, the best thing we can do to manage risk (keep our decisions within the bounds of the near, more certain, future) and ensure we deliver value early is to pick the most valuable requirement/problem to solve and come up with a simple, creative approach to fulfilling that requirement in a very short timeframe.

What’s next? One at a time…

The team can go away for, say, 1 month, after which time we holistically assess where we’re at in terms of fulfilling that requirement. What have we learned? Is this requirement still the most valuable one to work on (ignoring sunk costs)? Are we better off ditching what we’ve done and investing in attacking another requirement?

Our measure of what is valuable must reset after each iteration. It’s irrelevant how much we’ve already spent (sunk cost fallacy).

We need to constantly concern ourselves with what is the most valuable thing to do next. This is Agile. This is #NoEstimates.

And this is risk management. Yes, it’s an approach that requires a different way of thinking about how we choose what work to invest in, how much to invest and the decisions we make along the way. But it is risk management nonetheless.

But we can’t do this when $200m is at stake!

The #NoEstimates debate has hit a point where the main remaining arguments are around its application in big money projects. Most of the original dissenters — who have now spent time reading more about the ideas put forward by myself and the other #NoEstimates crew — are now in agreement with us that, at least for small scale projects, we can get away with not doing “micro-estimates”, and indeed it may be preferable to work this way.

But when it comes to “macro-estimates” — i.e. how much of the customer’s money are we going to spend — it is argued that a #NoEstimates approach is not viable. That when “you are spending someone else’s money” you need a plan (estimated schedule) to ensure you deliver what is required for the money, with some deterministic level of confidence.

The irony of this argument is that when the big number guys come out swinging with their big numbers, these numbers are estimates! When we call a project that we haven’t yet completed, or even started, a “$200m project”, what we are actually saying is “our customer has a $200m budget and we have to deliver what they want for their money”. In other words, the decision has been made to go ahead, and the budget is $200m. There is no go/no-go decision to be made — it’s already been decided that the project is going ahead, and they want a result for $200m.

For me, with such large sums and timeframes at play, there is all the more reason to manage risk by drip funding small amounts and iterating over a solution in the way I’ve described. Scaling up where required. Tuning and adjusting.

The alternative is to manage risk by using probabilistic estimation techniques based on past projects such as Monte Carlo simulations to derive a total estimated cost with a confidence interval, and then constantly adjust these calculations as the project progresses. But I maintain that the Agile way, where we start from a budget or fixed deadline and then actively build and manage scope along the way, is preferable because it harnesses the creativity of designing and building great software and allows us to welcome and embrace change every step of the way.

Create the future rather than predict it

Instead of trying to nail down a plan and predict outcomes, we are forging our own future based on current market conditions at any given time, and the way we feel about what we’ve built so far. We are controlling our costs by working with fixed teams in short time-boxes, and we are constantly assessing the value of what we’re building.

If we work this way we do not need to estimate things up front. Empirical data is being generated as we go along, and we can look at the market with fresh eyes after each iteration. We can see what we’re getting done and what we’re not. We can change our mind on whether we care that we didn’t get the things done that we wanted to get done. We can see which of our assumptions were true and which were false. We can steer our ship in whichever direction we need to avoid the iceberg ahead, while remaining focused on the destination.

This is at the heart of #NoEstimates from my point of view. It is possible to work this way. It is not easy to get to a position where you are able to, but if you can get to that place it is, as Ron Jeffries describes it, “the best known way to work”.