Perils of estimation
Date: 2008-08-31 00:24:51

I had my first opportunity to participate in release plan estimation over the last couple of weeks. I've done estimation before but never at such a high level.

When doing this it appeared clear that there were two situations that we were trying to avoid:

== Under estimating

Under estimating is where we predict that the amount of time taken to complete a piece of work will be less than it actually is.

This can happen for a variety of reasons, including:

* We make some invalid assumptions - e.g. we assume integration with a 3rd party will be easy and then encounter unforeseen problems when we try to do it.
* Not considering all the edge cases - at the time of estimation we may not have all the acceptance criteria which could mean that some scenarios are not considered when estimating.

If we do end up underestimating the size of work problems will eventually manifest. Although the business may be happy with you to start with, eventually either they are going to be very unhappy when these aren't met. The most likely outcome then is that the team is going to end up on a death march to meet the original estimates, probably resulting in resignations after the project reaches completion.

The key to handling the situation where we realise we have under estimated a piece of work is to inform the business about the situation as quickly as possible. Waiting until the delivery date before saying anything is pointless.

== Over estimating

This is where we predict that the amount of time taken to complete a piece of work with be greater than it actually is.

Unlike under estimating the effects of over estimating are likely to be felt immediately.

For example, you may end up with the business being unable to afford to do the project if the estimates are too far removed from what they are expecting. As I http://www.markhneedham.com/blog/2008/08/26/the-transparency-of-agile/[mentioned earlier] this up front transparency may in fact be in the best interests of the client even if it doesn't seem it at the time.

Alternatively estimates may be challenged and pressure applied for them to be reduced. If they were actually accurate in the first place and the over estimation is only a perception then we may now have the problem that the new estimates are in fact under estimated.

If we reach the end of the piece of work and realise that we have finished earlier than expected due to over estimating then it may actually prove to be a good thing. We can then go and add more testing around our code, refactor code to make it more maintainable or whatever else the business sees as a valuable use of this unexpected free time.

== So what should we do?

Clearly it is important to try and avoid under or over estimating, but from my experience it is often better to err on the side of caution when making estimates. The natural tendency for software developers is to http://www.overcomingbias.com/2007/09/planning-fallac.html[underestimate] how long it takes to do something. It nearly always takes longer to do something than you expect it to.

In terms of concrete things that we can do to try and avoid either of the above situations from happening:

* Estimate original velocity by taking cards without their point/ideal day estimates on them and get the developers to predict what they can complete in an iteration - average the point/ideal day totals to come up with an expected velocity
* Review this velocity after a couple of iterations to see if it's actually accurate - communicate a changed velocity with the business
* Make sure the requirements are clear and that the developers understand exactly what is required to complete each story

Obviously even after doing all these things it is possible to get an original estimate wrong. That's how it needs to be communicated though; it is an *estimate*, not a guarantee and it could end up being too high or too low.