Predicting the future: how long will a story take?
In the second of our four-part forecasting series, scrum master Dave Longman discusses his approach to estimating how long it takes to complete a story.
After 10 years of living in our house, my wife and I started thinking seriously about building an extension. Soon, we were immersed in the world of construction – a space neither of us are particularly knowledgeable about.
When you don’t know the industry – or what the work entails – you’re never sure if your ideas are workable, super-complicated, or totally out of your budget. Would an extension cost us £20k? Or £100k? We may like the look of local slate flagstones, but they’re £10k; is there an alternative that looks similar but costs, say, £2k instead?
This experience was a lot like what I’ve been helping clients through in my day job for years. They’re often unsure whether the ‘simple’ website they’re requesting is as straightforward as it sounds, or whether it’s far more complicated – because they don’t have the relevant experience. They don’t know whether the ‘Support Apple Pay’ feature is their equivalent of the local slate flagstones or the £2k alternative.
With our extension, what we needed was someone who had much more experience to listen to our ideas and tell us (roughly) how much it was likely to cost. We were looking for an idea of whether an extension was remotely feasible – we needed an estimate.
Estimates: useful tool or arbitrary guess?
An estimate is usually what clients are looking for, too. I’ve always struggled with them as a concept – in software development, I’ve seen them used to ‘beat up’ teams too many times for not hitting a deadline that shouldn’t have been made in the first place.
Over the years, that’s put me firmly in the #noestimates camp, a movement that rose out of Agile and Scrum practices in the early 2010s. Developers like Woody Zuill argued that estimates weren’t helpful (and could even be harmful) as we try and make predictions based on incomplete information. There are so many variables that can affect a software project, and demands can change so quickly – in those situations, estimates are useless. However, this argument has never stopped my clients valuing an estimate.
But trying to decide whether I could afford an extension put me firmly in the shoes of my clients and business-focused colleagues. Though my reservations about the value of estimates for developers remained, I understood why non-technical people feel they’re important. When you don’t have the expertise, estimates can help you contextualise a project – how long it’s going to take, how much work is involved, and how much it’s going to cost. And in large organisations, stakeholders often need estimates to satisfy their own internal processes. Overall though, they also want to understand if their idea is even possible.
As software professionals (as any sort of professional or expert, in fact) it’s really easy to forget that you have a unique insight into the problem domain that non-experts just don’t have. We need to be able to clearly articulate the effort required in a project, and explain where the risks and uncertainties are. We sometimes need to provide estimates but, when we do, we need to be clear about how confident we are, or how precise we can be.
If I asked you to estimate how much change I have right now in my pocket, you may tell me that there’s £4.56 in there. Now, that’s a precise estimate, but it’s probably not correct. However, if you said I had less than £5, there’s a far better chance that it’s accurate, even if it lacks precision. An imprecise but accurate range tends to be much easier to discover and, in a lot of cases, is at least as useful as an accurate-but-incorrect estimate. Ranges are always of some use – and they can easily be refined over time.
Keep an eye on the forecast
Forecasts and estimates are closely related and often conflated. For me, forecasts are simply estimates that change over time, usually becoming more accurate and precise the closer they get. They combine things we know with things we can guess, and give an evolving prediction of what will happen in the future. And they almost always use some type of statistical analysis to help refine the prediction.
A perfect example is weather forecasts: a weather forecast is usually pretty accurate for tomorrow, but you’d be much less confident in the forecast for two weeks’ time. And the odds of the annual ‘White Christmas’ forecast being right is usually no better than your odds of winning on red in roulette. But nonetheless, we still pay attention to the sun or snow forecast in the run-up to our summer or winter holidays.
Building an effective forecast
The greatest feature of forecasts is that they get more accurate the more data we can collect. We have no shortage of data in the software world, and Agile ways of working are great for generating data that shows how well we’ve delivered historically. For forecasting, this is gold dust.
Over the next three articles, we’ll explore three super-simple techniques development teams and stakeholders can use to build powerful forecasts that provide a continuously refined prediction for a software project’s progress.
We’ll start by looking at how to predict how long a single story will take to complete, without even needing the development team to take a look.
Related posts
Predicting the future: How big is this project – and when will we complete it?
In the third of a four-part series, Dave Longman takes you through a spreadsheet that uses Monte Carlo simulations to forecast project timelines.
Predicting the future: How much work is in our backlog?
In the final instalment of our four-part series, Dave Longman discusses how to use averages to estimate the undefined work in your backlog.