Lean Software Estimation

boat navigating the icebergs @anniesprat on Unsplash

Estimation is one of the hardest problems in software development. Trying to calculate the time required to solve complex technical and organizational problems is more magic than art. As Kelly Vaughn on Twitter put it…

When you give a developer 5 hours for a task, they’ll let you know they need 50 hours once they’ve used up 10.

Even looking at the most atomic of these calculations - estimating an engineering task or user story - it’s clear to see that estimations are no exact science. We engineers size stories based on technical complexity, but even that is no exact correlation to time spent, as there are countless variables to this measure - skillset, experience, motivation, relationships - just to name a few.

Now multiply that complexity by the number tasks or user stories required to make up a feature. Consider how those variables expand beyond one developer to a team, or group of teams as the complexity of the compounded problems build on each other. As features combine to form epics, and epics may be combined to form a product release, we start to spiral out of control.

Just as a 1 point story means something completely different team-to-team, the same is true of user story comparison or epic comparison. Estimates on this order of magnitude are practically meaningless, as even the most seasoned teams are only just estimating - and on a much smaller scale. Remember the tweet: 5 could be 50 when you’re standing in front of a pile of work. 5 months becomes 4 years.

Excellent advice could be to shut up and plan the sprint, remembering that our agile frameworks like Lean teach us to make decisions late and focus on the most important things first. As an engineer, I know that is usually the right thing to do to be as efficient as possible. On the other hand, I’m offloading a lot of the mental energy that it takes to decide “what’s next” to our product teams.

And in product, the eternal struggle is understanding how to plan. How to determine how to spend money and trying to calculate when investments are going to pay off. So the conversation turns to asking for commitments, and 2 weeks isn’t a very long time.

I fear the answer includes some nuance, but a few things I think you’ll need first.

Trust (and Invest) in Your Dev Teams

The entire agile philosophy is built on the idea of high-performing teams. If you go into the process doubting that your dev team is performing at their best, or that they’re not up to the task, the structure immediately begins to break down. When you don’t trust a developer’s estimate to be the truth - and instead some fake number meant to inflate the work or buy time - you’ve failed.

Even in the real scenario where your dev team isn’t so high-performing, trusting their work, estimates, and convictions is critical to the process.

To continue to build this trust, engineering teams should be invested in and empowered. It’s the goal of the engineering leadership to find, cultivate, and empower great engineers. They should be experts at helping engineers reach their high-performing potential, and clear-eyed about areas to be improved. The culture in the engineering organization should be one that reinforces leadership and learning. Good engineers should be rewarded.

Enable Incremental Improvement, and Ask For Proof

The faster the product is in your hands, the faster you can start to learn. An unreleased product is a net drain on resources, whereas any code in production provides value to users and teams.

This means that moving a product from development to production is more important than how much value you’re providing per-release. Focus less on understanding the path to deliver on an entire feature or feature-set, and more on how you can start to iterate on the smallest changes possible. Let product ideas evolve instead of trying to decide on perfect early.

Working software is preferred over planning. Meaning your team should be focused on building. Each sprint, or faster, your engineering team will deliver software that can be demonstrated. Asking for real work to be demonstrated is a cornerstone of the process. Focus less on the scope of work and more on the fact that work can be delivered every sprint.

Change, as a Result of Output

The concept of deciding late can be foreign to project teams used to slow, huge, disruptive releases. It’s scary to “let go and let Lean” when we aren’t used to seeing real code go fast.

In a productive team, you should get used to letting things go to users un-polished, and learn how to react quickly to feedback. Try to resist the urge to change direction before getting feedback from users.

Back to Kelly’s tweet - you aren’t going to be able to avoid engineers finding work that needs to be done. The point of the tweet is to focus on not putting too much time, effort, and value in zeroing in on a 5-hour estimate for something we don’t understand yet.

Instead, let engineers invest that planning time in writing real code. Let them learn from the code they do write. Let them get that code to production - fast! And then, change based on the code in users’ hands.