DevOps Vision Blog

DevOps Vision Blog

About the Blog

Various posts about software delivery in the 21th century. Managed by Anders Lundsgard, @anderslundsgard. Psots are my own but inspired by many.

Deadlines kill your Agility

NoEstimatesPosted by Anders Lundsgård Thu, June 16, 2016 12:39PM

Just a post to make you think about the value of your estimates to deliver software "on time".

It is the year of 2016, and large organizations still hire a bunch of agile coaches to make teams of developers more agile. I think they are mistaken if they think they can improve IT performance merely by getting everyone in the engineering department to adopt Scrum.

Before you continue reading. If you don’t believe that agility is good for your business, please do something better that continuing to read this post.

Agile thinking must extend beyond the boundaries of the Engineering Department for it to work at all. We need DevOps and Continuous Delivery for a fast and reliable path to production. Continuous Integration with automated tests in pipeline, trunk based development and so on are today’s mainstream even in large companies. Microservice architectures and Cloud environments make delivery and operation of software to a no-brainer. But all these practices only address the IT delivery cycle. It helps to build the thing right, not the right thing.

Building the right thing is an iterative process that requires full co-operation of the product management and other business stakeholders. However, when teams attempt to iterate at this level, they encounter friction on account of the organization’s structure, operational practices, politics, and culture. People in an organization act rationally in a way that maximizes their own success. Putting the emphasis on departmental output maximization, rather than optimizing the overall flow for the customer, means that the natural interests of the departmental manager come into conflict with the long-term goals of the company as a whole. In short, siloed organizations sub-optimize for their own success.

One of the things that people new to agile thinking have a hard time getting their heads around is that time (and estimates, and deadlines) is simply not a factor in an agile shop. They say that deadlines are a necessary part of the "real world." Yes, there are real-world issues that surround time. It's hard to move the date of the "2k bug" or the opening ceremony of the Olympics, but time is not a central part of the agile planning process. When you deal with complex systems (as we do in software delivery) my experience is that you should avoid deadlines wherever possible.

In a waterfall world, a deadline provides the illusion of security. A traditional deadline is a way to measure progress in order to manage a budget. The reasoning is that the product perhaps won't be totally finished for a year, but at least we know that they (the developer teams) will be "Done" by a year.


Agile processes promote sustainable development. The business, developers, and users should be able to maintain a constant pace indefinitely. Constant pace means that deadline-based management process simply can't work. If a team is punished for not meeting their deadlines (and overtime is a kind of punishment), then they'll pad the estimates to avoid the pain. Also in a code-monkey culture, the developers take shortcuts to meet an expected short-term business result. Omitting a reliable pipeline and test coverage increases technical dept. With that, the quality and ability to keep up the pace after the deadlines will decrease. And everyone asks why?!

In my experience deadlines are based on long-term estimates, and estimates of that sort almost never work. In reality everyone (including you) know that estimates always fails. But for some strange reason we keep behave like we don’t know that. Perhaps the simple reason is that we are to lazy to find better alternatives to master our software delivery. The hashtag #NoEstimates was born just because of that. It is a discussion on how to find alternatives to Estimates in software delivery.

A deadline-focused manager typically starts with the assumption that certain features are essential, and if you can't build these features by the deadline, then you've failed. When deadlines serve as a substitute to close collaboration between developers and the business they do not serve a good purpose. An agile company has realized that "essential features" will change as the product develops, so any estimation that was made in the past will be incorrect because the set of things on which you base the estimate will change. You always find flaws in the specification and encounter new implementation problems. Sorry, but that is probably true for all your software projects for the rest of your lifetime.


Questions

All projects are different. In your environment, have some thoughts on these questions:

1. Do you estimate the time team members should be out of office for various personal reasons?
2. Do you estimate the time to write tests and manage the CI/CD environment to run these tests and deploys?
3. Do you estimate the time your team will explore new territory to find better ways of solving your challenges?
4. Do you estimate the time your team will manage pull-requests?
5. Do you welcome new requirements between the time of estimate and the deadline?

The "solution"

When you manage to have a constant pace with short iterations, you can guarantee that you can deliver a working product every single day. The software is guaranteed to work when that trade-show arrives because it always works. The goal changes from "building a specific set of features by the 15th of September" to "building the best possible product in the time we have".

Continuous prioritizing based on feedback is important in this non-waterfall world. To achieve that we need to have a day to day collaboration (at the best also co-location) of development and business people. These days Outcome-oriented or Autonomous Teams are popular expressions for that. The responsibility for business outcome must belong to the developer teams building the product. As long as the company culture prevent that to happen the longer, the agile journey will be close to infinite.

Summary

A company with a culture of top down deadlines will die. The date of the funeral is based on how your competitors are doing in their "agile movement". Deadlines, especially those pushed from above, undermine teams with a constant push towards an illusory goal, and any attempt to work in an agile way will fail as a consequence. They will eventually also build products that nobody wants. Agile thinking must extend way beyond the boundaries of the Engineering Department for it to work at all.

And finally. I don't say do Estimates or do No Estimates (or deadlines). But probably all companies need to improve their current processes. Experiment with alternatives that fit your situation best. And of course - One size does not fits all.


References

#NoEstimates talk by Allen Holub
https://youtu.be/QVBlnCTu9Ms

There's No Room for Deadlines, June 30, 2014,
http://www.drdobbs.com/architecture-and-design/theres-no-room-for-deadlines/240168577

[Podcast] Designing Outcome-Oriented Teams: Part 1,
https://www.thoughtworks.com/insights/blog/podcast-designing-outcome-oriented-teams-part-1

NoEstimates book
By Vasco Duarte, http://noestimatesbook.com

Lean Enterprise book
By Jez Humble and Barry O’Reily, https://barryoreilly.com/lean-enterprise/

Toyota Kata book
By Mike Rother, http://www.lean.org/Bookstore/ProductDetails.cfm?SelectedProductId=324









  • Comments(2)//blog.devops.vision/#post2