There’s really only one way to travel down a waterfall – in a barrel. A lot of people died this way, but some survived. Software projects have been predominantly waterfall projects since the start of software projects. And stakeholders rode down those projects, basically in a barrel. The people riding Niagara Falls 100 years ago didn’t know if they would survive until they got to the end. Stakeholders in waterfall projects don’t know if they will succeed until the end.
An agile project is dependent upon tight interaction (and feedback) with stakeholders.
If you’re running an agile project, and your stakeholders are old-school barrel-riders, how do you make it work?
Expectations, Documentation, and Communication
The success of any project is dependent on setting and managing stakeholder expectations. In Managing Stakeholder Goals, we talked about assuring that those goals are addressed by our requirements. And in a later article, we proposed a way to balance the goals of different stakeholder groups when those goals are in opposition. Those are good tools for making sure we are initially aligned with what we are hearing from stakeholders.
We need to also apply active listening techniques to assure that what we thought we heard is what the stakeholders thought they said. One way to do that is to write use cases (or user stories) so that we can communicate the intent of the product back to the stakeholders. This still primarily helps with kicking off a project, in the desired direction, with the correct goals.
There are many ways to document this initial intent of the project and stakeholder goals. It is analogous to a stakeholder selecting a sturdy barrel and picking a good spot on the river bank to push off. We’re well positioned to eventually succeed, assuming nothing changes. After a harrowing fall, we find out how well we did.
A well-run waterfall project will provide status updates to the stakeholders along the way. Imagine your stakeholder wearing one of those in-helmet headsets that NFL quarter backs use. We publish effective status reports to let the stakeholder know what’s going on. Like the falling barrel, however, a waterfall project has inertia. The project, barring significant outside influence, will keep going in the direction it started. Projects have change control boards to manage that change, but emotionally, I find that a formal change-approval process tends to inhibit change, rather than encourage it. That barrel will keep falling.
Some project teams will try and do very heavyweight documentation to maximize the likelihood that the project will end up where it should. The problem is, this heavyweight documentation only improves the chances that the project will end up where we used to think it should go. It does not help us change the trajectory of the project as new insights are gained. Just as responding to those changes provides a competitive advantage, ignoring those changes exposes a competitive weakness. Someone will come along and exploit your weakness if you don’t respond to change.
From these dynamics, we can conclude that “big up-front requirements”, while better than “no requirements,” are actually a waste of energy and time, relative to an ongoing adaptation to change. That adaptation to change, however, should also be documented. It needs to be documented for two reasons – to manage expectations of stakeholders, and to keep the implementation team focused on creating the most valuable capabilities in your product. As you get smarter about what is valuable, you need to apply that knowledge and change the path of the project.
This is where communication becomes critical with stakeholders too. Especially the old-school barrel-riders. They are used to being in the barrel, maybe with an occasional “looking good – you’re still falling straight down” message along the way. They are not used to hearing “we’ve decided that gliding over the rocks is more valuable than falling – please extend your wings now” messages. The shocked “what wings?!” response is what they immediately think.
Agile Expectations
An agile project will avoid the big up-front requirements, and gather just enough requirements for right now. What your stakeholder might hear is “agile is magic – we don’t need detailed requirements.” The real message is “agile is better – we don’t need details about the requirements yet. But we will need them later.”
A given team needs the same amount of specificity in requirements to achieve the same result, regardless of project approach. One team I worked with would not set the tab-order in a form to go from top to bottom if you didn’t specify that. It simply didn’t occur to them. Another team was very effective with “gather the following data in a form…” – and they would layout the form, manage tab order, add reasonable field validation, assure proper markup and support for screen readers (for visually impaired users), and implement a candidate error-message/feedback design for users. The first team would never succeed without details, the second team would view those details as wasted effort by me to write them, and by them to read them.
I think most of the initial creators, proponents, and early adopters of agile processes are members of the second camp. They designed agile to not require detailed documents, but to allow it when needed – because they acknowledged that some people need the details. They were reacting to heavyweight processes that were designed to work for “any team” at the cost of slowing down the best teams. Kent Beck told me once that when people tell him about the importance of testing, his response is “so, lack of testing burned you before?” When people stressed the importance of gathering requirements, his response was “you’ve been burned by a disconnect with stakeholders.”
Enough is the operative word here. You have to document enough. You have to communicate enough. You have to set expectations with your stakeholders that things will change. And as those things change, you have to stay joined at the hip with your stakeholders to validate those changes.
While your team is responding to changes (based on feedback from stakeholders and the market and competition and implementation discoveries) with new plans, you have to feed that information back to the stakeholders. It is a two-way communication. And it needs to be a continual communication.
I joined a six month project once in the last month of the project. Here’s a sanitized sequence of events describing what happened.
- Initial vision for the project was defined and requirements defined and tied to goals and value.
- The estimates came back and said “no way it will happen before the business-imposed deadline.”
- The team said “let’s be agile” and chose a component of the vision to do first. That component could be completed by the deadline, and stakeholder expectations were set – (a) do this visible thing first, “meet” the deadline, and then (b) regroup, re-prioritize, and then do the next thing next.
- The team started developing the solution, and worked for 5 months. After 5 months, someone concluded “we won’t finish by the deadline.” A couple weeks later, the estimate was that the team still had between 1/3 and 1/2 of the work remaining to complete the first component of the vision.
- In presumably unrelated events, all but 2 of the internal stakeholders left the company. Literally.
- When the CIO and president of the company engaged the project team, they weren’t thinking “we have an over-run on the first component of our vision” – they were thinking “not only is it not done, but what you’re trying to do is not the right thing.”
- Project was stopped one week after the original deadline for the first component (which was not completed).
It may be that this was unavoidable, but one thing was clear – the “build the first thing first” expectation was not communicated effectively. It didn’t help that the team did not track velocity, so it wasn’t until the end of the project when the “you’re going to hit the rocks” message was first heard by the stakeholder in the barrel. Way too late to affect change. For this and other reasons, I contend that the project was not agile. It was a Chevrolet with a Ferrari sticker on it.
We can ignore the labeling of the process and focus on the lack of ongoing expectation management as the ultimate doom of the project.
Evolving Expectations
Patrick Masi had a couple great comments on our Simple Agile Model article. Patrick pointed out a problem with simple documentation artifacts like this. The early “here’s all we need right now” artifacts were insufficient to capture the full extent of what the stakeholders needed. I’m not sure exactly where things broke down, but Patrick implied that the ongoing clarification with stakeholders was not happening.
This is a completely different manifestation, I suspect, of the exact same problem described above. Thinking about Patrick’s comments is what got me heading down the whole “barrel rider” path in the first place.
I don’t believe I’ve worked with any stakeholders who would say “yes, ignore what we learned this year – it is more important to build last year’s product.” I’ve definitely worked with stakeholders who did not have time to commit to the support of an agile project. The ironic thing about agile projects is that they may do more requirements work through the course of the project than non-agile projects. An agile project will respond to change. That means some work is re-done. It also means that some valueless work is avoided. You can’t categorically say which influence is larger.
One possible source of the pain Patrick’s team felt is mis-set expectations of what is being delivered when. Imagine developing a checkout-process for an eCommerce website. The complete checkout process is too large for a single sprint, so you break it down into valuable atomic deliverables for each sprint:
- Registered customers can buy any products, using previously saved billing / shipping info.
- Individual product and “per order” discounting works and customers can use gift cards to pay for all or part of the order.
- Anonymous customers can place orders (without accounts) and registered customers can modify billing and shipping information.
- Orders can be placed as gifts (with gift receipts and wrapping services), and online call-center reps can interact with customers real-time to help with the process.
This is a nuanced message – basic checkout in sprint 1, with increasing capabilities in each sprint, until “complete checkout” is done in sprint 4. That is a reasonable plan, but requires a more detailed conversation with stakeholders so that they know what is coming and when. You don’t want someone freaking out after the 3rd sprint when they can’t place gift orders.
Another possibility is that the elicitation process before the third sprint (re-visiting checkout again with the same stakeholder) did not tease out that anonymous users must provide an email address (for order confirmation email, and to secretly create a “shadow account” for that customer). All of those details need to be gathered, and it can be harder to spread out the conversation over multiple sprints than it is to have it all up front. This is just incomplete discovery, but with delayed (and recurring) impacts.
In User Stories Applied, Mike Cohn stresses that the brevity of user stories is intentionally designed to facilitate (and I would add “dependent upon”) conversation. I find it to be both a strength and a weakness of user stories. It is strong because you can cover a lot of ground quickly (breadth) and capture a number of stories, much like the list above. It is weak because the requirements documentation does not stand on its own – it requires conversation to fill in the details. I’ve had some success using the “Verify that…” user acceptance tests as the method of documenting those details (depth), in conjunction with the brief, easily consumable stories.
You can write the stories quickly to decompose and schedule across sprints (revisiting the schedule as things change), and then write the verify statements as UAT for each sprint as it occurs.
Another benefit of the UAT is that it is explicit and cuts through the walls of the barrel for a stakeholder who “doesn’t get agile” as Patrick puts it. “Here’s our lightweight multi-sprint plan – now lets define the specific acceptance criteria you need” is a pretty effective conversation.
Conclusion
You have to stay closely connected to your stakeholders – not just for messaging, managing, and changing the big-picture direction of the project, but also to drill down into the details at the last practical moment.
Latest Articles
Comments on all Articles
