Planning by ROI. Hmmm. Isn’t that impractical? In an econometric way, yes. But you can still estimate the relative value of the capabilities / stories you’re planning for your scrum sprints. The point is – don’t look only at value – also look at costs. While “ROI” may be a poor choice of terms, “bang for the buck” is not.
Mea Culpa
In the previous article, Plan Your Next Spring By ROI: Part 1, I made a bad choice of sloppily using ROI (about a dozen times) to describe “bang for the buck.”
Just over a year ago, we showed how prioritizing by ROI delivers value faster. The reason this works is that each sprint (or timebox) has a fixed capacity for work, so you want to cram as much value into that box as possible – not just do the most valuable thing first. Here are a couple images from the previous article that show the impact of using this approach.
[…]
So far, we’ve argued that prioritization should be by ROI, and while accounting for other people, should be user focused. Actually, nothing above is sprint-specific, the same concepts apply to any product planning and development approach.
[Editor: Note – the author has also messed up again, this time by burying the lead at the end of the article.]
Giants. And Their Shoulders.
I had a great conversation over the weekend with Luke Hohmann, founder and CEO of Enthiosys, and realized my mistake. Consider the titles of three (excellent) articles Luke wrote earlier this summer:
- Why Prioritizing Your Product Backlog for ROI Doesn’t Work (Part 1 of 3)
- Developing Attributes for Backlog Prioritization (Part 2 of 3)
- Prioritizing for Profit (Part 3 of 3)
Not to mention giving a presentation at Agile’08.
You’d think I would have noticed, and been more careful in my use of language. Sorry about that.
Luke, in his first article, provides several arguments against econometric assessment of ROI, and I agree with all of his points. In his second article, Luke goes into details about defining attributes for reflecting the goals of internal stakeholders, external stakeholders (buyers and users), and the system. I promised to do the same in this article, but frankly, why reinvent the wheel? Our differences from Luke’s article would at best be nuanced, so the best use of your time (and mine) is to just go read his second article.
Our experience is that successful Agile Product Managers have an almost natural, intuitive feel for prioritizing backlog items to drive profitable growth. Fortunately, this natural ability is greatly enhanced when they make the link between the backlog and their ability to drive profit explicit. As an aside, it is this set of attributes that most clearly distinguish the Scrum-centric role of a Product Owner with the full responsibilities of an Agile Product Manager.
Developing Attributes for Backlog Prioritization (Part 2 of 3)
Including Costs in Sprint Planning
There are two dimensions by which to keep costs in mind when planning a sprint. The first is in determining how much work to schedule for any given sprint. When you use timeboxing to plan a sprint, you’re saying “we have the capacity to do X work per sprint” and “we should only schedule X work per sprint.” Your plans are based on estimates, so you need to have a feedback mechanism to make sure you’re doing a better job in planning each sprint than you did with the previous sprint. Burndown graphs are a great way to do this. The burndown provides feedback within the sprint too, not just at the end.
Mike Cohn, of Mountain Goat Software, gave a great presentation on how to estimate costs for an agile project. Without giving it away, he proposed (starting on slide 14) using planing poker to get quick estimates of user stories [more details] at the product backlog level. Then, more detailed estimates are created for the tasks within the sprint. If your team uses use cases, you can choose to use use case points as a low-overhead estimation method (but not nearly as low as planning poker) to determine the initial estimates of costs.
A Planning Example
You have the following set of items being evaluated in your product backlog. Note that there are a mixture of strategic, user-centric, and code-refactoring items under consideration.
- Profile Editing – As an insurance agent in the community, I want to be able to edit my profile to personalize my identity within the site.
- Data Abstraction Refactoring – The system’s data model is inelegant and needs to be refactored, if ongoing development is to be easier.
- Action Notification – As a manager of agents, I want to be notified whenever my agents submit quotes to potential customers.
- Grouping Items – As an insurance agent, I want to be able to group contacts, proposals, communications and other items, so that I can organize my work.
- Reporting – As a manager of agents, I want to be able to run reports to track financial metrics and activities by agent, product, geography, and time period, so that I can manage my team effectively.
- Serializable Objects – The system’s implementations for data backup and data export are redundant and brittle and should be refactored to clean up the code.
- Customer Centric UI – The user interface is currently product-centric in approach. Our strategy to offer multi-vendor product lines will require agents to focus on customers first.
Using the value-estimation and cost estimation techniques outlined by Luke and Mike you determine the following value and cost estimates for each “story.”
If you were to ignore the cost estimates you would do the data abstraction refactoring and customer-centric UI stories first, followed by the profile editing, action-notification, and other tasks in order of descending value. As it turns out, that is not the way to maximize value over time. At first glance, the profile editing story looks like the biggest bang for the buck.
Create a scatter plot of value versus cost.
[You can clearly see the differences in value and differences in cost of the different stories. If, however, you add the element of value/cost – bang for the buck – and prioritize by it, you will schedule stories in a different order. Value / cost is also the slope of a line from the origin of the graph to each story. If you draw those lines, you see the following:
[To maximize value delivery over time, work across the chart in a clock-wise direction. Profile editing is the first task, but grouping items and data abstraction refactoring are tied for second place. This is different because you’re prioritizing by bang-for-the-buck, not just bang.
Collaboration And Agile
When you consider the agile manifesto
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a planThat is, while there is value in the items on the right, we value the items on the left more.
Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, Dave Thomas
© 2001, the above authors. this declaration may be freely copied in any form, but only in its entirety through this notice.
you see that there is an opportunity to take this even further.
Have Developers Improve ‘The Plan’
I proposed an idea to the development lead on an agile team a couple weeks ago. A couple hours later, he pinged me to let me know he had just applied it and was very excited. A quick whiteboard session (started with a version of the diagrams above), and he was able to immediately make his sprint better.
I told him that when I was still slinging code, and later, when working with teams of engaged and talented developers, I often ran into the problem of having a “cool capability” and wanting to get that capability into the current release. I wasn’t pushing for the capability because it was high bang-for-the-buck, or even high-bang – I just thought it was cool. So I would implement it. Bad Scott, no biscuit.
What I proposed was that when a developer has a favorite feature, instead of just trying to squeeze it in, the developer should figure out how to make it cost less, until it had the highest bang for the buck. Each initial cost estimate is a planning-poker estimate. By spending cycles on design, a developer can figure out a cheaper way to implement it. And that moves the story to the left on the chart.
Your challenge, visually, if you want to bump reporting up in the planning backlog, looks like the following:
[You, as a member of the implementation team, have to come up with a design for reporting that is roughly one-third the cost that was originally estimated. Do that, and it gets bumped up in the queue. [And yes, I realize that very few developers get excited about reporting.]
The product manager‘s job is to make sure you get all the dots in the right place vertically.
The implementation team puts the dots in the right place horizontally.
Everyone embraces the ability to change the graph. The product manager moves the dots up and down as she learns more about the needs of the market. The implementation team moves them to the left (and sometimes to the right) as they design better solutions.
Quoting Luke again – “Our experience is that successful Agile Product Managers have an almost natural, intuitive feel for prioritizing backlog items to drive profitable growth. Fortunately, this natural ability is greatly enhanced when they make the link between the backlog and their ability to drive profit explicit.”
This feels intuitive to me. What about you?
Scott, I’m liking this better. One nagging concern deals with those backlog items that are related to strategic intent and/or political realities. To illustrate strategic intent, we’re working with a team that is merging together numerous web properties acquired over time via acquisition. Each acquired company had their own way of managing user identity and user profiles. They all work — just fine — and from the perspective of an individual Product Manager replacing (refactoring) their own idiosyncractic way of managing identity and profile information with the “corporate standard” is almost certainly going to be perceived as a pure cost. And they may be able to avoid it — but not forever. I handle this by simply allowing Product Managers to capture their need to “toe the line” for strategic intent. How do you handle this in your grid?
Luke
Hey Luke, thanks! And great complex-strategy question. I’ve worked on two customer projects in the last year that are both near your example. The first focused primarily on providing a cohesive and immersive online environment for users of a solution built by different companies. The second focused on the back-end, providing a consolidated view of “the customer” – compositing different notions of identity across the company (and across systems that evolved independently), to leverage silos of data to provide value across areas of the business.
The two projects had different political and strategic drivers. I’ll respond in more depth at lunch today or tonight when I get home.
Luke – maybe I’m missing something in your example, but I would say that if something is a pure cost, don’t do it. Or if the benefit is far in the future, wait as long as practical to do it.
It may be that the ‘value’ is not within the scope of a single web property, but there needs to be value for the business, or it wouldn’t be needed. Maybe I’m accidentally sidestepping your point – if I am, please let me know.
With my first example, several properties were connected by my client in an attempt to create a cohesive environment for shared users. In many cases, a single story would take them (given the current implementation) through multiple solutions (developed by different companies and managed by different teams). There was value in reaching a common understanding of ‘identity’ along a few dimensions: user experience (both in maintaining a consistent LnF and in sharing information “across contexts” from property to property); security (allowing a single authentication mechanism to be used across properties); and analytics (understanding behaviors, abandoned funnels, navigation paths, segmentation). This value was easy to articulate at a “solution” level, even when not isolatable within a “single property” context. On this particular project, we used this “solution level” value to drive a roadmap. Expectation setting with the user base was the penultimate value driver (as it was believed to be the most critical factor in growth, and therefore profit), so both identification of valuable capabilities and the ability to commit to a delivery time frame (measured in halves and quarters) were the dog to the “design ideas” tail.
In the other project, we took the approach of investing in a federated identity model, primarily to meet a political perception that this would make different parts of the business as agile as possible. This created a separate project to deploy an MDM solution (focused on identity) that drove value for the independent projects (that also had to do work) with a focus on eliminating problems in their business areas that were caused by the inefficient coupling of these different “idiosyncratic” ways of managing identity. I guess that would be a Kobayashi Maru to your example, but we were still able to articulate value. The value elements for the individual businesses (of leveraging the MDM solution) justified development in the existing systems, and also drove prioritization of the MDM system development.
Did I get to where you were going with your question?
Oh – FWIW, I believe, given the available budgets, scope of the data and systems, and political environment, that the federated identity model (verus allowing continued distributed models OR forcing a centralized model) was the right one. But the fact that it was (IMO) “right” technically, was not the driver – it was the fact that some people with juice in the organization believed it was right.
Scott –
Yeah, you missed my point. This time I won’t be so subtle. As I pointed out in my blog series, truly extraordinary product management is not just about maximizing the revenue and lowering the costs. It is about managing a complex social system. That’s one reason why I include stakeholder analysis and alignment with corporate strategy. In some cases, you just gotta “pay the corporate strategy tax” and show how your product aligns with corporate strategy, even when you can’t really find a way to justify it economically. I appreciate the power of the Kobayashi Maru, but use it sparingly, my friend. Sometimes, just shoving a few backlog items to demonstrate alignment to corporate strategy is better than wasting precious cycles trying to make it something that it’s not.
Thanks, Luke! Really good advice for all of us. Thinking of it as a ‘tax’ seems like a good way to put it in perspective. Personally, I only like process when it helps me, not when it hurts me. So the idea of scheduling something that isn’t “otherwise next in line” when it makes sense (regardless of what the process says) works for me.
I hope that folks who read this article also read your comments, and can benefit to create extraordinary products.
How does risk factor into backlog prioritization?
A key benefit of agile processes is that the confront and address risks, whether they be requirements risks, architectural risks, deployment risks, or any other risks that occur in the product release cycle.
While you certainly want to prioritize by value (to the customer) and cost, it’s also important to factor risk mitigation into the assessment of priority. If deploying with a particular feature runs the risk of being a deal braker for the entire project, move it to the top of the priority list, as you want to confront and address this risk before wasting resources on other features, no matter how high their value and low their cost.