Writing Verifiable Requirements should be a rule that does not need to be written.  Everyone reading this has seen or created requirements that can not be verified.  The primary reason for writing requirements is to communicate to the team what they need to accomplish.  If you can’t verify that what the team delivered is acceptable, neither can the team.  This may be the most obvious of the rules of writing requirements – but it is ignored every day.

Verifiable Requirements – Revisiting

In 2006, I first looked at how and why writing verifiable requirements is important - as part of the ongoing series on the rules of writing good requirements.  The focus of that article was on testability, as is this one.  When visiting verifiability again, however, I will add that not only must specifications be objectively measurable, but so must goals be written so that you can identify when a solution has met the objectives that justified its creation.

Directly Verifiable Requirements

Most of the requirements we come across can be directly verified.  The only problem with these requirements is that they lack specificity.  The following example is almost verifiable, and only needs a little help:

• The web site must make it easier for users who use site search to find the products they want to buy.

This requirement presents the challenge of resolving the ambiguity of the requirement. This lack of specificity prevents the requirement from being verifiable.  You have to identify how much easier the site search process must become for users.  Findability is a more is better characteristic in the Kano Analysis model for defining requirements.

You have to acknowledge that making it easier to find what you’re looking for is better for users than making it harder – there is an upward slope to the function.  However, there are also diminishing returns.

[larger image]

• A search that includes “the perfect item” is massively better than a search that does not include what you are looking for.
• A search that includes the perfect item on the first page of results is significantly better than one that returns the perfect item on the second page of results.
• A search that includes the perfect item as the first result is only marginally better than a search that returns that perfect item as the second or third result.

Improving findability as articulated above, from a user perspective, manifests in value to your company in two ways: immediate revenue impact and indirect revenue impact.

The immediate impact can be measured in terms of increases in commerce.  The indirect impact results from improving the user experience.  These improvements in experience result in increased word of mouth, as users altruistically encourage others to visit your website, and also result in an increase in satisfaction causing users to return to your site more frequently and make more purchases from you.

You can measure these impacts in terms of

• Conversion percentage (percentage of people who search and then purchase the products found in the results)
• Revenue attributed to users who search (an absolute measurement of purchases of products found via search)
• Site traffic levels (the number of people that visit your site over time)
• Visitor-recency statistics (the amount of time that elapses between return visits for returning visitors)

Conversion percentage, for users who search, normalized against users who do not search, is the most isolated (from other variables and noise) and fastest responding (as a delayed measurement of impact) measurement of the value of making it “easier” for users to search for the products they want to buy.

Your team will design different approaches to achieving these improvements.  You have to estimate both the cost and the potential benefit of each approach.  Balancing cost estimates with potential benefits will yield the ideal requirement – perhaps a 10% improvement.  [Note: I've collapsed at least another article's worth of balancing cost versus benefit, and multiple articles of "understanding and measuring site search" into one paragraph here, in hopes of staying on task with writing verifiable requirements.]

Rewriting the requirement as follows makes the requirement verifiable:

• Before: “The web site must make it easier for users who use site search to find the products they want to buy.”
• After: “Users who search on the site will be at least 10% more successful at finding the products they want to buy when using site search.”

Impossible to Verify Requirements

Often, stakeholders will present us with requirements that are impossible to verify (as requested).

• The home page needs to load fast.

At first blush, this looks just like the previous requirement (easier search is similar to faster page load).  You can use the same techniques to determine a measurable “requirement” like “the home page needs to load in under 1 second.”  However, that’s not a good requirement, because it is not an implicitly valuable requirement.  This statement provides no insight into why a fast-loading home page might be valuable to a particular user or why it would be valuable to the company.

To address this issue, you have to meet with the stakeholder(s) and understand the actual underlying requirements.  You will ask why, determine motivations and underlying problems, and apply active listening skills to discover the underlying requirement.

The problem is that the statement, “the home page needs to load fast” is a specification.  In Requirements Documents – One Man’s Trash…, I first used the following diagram to show how different people in the process of designing software view their piece of providing a solution.

A developer may receive a spec – “home page must load in under 5 seconds” and feel like the why component is perfectly well defined – it is a matter of context and perspective.  Another way to think about this is that the problem is with the problem statement.

A product manager, however, needs to do research that follows a path like the following:

Given that we are building an eCommerce website, we know that people have expectations around page load times (per Forrester / Akamai report (requires registration)).

[larger image]

Further, those expectations manifest as people abandoning slow-loading websites (from the same report).

[larger image].

We know from this market research that page-load times (for websites) represent another more is better Kano attribute, but one that has must be characteristics at the extremes.

[larger image]

While more speed is better, what the market research reveals is that for any given person, there is a minimum-speed threshold, at which speed is a must-be characteristic – too slow, and you lose customers (immediately).

A  combination of market research and elicitation will reveal that there is a true underlying goal of being fast enough to satisfy as many users as possible.  Combining this with practicalities of scaling your solution, and the associated costs; in the context of a particular solution design (an eCommerce website), you will arrive at a goal that allows you to rework your requirement so that it is verifiable.

Note: The requirement in this example is a subordinate requirement to a goal focused on maximizing conversion percentage (the percentage of customers who visit the site making purchases) – with a recognition that a major source of non-conversion is abandonment of the site, and that a large contributor to visitor abandonment is page load times.  An Ishikawa (or other model) will help you articulate this complexity and formulate requirements at a level of abstraction that is both valuable and actionable.

Rewriting the requirement as follows makes the requirement verifiable:

• Before: “The home page needs to load fast.”
• After: “No more than 10% of potential site visitors will abandon our site before viewing the page.”

Indirectly Verifiable Requirements

Some requirements are impossible to literally verify, and must be inferentially verified.

• The site must support 1 million visitors per day, with a peak of 10,000 simultaneous with page-load times under 5 seconds.

Models are Models, Not Reality.

Imagine trying to create a test by organizing a million visitors to hit your SaaS web solution on the same day, with at least 10,000 of them hitting the site simultaneously.  That is completely impractical.  That doesn’t however, invalidate the requirement or make it untestable.  The combination of modeling, hypothesis formulation, and extrapolation gives you a reasonable way to verify that your solution probably meets this requirement.

You’re already used to the concept that models are representations of reality – think about the maps you use – a map may have a scale like one inch equals one mile.  The map is a model of reality.  A map where one inch equals one inch would be impractical.

An effective approach to measuring this type of scalability requirement is to simulate users (with load testing and realistic user-behavior scripts) with automation.  That takes care of most of the coordination problem.  However, the cost of simulating a million users and 10,000 simultaneous users is high.  You can, more realistically, measure the site’s performance when 10, 100, and 1,000 simulated users simultaneously access a server.  You can extrapolate from those results to estimate how the system is likely to perform under 10,000 user loads.

Note: Extrapolation is dangerous (follow the Elvis link below)- you have to justify why extrapolation holds true, and identify when it does not, to minimize the risk of invalidating your hypothesis.  Make sure you have confidence in the engineering prowess of whoever is doing your extrapolation and modeling.

[see tip #5 of these ROI calculation tips for the Infinite Elvis anti-pattern]

While this requirement does not to be rewritten in order to be verifiable, it does need to be augmented:

• Original: “The site must support 1 million visitors per day, with a peak of 10,000 simultaneous with page-load times under 5 seconds.”
• Additional: “Testing of the site must show that 500 simultaneous users following <user script X> will yield no more than 1% page load times over 200 ms.”

Conclusion

Every requirement you write must be verifiable.  When you can’t verify something, and can’t rewrite it into a verifiable form, that should be a sign that either it is a vision statement or a red herring.  Vision statements guide how we approach creating products and engage markets – they are valuable, but they are not requirements.  Red herrings are well-meaning but ill-advised inputs into the process that need to be culled – they are neither valuable nor requirements.

Make sure all your requirements are verifiable.

Unless you live in a world filled with unicorns and rainbows, writing realistic requirements is critical.  When you set unattainable goals, the best result you can hope for is a frustrated engineering team.  Write requirements that are attainable, and your team will surprise you with what they can achieve.

Attainable Requirements – Revisiting

A little over three years ago (ten unicorn years), I wrote Writing Attainable Requirements, as part of the Big Rules of Writing Requirements series.  In that article, we focused on the unique situations that every team faces – politics and people, and the shared challenge of implicit practicality.  Those factors are as important today as they were then.

Defining Attainable

Attainability, for requirements, is simply the answer to the question – can this be reasonably done?  Most, but not all, of the answer to that question comes from understanding if your team can build and test a solution to the problem you identify with your requirement.  Being able to create a solution requires that the problem you’re solving is tractable, and that the team creating the solution has the skills to solve it.

Being able articulate a problem and it’s solution is necessary, but insufficient – solving the problem requires that there be a feasible solution.

As the founders of the agile development movement explained in their pitch for “low overhead” process – people trump process.  I think it was Alistair Cockburn who added, “but politics trumps people.”

Finally, process is the often-overlooked element of determining attainability.  Realizing the benefits of a novel solution to an existing problem usually involves process changes – sometimes significant ones.

The next few sections go into more detail on each of these – tractability, feasibility, people, and process.

Tractability

Goldilocks ruled out the large bed because it was too large.  Eating the elephant, drinking from the fire-hose, boiling the ocean – these are all common phrases in the American vernacular, because so many people try to do too much (at one time).  There are many ways to decompose a large, intractable problem into smaller, addressable components.  Practicality requires that the problem you’re trying to solve is not too large.

If everything you do is small, however, you won’t ever accomplish anything big.  That’s where strategic product management shines.  As part of understanding your market, you identify large, valuable problems to be solved.  To solve those large problems, you have to break them up into smaller problems, and then address them individually.  You also have to make sure your team understands the big picture and the larger problem that ultimately needs to be solved.  A great way to share context while decomposing problems is with the use of Ishikawa diagrams.

And zooming in…

Maintaining context is a particularly critical component of working with agile teams, where each deliverable’s possible size is constrained further to what a team can accomplish in an iteration.

[click for larger version]

For some problems (and some teams!) this may be enough information to provide the context to develop a product backlog.  And your agile team will move forward into managing their own execution from the product backlog stage.  For larger or more complex problems (such as this example), you will need more detailed communication before diving into user stories / use cases.

The same Ishikawa diagram, with more detail, looks like the following:

[click for larger version]

Agile Product Management: Providing Context

This reduced granularity represents the smallest go-to-market atomic requirement.  Further decomposition results in smaller requirements, but they become too small, because they are no longer independently valuable.  I’m not talking about “the sum of the whole is greater than the sum of the parts” stuff here – I’m talking about “if you only get X, without Y and Z, it has no value” stuff.

Feasibility

Even after decomposing market problems into addressable chunks, you still have to make sure that there are feasible solutions.  One of the brighter software developers I worked with back in the day used to say “We’re writing software – it’s all ones and zeros – we can do anything.”  He also followed that up with “…but should we be doing that?”  The team I was working with at the time probably could do just about anything.  But not everything can be done with a given budget and a specific deadline.  Some things are simply infeasible.

Scrum teams measure velocity – a reflection of how much software they deliver in a given iteration, as a function of how much work they believe is involved in delivering.  This is a great “protected” measure of efficiency.  The team is protected from requests to do work that has little or no value – by measuring throughput in terms of effort, a team can get more efficient at doing more.  A requirement is infeasible, and therefore unattainable, when it exceeds what the team can accomplish given a fixed amount of capacity.  Project managers refer to this as the Iron Triangle – given scope, cost, and quality, pick any two as inputs and the third will be a variable output (absorbing the impact of uncertainty that arises during the development process).  I prefer to use timeboxes rather than that rusty old triangle, to emphasize my perspective that quality is inherent in delivery, and should not be a “variable output of uncertainty.”  Because quality is what usually bears the brunt of uncertainty.

plus  yields

When the solution to a defined problem is prohibitively expensive, it is not attainable.  The notion of cost should also be considered in the context of value.  Requirements with higher value can justify implementing solutions with higher costs.

Prohibitively expensive solutions make requirements unattainable.  Relatively expensive (relative to their value) solutions also reflect unattainable requirements.  You need to collaborate with your implementation team to understand the costs associated with any particular requirement to know if it is attainable.  This is one of the strongest arguments against throw-it-over-the-wall interactions between product managers and implementation teams.  Without that feedback about feasibility and cost of a solution, you can’t assure that the requirements you are proposing are attainable.

Regular readers of Tyner Blain will realize that this is just an alternate way of stressing the importance of considering ROI when defining requirements.

Politics

There are political realities that every organization faces.  It could be a benevolent dictator CEO, or an executive with an agenda or pet project.  It may simply be that there is value in the problems you’ve identified, but solving them is not aligned with your corporate strategy.  You may be focusing on dominating an existing market, where expansion into new markets is the company’s focus – or vice versa.  You may be focusing on top-line growth opportunities, while the company is fixated on cost-reduction in a time of economic contraction.

Each of these scenarios causes you to be fighting an uphill battle internally.  When that hill is too steep, you’re writing unattainable requirements.

Process

When deciding what to build, you also have to think about how you launch.  Launching, as Dave Daniels regularly points out on his Launch Clinic blog, is not just putting your product up on your website for sale.  You have to think organizationally about all of the other moving parts:

• Do your customers have the ability to absorb an update to your software?
• Can your customer service reps be trained in time to promote and support the new capabilities?
• Will your sales team be able to communicate a message that helps them close deals based on what you’ve just built?

Process is more than just the process of typing, compiling, and testing code.  Process includes everything needed to successfully go to market and sustain your product.  If you’re defining requirements that are too far ahead of your market they are not attainable.  If your requirements embody process changes that are too difficult or expensive for your customers or your organization to absorb, your requirements are not attainable.

Conclusion

Attainable requirements are requirements that

• Have a reasonable cost to implement,
• Are aligned with your company’s strategy and stakeholder’s agendas, and
• Result in solutions that can be consumed by your company and your customers.

Make sure you’re writing attainable requirements.

Concise requirements give your team a useful, easy to read and easy to change understanding of what must be done.  Great requirements exist to do three things:

1. Identify the problems that need to be solved.
2. Explain why those problems are worth solving.
3. Define when those problems are solved.

Concise Requirements – Revisiting

In the three years since we last looked at Writing Concise Requirements from the Big Ten Rules of Writing Requirements, the iPod evolved to give us a better experience.  Let’s see if we can do the same with the topic of brevity in requirements.  The size of our community here has grown ten-fold, and the people who were here back then have grown just as much.  It makes sense to look at this again.

Writing concise requirements is not just minimizing the number of words you use.  Writing concise requirements is presenting the most important information in the easiest format for your audience to consume.

Concise Requirements Identify the Problems That Need to be Solved

Ultimately, requirements are the problems that we choose to solve.  A concise requirements artifact (formal document, index card, photo of a whiteboard, whatever) is one that has the highest signal-to-noise ratio possible.  You’re maximizing the amount of communication, and minimizing the cost of communicating.

You want your requirements document to read like the lines, not the points.  If the line (the signal) is what you really want, and you communicate a big pile of points (the signal, hidden in the noise), you run the very real risk that your audience will misinterpret the signal.

User stories provide the best example of clarity that comes from conciseness.  The format you should use, based on Mike Cohn’s great book, User Stories Applied, is

As a [role], I want to [do something] [with some frequency] so that I can/will [achieve some goal]

The user story is not always the right communication format – it depends on what problems you’re solving, and who is on your team.  Sometimes, use cases work better than user stories.  Conciseness is important for use cases too.  Start with a good use case name.

Concise Requirements Explain Why Those Problems Are Worth Solving

Last week’s article on Valuable Requirements focused on why particular problems should be solved.  Your focus should be on value, and that article discussed five ways to assure that your requirements are valuable.  One of the techniques, the use of an Ishikawa diagram, provides a method for identifying the root causes of problems.

[larger image]

Imagine that you have a collection of user stories, representing problems to be solved for your users.  You need a way to demonstrate why this user story should be implemented and why that one shouldn’t.  You can often use the Ishikawa diagram in the same way.  A particular goal is achieved when a user is able to do something.  Perhaps several somethings are required.  The point is that you can use the Ishikawa to drive home the point – if this set of user stories are all implemented, then this goal will be achieved.

There are two reasons this is important – first, you’re providing context for your team.  They understand why they are doing something.  Second, you can make changes easily, because you can see the impact of those changes.  By understanding the cause-and-effect relationships between problems and their values (user stories and their goals), you can see the impact of changing one or the other.

Concise Requirements Define When Those Problems Are Solved

Clarity is the goal of conciseness.  It isn’t enough to say “work on this.”  It’s important to know why you’re working on it, but that still isn’t enough.  You have to know when you’re done.  When you’re defining problems to be solved (and therefore solutions), you must also define the measures by which the solution will be judged.

A measurement of success for “Cost of Operation is Too High” might be “reduce costs of operation by 10%.”  This gives you a testable criteria for knowing when that problem has been sufficiently solved.  Sticking with the Ishikawa, you can also map out the strategy for achieving that lofty goal.  You can say that the goal is to reduce fuel expenses by 20%, reduce cost of maintenance by 5%, and reduce payments by 15%.  This process continues – a 20% reduction in fuel spend requires that you operate with your tires within 5% of nominal pressure, and that you reduce the aerodynamic drag coefficient by 7% (or whatever).

This gives you clarity in your objectives.

User stories, when combined with user acceptance criteria, provide that last connection of testability that lets your team know when they are done.

There aren’t many things more frustrating to a development team than having them solve the wrong problems.  One of those things might be having their solution be rejected because it isn’t enough.  Writing acceptance criteria clearly and concisely lets the team know exactly when they can move on to the next problem.

Providing a crisp understanding of acceptance criteria also facilitates iterative development.  One challenge teams always face is making improvements that fit within the timebox of a given iteration.  Imagine a user story with 4 acceptance criteria, where the story is too big to complete in one sprint.  When talking with your development team, you may find that the story is too big because satisfying all of the acceptance criteria is too big.  This is where many teams miss an opportunity – by defining all of the acceptance criteria that are believed to be needed eventually and requiring that they all be implemented now.  One of those criteria is going to be more important than the others.  Implement the story such that is satisfies the most important criteria (timeboxing) now, and rewrite or enhance it to meet the additional criteria later.

Conclusion

Software Product Success depends not only on identifying the “right stuff” to build, but on making sure the team builds it and builds it right.

Concise requirements improve your ability to communicate with your team, thereby improving their ability to build the right stuff right.

Writing valuable requirements is important.  It doesn’t matter how well your teams execute if they are off building the wrong products / capabilities / features.  The right products and capabilities are the ones that have relevant value.

• Valuable requirements solve problems in your market.
• Valuable requirements support your business strategy.
• Valuable requirements solve problems for your users.
• Valuable requirements meet your buyers’ criteria.
• Valuable requirements don’t over-solve the problems.

Valuable Requirements – Revisiting

A little over three years ago, I compiled the Big Ten Rules of Writing Good Requirements, which ended up with an even dozen “rules.”  The first rule was writing valuable requirements.  Three years is a long time.  Our environment has changed.  Many great insights have come into our neck of the software development woods from many inspired voices.  I’ve personally learned a lot.  My focus has changed from being more focused on product specification (back then) to being more engaged in business strategy (today).  Our audience here has grown ~ 10x since the original rules were published.  My dog has tripled in size.  Perhaps my writing has even improved.

Given that, it makes sense to revisit the topic now.

Valuable Requirements Solve Problems in your Market

I struggled a little about starting with “strategy” or starting with “the market.”  Pragmatic Marketing’s Framework (recently updated) starts with the market.  When I first watched the great video, where Jim Foxworthy introduces the updates to the grid, I was a little concerned about that.  Market first, strategy second.  Why not strategy first and market second?  I decided that I was happy with their presentation, since the framework is a tool for product managers.  Product managers usually have responsibility for a market, as a component of their company’s strategy – so the sequencing makes sense for a product manager audience.  Jim also puts it in perspective – their framework is focused on the company’s strategy for attacking a particular market.  I’ll stick with that here too.

To understand what problems are valuable for a particular market (or market segment) you have to approach it from your customer’s perspective.  What are the problems they are trying to solve?

Your customers might be distributors of retail products, who’s business it is to acquire, store, and redistribute small products.  They are in a business where their customers value accuracy, timeliness, and low cost of delivery.  Your customers may value solutions that allow them to more accurately redistribute products (they receive pallets full of items, and then redistribute them a case at a time).  They may value solutions that allow them to adapt to changes in orders with minimal turn-around time.  Or they may get the most value out of cost reductions.  Talk with your customers, listen to them talk about their problems.  When I was working for an enterprise software company years ago, our CEO stressed that he wanted to be solving the problems that keep our customer CEOs up at night.  Find out what those problems are.

The next challenge is in breaking those problems down into something actionable.  If your customers’ biggest problem is cost reduction, how do you solve it?  The first step is to understand where the costs are.  One way to visualize and decompose the problems your customers face is with an Ishikawa diagram (check this out to learn how to use an Ishikawa).

The following diagram shows a decomposition of “The cost of driving is too high” problem:

[larger image]

Now you’ve identified a set of problems that are valuable to your customers.  The Ishikawa can help you make sure you’re identifying problems, not the manifestations of problems.

You also have to understand what solutions are already available to your customers.  If you have a competitor that already offers a very good solution to one of the identified problems, your customers will find less value in a solution from you.

Valuable Requirements Support Your Business Strategy

With a set of problems in hand that are worth solving, you have to figure out which ones are aligned with your company’s strategy.

In the Ishikawa above, one of the problems that car owners face is excessively high payments (on their car loan).  Another problem is that the cost of maintenance is too high.  If your company provides financing products (loans and leases), trying to solve the cost of maintenance problem for your customers is probably out of alignment with your strategy.  Making the financing of a vehicle more affordable (while still profitable for your company) is probably a well-aligned problem.

Your company will also have a strategy for engaging the market – target market share levels, target market segments to penetrate, etc.  You may be trying to become a visionary company with your finger on the pulse of your market – or you may be trying to get mass adoption of your product by solving a very common problem, but solving it better than your competitors.  Your strategy for winning in this market may be to differentiate your offerings by solving different problems than your competitors, or defining a unique market.

Another way to think about alignment with your business strategy is to think about the owners of your company’s strategic goals – your internal stakeholders.  Most projects will fail (or be killed) without internal champions who believe in the ideas.

When you’re aligned with a part of your company strategy,  you’re aligned with whoever is the owner of that component of the strategy, and you’re providing value to that stakeholder.  Sometimes there are many components and stakeholders.  You can adapt some process-improvement prioritization techniques that Roger Burlton teaches to get an understanding of which goals are important to whom.

Valuable Requirements Solve Problems for Your Users

Products are used by people.  People use those products to accomplish goals.  They may be using your product for their employers, in which case they have “practical goals” (finish quickly) that represent how their company’s goals (lowered costs) are realized.  And those people usually interact with other people.  You can quickly build a visualization of who are the users (and indirect users).

[larger image]

You can think of this as an ecosystem of users of your product.  Develop personas and their goals to represent these users.  You can apply the same Ishikawa-based problem-decomposition technique when needed to make sure you’re uncovering the real problems (too many people abandon the sign-up process) and not the manifestations of those problems (users have to click too much).

When your users are not your customers, your users have personal and practical goals that represent their individual contributions to achieving your customer’s goals.  And your users achieve those goals by doing stuff.  You can represent that stuff with user stories or use cases.  The key element is to make sure that you’ve identified the valuable activities, the ones that are required to achieve goals.

[larger version]

The above diagram is used when assessing the completeness of requirements, which is an element of assuring valuable requirements.  If a goal has value, you have to identify the set of user stories required to achieve the goal.  Those user stories therefore have value.  User stories that don’t support a goal don’t have value.

Valuable Requirements Meet Your Buyer’s Criteria

The people who buy your products are sometimes not the people who use your products.  Even when the same person is both buyer and user, that person’s mental model for buying is different from their model for using a product.  If you can’t convince someone to buy your product, it doesn’t matter how great it would have been had they bought it.

Here are the opening soundbites from an earlier article on buyer personas.

• Buyer Persona: If you build what he thinks he wants, he’ll come.
• User Persona: If you build what he actually needs, he’ll come back.

And the closing summary points (it’s a long article):

• Buyer personas make purchases when products appear to address their internal view of what the problems are.
• User personas love products when those products solve the real problems.
• Don’t confuse buyers (who need to buy products to solve user problems) with users (who need to solve their own problems).
• When buyers and users are the same people, acknowledge the buyer-goals distinctly from the user-goals.

There are also several really great insights in the discussion thread – including great comments from Shaun Connolly, Edele Revella, and David Meerman Scott!

Valuable Requirements Don’t Over-Solve the Problems

More is better, right?  Not always.  Sometimes, “more” is a waste.  There are really two ways to think about how much is too much.

The ROI of Incremental Improvements

Because of the law of diminishing returns, the next “more” is always worth less than the previous “more.”  Think of it in terms of improving fuel-economy.  If you have a car that gets 10 mpg, and you drive 100 miles a day, you have to buy 10 gallons of gas a day.  If you improve the mpg by 10 mpg (to 20 mpg), you’ll save 5 gallons of gas per day.  Huge value.  If you improve the mpg by another 10 mpg (to 30 mpg), you’ll save an additional 1.3 gallons of gas per day.  Some value.  Another 10 mpg buys you 0.8 gallons per day.  Diminishing returns.

The cost of achieving “more” is also a factor.  The simplified diagram above shows a linear representation of cost as a black line.  The diminishing returns of value are represented as the red curve.  The optimal investment point is where the two curves are tangent (have the same slope) – marked with the blue circle.  Less investment leaves money on the table – additional investment is done at a loss.

Good Enough is Enough

You don’t need to super-solve the problem.  How much more would you pay for a car that got 10,000 mpg than one that got 1,000 mpg?  If you drove 100 miles a day, the difference between the two (from a value standpoint) is trivial.  Over the course of 100 days, you would save 9 gallons total with the car that had ten times the fuel efficiency.

Why haven’t microwave ovens been getting faster every year? Because the move from a 1-hr baked potato to a 5-minute baked potato is good enough.  You don’t need to get it under 4 minutes.

This is known as satisficing - stopping when something is good, not trying to make it “perfect.”  Additional “goodness” will not result in enough additional sales to justify the investment.

Conclusion

The tagline for Tyner Blain is Software Product Success.  On an elevator, I explain that you have to not only “build stuff right” but you have to “build the right stuff.”  And the right stuff is the valuable stuff.

Define valuable requirements to make sure you’re building the right stuff.

Jump forward in time to the day of your next big product launch (first release, new features, new market segment, etc).  And your site/application crashes due to the “unexpected” demand.  All you can do now is look for a bucket of water to put out the fire.  What could you have done to prevent this disaster?  Jump back to today and start doing it!

Backwards Planning

Depending on how you look at things, this is a backwards planning exercise, or a variation of the  remember the future innovation game, or risk management, or proactive product management.  You can avoid a disaster by imagining what might happen, then hypothetically figuring out why it (would have) happened.  That leads to planning how you could prevent it.  And now you’ve left the dream world of a Gedanken experiment and returned to the real world of product management.

Problem Triage

The way to approach this is straightforward.  Imagine some failure scenarios and the importance of preventing them:

1. Imagine a failure scenario.
2. “Predict” the likelihood of the failure.
3. “Estimate the impact of the failure.
4. Repeat for each scenario

You can prioritize your failure scenarios by multiplying the likelihood of each with the impact of each, and sorting them from largest to smallest.  Then determine which ones you’re willing to address, and which ones you’re willing to risk.  You may not be able to predict the likelihood of some failures (at least until you do a root cause analysis).  Take each of these and put them directly above the scenario with the next highest impact.  The rationalle is that these are so bad, that you really want to find out how likely they are to happen.  Once you predict likelihood (see below) you can reprioritize.

Root Cause Analysis

For the failure scenarios you choose to address, the next step is to do a root cause analysis that identifies why it might have happened.  The best tool for capturing this analysis is an Ishikawa diagram.  Consider that one problem you might face is your website crashing.

[click for larger version]

Essentially, you can crash your site by having too many users, too many concurrent users, or too many concurrent sign-ups.  Developing a cause-and-effect diagram (another name for an Ishikawa diagram) is usually an iterative and exploratory process.  You probably won’t create the simple version above first.  You may ask your implementation team “What can cause the website to crash?”  For each of their answers, you identify when that situation can happen.  Or you start top down.  Most likely, a mix of the two.  Your completed root cause analysis may look like the following:

[click for larger version]

At this point, your team can probably predict many (maybe all) of the root causes of a website crash.  The predictions may be conditional – “we can handle 10 concurrent users, but 20 probably kill us, and 100 definitely would.”  Developers are notoriously good at answering questions with conditional statements that reveal the nuances of their thinking.

Remember that you’re looking back from the future.  At product launch, what are you hoping for / reasonably expecting?  For this example, assume it is 10,000 total users, with 100 concurrent users (normally) and 500 concurrent signups.  You determine these numbers by working with your PR, marketing, or mar-com people (or wearing those hats, when it is all you).  Your plan is to do a big launch with a demo and a promo code for signup.  You know your audience will have internet connections, and will have twitter running at the time of your presentation.  You expect/dream of an immediate burst of signups, followed by tweets and word of mouth, and eventually blog articles causing additional growth over the next couple of weeks.

Use this data to feed back into the developer’s conditional responses.  If you’re like me, you will have found “absolute certainty of failure” from something.  And you may have even identified the thresholds for each element.  For example, database loading can handle 75 concurrent users, but with the current implementation, you only have enough database connections available to support 25 concurrent users.

Jumping back to the present, you now have some very discrete, and very important things to do before your launch.  If you need to, revisit the prioritized list of failure scenarios.  By looking at the next level of detail, have you found that the order of importance (to fix) has changed?  What about the “must fix” versus “willing to risk” line?  Has it moved?

Fold the “must fix” items into your backlog, and prioritize them relative to the other capabilities on your roadmap.  As a side note – make sure you’ve built in some testing to make sure you actually prevent the problems.  This might even be a great opportunity to implement “performance regression tests” – it is not enough to prevent bugs, you have to prevent slowdowns.

Rethinking the Problem

Without going into details on how the team will solve each problem, make sure that together you keep the Ishikawa diagrams in mind, and see how any proposed solutions might “reappear” on the diagram.  For example, rewriting your database connections to use asynchronous processes and a set of pooled connections may prevent a crash, but it may really hurt performance.  You may not have time to find an elegant solution.  So stop and rethink the problem.

At this point, you’ve said

1. Given a marketing plan / launch strategy, we would crash the website.
2. We can make changes between now and the launch that will double the number of concurrent users we can support (or whatever), but that is not enough to support the launch strategy.
3. Solution: Change the launch strategy.

Maybe you can’t support a wide-open promo-code based signup.  You should modify your launch so that it can only create as much demand as your product (including pending improvements) can support.  Maybe you limit it to the first 1,000 new users (probably more code to write to enforce the limit).  Maybe you launch with per-user invitations, where you can control the speed of propagation of invites (start with 100, when those have been sent, make another 100 available, etc).

Entire Team Problem

This is a problem that is solved collaboratively, by the entire team.  It is not just a “go write the code” problem.  What your product can support at a launch should drive how you choose to launch, just as how you choose to launch should drive what you want your product to support.  

You may have to delay a key capability in order to scale.  Does your marketing team know this?  Slightly less bad than crashing would be announcing a feature that is disabled.  Still need to announce the feature?  Pre-announce it: “Coming in a month…”

This stuff is important for every company and product, but it is especially critical for start-ups.  As a start-up, you have limited opportunities to grow, and a limited safety-net to catch you when you fail to capitalize on those opportunities.  So make sure everyone (not just the development team) is aligned to make the best use of each opportunity.

Conclusion

You have an opportunity to prevent problems.  All you have to do is imagine that they have happened in the future, figure out why they would have happened, then do what it takes (in software, or organizationally) to prevent them.

Agile development methodologies succeed because they help development teams be as effective as possible.  Development teams do not, however, work in complete isolation.  The company they work for has a strategy.  The company manages a portfolio of products, and targets a particular product at specific market problems.  Within that context, an agile team can thrive.  What’s the best way to provide that context?

Agile Development in Context

Mike Cohn of Mountain Goat Software gave a presentation to the bayXP group in March 2007 on agile estimation and planning (video and pdf at link).  As part of setting the stage for his planning presentation, he describes the software development ecosystem as an onion.  So did I, coincidentally, in 2006.  The gist of both onions is that software development is implementation, in the context of design, driven by requirements, formed to address market opportunities.  Mike’s onion provides specific insights into the execution approach of an agile team, so let’s frame this conversation using his terms (but a new visual).

• A company has a strategy.  [Example: "Organize the world's information..."]
• A company has a portfolio of products that is intended to provide a mechanism by which the company can achieve its strategy.  [Example: Gmail, Chrome, Gears, Reader, Docs, etc.]
• Each product (or service) the company creates is an intentional part of the portfolio, targeted at specific markets or market segments, with an intention to solve specific problems.  [Example: Gears allows people to consume online content when offline.]
• A product is delivered through a series of releases. [Example: Chrome release 0.2.149.30.]
• The work that goes into a release, when practicing incremental development, may be decomposed into multiple iterations.  [Example: Chrome iteration 0.2.149.29, not released, but built and tested]
• In scrum, the completion of tasks within an iteration are managed daily.  [Example: Today, I will add the 'check for updates' to the 'About Chrome' popup.]

Many people talk about the agile process as if it encompassed the entire perspective shown above.  It doesn’t.  When you talk about the software development lifecycle from the strategy level down, you’re really talking about business strategy.  Agile development happens in the context of a business strategy, a product portfolio designed to achieve those strategic goals, and the product in that portfolio that is being developed.

Saeed Khan puts it pretty well in his first article about agile / scrum:

Keep in mind, Agile/Scrum is a DEVELOPMENT methodology. It is a great model for developers and engineers and other R&D team members to work and communicate more efficiently. There are very clear benefits to this model. It provides greater visibility into current work status, work remaining, can identify development hurdles earlier and can communicate them outward more easily.

I agree with Saeed.  In the diagram above, the layers that represent “what we do” have black text and borders (Strategy, Portfolio, and Product).  The layers that represent “how we do it” have white text and borders (Release, Iteration, Daily).  In Mike Cohn’s presentation on agile planning, he specifically calls out that agile planning happens in this (inner) scope.

Managing a product backlog (in scrum) is the place where things blur a little.  Determining what goes into the product backlog is “what we do”, prioritizing those product backlog items is “when do we do it”, and fitting them into individual releases (timeboxes and rolling-wave planning) is “how we do it.”  As agile teams stress, communication here is the key.

The true challenge that companies face is to provide agile development teams with the context needed to develop software.

Providing Context to Agile Teams

Product management is primarily a strategic activity, combining market insights () with company strategy to design a product portfolio, and to determine which problems a particular product should solve, for a particular market segment.  This leads to a focus on buyer personas and user personas.

Establishing and maintaining the connection between market insights and agile development teams will develop a distinctive competence for your company.  One key is to connect stakeholder goals with implementation activities.  This involves translation from the language of business to the language of developers.

An effective bridge across that communication divide has to be visceral and very straightforward.  Many agile team members rail against anything that looks like overhead, and the manifesto even codifies this disposition – “working software over comprehensive documentation.”

Our magic decoder ring must be simple, explicit, and light-weight.  Sounds like a job for the Ishikawa diagram.

Ishikawa Diagrams for Agile Product Management

The Ishikawa diagram, also known as a cause-and-effect diagram or as a fishbone diagram, can be used to describe and decompose market problems.  Consider the following near-real-world example:

You’re developing a product for sales reps.  The key to your product’s success is user adoption, and you believe the way to get that adoption is by helping sales reps to maximize their compensation.  Sales reps are generally managed via commission-based compensation models.  You establish “Maximize Sales Compensation” as a goal for your software (for these users, in this market segment).  You then acknowledge that there are a series of “smaller” problems that have to be solved before sales reps actually can maximize their compensation.  An Ishikawa diagram of your results would look like the following (consider only the main branches):

[click for larger version]

For some problems (and some teams!) this may be enough information to provide the context to develop a product backlog.  And your agile team will move forward into managing their own execution from the product backlog stage.  For larger or more complex problems (such as this example), you will need more detailed communication before diving into user stories / use cases.

The same Ishikawa diagram, with more detail, looks like the following:

[click for larger version]

For the largest and most complex problems, this is still not enough.  You can take each major branch of the Ishikawa diagram and create a child Ishikawa diagram.  However, from an agile standpoint (as in “staying close to your market”), you don’t want to do all of that detailed analysis up front.  Once you’ve prioritized the main branches in your main Ishikawa diagram, you will want to go to the next level of detail only for the first branch.  You want to make sure that you are delivering “working software” – not “comprehensive documentation.”  Delay your detailed analysis to the last practical moment.

Assuming you chose “Improve Predictability of Sales” as the first market problem to address with your product.  And within that choice, you are going to tackle “Align Solutions to Customer Problems” first.  Your child Ishikawa diagram could look like the following:

[click for larger version]

At this point, you would start writing user stories (or defining use cases) around recording customer problems, searching for ‘comparable problems’, predicting the problems a customer might face (before your first sales call), etc.  Those user stories will then get mapped to releases and iterations, and their implementation will be managed through the daily standups.

Outbound Communication of Agile Delivery

This problem-decomposition approach was presented top-down, specifically around providing the needed context to an agile development team to guide their design and implementation activities, giving them the opportunity to intentionally solve valuable problems, in alignment with the company’s strategy.

This same approach can be leveraged in outbound communication of what the team will deliver (and when).  You can easily construct a product roadmap that is actually an “agile problem roadmap.” [Ed: can I trademark that?  Probably not, since Enthiosys already does it, but without the word problem.]  In the short term, talk about specific problems (“Align Solutions to Customer Problems”) that you are solving.  In the longer range plan, talk about the next level of problems (“Improve Predictability of Sales”).  The team is not committing to a widget or feature.  The team is committing to providing a solution to a particular problem in a given release.

When you commit to a particular feature, you inhibit your ability to change as you learn.  And that’s bad.  You need to be able (after each iteration) to say “This problem is still valuable, but our previous idea about how to solve it turned out to be a bad idea.”  If you communicate at the feature level, you’ve added overhead to your process – you now have to manage expectations and update your communications, EVERY time you change designs.

You introduce another problem – these Ishikawa diagrams are so easy to read, and you’re now managing your roadmap based on problems being solved in a given iteration/release – should you tell everyone?  Many product managers sidestep the “Do we make our roadmap public?” question because they don’t create an easy to consume roadmap.  With this approach, you do.  So you have to decide if it needs to be a secret.

Zero-Overhead Planning

Early in this article, I said “agile team members rail against anything that looks like overhead.”  Creating Ishikawa diagrams is zero-overhead.  Or nearly so.  99% of the work that is displayed through the Ishikawa is work that you have to do anyway, to have an intentional approach to product creation.  The only overhead that is introduced is in the creation of the Ishikawa diagram.  If your thoughts aren’t already organized this way, you’re in trouble.  It takes almost no time to create the diagram that reflects what you should already be thinking.

And that qualifies as low-overhead.  And high value.  That should win over any resistance from the team.

Conclusion

As a product manager, you have to synthesize market problems, and develop a product vision to solve those problems that is aligned with your strategy.  As someone who is “part of” or “working with” (whichever matches your situation) an agile development team, you need a way to provide context to your implementation team.  That context can easily be conveyed with an Ishikawa diagram, with almost no incremental effort.  Your team can easily consume the diagram, and it gives them the freedom to explore different solution designs.  It also enables you to communicate your plans with the rest of the company (and externally, if desired).

Jun 14th was the first productcamp in Austin (and the second one anywhere).  It was a great event, and here’s the presentation that I did on how to define the strategic problems that drive our products.

Defining Problems

Here’s the presentation I gave at ProductCampAustin 1, posted on slideshare.  Just click on the forward/back arrows in the center of the bottom of the image, and it will cycle through the presentation – you don’t even have to leave Tyner Blain.

| View | Upload your own

Following Along

In keeping with the “don’t put a lot of words on your slides” tradition, the presentation is probably all but impossible to follow without some notes.  So here are some notes, organized by slide.  The presentation builds on concepts we’ve talked about here over the years, and in the notes below, I’ll link to some of those previous articles to provide more depth (instead of typing out everything I said).

• Slide 1.  There are two key elements to achieving software product success.  Build the right stuff, and build it right.  This presentation is about building the right stuff.
• Slide 2.  Specifically, when we start a project, it is to achieve some business objective.  The challenge is to find the right level of abstraction for that objective.  “Increase shareholder value” is too nebulous, and “Replace a legacy system” is too lacking in context.  Today we will apply a very powerful, but simple technique to help define the business objectives at the right level of detail to be actionable, while providing the right context to validate that we’re doing the right thing. My goal is for everyone to leave here with a new skill that they can immediately apply.
• Slide 3.  Everyone has a different perspective on a software project, and from those vantage points, perceives different elements of the project as the “why, what, and how” of the project.
• Slide 4.  Keeping in mind that people have different perspectives, let’s also look at a modified version of Karl Wiegers’ structured approach to requirements definition.
• Slide 5.  Introducing the Ishikawa diagram.
• Slide 6.  Set up the “deflated tires” problem from the article linked in (slide 5).
• Slide 7.  Showed the discovery process and representation of “real problems” closing out the theme from slides 5 and 6.
• Slide 8.  A real-world project I joined mid-stream for a client.  The project was a “collection of stuff” and no two people would give the same answer for “why are we doing it” and many people unashamedly admitted that they had no idea why.  No idea why!
• Slide 9.  Turns out, here’s what a view of the value for that program really looked like.  The team went from chaos to context.  Suddenly, scope discussions and prioritization decisions had a framework for validation.  There was also an organized way to begin completeness analysis.  Much easier to say “we’re getting this value” than to say “we’re doing these things – did we miss anything?”
• Slide 10.  I facilitated the folks in the room creating an Ishikawa from scratch.  We started with what we thought was the problem statement (provided by John Milburn from the back of the room) – “Traffic in Austin on Fridays is too bad.”  We ended with “John has work-life balance problems”, one of which comes from missing dinner with his wife, which can happen because of a combination of bad traffic and bad planning by John.  We also explored several solution-paths, including increasing the capacity of the roads and of the vehicles.  The ideas clicked for several people in the room.
• Slide 11.  Another real-world example, this one used in planning of a product roadmap for an 18-24 month view of the problems the team was signing up to solve.
• Slide 12.  Special thanks to the sponsors that made our inaugural ProductCampAustin a success!  My prediction for the next one (fall/winter 2008): 250 attendees.  So if you’re the sponsoring type, start planning on how you can help out.

Thanks again to everyone who attended, and special thanks to Tal Boyd of Seilevel, who video taped my whole presentation.  I just haven’t figured out how to split the 400+MB m4v file into something I can upload onto youtube.

The Cause and Effect diagram is also known as a fish bone diagram, because it resembles the skeleton of a fish. Using a cause and effect diagram can be the most effective way to define the problems that you intend to solve with your product. Get your stakeholders engaged in your program with this compelling visual!

Getting To The Root Of The Problem

In our recent article about writing good problem statements, we pointed out a common mistake people make with problem statements – they confuse the manifestation of a problem with a problem.

Problem manifestation [noun] – an example of a way in which a problem is exhibited, without appreciating the true nature of the problem. Ex: The problem manifestation is that the tires on my car are under-inflated. The problem is that my car is too expensive to maintain.

This distinction is relevant. The cost of operating the car is too high. That is the problem. It happens to be that one reason that the cost is too high is under-inflated tires. If you focus your energy on getting properly inflated tires, it will help (by improving fuel economy a little, and by reducing the frequency of tire replacement) with costs anecdotally. But you will not have solved the problem that costs are too high. Unless you get lucky. Costs can be high because the engine is inefficient or damaged, the aerodynamics of the car are bad, or any of a number of reasons. If you solve the problem by addressing a single manifestation of the problem, without understanding the whole problem, it is only because of luck.

Your Problem Statement is The Problem

In the comments on that article, The Demon points out that it is not always easy to identify the right level of abstraction for your problem. The cause and effect diagram makes it brilliantly simple not only to get to the root of the problem, but to communicate this cause-and-effect hierarchy of problem decomposition.

Cause And Effect Diagram Example

The cause and effect diagram is so visceral that the easiest way to communicate how it works is to show an example. Here’s what the cause and effect diagram would look like for the example problem above, where the cost of operating the car is too high.

[larger image]

The main problem is that the cost of operation is too high. This is the far-right, or fish-head part of the diagram (it is sometimes called a fish bone diagram).

The problem can be decomposed into three separate problems: spending too much on fuel, maintenance, and payments. Each of those problems can be further decomposed. Note that “under-inflated tires” appears twice – once as a cause of low miles per gallon (MPG) and once as an excess maintenance cost.

Alternately, you could recognize that spending too much on fuel could be due to lower fuel economy or excessively high prices. You could then choose to decompose the problem slightly differently:

[larger image]

Either approach results in crystal clarity that under-inflated tires is one root cause of low fuel economy, which is one cause of excessive spending on fuel, which is one cause of excessive operating costs. This visual approach helps significantly when trying to identify the right level of abstraction for expressing the problems in your problem statement.

Problem Abstraction Is A Side Benefit

The really cool part is that the help you get in finding the right level of abstraction for your problem is just icing on the cake. [Ed: No jokes about fish-bone cake. Ick!]

The real benefit comes in communicating and validating the problem decomposition with your stakeholders.

Someone questioned me once on the value of writing passionate requirements. Show one of these to your team, and you’ll get enthusiastic, passionate responses. You’ll get kudos from the business for demonstrating that you understand their needs. You’ll get praise from the implementation team for providing them with context.

Using Visio To Create A Cause And Effect Diagram

Creating a cause and effect diagram in Microsoft Visio is really easy, there’s a built in template, and it’s a good one. Create a new drawing and select the “Cause and Effect Diagram Shapes” template (under “Business Process”):

Visio will create a new drawing with a blank cause and effect diagram set up for you:

Fill in the boxes with the large problems. To get to the next level of detail (such as “Fuel Economy is Too low” in the last example), select the “Primary Cause” shape and drag it onto the diagram. Attach the arrowhead to one of the branches (the fish “ribs”) and start typing. For once, Visio’s default layout is where you want it.

To get a secondary cause shape (such as “Bad Aerodynamics” in the last example), select the “Secondary Cause” shape and drag it onto the diagram. Attach the arrowhead to the “primary cause” arrow you just created.

Conclusion

You already have a good justification for defining problems at the right level of abstraction. Now you know how to easily create a cause and effect diagram to find the right problem definition. As a bonus, communicating with stakeholders just got a lot easier – include this in your BRD.