People who already use Scrum will only find one new thing in this article – a way to communicate what happens inside a sprint that has proven effective for me.  People who are new to Scrum who wonder “how do things work inside a sprint?” will see how things work in a way that avoids hyperbole and is easy to map to what they already understand from traditional software development processes.

Two Teams Separated by a Common Process

George Bernard Shaw was an Irish author in London who memorably said “England and America are two countries separated by a common language.”  The same can be said about teams developing in agile and waterfall processes.

The story of agile adoption is one that has many colorful examples of adoption and of spreading the gospel.  Some practitioners traveled across the ocean from the agile continent to the lands of agile and opportunity to reap the rewards.  A few of those, after realizing the benefits of agile, invade our consciousness with a hellfire and brimstone tale of doom for anyone who doesn’t convert to the new religion.

Others seem hell-bent on kidnapping entire development teams, smuggling them across the ocean in the belly of steamer ships and unceremoniously dumping them in the land of agile, ready or not.  Scrum is but one branch of the agile movement.

Stalwarts of the old way, which has worked fine for me, thank you very much, refuse to leave their comfortable lives to step into the unknown wilds of agile.  Open-minded but responsible potential converts ask questions to gain an understanding of what life in the new world might be like.  What will life be like if I join this Scrum congregation?

How Does Scrum Really Work?

The best explanations of how scrum works that I’ve read come from Mike Cohn and Mountain Goat Software.  The training, videos, and explanations they share provide fantastic top-down introductions (as well as guidance after adopting Scrum).  His book, User Stories Applied, is the ultimate reference when gaining an understanding of the mechanics of using user stories as the central artifacts for developing software with scrum.

[larger image]

The process diagram above provides a good outside-in, top-down view of how “this newfangled process” results in shippable product, incrementally.  It is a great way to present the concept of Scrum, and incremental development in general.

When people I’ve worked with first gain this understanding, they often ask, what are the artifacts that live in these backlogs?  User stories live there.  Sometimes use cases make more sense than user stories for Scrum- it depends (read the article to see which works best in your circumstance).  The common element is an outside-in, user-centric approach to describing which problems the software is intended to solve.  This is also known as goal-driven development.

Once people understand the top-down view of the process, and have an idea of what user stories are, the next question they tend to ask is “how do stories move through the Scrum process?”

The first attempt at answering this question leans on the process diagram above:

1. The product owner takes the most important stories from the product backlog and puts them into the sprint backlog – collaborating with the team to agree on the scope of work for this sprint (based on the amount of work, capacity of the team, real-world constraints, etc).
2. The team “works the stories“, meeting every day to communicate effectively and provide progress updates – best visualized with a burndown chart.  This chart says “we started with X work to do, and every day, we track how much of X remains to be done.”
3. At the end of the sprint, the work is done, and the software (or an update to the software) is ready to deliver.
4. Sometimes, multiple sprints happen between releases – launches of the updated software, because customers (and the rest of your organization) incur a cost when changing to the latest version.  But the output of each sprint is deliverable – that’s a key concept – even if you choose not to deliver it just yet.

What Does it Mean to Work The Stories?

I’ve repeatedly had people ask, after absorbing the above explanation, “what does it mean to work the stories?”

I’ve had consistent success (in bridging the language divide) using the following diagrams (drawn on a whiteboard) to explain how stories flow through the sprint process in Scrum.

The first diagram shows that user stories have a structure – the story itself, and the acceptance criteria for the story. I also establish that the story is going to go through a life cycle within the sprint.

[larger image]

Next I remind folks that the user stories make it into the sprint backlog because they are the most important (highest priority) not-yet-implemented stories in the product backlog.

[larger image]

Developers on the team pull stories from the sprint backlog and begin working on them.  These stories are work in progress, and are owned by individuals.  Some teams maintain that priority order within the sprint backlog (e.g. implement the most important story in the sprint first, etc.), while other teams use a more coarse-grained approach, relying on the product backlog prioritization to assure that everything in this sprint is the next most important stuff, and the order that the team implements, within the sprint, is up to the team.  Some folks like to strenuously debate this decision (don’t prioritize within sprints vs. do prioritize within sprints).  Both approaches are valid, and you should choose the right one for your team.

[larger image]

As each user story owner believes that her work is complete (all unit tests pass, and the developer believes that the acceptance criteria have been met), the story is ready to be tested by QA.  This step, in particular, helps people who are new to scrum and who think of things as “code and test”, to make some sense of the inner workings of a sprint.  When working with a team that was under-staffed in the QA department, I discovered that calling out this queue of user stories waiting to be tested helped convince management to increase QA investment in the team.  When too many stories pile up here, you know you have a problem.

[larger image]

The scrum team member responsible for testing the user story has the responsibility of assuring that it meets all of the acceptance criteria.  He pulls the story from the queue of “done” stories, and tests.

[larger image]

If the story fails to meet any of the acceptance criteria, it moves back into either the sprint backlog or the being developed column, where a member of the team resumes ownership of the user story and works to resolve the issue.  Once the issue is resolved, the user story is re-queued for testing.  When the user story meets the acceptance criteria, it is moved into the done done column.

[larger image]

Conclusion

If you are new to Scrum, and wondered how the sausage is made inside a sprint, this gives you a framework for understanding what is going on, and how the team delivers.  If you already know or use Scrum, you may have learned a couple things.

• Using this framework to describe how a sprint works is effective when explaining to people who do not have any experience with agile.
• Laying things out this way visually, will give you an easy-to-see signal if your team is unbalanced between developers and testers.

I’ve been involved with agile teams and processes for just over 11 years now, and incremental development is burned into my brain.  If you are new to scrum, please let me know if this seemed like a good way for you to consume this material.  If you’re an old hat, and have other suggestions or presentations, please add them in the comments below, because future readers will benefit from the additional ideas.

“For what one idea do you want your product to stand in the mind of your customer?”  I heard Roger Cauvin ask that question at the most recent ProductCamp Austin [correction - he said it here - thanks Roger], and the quote has been jumping to the front of my mind almost daily ever since.  Maybe by writing about it I can exorcise the demon and get back to using the idea instead of being haunted by it.

The One Idea

You’ve just been given a new assignment – your company needs a new software thingamajig so that you can play in the whatchamacallit space, and you’re going to drive product management for it.  You’ve also been asked to deliver the first version in six weeks.  Of course you’ll be able to do follow-on releases, after all, you are agile.  That’s how “agile” works, isn’t it?

Cool.  Exciting.  Challenging.

You sit down with the folks who will be building the product, and find out that they have already spoken with several stakeholders.  Excellent. You circle back with the stakeholders, and start to gather data about your market and audience.

Your schedule is aggressive, so you think about what is realistic to get done in a very small timebox.  With the current schedule, you can’t afford to get caught in analysis paralysis, and you can’t realistically do in depth market analysis, persona development, hyper-accurate requirements prioritization, etc.  You do, however, have time to do the most important thing – define the one idea that will define your product.

If you cannot come up with this idea, you need to push back on your management team, and delay the launch of your product until you have that one idea.

Don’t Abuse Agile

Agile is designed to help you rapidly create iteratively better products, with iterative development and continuous infusion of market feedback and data.  Agile is not a process by which you start typing without any idea of what you intend, releasing it and then getting feedback in an iterative process.  If that’s how you’re approaching agile, your process is broken.

[image from Market Driven Competitive Advantage]

Unless you’re very lucky, your first iteration will be a waste of time and money.  Wouldn’t you rather be intentional than lucky? Make sure you have that one idea before you start developing.

Strategic Alignment

You need to come up with a one idea that is aligned with and supports your corporate strategy.  At a minimum, you have to make sure that you have an idea that is aligned with your stakeholder’s goals, based on the assumption that those goals are aligned with corporate strategy.

Wow

Your one idea really needs to make your users say “Wow!”

I was explaining the importance of this to a colleague (who is not a product manager), and his response was “Oh, you mean the hook.”

I was reminded that you not only have to provide a wow experience for your users, but you have to present a hook that will capture the imagination of your buyers.  Remember that buyer personas make decisions based on their perceptions of user problems [more on convincing buyers].

Satisfice

Remember that you don’t want to try and do everything perfectly in the first release – you want to satisfice.  You may only be able to target a single persona with a solution to a single problem.  You just need to make sure you are solving the right problem – which is where Kano analysis is very handy for identifying the Minimum Market Acceptance criteria.

[From Minimum Market Acceptance]

Conclusion: A Balancing Act

Defining the one idea for your product is a balancing act.

• You have to align the one idea with your corporate strategy and stakeholder goals
• You have to solve a valuable problem for your users, presented in a way that captivates your buyers
• You have to create a product that is compelling right away – good enough in execution to enter your market
• And you have to get the first release out in six weeks!

Good luck!  Or better yet – Good Intent!

I came across a really interesting article LukeW.com, showing how making changes to the way an input form on a website increased interaction by 25 to 40%. The changes reflect the value of thinking outside-in, investing in user experience, and performance measurement.

Bonus: the idea is cool.

Mad Libs Input Form

[full size image available in Luke's article]

The idea being presented is to replace the old boring web input form designed for a computer. The new, fun form is a fill-in-the-blank (aka Mad Libs) layout. The article is on Luke Wroblewski’s site.  The team at Vast.com created and tested a version of this form for the Kelly Blue Book site. [Luke cites Huffduffer.com as the first place where he saw this technique.] Thanks, Luke for sharing this with all of us!

Outside-In Development

Product managers are acutely aware of the need to solve market problems. We are regularly reminded that we’re creating solutions for people in our markets who use our products to solve their problems. When we write requirements, we avoid writing design specifications, and thereby lose the ability to enforce that the proposed solutions also take an outside-in approach. We maintain an outside-in perspective, but we lost the ability to influence an outside-in aesthetic.

Note: Outside-In Software Development is a great book, you should read it.

That’s one area where user experience works well in concert with product management – assuring that the same drivers of what to build are informing the design of how it is built.

The genius (or at least elegance) of this Mad Libs form from Vast / Kelly Blue Book is that it humanizes the experience of requesting that a nearby dealer contact you to discuss a possible purchase of a car from them. The forms we’ve all had to fill out on countless web sites are very inside-out. They expose the inner workings of the program – “gather this info, and that info, and this other info.” Those forms force users to do what the program wants. Blech. The form that Ron Kurti designed, however, forces the program to do what the users want. Huzzah!

Bad Requirements Prevent Elegance

I’ve written about the importance of writing design-free requirements several times.  Avoiding design constraints in your requirements is even one of the pillars of the big 10 rules of writing requirements.

If you had written the requirements for this contact the dealer page, would you have specified the design of the form?

Most of the requirements analysts I’ve worked with would have. I’ve even worked with companies that have developed templates / stencils defining how these interface specifications must be documented – requiring that all the fields be aligned, with specific pixel gaps, etc.

If you had specified the design (read: limited the choices of the designer), you would have prevented this solution.

Mr. Kurti’s elegant solution is clearly better.

One way to write the requirements that would not have prevented this solution would be with a user story and acceptance criteria:

As a car-shopper, I want to engage a car dealer, once every few years, so that I can purchase the car I selected.

• The car-shopper can provide his contact info (name, email, phone) for the car dealer.
• The system will display the information identifying the selected vehicle.
• The system will display the information identifying the dealer being contacted.
• The system will include an opt-in newsletter-subscription option.
• The system will notify the specified dealer when the car-shopper indicates.

The user story nudges the developers into an outside-in perspective by emphasizing the user’s goal. The acceptance criteria make it clear that any solution that meets those criteria is acceptable to the product manager. It allows the interaction designer to create a compelling solution, rather than forcing him to recreate a boring experience.

Measurement Rocks

The Vast team measured the impact of making this change to their form, and saw between a 25% and 40% increase in conversion (people contacting dealers versus people abandoning the process when they see the form). That’s a lot more leads for the dealers, and presumably a lot more money for Kelly Blue Book and Vast. This is a great example of how you can measure the impact of investing in user experience, because it should spark ideas for you.

What can you measure and test and improve?

Now that’s ROI for you.

Engagement – that’s what this whole product management blogging thing is about.  Check out what Tyner Blain readers found to be the most engaging articles in 2009.

Deep Dives

If you’re new to Tyner Blain, you may be surprised by the length of the articles here.  I joke that they are long because I don’t have time to edit.  Stewart Rogers jokes that they are long because I’m incapable of writing a short article.  If you’ve been here a while, you know what you’re in for.  If you’ve been here a long while, then you’re glad (like me) that I don’t write one per day any more.

Product Management is simultaneously a broad and deep discipline, requiring us to have a breadth of perspective combined with a depth of insight.  We then have to apply that in a market context, effectively navigating the political waters of our organizations.  Most of the articles here either try to skim the breadth of a range of related topics, or plumb the depths of a single topic.  Doing that in under a thousand words is pretty hard.  Most of the articles here also link to other articles, to try and provide even more depth and context, and encourage additional critical thinking.

One measure of the quality of the articles here is how often they stimulate readers to read further, or dive deeper into the topics.  While web analytics won’t allow us to measure how thought-provoking an article is, we can look to see how many people dive into the linked articles, versus how many people move on to something else.  We can measure the bounce rate of an article to see how often people leave a page without following any of the “tell me more” links.

Looking at ~170,000 page views at Tyner Blain in 2009, narrowed down to only those articles with at least 100 page views, here is the top-ten list of most engaging (or “least abandoned” if you’re a “cup is half empty” person) articles:

1. Use Case Series: Introduction: A collection of articles on the “traditional” forms of use cases – informal, formal, UML.
2. Agile Development of Use Cases: A dive into the dynamics and cadence of an agile process for developing use cases.
3. How Do You Manage Market Data?: A collection of articles exploring different market-immersion techniques in depth.
4. Scheduling Requirements Changes – Part 2: A focus on practical techniques for managing change with an agile development process.
5. Yesterday’s BPMN Post Was A Big Fat Lie: A mea culpa and clarification of interest to folks following the BPMN Series.
6. Everything I Needed To Know I Forgot in Kindergarten: Why asking why? is important.
7. Plan Your Next Sprint by ROI – Part 1: Prioritizing by Bang for the Buck, not just Bang.
8. Ten Common Mistakes of Going Agile: A collection of articles about common pitfalls encountered when adopting agile practices.
9. Perpetually Almost Finished Projects: Organizing a project with discrete deliverables and rolling-wave planning to avoid the “90% done, 90% remaining” problem.
10. How To Use Timeboxes for Scheduling Software Delivery: One of my personal favorites.  A rational approach to making trade-offs when your “perfect” plan has to change.

OK Stewart, that’s only 519 words.

Design-Free requirements are important for two reasons, and hard for two other reasons.

Design-free requirements are hard because you “know what you want” when you should be documenting “why you want it.”  Writing design-free requirements can be hard when you don’t trust your development team to “do the right thing” even though it is not your job to design the solution.

Design-Free Requirements – Revisiting

It has been three years since I wrote Writing Design-Free Requirements as part of The Big Ten Rules of Writing Requirements.  In that time, agile development practices have moved from being an esoteric development methodology to being the topic on the tips of everyone’s tongues as executives and organizations try to either (1) get the benefits of the state of the art in software-development process or (2) do something shiny and fashionable.

The previous article centered on elements of designs within MRD, PRD, and SRS artifacts.  A regular alphabet soup of artifacts.  Honoring the values behind the agile manifesto encourages us to emphasize working software over comprehensive documentation.  In that light, we’ll take a “what is needed” approach to talking about how requirements, design, and implementation are all needed – rather than issue an edict about where that information should be captured.

Agile Development Inputs

When creating software, someone needs to know:

• What problems are being solved, and how important (valuable) they are to be solved.
• Who has the problems and who is using the software to (help) solve those problems.
• What constraints limit the space that defines the universe of possible viable solutions.
• What acceptance criteria define if the delivered solution will be acceptable.

Subject to those inputs, someone needs to make design decisions:

• User experience design - what interactions will a user of our solution love?
• Program design – how (in a nuts and bolts way) will our solution function?

When talking about agile and talking about design, we should take a look at how Kent Beck and Alan Cooper, as respective though leaders in each space, view this.

Cooper doesn’t talk much about creating the requirements to support the daily use scenarios – he proposes moving directly into design of the solution. This differs from the more traditional technique of writing functional requirements to support use cases. Cooper also breaks down design into two components – program design and interaction design. Program design is everything you don’t see. Interaction design is everything you do see.

[...]

Cooper argues that designing the interaction should happen before any code is written. He uses a construction analogy to drive home his perspective – you can’t pour the concrete before you build the forms. Kent Beck, founder of the XP programming philosophy disagrees with the premise. Beck believes that the cost of changing software is low, and the imagery Cooper uses is hyperbole. We touch on, and link to that debate in this post.

Interaction Design Process Overview

I don’t believe that Beck is arguing for “don’t do design” – I believe he is arguing for “don’t do big upfront design that would delay implementation.”  He’s championing the agile values that emphasize creating working software and responding to change.  I can imagine him saying “no one will buy a design, they want a solution.”  Cooper’s point is that create a solution without first understanding how someone will use it, you can’t create a great product.

Program design has many hidden impacts – cost of maintenance, cost to change, and the performance of the solution.  You can have a design (or architecture) that makes everything easier to do in the future – at the cost of delaying the delivery of anything.  Or you can have a design that minimizes the time to deliver the first thing, while increasing the cost to deliver any particular next thing.  Or you can design a solution that falls somewhere between those extremes.

If you say “we don’t do design” you’re lying.  Every solution includes design.  Your team’s design process may be “big and upfront” or it could be a couple sketches on a whiteboard, or some email exchanges.  You may create storyboards and wireframes.  Or design may be the thing that happens right before the programmer’s fingers strike the keys.  Design may be explicit or it may be implicit – but it never “doesn’t happen.”

In the movie Amadeus, Salieri is astounded that there are no rough drafts of Mozart’s compositions.  That’s because Mozart did the design in his head, not because he didn’t do design.  I’ve worked with programmers who’s “implicit designs” were great, but that is the exception, not the rule.

Who Owns Design?

Since design happens sometime before fingers strike the keyboard, the real question is – “who owns the design?”

• You have a product manager developing an understanding of market problems, and prioritizing the problems that should be solved with your product.
• You may have a product owner managing the backlog and clarifying those requirements (problems) for the development team.
• You may have an interface (or interaction) designer or team of designers who are broadly responsible for “how users interact with our products.”
• You may have an architect or lead engineer who is responsible for the “big picture design” of how your product works.
• You have developers and testers who are responsible for delivering your product. [Note: coding without testing is like typing code without compiling - maybe it works, but probably not.]

You may not have distinct individuals with each of these titles – every small team I’ve worked with has people wearing multiple hats.  This is especially true when you have an agile team full of specializing generalists – any given story (or task) may have a different architect and different implementer.  I’ve only worked with one company where the architects are NOT part of the development team.  If your team is set up that way, please comment below – I had never seen that before, and I’m not convinced that it is the best way to organize – what have your experiences been?

Generally, “program design” is clearly owned by the development team – and product managers (and product owners) know better than to specify program design in their requirements.  Neither the lead engineer nor the product manager believes that the product manager is a better programmer – so the product manager better not be specifying program design and mislabeling it as requirements.  Coincidentally, Steve Johnson, at Pragmatic Marketing has a post running right now with a bit of a quiz – is this a req(uirment) or a spec (design)?

Where the line is more blurred is around interface and/or interaction design.  Some development teams have interface designers as part of the team.  Some companies organize with interface design as a “shared service” within the company.  Either approach can be the “right one” – it depends on too many details to make a sweeping generalization.  When the designers are members of the development team, the solution (from a product management perspective) is the same – the “team” is responsible for all design.  When the designers are not part of the development team, the developers have to consume two sets of guidance – “solve this problem” from product management, and “the solution needs to look like / act like this” from the designers.

Collaboration between product management and user experience people is the ideal solution.  The “requirements” and “design” inputs to the development team are comprehensive and consistent.

Design-Free Requirements

There are benefits – especially when being agile and minimizing documentation – to delivering requirements and design at the same time.  Just don’t do it by embedding design constraints within the requirements.

• When people on your team misinterpret design as requirements, they are unnecessarily constrained.
• As a product manager – are you the best designer on your team?  If you’re busy designing, who’s product managing?

This is trickiest when writing use cases – sequencing a set of interactions is interaction design.  That is one benefit of writing a user story instead of a use case.  An approach that has worked well for me with multiple teams is to deliver user stories (requirements) combined with storyboards (interaction design) and wireframes (interface design).  When details are needed (usually when “changing” versus “creating” an interface), screenshots can replace wireframes.  When business processes are complicated, process flows can replace storyboards.

Just don’t embed the designs within the requirements.

Your company is building out a toolkit to support third-party developers.  You’ll need a bunch of different types of widgets – combo-boxes, text entry fields, domain-specific controls, etc.  You’ve got a long list of desired controls from your customers.  You’re agile.  What do you build first?

Agile In A Soundbite

Being agile is about delivering incremental value, quickly, getting feedback, and then delivering more incremental value.  Repeat until “done.”  Good agile adds a qualifier – do the most valuable thing quickly, get feedback, do the most valuable thing (that has not already been done) quickly. Better agile optimizes the rate at which you deliver value by taking into account both benefit and cost.  Great agile overlays a focus on getting better at doing all of these things while you do them – becoming a learning organization.

Boiling the Ocean

A product manager I was coaching was faced with a challenge commonly referred to as boiling the ocean. His team was tasked with solving a market problem, and they were constrained to doing it with a service-oriented architecture that exposed a set of widgets (user interface controls) that customers could easily integrate into their products.  This approach was designed to provide competitive differentiation by reducing the time and cost to deploy solutions that allowed customers to integrate his product into their existing platforms.

In initial conversations with customers, technologists, and architects, this product manager quickly amassed a list of desired widgets (controls) and scenarios (stories) in which they could be used.  The product manager’s team had recently switched to an agile development methodology.  The internal stakeholders, not yet accustomed to agile development, wanted “all of the widgets,” now.

This product manager was able to convince them that the team would deliver the widgets incrementally, following agile principles.

His question was – How do I sequence the widgets in the backlog?

Defining Widget Priority

The product manager had a list of widgets, combo-boxes, data-grids, text fields, radio buttons, etc., and for each widget, he had a real-world scenario showing how the widget could be used.  Most of the scenarios involved customers using multiple widgets.  He wondered if he should do some sort of analysis that detailed the frequency of use of each widget.  ”This widget is used in 7 scenarios, so it should be done first; this widget is used in 5 scenarios…”

There was a strong temptation to add several “Create a widget” tasks to his backlog.  It would be easy for his development team to estimate the effort required to deliver each widget.  His team wanted to deliver incrementally, and “one widget at a time” felt like logical, discrete chunks of work to them.  They could easily sink their teeth into estimating, building, and testing each widget.

A quick reminder of a main tenet of agile, delivering incremental value, illuminated the flaw in this approach.  This approach would have been an inside-out prioritization, when value delivery requires an outside-in perspective. (See the ‘recommended reading’ suggestions on this page to check out Kessler and Sweitzer’s great book.)

If the team used this approach, after the first widget was complete, they would be able to deliver exactly 1 task, and 0 stories.  Development teams don’t deliver tasks, however.  The team’s customers would not be able to get incremental value from having a single widget.  The team would have delivered seven incomplete stories – so they would have delivered no stories at all!

We reworked his approach as follows:

1. Assess a value for each story, making sure that each story would enable his customers to accomplish something valuable.
2. Engage their user interface / user experience designer to design a solution for the most valuable story.  This design was constrained to use widgets in the user interface.  The suggested way to communicate this design was with a storyboard and low-fidelity wireframes.
3. Identify the widgets needed to be built to deliver that story, using the designer’s design.
4. Deliver the story to the development team, including the storyboard, wireframes, and list of widgets to be used as acceptance criteria.
5. Get the development team to size (estimate) the effort to complete each story.
6. Where stories were too big (epics), collaborate to identify good ways to break up the stories into manageable chunks.
7. Repeat steps 2 through 6 until the amount of identified effort was likely to fill the release (keep the dev team busy delivering value).
8. As the team delivers each story, get feedback from the market, revisit the prioritization, and revise the “next” story.

What we discovered was that a scenario like the following (modified for this article) played out:

• The first story required two widgets.  As no widgets existed, both had to be built.
• The second story required three widgets, but two of them had been built to support the first story – only one incremental widget had to be created.
• The third story used one of the widgets from the first story, but required additional behaviors.  The original widget was modified to provide incremental capabilities (and value).

A Note On Team Structure

Astute readers will notice that there was a design step “inside the requirements process” – before the stories were delivered to the development team.  Technically, that is true, for the way this particular company was organized.  The user experience designer was not a member of the scrum team, but rather, an external consultant who supported multiple teams.  The product manager engaged the designer prior to engaging the development teams.

This just as easily could have happened as follows:

1. Product manager identifies, values, and prioritizes stories to be added to the backlog.
2. Development team, as part of sizing the stories, engages their user experience designer to select the appropriate widgets, and those design decisions inform the estimation process.
3. Everything else is the same.

This procedural variation does not have design “inside the requirements process.”  The same people do the same things, in the same sequence in this scenario.  The only difference is that the interface / interaction designer is a member of the development team.  That would be ideal, but that was not how the company was organized.

In this particular example, having the product manager collaborate with the UX designer made the most sense.  It introduced less complexity for the product manager to accept responsibility for this activity than it would have for the development team (who was still new to using an agile delivery cadence) to do it.

Conclusion

The key ideas at play here:

• Focus on realizable value to the customer (outside-in development), not tractable tasks (inside-out development).
• Collaboration with a UX professional is key to driving “interface requirements.”
• Deliver frequently and valuably, get feedback (learn), and incorporate that knowledge into whatever is next.

Perpetually intermediate (competent) users.  Users who briefly exist as novice users and never become experts. Most of your users are competent, and you should design for them.  Competent users have different needs and different expectations than novice or expert users.  How do you know your user’s competency levels, so you can design for them?

User Competency

User competency is a concept I first read about in Alan Cooper’s The Inmates Are Running The Asylum.  Cooper’s contention is that the level of expertise of your users follows a bell curve, or normal distribution.

When we’re designing software we need to keep in mind that most of our users will be competent – neither experts nor beginners. Alan Cooper’s studies tell us that user skill levels follow a bell curve. He talks about competent users as perpetual intermediates. Some users drop out of the bell curve when they stop using our software. The rare user becomes an expert. Most users only learn enough to get their real job done.
Competent Users and Software Requirements

Cooper contends that the combination of learning curves and natural user tendencies to stop learning or abandon a software application are the sources of this distribution of user competency.

Experience Curves

Experience curves represent the diminishing costs over time of manufacturing something repeatedly.  The process of manufacturing something gets more efficient as you get smarter about the process.  The process of using software is at least analogous to, if not a specific example of a manufacturing process.  If you’re using an email application, you are manufacturing email messages.  In a CRM system, you are manufacturing contacts, or contact reports, etc.

By treating any “using the software to do something” interactions as a process, you can measure the cost (how long it takes) of the user’s interactions.  Applying the math behind experience curves, you can predict the reduction in cost (to your users) over time, for any set of interactions.  Experience curves take into account that some processes are inherently more learnable than others.  This property of learnability is reflected as an efficiency coefficient – how efficiently someone can learn ways to reduce the cost (time) needed to perform the interactions.

This gives us an approach to quantitatively model user competency.  Having a definition allows us to model competence.  And measuring competence allows us to manage product design in the context of user competency - designing for competent users.

Defining Competence

The first step to measuring competency is to define the model.  I am proposing a definition in this article that I suspect will yield insights (to help us manage our products). I was unable to find any quantified definitions of competence, when researching it as part of a client engagement.  If you have, or know of a model, please share it in the discussion below this article.

• A competent user is someone who learns to perform a task in half the time it initially took them.
• An expert user is someone who can complete a task in 10% of the initial time.

This definition is guided by an expectation that Alan Cooper’s premise about perpetually intermediate users is true.  Being a novice user is a very transient state, and becoming an expert is very infrequent.  The goal of the definition is to be able to segment your users and make well-informed design decisions.

A Proposal, Not a Doctrine

This is a proposal that doubling performance reflects competence, and achieving a ten-times improvement represents expertise.  It may be that some different measure of performance improvement more accurately reflects competence and expertise.  We have to test it to know.

The experience curve is defined mathematically by Henderson’s Law.  It states that the time to complete a task is a function of the number of times you have previously done that task, adjusted by the “elasticity” of the cost of that task.  In other words, some tasks are easier to improve than others.  If you populate a table with the results of applying Henderson’s Law, you get the following:

[larger image]

• Each row in the table represents the Nth repetition of a task.
• Each column represents how easy that task is to learn – progressing from “hard to improve” on the left, to “easy to improve” on the right.
• Each cell in the table represents how long it would take to perform the Nth repetition of the task, as a function of how easy it is to improve your performance at the task.

The above definitions represent what experience curves predict mathematically, when the task initially takes 60 seconds.  The following definitions reflect the proposed user competency model:

• A user is a novice user until she has learned enough to cut the time-on-task in half.  These cells have a white background (and are in the upper left area of the table).
• A user is a competent user when the time needed to complete the task is between one half and one tenth of the initial time.  These cells are shown with a yellow background (and are in the central area of the table).
• A user is an expert user when she can complete the task in less than one tenth of the time required to initially complete the task.  Theses cells are shown with a red background (and are in the lower right area of the table).

In the absence of empirical data, I used my intuition to suggest that a representative experience curve for a typical task performed in software would be one with an “elasticity” of 50%.  For a task with those learning characteristics:

• A novice user would cut the needed time in half on the 4th repetition of the task, and would be considered to be a competent user.
• A competent user would further reduce the time needed to one tenth of the initial time on the 100th repetition of the task, and would be considered an expert user.
• The 50% elasticity column is surrounded by a black border, and the number of repetitions required to advance to competent or expert status is also highlighted with a black border.

Inferring Competence

Since different processes have different learning characteristics, you have to figure out how easy it is for your users to improve at your processes.  To do that, you have to study (or at least measure) your users’ interactions with your software.  In the 50% curve highlighted above, a user is capable of cutting their time-on-task in half by the fourth time they perform the task.

If data from your initial testing (or measurement) reveals this to be true, then you have selected the correct curve (the correct column in the table).  If it takes more or less time to reach this level of improvement, shift to the left or right to find the appropriate curve.  If software-interactions are reasonable analogs to manufacturing processes, then the experience curve projects an expected rate of improvement on task.

The following graph isolates the time-on-task data for a user who is learning to improve when repeating a task (process) that matches the 50% elasticity curve.

[larger image]

Note that the number of repetitions of the task (the X axis) is represented as a logarithmic scale.  The data points along the curve correspond to the cell values in the table above (for the 50% column).

Shades of Gray

One nice thing about this quantitative approach to inferring competency by measuring usage is that your measurements are per-process.  Users are not “purely novice” or “purely expert” – they can be experts at some processes, while remaining neophytes at other.  There is also awareness, for any particular process, of “how much competency” a user has.  This allows you to refine your assumptions of the steepness of the learning curve, and of the thresholds (doubled performance and ten-times performance improvements).

Improvement Over Time

Any particular learning curve can be considered relative to calendar time, to see how quickly a user will progress along the curve (as a function of frequency of use).  This can be useful for determining the ROI of improvement in a particular process.

The following graph shows how an 80% learning curve overlays a calendar for tasks that happen daily, weekly, and hourly.

[larger image]

The graph shows that for a weekly frequency, after 16 weeks, the task time has reduced from 300 seconds to 100 seconds. With a daily frequency, the task time is even lower – about 60 seconds. This graph shows nothing other than converting the academic learning curve graph into one that incorporates calendar time and frequency of occurrence.

Software Usability and Learning Curves

One approach to inferring user competency would be to measure how long a user has been using your software.  The variation in how frequently different users perform the same task will introduce an error into that inference.  You can avoid introducing that error into your modeling by counting the number of times a user has performed a task.

Applying the User Competency Model

The advice in previous articles, and from Cooper’s book, and from this great article on the  Coding Horror site, encourages us to focus on the competent users.

I’m working with a client who needs to prioritize a set of capabilities and establish design principles for a product.  We will incorporate this user competency model as part of our analysis.

Hopefully we’ll have an opportunity to collect data to validate and / or refine the model.  I’m proposing that we first gain some insight into the which users (novice, competent, expert) drive the most revenue and profit from use of the product – to establish the importance of each category of user.

For this product, I suspect that we will find many more novice users than a normal distribution would predict.  If that is true, the next question will be to understand if we are dealing with a normal behavioral dynamic, or if characteristics of the current product “force” novice users to abandon it before they achieve competence.

Either way, we will have a framework for prioritizing the goals of the novice, competent, and expert users.

How would you apply a model like this to improving your product?

Concise requirements give your team a useful, easy to read and easy to change understanding of what must be done.  Great requirements exist to do three things:

1. Identify the problems that need to be solved.
2. Explain why those problems are worth solving.
3. Define when those problems are solved.

Concise Requirements – Revisiting

In the three years since we last looked at Writing Concise Requirements from the Big Ten Rules of Writing Requirements, the iPod evolved to give us a better experience.  Let’s see if we can do the same with the topic of brevity in requirements.  The size of our community here has grown ten-fold, and the people who were here back then have grown just as much.  It makes sense to look at this again.

Writing concise requirements is not just minimizing the number of words you use.  Writing concise requirements is presenting the most important information in the easiest format for your audience to consume.

Concise Requirements Identify the Problems That Need to be Solved

Ultimately, requirements are the problems that we choose to solve.  A concise requirements artifact (formal document, index card, photo of a whiteboard, whatever) is one that has the highest signal-to-noise ratio possible.  You’re maximizing the amount of communication, and minimizing the cost of communicating.

You want your requirements document to read like the lines, not the points.  If the line (the signal) is what you really want, and you communicate a big pile of points (the signal, hidden in the noise), you run the very real risk that your audience will misinterpret the signal.

User stories provide the best example of clarity that comes from conciseness.  The format you should use, based on Mike Cohn’s great book, User Stories Applied, is

As a [role], I want to [do something] [with some frequency] so that I can/will [achieve some goal]

The user story is not always the right communication format – it depends on what problems you’re solving, and who is on your team.  Sometimes, use cases work better than user stories.  Conciseness is important for use cases too.  Start with a good use case name.

Concise Requirements Explain Why Those Problems Are Worth Solving

Last week’s article on Valuable Requirements focused on why particular problems should be solved.  Your focus should be on value, and that article discussed five ways to assure that your requirements are valuable.  One of the techniques, the use of an Ishikawa diagram, provides a method for identifying the root causes of problems.

[larger image]

Imagine that you have a collection of user stories, representing problems to be solved for your users.  You need a way to demonstrate why this user story should be implemented and why that one shouldn’t.  You can often use the Ishikawa diagram in the same way.  A particular goal is achieved when a user is able to do something.  Perhaps several somethings are required.  The point is that you can use the Ishikawa to drive home the point – if this set of user stories are all implemented, then this goal will be achieved.

There are two reasons this is important – first, you’re providing context for your team.  They understand why they are doing something.  Second, you can make changes easily, because you can see the impact of those changes.  By understanding the cause-and-effect relationships between problems and their values (user stories and their goals), you can see the impact of changing one or the other.

Concise Requirements Define When Those Problems Are Solved

Clarity is the goal of conciseness.  It isn’t enough to say “work on this.”  It’s important to know why you’re working on it, but that still isn’t enough.  You have to know when you’re done.  When you’re defining problems to be solved (and therefore solutions), you must also define the measures by which the solution will be judged.

A measurement of success for “Cost of Operation is Too High” might be “reduce costs of operation by 10%.”  This gives you a testable criteria for knowing when that problem has been sufficiently solved.  Sticking with the Ishikawa, you can also map out the strategy for achieving that lofty goal.  You can say that the goal is to reduce fuel expenses by 20%, reduce cost of maintenance by 5%, and reduce payments by 15%.  This process continues – a 20% reduction in fuel spend requires that you operate with your tires within 5% of nominal pressure, and that you reduce the aerodynamic drag coefficient by 7% (or whatever).

This gives you clarity in your objectives.

User stories, when combined with user acceptance criteria, provide that last connection of testability that lets your team know when they are done.

There aren’t many things more frustrating to a development team than having them solve the wrong problems.  One of those things might be having their solution be rejected because it isn’t enough.  Writing acceptance criteria clearly and concisely lets the team know exactly when they can move on to the next problem.

Providing a crisp understanding of acceptance criteria also facilitates iterative development.  One challenge teams always face is making improvements that fit within the timebox of a given iteration.  Imagine a user story with 4 acceptance criteria, where the story is too big to complete in one sprint.  When talking with your development team, you may find that the story is too big because satisfying all of the acceptance criteria is too big.  This is where many teams miss an opportunity – by defining all of the acceptance criteria that are believed to be needed eventually and requiring that they all be implemented now.  One of those criteria is going to be more important than the others.  Implement the story such that is satisfies the most important criteria (timeboxing) now, and rewrite or enhance it to meet the additional criteria later.

Conclusion

Software Product Success depends not only on identifying the “right stuff” to build, but on making sure the team builds it and builds it right.

Concise requirements improve your ability to communicate with your team, thereby improving their ability to build the right stuff right.

The maturity model approach to describing organizations and processes comes and goes out of fashion.  It is a repeating framework de jour.  In the game of agile jargon whack-a-mole, the agile maturity model is poking its head up again.

Agile Maturity Model

Ellen Gottesdeiner, author of Requirements by Collaboration, tweeted (@ellengott) a few hours ago with a link to an article proposing a set of maturity models.  She graciously passed on the opportunity to comment about the “agile maturity model” when pointing out her like of the collaboration maturity model.  The article proposed the following as an agile maturity model:

Agile Maturity Model (AMM)

0 – Dormant

1 – Speed: Focusing on being expeditious.

2 – Reactive: Focusing on acting relative to change from the perspective of the moment rather than a longer timeframe.

3 – Responsive: Focusing on acting relative to change from the perspective of the moment balanced with a longer timeframe.

I didn’t find it to be a particularly useful model.  Although descriptive, it won’t help your organization improve.

What Does Maturity Mean?

I wrote a series of articles a couple years ago that explored the CMMI and RMM – the capability maturity model and the requirements maturity model.  These are two different models that use maturity as a concept for articulating different levels, or grades (or enlightenment, or rigor), with respect to organizational behaviors.  At the time, there was both momentum and confusion around the notion of a requirements maturity model.  CMMI is a description of an organization’s rigor in “saying what it does, and doing what it says.”  RMM is an assessment of the level of critical thinking incorporated into the ways an organization is using requirements to develop products.

The problem is that people were incorrectly assuming that an organization “at level 4 CMMI” would approach requirements management at a comparably enlightened level.  The problem is that they are putting entirely unrelated concepts into similarly named classifications.  An organization could be at CMMI 1 and RMM 4 or RMM 2 and CMMI 3.  That series of articles explored what it would mean to be in any of the combinations of “maturity” for those two models.  Check out all the articles in the series if you want to put it all in perspective:

• Foundation Series: CMMI Levels Explained
• What CMMI Level Should We Use?
• CMMI Levels and RMM Introduction
• CMMI Levels and RMM Level 1 – Written Requirements
• CMMI Levels and RMM Level 2 – Organized Requirements
• CMMI Levels and RMM Level 3 – Structured Requirements
• CMMI Levels and RMM Level 4 – Traced Requirements
• CMMI Levels and RMM Level 5 – Integrated Requirements
• Don’t forget to take our One Minute Survey on CMMI and RMM.

Making a Maturity Model Useful

Why bother with a maturity model?  Not so you can “keep score” relative to others – rather, so that you can improve your own organization.  For a maturity model to be useful, you have to be able to do two things.

1. You must be able to determine where your organization is in the model.
2. You must be able to identify actions you can take to “improve” your organization relative to the model.

If you can’t measure maturity, and the model does not provide guidance about how to improve, it’s useless.  One challenge with maturity models is that they risk becoming contextually narrow in their application.  The more concrete a measurement or suggestion becomes, the less extensible it is likely to be.  Ideally, your model would be broadly applicable to many organizations.

With an agile manifesto that emphasizes people over process, it is ironic to consider applying a metric that measures your agile process.  So – goal #1 is immediately undermined.  Goal #2 – improving your organization, is, however, very valuable.

Improving An Agile Process

I’ve worked in several agile environments, both as a practicioner and as someone helping to transform organizations to become agile (or better at agile).  I’ve worked in enterprise software, helping large teams adopt agile processes; with a SaaS consumer product team, and in large IT departments with collections of teams adopting agile processes at varying paces.  I’ve played on both sides of the fence (delivery/QA & product management/ownership).

Going from zero to sixty on agile is a big task.  You can’t solve all of the organizational, practical, and interpersonal challenges at the same time.  Or at least you would be more effective tackling and resolving each issue in sequence.  In fact, you should solve the most important / largest problem first – then move on to the next, and so on.  You can call it agile agile adoption, or you can call it eating the elephant.

One powerful use of a maturity model is highlighting the next-biggest hurdle you have to overcome.  Unfortunately, most communication about maturity models is “look which hurdle we just overcame” – but the focus should be on “what’s next?”

When helping organizations be effective with agile processes, there are some clear “you have to overcome this first” challenges, that if unresolved, make other challenges irrelevant.

If you’re a software developer, I’m sure you’ve experienced the following scenario:

• A bug is reported.
• You fix the bug, and write some tests to prevent it from reoccuring.
• While testing, you discover another bug that was hidden by the first bug.

Fixing that second bug doesn’t do any good until you’ve fixed the first one.

If you’re a product manager or business analyst, think about it in terms of capability enhancements.  Is a faster search important if the current search algorithm is not returning the right results?  Get good results first, then make it faster.

You have to solve the biggest/closest/roadblockiest issue first.  Then move on to the next issue.  That’s how you should use a maturity model – as guidance about “what’s next?”

When assessing / improving your organization’s ability to be agile, you need to be addressing whatever is next.

What’s Next?

Let’s discard the “maturity model” notion and focus on “what’s next?” instead.  And that of course starts with “what’s first?”  Here’s how I’ve presented an agile hierarchy of needs to in the past:

1. Staffing the engineering team correctly.  People over process is the right emphasis.  If you can’t find people that are “good enough” you might as well go home.  Doesn’t matter how agile you are if you don’t have the horsepower.  You also need people who are excited to “do agile” – they like to communicate, they enjoy the project and team dynamics of an agile process.  You’re also better off with specializing generalists – ideally, every member of the team can do any work that is needed.  This is an efficiency play – you risk introducing bottlenecks when you have a specialist who is the “only one” who can do particular types of work – because you will not have a consistent mix of types of work from release to release.
2. Assuring Quality is in your team’s DNA.  Arguably, this is part of what’s first, but there are a lot of teams that get cood at cranking out code, before they get good at cranking out good code.  Continuous integration is the approach you must have.  Test-driven development, spec-driven development, and other testing-integration approaches are extremely important, but are really reflective of different flavors of agile and quality, not different degrees.
3. Reducing overhead in the release process.  There’s a cost associated with releasing a product.  Once you get good at releasing, and are releasing good product, your next focus is on finding the right release cadence.  There are three factors that essentially dictate your release frequency.  The size of atomic deliverables (user stories and use cases, non-functional requirements, etc) your team is creating, the overhead (time and cost) of releasing, and your customer’s capacity to consume the releases.  My anecdotal experience is that teams are usually constrained initially by the cost of releasing – dedicated hours to creating a build, code freezes, etc.  Automating the build process (to reduce both costs and errors introduced while building) is first.  Integrating automated build and automated test processes together is next.
4. Feeding the beast.  Once you have an engineering / delivery team that is operating efficiently, you run into the problem of making sure they have enough important work to do.  There’s pressure on product owners and  product managers to schedule something because it is easy to define, not because it is important.  It is also important that the team understand the context and importance of what they are doing.  So you have to focus on making sure your product management team has enough capacity to keep the engineering team busy on delivering really valuable stuff.  The “right” solution to this depends on how the problem manifests in your organization. Is your product manager spread too thin across multiple products?  Too junior?  Too bogged down in sales support or trade show attendance or or or?
5. Managing stakeholder expectations.  A big challenge in changing an organization to become agile is in resetting and managing the expectations of stakeholders.  Execs are used to having an annual budget exercise where they dedicate X dollars in the pursuit of Y objectives.  We’ll set aside the reality that they’ll only achieve 1/3 of the objectives, at 2X the cost, much later than originally forecasted.  We aren’t looking at the failings of waterfall, we’re looking at the risks of agile.  There’s a ton of stuff here, from doing damage control to reverse perceptions that “people who want to do agile just don’t want to be accountable” or addressing the “we can’t tell you what X dollars gets you” message.  Again – the particular solution is a function of how the problem manifests in your organization.  Once your team is delivering valuable stuff efficiently, you have to make sure your management team is happy and engaged and knows what to expect.
6. Continuously learning from your markets.  This is really what differentiates an agile organization from an organization with an agile development team.  Agile processes do enable you to develop code more efficiently.  If you aren’t taking advantage of the faster development, feedback loops, and ability to change that agile enables, you’re leaving money on the table.  And one of your competitors will pick it up.  There’s a whole community of MBAs that talk about business agility without once thinking about software development.  They’re talking about an organization’s capacity for rapid response to changing market conditions.  It’s what separates the winners from everybody else.  And agile development makes business agility possible.  As long as your team is able to identify and value industry trends, competitive threats, and market opportunities.  When you’re good at this, you have an agile organization.  And you win.

Conclusion

Any agile “maturity model”, irony aside, needs to be structured to help organizations progress through the stages of enlightened operations – continuously providing insights into “what’s next?” for those teams to grow.

I know there are many readers here with years of agile experience – how would you improve the list above, to better provide a framework for “what’s next?”  When that framework is solid, we can do some wordsmithing and repackage it as a maturity model.  Until then, just focus on “what’s next” – that’s all a maturity model should be used for anyway.

When defining requirements, you always start in the context of a goal – either a user goal or a corporate goal.  You need to be aware of both.  Having a positive user experience is important, and requires a user-centered understanding.  Achieving your corporate goals might be in conflict with some user goals.

User Goals

A user centered, or user-centric approach to developing software is one where you start by identifying the key persona in your target market.  For each of those personas, you identify their most important goals.  You then identify the user stories or use cases that represent the things these people do in order to achieve their goals.  These “things” are manifested as practical goals.  For these activities, you design user interfaces (process, layout, architecture, look and feel, etc) that create customer delight for those users, doing those things.  This is how your product exceeds the suck-threshold (good enough that it doesn’t suck to use your product).

Corporate Goals

There’s a reason your company is creating or improving a product.  It may be to gain market share, or improve profitability, or increase sales.  Whatever it is, there’s a corporate goal that your decisions should be supporting.  The (subset of) corporate goals that are relevant to your product could be communicated in a vision and scope document, that constrains the domain of problems to be solved, and provides nuanced insights into viable approaches to solving them (example vision and scope).

Corporate goals are achieved when customers do things with the products.  These “things” are manifested as practical goals.  For those activities, you document what should be accomplishable and why.

Alignment and Conflict of Goals

Notice that both the user goals and corporate goals manifest in terms of people doing things.  When the person (user) wants to do the same things that your company wants them to do, you’re in alignment.  When the user goals and corporate goals suggest different activities, you’re in conflict.

You can see visually that these worlds collide at the “practical goals” stage  in the following diagram, from an article on combining interaction design and structured requirements principles.

It may be great – when the user and corporate goals are in alignment, the practical goal and associated scenario (activity) are easily defined.  What about when the goals are in conflict?

Vending Machine Example

Last Thursday morning, as I interacted with a vending machine, the idea for this article was dispensed along with my Diet Mountain Dew.  Yes, I know that’s odd.  Move on.

It occured to me that the process of buying a cold, caffeinated beverage can be both aligned and unaligned from the perspective of user and corporate goals.  If you imagine writing a use case for purchasing a beverage from the vending machine, your most important scenario is the one where a person wants to purchase a beverage that is available in the machine.

• User Goal: Get refreshed and vitalized.
• Corporate Goal: Sell a beverage.

I want to buy a Diet Mountain Dew, as a means to realize my personal goal.  The owner of the vending machine wants to realize her corporate goal of selling me the beverage I want to buy.  We’re in alignment.  Our requirements definitions and design decisions will be pretty easy – everything that increases the likelihood of and improves the experience of purchasing that beverage is good.  I put my money in the machine and push the button.

Then it occurs to me – there’s a situation where my personal goals are clearly not aligned with the corporate goals of the vending machine owner.  When there is no Diet Mountain Dew available to be purchased.

Consider the following procedure I’m forced to endure:

1. I view the available beverages that this machine dispenses – Diet Mountain Dew is one of them.
2. I put my money in the machine.
3. I push the button for Diet Mountain Dew.
4. The tiny display on the machine presents a message in taunting, scrolling red LED lights: SOLD OUT.

At this point, I’m faced with a choice – select a less desireable beverage, or request my money back and try to satisfy my user goal in some other way.

Consider an alternate procedure:

1. I view the available beverages that this machine dispenses – Diet Mountain Dew is one of them.
2. I view the ‘current availability’ of Diet Mountain Dew and see that this machine is currently sold out of Diet Mountain Dew.

At this point, I’m faced with a choice – select a less desireable beverage, or request my money back and try to satisfy my user goal in some other way.

The differences in these two possible procedures indicate that there is a conflict between the corporate goal and the user goal.  With the first procedure, the “Sell a beverage” goal is given more importance, because it makes me more likely to purchase a beverage that I don’t particularly want.  My utility is potentially sacrificed, while the owner of the vending machine is just as satisfied.

By failing to tell me that I can’t get what I want, until I’m further invested in the process, I am more likely to purchase something else.  That purchase creates just as much value for the owner of the vending machine, but less value for me.

If the vending machine owner had allowed me to use the second procedure, it would demonstrate that the  owner believed the improved customer experience, while risking the loss of this sale, would encourage me to purchase more over time.  The owner would have been prioritizing the requirements and design that supported the user goal ahead of those supporting the corporate goal – in hopes that my user loyalty would result in better word of mouth (more business from other people) and more business from me over time.

Generalizing User and Corporate Goal Conflicts

In this vending machine example, the annoying procedure is probably the right one – I’m going to treat the “point of use vending of cold, caffeinated beverages” as a commodity product.  This market has a very low barrier to entry, and relatively small problem being solved.  I’m not going to tell my friends about how much I love the vending machines from Joey’s Vending and encourage them to go out of their way to find and use those machines.  Nor will I purchase more Diet Mountain Dew because the emotional cost of the occasional “sold out” scenario is trivially smaller with Joey’s machines.  I would, however, be more likely to buy a Diet Pepsi when I’ve already invested time in the process and put my money in the machine – especially when the alternative is to find a water fountain.  I’m emotionally invested at this point.

However, this experience did jump out at me as one that is worth explicit consideration when defining requirements and designing solutions.  It does come up regularly.

The new iPhone 3GS hardware allows tethering (using the phone as an internet-connection device for your laptop).  ATT (the exclusive carrier for iPhones in the USA) is indicating that they will charge customers an extra monthly fee for this capability, when they eventually allow it on their network.  The user already has a data plan, and the traffic that ATT would have to carry is just “data.”  The user just wants it to work.  ATT, however, may want more money, or may want to limit data traffic on its network – perhaps to improve the quality of service to all customers.  This could be ATT’s corporate goal of “provide better service quality” conflicting with the user’s “increase the flexibilty of how I use my data plan.”  The former would likely influence customer satisfaction (and abandonment) over time, while the latter would influence customer acquisition rates.  Another conflict in goals.

Think about how different goals can lead to conflicting solutions, requirements, and designs.  And tell me when you’re out of Diet Mountain Dew before I’m emotionally invested in the purchase.

Creating a PERT estimate for a single task is both easy and straightforward. Creating an estimate for a set of tasks is still easy, but requires a little bit of math. Combining PERT estimates for tasks is easy, but not as obvious. Roll up your sleeves and dive in.

PERT Estimate Refresher

When estimating how long it will take you to complete a task, you shouldn’t estimate with a single value like “four hours.” “Four hours” does not provide enough information. Estimates reflect that there is uncertainty, and that single value does not give you any insights into how uncertain your estimate is. Your estimate could be “four hours plus or minus one hour” or it could be “at least four hours, maybe as much as sixteen hours.”

A PERT estimate, as we showed in our earlier PERT estimation article represents a distribution of likely effort for a particular task. To create a PERT estimate, you create three values -

1. The “best case” (shortest) amount of time it will take to complete the task.
2. The most likely amount of time it will take to complete the task.
3. The “worst case” (longest) amount of time it will take to complete the task.

A PERT estimate is presented in the form “Best Case / Most Likely Case / Worst Case.” If your estimate is “four plus or minus two hours” you would write 2/4/6 as a PERT estimate. Sharing “2/4/6″ as an estimate still doesn’t tell anyone else what you mean. A PERT estimate embodies some probability around completion time. You are precisely saying (for a 2/4/6 PERT estimate):

1. There is a less than 1% chance that this task will take less than 2 hours.
2. There is a 50% chance that this task will take less than 4 hours.
3. There is a greater than 99% chance that this task will take less than 6 hours.

This explicit statement of probabilities represents a distribution of likely outcomes. Statisticians would phrase it in a confusing (but mathematically important) way. They would say “if we sampled a large population of people (like you) doing this task, no more than 1% of them would complete the task in under 2 hours, no more than 1% of them would spend more than 6 hours on this task, and half of the people would complete the task in under 4 hours.” For the purpose of estimation, you can ignore the statistic-speak and just look at the “odds” of how long it will take you to complete the task once.

[larger image]

If you were to create a histogram of thousands of executions of the task, it would look like the diagram above.  This is the traditional bell curve shape we are used to associating with a normal distribution, but it actually represents a PERT estimate.  A PERT estimate is a distribution of possible outcomes, based on a beta distribution.  Another way to look at a PERT estimate is in terms of cumulative probability of a task being completed.  The same data in the graph above, when presented as a cumulative distribution function (CDF) looks like the following:

[larger image]

Here’s the rub – you don’t really know if the beta distribution is the right one.  Primarily because you can’t precisely know the actual probabilities.  So, you have to pick a distribution function to represent those probabilities.  There is debate about the right distribution function to use for estimation – check out this great article for hard core stats guys.  Here’s a cautionary warning from their conclusions:

Although bias stemming from misspecified activity time probability models is rarely mentioned in introductory discussions, we have seen several instances of this bias in simple examples. First, and perhaps most important is the uncertainty as regards the underlying activity time probability models. The literature offers no less than five procedures for translating the subjective estimates (a,m,b) into specific β-distributions. As shown, the methods lead to distinct β-distributions, and the PERT approximation need not satisfactorily estimate any of them.

Essentially, they are saying “you can’t really know the right shape for a distribution curve of possible outcomes – so don’t get carried away.”  And they then go on to suggest using a simpler model than the beta distribution for estimation.  Their suggestion is to use a triangular distribution (looks like a triangle, instead of a bell curve), because the math is easy.  With spreadsheets, we can still do “easy” math with a better distribution

The beta distribution for a symmetrical PERT estimate looks very much like a normal distribution.  You can see this by comparing the cumulative distribution function of the PERT and normal models for this 2/4/6 example.

[larger image]

Based on this similarity in distribution, and the inherent uncertainty in what the shape of the distribution really looks like, there are some benefits to treating the PERT estimate (2/4/6) as a normal distribution where the high and low values represent +/- 3 sigma bounding.  This approximation provides us some benefits when aggregating individual task estimates into combined project estimates.

Continued on the next page…

Blue Ocean Strategy provides an interesting reactive analysis of companies and markets.  Personas are used to understand your customer’s needs.  Combining the two provides powerful proactive insights when positioning your product for market success.

Blue Ocean Strategy

The book, Blue Ocean Strategy: How to Create Uncontested Market Space and Make Competition Irrelevant, by Kim and Mauborgne, was written in 2005.  The book presents a very compelling way to visualize competition in your market by contrasting the emphasis (or effectiveness) of each company at solving particular problems.  The authors argue that red oceans are competitive markets where companies compete to solve the same problems for the same customers.  Their main idea is that by identifying (and solving) unaddressed problems, you can define a new market – a blue ocean with no relevant competitors.  The “red” in their metaphor implies blood in the water, caused by cut-throat competition.  Their position – by defining a new market boundaries with no competition, you eliminate the blood in the water and give yourself a calm, blue ocean in which to navigate your company.

This is a compelling idea, and has a lot of merit.  The Smarter Product Managers book club (started by Cindy Alvarez) reviewed this book last month, and one conclusion we reached during the discussion is that the examples in the book felt “reverse-engineered.”  I feel that the lack of prescriptive advice from the authors created a sense of “that’s fine, but how do I apply these ideas proactively?”   Some of the examples in the book, like Cirque de Solei and Southwest Airlines, felt very compelling (and are often cited), while others felt a bit more contrived.  Almost as if the authors were searching for data to support their arguments – a big no-no for product managers.

Creating a Blue Ocean Strategy Map

The book presents examples of “mapping” markets based upon the strength of the offerings from companies, along different dimensions.  The following example was created by me (and is used throughout this article), but follows the pattern of analysis described in the book.  This is an unresearched, hypothetical example analysis of the market for vaccuum cleaners – or more generically, floor cleaning products.

 [larger image]

The diagram above identifies seven dimensions by which you could characterize the offerings from each company.

• Breathe Easier / Anti-Allergen – sometimes, people vaccuum their rugs in order to reduce allergens (by sucking up dust), making it easier for them to breathe in their homes.  Vaccuum cleaners work by sucking up air through the carpet, collecting the dirt (that is sucked up along with the air), and filtering the exhaust air (that is expelled into the room).  Different vaccuum cleaners pay different levels of attention to removing allergens during this cleaning process, by (1) getting the allergens out of the carpet and (2) filtering the allergens out of the exhaust air.
• Remove Stains – one motivation to clean your floor is to remove stains.  This is not generally a focus of vaccuum cleaners, but products like the Green Machine do focus on removing stains, while also “picking up dirt.”
• Physically Easy to Use – after back surgery, one of the key pieces of advice my mother received from her doctor was “no vaccuuming.”  When you are young and healthy, you don’t expect vaccuuming to be something that is forbidden by your doctor.  Shoveling snow, climbing a mountain, running a marathon, yes – but vaccuuming?
• Reducing Hassle – Why don’t we vaccuum every day?  Because it can be a hassle to get out the vaccuum, set up the equipment, and actually use it.  When you reduce the hassle involved in vaccuuming, you are likely to vaccuum more often, thereby realizing more benefits from vaccuuming.
• Saving Time – Hassle is not the only barrier to vaccuuming.  If you could vaccuum your house in 5 minutes instead of 105 minutes, you would do it more often.  As a teenager, I used to jog behind the lawn mower, just to get the yard done so I could move on to other things.
• Reliability – While reliability is not a major factor when vaccuuming your house (once), it is a key component of making purchasing decisions – because you don’t vaccuum just once.  The more you vaccuum, the more you care about reliability.  This is also an indirect cost factor – a vaccuum cleaner that has to be frequently repaired or replaced will cost more per year than one that has the same purchase price and is more reliable.  This is also an indirect hassle factor – you may delay vaccuuming your home until you can make time to get a broken vaccuum cleaner repaired.
• Avoiding Cost – A direct focus on cost is primarily driven by purchase price.  Other factors (like reliability, and the cost of replacement bags or filters), but people place greater emphasis on purchase price when thinking about cost.  Consumer Reports and other “product review” companies will often “do the math” for you, taking into account all of the post-purchase costs to calculate total-cost-of-ownership.

The above diagram, with fictitous values for “Strength of Competitive Offering”, shows the competitive landscape for the vaccuum cleaner market.  One of the very powerful features of this visualization is that the Roomba faces “no competition” from the other companies in three of the seven categories – Ease of Use, Hassle, and Time.  A Blue Ocean Strategy analysis would say that the Roomba has created their own market, with an absense of competition.

This gets back to our insight that the examples felt “reverse engineered.”  The Roomba does have a dramatically different value-proposition, and focused on dimensions that are not addressed by their competition.  So why isn’t the Roomba in the book?  Because they still compete in the red ocean.  Unlike Southwest Airlines, they did not differentiate their offering in a relevant way.  And that’s the problem we found in trying to apply the principles from the book.  

It is not enough to be innovative and differentiated, which the Roomba certainly is, you have to be valuably differentiated.

So how do you differentiate valuably and proactively?  Identify the problems that your customers value, but don’t yet have solutions for.  How do you do that?  With personas.

Personas

Personas represent customers in your target markets.  Markets are not homogenous – different people in your markets value different things for different reasons.  You develop personas as archetypes to represent subsets of the customers in a given market who share common problems.  More concretely, personas share common valuations of shared problems.

To combine persona development with the market-mapping concept from Blue Ocean Strategy, create the same map, but for personas.  Instead of identifying the strength of each company/product along each dimension, identify how much each persona cares about each particular dimension.

 [larger image]

In the chart above, I’ve created 6 hypothetical personas who populate our market:

• Single Parent – the classic superman / superwoman who does everything, including keeping the house clean.
• Housekeeper – someone employed to keep someone else’s house clean.
• Office Cleaner – the person who vaccuums your office at 2am every night and picks up all the empty Diet Coke cans and Twinkee wrappers.
• Kid (Doing Chores) – most kids won’t ask for a vaccuum cleaner for their birthday, but when you child is responsible for vaccuuming, do you want a battle every week because they hate it?
• Homeowner – like the single parent, but maybe without kids, and definitely with fewer responsibilities (and more time).
• Retiree – like the homeowner, but with a lot more time, and a less robust physical condition.

[Note - the above examples are really stereotypes and not personas (there is no research to back them up - they are entirely fictional) - don't fall into the trap of thinking persona development is this easy.]

OK, now we have an understanding of how much importance each persona places on each dimension.  It looks like a mess, for this example, when you try and absorb the big picture.  If you focus on a single persona, you get great clarity about what that type of customer cares about.  Look at each curve individually, then look at them all together.  If you’re thinking ahead, you’ll suspect that the Roomba is perfect for the kid doing chores.  But wait, we’ll come back to that.

One insight we can gain from this messy view of the market is that you can’t create a product that is “perfect” for all of these personas.  You have to target a specific persona, and tailor your product for that persona.

Mapping Personas to Products

Now we have two views of the market, along 7 dimensions.  We have assessments of the relative strength of each competitor’s offering, and we have estimates of the relative importance to each persona – for each dimension.  What we need to do is combine the two.

All of this analysis runs the risk of introducing a notion of false precision - we have a lot of data, therefore it must be accurate.  So our inclination may be to try and do some mathematically refined, scientific analysis.  I suspect that would be wasted effort, providing no additional insights, and risking leaps to the wrong conclusions.  I propose a simpler approach.

The analysis we really care about – which product offering is best aligned with the needs of each persona?  A simple mathematical equivalent of “which curves match the best” is an easy analysis.  By comparing the values from each competitor curve with those of each persona curve, we can create a series of “how good a fit is it?” values.

 [larger image]

Imagine a persona saying “how well does each company’s product align with my goals?”  This “simple math mashup” takes into account both “they succeed at what I care about” and “they haven’t invested in something I don’t care about (at the expense of something I do care about).”

You can see that the Roomba clearly wins with kids doing chores.  The rest of offerings stand out less, but do provide insight.  Now you can easily drive strategic decisions – “we want to be the vaccuum of choice for housekeepers” now drives some obvious emphasis on ease of use and a reduced emphasis on reducing costs.

Improving The Model

My two main problems with the blue ocean strategy were lack of relevance (who cares about dimension X) and magnitude (which dimension is most important?).  The model above addresses only relevance – a focus on target personas.  We can overlay some “relative importance of each persona” data, or just manually focus our efforts on each persona in series.  

The other magnitude challenge is in understanding not just which problems are important in absolute terms (kids care a lot about saving time, but not cost), but in relative terms (kids would trade an hour of time for a modicum of ease-of-use).  Essentially, defining the utility-curves for each persona.  For any but the largest, most saturated markets, I would hesitate to do the utility curve analysis in detail – it feels too heavyweight and un-agile.

Conclusion

The Blue Ocean Strategy book provides us with a visceral tool for visualizing relative offerings from competitors in a given market.  Combine it with the same visualization approach for personas that participate in that market, and you gain insights into which problems to solve next to achieve product success.

Jump forward in time to the day of your next big product launch (first release, new features, new market segment, etc).  And your site/application crashes due to the “unexpected” demand.  All you can do now is look for a bucket of water to put out the fire.  What could you have done to prevent this disaster?  Jump back to today and start doing it!

Backwards Planning

Depending on how you look at things, this is a backwards planning exercise, or a variation of the  remember the future innovation game, or risk management, or proactive product management.  You can avoid a disaster by imagining what might happen, then hypothetically figuring out why it (would have) happened.  That leads to planning how you could prevent it.  And now you’ve left the dream world of a Gedanken experiment and returned to the real world of product management.

Problem Triage

The way to approach this is straightforward.  Imagine some failure scenarios and the importance of preventing them:

1. Imagine a failure scenario.
2. “Predict” the likelihood of the failure.
3. “Estimate the impact of the failure.
4. Repeat for each scenario

You can prioritize your failure scenarios by multiplying the likelihood of each with the impact of each, and sorting them from largest to smallest.  Then determine which ones you’re willing to address, and which ones you’re willing to risk.  You may not be able to predict the likelihood of some failures (at least until you do a root cause analysis).  Take each of these and put them directly above the scenario with the next highest impact.  The rationalle is that these are so bad, that you really want to find out how likely they are to happen.  Once you predict likelihood (see below) you can reprioritize.

Root Cause Analysis

For the failure scenarios you choose to address, the next step is to do a root cause analysis that identifies why it might have happened.  The best tool for capturing this analysis is an Ishikawa diagram.  Consider that one problem you might face is your website crashing.

[click for larger version]

Essentially, you can crash your site by having too many users, too many concurrent users, or too many concurrent sign-ups.  Developing a cause-and-effect diagram (another name for an Ishikawa diagram) is usually an iterative and exploratory process.  You probably won’t create the simple version above first.  You may ask your implementation team “What can cause the website to crash?”  For each of their answers, you identify when that situation can happen.  Or you start top down.  Most likely, a mix of the two.  Your completed root cause analysis may look like the following:

[click for larger version]

At this point, your team can probably predict many (maybe all) of the root causes of a website crash.  The predictions may be conditional – “we can handle 10 concurrent users, but 20 probably kill us, and 100 definitely would.”  Developers are notoriously good at answering questions with conditional statements that reveal the nuances of their thinking.

Remember that you’re looking back from the future.  At product launch, what are you hoping for / reasonably expecting?  For this example, assume it is 10,000 total users, with 100 concurrent users (normally) and 500 concurrent signups.  You determine these numbers by working with your PR, marketing, or mar-com people (or wearing those hats, when it is all you).  Your plan is to do a big launch with a demo and a promo code for signup.  You know your audience will have internet connections, and will have twitter running at the time of your presentation.  You expect/dream of an immediate burst of signups, followed by tweets and word of mouth, and eventually blog articles causing additional growth over the next couple of weeks.

Use this data to feed back into the developer’s conditional responses.  If you’re like me, you will have found “absolute certainty of failure” from something.  And you may have even identified the thresholds for each element.  For example, database loading can handle 75 concurrent users, but with the current implementation, you only have enough database connections available to support 25 concurrent users.

Jumping back to the present, you now have some very discrete, and very important things to do before your launch.  If you need to, revisit the prioritized list of failure scenarios.  By looking at the next level of detail, have you found that the order of importance (to fix) has changed?  What about the “must fix” versus “willing to risk” line?  Has it moved?

Fold the “must fix” items into your backlog, and prioritize them relative to the other capabilities on your roadmap.  As a side note – make sure you’ve built in some testing to make sure you actually prevent the problems.  This might even be a great opportunity to implement “performance regression tests” – it is not enough to prevent bugs, you have to prevent slowdowns.

Rethinking the Problem

Without going into details on how the team will solve each problem, make sure that together you keep the Ishikawa diagrams in mind, and see how any proposed solutions might “reappear” on the diagram.  For example, rewriting your database connections to use asynchronous processes and a set of pooled connections may prevent a crash, but it may really hurt performance.  You may not have time to find an elegant solution.  So stop and rethink the problem.

At this point, you’ve said

1. Given a marketing plan / launch strategy, we would crash the website.
2. We can make changes between now and the launch that will double the number of concurrent users we can support (or whatever), but that is not enough to support the launch strategy.
3. Solution: Change the launch strategy.

Maybe you can’t support a wide-open promo-code based signup.  You should modify your launch so that it can only create as much demand as your product (including pending improvements) can support.  Maybe you limit it to the first 1,000 new users (probably more code to write to enforce the limit).  Maybe you launch with per-user invitations, where you can control the speed of propagation of invites (start with 100, when those have been sent, make another 100 available, etc).

Entire Team Problem

This is a problem that is solved collaboratively, by the entire team.  It is not just a “go write the code” problem.  What your product can support at a launch should drive how you choose to launch, just as how you choose to launch should drive what you want your product to support.  

You may have to delay a key capability in order to scale.  Does your marketing team know this?  Slightly less bad than crashing would be announcing a feature that is disabled.  Still need to announce the feature?  Pre-announce it: “Coming in a month…”

This stuff is important for every company and product, but it is especially critical for start-ups.  As a start-up, you have limited opportunities to grow, and a limited safety-net to catch you when you fail to capitalize on those opportunities.  So make sure everyone (not just the development team) is aligned to make the best use of each opportunity.

Conclusion

You have an opportunity to prevent problems.  All you have to do is imagine that they have happened in the future, figure out why they would have happened, then do what it takes (in software, or organizationally) to prevent them.

Just because your requirement is not a user story does not mean you have to throw it out when planning your next sprint.  See one way (that is working) for managing non-functional requirements with an agile team.

Product Backlog Stories

Every article* I can remember reading that explains how to manage a product backlog talks about user stories.  Those articles are necessary, but not sufficient .  You’ll create better products by developing them from the outside-in, with a user-centric point of view.

*One loophole – scheduling refactoring (or the payback of technical debt).  A lot of articles talk about the need to do this, and refactoring is pretty much the opposite of a user story, since by definition, refactoring improves the software without introducing new capabilities.  The best idea I’ve come across for incorporating refactoring as part of a sprint is to write a user story, with the system as the user, and the lead architect / developer as the primary stakeholder.  This actually works really well, but it works by treating the work as a user story.

Atomicity

What is really important, when scheduling sprints (or releases, if you are not doing scrum) is that you are scheduling solutions to atomic, measureable, valuable problems.  Atomicity is the reason for breaking epics (really big stories) up into smaller stories.  It is also important that you communicate them unambiguously.  That might mean writing user stories or writing use cases instead of user stories, when things are complicated.

The problem is, not every atomic requirement can be represented with a user story.  Some things that must be, are not stories.

Non-Functional Requirements

Non-functional requirements always seem to be under-emphasized when writing requirements.  The Twitter fail whale has become famous, because twitter could not scale to meet the demands of a rapidly growing user base.  Maybe the Twitter team planned for scalability, but demand simply outstripped it.  Or maybe they failed to plan for it.  Either way, they failed to meet the non-functional requirements of supporting the growth that they did have.  (Un)Luckily, this type of problem self-corrects.  Scaling failures drive away users, reducing the need to scale, until balance is achieved.

Product managers and business analysts tend to neglect non-functional requirements.  Unfortunately, this is especially true when managing with a focus on users and their goals.  Not because goals don’t drive non-functional requirements – they do.  I believe this has happened because historically, non-functional requirements were treated as an after-thought.  In reality, they are explicitly driven by goals.  I proposed an equal rights amendment to the structured requirements domain model almost three years ago.  In short, it explicitly calls out the relationship between goals and non-functional requirements.

Agile Non-Functional Requirements

Getting non-functional requirements into your sprint planning is actually not that hard.  You only have to make two tiny adjustments to get from the waterfall world to the agile world.

The first adjusment is that you have to treat non-functional requirements incrementally.  Non-functional requirements often affect all of the other requirements – so they seem massive and unweildy.  You have to decompose them.  Consider the platform-compatibility requirements for a web application.  You may have to support IE6,7,8; Safari on Windows, Safari on OS X, and Firefox on Windows XP and Vista.  That could be incredibly daunting.  So break it down.  Your first group of users (key persona) are primarily Firefox/XP users.  So the first platform you support is that one.  The next big platform for your persona group is Safari on OS X.  Add support for that next without breaking the previous support for FF/XP.  With each release, you add a platform (or two, or none).  You are conspicuoulsy addressing the needs of your target users.  The key is that once support is added for a platform, all future development is required to “not break it.”

Each non-functional requirement is cumulative.  This is the second adjustment.  All development, once a non-functional requirement is in place, must continue to honor it.  You wouldn’t break a previously released capability (functional requirement), so don’t break a non-functional requirement.  You have to determine, in each sprint, if additional functionality is more important than additional platform support.  And add in the platforms as they become the most important “next things to do.”  In waterfall projects, I’ve seen many teams break and re-break platform support throughout the development process, with the knowledge that it only has to work “at the end.”  Include platform-specific support in your continuous integration tests.

You have a launch event coming up in six weeks.  You have an established user base.  You’re also developing a key new set of capabilities for your product that you believe will be a big hit and drive significant growth for your product.  You have a small group of people in a private beta, providing you with feedback about the new development.  If you believe the launch will cause your customer base to double very quickly (maybe over a month), how do you plan for that?  This is a serious scalability non-functional requirement.

Break the non-functional requirement up into cumulative requirements.  Assuming your plan is to add 10,000 users “at once” – have your implementation team brainstorm what that could/would mean for the system.  [Also, make sure you coordinate with your community manager and marketing folks, both to validate the anticipated growth, and to device any contingency strategies in advance.]  After talking with your development team, perhaps you learn that “at once” is a nuanced proposition.  Literally at once is very bad.  Spread out over a few days, not so bad.  OK – you can deal with this too.

Imagine you already have the following non-functional requirements in place:

• The system must be available 24/7, with no more than one hour of down time per day, and no more than one outage per day.
• The system must respond in under 2 seconds for >95% of uses [ in key user story].
• The system must respond in under 20 seconds for 100% of uses [in same key user story].

Based on what we’ve said above, these non-functional requirements must not be broken.  They essentially define the acceptance criteria for our scalability requirements.

Consider the following as the non-functional requirements that must be deployed by the time of launch (six weeks, or three releases from now):

• The system must support 10,000 additional users added in the month following SXSW.
• The system must support 500 new users signing up and initiating [troublesome user story] within the same hour.

Your dev team does not really know exactly what needs to be done – they just know that the current solution won’t scale – it barely meets the existing non-functional requirements.  They propose a couple redesigns that may get the job done.  But they point out the need to actually test that the designs work.  Schedule the following for the first release:

• The system must support 100 additional private beta users.
• The additional users will all have [troublesome user story] initiated within the same hour.

The second one is really a pragmatic solution – artificially creating a spike in demand, to test out the scalability of the new code.  From that data, we can determine what we need to do to hit 10,000 additional users, and to support 500 concurrent [troublesome user story] instances.  By managing expectations of the new users (that you’re queuing them up to test scalability), you can get the data you need.  And you have a couple more iterations to improve if needed.

As a benefit, you get to completely avoid the fail whale.  The other half of this is making sure you can constrain the rate of new-user sign-ups.  Work with your community manager and marketer to make sure you position this effectively.  You are creating scarcity, which may increase demand.  A crashed server won’t.  If your team can’t support 10,000 in time for the launch, plan on 2,500.

Conclusion

You can plan and schedule more than just user stories.  And your product will be better for it.  Give those non-functional requirements a chance.

User Stories are one of the key agile artifacts for helping implementation teams deliver the most important capabilities first.  They differ from use cases in some important ways, but share more commonalities than you might think.

User Stories Applied

Mike Cohn wrote User Stories Applied: For Agile Software Development.  It is the book for understanding how to write, estimate, and use user stories.  If you’re thinking about trying out “agile” for the first time – and you haven’t read his book, you need to.  He provides great detail and anecdotes about how to write, manage, and utilize user stories.

What are user stories?

A user story is a user-centric description of the goals that one or more people will achieve by using your product.  User stories are applicable whenever there is a person, interacting with an interface to a product, to achieve a goal.  They aren’t just for software – even though the tone of this article may imply it (since I live in a software mindset most of the time).

A user story is written in the format

• As a [person in a role] I want to [perform some activity] so that [some goal is achieved].

That’s it.  No more.  Perfectly atomic, very concise and unambiguous.  Where did this elegant idea come from?

User-centered design (UCD) is an approach to product design that can almost be considered a philosophy.  In college, I used to say that industrial designers designed things from the outside in, and engineers designed things from the inside out.  The ideal products are the ones that marry form (outside) and function (inside) in an elegant solution that blurs the lines of distinction.  The first designers of engineered products were the engineers, and the first designers of software were the programmers – they were the only ones who knew what could be accomplished.  Anyone who has used an application with a user interface designed by a database expert can remember what that was like.  Eventually, someone successfully adopted the mindset of designing a product based on what someone could use it to do instead of looking at what it could do and figuring out how to use it.  That under-emphasizes the importance of UCD, but you get the idea.

Out of UCD come different things that really drive how we create products today.  Kessler and Sweitzer focus on understanding these user goals in Outside-In Software Development.  Stakeholder goals drive (arguably, are) requirements.  But it is difficult to develop actionable implementation plans directly from goals.  You need something to bridge that gap.

A user story, in an agile process, bridges that gap.  In Wiegers’ world of structured requirements, that gap is spanned with use cases.

Conceptually, a user story crosses the same chasm as a use case.  A user story defines what people need to accomplish (e.g. faster call processing), in order to achieve the goals of the company (e.g. lower call-center costs), given a solution approach (write software versus outsource to lower-cost call center employees).

Now we find ourselves in a confusing situation – there are user stories, use case scenarios, and at least three different kinds of use cases – formal and informal use cases, and use case briefs.  How are they different, and how are they the same?

Usage Descriptors

Use cases, use case scenarios, and user stories all document descriptions of how a product will be used.  They vary, along a continuum, in terms of the amount of overhead required to create each.

• Formal use cases require the most effort.  They have a lot of pomp and circumstance going on, but they describe all the permutations of how some person does some activity (or its variations).
• Informal use cases are pretty much the same – just less formal.  The challenge is to provide the right level of detail, without the guide-rails of the formal use case to remind you.
• Use case briefs have almost no overhead, but have the same “how much is enough” challenges of informal use cases, only more so.  Think of it as a single-paragraph description of a formal use case.
• User stories have the least overhead, and provide the least context.

Use case scenarios are a slightly different breed.  Like a user story, a use case scenario describes one path through a multi-path use case.  But you can’t (or at least I’ve never seen anyone) create a use case scenario without first creating the formal use cases.  This artifact basically combines the weakness of a formal use case (high overhead) with that of a user story (limited context).  It can be very handy for developing test cases if your testing team is not well versed in writing test cases.  We’ll leave use case scenarios out of the rest of the discussion.

The cost of high overhead in documentation comes with a benefit – increased detail.

Because a user story captures a single path through a use case, while having less overhead, it also captures less detail.  This is ok, because an agile process stresses communication over comprehensive documentation.   You start to run into inefficiencies when the amount of conversation (especially when repeated) is burdensome.  The amount of conversation required is a function of the amount of domain expertise, or context, that the implementation team already has.

Should I write User Stories or Use Cases?

It depends on your audience.  The formal and informal use cases, in addition to having more overhead, also provide more context, and allow you to describe more complex usage patterns of your product.  Some uses are so complicated that you have to use a use case to describe them.  Others are so simple that anything other than a story is wasted effort.  The interpretation of complicated and simple, though, is not purely an assessment of what the users want to do – it varies with the level of domain expertise of your implementation team.

Some teams simply aren’t equipped to consume user stories or use case briefs.  You have to write your requirements for the readers, so you need to know when people are struggling to implement solutions based upon stories or briefs, and give them more structure and formality.  You have to adapt to the realities of your current situation, and the experience of your team.

You could replace the words “Reader Domain Expertise” with “Writer Trust of the Team” if you wanted, and the graph would look about the same.  This is another concept that is critical to a team working effectively – trust equates to delegation.

If you can trust your team to come up with a solution that matches the story, delegate that “next level of detail” to them.  If you can’t trust them, then don’t.  But you should.  One of the benefits of agile is that your team will quickly give you a solution, and then you can give them feedback.  If they don’t create a solution that meets your interpretation of your user’s needs, then give them feedback, and they will change it.  The agile process actually relies on this dynamic to allow less-verbose artifacts to still be effective.  Someone on the team will sketch out what the solution will look like, and get your feedback, before implementing.  The more this collaboration happens, the more effective a light-weight document will be.

Conclusion

Don’t just rely on platitudes about stories and use cases.  Think about the nature of the different artifacts.  Think about their strengths and weaknesses.  Consider how you interact with your team.  Should you trust your team?  Do you trust your team?  Determine what they need, and give it to them.

Netlife Research brings us the 2009 Bad Usability calendar.  Get it while it’s hot.

Download it from the 2009 bad usability calendar page on their website.  Or check out our article from last year for the 2007-2008 bad usability calendars.  They really do have some great points to make.  Two in particular got my attention this year – any guesses which months?

Last year, over 100K copies were downloaded.  Act now and be on the bleeding edge of the first 2% to download for 2009.

There’s really only one way to travel down a waterfall – in a barrel.  A lot of people died this way, but some survived.  Software projects have been predominantly waterfall projects since the start of software projects.  And stakeholders rode down those projects, basically in a barrel.  The people riding Niagara Falls 100 years ago didn’t know if they would survive until they got to the end.  Stakeholders in waterfall projects don’t know if they will succeed until the end.

An agile project is dependent upon tight interaction (and feedback) with stakeholders.

If you’re running an agile project, and your stakeholders are old-school barrel-riders, how do you make it work?

Expectations, Documentation, and Communication

The success of any project is dependent on setting and managing stakeholder expectations.  In Managing Stakeholder Goals, we talked about assuring that those goals are addressed by our requirements.  And in a later article, we proposed a way to balance the goals of different stakeholder groups when those goals are in opposition.  Those are good tools for making sure we are initially aligned with what we are hearing from stakeholders.

We need to also apply active listening techniques to assure that what we thought we heard is what the stakeholders thought they said.  One way to do that is to write use cases (or user stories) so that we can communicate the intent of the product back to the stakeholders.  This still primarily helps with kicking off a project, in the desired direction, with the correct goals.

There are many ways to document this initial intent of the project and stakeholder goals.  It is analogous to a stakeholder selecting a sturdy barrel and picking a good spot on the river bank to push off.  We’re well positioned to eventually succeed, assuming nothing changes.  After a harrowing fall, we find out how well we did.

A well-run waterfall project will provide status updates to the stakeholders along the way.  Imagine your stakeholder wearing one of those in-helmet headsets that NFL quarter backs use.  We publish effective status reports to let the stakeholder know what’s going on.  Like the falling barrel, however, a waterfall project has inertia.  The project, barring significant outside influence, will keep going in the direction it started.  Projects have change control boards to manage that change, but emotionally, I find that a formal change-approval process tends to inhibit change, rather than encourage it.  That barrel will keep falling.

Some project teams will try and do very heavyweight documentation to maximize the likelihood that the project will end up where it should.  The problem is, this heavyweight documentation only improves the chances that the project will end up where we used to think it should go.  It does not help us change the trajectory of the project as new insights are gained.  Just as responding to those changes provides a competitive advantage, ignoring those changes exposes a competitive weakness.  Someone will come along and exploit your weakness if you don’t respond to change.

From these dynamics, we can conclude that “big up-front requirements”, while better than “no requirements,” are actually a waste of energy and time, relative to an ongoing adaptation to change.  That adaptation to change, however, should also be documented.  It needs to be documented for two reasons – to manage expectations of stakeholders, and to keep the implementation team focused on creating the most valuable capabilities in your product.  As you get smarter about what is valuable, you need to apply that knowledge and change the path of the project.

This is where communication becomes critical with stakeholders too.  Especially the old-school barrel-riders.  They are used to being in the barrel, maybe with an occasional “looking good – you’re still falling straight down” message along the way.  They are not used to hearing “we’ve decided that gliding over the rocks is more valuable than falling – please extend your wings now” messages.  The shocked “what wings?!” response is what they immediately think.

Agile Expectations

An agile project will avoid the big up-front requirements, and gather just enough requirements for right now.  What your stakeholder might hear is “agile is magic – we don’t need detailed requirements.”  The real message is “agile is better – we don’t need details about the requirements yet.  But we will need them later.”

A given team needs the same amount of specificity in requirements to achieve the same result, regardless of project approach.  One team I worked with would not set the tab-order in a form to go from top to bottom if you didn’t specify that.  It simply didn’t occur to them.  Another team was very effective with “gather the following data in a form…” – and they would layout the form, manage tab order, add reasonable field validation, assure proper markup and support for screen readers (for visually impaired users), and implement a candidate error-message/feedback design for users.  The first team would never succeed without details, the second team would view those details as wasted effort by me to write them, and by them to read them.

I think most of the initial creators, proponents, and early adopters of agile processes are members of the second camp.  They designed agile to not require detailed documents, but to allow it when needed – because they acknowledged that some people need the details.  They were reacting to heavyweight processes that were designed to work for “any team” at the cost of slowing down the best teams.  Kent Beck told me once that when people tell him about the importance of testing, his response is “so, lack of testing burned you before?”  When people stressed the importance of gathering requirements, his response was “you’ve been burned by a disconnect with stakeholders.”

Enough is the operative word here.  You have to document enough.  You have to communicate enough.  You have to set expectations with your stakeholders that things will change.  And as those things change, you have to stay joined at the hip with your stakeholders to validate those changes.

While your team is responding to changes (based on feedback from stakeholders and the market and competition and implementation discoveries) with new plans, you have to feed that information back to the stakeholders.  It is a two-way communication.  And it needs to be a continual communication.

I joined a six month project once in the last month of the project.  Here’s a sanitized sequence of events describing what happened.

1. Initial vision for the project was defined and requirements defined and tied to goals and value.
2. The estimates came back and said “no way it will happen before the business-imposed deadline.”
3. The team said “let’s be agile” and chose a component of the vision to do first.  That component could be completed by the deadline, and stakeholder expectations were set – (a) do this visible thing first, “meet” the deadline, and then (b) regroup, re-prioritize, and then do the next thing next.
4. The team started developing the solution, and worked for 5 months.  After 5 months, someone concluded “we won’t finish by the deadline.”  A couple weeks later, the estimate was that the team still had between 1/3 and 1/2 of the work remaining to complete the first component of the vision.
5. In presumably unrelated events, all but 2 of the internal stakeholders left the company.  Literally.
6. When the CIO and president of the company engaged the project team, they weren’t thinking “we have an over-run on the first component of our vision” – they were thinking “not only is it not done, but what you’re trying to do is not the right thing.”
7. Project was stopped one week after the original deadline for the first component (which was not completed).

It may be that this was unavoidable, but one thing was clear – the “build the first thing first” expectation was not communicated effectively.  It didn’t help that the team did not track velocity, so it wasn’t until the end of the project when the “you’re going to hit the rocks” message was first heard by the stakeholder in the barrel.  Way too late to affect change.  For this and other reasons, I contend that the project was not agile.  It was a Chevrolet with a Ferrari sticker on it.

We can ignore the labeling of the process and focus on the lack of ongoing expectation management as the ultimate doom of the project.

Evolving Expectations

Patrick Masi had a couple great comments on our Simple Agile Model article.  Patrick pointed out a problem with simple documentation artifacts like this.  The early “here’s all we need right now” artifacts were insufficient to capture the full extent of what the stakeholders needed.  I’m not sure exactly where things broke down, but Patrick implied that the ongoing clarification with stakeholders was not happening.

This is a completely different manifestation, I suspect, of the exact same problem described above.  Thinking about Patrick’s comments is what got me heading down the whole “barrel rider” path in the first place.

I don’t believe I’ve worked with any stakeholders who would say “yes, ignore what we learned this year – it is more important to build last year’s product.”  I’ve definitely worked with stakeholders who did not have time to commit to the support of an agile project.  The ironic thing about agile projects is that they may do more requirements work through the course of the project than non-agile projects.  An agile project will respond to change.  That means some work is re-done.  It also means that some valueless work is avoided.  You can’t categorically say which influence is larger.

One possible source of the pain Patrick’s team felt is mis-set expectations of what is being delivered when.  Imagine developing a checkout-process for an eCommerce website.  The complete checkout process is too large for a single sprint, so you break it down into valuable atomic deliverables for each sprint:

1. Registered customers can buy any products, using previously saved billing / shipping info.
2. Individual product and “per order” discounting works and customers can use gift cards to pay for all or part of the order.
3. Anonymous customers can place orders (without accounts) and registered customers can modify billing and shipping information.
4. Orders can be placed as gifts (with gift receipts and wrapping services), and online call-center reps can interact with customers real-time to help with the process.

This is a nuanced message – basic checkout in sprint 1, with increasing capabilities in each sprint, until “complete checkout” is done in sprint 4.  That is a reasonable plan, but requires a more detailed conversation with stakeholders so that they know what is coming and when.  You don’t want someone freaking out after the 3rd sprint when they can’t place gift orders.

Another possibility is that the elicitation process before the third sprint (re-visiting checkout again with the same stakeholder) did not tease out that anonymous users must provide an email address (for order confirmation email, and to secretly create a “shadow account” for that customer).  All of those details need to be gathered, and it can be harder to spread out the conversation over multiple sprints than it is to have it all up front.  This is just incomplete discovery, but with delayed (and recurring) impacts.

In User Stories Applied, Mike Cohn stresses that the brevity of user stories is intentionally designed to facilitate (and I would add “dependent upon”) conversation.  I find it to be both a strength and a weakness of user stories.  It is strong because you can cover a lot of ground quickly (breadth) and capture a number of stories, much like the list above.  It is weak because the requirements documentation does not stand on its own – it requires conversation to fill in the details.  I’ve had some success using the “Verify that…” user acceptance tests as the method of documenting those details (depth), in conjunction with the brief, easily consumable stories.

You can write the stories quickly to decompose and schedule across sprints (revisiting the schedule as things change), and then write the verify statements as UAT for each sprint as it occurs.

Another benefit of the UAT is that it is explicit and cuts through the walls of the barrel for a stakeholder who “doesn’t get agile” as Patrick puts it.  “Here’s our lightweight multi-sprint plan – now lets define the specific acceptance criteria you need” is a pretty effective conversation.

Conclusion

You have to stay closely connected to your stakeholders – not just for messaging, managing, and changing the big-picture direction of the project, but also to drill down into the details at the last practical moment.

The second productcamp for Austin is just around the corner!  Are you going to be there?  You should.

Announcement

Paul Young has written a great announcement and explanation of what product camp is.  I really enjoyed and benefited from the one we did last spring, and I can’t wait to attend and present at this one!  If you are or can be in the Austin area on Jan 24th, make sure you head to the UT Austin College of Communications CMB building, studios 4B through 4E from 8 to 5:30.  And the networking afterwords is sure to be even better than last year too.

Presentation Topics

A key takeaway from our first productcamp was that people most enjoyed the presentations that were more collaborative.  These sessions tended to give attendees the ability to immediately put to use some really good ideas.  I’ll wager that we’ll have a mix of presentation styles again this year, but that it will skew to the interactive and practical.  The roundtables were also very well received, and I hope we’ll have more of those too.

I’m really looking forward to presenting again, and would love suggestions from you about topics you think would be valuable.  I guess this is pre-collaboration :).  At the moment, I’m leaning towards a presentation around using wireframes to rapid-prototype ideas in support of an agile development team. I’m currently doing this with the developers and stakeholders at OtherInbox, an Austin start-up that was one of this year’s TechCrunch 50, doing some awesome stuff to eliminate email overload.

So far, this is proving to be an effective way to rapidly evolve ideas and minimize the overhead of communication – and we’re doing it with a distributed team.  We’ve been using a combination of balsamiq mockups and google docs.  By the time product camp rolls around, we’ll have many sprints worth of experience under our belts, and I should be able to provide a mix of how-to’s and lessons learned.

Add a comment below if you have another idea, or if you have any feedback on this one.

A picture is worth a thousand words.  Agile values working software over comprehensive documentation, and it values customer collaboration over contract negotiation.  With that in mind, how much is a picture of a model worth?  Check out a simple example, how it helped, and what we didn’t do.

A Complex Conversation

I was working with a client on an eCommerce website, where one of the things that was important to the client was having and managing an affiliate network that refers visitors (and ultimately customers) to the website. The conversations around exactly what the stakeholders of the eCommerce site wanted were pretty convoluted, and different stakeholders described the “requirements” differently.

Affiliate Network Background

An affiliate network can work in the following way:

1. When someone visits a website, and does so by directly clicking on a link on an affiliate partner’s page, the website is able to identify the affiliate partner who “sent” the visitor to the website.
2. The website keeps track of who “sent” the visitor.
3. If the visitor becomes a customer, the affiliate partner who sent the visitor is compensated.  One way to compensate that affiliate partner is by giving them a percentage of the revenue that the customer generates for the website.

This can be a lucrative model for both the eCommerce site and the affiliate partner.  The partner gets paid for funneling traffic to the eCommerce site, but not necessarily doing any other work (beyond what it takes to encourage people to go to the eCommerce site).  The eCommerce site gets increased traffic, at a manageable cost – proportional to the revenue generated by the incremental visitors (who would probably not otherwise visit the site).

Each business gets to focus on its core competency (sending traffic, for the affiliate partner; converting that traffic into revenue, for the eCommerce site).  The eCommerce site invests the majority of the time and money (probably orders of magnitude more investment), and the affiliate partner shoulders all of the risk, but can do this for almost no cost (if they want), in exchange for an uncertain revenue stream.  This blog is currently trying out an affiliate relationship with Carbonite – because I love the product (and therefore think people who visit Tyner Blain will too), and because the revenue generated, if any, will help defray the costs of writing and maintaining the blog.

Specific Complexity of an Affiliate Model

The complexity of this “simple” model becomes apparent when you ask a few clarifying questions.

• Will there be multiple affiliate partners?  (Yes)
• If someone is referred by multiple affiliate partners (on separate occasions), who gets credit? (The last affiliate partner who sends the customer to the site)
• Does the visitor have to purchase during the visit that was made from the affiliate partner’s site? (No)
• Is there an expiration date, after which an affiliate will not get credit for customer purchases?  (Yes, “N” days).  [Note: I am obscuring the actual value for this article, but N is a real number.]
• If a customer who was referred by an affiliate makes multiple purchases, for how many of them is the affiliate partner credited?  (One)

Asking these questions in different ways generated some different, and potentially conflicting answers.  The list above is the “after we put it all together” answer.

A Simple Model

We solved this problem by grabbing the immediately available stakeholders and pulling them into an office with a whiteboard for about 3 minutes.  [Ed: Our first article about using object-oriented analysis as part of requirements gathering is from three[!] years ago, this is not a new idea, and wasn’t then, either.]  I drew the following state transition diagram on the whiteboard, using an active listening technique, to confirm that I had consolidated the inputs into the right set of business requirements.  Documenting business rules with a state transition diagram was very effective at highlighting them.

Most of the explanations from stakeholders were in more of a use-case format, but a state-transition diagram was more effective at validating the rules and requirements in this situation.

The diagram above uses the following approach to describing the problem:

• Some customers are considered to be “actively referred” by an affiliate partner – and when that is true, an affiliate receives compensation for the order placed by the customer.  Other customers are not considered to be “actively referred” and no compensation is due to the affiliate partner.
• Customers can be “temporarily actively referred” and events modify that state.

What the diagram shows, given that context is the following business rules:

• When a customer is referred to the site, the “to be compensated” affiliate partner is identified, and associated with the customer.
• When the same customer is referred to the site by a different affiliate partner, the new affiliate replaces the old one.
• If “N” days pass after a customer was referred by an affiliate partner, that affiliation expires, and the affiliate partner is not compensated for future purchases.
• An affiliate partner is only compensated for the first purchase made by the referred customer.

The stakeholders that were present confirmed that this perfectly represents their business requirements.  Even cooler – a couple days later, the business person most-directly-responsible for implementing the programs came by, looked at the whiteboard, and signed her name on it to record her approval.  We also have a camera-phone snapshot of that.

What We Didn’t Do

I could have titled this article Mini-Model and Requirements, but Agile seemed more appropriate, because we didn’t go over the top with it (and it is a rapid-development, incremental project).  The diagram above also represents the minimum deployable version of the affiliate program for a given release.

We stopped short of creating the following diagram:

This diagram is more of a “classic” artifact that you would see in a BUFR (big, up-front requirements) project.  Our team collaborates regularly, and the affiliate program details were now relevant/timely.  We had the conversations, drew the diagram, confirmed (in about 10 minutes), took a picture, and shared it with the rest of the team.  There would have been marginal incremental value in making a pretty version of the diagram, so we didn’t do it.  Our goal is working software, not comprehensive documentation.

Conclusion

Just do what you need to do, then move on.  For our team, clarifying the intent was important.  Creating a formal, pretty picture added no extra value.  So we didn’t do it.

Satisficing probably makes more sense than perfecting your product.

Can?  Open.

Worms?  Everywhere.

Are we really saying “don’t make it perfect?”  Yup.

Product Perfection

A perfect product will be successfull.  It will solve the right problems, generate as much value as possible, and be so intuitive to use that you don’t even realize you’re using it.

Perfection is a bit of a catch-22, however.  When I was still working as a mechanical engineer, I joined a team who introduced me to my new role with the following statement about the product I was going to replace:  “the design engineers are really good, they designed a great <product> – the only problem is that we can’t build it.”

My first thought – “you’re not using the same definition of great that I use.”  The same holds true for the perfect product.  Perfect at what cost?  If you spend too much making the product otherwise perfect, you won’t make enough money selling them, or your customers won’t benefit enough from using them, so you won’t sell as many, so your profits will be lower, so …, so …, and so on.

You can aim towards perfection, and still probably be ok, because you’ll never get there.  Your ideal product is a moving target, because your market is moving.  Since you’re prioritizing the most important capabilities to build at any given point in time, you’ll never reach the end state of having done “everything.”

There was a math problem that made my head hurt when I was in college.  It went something like this:  If you were 100 feet from your house, and every hour cut the distance by half (after one hour, you were 50 feet away, after another hour – 25 feet, etc), and it was going to start raining in 10 hours, would you be inside before it started?

What if it wasn’t going to rain for 10 days?  Or 10 years?

That may be why I became an engineer, and not a mathematician.  To me, at some point, you would simply take one reasonably sized step, and be in the house.  An engineer would be inside the house before it started raining – and he’d probably be fixing (or breaking) something.  A mathematician would never make it into the house.  I could never appreciate that kind of math.  But I like fixing things.

Every capability you add, or every problem you solve, gets you closer to perfection.  The problem is, it doesn’t get you “10 feet closer”, it is more like “30% closer.”  And like the mathematician, you’ll never get there.

Even the engineers won’t get there, however.  Every solution you deliver to your customers changes the definition of perfection.  Each solution enables new opportunities, and exposes new challenges.  Perfection becomes a moving target.

Product Paralysis

People talk about analysis paralysis fairly often.  Product paralysis comes when you spend so many cycles agonizing over the perfect thing to do, or the perfect thing to do next, that you don’t do anything.  Its the product management version.

I was on a call today, introducing some users to a product we’re developing.  We just stood up a website release – the first sprint that has enough content to be independently consumable by non-software folks.  One of the folks on the call, when trying to explain how agile works, and how user acceptance testing would work, made a statement that I thought failed to capture the essence of what we were doing (in a language that the audience could appreciate).  Paraphrased, the statement was – “in an agile project, there is no ‘release date’, the website will be done when it is done.”

I interrupted to share how I think about this project, and agile in general – “In this project, we’re building a release for you every two weeks.  Every two weeks, you get to decide if it is ‘enough’ to share with customers, or share with more customers.”  [Note: In that statement, "customers" are people who use the website to purchase products from our client, who is also our customer.]  This project involves replacing a legacy system, as quickly as makes sense.  The new solution introduces new capabilities (and value) relative to the existing solution.  Today, it also lacks many capabilities (and value) provided by the current solution.

The question we are working with the business to answer is “when do we switch from the old system to the new one?”  There are at least some people who insist that we cannot switch over until all of the current capabilities are live in the new solution.  If we use that metric (and we may – that has not been decided yet), there will be a period of time where the new solution is more valuable than the current one, but that value will go unrealized.  I think that is another form of product paralysis.  We won’t have perfect data, so it will be a qualitative decision.  Right now, we’re trying to establish an alternate view of “enough”, and then predicting how long that will take, to manage expectations with stakeholders.  Fun stuff.

Sprint Satisficing

Ivan Chalif just published a review of the book, Don’t Make Me Think, by Steve Krug.  In his review, Ivan points out one of the themes in Krug’s book – doing “enough.”

Many choices that a Product Manager has to make are based on having to balance between the optimal solution and what can be done in a given time frame with the available resources. We bet that if we satisfice, it will be enough to address the short-term requirement or at least buy some time to refine the feature in a later release.

That made me remember another book, The Paradox of Choice, by Barry Schwartz.  Another good book, where a central theme is that when given an arbitrarily large set of choices, people don’t choose optimally – they focus on eliminating choices, then choosing the best from a select few options.  If I remember correctly, Schwartz argues that people don’t optimize when faced with too many choices, they satisfice.

When it comes to determining what to include in a particular sprint, we’re faced with an arbitrarily large set of choices, so we prune.  And then we satisfice.

In another conversation today, one of the developers asked an insightful question about one of the stories that was committed for the sprint.  As part of answering that question, we realized that the “perfect” answer was to change the design approach for that story, resulting in extra work that would push the team beyond capacity for the sprint.  We have a theme for the sprint, and this story is a key element in that theme.  If we have to push something to a later sprint, we want to push something else.

The story was written for three classes of users – logged in users with existing accounts, people who just created their accounts, and people who didn’t have (or want) accounts.  The project involves integrating COTS (customized, off-the-shelf) software with legacy systems.  We discovered that the design changes would only be needed for people who did not have accounts.  The theme of this sprint is to enable an end-to-end path through the site (essentially, completing an epic).  We decided that we would be satisfied by completing the epic for the logged-in users in this sprint, and add the other users in the next sprint.  That met our tactical goals for completing the epic.  Our alternative would have been to delay another story (or two) to a later sprint, just to be “perfect” with our epic.  This is the better solution, in this case.

What made this possible was that we remembered that each sprint asks the question about the solution – “is it good enough?”  And by implication, each sprint asks the question about each story – “is this approach good enough?”

We decided that today landed in the “a good day” column on the ledger.