The ideal agile team is made up of specializing generalists – but what does that really mean?  The goal isn’t to prevent functional silos of expertise, it is to allow people to cover for each other.

Great Conversation

Elena Yatzeck (@eyatzeck) posted a comment on an earlier article about agile maturity models.

In terms of refinement, I’m thinking a lot these days about “staffing the engineering team correctly.” I’m not sure I agree in practice that you can or should try to staff all teams with “specializing generalists,” or at least not as taken to an extreme. (If you’ll forgive the self-promotion, I talked more about this here: http://pagilista.blogspot.com/2012/01/no-blender-zone-cross-functional-doesnt.html.)

I’ll not only “forgive” the promotion, I’ll re-promote it.  Good stuff.

When re-reading the maturity-model article, this snippet popped out at me:

People over process is the right emphasis. If you can’t find people that are “good enough” you might as well go home. Doesn’t matter how agile you are if you don’t have the horsepower. You also need people who are excited to “do agile” – they like to communicate, they enjoy the project and team dynamics of an agile process. You’re also better off with specializing generalists – ideally, every member of the team can do any work that is needed. This is an efficiency play – you risk introducing bottlenecks when you have a specialist who is the “only one” who can do particular types of work – because you will not have a consistent mix of types of work from release to release.
Agile Maturity Model

Thirty months later, my experiences have increased my conviction that this is true – and have realized that the way I wrote the quote above fails to provide a key clarification.

Following that link to an (even earlier) article on specializing generalists, brings the following (emphasis added):

The idea of specializing generalists is easiest to grasp by first saying what it is not. It is not staffing a team with a database expert, a user interface coder, a SOA (service oriented architecture) guru and an architect. With four specialists, each development task has an obvious owner. Database changes and refactoring go to the database expert. Reworking the UI goes into the queue for the AJAX hotshot. The problem is that this approach is only efficient when each team member is equally loaded with work. Since an agile team is continuously reprioritizing their work based on repeated feedback cycles as part of each release, this doesn’t work. The team will never face a situation where the (for example) four most important things to do are one item for each specialist. You can very easily have a release where all of the most important tasks are focused on the user interface. So all of the non-interface-experts are either working on lower-priority tasks, or even worse – they are idle. And you delay the most important work until the specialist can get to it.

By staffing a team with people who have an area of expertise, but can do anything, you can maximize the value of each delivery cycle. In our example, where all of the tasks for a release are UI tasks, they can be interchangeably assigned to any of the developers. The UI expert may suggest an implementation approach, do code reviews, or provide guidance to all the other developers. But every developer (including the database guy) can sling code effectively to get the job done. Specializing generalists.

This is very effective for making the “development engine” a black-box. Feed it the highest priority stuff, and it all gets done. We can take that approach to the next level. Designers can implement, project managers can design test plans, and yes, product managers can specify design. Twitch. Back up a sentence and read it again.

Specifying design is not the job of the product manager. True. Very true. Emphatically true. But specifying design can be what a specializing generalist does, even when that person is also responsible for defining market needs.
Specializing Generalists 2008

Elena’s article identifies a common misconception – that “specializing generalist” is a fancy way of saying “a bunch of people who can all do everything:

It’s a seductively simple fallacy of division to interpret the concept of “cross functional” team to mean a “collection of cross-functional individuals.” New agilists are quick to apologize that “we still have functional silos here” as though it would be much better if everyone could do all the same things. Grab some equally skilled poly-functional people, have them all take turns doing all of the jobs as needed, and you’ll all laugh your way to on-time, high-quality, and valuable working software.

Not so fast!

The power of an effective agile team, like the power of any other effective team, doesn’t come from its homogeneity, but from its ability to harness its diversity.
No Blender Zone: Cross Functional Doesn’t Mean Homogenous

Elena goes on to say (emphasis mine)

Team members shouldn’t attempt to Harrison Bergeron themselves into a mish-mash of mediocre (but working!) software. Someone needs to facilitate the stakeholders into some sensible semblance of a business case. Someone needs to build functional test suites that mercilessly beat on the code to prevent it from breaking in production. Neither of these are exactly the same skills it takes to gradually evolve the design of a complex system in modules of 100 lines of code or less. If people want to try new things, that’s great, but it needs to be with the realization that other jobs on the team are actual professions with skills and the need for experience in order to excel.

I completely agree.

Specializing Generalist

Specializing Generalist.

• Not a specialist.
• Not a generalist.

You need best of breed team members who specialize in areas of experise – “actual professions with skills,” as Elena puts it.  Without people who excel in the needed areas, you end up with a mediocre product.  How many times have you gone to the store and asked for the “middle of the pack” product?

That’s not even table stakes anymore.  Just the ability to create “something” isn’t interesting in the market, and isn’t interesting to the members of the team.  How many times have you heard someone brag “I love my job, I’m a cog in the machine?”  You have to have people who specialize in all of the needed areas (interface design, market insight, coding, quality, etc) in order to create a viable product.

If you staff your team with (only) generalists you will fail.

Pure generalists cannot create a product that is “good enough” – because they aren’t good enough at the creating the parts, from which the product is the sum.  You have to have people who specialize in creating great “parts” of the solution.  That’s what you need to have a shot at creating a great product.  But it isn’t really enough.  The problem is in how you define “great.”  Great means that customers buy it, users love it, and your competition is knocked back on their heels by it. Everyone agrees on this, but most people miss one thing.

Your market is changing – you also have to be fast.  You can’t solve the right problems if you aren’t fast, because the problems that are “right” are constantly changing – your market is a moving target.

Specialists, as individuals, are capable of creating great “parts” in their silos, and those parts all add up to a “great” product, so what’s the problem?  The problem is that collectively, by the time the specialists are done, they are no longer solving the right problem.

If you staff your team with (pure) specialists you will fail.

The most important tasks for the team, in any given sprint, will not balance into a perfectly allocated workload, where each “part” is worked on by each specialist, where no one is idle, and no one is a bottleneck.  It just doesn’t happen.  I haven’t seen it in 15 years in the software world, or in my prior decade as a mechanical engineer.

When one specialist is waiting for something important, she isn’t idle, she’s just working on something that is by definition not as important.  OK, you’re minimizing the damage – but you’re still taking damage.  When another specialist is the bottleneck, you lose.  Nothing magical to do here.

If you staff your team with specializing generalists you may succeed.

The work that piles up in any one specialized silo is of varying degrees of complexity.  The “UI specialist” may be backed up with a bunch of CSS tweaks, some straightforward AJAX calls to write, and a gnarly refactoring of the model-view-controller model to adapt to changing understanding of market needs.  No one can solve the MVC problem without specialized skills – but with guidance from the UI expert, one of  the other team members can handle the AJAX calls and CSS updates.  Extend this same model across other aspects of the product.  Your database expert may be needed to optimize query performance or resolve locking problems, but other members of the team could make straightforward schema changes.

It is the collective ability of the team to optimize what they collectively work on that accelerates the team’s delivery of the most important capabilities.

You have to have people who specialize, in order to optimize individual performance.  But your team needs to be built with specializing generalists in order to optimize for team performance.

T-Shaped People

From an HR perspective, I was taught about “T-Shaped People” – people who have breadth and depth of skills.

• Specialists are “I-Shaped People” – people who have depth of expertise, without breadth
• Generalists are “Minus-Shaped People” – people who have a breadth of skills, but no depth of expertise.
• Specializing Generalists are “T-Shaped People” – people who have depth of expertise in one area, combined with a breadth of skills across many areas.

These are the people you’re going for.

Thanks Elena for re-invigorating the discussion!

Your boss wants a commitment.  You want to offer a prediction.  Agile, you say, only allows you to estimate and predict – not to commit.  ”Horse-hockey!” your boss exclaims, “I want one throat to choke, and it will be yours if you don’t make a commitment and meet it.”  There’s a way to keep yourself off the corporate gallows – estimate, predict, and commit – using agile principles.

This is an article about agile product management and release planning.

Change and Uncertainty

In the dark ages before your team became agile, you would make estimates and commitments.  You never exactly met your commitments, and no one really noticed.  That was how the game was played.  You made a commitment, everyone knew it would be wrong, but they expected it anyway.  Maybe your boss handicapped your commitment, removing scope, lowering expectations, padding the schedule.  Heck, that’s been the recipe for success since they planned the pyramids.

It makes sense.

1. Your early estimates are wrong. When you add them up, the total will be wrong.  If you do PERT estimation, the law of large numbers will help you in aggregate.  But you’ll still be wrong.
2. The outside demands on, and availability of, your people will change. Unplanned sick time, attrition, levels of commitment over time, lots of “people stuff” is really unknown.
3. The needs of your customers will change. Markets evolve over time.  You get smarter, your competitors get better, your customer’s expectations change.

Agile processes are designed to help you deliver what your customer actually needs, not what was originally asked for. Contrast the two worlds.

In the old world, you would commit to delivering a couple pyramids. After spending double your budget, with double the project duration, you would have delivered one pyramid. When you deliver it, you find out that sphinxes are all the rage. Oops.

Your team changed to agile, so that you could deliver the sphinx. But your Pharaoh still wants a commitment to deliver a couple pyramids (the smart ones will be expecting to get just one).  You can stay true to agile, and still mollify your boss’ need to have a commitment, if you take advantage of the first-principles of why agile estimation works.

Estimation

A commitment is a factual prediction of the future.  ”This will take two weeks.”  Nobody is prescient.

A factual prediction has to be nuanced.  ”I expect* this will take no more than two weeks.”

*in reality, this is shorthand for a mathematical prediction, such as “I expect, with 95% confidence, that this will take no more than two weeks.”

Few non-scientist, non-engineers, non-mathematicians understand that 95% confidence has a precise meaning.  People usually interpret it to mean “a 5% chance that it will take more than two weeks.”  What it really means is that if this exact same task were performed twenty thousand times (in a hypothetical world, of course), then nineteen thousand of those times, it would be completed in under two weeks – do you feel lucky?

To make a statement like this, you actually have to create a PERT estimate – identifying the best-case, worst-case, and most-likely case for how long a task will take.

Unfortunately, we’re rarely asked to make a commitment about a single task – but rather a large collection of tasks – well-defined, ill-defined, and undefined.

You can combine PERT estimates for the individual tasks, resulting in an overall estimate of the collection of tasks.

The beauty of this approach is that the central limit theorem, and the law of large numbers, work to help you estimate a collection of tasks – you can actually provide better estimates of a group of tasks than a single task.  This obviously helps with the well-defined tasks that you know about at the start of the project.  This even helps with the ill-defined tasks.  Rationalists will argue that the key, then, is to do more up-front research to discover the undefined tasks – and then we’re set.  As Frederick Brooks (Mythical Man-Month) points out in The Design of Design, this debate has been going on since Descartes and Locke.  It is not a new idea.

Big Up-Front Design and Requirements (BUFD & BUFR) hasn’t worked particularly well, so far.

Don’t throw out the baby with the bath-water, however.  The math of estimation is still important and useful.  Even if empiricism is not the silver bullet.

Prediction

Estimation is a form of prediction.  Even agile teams do it.  In Scrum, you estimate a collection of user stories – in story points that represent complexity, and you predict how many points the team can complete in this sprint.  Note the time factor.  If you’re working a two-week sprint, there is very little risk of changes in staffing during a two-week period.  There’s also very little risk that your market will change significantly in two weeks – and if it does, what are the odds that you will notice and materially change your requirements in two weeks?

Visually, let’s take that PERT estimate and turn it sideways – so we can introduce the dimension of time.  Imagine you estimated all of the tasks (well-defined, ill-defined, and a guess about the undefined), as if they were all to happen in the first sprint.  Ignore inter-task dependencies, and pretend you had unlimited resources and the ability to perform all tasks in parallel.

The graph above shows the aggregate estimate – the circle is your best prediction, with error bars representing your confidence interval in the estimate.  If you were using PERT estimates, these could represent that 5% and 95% confidence lines.  Subjectively pick something based on your team’s experience in the domain and your confidence in your guesses (about the undefined tasks).

We need a segue into the “best of waterfall” approach to estimating projects, to steal and invert a good idea.

The Cone of Uncertainty

The folks at Construx have published a nice explanation of the cone of uncertainty – an adaptation of an idea from Steven McConnell’s Software Estimation: Demystifying The Black Art (2006).  That article uses his imagery with permission – so please go look at it there.  The idea is that as the project becomes better defined (e.g. during the project), the amount of uncertainty is reduced.

The findings show that initial estimates are off by 400% (either low by a factor of 4 or high by a factor of 4)!  Even after “nailing down” requirements, estimates are still off by 30% to 50%!

As bad as that sounds, it is actually worse.  This is a prediction for the original project (delivering pyramids).  Not only are your estimates wrong – but they are bad estimates for delivering the wrong product.

But – the core idea is sound – the further into the future you have to execute, the greater the mistakes in your estimate.

Taking that concept, and applying it to our diagram, we get the following:

The further into the future you are trying to predict, the less accuracy you have in your prediction.  This reduction in accuracy is reflected as a widening of the confidence bands for your estimate.

• A couple sprints’ worth of work is not much different than one sprint – so your estimation range is not much changed.
• An entire release of sprints (say 6 to 10 sprints) has much more opportunity for the unknown to rear its head.

Now, your prediction is (probably) unusably vague and imprecise.  ”This set of tasks will take X plus or minus a factor of two.”

That’s the reality.

Note: This has always been the reality.  People have historically reduced this “risk to timing” by hiding the “risk of change” aspects – and waterfall processes encourage you to deliver the wrong thing, as close to on-time as possible.

That’s not what we want to do, however.

We still want to deliver the (not-yet-defined) right product, as efficiently as possible.  That’s the goal of agile.  (For folks who haven’t been here at Tyner Blain for long – “right” includes both value and quality).

Refinement

Because we’re agile, and we’re willing to “get smarter” about our product over time, we have an opportunity to improve.  Because of the nature of compounding estimates and the cone of uncertainty, our uncertainty gets smaller over time.

Let’s remove our artificial simplification that we could do everything “right now” and look at what we think we know right now, about the end of the release.

Our ability to predict the amount of effort (for today’s definition of the product) at the end of the release is not very good.

Our ability to predict (today’s definition of the product) one sprint into the future is much better.

After completing the first sprint, we are a little bit smarter – the ill-defined tasks are better defined.  Maybe some of the undefined tasks are now ill-defined.  The same cone of uncertainty is now a little bit smaller – we are a little bit smarter, and the time horizon of the release date is a little bit closer.

The trend continues – each sprint gets us closer to the release date, and with each sprint (assuming we get feedback from our customers, and continue to study our markets) we get a little bit smarter.  We also get better at predicting the team’s velocity (how much “product” they can deliver during each sprint).

Commitment

Your boss still wants a commitment, however.  And that’s where we get to change the way we look at this (again).

The above diagrams all display how we converge on an estimate for a stable body of work.  However, we know that the body of work is constantly changing.

Backlog! [you say]

Yes!  The backlog.  The backlog is an ordered, prioritized list of user stories and bugs.  I was talking with Luke Hohmann of Innovation Games last month, and one of the most popular online Innovation Games is now the one they created based on prioritizing by bang for the buck.  Play it today online (for free!).  How cool is that?

The backlog represents the work the team is going to do – in the order in which the team is going to do it.  Over time, as we get smarter, we will add and remove items from the backlog – because we discover new capabilities that are important, and because we learn that some things aren’t worth doing.  We will even re-order the backlog as we recognize shifting priorities in the markets (or in our changing strategy).

As this happens, it turns out that the items at the top of the list are least likely to get displaced, and therefore most likely to still be part of the product by the time we get to the release.

Instead of thinking about uncertainty in terms of how long it takes, think about uncertainty in terms of how much we complete in a fixed amount of time. In agile, generally, we apply a timebox approach to determining what gets built.

Now, uncertainty, instead of manifesting as “when do we finish?” becomes “what will we finish?”

Your boss is rational.  She appreciates the constraints, she just wants to know what you can commit.  Every boss I’ve worked with has been willing (sometimes only after much discussion) to treat this uncertainty in terms of what instead of when.  They acknowledge that they need to translate (usually for their boss) into a “fixed” commitment.

The solution: commit to a subset of what you predict you can complete.

At the start of the release, you may have 500 points worth of stories.  Based on your team’s expected velocity, and the number of sprints in the release, you predict that you can complete 320 points worth of stories (5 people on the team, a team velocity of 40 points per sprint, and 8 sprints in the release).  Starting at the top of the backlog and working down, draw a cut-line at the last story you can complete (when you reach 320 points).  This is your prediction.

Now the commitment part.  You’ll have to figure out what you’re comfortable with.  Maybe for 8 sprints (say, 16 weeks into the future), you may only be comfortable committing to half that amount – 160 points.  Go back to the top of the backlog, and count down until you reach 160 points.  Everything above the line is what you commit to delivering.

Maybe you are comfortable committing to 240 points, maybe only 80.  This is like playing spades.  The more you can commit to, without missing, the better off you are.  Your tolerance for risk is different than mine.

You can also negotiate with your boss.  Commit to 160 points now, and provide an update after every other sprint.  More likely than not, you will be increasing the scope of your commitment with every update.

Mid-project updates of “we can do more” are always better than “we can do less.”  And both are better than end-of-project surprises.  This also allows you to have updates that look like this:

We didn’t know this at the start of the release, but X is really important to our customers – and we will be able to deliver X in addition to what we already committed.  Without slipping the release date.

Conclusion

Making commitments with an agile process is not impossible.  It just needs to be approached differently (if you want to stay true to agile).  The end result: better predictions, more realistic commitments, and the likelihood that each update will be good news instead of bad.

[Update: Changed initial image.  Thanks Dennis for the great photo!]

Agile values working software over comprehensive documentation – it is 1/4th of the original manifesto.  That doesn’t mean don’t document!  It means don’t document more than you need to document.  Documentation does have value, but the practice of documenting got excessive – that’s why a reaction to the bad stuff earned a spot as one of the pillars of agile.  How do you avoid over-reacting when changing a culture of over-documentation?

The Need for Documentation

We need documentation to help us communicate.  You can define an agile team as “the people creating the product.”  You can interpret “creating” as the people we normally think of as executing to accomplish the goals, or you can be more inclusive, and think of the people who have the goals as being part of the team too.

Philosophically, agile stresses collaboration.  Usually it is couched in terms of (1) people who are executing collaborate to devise the best solutions and (2) people who are part of the team collaborate with the people for whom they are creating the product.  Personally, my experience has been that projects with committed sponsors succeed, and projects without them fail – so I always think of the sponsors as part of the team.

Regardless of your definition – collaboration requires communication, and communication benefits from documentation.

Temporal Communication

Communication among groups of people happens over time.

Some collaboration is transient – communication happens right now, and is only important right now.  Other communications are persistent – the collaboration happens right now, but we need to remember later what we agreed upon and why.  There are variations of “right now” and varying degrees of “later”, but slightly oversimplifying,

• There is communication for now.
• There is communication for later.

Both are important.

Documentation, in the vision from your nightmares – giant requirements documents, architectural analyses, and detailed psychographic profiles – is very inefficient when communicating for now.

These massive documents, while getting in the way of a conversation, do provide the opportunity to remember (in the future) why we make the decisions we make (today).

Documentation, as emphasized in most agile discussions – user stories and acceptance criteria, on an index card, taped on the wall – is not a robust solution for communication that happens later.

This low-overhead approach to capturing today’s ideas actually facilitates and improves today’s conversation – but does not give us a record we can use in the future for why we made our decisions.

As an agile team, there are ways we can approach both the persistent and transient documentation tactics to make them more valuable.  We can tweak our transient documents to make them more useful for persistence.  We can take an agile approach to development of persistent documents, so that we only do just enough documentation – trading off the minimum amount of short term efficiency to avoid long-term blunders.

Transient Documents for the Future

The image above is from a workshop defining the product backlog for an initial sprint for a new project.  The team identified the most important stories for the first sprint, their acceptance criteria, and the size, in story points, for each story.  As part of scrum, the stories are written in user-story format, with acceptance criteria for each story identified and documented on additional index cards (taped to the index cards for the associated stories).  Great transient way to capture the must do and makes it better stories that the product owner is thinking about.

As a team, we made a couple minor tweaks to make these story cards more useful.  First, all of the stories were organized into themes that help establish the context (for the team), and combine the stories into “meaningful areas of focus” for communication with stakeholders.

Each story got a blob of color in the top left hand corner, identifying which of the five themes were being supported.  You can see a few of them circled in the image above.

We also developed a set of 8 proto-personas (prototype, early draft, very rough personas – almost stereotypes) that the overall product / project was designed to support.  Each persona got a different color star.

Each user story got a corresponding colored-star sticker to indicate for which persona the story has been written.  Specifically, when multiple personas could use a story, we identified for which persona we were implementing the story in this sprint.

In the “right now” world, these tweaks helped us understand, question, and confirm the themes and personas being supported in the first story.  Those conversations allowed us to reach pragmatic compromises about which stakeholders were going to benefit and when.  They also allowed us to craft the outbound message that is step one of stakeholder expectation management.

The list of themes changed while we were doing this – moving from 4 to 5 themes, as we realized that there was value in splitting one of the themes.  The group of proto-personas grew from an initial set of 3 to a set of 8, as we quickly realized that the team was trying to “build something for everyone” and we wanted to avoid the elastic user problem, and instead, roll out capabilities sequentially that satisfied valuable groups of users with similar goals and perspectives.

Culture Change for Stakeholders

As you would expect, having a bunch of cards stuck on the wall provides a great, visceral, now communication for the team – both in implementing and communicating.  However, cards on the wall is very transient.  It also doesn’t provide a good way to communicate with old-school stakeholders (imagine the execs who have their admins print out their emails for them to read and annotate – those folks still exist!).

This can make it hard for the “not part of the implementation” part of the team to collaborate and communicate.

In User Stories Applied, Mike Cohn stresses that the brevity of user stories is intentionally designed to facilitate (and I would add “dependent upon”) conversation.  I find it to be both a strength and a weakness of user stories.  It is strong because you can cover a lot of ground quickly (breadth) and capture a number of stories, much like the list above.  It is weak because the requirements documentation does not stand on its own – it requires conversation to fill in the details.  I’ve had some success using the “Verify that…” user acceptance tests as the method of documenting those details (depth), in conjunction with the brief, easily consumable stories.

Stakeholders in a Barrel

Those conversations happened as part of getting the stories defined, getting them on the board in the right order, and defining the acceptance criteria.  Stakeholders often have a short memory, especially when they were the ones making a compromise.  You need some way to retain the context of the conversation that led to the particular organization and content of the stories on the wall.

Persistent Documents in the Present

Agile proponents complain about the massive get-in-your-way documents that slow down the start of projects, prevent the teams from adapting to changing market conditions, and otherwise make it harder to deliver great products.  I agree.  Agile proponents also rail against the “build it all, then release it,” waterfall, process for creating software – and propose an alternative – iterative development of the software.  I agree again.

Why not apply the same principle of iterative development to persistent documents?

Consider the example above of using colored-star stickers to identify the people for whom each story is being implemented in any given sprint.  The majority of the time spent on this analysis is around understanding which people are important (to the success of the product), and for which people the ability to perform this user story is important (to the person).  The combination of these two importances is an input to prioritization.  A minimal amount of time is spent putting stickers on cards, and creating multiple cards for the same story (each with a different sticker for a different persona and potentially different acceptance criteria).

Add a tiny bit of overhead, and capture the information in a spreadsheet (or whatever).

The image above shows a mapping of use cases to actors.  It could just as easily show a mapping of user stories to personas.  The transition from thinking about actors to thinking about personas is a small one.  This chart comes from an earlier article on communicating a delivery schedule with use cases – it helps stakeholders get a big picture view of who benefits when – a user-centric, yet still top-down perspective.

Note: Another nuance to incremental delivery is that you can introduce the “bare bones” version of a story now, and the “better” version of it later.  You can also communicate that stories get better over time, for some users, with the same type of chart.

A very similar, and also simple to create table can show how each theme is getting “representation” within each sprint.  That view facilitates great discussions about trying to emphasize one theme and then another sequentially, versus doing the most important items in each theme first – gradually making progress in each theme.  Creating that view, and using it to communicate, has helped me address concerns that “external” stakeholders have shared about how a team appeared to be “chaotic and random” in their approach to prioritization and implementation.

Many Models

There are many types of documentation – and many types of models for emphasizing and discussing particular aspects of a complex system (like users and goals and solutions).  Creating a simple diagram like the following whiteboard sketch is almost required for effective transient communication in some projects.

[from Simple Agile Model Example]

When most folks are complaining about documentation, they are not complaining about the sketch above.  They would consider that sketch to be augmenting a conversation.  Fine.  That’s what I did, at the time.  Then I took a picture of it.  Suddenly, it was documentation.  A stakeholder came by later, confirmed that the system needed to work this way, and signed the whiteboard – and the photo was updated.

Every heavyweight document can start out this way.  And it can evolve.  It should evolve.  If you’re going to succeed, it must evolve as your team gets smarter, your competitors react, and your market changes.

The trick is that you don’t have to write the final version before you get started.  Capture at a high level what you know right now – just like a roadmap.  Capture in just enough detail what you need right now – just like a story backlog.  And change it as you go.

Conclusion

Documentation is not bad.  Documenting stuff you will never use is bad.  Documenting stuff you don’t need for a long time is risky, because it will probably change before you refer back to your document.

Documenting why you made decisions right now, as part of transient collaboration and communication is important, so that you collectively remember.  That documentation persists and your team can build upon knowledge, incrementally – just as your team builds upon the evolving code base, incrementally.

Best practices for user experience design and agile.  I don’t have the brainpower at the moment, or the experience and eloquence in general, to say it better than these guys.  So this week, I’m phoning it in, and deferring to these folks to say it far better than I can.

UXD and Agile

Twelve emerging best practices for adding UX work to Agile development, by Jeff Patton in 2008.  Incredibly comprehensive article here – both summarizing in breadth, and covering in depth.  If you’re interested in how user experience design (UXD) should work in an agile environment, this is the article you need to read.  If you’re really interested in UXD and agile – this is where you need to start!

Scrum and UX – a presentation by Morten Just from 2009.  Morten tells a great story about how many of the different disciplines within the “catchall” of UXD can fit in the world of projects being run in the Scrum framework.

My soundbite addition:

I feel like UXD is such a broad term that you can’t simply answer “how does UXD work in Scrum?” (a question I was asked today).  Some elements (outputs traditionally labeled as UXD) represent requirements (what should people be doing, what should some of the acceptance criteria be), and others represent design (how should things work / look / feel / exist, given a solution approach).

Every team will be organized differently, and parts of UXD that are “requirements” should be driven – in Scrum – by the product owner and UXD professional working together, expressing the need to enable the right stories with “good (big picture) design.”  The parts of UXD that are “design” should be handled inside the team, and the product owner (and doubly-so the stakeholders) should rely on their trust of the Scrum team to include “good UXD” as part of their design of solutions that enable stories.

As I mentioned in the pre-amble, Jeff and Morten do a much better job of articulating this point of view, so this article is 99% “go read their stuff” and 1% “my mental model that some of it should happen as part of expressing what needs to be done, and some of it should happen as expressing how it should be done.”

A picture is worth a thousand words.  A prototype is worth a thousand lines of code.  Two key elements of product management – and of agile development are elicitation and feedback.  Low fidelity artifacts can significantly improve both.  Polished, codified prototypes can create problems that prevent you from getting the benefits of communication.

Prototyping Anti-Pattern

David Bernstein has three good quick-read articles about prototyping in the agile zone.  The first one depicts the primary anti-pattern of prototyping – mis-set expectations.

As I was walking him through the different screens I could see he was starting to get uneasy. As I talked I could see him becoming paler and tenser, as if he saw a ghost. After a while I asked him if he was ok. He finally said, “You mean to tell me that I just wrote you a check for over $100,000 for less than a week of work?”

Prototyping Caveat

The second article is quick reminder of one of the goals of a prototype.

The purpose of a prototype is to elicit feedback from others or verify a design approach will work. When doing this we generally only concern ourselves with the “happy path” through our code.

A Prototype is Not a Product

David’s third article explores a little more about the mis-setting of expectations, and provides a good parallel from the film industry.

Prototypes are rough sketches without the robustness of a final product and it is often confusing to users when we show them a half-baked version of their project for feedback, even though they know it is not yet finished.

Agile Prototyping on a Napkin

David’s advice is (good and) primarily about using low-fidelity prototypes to avoid disrupting the elicitation and feedback process.  Folks in the user experience community have known this for a long time, and adapted their processes accordingly.  Back in 2006, I wrote about prototype fidelity as part of exploring ways to use this insight in requirements gathering.  Prototyping, in particular, is a great way to elicit implicit requirements – those unspoken (“you should have asked!”) requirements that your customer or stakeholder assumed you knew, or didn’t think to tell you.

Multiple Levels of Interaction

David focused on the initial “will this approach work?” elements of getting feedback on a design – and by inference, some low-fidelity user acceptance testing that the proposed approach will meet your customer’s requirements.  Jan Miksovsky wrote an excellent article (also in 2006) that provides excellent guidance on how much fidelity to build into your prototype, depending on where you are in the design process.  This is important – the type of feedback you need at different stages of your design process varies.  Jan proposes that the first prototypes be rough sketches, just as David points out.  Jan goes on to show how and when to add additional fidelity to your prototypes (read Jan’s article to see his great visual examples).  Like I said, the user experience community has known about this for a long time.

Prototypes Are For Two-Way Communication

The key element, and the reason for creating prototypes, is to get two-way communication.  A prototype is not just a status update about your design.

A prototype is the start of a conversation.

[thanks uninen for the image]

Prototyping is not only useful for design conversations, it is critical to understanding your requirements.  Yes, a prototype will you get feedback about the design-execution of your proposed solution.  More importantly, a prototype can help you make sure you’re solving the right market problems.

Prototypes Are More Than Interface Mock-Ups

The first thing we think about, and everything you’ve read so far in this article, is about getting feedback on the user interface of the product.  Prototypes are also useful for gathering business rules*, understanding system complexity, and defining, clarifying, and validating requirements.

*In that article, I show prototypes as “not being useful” for gathering business rules.  At that time, I was thinking in terms of interface mock-ups only, not other prototypes.

Use interface mock-ups when you need feedback about user interaction.  Use other artifacts when you need feedback about other aspects of your product.  You can use prototypes as an active-listening technique when gathering market data too.  There are lots of ways to draw stuff to help with communication.

I’ve regularly used flow charts to prototype processes.  Within those flow-charts, which are initially prototypes of how the product will behave, are decision diamonds.  Those decision diamonds prototype the enforcement of business rules within the to-be-created product.  In the spirit of DRY (don’t repeat yourself), a quick polishing of your process diagram can serve as a persistent artifact of the requirements.  Just like a user interface mock-up.

In more complicated scenarios, I’ve also used UML statecharts and data flow diagrams to prototype behavior.

In the section on up-front planning in last week’s article on risk management and release cadence, I talked about the mental leap that people need to make to envision a product that doesn’t yet exist.  Prototypes are like big springboards that help people leap that chasm of imagination.

Important Safety Tip

Don’t use the wrong prototype (mock-up, flow chart, etc) for the wrong conversation or with the wrong stakeholder.  Showing when and where business rules are being enforced (in the process being designed) is great for getting feedback about your interpretation and potential embodiment of those rules.  It will be a disaster if you try and have this conversation with someone who is not the owner of those policies – like a representative user.  Sometimes, policies are “owned” by non-technical people who struggle to read diagrams.  You may have to walk them through how the diagram works.  They’re smart – they’ll get it.

Just remember – use the right prototype to emphasize the right ideas, and elicit the right feedback.

I’ve been thinking about the software development process.  Big, upfront, design and requirements.  User research and analysis.  Market insights, gained on exploration or over time.  Release cadence – how quickly you get, and incorporate, feedback from your customers about your product.  How quickly you react to your competitors’ reactions to your actions.

Big and Up-Front

Leander Kahney published the full transcript of an interview with John Sculley, former CEO of Apple, last week.  Of particular interest are the discussions about how Steve Jobs is a designer first, CEO second.

He always looked at things from the perspective of what was the user’s experience going to be? But unlike a lot of people in product marketing in those days, who would go out and do consumer testing, asking people, “What did they want?” Steve didn’t believe in that.

John Sculley interview

The theme of understanding what users want and need comes up repeatedly in the article.  A few years ago, Alan Cooper and Kent Beck got in a debate about the merits of interaction-design-driven versus emergent-design software development.  Four years later, I don’t accept the premise that you have to choose between the two.  You can treat both decisions independently and make the right choice for your team and market.

Thanks Jim Holland (also PM Tribe author), for bringing this article from Wayne Mulligan (guesting on On PM) to our attention today.  It’s a great read, and the money-quote from Wayne about going back to first-principles is:

And that’s the biggest lesson I learned here – you can reap all of the benefits of an agile product development approach without following the rules to the letter.  It’s the spirit of the manifesto that counts and not the precise implementation.

4 Ways that Agile Methods Brought Sanity to My Company

In the spirit of that reality – I believe decoupling how do you plan? from how do you deploy? is fair game.

The Risk of Being (Big Picture) Wrong

[larger image]

When you do no upfront analysis at all, you have complete risk of creating the wrong product.  The more analysis you do, the more likely you are to create a product that solves the right problems – problems that your target customers, in your target market, are willing to pay to solve.  The units in the graphs in this article can be mostly ignored – all you can really capture, when talking generally about this is the trending.  The more time you spend understanding your market, the less understanding you gain.  There is definitely a law of diminishing returns.  This effect is one of the reasons analysis paralysis is so bad.  With analysis paralysis, you delay the launch of your product (at some cost), for no appreciable gain.

The two curves are labeled complex market, and simple market.  The idea here is that the easier it is to understand your market, the less benefit you get from analyzing it.  Complexity can be in the form of competitive concentration, nuances of problems being solved, specifics of the customers for whom you are solving the problem, and even the notion of just how innovative your solution is (think cars, when horses were common, or smart phones in a world of land-lines).

As a product manager, you have to act with imperfect information.  Even if you try and uncover every relevant piece of information (which you can’t do), you will misunderstand some things.  You will develop a mental model of your customers’ problems – and that model will be wrong.  You will also predict competitive responses incorrectly.  To top it off, you’ll make mistakes about “what’s next?” for your customers.

That does not mean you should do no analysis – it means you should not do too much analysis.

Not all product managers are equally good at distilling insights from the vapor of nuanced market data.  Great product managers will have (correct) epiphanies about the important problems to solve, and the impacts of disruption on the market.  Others will not.  The better you are at having these insights, the faster you reduce risk – and the faster you reach the point of diminishing returns of additional analysis.  Some product managers and markets may require weeks of analysis, others may require days.

The Risk of Being (Execution) Wrong

Another great quote from the Sculley interview:

[Jobs] said, “How can I possibly ask somebody what a graphics-based computer ought to be when they have no idea what a graphic based computer is? No one has ever seen one before.” He believed that showing someone a calculator, for example, would not give them any indication as to where the computer was going to go because it was just too big a leap.

John Sculley

In The Design of Design, Frederick Brooks discusses the debate between empiricists and rationalists.  Empiricists, in product creation, emphasize feedback as the means of gaining insight about what your product should be.  Rationalists use intuition and deduction to presuppose what a product should be.  In the hyperbole of many agile-waterfall debates, empiricists are burdened with the impossibility of designing the right product in advance, while rationalists struggle under the yoke of being wrong from the start and wasting their efforts up until the first failed product release.

Obviously, neither of these extreme perspectives is wholly true – while both have significant elements of truth.

True insights about your market grow stale with time.  The damage from erroneous conclusions about your market grows over time.  The risk of creating the wrong product grows over time.

[larger image]

If you released continuously, including getting feedback continuously, you would face the minimal possible risk that the market has changed since last you checked.  You would also maximize the opportunities to correct mistaken conclusions – thus minimizing the risk of being wrong.

You reduce the size of this risk by getting feedback.  You get feedback by “releasing” your product to customers, either as a prototype or as a product.  Jobs (per Mr. Sculley) realizes the catch-22 of not being able to get inputs from others, until they can realize a version of your idea. Sculley attributes the following to Mr. Jobs: “There was no way to do consumer research on it so I had to go and create it and then show it to people and say now what do you think?“

The longer you wait to get feedback, the greater the cost of effort invested in the wrong ideas.  The longer you wait to get feedback, the greater the risk that your market has changed since you formed your ideas.

In stagnant markets, where problems and their solutions are well understood, and change happens slowly and competitors don’t innovate, you can release less frequently with less risk.  Until someone decides to innovate, and make your market more dynamic.  Markets with active competitors, customers who’s expectations change, and problems that evolve are more dynamic.  The risk of delays is greater.

Rationalism and Empiricism Can Co-Exist

My experience has been that both concepts – “think before you act” and “fail fast, get feedback” can happily co-exist.

• You can develop an understanding of your market, your customers, and the very real problems they face, before creating a product that solves those problems.
• Introducing a new product / solution into a market changes that market – so you have to revisit what you know at least as frequently as your market changes.
• Even when you understand the problem, you need feedback to know if the product is actually the solution.  The greater the “leap,” the more representative your prototype must be for people to give you feedback about the (eventual) product.

What have your experiences been?

Almost everything I’ve read about use cases focuses on describing what needs to be added to your product.  Agile development says “get it working first, make it better second.”  That means changing the way the software enables a user to do something they can already do.  How do you manage requirements for incremental improvement?

Iterative Development and Incremental Improvement

Iterative development is the process by which you release software in iterations – small, incrementally better versions of your software.  People usually think about this solely in terms of enable the most valuable capabilities first, then the next most valuable capability next.

That’s fine, when your product is new.  Eventually (or quickly!) you will reach the point when the next most valuable improvement is not to add a new capability, but rather, to make an existing capability more valuable.

Everything is an Upgrade

In a couple earlier articles about organizing migration projects and requirements for migration projects, I wrote about how migration is a bit of a misnomer.  Everything is a migration.

My argument in those earlier articles is that the problems you’re solving with your “new” solution are not new.  Your customers are currently solving them in different ways.  The relevant question for migration projects is around how much your users are changing the way they solve their problems.

[As a slight segue, failing to appreciate this distinction can lead you to a project that is based on a faulty problem statement.]

When you are delivering incremental improvements to your product, through iterative development, you are upgrading your software.  You are migrating your users from their old solution to their new solution.  It just so happens that you are replacing yourself, as you migrate them from the old version to the new one.

A viable strategy, that appeals to me personally, is to continuously innovate, disrupting your market, before someone else does.  With this approach, you are intentionally reinventing your solution, making it difficult for someone else to out-innovate you.  This is a great way to approach developing your roadmap strategically.

Your market will change.  Rapidly. Do you want to react to your competitors, or keep them on their heels while you stay on the balls of your feet?

The reverberations of the “new” Twitter still haven’t settled, and many of Twitter’s competitors are on their heels – some of them only now realizing that Twitter has always been a competitor and not just “a platform.”  The new Twitter doesn’t really change anything about how people use Twitter, except make it (markedly) better.

Avoiding Featuritis

When is it more valuable to improve what your software already enables, versus enabling users to accomplish more with your software?  Kathy Sierra introduced us to the concept of featuritis, the idea that too much is too much.

In an article on viral product management, I proposed that by making your software better, you can tap into an altruistic mechanism by which people will promote your product.  As an example, I proposed improving the usability of your software as a means to cross this viral tipping point.

One of the key ideas is that making it better for users makes your product better for your company.

Managing “Improvement” Requirements

Now that we all agree that there is value in making your software better [comment below if you disagree] the question becomes “How?!”

Use Cases can be used as the keystone to your requirements arch.  Sometimes, user stories are better than use cases, but use cases are perfectly agile.  When you’re starting a project, you want to apply an outside-in approach to defining the scope of that project – or more particularly, to define the scope of problems you are solving in your roadmap. This article will talk in the language of use cases, but identical concepts and approaches apply when using user stories as well.

When you have prioritized a use case as “the next most valuable capability to introduce” you add it to the backlog for your sprint.  The implementation team then provides feedback that use case is “too big” and you have to slice it up.  It is important that you don’t split the use case  in such a way that you only solve half the problem.

You also shouldn’t deliver a “bad” product.  In fact, the ideal way to introduce a new capability is to satisfice, and not introduce the “perfect” solution in your first iteration.  Make your first solution “good enough.”  Part of the art is defining “good enough,” but rest assured, it is not the same as “the best you can possibly do.”

Summarizing some key elements from above demonstrates the eventual reality you will face:

1. Every problem* your users face is already being solved, you are just providing a better solution.
2. There is diminishing, and eventually negative return on adding more capabilities to your product.
3. Your first release of a given capability will only be “good enough.”  That leaves room for improvement.

*Yes, you can pedantically define “the problem” as “the existing solution is not good enough” – but you still end up in the same place, so why bother?

Eventually, the value of improving something you’ve already released will be greater than the value of releasing something new.

What do you do?  You implement the same use case again.

If your product is Software as a Service (SaaS), you absolutely should be thinking about things this way.  Becoming better is a continuous improvement objective that is implicit in the economics of SaaS products.

Revisiting Use Cases

Not to be confused with re-use of use cases, revisiting a use case is specifically revisiting – with the goal of improving – the implementation that is already in place within your product.  This is a special case of a migration project – you’re migrating your users from your old solution to your new and improved solution.

You may be migrating to an identical process, perhaps with faster performance than your previous release was capable of.  Or you may be improving the procedure (improving usability, interaction design, visual design, etc) without affecting the process. Generally, a “make it better” use case improvement effort will have no more than a minor process impact.  Your users are still doing the same thing, they are just enjoying it more, or doing it more effectively.

When slicing use cases to make them fit within a single sprint, you may design them for a single user persona in the first release, knowing that you won’t meet the expectations of a different user persona until the next release.  In this case, your non-functional requirements, constraints, or acceptance criteria will be different.  Your use case may also be different (in the details) while appearing to be the same.  This also common scenario can be addressed with the same approach.

Find that old use case (from several iterations ago) and put it back in the backlog.  Remember, agile is about conversation, not artifacts.  If you need to add an explanation to the use case, because your team fixates on the fact that it is “already working,” add a qualifier that it needs to be “better.”  Then have a conversation, and explain how it needs to be better.

Is Design Part of Implementation?

Some teams organize such that designers (both architects designing code and designers designing interfaces are being addressed here) are part of the implementation team.  Other companies treat designers as stakeholders – providing guidance and input to the product managers and product owners.

When a “new design” is being requested from outside the team, include that design guidance as part of the input to the implementation team – through artifact and / or clarification.

When a “new design” is being requested of the team (the designer is part of the team), express the new acceptance criteria to the team.  Note that these need to be measurable requirements if they are to be considered good requirements.

Summary

There will come a time when the most valuable thing you can do is improve the user experience for a capability your product already embodies.  When that time comes, use the same use case again, updated with the new constraints, non-functional requirements, and acceptance criteria.

Each requirement you write represents a single market need, that you either satisfy or fail to satisfy.  A well written requirement is independently deliverable and represents an incremental increase in the value of your software.  That is the definition of an atomic requirement.  Read on to see why atomic requirements are important.

Atomic Requirements – Revisiting

As part of the ongoing series, Writing Good Requirements – The Big Ten Rules, I wrote about the importance of atomic requirements first in 2006.  That article touched on only one aspect of atomic requirements – being able to ask “is it done?”

Writing atomic requirements is important from two perspectives – delivering value to your customers, and operating efficiently.  You get benefits both operationally, and in how you are delivering value when you write atomic requirements.

Atomic Requirements Accelerate Value Delivery

In a recent article, I proposed methods for splitting user stories when they are too large to be delivered within a single Scrum sprint.  That article looked at ways to break up complete user stories (an agile requirement artifact, similar to a use case) that were already atomic, making them even smaller.  Think of that article as Atomic Requirements 301 – sort of an advanced class on splitting the atom.  There are opportunities to subdivide molecular requirements – those made up of multiple atomic goals – the topic of this article, which would be Atomic Requirements 101.

If you’re using an agile development process like Scrum, consider a user story like the following:

• As an online shopper, I need to find and compare similar products, a few times a year, so that I can make an informed purchasing decision.

The key element, from an atomicity perspective, is find and compare.  These are actually discrete user goals, although it may not be immediately obvious.  Rewriting as follows, will highlight this:

• As an online shopper, I need to find products and compare similar products, a few times a year, so that I can make an informed purchase.

Rewritten like this, find products and compare similar products are clearly different activities supporting different user goals.  They should be split into separate, atomic user stories.

1. As an online shopper, I need to find products, a few times a year, so that I can purchase desired products.
2. As an online shopper, I need to compare similar products when shopping online, so that I can make an informed purchase.

These two user stories can be delivered separately.  If this example were for B2B (business-to-business) eCommerce, the example would be different – because the online shopper is (often) not the right persona.  A typical situation in B2B is that one person will do the research to determine what to buy, and another person will make the decision of from whom to buy it.

This rule applies for non-agile requirements development as well – consider the following “BRD-style” requirement example:

• The system must record all purchases and submit them to the fulfillment system for processing.

Recording purchases and submitting them for processing are two different activities.  While they likely support the same goal, they may also support different goals and could be implemented separately.  Recording purchases is important not only for fulfillment, but potentially also for analytics.  Pushing data to a fulfillment system can be done in either a manual or automated fashion – providing a distinct cost-reduction opportunity.  To write these as atomic requirements, they must be separated.

1. The system must record all purchases.
2. The system must submit all recorded purchases to the fulfillment system for processing.

This example shows only the bare bones – there is more involved in writing either requirement, particularly around defining the non-functional requirements and business rules (see also: why you benefit when you separate rules from requirements, and the difference between business rules and business requirements) that affect and constrain each requirement.  This is easier to see when the requirements are atomic. These two atomic requirements may be rewritten as follows:

1. The system must record all purchases on the same day that they are created, recording information per [BR117: Purchase Data Rules].
2. The system must submit all recorded purchases to the fulfillment system for processing per [BR231: Fulfillment System Data Interchange Specification].

These additional constraints are not non-atomic additional requirements, they are constraints that specify how the system must perform the atomic actions.

The previous examples allude to another key benefit of atomicity – clean traceability.  Each requirement (or user story) exists to support one or more goals.  Requirements traceability can get complex, when many goals are dependent upon multiple requirements to be realized and many requirements enable multiple goals.  This approach to representing goal-decomposition (or user story decomposition) is critical to assuring that you are writing valuable requirements.

The simple version of traceability can be visualized with this view of structured requirements, presenting (and slightly extending) some of Karl Wiegers’ work on requirements.  If you’re doing any work in requirements of any kind, you need to read Karl’s stuff – his Software Requirements is a must-have.

This diagram makes it seem pretty straightforward.  Where it gets messy is when the multi-goal-per-requirement and multi-requirement-per-goal dependencies (that always exist) turn it from a simple tree into a complex graph.  Non-atomic requirements make that graph messier (a hassle, but not a problem)as each “requirement that is really multiple requirements” ads extra dependency relationships to your traceability model.  This becomes a problem, however, when you need to change.

Every project has implementation tasks that take longer than originally expected.  Using a time-box approach to managing the content of each release gives you a straightforward framework for determining how much stuff will be delivered in each release.  When you have to slip something, you have to revisit prioritization.  With well-defined requirements, each valuable requirement that could be delivered provides value by enabling users (or systems as users) to achieve a goal.  When you have to delay the implementation of a requirement, you delay the goals that it supports.

As soon as you realize you are considering delaying part of a requirement, that is a red flag that your requirement is not atomic.

Atomic Requirements Simplify Operations

Atomic requirements allow you to make these on-the-fly prioritization decisions with much better insight into the resultant delay in value.  This allows you to better manage communication with your stakeholders – validating that you are delaying the realization of the right goals.

The same benefits apply when originally planning your software iterations and releases.  Conceptually, you can imagine “scheduling” everything for the first release, immediately discovering that most things need to slip to future releases.  That’s how most* software projects really play out.

*There are times when you intentionally delay particular capabilities, based on external factors – market positioning, customer-adoption cadences, sales-team capacity (to absorb change), etc.

Conclusion

Writing atomic requirements helps you

• Organize and describe why you are asking the team to build what they are building, and how stakeholders will benefit from what will be built.
• Understand the assignable value of each requirement by maximizing clarity in the dependency tree – each requirement is in place for specific reasons.
• Communicate the relevance of each requirement to the implementation team – informing cost-benefit discussions around the inevitable schedule-change discussions.
• Test the deliverables – atomic requirements get graded as pass-fail, not “X% working.”

Attributions:

• Thanks clix for both “atomic” photos!

Every team that transitions to agile faces this problem – some stories are too big to fit in a single sprint.  Most of the teams that I have worked with have the wrong instinct – to solve half of the problem for all users.

The right approach is to first solve all of the problem for a subset of the users.

The Goal of the Story

User stories were created as lower-overhead use cases that served to better describe chunks of valuable software that could be rapidly developed, delivered, reviewed, used, and improved. Writing user stories that undermine that intent – rapid, valuable, incremental delivery – is a bad way to write user stories.

Imagine you are part of the team building GoToAssist, a product designed to do this. Your product manager and product owner put the following story into the top of your backlog – it is the most valuable capability you could add.

As a provider of IT support, I need to remotely resolve issues on multiple computers simultaneously, every day, so that those computers will run well.

The acceptance criteria include requiring that the solution working when accessing Apple computers, and PCs running various versions of windows and Linux. Other acceptance criteria include allowing the solution to work well through network connections having minimum characteristics, not requiring the remote users to take action, and other elements that characterize what would be considered acceptable solutions by the stakeholders.

Your team immediately raises a red flag that this user story is “too big” for a single sprint.

Since the goal of the sprint is to deliver valuable, shippable software, you have a problem with any user story that cannot be completed within a single sprint.

No problem, just decompose the user story into smaller user stories…

Inside-Out User Stories (Bad)

Note: Don’t confuse making smaller user stories with defining the tasks required to implement a user story. Task definition is important and valuable for helping the development team organize their activities.  Making smaller user stories is not a good approach for organizing the team’s activities – use tasks instead.  The size and structure of user stories should align with the user’s goals.

An inside out perspective or approach is one that starts with an awareness of what is involved in creating the software, and either never, or only later considers the perspective of what it is like use the software.

When approaching the problem inside-out, you will break the story down as follows:

1. As a provider of IT support, I need to remotely gain access to multiple computers simultaneously…
2. As a provider of IT support, I need to remotely authenticate as an administrator to each computer I access…
3. As a provider of IT support, I need to remotely control each computer I access…
4. As a provider of IT support, I need to remotely restore control of each computer…

This approach looks very appealing, because it is very clear what has to be done to support each new user story, and each new user story is appreciably smaller. Assume that each new user story is “small enough” for the team and does not need to be further reduced in size. It is very easy to see how the scope of development work has been reduced for each user story.

The problem shows up when you approach this schedule from a user’s perspective. Imagine that this product shipped with only stories 1 & 2, and not 3 & 4. How useful would the user find the product to be if she could remotely access a computer and authenticate, but not actually control the computer (to do her remote support work) or return control to the computer’s owner?

The reason you should not break the original user story up into these pieces is that the new stories do not independently represent delivery of valuable software. These proposed smaller user stories are atomic (each step is discrete), but each step has no independent value. The value is implicit in the sequence of steps, and is only realized when all four steps can be completed. The original story is already the smallest set of activities that allow the user to achieve his goal – a complete user story.

The problem with decomposing the original user story from the inside-out is that the new user stories don’t provide independent value to any of the users.

What about trying an approach that provides all of the value to some of the users?

Outside-In Story Decomposition (Good)

You are still faced with the challenge of making the story smaller, but now with the constraint of only delivering pieces that provide independent value. You need an approach that will work to split the story up per-user. “Provider of IT support” is an overly broad category of users – it is heterogeneous, within which different groups of users will use the product differently. I am a “provider of IT support” to my family, but I never have a need to access multiple computers simultaneously. I also don’t need to do this more than once a month on average. A corporate IT specialist responsible for 100 users would really benefit from simultaneous remote access to multiple machines – allowing her to multi-task, starting updates on a second machine while the first machine is working, etc.

There are at least two different personas that care about the “remote administration” user story, within the “provider of IT support” user category. One persona who needs to access only one machine at a time, and one persona who needs to access multiple machines simultaneously. This split allows you to decouple all of the work associated with managing simultaneous connections from the work associated with the sequence of steps (as applied through a single connection). This may reduce the size of the original user story enough to complete it in one sprint, but probably not, since it only “shaves off a little work.”

Another way to discover different sets of people who could benefit from delivering the user story differently is to look at the acceptance criteria.

Right off the bat, we’ve called out that the solution needs to work when the remotely accessed computers are running several different operating systems. You can make the user story smaller by constraining to one supported platform at a time. Whichever platform you pick first – say Linux – has some users who will value your solution. Start with them, then add another platform. Work with the product manager to determine how many customers in the market will require each platform or combination of platforms – that will guide the sequence in which you build support for each platform.

Another acceptance criteria required that the remote users not need to be involved – enabling “unattended” remote support – a key market differentiator. A key differentiator for the corporate IT specialist persona, who gets value from not having to coordinate support activities with multiple remote users. In fact, it is probably a must-have feature for those customers. As a family-support specialist, I don’t really have this need. My mom is going to call me, and I can fix her pc while we’re on the phone. I’m not trying to remotely administer her machine when a corporate policy has changed (on my schedule) – I’m doing it when a problem arises (on her schedule). This criteria can be deferred for a later iteration of the story

This approach works because, while it may be not be intuitive (from the inside) that this is a viable way to decompose the story, it makes send from an outside-in, user perspective.  Think about the different sources of value / problems, and how different people value solutions to the same problems differently.  The important characteristics of the problem will vary from one persona to another.

What About Development Efficiency?

As a developer, you may very correctly complain that this is a really inefficient development process. Some portion of the work done to enable controlling a single machine may need to be discarded and recreated in order to control multiple machines in parallel.

While these extra development costs are very visible, especially to developers, they are much smaller than the benefits that come from delivering the working software earlier. The cost of resolving bugs increases by orders of magnitude as you move through the product creation life cycle (from development to testing to deployment). The value of solutions also increases by orders of magnitude as they move through the process. Getting working product deployed earlier allows you to start realizing value (and therefore revenue) earlier.

That increase in revenue far outweighs the cost of a less-efficient development approach. Ignoring revenue side and focusing only on the cost side is being penny-wise but pound-foolish.

Dogma and Pragmatism

None of this should be taken as dogma. My experience has been that this approach has almost always worked, sometimes requiring more radical thinking than others. When it does not work (e.g. does not provide additional value, or does not sufficiently reduce the size of the stories), don’t do it. I’ll always remember Kent Beck saying to me – “If it costs more to test (in advance) than it costs to fix the bugs (after the fact), then don’t test.”

The same applies here, be pragmatic about it.

If you’ve been biting your tongue since the middle of the “Inside-Out” section, because you can always just wait until all four user stories are completed in separate sprints, before releasing the software, then my apologies. Of course you can do that. But you are undermining a precept of Scrum – that every sprint could be released, if you needed to release it.

That may not apply to you – I work with some teams that develop enterprise software, and for them, releasing every sprint makes no sense – their customers can’t absorb the rapid improvements, their sales teams aren’t willing to sell differently, etc. So, please forgive me, and go back and read the Outside-In section again, since you were probably ignoring that section as you started planning a great comment about this.

Who knows – a “too big” user story may appear in the backlog for the last sprint within your current release – wouldn’t it be great if you had the choice to release something (valuable) in the current release?

Think about this problem from an outside-in perspective. The smallest story is one that allows a single customer to perform a single task, for which he will pay you.

People Over Process – Conclusion

The objective that every sprint deliver valuable, working software, is not a requirement. It is a design principle of the Scrum process.

Agile is not about mandating processes, it is about getting better and faster. When you can’t reasonably decompose a story into something that can be delivered in a single sprint, don’t. You can have a sprint where you don’t deliver anything, because you only complete part of one thing. Yes, it will look bad on the burn-down chart.

Will it mess with your velocity? A bit. But who cares? Velocity is a measurement that helps inform projected delivery schedules. You may have one sprint with zero velocity, but in the next sprint, when you do deliver the “two sprint story”, you’ll have double the velocity. Magically, your running average is back to normal.

Don’t let the rulers you use to measure the team’s output cause you to act irrationally.

Attributions:

• Thanks to John Loo for the half-a-coin image.
• Thanks to Handige Harry for the puzzle pieces image.

People who already use Scrum will only find one new thing in this article – a way to communicate what happens inside a sprint that has proven effective for me.  People who are new to Scrum who wonder “how do things work inside a sprint?” will see how things work in a way that avoids hyperbole and is easy to map to what they already understand from traditional software development processes.

Two Teams Separated by a Common Process

George Bernard Shaw was an Irish author in London who memorably said “England and America are two countries separated by a common language.”  The same can be said about teams developing in agile and waterfall processes.

The story of agile adoption is one that has many colorful examples of adoption and of spreading the gospel.  Some practitioners traveled across the ocean from the agile continent to the lands of agile and opportunity to reap the rewards.  A few of those, after realizing the benefits of agile, invade our consciousness with a hellfire and brimstone tale of doom for anyone who doesn’t convert to the new religion.

Others seem hell-bent on kidnapping entire development teams, smuggling them across the ocean in the belly of steamer ships and unceremoniously dumping them in the land of agile, ready or not.  Scrum is but one branch of the agile movement.

Stalwarts of the old way, which has worked fine for me, thank you very much, refuse to leave their comfortable lives to step into the unknown wilds of agile.  Open-minded but responsible potential converts ask questions to gain an understanding of what life in the new world might be like.  What will life be like if I join this Scrum congregation?

How Does Scrum Really Work?

The best explanations of how scrum works that I’ve read come from Mike Cohn and Mountain Goat Software.  The training, videos, and explanations they share provide fantastic top-down introductions (as well as guidance after adopting Scrum).  His book, User Stories Applied, is the ultimate reference when gaining an understanding of the mechanics of using user stories as the central artifacts for developing software with scrum.

[larger image]

The process diagram above provides a good outside-in, top-down view of how “this newfangled process” results in shippable product, incrementally.  It is a great way to present the concept of Scrum, and incremental development in general.

When people I’ve worked with first gain this understanding, they often ask, what are the artifacts that live in these backlogs?  User stories live there.  Sometimes use cases make more sense than user stories for Scrum- it depends (read the article to see which works best in your circumstance).  The common element is an outside-in, user-centric approach to describing which problems the software is intended to solve.  This is also known as goal-driven development.

Once people understand the top-down view of the process, and have an idea of what user stories are, the next question they tend to ask is “how do stories move through the Scrum process?”

The first attempt at answering this question leans on the process diagram above:

1. The product owner takes the most important stories from the product backlog and puts them into the sprint backlog – collaborating with the team to agree on the scope of work for this sprint (based on the amount of work, capacity of the team, real-world constraints, etc).
2. The team “works the stories“, meeting every day to communicate effectively and provide progress updates – best visualized with a burndown chart.  This chart says “we started with X work to do, and every day, we track how much of X remains to be done.”
3. At the end of the sprint, the work is done, and the software (or an update to the software) is ready to deliver.
4. Sometimes, multiple sprints happen between releases – launches of the updated software, because customers (and the rest of your organization) incur a cost when changing to the latest version.  But the output of each sprint is deliverable – that’s a key concept – even if you choose not to deliver it just yet.

What Does it Mean to Work The Stories?

I’ve repeatedly had people ask, after absorbing the above explanation, “what does it mean to work the stories?”

I’ve had consistent success (in bridging the language divide) using the following diagrams (drawn on a whiteboard) to explain how stories flow through the sprint process in Scrum.

The first diagram shows that user stories have a structure – the story itself, and the acceptance criteria for the story. I also establish that the story is going to go through a life cycle within the sprint.

[larger image]

Next I remind folks that the user stories make it into the sprint backlog because they are the most important (highest priority) not-yet-implemented stories in the product backlog.

[larger image]

Developers on the team pull stories from the sprint backlog and begin working on them.  These stories are work in progress, and are owned by individuals.  Some teams maintain that priority order within the sprint backlog (e.g. implement the most important story in the sprint first, etc.), while other teams use a more coarse-grained approach, relying on the product backlog prioritization to assure that everything in this sprint is the next most important stuff, and the order that the team implements, within the sprint, is up to the team.  Some folks like to strenuously debate this decision (don’t prioritize within sprints vs. do prioritize within sprints).  Both approaches are valid, and you should choose the right one for your team.

[larger image]

As each user story owner believes that her work is complete (all unit tests pass, and the developer believes that the acceptance criteria have been met), the story is ready to be tested by QA.  This step, in particular, helps people who are new to scrum and who think of things as “code and test”, to make some sense of the inner workings of a sprint.  When working with a team that was under-staffed in the QA department, I discovered that calling out this queue of user stories waiting to be tested helped convince management to increase QA investment in the team.  When too many stories pile up here, you know you have a problem.

[larger image]

The scrum team member responsible for testing the user story has the responsibility of assuring that it meets all of the acceptance criteria.  He pulls the story from the queue of “done” stories, and tests.

[larger image]

If the story fails to meet any of the acceptance criteria, it moves back into either the sprint backlog or the being developed column, where a member of the team resumes ownership of the user story and works to resolve the issue.  Once the issue is resolved, the user story is re-queued for testing.  When the user story meets the acceptance criteria, it is moved into the done done column.

[larger image]

Conclusion

If you are new to Scrum, and wondered how the sausage is made inside a sprint, this gives you a framework for understanding what is going on, and how the team delivers.  If you already know or use Scrum, you may have learned a couple things.

• Using this framework to describe how a sprint works is effective when explaining to people who do not have any experience with agile.
• Laying things out this way visually, will give you an easy-to-see signal if your team is unbalanced between developers and testers.

I’ve been involved with agile teams and processes for just over 11 years now, and incremental development is burned into my brain.  If you are new to scrum, please let me know if this seemed like a good way for you to consume this material.  If you’re an old hat, and have other suggestions or presentations, please add them in the comments below, because future readers will benefit from the additional ideas.

“For what one idea do you want your product to stand in the mind of your customer?”  I heard Roger Cauvin ask that question at the most recent ProductCamp Austin [correction - he said it here - thanks Roger], and the quote has been jumping to the front of my mind almost daily ever since.  Maybe by writing about it I can exorcise the demon and get back to using the idea instead of being haunted by it.

The One Idea

You’ve just been given a new assignment – your company needs a new software thingamajig so that you can play in the whatchamacallit space, and you’re going to drive product management for it.  You’ve also been asked to deliver the first version in six weeks.  Of course you’ll be able to do follow-on releases, after all, you are agile.  That’s how “agile” works, isn’t it?

Cool.  Exciting.  Challenging.

You sit down with the folks who will be building the product, and find out that they have already spoken with several stakeholders.  Excellent. You circle back with the stakeholders, and start to gather data about your market and audience.

Your schedule is aggressive, so you think about what is realistic to get done in a very small timebox.  With the current schedule, you can’t afford to get caught in analysis paralysis, and you can’t realistically do in depth market analysis, persona development, hyper-accurate requirements prioritization, etc.  You do, however, have time to do the most important thing – define the one idea that will define your product.

If you cannot come up with this idea, you need to push back on your management team, and delay the launch of your product until you have that one idea.

Don’t Abuse Agile

Agile is designed to help you rapidly create iteratively better products, with iterative development and continuous infusion of market feedback and data.  Agile is not a process by which you start typing without any idea of what you intend, releasing it and then getting feedback in an iterative process.  If that’s how you’re approaching agile, your process is broken.

[image from Market Driven Competitive Advantage]

Unless you’re very lucky, your first iteration will be a waste of time and money.  Wouldn’t you rather be intentional than lucky? Make sure you have that one idea before you start developing.

Strategic Alignment

You need to come up with a one idea that is aligned with and supports your corporate strategy.  At a minimum, you have to make sure that you have an idea that is aligned with your stakeholder’s goals, based on the assumption that those goals are aligned with corporate strategy.

Wow

Your one idea really needs to make your users say “Wow!”

I was explaining the importance of this to a colleague (who is not a product manager), and his response was “Oh, you mean the hook.”

I was reminded that you not only have to provide a wow experience for your users, but you have to present a hook that will capture the imagination of your buyers.  Remember that buyer personas make decisions based on their perceptions of user problems [more on convincing buyers].

Satisfice

Remember that you don’t want to try and do everything perfectly in the first release – you want to satisfice.  You may only be able to target a single persona with a solution to a single problem.  You just need to make sure you are solving the right problem – which is where Kano analysis is very handy for identifying the Minimum Market Acceptance criteria.

[From Minimum Market Acceptance]

Conclusion: A Balancing Act

Defining the one idea for your product is a balancing act.

• You have to align the one idea with your corporate strategy and stakeholder goals
• You have to solve a valuable problem for your users, presented in a way that captivates your buyers
• You have to create a product that is compelling right away – good enough in execution to enter your market
• And you have to get the first release out in six weeks!

Good luck!  Or better yet – Good Intent!

I came across a really interesting article LukeW.com, showing how making changes to the way an input form on a website increased interaction by 25 to 40%. The changes reflect the value of thinking outside-in, investing in user experience, and performance measurement.

Bonus: the idea is cool.

Mad Libs Input Form

[full size image available in Luke's article]

The idea being presented is to replace the old boring web input form designed for a computer. The new, fun form is a fill-in-the-blank (aka Mad Libs) layout. The article is on Luke Wroblewski’s site.  The team at Vast.com created and tested a version of this form for the Kelly Blue Book site. [Luke cites Huffduffer.com as the first place where he saw this technique.] Thanks, Luke for sharing this with all of us!

Outside-In Development

Product managers are acutely aware of the need to solve market problems. We are regularly reminded that we’re creating solutions for people in our markets who use our products to solve their problems. When we write requirements, we avoid writing design specifications, and thereby lose the ability to enforce that the proposed solutions also take an outside-in approach. We maintain an outside-in perspective, but we lost the ability to influence an outside-in aesthetic.

Note: Outside-In Software Development is a great book, you should read it.

That’s one area where user experience works well in concert with product management – assuring that the same drivers of what to build are informing the design of how it is built.

The genius (or at least elegance) of this Mad Libs form from Vast / Kelly Blue Book is that it humanizes the experience of requesting that a nearby dealer contact you to discuss a possible purchase of a car from them. The forms we’ve all had to fill out on countless web sites are very inside-out. They expose the inner workings of the program – “gather this info, and that info, and this other info.” Those forms force users to do what the program wants. Blech. The form that Ron Kurti designed, however, forces the program to do what the users want. Huzzah!

Bad Requirements Prevent Elegance

I’ve written about the importance of writing design-free requirements several times.  Avoiding design constraints in your requirements is even one of the pillars of the big 10 rules of writing requirements.

If you had written the requirements for this contact the dealer page, would you have specified the design of the form?

Most of the requirements analysts I’ve worked with would have. I’ve even worked with companies that have developed templates / stencils defining how these interface specifications must be documented – requiring that all the fields be aligned, with specific pixel gaps, etc.

If you had specified the design (read: limited the choices of the designer), you would have prevented this solution.

Mr. Kurti’s elegant solution is clearly better.

One way to write the requirements that would not have prevented this solution would be with a user story and acceptance criteria:

As a car-shopper, I want to engage a car dealer, once every few years, so that I can purchase the car I selected.

• The car-shopper can provide his contact info (name, email, phone) for the car dealer.
• The system will display the information identifying the selected vehicle.
• The system will display the information identifying the dealer being contacted.
• The system will include an opt-in newsletter-subscription option.
• The system will notify the specified dealer when the car-shopper indicates.

The user story nudges the developers into an outside-in perspective by emphasizing the user’s goal. The acceptance criteria make it clear that any solution that meets those criteria is acceptable to the product manager. It allows the interaction designer to create a compelling solution, rather than forcing him to recreate a boring experience.

Measurement Rocks

The Vast team measured the impact of making this change to their form, and saw between a 25% and 40% increase in conversion (people contacting dealers versus people abandoning the process when they see the form). That’s a lot more leads for the dealers, and presumably a lot more money for Kelly Blue Book and Vast. This is a great example of how you can measure the impact of investing in user experience, because it should spark ideas for you.

What can you measure and test and improve?

Now that’s ROI for you.

Engagement – that’s what this whole product management blogging thing is about.  Check out what Tyner Blain readers found to be the most engaging articles in 2009.

Deep Dives

If you’re new to Tyner Blain, you may be surprised by the length of the articles here.  I joke that they are long because I don’t have time to edit.  Stewart Rogers jokes that they are long because I’m incapable of writing a short article.  If you’ve been here a while, you know what you’re in for.  If you’ve been here a long while, then you’re glad (like me) that I don’t write one per day any more.

Product Management is simultaneously a broad and deep discipline, requiring us to have a breadth of perspective combined with a depth of insight.  We then have to apply that in a market context, effectively navigating the political waters of our organizations.  Most of the articles here either try to skim the breadth of a range of related topics, or plumb the depths of a single topic.  Doing that in under a thousand words is pretty hard.  Most of the articles here also link to other articles, to try and provide even more depth and context, and encourage additional critical thinking.

One measure of the quality of the articles here is how often they stimulate readers to read further, or dive deeper into the topics.  While web analytics won’t allow us to measure how thought-provoking an article is, we can look to see how many people dive into the linked articles, versus how many people move on to something else.  We can measure the bounce rate of an article to see how often people leave a page without following any of the “tell me more” links.

Looking at ~170,000 page views at Tyner Blain in 2009, narrowed down to only those articles with at least 100 page views, here is the top-ten list of most engaging (or “least abandoned” if you’re a “cup is half empty” person) articles:

1. Use Case Series: Introduction: A collection of articles on the “traditional” forms of use cases – informal, formal, UML.
2. Agile Development of Use Cases: A dive into the dynamics and cadence of an agile process for developing use cases.
3. How Do You Manage Market Data?: A collection of articles exploring different market-immersion techniques in depth.
4. Scheduling Requirements Changes – Part 2: A focus on practical techniques for managing change with an agile development process.
5. Yesterday’s BPMN Post Was A Big Fat Lie: A mea culpa and clarification of interest to folks following the BPMN Series.
6. Everything I Needed To Know I Forgot in Kindergarten: Why asking why? is important.
7. Plan Your Next Sprint by ROI – Part 1: Prioritizing by Bang for the Buck, not just Bang.
8. Ten Common Mistakes of Going Agile: A collection of articles about common pitfalls encountered when adopting agile practices.
9. Perpetually Almost Finished Projects: Organizing a project with discrete deliverables and rolling-wave planning to avoid the “90% done, 90% remaining” problem.
10. How To Use Timeboxes for Scheduling Software Delivery: One of my personal favorites.  A rational approach to making trade-offs when your “perfect” plan has to change.

OK Stewart, that’s only 519 words.

Design-Free requirements are important for two reasons, and hard for two other reasons.

Design-free requirements are hard because you “know what you want” when you should be documenting “why you want it.”  Writing design-free requirements can be hard when you don’t trust your development team to “do the right thing” even though it is not your job to design the solution.

Design-Free Requirements – Revisiting

It has been three years since I wrote Writing Design-Free Requirements as part of The Big Ten Rules of Writing Requirements.  In that time, agile development practices have moved from being an esoteric development methodology to being the topic on the tips of everyone’s tongues as executives and organizations try to either (1) get the benefits of the state of the art in software-development process or (2) do something shiny and fashionable.

The previous article centered on elements of designs within MRD, PRD, and SRS artifacts.  A regular alphabet soup of artifacts.  Honoring the values behind the agile manifesto encourages us to emphasize working software over comprehensive documentation.  In that light, we’ll take a “what is needed” approach to talking about how requirements, design, and implementation are all needed – rather than issue an edict about where that information should be captured.

Agile Development Inputs

When creating software, someone needs to know:

• What problems are being solved, and how important (valuable) they are to be solved.
• Who has the problems and who is using the software to (help) solve those problems.
• What constraints limit the space that defines the universe of possible viable solutions.
• What acceptance criteria define if the delivered solution will be acceptable.

Subject to those inputs, someone needs to make design decisions:

• User experience design - what interactions will a user of our solution love?
• Program design – how (in a nuts and bolts way) will our solution function?

When talking about agile and talking about design, we should take a look at how Kent Beck and Alan Cooper, as respective though leaders in each space, view this.

Cooper doesn’t talk much about creating the requirements to support the daily use scenarios – he proposes moving directly into design of the solution. This differs from the more traditional technique of writing functional requirements to support use cases. Cooper also breaks down design into two components – program design and interaction design. Program design is everything you don’t see. Interaction design is everything you do see.

[...]

Cooper argues that designing the interaction should happen before any code is written. He uses a construction analogy to drive home his perspective – you can’t pour the concrete before you build the forms. Kent Beck, founder of the XP programming philosophy disagrees with the premise. Beck believes that the cost of changing software is low, and the imagery Cooper uses is hyperbole. We touch on, and link to that debate in this post.

Interaction Design Process Overview

I don’t believe that Beck is arguing for “don’t do design” – I believe he is arguing for “don’t do big upfront design that would delay implementation.”  He’s championing the agile values that emphasize creating working software and responding to change.  I can imagine him saying “no one will buy a design, they want a solution.”  Cooper’s point is that create a solution without first understanding how someone will use it, you can’t create a great product.

Program design has many hidden impacts – cost of maintenance, cost to change, and the performance of the solution.  You can have a design (or architecture) that makes everything easier to do in the future – at the cost of delaying the delivery of anything.  Or you can have a design that minimizes the time to deliver the first thing, while increasing the cost to deliver any particular next thing.  Or you can design a solution that falls somewhere between those extremes.

If you say “we don’t do design” you’re lying.  Every solution includes design.  Your team’s design process may be “big and upfront” or it could be a couple sketches on a whiteboard, or some email exchanges.  You may create storyboards and wireframes.  Or design may be the thing that happens right before the programmer’s fingers strike the keys.  Design may be explicit or it may be implicit – but it never “doesn’t happen.”

In the movie Amadeus, Salieri is astounded that there are no rough drafts of Mozart’s compositions.  That’s because Mozart did the design in his head, not because he didn’t do design.  I’ve worked with programmers who’s “implicit designs” were great, but that is the exception, not the rule.

Who Owns Design?

Since design happens sometime before fingers strike the keyboard, the real question is – “who owns the design?”

• You have a product manager developing an understanding of market problems, and prioritizing the problems that should be solved with your product.
• You may have a product owner managing the backlog and clarifying those requirements (problems) for the development team.
• You may have an interface (or interaction) designer or team of designers who are broadly responsible for “how users interact with our products.”
• You may have an architect or lead engineer who is responsible for the “big picture design” of how your product works.
• You have developers and testers who are responsible for delivering your product. [Note: coding without testing is like typing code without compiling - maybe it works, but probably not.]

You may not have distinct individuals with each of these titles – every small team I’ve worked with has people wearing multiple hats.  This is especially true when you have an agile team full of specializing generalists – any given story (or task) may have a different architect and different implementer.  I’ve only worked with one company where the architects are NOT part of the development team.  If your team is set up that way, please comment below – I had never seen that before, and I’m not convinced that it is the best way to organize – what have your experiences been?

Generally, “program design” is clearly owned by the development team – and product managers (and product owners) know better than to specify program design in their requirements.  Neither the lead engineer nor the product manager believes that the product manager is a better programmer – so the product manager better not be specifying program design and mislabeling it as requirements.  Coincidentally, Steve Johnson, at Pragmatic Marketing has a post running right now with a bit of a quiz – is this a req(uirment) or a spec (design)?

Where the line is more blurred is around interface and/or interaction design.  Some development teams have interface designers as part of the team.  Some companies organize with interface design as a “shared service” within the company.  Either approach can be the “right one” – it depends on too many details to make a sweeping generalization.  When the designers are members of the development team, the solution (from a product management perspective) is the same – the “team” is responsible for all design.  When the designers are not part of the development team, the developers have to consume two sets of guidance – “solve this problem” from product management, and “the solution needs to look like / act like this” from the designers.

Collaboration between product management and user experience people is the ideal solution.  The “requirements” and “design” inputs to the development team are comprehensive and consistent.

Design-Free Requirements

There are benefits – especially when being agile and minimizing documentation – to delivering requirements and design at the same time.  Just don’t do it by embedding design constraints within the requirements.

• When people on your team misinterpret design as requirements, they are unnecessarily constrained.
• As a product manager – are you the best designer on your team?  If you’re busy designing, who’s product managing?

This is trickiest when writing use cases – sequencing a set of interactions is interaction design.  That is one benefit of writing a user story instead of a use case.  An approach that has worked well for me with multiple teams is to deliver user stories (requirements) combined with storyboards (interaction design) and wireframes (interface design).  When details are needed (usually when “changing” versus “creating” an interface), screenshots can replace wireframes.  When business processes are complicated, process flows can replace storyboards.

Just don’t embed the designs within the requirements.

Your company is building out a toolkit to support third-party developers.  You’ll need a bunch of different types of widgets – combo-boxes, text entry fields, domain-specific controls, etc.  You’ve got a long list of desired controls from your customers.  You’re agile.  What do you build first?

Agile In A Soundbite

Being agile is about delivering incremental value, quickly, getting feedback, and then delivering more incremental value.  Repeat until “done.”  Good agile adds a qualifier – do the most valuable thing quickly, get feedback, do the most valuable thing (that has not already been done) quickly. Better agile optimizes the rate at which you deliver value by taking into account both benefit and cost.  Great agile overlays a focus on getting better at doing all of these things while you do them – becoming a learning organization.

Boiling the Ocean

A product manager I was coaching was faced with a challenge commonly referred to as boiling the ocean. His team was tasked with solving a market problem, and they were constrained to doing it with a service-oriented architecture that exposed a set of widgets (user interface controls) that customers could easily integrate into their products.  This approach was designed to provide competitive differentiation by reducing the time and cost to deploy solutions that allowed customers to integrate his product into their existing platforms.

In initial conversations with customers, technologists, and architects, this product manager quickly amassed a list of desired widgets (controls) and scenarios (stories) in which they could be used.  The product manager’s team had recently switched to an agile development methodology.  The internal stakeholders, not yet accustomed to agile development, wanted “all of the widgets,” now.

This product manager was able to convince them that the team would deliver the widgets incrementally, following agile principles.

His question was – How do I sequence the widgets in the backlog?

Defining Widget Priority

The product manager had a list of widgets, combo-boxes, data-grids, text fields, radio buttons, etc., and for each widget, he had a real-world scenario showing how the widget could be used.  Most of the scenarios involved customers using multiple widgets.  He wondered if he should do some sort of analysis that detailed the frequency of use of each widget.  ”This widget is used in 7 scenarios, so it should be done first; this widget is used in 5 scenarios…”

There was a strong temptation to add several “Create a widget” tasks to his backlog.  It would be easy for his development team to estimate the effort required to deliver each widget.  His team wanted to deliver incrementally, and “one widget at a time” felt like logical, discrete chunks of work to them.  They could easily sink their teeth into estimating, building, and testing each widget.

A quick reminder of a main tenet of agile, delivering incremental value, illuminated the flaw in this approach.  This approach would have been an inside-out prioritization, when value delivery requires an outside-in perspective. (See the ‘recommended reading’ suggestions on this page to check out Kessler and Sweitzer’s great book.)

If the team used this approach, after the first widget was complete, they would be able to deliver exactly 1 task, and 0 stories.  Development teams don’t deliver tasks, however.  The team’s customers would not be able to get incremental value from having a single widget.  The team would have delivered seven incomplete stories – so they would have delivered no stories at all!

We reworked his approach as follows:

1. Assess a value for each story, making sure that each story would enable his customers to accomplish something valuable.
2. Engage their user interface / user experience designer to design a solution for the most valuable story.  This design was constrained to use widgets in the user interface.  The suggested way to communicate this design was with a storyboard and low-fidelity wireframes.
3. Identify the widgets needed to be built to deliver that story, using the designer’s design.
4. Deliver the story to the development team, including the storyboard, wireframes, and list of widgets to be used as acceptance criteria.
5. Get the development team to size (estimate) the effort to complete each story.
6. Where stories were too big (epics), collaborate to identify good ways to break up the stories into manageable chunks.
7. Repeat steps 2 through 6 until the amount of identified effort was likely to fill the release (keep the dev team busy delivering value).
8. As the team delivers each story, get feedback from the market, revisit the prioritization, and revise the “next” story.

What we discovered was that a scenario like the following (modified for this article) played out:

• The first story required two widgets.  As no widgets existed, both had to be built.
• The second story required three widgets, but two of them had been built to support the first story – only one incremental widget had to be created.
• The third story used one of the widgets from the first story, but required additional behaviors.  The original widget was modified to provide incremental capabilities (and value).

A Note On Team Structure

Astute readers will notice that there was a design step “inside the requirements process” – before the stories were delivered to the development team.  Technically, that is true, for the way this particular company was organized.  The user experience designer was not a member of the scrum team, but rather, an external consultant who supported multiple teams.  The product manager engaged the designer prior to engaging the development teams.

This just as easily could have happened as follows:

1. Product manager identifies, values, and prioritizes stories to be added to the backlog.
2. Development team, as part of sizing the stories, engages their user experience designer to select the appropriate widgets, and those design decisions inform the estimation process.
3. Everything else is the same.

This procedural variation does not have design “inside the requirements process.”  The same people do the same things, in the same sequence in this scenario.  The only difference is that the interface / interaction designer is a member of the development team.  That would be ideal, but that was not how the company was organized.

In this particular example, having the product manager collaborate with the UX designer made the most sense.  It introduced less complexity for the product manager to accept responsibility for this activity than it would have for the development team (who was still new to using an agile delivery cadence) to do it.

Conclusion

The key ideas at play here:

• Focus on realizable value to the customer (outside-in development), not tractable tasks (inside-out development).
• Collaboration with a UX professional is key to driving “interface requirements.”
• Deliver frequently and valuably, get feedback (learn), and incorporate that knowledge into whatever is next.

Perpetually intermediate (competent) users.  Users who briefly exist as novice users and never become experts. Most of your users are competent, and you should design for them.  Competent users have different needs and different expectations than novice or expert users.  How do you know your user’s competency levels, so you can design for them?

User Competency

User competency is a concept I first read about in Alan Cooper’s The Inmates Are Running The Asylum.  Cooper’s contention is that the level of expertise of your users follows a bell curve, or normal distribution.

When we’re designing software we need to keep in mind that most of our users will be competent – neither experts nor beginners. Alan Cooper’s studies tell us that user skill levels follow a bell curve. He talks about competent users as perpetual intermediates. Some users drop out of the bell curve when they stop using our software. The rare user becomes an expert. Most users only learn enough to get their real job done.
Competent Users and Software Requirements

Cooper contends that the combination of learning curves and natural user tendencies to stop learning or abandon a software application are the sources of this distribution of user competency.

Experience Curves

Experience curves represent the diminishing costs over time of manufacturing something repeatedly.  The process of manufacturing something gets more efficient as you get smarter about the process.  The process of using software is at least analogous to, if not a specific example of a manufacturing process.  If you’re using an email application, you are manufacturing email messages.  In a CRM system, you are manufacturing contacts, or contact reports, etc.

By treating any “using the software to do something” interactions as a process, you can measure the cost (how long it takes) of the user’s interactions.  Applying the math behind experience curves, you can predict the reduction in cost (to your users) over time, for any set of interactions.  Experience curves take into account that some processes are inherently more learnable than others.  This property of learnability is reflected as an efficiency coefficient – how efficiently someone can learn ways to reduce the cost (time) needed to perform the interactions.

This gives us an approach to quantitatively model user competency.  Having a definition allows us to model competence.  And measuring competence allows us to manage product design in the context of user competency - designing for competent users.

Defining Competence

The first step to measuring competency is to define the model.  I am proposing a definition in this article that I suspect will yield insights (to help us manage our products). I was unable to find any quantified definitions of competence, when researching it as part of a client engagement.  If you have, or know of a model, please share it in the discussion below this article.

• A competent user is someone who learns to perform a task in half the time it initially took them.
• An expert user is someone who can complete a task in 10% of the initial time.

This definition is guided by an expectation that Alan Cooper’s premise about perpetually intermediate users is true.  Being a novice user is a very transient state, and becoming an expert is very infrequent.  The goal of the definition is to be able to segment your users and make well-informed design decisions.

A Proposal, Not a Doctrine

This is a proposal that doubling performance reflects competence, and achieving a ten-times improvement represents expertise.  It may be that some different measure of performance improvement more accurately reflects competence and expertise.  We have to test it to know.

The experience curve is defined mathematically by Henderson’s Law.  It states that the time to complete a task is a function of the number of times you have previously done that task, adjusted by the “elasticity” of the cost of that task.  In other words, some tasks are easier to improve than others.  If you populate a table with the results of applying Henderson’s Law, you get the following:

[larger image]

• Each row in the table represents the Nth repetition of a task.
• Each column represents how easy that task is to learn – progressing from “hard to improve” on the left, to “easy to improve” on the right.
• Each cell in the table represents how long it would take to perform the Nth repetition of the task, as a function of how easy it is to improve your performance at the task.

The above definitions represent what experience curves predict mathematically, when the task initially takes 60 seconds.  The following definitions reflect the proposed user competency model:

• A user is a novice user until she has learned enough to cut the time-on-task in half.  These cells have a white background (and are in the upper left area of the table).
• A user is a competent user when the time needed to complete the task is between one half and one tenth of the initial time.  These cells are shown with a yellow background (and are in the central area of the table).
• A user is an expert user when she can complete the task in less than one tenth of the time required to initially complete the task.  Theses cells are shown with a red background (and are in the lower right area of the table).

In the absence of empirical data, I used my intuition to suggest that a representative experience curve for a typical task performed in software would be one with an “elasticity” of 50%.  For a task with those learning characteristics:

• A novice user would cut the needed time in half on the 4th repetition of the task, and would be considered to be a competent user.
• A competent user would further reduce the time needed to one tenth of the initial time on the 100th repetition of the task, and would be considered an expert user.
• The 50% elasticity column is surrounded by a black border, and the number of repetitions required to advance to competent or expert status is also highlighted with a black border.

Inferring Competence

Since different processes have different learning characteristics, you have to figure out how easy it is for your users to improve at your processes.  To do that, you have to study (or at least measure) your users’ interactions with your software.  In the 50% curve highlighted above, a user is capable of cutting their time-on-task in half by the fourth time they perform the task.

If data from your initial testing (or measurement) reveals this to be true, then you have selected the correct curve (the correct column in the table).  If it takes more or less time to reach this level of improvement, shift to the left or right to find the appropriate curve.  If software-interactions are reasonable analogs to manufacturing processes, then the experience curve projects an expected rate of improvement on task.

The following graph isolates the time-on-task data for a user who is learning to improve when repeating a task (process) that matches the 50% elasticity curve.

[larger image]

Note that the number of repetitions of the task (the X axis) is represented as a logarithmic scale.  The data points along the curve correspond to the cell values in the table above (for the 50% column).

Shades of Gray

One nice thing about this quantitative approach to inferring competency by measuring usage is that your measurements are per-process.  Users are not “purely novice” or “purely expert” – they can be experts at some processes, while remaining neophytes at other.  There is also awareness, for any particular process, of “how much competency” a user has.  This allows you to refine your assumptions of the steepness of the learning curve, and of the thresholds (doubled performance and ten-times performance improvements).

Improvement Over Time

Any particular learning curve can be considered relative to calendar time, to see how quickly a user will progress along the curve (as a function of frequency of use).  This can be useful for determining the ROI of improvement in a particular process.

The following graph shows how an 80% learning curve overlays a calendar for tasks that happen daily, weekly, and hourly.

[larger image]

The graph shows that for a weekly frequency, after 16 weeks, the task time has reduced from 300 seconds to 100 seconds. With a daily frequency, the task time is even lower – about 60 seconds. This graph shows nothing other than converting the academic learning curve graph into one that incorporates calendar time and frequency of occurrence.

Software Usability and Learning Curves

One approach to inferring user competency would be to measure how long a user has been using your software.  The variation in how frequently different users perform the same task will introduce an error into that inference.  You can avoid introducing that error into your modeling by counting the number of times a user has performed a task.

Applying the User Competency Model

The advice in previous articles, and from Cooper’s book, and from this great article on the  Coding Horror site, encourages us to focus on the competent users.

I’m working with a client who needs to prioritize a set of capabilities and establish design principles for a product.  We will incorporate this user competency model as part of our analysis.

Hopefully we’ll have an opportunity to collect data to validate and / or refine the model.  I’m proposing that we first gain some insight into the which users (novice, competent, expert) drive the most revenue and profit from use of the product – to establish the importance of each category of user.

For this product, I suspect that we will find many more novice users than a normal distribution would predict.  If that is true, the next question will be to understand if we are dealing with a normal behavioral dynamic, or if characteristics of the current product “force” novice users to abandon it before they achieve competence.

Either way, we will have a framework for prioritizing the goals of the novice, competent, and expert users.

How would you apply a model like this to improving your product?

Concise requirements give your team a useful, easy to read and easy to change understanding of what must be done.  Great requirements exist to do three things:

1. Identify the problems that need to be solved.
2. Explain why those problems are worth solving.
3. Define when those problems are solved.

Concise Requirements – Revisiting

In the three years since we last looked at Writing Concise Requirements from the Big Ten Rules of Writing Requirements, the iPod evolved to give us a better experience.  Let’s see if we can do the same with the topic of brevity in requirements.  The size of our community here has grown ten-fold, and the people who were here back then have grown just as much.  It makes sense to look at this again.

Writing concise requirements is not just minimizing the number of words you use.  Writing concise requirements is presenting the most important information in the easiest format for your audience to consume.

Concise Requirements Identify the Problems That Need to be Solved

Ultimately, requirements are the problems that we choose to solve.  A concise requirements artifact (formal document, index card, photo of a whiteboard, whatever) is one that has the highest signal-to-noise ratio possible.  You’re maximizing the amount of communication, and minimizing the cost of communicating.

You want your requirements document to read like the lines, not the points.  If the line (the signal) is what you really want, and you communicate a big pile of points (the signal, hidden in the noise), you run the very real risk that your audience will misinterpret the signal.

User stories provide the best example of clarity that comes from conciseness.  The format you should use, based on Mike Cohn’s great book, User Stories Applied, is

As a [role], I want to [do something] [with some frequency] so that I can/will [achieve some goal]

The user story is not always the right communication format – it depends on what problems you’re solving, and who is on your team.  Sometimes, use cases work better than user stories.  Conciseness is important for use cases too.  Start with a good use case name.

Concise Requirements Explain Why Those Problems Are Worth Solving

Last week’s article on Valuable Requirements focused on why particular problems should be solved.  Your focus should be on value, and that article discussed five ways to assure that your requirements are valuable.  One of the techniques, the use of an Ishikawa diagram, provides a method for identifying the root causes of problems.

[larger image]

Imagine that you have a collection of user stories, representing problems to be solved for your users.  You need a way to demonstrate why this user story should be implemented and why that one shouldn’t.  You can often use the Ishikawa diagram in the same way.  A particular goal is achieved when a user is able to do something.  Perhaps several somethings are required.  The point is that you can use the Ishikawa to drive home the point – if this set of user stories are all implemented, then this goal will be achieved.

There are two reasons this is important – first, you’re providing context for your team.  They understand why they are doing something.  Second, you can make changes easily, because you can see the impact of those changes.  By understanding the cause-and-effect relationships between problems and their values (user stories and their goals), you can see the impact of changing one or the other.

Concise Requirements Define When Those Problems Are Solved

Clarity is the goal of conciseness.  It isn’t enough to say “work on this.”  It’s important to know why you’re working on it, but that still isn’t enough.  You have to know when you’re done.  When you’re defining problems to be solved (and therefore solutions), you must also define the measures by which the solution will be judged.

A measurement of success for “Cost of Operation is Too High” might be “reduce costs of operation by 10%.”  This gives you a testable criteria for knowing when that problem has been sufficiently solved.  Sticking with the Ishikawa, you can also map out the strategy for achieving that lofty goal.  You can say that the goal is to reduce fuel expenses by 20%, reduce cost of maintenance by 5%, and reduce payments by 15%.  This process continues – a 20% reduction in fuel spend requires that you operate with your tires within 5% of nominal pressure, and that you reduce the aerodynamic drag coefficient by 7% (or whatever).

This gives you clarity in your objectives.

User stories, when combined with user acceptance criteria, provide that last connection of testability that lets your team know when they are done.

There aren’t many things more frustrating to a development team than having them solve the wrong problems.  One of those things might be having their solution be rejected because it isn’t enough.  Writing acceptance criteria clearly and concisely lets the team know exactly when they can move on to the next problem.

Providing a crisp understanding of acceptance criteria also facilitates iterative development.  One challenge teams always face is making improvements that fit within the timebox of a given iteration.  Imagine a user story with 4 acceptance criteria, where the story is too big to complete in one sprint.  When talking with your development team, you may find that the story is too big because satisfying all of the acceptance criteria is too big.  This is where many teams miss an opportunity – by defining all of the acceptance criteria that are believed to be needed eventually and requiring that they all be implemented now.  One of those criteria is going to be more important than the others.  Implement the story such that is satisfies the most important criteria (timeboxing) now, and rewrite or enhance it to meet the additional criteria later.

Conclusion

Software Product Success depends not only on identifying the “right stuff” to build, but on making sure the team builds it and builds it right.

Concise requirements improve your ability to communicate with your team, thereby improving their ability to build the right stuff right.

The maturity model approach to describing organizations and processes comes and goes out of fashion.  It is a repeating framework de jour.  In the game of agile jargon whack-a-mole, the agile maturity model is poking its head up again.

Agile Maturity Model

Ellen Gottesdeiner, author of Requirements by Collaboration, tweeted (@ellengott) a few hours ago with a link to an article proposing a set of maturity models.  She graciously passed on the opportunity to comment about the “agile maturity model” when pointing out her like of the collaboration maturity model.  The article proposed the following as an agile maturity model:

Agile Maturity Model (AMM)

0 – Dormant

1 – Speed: Focusing on being expeditious.

2 – Reactive: Focusing on acting relative to change from the perspective of the moment rather than a longer timeframe.

3 – Responsive: Focusing on acting relative to change from the perspective of the moment balanced with a longer timeframe.

I didn’t find it to be a particularly useful model.  Although descriptive, it won’t help your organization improve.

What Does Maturity Mean?

I wrote a series of articles a couple years ago that explored the CMMI and RMM – the capability maturity model and the requirements maturity model.  These are two different models that use maturity as a concept for articulating different levels, or grades (or enlightenment, or rigor), with respect to organizational behaviors.  At the time, there was both momentum and confusion around the notion of a requirements maturity model.  CMMI is a description of an organization’s rigor in “saying what it does, and doing what it says.”  RMM is an assessment of the level of critical thinking incorporated into the ways an organization is using requirements to develop products.

The problem is that people were incorrectly assuming that an organization “at level 4 CMMI” would approach requirements management at a comparably enlightened level.  The problem is that they are putting entirely unrelated concepts into similarly named classifications.  An organization could be at CMMI 1 and RMM 4 or RMM 2 and CMMI 3.  That series of articles explored what it would mean to be in any of the combinations of “maturity” for those two models.  Check out all the articles in the series if you want to put it all in perspective:

• Foundation Series: CMMI Levels Explained
• What CMMI Level Should We Use?
• CMMI Levels and RMM Introduction
• CMMI Levels and RMM Level 1 – Written Requirements
• CMMI Levels and RMM Level 2 – Organized Requirements
• CMMI Levels and RMM Level 3 – Structured Requirements
• CMMI Levels and RMM Level 4 – Traced Requirements
• CMMI Levels and RMM Level 5 – Integrated Requirements
• Don’t forget to take our One Minute Survey on CMMI and RMM.

Making a Maturity Model Useful

Why bother with a maturity model?  Not so you can “keep score” relative to others – rather, so that you can improve your own organization.  For a maturity model to be useful, you have to be able to do two things.

1. You must be able to determine where your organization is in the model.
2. You must be able to identify actions you can take to “improve” your organization relative to the model.

If you can’t measure maturity, and the model does not provide guidance about how to improve, it’s useless.  One challenge with maturity models is that they risk becoming contextually narrow in their application.  The more concrete a measurement or suggestion becomes, the less extensible it is likely to be.  Ideally, your model would be broadly applicable to many organizations.

With an agile manifesto that emphasizes people over process, it is ironic to consider applying a metric that measures your agile process.  So – goal #1 is immediately undermined.  Goal #2 – improving your organization, is, however, very valuable.

Improving An Agile Process

I’ve worked in several agile environments, both as a practicioner and as someone helping to transform organizations to become agile (or better at agile).  I’ve worked in enterprise software, helping large teams adopt agile processes; with a SaaS consumer product team, and in large IT departments with collections of teams adopting agile processes at varying paces.  I’ve played on both sides of the fence (delivery/QA & product management/ownership).

Going from zero to sixty on agile is a big task.  You can’t solve all of the organizational, practical, and interpersonal challenges at the same time.  Or at least you would be more effective tackling and resolving each issue in sequence.  In fact, you should solve the most important / largest problem first – then move on to the next, and so on.  You can call it agile agile adoption, or you can call it eating the elephant.

One powerful use of a maturity model is highlighting the next-biggest hurdle you have to overcome.  Unfortunately, most communication about maturity models is “look which hurdle we just overcame” – but the focus should be on “what’s next?”

When helping organizations be effective with agile processes, there are some clear “you have to overcome this first” challenges, that if unresolved, make other challenges irrelevant.

If you’re a software developer, I’m sure you’ve experienced the following scenario:

• A bug is reported.
• You fix the bug, and write some tests to prevent it from reoccuring.
• While testing, you discover another bug that was hidden by the first bug.

Fixing that second bug doesn’t do any good until you’ve fixed the first one.

If you’re a product manager or business analyst, think about it in terms of capability enhancements.  Is a faster search important if the current search algorithm is not returning the right results?  Get good results first, then make it faster.

You have to solve the biggest/closest/roadblockiest issue first.  Then move on to the next issue.  That’s how you should use a maturity model – as guidance about “what’s next?”

When assessing / improving your organization’s ability to be agile, you need to be addressing whatever is next.

What’s Next?

Let’s discard the “maturity model” notion and focus on “what’s next?” instead.  And that of course starts with “what’s first?”  Here’s how I’ve presented an agile hierarchy of needs to in the past:

1. Staffing the engineering team correctly.  People over process is the right emphasis.  If you can’t find people that are “good enough” you might as well go home.  Doesn’t matter how agile you are if you don’t have the horsepower.  You also need people who are excited to “do agile” – they like to communicate, they enjoy the project and team dynamics of an agile process.  You’re also better off with specializing generalists – ideally, every member of the team can do any work that is needed.  This is an efficiency play – you risk introducing bottlenecks when you have a specialist who is the “only one” who can do particular types of work – because you will not have a consistent mix of types of work from release to release.
2. Assuring Quality is in your team’s DNA.  Arguably, this is part of what’s first, but there are a lot of teams that get cood at cranking out code, before they get good at cranking out good code.  Continuous integration is the approach you must have.  Test-driven development, spec-driven development, and other testing-integration approaches are extremely important, but are really reflective of different flavors of agile and quality, not different degrees.
3. Reducing overhead in the release process.  There’s a cost associated with releasing a product.  Once you get good at releasing, and are releasing good product, your next focus is on finding the right release cadence.  There are three factors that essentially dictate your release frequency.  The size of atomic deliverables (user stories and use cases, non-functional requirements, etc) your team is creating, the overhead (time and cost) of releasing, and your customer’s capacity to consume the releases.  My anecdotal experience is that teams are usually constrained initially by the cost of releasing – dedicated hours to creating a build, code freezes, etc.  Automating the build process (to reduce both costs and errors introduced while building) is first.  Integrating automated build and automated test processes together is next.
4. Feeding the beast.  Once you have an engineering / delivery team that is operating efficiently, you run into the problem of making sure they have enough important work to do.  There’s pressure on product owners and  product managers to schedule something because it is easy to define, not because it is important.  It is also important that the team understand the context and importance of what they are doing.  So you have to focus on making sure your product management team has enough capacity to keep the engineering team busy on delivering really valuable stuff.  The “right” solution to this depends on how the problem manifests in your organization. Is your product manager spread too thin across multiple products?  Too junior?  Too bogged down in sales support or trade show attendance or or or?
5. Managing stakeholder expectations.  A big challenge in changing an organization to become agile is in resetting and managing the expectations of stakeholders.  Execs are used to having an annual budget exercise where they dedicate X dollars in the pursuit of Y objectives.  We’ll set aside the reality that they’ll only achieve 1/3 of the objectives, at 2X the cost, much later than originally forecasted.  We aren’t looking at the failings of waterfall, we’re looking at the risks of agile.  There’s a ton of stuff here, from doing damage control to reverse perceptions that “people who want to do agile just don’t want to be accountable” or addressing the “we can’t tell you what X dollars gets you” message.  Again – the particular solution is a function of how the problem manifests in your organization.  Once your team is delivering valuable stuff efficiently, you have to make sure your management team is happy and engaged and knows what to expect.
6. Continuously learning from your markets.  This is really what differentiates an agile organization from an organization with an agile development team.  Agile processes do enable you to develop code more efficiently.  If you aren’t taking advantage of the faster development, feedback loops, and ability to change that agile enables, you’re leaving money on the table.  And one of your competitors will pick it up.  There’s a whole community of MBAs that talk about business agility without once thinking about software development.  They’re talking about an organization’s capacity for rapid response to changing market conditions.  It’s what separates the winners from everybody else.  And agile development makes business agility possible.  As long as your team is able to identify and value industry trends, competitive threats, and market opportunities.  When you’re good at this, you have an agile organization.  And you win.

Conclusion

Any agile “maturity model”, irony aside, needs to be structured to help organizations progress through the stages of enlightened operations – continuously providing insights into “what’s next?” for those teams to grow.

I know there are many readers here with years of agile experience – how would you improve the list above, to better provide a framework for “what’s next?”  When that framework is solid, we can do some wordsmithing and repackage it as a maturity model.  Until then, just focus on “what’s next” – that’s all a maturity model should be used for anyway.

When defining requirements, you always start in the context of a goal – either a user goal or a corporate goal.  You need to be aware of both.  Having a positive user experience is important, and requires a user-centered understanding.  Achieving your corporate goals might be in conflict with some user goals.

User Goals

A user centered, or user-centric approach to developing software is one where you start by identifying the key persona in your target market.  For each of those personas, you identify their most important goals.  You then identify the user stories or use cases that represent the things these people do in order to achieve their goals.  These “things” are manifested as practical goals.  For these activities, you design user interfaces (process, layout, architecture, look and feel, etc) that create customer delight for those users, doing those things.  This is how your product exceeds the suck-threshold (good enough that it doesn’t suck to use your product).

Corporate Goals

There’s a reason your company is creating or improving a product.  It may be to gain market share, or improve profitability, or increase sales.  Whatever it is, there’s a corporate goal that your decisions should be supporting.  The (subset of) corporate goals that are relevant to your product could be communicated in a vision and scope document, that constrains the domain of problems to be solved, and provides nuanced insights into viable approaches to solving them (example vision and scope).

Corporate goals are achieved when customers do things with the products.  These “things” are manifested as practical goals.  For those activities, you document what should be accomplishable and why.

Alignment and Conflict of Goals

Notice that both the user goals and corporate goals manifest in terms of people doing things.  When the person (user) wants to do the same things that your company wants them to do, you’re in alignment.  When the user goals and corporate goals suggest different activities, you’re in conflict.

You can see visually that these worlds collide at the “practical goals” stage  in the following diagram, from an article on combining interaction design and structured requirements principles.

It may be great – when the user and corporate goals are in alignment, the practical goal and associated scenario (activity) are easily defined.  What about when the goals are in conflict?

Vending Machine Example

Last Thursday morning, as I interacted with a vending machine, the idea for this article was dispensed along with my Diet Mountain Dew.  Yes, I know that’s odd.  Move on.

It occured to me that the process of buying a cold, caffeinated beverage can be both aligned and unaligned from the perspective of user and corporate goals.  If you imagine writing a use case for purchasing a beverage from the vending machine, your most important scenario is the one where a person wants to purchase a beverage that is available in the machine.

• User Goal: Get refreshed and vitalized.
• Corporate Goal: Sell a beverage.

I want to buy a Diet Mountain Dew, as a means to realize my personal goal.  The owner of the vending machine wants to realize her corporate goal of selling me the beverage I want to buy.  We’re in alignment.  Our requirements definitions and design decisions will be pretty easy – everything that increases the likelihood of and improves the experience of purchasing that beverage is good.  I put my money in the machine and push the button.

Then it occurs to me – there’s a situation where my personal goals are clearly not aligned with the corporate goals of the vending machine owner.  When there is no Diet Mountain Dew available to be purchased.

Consider the following procedure I’m forced to endure:

1. I view the available beverages that this machine dispenses – Diet Mountain Dew is one of them.
2. I put my money in the machine.
3. I push the button for Diet Mountain Dew.
4. The tiny display on the machine presents a message in taunting, scrolling red LED lights: SOLD OUT.

At this point, I’m faced with a choice – select a less desireable beverage, or request my money back and try to satisfy my user goal in some other way.

Consider an alternate procedure:

1. I view the available beverages that this machine dispenses – Diet Mountain Dew is one of them.
2. I view the ‘current availability’ of Diet Mountain Dew and see that this machine is currently sold out of Diet Mountain Dew.

At this point, I’m faced with a choice – select a less desireable beverage, or request my money back and try to satisfy my user goal in some other way.

The differences in these two possible procedures indicate that there is a conflict between the corporate goal and the user goal.  With the first procedure, the “Sell a beverage” goal is given more importance, because it makes me more likely to purchase a beverage that I don’t particularly want.  My utility is potentially sacrificed, while the owner of the vending machine is just as satisfied.

By failing to tell me that I can’t get what I want, until I’m further invested in the process, I am more likely to purchase something else.  That purchase creates just as much value for the owner of the vending machine, but less value for me.

If the vending machine owner had allowed me to use the second procedure, it would demonstrate that the  owner believed the improved customer experience, while risking the loss of this sale, would encourage me to purchase more over time.  The owner would have been prioritizing the requirements and design that supported the user goal ahead of those supporting the corporate goal – in hopes that my user loyalty would result in better word of mouth (more business from other people) and more business from me over time.

Generalizing User and Corporate Goal Conflicts

In this vending machine example, the annoying procedure is probably the right one – I’m going to treat the “point of use vending of cold, caffeinated beverages” as a commodity product.  This market has a very low barrier to entry, and relatively small problem being solved.  I’m not going to tell my friends about how much I love the vending machines from Joey’s Vending and encourage them to go out of their way to find and use those machines.  Nor will I purchase more Diet Mountain Dew because the emotional cost of the occasional “sold out” scenario is trivially smaller with Joey’s machines.  I would, however, be more likely to buy a Diet Pepsi when I’ve already invested time in the process and put my money in the machine – especially when the alternative is to find a water fountain.  I’m emotionally invested at this point.

However, this experience did jump out at me as one that is worth explicit consideration when defining requirements and designing solutions.  It does come up regularly.

The new iPhone 3GS hardware allows tethering (using the phone as an internet-connection device for your laptop).  ATT (the exclusive carrier for iPhones in the USA) is indicating that they will charge customers an extra monthly fee for this capability, when they eventually allow it on their network.  The user already has a data plan, and the traffic that ATT would have to carry is just “data.”  The user just wants it to work.  ATT, however, may want more money, or may want to limit data traffic on its network – perhaps to improve the quality of service to all customers.  This could be ATT’s corporate goal of “provide better service quality” conflicting with the user’s “increase the flexibilty of how I use my data plan.”  The former would likely influence customer satisfaction (and abandonment) over time, while the latter would influence customer acquisition rates.  Another conflict in goals.

Think about how different goals can lead to conflicting solutions, requirements, and designs.  And tell me when you’re out of Diet Mountain Dew before I’m emotionally invested in the purchase.

Creating a PERT estimate for a single task is both easy and straightforward. Creating an estimate for a set of tasks is still easy, but requires a little bit of math. Combining PERT estimates for tasks is easy, but not as obvious. Roll up your sleeves and dive in.

PERT Estimate Refresher

When estimating how long it will take you to complete a task, you shouldn’t estimate with a single value like “four hours.” “Four hours” does not provide enough information. Estimates reflect that there is uncertainty, and that single value does not give you any insights into how uncertain your estimate is. Your estimate could be “four hours plus or minus one hour” or it could be “at least four hours, maybe as much as sixteen hours.”

A PERT estimate, as we showed in our earlier PERT estimation article represents a distribution of likely effort for a particular task. To create a PERT estimate, you create three values -

1. The “best case” (shortest) amount of time it will take to complete the task.
2. The most likely amount of time it will take to complete the task.
3. The “worst case” (longest) amount of time it will take to complete the task.

A PERT estimate is presented in the form “Best Case / Most Likely Case / Worst Case.” If your estimate is “four plus or minus two hours” you would write 2/4/6 as a PERT estimate. Sharing “2/4/6″ as an estimate still doesn’t tell anyone else what you mean. A PERT estimate embodies some probability around completion time. You are precisely saying (for a 2/4/6 PERT estimate):

1. There is a less than 1% chance that this task will take less than 2 hours.
2. There is a 50% chance that this task will take less than 4 hours.
3. There is a greater than 99% chance that this task will take less than 6 hours.

This explicit statement of probabilities represents a distribution of likely outcomes. Statisticians would phrase it in a confusing (but mathematically important) way. They would say “if we sampled a large population of people (like you) doing this task, no more than 1% of them would complete the task in under 2 hours, no more than 1% of them would spend more than 6 hours on this task, and half of the people would complete the task in under 4 hours.” For the purpose of estimation, you can ignore the statistic-speak and just look at the “odds” of how long it will take you to complete the task once.

[larger image]

If you were to create a histogram of thousands of executions of the task, it would look like the diagram above.  This is the traditional bell curve shape we are used to associating with a normal distribution, but it actually represents a PERT estimate.  A PERT estimate is a distribution of possible outcomes, based on a beta distribution.  Another way to look at a PERT estimate is in terms of cumulative probability of a task being completed.  The same data in the graph above, when presented as a cumulative distribution function (CDF) looks like the following:

[larger image]

Here’s the rub – you don’t really know if the beta distribution is the right one.  Primarily because you can’t precisely know the actual probabilities.  So, you have to pick a distribution function to represent those probabilities.  There is debate about the right distribution function to use for estimation – check out this great article for hard core stats guys.  Here’s a cautionary warning from their conclusions:

Although bias stemming from misspecified activity time probability models is rarely mentioned in introductory discussions, we have seen several instances of this bias in simple examples. First, and perhaps most important is the uncertainty as regards the underlying activity time probability models. The literature offers no less than five procedures for translating the subjective estimates (a,m,b) into specific β-distributions. As shown, the methods lead to distinct β-distributions, and the PERT approximation need not satisfactorily estimate any of them.

Essentially, they are saying “you can’t really know the right shape for a distribution curve of possible outcomes – so don’t get carried away.”  And they then go on to suggest using a simpler model than the beta distribution for estimation.  Their suggestion is to use a triangular distribution (looks like a triangle, instead of a bell curve), because the math is easy.  With spreadsheets, we can still do “easy” math with a better distribution

The beta distribution for a symmetrical PERT estimate looks very much like a normal distribution.  You can see this by comparing the cumulative distribution function of the PERT and normal models for this 2/4/6 example.

[larger image]

Based on this similarity in distribution, and the inherent uncertainty in what the shape of the distribution really looks like, there are some benefits to treating the PERT estimate (2/4/6) as a normal distribution where the high and low values represent +/- 3 sigma bounding.  This approximation provides us some benefits when aggregating individual task estimates into combined project estimates.

Continued on the next page…