Best practices for user experience design and agile.  I don’t have the brainpower at the moment, or the experience and eloquence in general, to say it better than these guys.  So this week, I’m phoning it in, and deferring to these folks to say it far better than I can.

UXD and Agile

Twelve emerging best practices for adding UX work to Agile development, by Jeff Patton in 2008.  Incredibly comprehensive article here – both summarizing in breadth, and covering in depth.  If you’re interested in how user experience design (UXD) should work in an agile environment, this is the article you need to read.  If you’re really interested in UXD and agile – this is where you need to start!

Scrum and UX – a presentation by Morten Just from 2009.  Morten tells a great story about how many of the different disciplines within the “catchall” of UXD can fit in the world of projects being run in the Scrum framework.

My soundbite addition:

I feel like UXD is such a broad term that you can’t simply answer “how does UXD work in Scrum?” (a question I was asked today).  Some elements (outputs traditionally labeled as UXD) represent requirements (what should people be doing, what should some of the acceptance criteria be), and others represent design (how should things work / look / feel / exist, given a solution approach).

Every team will be organized differently, and parts of UXD that are “requirements” should be driven – in Scrum – by the product owner and UXD professional working together, expressing the need to enable the right stories with “good (big picture) design.”  The parts of UXD that are “design” should be handled inside the team, and the product owner (and doubly-so the stakeholders) should rely on their trust of the Scrum team to include “good UXD” as part of their design of solutions that enable stories.

As I mentioned in the pre-amble, Jeff and Morten do a much better job of articulating this point of view, so this article is 99% “go read their stuff” and 1% “my mental model that some of it should happen as part of expressing what needs to be done, and some of it should happen as expressing how it should be done.”

A picture is worth a thousand words.  A prototype is worth a thousand lines of code.  Two key elements of product management – and of agile development are elicitation and feedback.  Low fidelity artifacts can significantly improve both.  Polished, codified prototypes can create problems that prevent you from getting the benefits of communication.

Prototyping Anti-Pattern

David Bernstein has three good quick-read articles about prototyping in the agile zone.  The first one depicts the primary anti-pattern of prototyping – mis-set expectations.

As I was walking him through the different screens I could see he was starting to get uneasy. As I talked I could see him becoming paler and tenser, as if he saw a ghost. After a while I asked him if he was ok. He finally said, “You mean to tell me that I just wrote you a check for over $100,000 for less than a week of work?”

Prototyping Caveat

The second article is quick reminder of one of the goals of a prototype.

The purpose of a prototype is to elicit feedback from others or verify a design approach will work. When doing this we generally only concern ourselves with the “happy path” through our code.

A Prototype is Not a Product

David’s third article explores a little more about the mis-setting of expectations, and provides a good parallel from the film industry.

Prototypes are rough sketches without the robustness of a final product and it is often confusing to users when we show them a half-baked version of their project for feedback, even though they know it is not yet finished.

Agile Prototyping on a Napkin

David’s advice is (good and) primarily about using low-fidelity prototypes to avoid disrupting the elicitation and feedback process.  Folks in the user experience community have known this for a long time, and adapted their processes accordingly.  Back in 2006, I wrote about prototype fidelity as part of exploring ways to use this insight in requirements gathering.  Prototyping, in particular, is a great way to elicit implicit requirements – those unspoken (“you should have asked!”) requirements that your customer or stakeholder assumed you knew, or didn’t think to tell you.

Multiple Levels of Interaction

David focused on the initial “will this approach work?” elements of getting feedback on a design – and by inference, some low-fidelity user acceptance testing that the proposed approach will meet your customer’s requirements.  Jan Miksovsky wrote an excellent article (also in 2006) that provides excellent guidance on how much fidelity to build into your prototype, depending on where you are in the design process.  This is important – the type of feedback you need at different stages of your design process varies.  Jan proposes that the first prototypes be rough sketches, just as David points out.  Jan goes on to show how and when to add additional fidelity to your prototypes (read Jan’s article to see his great visual examples).  Like I said, the user experience community has known about this for a long time.

Prototypes Are For Two-Way Communication

The key element, and the reason for creating prototypes, is to get two-way communication.  A prototype is not just a status update about your design.

A prototype is the start of a conversation.

[thanks uninen for the image]

Prototyping is not only useful for design conversations, it is critical to understanding your requirements.  Yes, a prototype will you get feedback about the design-execution of your proposed solution.  More importantly, a prototype can help you make sure you’re solving the right market problems.

Prototypes Are More Than Interface Mock-Ups

The first thing we think about, and everything you’ve read so far in this article, is about getting feedback on the user interface of the product.  Prototypes are also useful for gathering business rules*, understanding system complexity, and defining, clarifying, and validating requirements.

*In that article, I show prototypes as “not being useful” for gathering business rules.  At that time, I was thinking in terms of interface mock-ups only, not other prototypes.

Use interface mock-ups when you need feedback about user interaction.  Use other artifacts when you need feedback about other aspects of your product.  You can use prototypes as an active-listening technique when gathering market data too.  There are lots of ways to draw stuff to help with communication.

I’ve regularly used flow charts to prototype processes.  Within those flow-charts, which are initially prototypes of how the product will behave, are decision diamonds.  Those decision diamonds prototype the enforcement of business rules within the to-be-created product.  In the spirit of DRY (don’t repeat yourself), a quick polishing of your process diagram can serve as a persistent artifact of the requirements.  Just like a user interface mock-up.

In more complicated scenarios, I’ve also used UML statecharts and data flow diagrams to prototype behavior.

In the section on up-front planning in last week’s article on risk management and release cadence, I talked about the mental leap that people need to make to envision a product that doesn’t yet exist.  Prototypes are like big springboards that help people leap that chasm of imagination.

Important Safety Tip

Don’t use the wrong prototype (mock-up, flow chart, etc) for the wrong conversation or with the wrong stakeholder.  Showing when and where business rules are being enforced (in the process being designed) is great for getting feedback about your interpretation and potential embodiment of those rules.  It will be a disaster if you try and have this conversation with someone who is not the owner of those policies – like a representative user.  Sometimes, policies are “owned” by non-technical people who struggle to read diagrams.  You may have to walk them through how the diagram works.  They’re smart – they’ll get it.

Just remember – use the right prototype to emphasize the right ideas, and elicit the right feedback.

I’ve been thinking about the software development process.  Big, upfront, design and requirements.  User research and analysis.  Market insights, gained on exploration or over time.  Release cadence – how quickly you get, and incorporate, feedback from your customers about your product.  How quickly you react to your competitors’ reactions to your actions.

Big and Up-Front

Leander Kahney published the full transcript of an interview with John Sculley, former CEO of Apple, last week.  Of particular interest are the discussions about how Steve Jobs is a designer first, CEO second.

He always looked at things from the perspective of what was the user’s experience going to be? But unlike a lot of people in product marketing in those days, who would go out and do consumer testing, asking people, “What did they want?” Steve didn’t believe in that.

John Sculley interview

The theme of understanding what users want and need comes up repeatedly in the article.  A few years ago, Alan Cooper and Kent Beck got in a debate about the merits of interaction-design-driven versus emergent-design software development.  Four years later, I don’t accept the premise that you have to choose between the two.  You can treat both decisions independently and make the right choice for your team and market.

Thanks Jim Holland (also PM Tribe author), for bringing this article from Wayne Mulligan (guesting on On PM) to our attention today.  It’s a great read, and the money-quote from Wayne about going back to first-principles is:

And that’s the biggest lesson I learned here – you can reap all of the benefits of an agile product development approach without following the rules to the letter.  It’s the spirit of the manifesto that counts and not the precise implementation.

4 Ways that Agile Methods Brought Sanity to My Company

In the spirit of that reality – I believe decoupling how do you plan? from how do you deploy? is fair game.

The Risk of Being (Big Picture) Wrong

[larger image]

When you do no upfront analysis at all, you have complete risk of creating the wrong product.  The more analysis you do, the more likely you are to create a product that solves the right problems – problems that your target customers, in your target market, are willing to pay to solve.  The units in the graphs in this article can be mostly ignored – all you can really capture, when talking generally about this is the trending.  The more time you spend understanding your market, the less understanding you gain.  There is definitely a law of diminishing returns.  This effect is one of the reasons analysis paralysis is so bad.  With analysis paralysis, you delay the launch of your product (at some cost), for no appreciable gain.

The two curves are labeled complex market, and simple market.  The idea here is that the easier it is to understand your market, the less benefit you get from analyzing it.  Complexity can be in the form of competitive concentration, nuances of problems being solved, specifics of the customers for whom you are solving the problem, and even the notion of just how innovative your solution is (think cars, when horses were common, or smart phones in a world of land-lines).

As a product manager, you have to act with imperfect information.  Even if you try and uncover every relevant piece of information (which you can’t do), you will misunderstand some things.  You will develop a mental model of your customers’ problems – and that model will be wrong.  You will also predict competitive responses incorrectly.  To top it off, you’ll make mistakes about “what’s next?” for your customers.

That does not mean you should do no analysis – it means you should not do too much analysis.

Not all product managers are equally good at distilling insights from the vapor of nuanced market data.  Great product managers will have (correct) epiphanies about the important problems to solve, and the impacts of disruption on the market.  Others will not.  The better you are at having these insights, the faster you reduce risk – and the faster you reach the point of diminishing returns of additional analysis.  Some product managers and markets may require weeks of analysis, others may require days.

The Risk of Being (Execution) Wrong

Another great quote from the Sculley interview:

[Jobs] said, “How can I possibly ask somebody what a graphics-based computer ought to be when they have no idea what a graphic based computer is? No one has ever seen one before.” He believed that showing someone a calculator, for example, would not give them any indication as to where the computer was going to go because it was just too big a leap.

John Sculley

In The Design of Design, Frederick Brooks discusses the debate between empiricists and rationalists.  Empiricists, in product creation, emphasize feedback as the means of gaining insight about what your product should be.  Rationalists use intuition and deduction to presuppose what a product should be.  In the hyperbole of many agile-waterfall debates, empiricists are burdened with the impossibility of designing the right product in advance, while rationalists struggle under the yoke of being wrong from the start and wasting their efforts up until the first failed product release.

Obviously, neither of these extreme perspectives is wholly true – while both have significant elements of truth.

True insights about your market grow stale with time.  The damage from erroneous conclusions about your market grows over time.  The risk of creating the wrong product grows over time.

[larger image]

If you released continuously, including getting feedback continuously, you would face the minimal possible risk that the market has changed since last you checked.  You would also maximize the opportunities to correct mistaken conclusions – thus minimizing the risk of being wrong.

You reduce the size of this risk by getting feedback.  You get feedback by “releasing” your product to customers, either as a prototype or as a product.  Jobs (per Mr. Sculley) realizes the catch-22 of not being able to get inputs from others, until they can realize a version of your idea. Sculley attributes the following to Mr. Jobs: “There was no way to do consumer research on it so I had to go and create it and then show it to people and say now what do you think?“

The longer you wait to get feedback, the greater the cost of effort invested in the wrong ideas.  The longer you wait to get feedback, the greater the risk that your market has changed since you formed your ideas.

In stagnant markets, where problems and their solutions are well understood, and change happens slowly and competitors don’t innovate, you can release less frequently with less risk.  Until someone decides to innovate, and make your market more dynamic.  Markets with active competitors, customers who’s expectations change, and problems that evolve are more dynamic.  The risk of delays is greater.

Rationalism and Empiricism Can Co-Exist

My experience has been that both concepts – “think before you act” and “fail fast, get feedback” can happily co-exist.

• You can develop an understanding of your market, your customers, and the very real problems they face, before creating a product that solves those problems.
• Introducing a new product / solution into a market changes that market – so you have to revisit what you know at least as frequently as your market changes.
• Even when you understand the problem, you need feedback to know if the product is actually the solution.  The greater the “leap,” the more representative your prototype must be for people to give you feedback about the (eventual) product.

What have your experiences been?

Almost everything I’ve read about use cases focuses on describing what needs to be added to your product.  Agile development says “get it working first, make it better second.”  That means changing the way the software enables a user to do something they can already do.  How do you manage requirements for incremental improvement?

Iterative Development and Incremental Improvement

Iterative development is the process by which you release software in iterations – small, incrementally better versions of your software.  People usually think about this solely in terms of enable the most valuable capabilities first, then the next most valuable capability next.

That’s fine, when your product is new.  Eventually (or quickly!) you will reach the point when the next most valuable improvement is not to add a new capability, but rather, to make an existing capability more valuable.

Everything is an Upgrade

In a couple earlier articles about organizing migration projects and requirements for migration projects, I wrote about how migration is a bit of a misnomer.  Everything is a migration.

My argument in those earlier articles is that the problems you’re solving with your “new” solution are not new.  Your customers are currently solving them in different ways.  The relevant question for migration projects is around how much your users are changing the way they solve their problems.

[As a slight segue, failing to appreciate this distinction can lead you to a project that is based on a faulty problem statement.]

When you are delivering incremental improvements to your product, through iterative development, you are upgrading your software.  You are migrating your users from their old solution to their new solution.  It just so happens that you are replacing yourself, as you migrate them from the old version to the new one.

A viable strategy, that appeals to me personally, is to continuously innovate, disrupting your market, before someone else does.  With this approach, you are intentionally reinventing your solution, making it difficult for someone else to out-innovate you.  This is a great way to approach developing your roadmap strategically.

Your market will change.  Rapidly. Do you want to react to your competitors, or keep them on their heels while you stay on the balls of your feet?

The reverberations of the “new” Twitter still haven’t settled, and many of Twitter’s competitors are on their heels – some of them only now realizing that Twitter has always been a competitor and not just “a platform.”  The new Twitter doesn’t really change anything about how people use Twitter, except make it (markedly) better.

Avoiding Featuritis

When is it more valuable to improve what your software already enables, versus enabling users to accomplish more with your software?  Kathy Sierra introduced us to the concept of featuritis, the idea that too much is too much.

In an article on viral product management, I proposed that by making your software better, you can tap into an altruistic mechanism by which people will promote your product.  As an example, I proposed improving the usability of your software as a means to cross this viral tipping point.

One of the key ideas is that making it better for users makes your product better for your company.

Managing “Improvement” Requirements

Now that we all agree that there is value in making your software better [comment below if you disagree] the question becomes “How?!”

Use Cases can be used as the keystone to your requirements arch.  Sometimes, user stories are better than use cases, but use cases are perfectly agile.  When you’re starting a project, you want to apply an outside-in approach to defining the scope of that project – or more particularly, to define the scope of problems you are solving in your roadmap. This article will talk in the language of use cases, but identical concepts and approaches apply when using user stories as well.

When you have prioritized a use case as “the next most valuable capability to introduce” you add it to the backlog for your sprint.  The implementation team then provides feedback that use case is “too big” and you have to slice it up.  It is important that you don’t split the use case  in such a way that you only solve half the problem.

You also shouldn’t deliver a “bad” product.  In fact, the ideal way to introduce a new capability is to satisfice, and not introduce the “perfect” solution in your first iteration.  Make your first solution “good enough.”  Part of the art is defining “good enough,” but rest assured, it is not the same as “the best you can possibly do.”

Summarizing some key elements from above demonstrates the eventual reality you will face:

1. Every problem* your users face is already being solved, you are just providing a better solution.
2. There is diminishing, and eventually negative return on adding more capabilities to your product.
3. Your first release of a given capability will only be “good enough.”  That leaves room for improvement.

*Yes, you can pedantically define “the problem” as “the existing solution is not good enough” – but you still end up in the same place, so why bother?

Eventually, the value of improving something you’ve already released will be greater than the value of releasing something new.

What do you do?  You implement the same use case again.

If your product is Software as a Service (SaaS), you absolutely should be thinking about things this way.  Becoming better is a continuous improvement objective that is implicit in the economics of SaaS products.

Revisiting Use Cases

Not to be confused with re-use of use cases, revisiting a use case is specifically revisiting – with the goal of improving – the implementation that is already in place within your product.  This is a special case of a migration project – you’re migrating your users from your old solution to your new and improved solution.

You may be migrating to an identical process, perhaps with faster performance than your previous release was capable of.  Or you may be improving the procedure (improving usability, interaction design, visual design, etc) without affecting the process. Generally, a “make it better” use case improvement effort will have no more than a minor process impact.  Your users are still doing the same thing, they are just enjoying it more, or doing it more effectively.

When slicing use cases to make them fit within a single sprint, you may design them for a single user persona in the first release, knowing that you won’t meet the expectations of a different user persona until the next release.  In this case, your non-functional requirements, constraints, or acceptance criteria will be different.  Your use case may also be different (in the details) while appearing to be the same.  This also common scenario can be addressed with the same approach.

Find that old use case (from several iterations ago) and put it back in the backlog.  Remember, agile is about conversation, not artifacts.  If you need to add an explanation to the use case, because your team fixates on the fact that it is “already working,” add a qualifier that it needs to be “better.”  Then have a conversation, and explain how it needs to be better.

Is Design Part of Implementation?

Some teams organize such that designers (both architects designing code and designers designing interfaces are being addressed here) are part of the implementation team.  Other companies treat designers as stakeholders – providing guidance and input to the product managers and product owners.

When a “new design” is being requested from outside the team, include that design guidance as part of the input to the implementation team – through artifact and / or clarification.

When a “new design” is being requested of the team (the designer is part of the team), express the new acceptance criteria to the team.  Note that these need to be measurable requirements if they are to be considered good requirements.

Summary

There will come a time when the most valuable thing you can do is improve the user experience for a capability your product already embodies.  When that time comes, use the same use case again, updated with the new constraints, non-functional requirements, and acceptance criteria.

I came across a really interesting article LukeW.com, showing how making changes to the way an input form on a website increased interaction by 25 to 40%. The changes reflect the value of thinking outside-in, investing in user experience, and performance measurement.

Bonus: the idea is cool.

Mad Libs Input Form

[full size image available in Luke's article]

The idea being presented is to replace the old boring web input form designed for a computer. The new, fun form is a fill-in-the-blank (aka Mad Libs) layout. The article is on Luke Wroblewski’s site.  The team at Vast.com created and tested a version of this form for the Kelly Blue Book site. [Luke cites Huffduffer.com as the first place where he saw this technique.] Thanks, Luke for sharing this with all of us!

Outside-In Development

Product managers are acutely aware of the need to solve market problems. We are regularly reminded that we’re creating solutions for people in our markets who use our products to solve their problems. When we write requirements, we avoid writing design specifications, and thereby lose the ability to enforce that the proposed solutions also take an outside-in approach. We maintain an outside-in perspective, but we lost the ability to influence an outside-in aesthetic.

Note: Outside-In Software Development is a great book, you should read it.

That’s one area where user experience works well in concert with product management – assuring that the same drivers of what to build are informing the design of how it is built.

The genius (or at least elegance) of this Mad Libs form from Vast / Kelly Blue Book is that it humanizes the experience of requesting that a nearby dealer contact you to discuss a possible purchase of a car from them. The forms we’ve all had to fill out on countless web sites are very inside-out. They expose the inner workings of the program – “gather this info, and that info, and this other info.” Those forms force users to do what the program wants. Blech. The form that Ron Kurti designed, however, forces the program to do what the users want. Huzzah!

Bad Requirements Prevent Elegance

I’ve written about the importance of writing design-free requirements several times.  Avoiding design constraints in your requirements is even one of the pillars of the big 10 rules of writing requirements.

If you had written the requirements for this contact the dealer page, would you have specified the design of the form?

Most of the requirements analysts I’ve worked with would have. I’ve even worked with companies that have developed templates / stencils defining how these interface specifications must be documented – requiring that all the fields be aligned, with specific pixel gaps, etc.

If you had specified the design (read: limited the choices of the designer), you would have prevented this solution.

Mr. Kurti’s elegant solution is clearly better.

One way to write the requirements that would not have prevented this solution would be with a user story and acceptance criteria:

As a car-shopper, I want to engage a car dealer, once every few years, so that I can purchase the car I selected.

• The car-shopper can provide his contact info (name, email, phone) for the car dealer.
• The system will display the information identifying the selected vehicle.
• The system will display the information identifying the dealer being contacted.
• The system will include an opt-in newsletter-subscription option.
• The system will notify the specified dealer when the car-shopper indicates.

The user story nudges the developers into an outside-in perspective by emphasizing the user’s goal. The acceptance criteria make it clear that any solution that meets those criteria is acceptable to the product manager. It allows the interaction designer to create a compelling solution, rather than forcing him to recreate a boring experience.

Measurement Rocks

The Vast team measured the impact of making this change to their form, and saw between a 25% and 40% increase in conversion (people contacting dealers versus people abandoning the process when they see the form). That’s a lot more leads for the dealers, and presumably a lot more money for Kelly Blue Book and Vast. This is a great example of how you can measure the impact of investing in user experience, because it should spark ideas for you.

What can you measure and test and improve?

Now that’s ROI for you.

Design-Free requirements are important for two reasons, and hard for two other reasons.

Design-free requirements are hard because you “know what you want” when you should be documenting “why you want it.”  Writing design-free requirements can be hard when you don’t trust your development team to “do the right thing” even though it is not your job to design the solution.

Design-Free Requirements – Revisiting

It has been three years since I wrote Writing Design-Free Requirements as part of The Big Ten Rules of Writing Requirements.  In that time, agile development practices have moved from being an esoteric development methodology to being the topic on the tips of everyone’s tongues as executives and organizations try to either (1) get the benefits of the state of the art in software-development process or (2) do something shiny and fashionable.

The previous article centered on elements of designs within MRD, PRD, and SRS artifacts.  A regular alphabet soup of artifacts.  Honoring the values behind the agile manifesto encourages us to emphasize working software over comprehensive documentation.  In that light, we’ll take a “what is needed” approach to talking about how requirements, design, and implementation are all needed – rather than issue an edict about where that information should be captured.

Agile Development Inputs

When creating software, someone needs to know:

• What problems are being solved, and how important (valuable) they are to be solved.
• Who has the problems and who is using the software to (help) solve those problems.
• What constraints limit the space that defines the universe of possible viable solutions.
• What acceptance criteria define if the delivered solution will be acceptable.

Subject to those inputs, someone needs to make design decisions:

• User experience design - what interactions will a user of our solution love?
• Program design – how (in a nuts and bolts way) will our solution function?

When talking about agile and talking about design, we should take a look at how Kent Beck and Alan Cooper, as respective though leaders in each space, view this.

Cooper doesn’t talk much about creating the requirements to support the daily use scenarios – he proposes moving directly into design of the solution. This differs from the more traditional technique of writing functional requirements to support use cases. Cooper also breaks down design into two components – program design and interaction design. Program design is everything you don’t see. Interaction design is everything you do see.

[...]

Cooper argues that designing the interaction should happen before any code is written. He uses a construction analogy to drive home his perspective – you can’t pour the concrete before you build the forms. Kent Beck, founder of the XP programming philosophy disagrees with the premise. Beck believes that the cost of changing software is low, and the imagery Cooper uses is hyperbole. We touch on, and link to that debate in this post.

Interaction Design Process Overview

I don’t believe that Beck is arguing for “don’t do design” – I believe he is arguing for “don’t do big upfront design that would delay implementation.”  He’s championing the agile values that emphasize creating working software and responding to change.  I can imagine him saying “no one will buy a design, they want a solution.”  Cooper’s point is that create a solution without first understanding how someone will use it, you can’t create a great product.

Program design has many hidden impacts – cost of maintenance, cost to change, and the performance of the solution.  You can have a design (or architecture) that makes everything easier to do in the future – at the cost of delaying the delivery of anything.  Or you can have a design that minimizes the time to deliver the first thing, while increasing the cost to deliver any particular next thing.  Or you can design a solution that falls somewhere between those extremes.

If you say “we don’t do design” you’re lying.  Every solution includes design.  Your team’s design process may be “big and upfront” or it could be a couple sketches on a whiteboard, or some email exchanges.  You may create storyboards and wireframes.  Or design may be the thing that happens right before the programmer’s fingers strike the keys.  Design may be explicit or it may be implicit – but it never “doesn’t happen.”

In the movie Amadeus, Salieri is astounded that there are no rough drafts of Mozart’s compositions.  That’s because Mozart did the design in his head, not because he didn’t do design.  I’ve worked with programmers who’s “implicit designs” were great, but that is the exception, not the rule.

Who Owns Design?

Since design happens sometime before fingers strike the keyboard, the real question is – “who owns the design?”

• You have a product manager developing an understanding of market problems, and prioritizing the problems that should be solved with your product.
• You may have a product owner managing the backlog and clarifying those requirements (problems) for the development team.
• You may have an interface (or interaction) designer or team of designers who are broadly responsible for “how users interact with our products.”
• You may have an architect or lead engineer who is responsible for the “big picture design” of how your product works.
• You have developers and testers who are responsible for delivering your product. [Note: coding without testing is like typing code without compiling - maybe it works, but probably not.]

You may not have distinct individuals with each of these titles – every small team I’ve worked with has people wearing multiple hats.  This is especially true when you have an agile team full of specializing generalists – any given story (or task) may have a different architect and different implementer.  I’ve only worked with one company where the architects are NOT part of the development team.  If your team is set up that way, please comment below – I had never seen that before, and I’m not convinced that it is the best way to organize – what have your experiences been?

Generally, “program design” is clearly owned by the development team – and product managers (and product owners) know better than to specify program design in their requirements.  Neither the lead engineer nor the product manager believes that the product manager is a better programmer – so the product manager better not be specifying program design and mislabeling it as requirements.  Coincidentally, Steve Johnson, at Pragmatic Marketing has a post running right now with a bit of a quiz – is this a req(uirment) or a spec (design)?

Where the line is more blurred is around interface and/or interaction design.  Some development teams have interface designers as part of the team.  Some companies organize with interface design as a “shared service” within the company.  Either approach can be the “right one” – it depends on too many details to make a sweeping generalization.  When the designers are members of the development team, the solution (from a product management perspective) is the same – the “team” is responsible for all design.  When the designers are not part of the development team, the developers have to consume two sets of guidance – “solve this problem” from product management, and “the solution needs to look like / act like this” from the designers.

Collaboration between product management and user experience people is the ideal solution.  The “requirements” and “design” inputs to the development team are comprehensive and consistent.

Design-Free Requirements

There are benefits – especially when being agile and minimizing documentation – to delivering requirements and design at the same time.  Just don’t do it by embedding design constraints within the requirements.

• When people on your team misinterpret design as requirements, they are unnecessarily constrained.
• As a product manager – are you the best designer on your team?  If you’re busy designing, who’s product managing?

This is trickiest when writing use cases – sequencing a set of interactions is interaction design.  That is one benefit of writing a user story instead of a use case.  An approach that has worked well for me with multiple teams is to deliver user stories (requirements) combined with storyboards (interaction design) and wireframes (interface design).  When details are needed (usually when “changing” versus “creating” an interface), screenshots can replace wireframes.  When business processes are complicated, process flows can replace storyboards.

Just don’t embed the designs within the requirements.

Your company is building out a toolkit to support third-party developers.  You’ll need a bunch of different types of widgets – combo-boxes, text entry fields, domain-specific controls, etc.  You’ve got a long list of desired controls from your customers.  You’re agile.  What do you build first?

Agile In A Soundbite

Being agile is about delivering incremental value, quickly, getting feedback, and then delivering more incremental value.  Repeat until “done.”  Good agile adds a qualifier – do the most valuable thing quickly, get feedback, do the most valuable thing (that has not already been done) quickly. Better agile optimizes the rate at which you deliver value by taking into account both benefit and cost.  Great agile overlays a focus on getting better at doing all of these things while you do them – becoming a learning organization.

Boiling the Ocean

A product manager I was coaching was faced with a challenge commonly referred to as boiling the ocean. His team was tasked with solving a market problem, and they were constrained to doing it with a service-oriented architecture that exposed a set of widgets (user interface controls) that customers could easily integrate into their products.  This approach was designed to provide competitive differentiation by reducing the time and cost to deploy solutions that allowed customers to integrate his product into their existing platforms.

In initial conversations with customers, technologists, and architects, this product manager quickly amassed a list of desired widgets (controls) and scenarios (stories) in which they could be used.  The product manager’s team had recently switched to an agile development methodology.  The internal stakeholders, not yet accustomed to agile development, wanted “all of the widgets,” now.

This product manager was able to convince them that the team would deliver the widgets incrementally, following agile principles.

His question was – How do I sequence the widgets in the backlog?

Defining Widget Priority

The product manager had a list of widgets, combo-boxes, data-grids, text fields, radio buttons, etc., and for each widget, he had a real-world scenario showing how the widget could be used.  Most of the scenarios involved customers using multiple widgets.  He wondered if he should do some sort of analysis that detailed the frequency of use of each widget.  ”This widget is used in 7 scenarios, so it should be done first; this widget is used in 5 scenarios…”

There was a strong temptation to add several “Create a widget” tasks to his backlog.  It would be easy for his development team to estimate the effort required to deliver each widget.  His team wanted to deliver incrementally, and “one widget at a time” felt like logical, discrete chunks of work to them.  They could easily sink their teeth into estimating, building, and testing each widget.

A quick reminder of a main tenet of agile, delivering incremental value, illuminated the flaw in this approach.  This approach would have been an inside-out prioritization, when value delivery requires an outside-in perspective. (See the ‘recommended reading’ suggestions on this page to check out Kessler and Sweitzer’s great book.)

If the team used this approach, after the first widget was complete, they would be able to deliver exactly 1 task, and 0 stories.  Development teams don’t deliver tasks, however.  The team’s customers would not be able to get incremental value from having a single widget.  The team would have delivered seven incomplete stories – so they would have delivered no stories at all!

We reworked his approach as follows:

1. Assess a value for each story, making sure that each story would enable his customers to accomplish something valuable.
2. Engage their user interface / user experience designer to design a solution for the most valuable story.  This design was constrained to use widgets in the user interface.  The suggested way to communicate this design was with a storyboard and low-fidelity wireframes.
3. Identify the widgets needed to be built to deliver that story, using the designer’s design.
4. Deliver the story to the development team, including the storyboard, wireframes, and list of widgets to be used as acceptance criteria.
5. Get the development team to size (estimate) the effort to complete each story.
6. Where stories were too big (epics), collaborate to identify good ways to break up the stories into manageable chunks.
7. Repeat steps 2 through 6 until the amount of identified effort was likely to fill the release (keep the dev team busy delivering value).
8. As the team delivers each story, get feedback from the market, revisit the prioritization, and revise the “next” story.

What we discovered was that a scenario like the following (modified for this article) played out:

• The first story required two widgets.  As no widgets existed, both had to be built.
• The second story required three widgets, but two of them had been built to support the first story – only one incremental widget had to be created.
• The third story used one of the widgets from the first story, but required additional behaviors.  The original widget was modified to provide incremental capabilities (and value).

A Note On Team Structure

Astute readers will notice that there was a design step “inside the requirements process” – before the stories were delivered to the development team.  Technically, that is true, for the way this particular company was organized.  The user experience designer was not a member of the scrum team, but rather, an external consultant who supported multiple teams.  The product manager engaged the designer prior to engaging the development teams.

This just as easily could have happened as follows:

1. Product manager identifies, values, and prioritizes stories to be added to the backlog.
2. Development team, as part of sizing the stories, engages their user experience designer to select the appropriate widgets, and those design decisions inform the estimation process.
3. Everything else is the same.

This procedural variation does not have design “inside the requirements process.”  The same people do the same things, in the same sequence in this scenario.  The only difference is that the interface / interaction designer is a member of the development team.  That would be ideal, but that was not how the company was organized.

In this particular example, having the product manager collaborate with the UX designer made the most sense.  It introduced less complexity for the product manager to accept responsibility for this activity than it would have for the development team (who was still new to using an agile delivery cadence) to do it.

Conclusion

The key ideas at play here:

• Focus on realizable value to the customer (outside-in development), not tractable tasks (inside-out development).
• Collaboration with a UX professional is key to driving “interface requirements.”
• Deliver frequently and valuably, get feedback (learn), and incorporate that knowledge into whatever is next.

Perpetually intermediate (competent) users.  Users who briefly exist as novice users and never become experts. Most of your users are competent, and you should design for them.  Competent users have different needs and different expectations than novice or expert users.  How do you know your user’s competency levels, so you can design for them?

User Competency

User competency is a concept I first read about in Alan Cooper’s The Inmates Are Running The Asylum.  Cooper’s contention is that the level of expertise of your users follows a bell curve, or normal distribution.

When we’re designing software we need to keep in mind that most of our users will be competent – neither experts nor beginners. Alan Cooper’s studies tell us that user skill levels follow a bell curve. He talks about competent users as perpetual intermediates. Some users drop out of the bell curve when they stop using our software. The rare user becomes an expert. Most users only learn enough to get their real job done.
Competent Users and Software Requirements

Cooper contends that the combination of learning curves and natural user tendencies to stop learning or abandon a software application are the sources of this distribution of user competency.

Experience Curves

Experience curves represent the diminishing costs over time of manufacturing something repeatedly.  The process of manufacturing something gets more efficient as you get smarter about the process.  The process of using software is at least analogous to, if not a specific example of a manufacturing process.  If you’re using an email application, you are manufacturing email messages.  In a CRM system, you are manufacturing contacts, or contact reports, etc.

By treating any “using the software to do something” interactions as a process, you can measure the cost (how long it takes) of the user’s interactions.  Applying the math behind experience curves, you can predict the reduction in cost (to your users) over time, for any set of interactions.  Experience curves take into account that some processes are inherently more learnable than others.  This property of learnability is reflected as an efficiency coefficient – how efficiently someone can learn ways to reduce the cost (time) needed to perform the interactions.

This gives us an approach to quantitatively model user competency.  Having a definition allows us to model competence.  And measuring competence allows us to manage product design in the context of user competency - designing for competent users.

Defining Competence

The first step to measuring competency is to define the model.  I am proposing a definition in this article that I suspect will yield insights (to help us manage our products). I was unable to find any quantified definitions of competence, when researching it as part of a client engagement.  If you have, or know of a model, please share it in the discussion below this article.

• A competent user is someone who learns to perform a task in half the time it initially took them.
• An expert user is someone who can complete a task in 10% of the initial time.

This definition is guided by an expectation that Alan Cooper’s premise about perpetually intermediate users is true.  Being a novice user is a very transient state, and becoming an expert is very infrequent.  The goal of the definition is to be able to segment your users and make well-informed design decisions.

A Proposal, Not a Doctrine

This is a proposal that doubling performance reflects competence, and achieving a ten-times improvement represents expertise.  It may be that some different measure of performance improvement more accurately reflects competence and expertise.  We have to test it to know.

The experience curve is defined mathematically by Henderson’s Law.  It states that the time to complete a task is a function of the number of times you have previously done that task, adjusted by the “elasticity” of the cost of that task.  In other words, some tasks are easier to improve than others.  If you populate a table with the results of applying Henderson’s Law, you get the following:

[larger image]

• Each row in the table represents the Nth repetition of a task.
• Each column represents how easy that task is to learn – progressing from “hard to improve” on the left, to “easy to improve” on the right.
• Each cell in the table represents how long it would take to perform the Nth repetition of the task, as a function of how easy it is to improve your performance at the task.

The above definitions represent what experience curves predict mathematically, when the task initially takes 60 seconds.  The following definitions reflect the proposed user competency model:

• A user is a novice user until she has learned enough to cut the time-on-task in half.  These cells have a white background (and are in the upper left area of the table).
• A user is a competent user when the time needed to complete the task is between one half and one tenth of the initial time.  These cells are shown with a yellow background (and are in the central area of the table).
• A user is an expert user when she can complete the task in less than one tenth of the time required to initially complete the task.  Theses cells are shown with a red background (and are in the lower right area of the table).

In the absence of empirical data, I used my intuition to suggest that a representative experience curve for a typical task performed in software would be one with an “elasticity” of 50%.  For a task with those learning characteristics:

• A novice user would cut the needed time in half on the 4th repetition of the task, and would be considered to be a competent user.
• A competent user would further reduce the time needed to one tenth of the initial time on the 100th repetition of the task, and would be considered an expert user.
• The 50% elasticity column is surrounded by a black border, and the number of repetitions required to advance to competent or expert status is also highlighted with a black border.

Inferring Competence

Since different processes have different learning characteristics, you have to figure out how easy it is for your users to improve at your processes.  To do that, you have to study (or at least measure) your users’ interactions with your software.  In the 50% curve highlighted above, a user is capable of cutting their time-on-task in half by the fourth time they perform the task.

If data from your initial testing (or measurement) reveals this to be true, then you have selected the correct curve (the correct column in the table).  If it takes more or less time to reach this level of improvement, shift to the left or right to find the appropriate curve.  If software-interactions are reasonable analogs to manufacturing processes, then the experience curve projects an expected rate of improvement on task.

The following graph isolates the time-on-task data for a user who is learning to improve when repeating a task (process) that matches the 50% elasticity curve.

[larger image]

Note that the number of repetitions of the task (the X axis) is represented as a logarithmic scale.  The data points along the curve correspond to the cell values in the table above (for the 50% column).

Shades of Gray

One nice thing about this quantitative approach to inferring competency by measuring usage is that your measurements are per-process.  Users are not “purely novice” or “purely expert” – they can be experts at some processes, while remaining neophytes at other.  There is also awareness, for any particular process, of “how much competency” a user has.  This allows you to refine your assumptions of the steepness of the learning curve, and of the thresholds (doubled performance and ten-times performance improvements).

Improvement Over Time

Any particular learning curve can be considered relative to calendar time, to see how quickly a user will progress along the curve (as a function of frequency of use).  This can be useful for determining the ROI of improvement in a particular process.

The following graph shows how an 80% learning curve overlays a calendar for tasks that happen daily, weekly, and hourly.

[larger image]

The graph shows that for a weekly frequency, after 16 weeks, the task time has reduced from 300 seconds to 100 seconds. With a daily frequency, the task time is even lower – about 60 seconds. This graph shows nothing other than converting the academic learning curve graph into one that incorporates calendar time and frequency of occurrence.

Software Usability and Learning Curves

One approach to inferring user competency would be to measure how long a user has been using your software.  The variation in how frequently different users perform the same task will introduce an error into that inference.  You can avoid introducing that error into your modeling by counting the number of times a user has performed a task.

Applying the User Competency Model

The advice in previous articles, and from Cooper’s book, and from this great article on the  Coding Horror site, encourages us to focus on the competent users.

I’m working with a client who needs to prioritize a set of capabilities and establish design principles for a product.  We will incorporate this user competency model as part of our analysis.

Hopefully we’ll have an opportunity to collect data to validate and / or refine the model.  I’m proposing that we first gain some insight into the which users (novice, competent, expert) drive the most revenue and profit from use of the product – to establish the importance of each category of user.

For this product, I suspect that we will find many more novice users than a normal distribution would predict.  If that is true, the next question will be to understand if we are dealing with a normal behavioral dynamic, or if characteristics of the current product “force” novice users to abandon it before they achieve competence.

Either way, we will have a framework for prioritizing the goals of the novice, competent, and expert users.

How would you apply a model like this to improving your product?

When defining requirements, you always start in the context of a goal – either a user goal or a corporate goal.  You need to be aware of both.  Having a positive user experience is important, and requires a user-centered understanding.  Achieving your corporate goals might be in conflict with some user goals.

User Goals

A user centered, or user-centric approach to developing software is one where you start by identifying the key persona in your target market.  For each of those personas, you identify their most important goals.  You then identify the user stories or use cases that represent the things these people do in order to achieve their goals.  These “things” are manifested as practical goals.  For these activities, you design user interfaces (process, layout, architecture, look and feel, etc) that create customer delight for those users, doing those things.  This is how your product exceeds the suck-threshold (good enough that it doesn’t suck to use your product).

Corporate Goals

There’s a reason your company is creating or improving a product.  It may be to gain market share, or improve profitability, or increase sales.  Whatever it is, there’s a corporate goal that your decisions should be supporting.  The (subset of) corporate goals that are relevant to your product could be communicated in a vision and scope document, that constrains the domain of problems to be solved, and provides nuanced insights into viable approaches to solving them (example vision and scope).

Corporate goals are achieved when customers do things with the products.  These “things” are manifested as practical goals.  For those activities, you document what should be accomplishable and why.

Alignment and Conflict of Goals

Notice that both the user goals and corporate goals manifest in terms of people doing things.  When the person (user) wants to do the same things that your company wants them to do, you’re in alignment.  When the user goals and corporate goals suggest different activities, you’re in conflict.

You can see visually that these worlds collide at the “practical goals” stage  in the following diagram, from an article on combining interaction design and structured requirements principles.

It may be great – when the user and corporate goals are in alignment, the practical goal and associated scenario (activity) are easily defined.  What about when the goals are in conflict?

Vending Machine Example

Last Thursday morning, as I interacted with a vending machine, the idea for this article was dispensed along with my Diet Mountain Dew.  Yes, I know that’s odd.  Move on.

It occured to me that the process of buying a cold, caffeinated beverage can be both aligned and unaligned from the perspective of user and corporate goals.  If you imagine writing a use case for purchasing a beverage from the vending machine, your most important scenario is the one where a person wants to purchase a beverage that is available in the machine.

• User Goal: Get refreshed and vitalized.
• Corporate Goal: Sell a beverage.

I want to buy a Diet Mountain Dew, as a means to realize my personal goal.  The owner of the vending machine wants to realize her corporate goal of selling me the beverage I want to buy.  We’re in alignment.  Our requirements definitions and design decisions will be pretty easy – everything that increases the likelihood of and improves the experience of purchasing that beverage is good.  I put my money in the machine and push the button.

Then it occurs to me – there’s a situation where my personal goals are clearly not aligned with the corporate goals of the vending machine owner.  When there is no Diet Mountain Dew available to be purchased.

Consider the following procedure I’m forced to endure:

1. I view the available beverages that this machine dispenses – Diet Mountain Dew is one of them.
2. I put my money in the machine.
3. I push the button for Diet Mountain Dew.
4. The tiny display on the machine presents a message in taunting, scrolling red LED lights: SOLD OUT.

At this point, I’m faced with a choice – select a less desireable beverage, or request my money back and try to satisfy my user goal in some other way.

Consider an alternate procedure:

1. I view the available beverages that this machine dispenses – Diet Mountain Dew is one of them.
2. I view the ‘current availability’ of Diet Mountain Dew and see that this machine is currently sold out of Diet Mountain Dew.

At this point, I’m faced with a choice – select a less desireable beverage, or request my money back and try to satisfy my user goal in some other way.

The differences in these two possible procedures indicate that there is a conflict between the corporate goal and the user goal.  With the first procedure, the “Sell a beverage” goal is given more importance, because it makes me more likely to purchase a beverage that I don’t particularly want.  My utility is potentially sacrificed, while the owner of the vending machine is just as satisfied.

By failing to tell me that I can’t get what I want, until I’m further invested in the process, I am more likely to purchase something else.  That purchase creates just as much value for the owner of the vending machine, but less value for me.

If the vending machine owner had allowed me to use the second procedure, it would demonstrate that the  owner believed the improved customer experience, while risking the loss of this sale, would encourage me to purchase more over time.  The owner would have been prioritizing the requirements and design that supported the user goal ahead of those supporting the corporate goal – in hopes that my user loyalty would result in better word of mouth (more business from other people) and more business from me over time.

Generalizing User and Corporate Goal Conflicts

In this vending machine example, the annoying procedure is probably the right one – I’m going to treat the “point of use vending of cold, caffeinated beverages” as a commodity product.  This market has a very low barrier to entry, and relatively small problem being solved.  I’m not going to tell my friends about how much I love the vending machines from Joey’s Vending and encourage them to go out of their way to find and use those machines.  Nor will I purchase more Diet Mountain Dew because the emotional cost of the occasional “sold out” scenario is trivially smaller with Joey’s machines.  I would, however, be more likely to buy a Diet Pepsi when I’ve already invested time in the process and put my money in the machine – especially when the alternative is to find a water fountain.  I’m emotionally invested at this point.

However, this experience did jump out at me as one that is worth explicit consideration when defining requirements and designing solutions.  It does come up regularly.

The new iPhone 3GS hardware allows tethering (using the phone as an internet-connection device for your laptop).  ATT (the exclusive carrier for iPhones in the USA) is indicating that they will charge customers an extra monthly fee for this capability, when they eventually allow it on their network.  The user already has a data plan, and the traffic that ATT would have to carry is just “data.”  The user just wants it to work.  ATT, however, may want more money, or may want to limit data traffic on its network – perhaps to improve the quality of service to all customers.  This could be ATT’s corporate goal of “provide better service quality” conflicting with the user’s “increase the flexibilty of how I use my data plan.”  The former would likely influence customer satisfaction (and abandonment) over time, while the latter would influence customer acquisition rates.  Another conflict in goals.

Think about how different goals can lead to conflicting solutions, requirements, and designs.  And tell me when you’re out of Diet Mountain Dew before I’m emotionally invested in the purchase.

Blue Ocean Strategy provides an interesting reactive analysis of companies and markets.  Personas are used to understand your customer’s needs.  Combining the two provides powerful proactive insights when positioning your product for market success.

Blue Ocean Strategy

The book, Blue Ocean Strategy: How to Create Uncontested Market Space and Make Competition Irrelevant, by Kim and Mauborgne, was written in 2005.  The book presents a very compelling way to visualize competition in your market by contrasting the emphasis (or effectiveness) of each company at solving particular problems.  The authors argue that red oceans are competitive markets where companies compete to solve the same problems for the same customers.  Their main idea is that by identifying (and solving) unaddressed problems, you can define a new market – a blue ocean with no relevant competitors.  The “red” in their metaphor implies blood in the water, caused by cut-throat competition.  Their position – by defining a new market boundaries with no competition, you eliminate the blood in the water and give yourself a calm, blue ocean in which to navigate your company.

This is a compelling idea, and has a lot of merit.  The Smarter Product Managers book club (started by Cindy Alvarez) reviewed this book last month, and one conclusion we reached during the discussion is that the examples in the book felt “reverse-engineered.”  I feel that the lack of prescriptive advice from the authors created a sense of “that’s fine, but how do I apply these ideas proactively?”   Some of the examples in the book, like Cirque de Solei and Southwest Airlines, felt very compelling (and are often cited), while others felt a bit more contrived.  Almost as if the authors were searching for data to support their arguments – a big no-no for product managers.

Creating a Blue Ocean Strategy Map

The book presents examples of “mapping” markets based upon the strength of the offerings from companies, along different dimensions.  The following example was created by me (and is used throughout this article), but follows the pattern of analysis described in the book.  This is an unresearched, hypothetical example analysis of the market for vaccuum cleaners – or more generically, floor cleaning products.

 [larger image]

The diagram above identifies seven dimensions by which you could characterize the offerings from each company.

• Breathe Easier / Anti-Allergen – sometimes, people vaccuum their rugs in order to reduce allergens (by sucking up dust), making it easier for them to breathe in their homes.  Vaccuum cleaners work by sucking up air through the carpet, collecting the dirt (that is sucked up along with the air), and filtering the exhaust air (that is expelled into the room).  Different vaccuum cleaners pay different levels of attention to removing allergens during this cleaning process, by (1) getting the allergens out of the carpet and (2) filtering the allergens out of the exhaust air.
• Remove Stains – one motivation to clean your floor is to remove stains.  This is not generally a focus of vaccuum cleaners, but products like the Green Machine do focus on removing stains, while also “picking up dirt.”
• Physically Easy to Use – after back surgery, one of the key pieces of advice my mother received from her doctor was “no vaccuuming.”  When you are young and healthy, you don’t expect vaccuuming to be something that is forbidden by your doctor.  Shoveling snow, climbing a mountain, running a marathon, yes – but vaccuuming?
• Reducing Hassle – Why don’t we vaccuum every day?  Because it can be a hassle to get out the vaccuum, set up the equipment, and actually use it.  When you reduce the hassle involved in vaccuuming, you are likely to vaccuum more often, thereby realizing more benefits from vaccuuming.
• Saving Time – Hassle is not the only barrier to vaccuuming.  If you could vaccuum your house in 5 minutes instead of 105 minutes, you would do it more often.  As a teenager, I used to jog behind the lawn mower, just to get the yard done so I could move on to other things.
• Reliability – While reliability is not a major factor when vaccuuming your house (once), it is a key component of making purchasing decisions – because you don’t vaccuum just once.  The more you vaccuum, the more you care about reliability.  This is also an indirect cost factor – a vaccuum cleaner that has to be frequently repaired or replaced will cost more per year than one that has the same purchase price and is more reliable.  This is also an indirect hassle factor – you may delay vaccuuming your home until you can make time to get a broken vaccuum cleaner repaired.
• Avoiding Cost – A direct focus on cost is primarily driven by purchase price.  Other factors (like reliability, and the cost of replacement bags or filters), but people place greater emphasis on purchase price when thinking about cost.  Consumer Reports and other “product review” companies will often “do the math” for you, taking into account all of the post-purchase costs to calculate total-cost-of-ownership.

The above diagram, with fictitous values for “Strength of Competitive Offering”, shows the competitive landscape for the vaccuum cleaner market.  One of the very powerful features of this visualization is that the Roomba faces “no competition” from the other companies in three of the seven categories – Ease of Use, Hassle, and Time.  A Blue Ocean Strategy analysis would say that the Roomba has created their own market, with an absense of competition.

This gets back to our insight that the examples felt “reverse engineered.”  The Roomba does have a dramatically different value-proposition, and focused on dimensions that are not addressed by their competition.  So why isn’t the Roomba in the book?  Because they still compete in the red ocean.  Unlike Southwest Airlines, they did not differentiate their offering in a relevant way.  And that’s the problem we found in trying to apply the principles from the book.  

It is not enough to be innovative and differentiated, which the Roomba certainly is, you have to be valuably differentiated.

So how do you differentiate valuably and proactively?  Identify the problems that your customers value, but don’t yet have solutions for.  How do you do that?  With personas.

Personas

Personas represent customers in your target markets.  Markets are not homogenous – different people in your markets value different things for different reasons.  You develop personas as archetypes to represent subsets of the customers in a given market who share common problems.  More concretely, personas share common valuations of shared problems.

To combine persona development with the market-mapping concept from Blue Ocean Strategy, create the same map, but for personas.  Instead of identifying the strength of each company/product along each dimension, identify how much each persona cares about each particular dimension.

 [larger image]

In the chart above, I’ve created 6 hypothetical personas who populate our market:

• Single Parent – the classic superman / superwoman who does everything, including keeping the house clean.
• Housekeeper – someone employed to keep someone else’s house clean.
• Office Cleaner – the person who vaccuums your office at 2am every night and picks up all the empty Diet Coke cans and Twinkee wrappers.
• Kid (Doing Chores) – most kids won’t ask for a vaccuum cleaner for their birthday, but when you child is responsible for vaccuuming, do you want a battle every week because they hate it?
• Homeowner – like the single parent, but maybe without kids, and definitely with fewer responsibilities (and more time).
• Retiree – like the homeowner, but with a lot more time, and a less robust physical condition.

[Note - the above examples are really stereotypes and not personas (there is no research to back them up - they are entirely fictional) - don't fall into the trap of thinking persona development is this easy.]

OK, now we have an understanding of how much importance each persona places on each dimension.  It looks like a mess, for this example, when you try and absorb the big picture.  If you focus on a single persona, you get great clarity about what that type of customer cares about.  Look at each curve individually, then look at them all together.  If you’re thinking ahead, you’ll suspect that the Roomba is perfect for the kid doing chores.  But wait, we’ll come back to that.

One insight we can gain from this messy view of the market is that you can’t create a product that is “perfect” for all of these personas.  You have to target a specific persona, and tailor your product for that persona.

Mapping Personas to Products

Now we have two views of the market, along 7 dimensions.  We have assessments of the relative strength of each competitor’s offering, and we have estimates of the relative importance to each persona – for each dimension.  What we need to do is combine the two.

All of this analysis runs the risk of introducing a notion of false precision - we have a lot of data, therefore it must be accurate.  So our inclination may be to try and do some mathematically refined, scientific analysis.  I suspect that would be wasted effort, providing no additional insights, and risking leaps to the wrong conclusions.  I propose a simpler approach.

The analysis we really care about – which product offering is best aligned with the needs of each persona?  A simple mathematical equivalent of “which curves match the best” is an easy analysis.  By comparing the values from each competitor curve with those of each persona curve, we can create a series of “how good a fit is it?” values.

 [larger image]

Imagine a persona saying “how well does each company’s product align with my goals?”  This “simple math mashup” takes into account both “they succeed at what I care about” and “they haven’t invested in something I don’t care about (at the expense of something I do care about).”

You can see that the Roomba clearly wins with kids doing chores.  The rest of offerings stand out less, but do provide insight.  Now you can easily drive strategic decisions – “we want to be the vaccuum of choice for housekeepers” now drives some obvious emphasis on ease of use and a reduced emphasis on reducing costs.

Improving The Model

My two main problems with the blue ocean strategy were lack of relevance (who cares about dimension X) and magnitude (which dimension is most important?).  The model above addresses only relevance – a focus on target personas.  We can overlay some “relative importance of each persona” data, or just manually focus our efforts on each persona in series.  

The other magnitude challenge is in understanding not just which problems are important in absolute terms (kids care a lot about saving time, but not cost), but in relative terms (kids would trade an hour of time for a modicum of ease-of-use).  Essentially, defining the utility-curves for each persona.  For any but the largest, most saturated markets, I would hesitate to do the utility curve analysis in detail – it feels too heavyweight and un-agile.

Conclusion

The Blue Ocean Strategy book provides us with a visceral tool for visualizing relative offerings from competitors in a given market.  Combine it with the same visualization approach for personas that participate in that market, and you gain insights into which problems to solve next to achieve product success.

Netlife Research brings us the 2009 Bad Usability calendar.  Get it while it’s hot.

Download it from the 2009 bad usability calendar page on their website.  Or check out our article from last year for the 2007-2008 bad usability calendars.  They really do have some great points to make.  Two in particular got my attention this year – any guesses which months?

Last year, over 100K copies were downloaded.  Act now and be on the bleeding edge of the first 2% to download for 2009.

Learning how to write use cases can be tough, but it is simple compared to the balancing act of determining which use cases to write and how to manage the expectations of all the stakeholders that are involved. It can be a difficult balancing act to prioritize use cases to assure that you meet the goals of the business while satisfying the needs of the users.

Goal Driven Use Cases

In the world of structured requirements, we apply an approach of top-down decomposition to determine what we need to build.

In an article stressing the importance of non-functional requirements, we summarized the relationships from the above diagram as follows:

• Goals are achieved by enabling use cases.
• Goals drive non-functional requirements.
• Use cases are enabled by implementing functional requirements.
• Use cases influence non-functional requirements.
• Non-Functional Requirements define the characteristics of functional requirements.
• Functional requirements drive design decisions
• Design choices are restricted by constraints
• Design choices guide implementation
• Implementation is product

Non-Functional Requirements Equal Rights Amendment

The key element, with respect to this article, is that goals are achieved by enabling use cases. In other words, without the use cases that people* perform, the goals of the business can not be achieved.

[*Sometimes the use case is performed by a system - not all actors are people]

Principal Stakeholders Own Business Goals

We learned a lot from Outside-In Software Development, by Carl Kessler and John Sweitzer. One of the important things we learned is how to understand the goals of principal stakeholders. Principal stakeholders are the “owners” of the ROI that we are attempting to achieve with our product or solution. The business has the goals, but the business is an abstract entity. If the goal of the business is to increase sales by 10%, we can’t call up the business and ask “did we meet your goal?” We have to call up the vice president of sales. The business is organized so that there are people who are responsible for the initiatives and goals of the business – they are responsible for growth, profitability, costs, strategy. The insight we got from Kessler and Sweitzer is an appreciation that this ownership lies with one or more individuals.

When a goal has an owner, we can validate that we have met the goal. We can validate that we have satisfied the owner of the goal. You can’t do that validation with an abstract entity, a “business”, but you can with a person. Certainly, we can do the math ourselves, but if we want someone to pay the bills – and more importantly – invite us back to do more projects, we have to satisfy an individual. And that individual is our principal stakeholder.

Principals Own Use Cases

Our principal stakeholder owns the business goal – and therefore owns the use cases that enable that goal to be achieved. Our stakeholder will validate the completeness of our use cases by answering the question, “If we enable all of these use cases, will we completely meet your goal?”

End Users Own Use Cases

User-centered design is also one of our core principles, and an acknowledged best-practice. For many companies, it is more than mandatory – it is a way of life. The entire software development process is sometimes built around a focus on the end users. This approach puts the user in the center.

You can identify end user goals with the following approach:

1. Identify the actors that interact with the software.
2. Identify the personas that represent those actors – thus avoiding the elastic user syndrome.
3. Develop an understanding of the personal goals of the personas – and by extension, the actors through ethnographic research.

Stakeholder Goals: Principal vs. End User

This creates what appears to be a conflict – do principals own the use cases, or do end users own the use cases?

Mixing Interaction Design With Structured Requirements

One approach to resolving this conflict is to eliminate it by mixing interaction design with structured requirements. In this approach, the interaction design component applies end user ownership as an input to the design, but not the content of the use cases, as the following diagram shows:

In the diagram above, the changes (relative to the Wiegers process) are marked in blue.

The interaction design process starts with two parallel activities. On the left is the identification of corporate goals, which is analogous to the creation of an MRD. An MRD can be a reasonable mechanism for documenting the corporate goals. It provides more detail than “Increase profits” with statements like “Increase profits with improved pricing.” On the right side is the creation of personas to represent the most important users.

Interaction Design and Structured Requirements

This approach side-steps the issue by relegating the end users to “design input” and not allowing them to contribute to prioritization and management. For many applications, this is a great way to approach things. Specifically, it works very well when the user’s goals are tightly coupled or highly aligned with the principal stakeholder’s goals.

When Goals Conflict

User goals are not always aligned with stakeholder goals. In our earlier article on identifying these conflicts, we proposed a simple grid view for identifying these conflicts.

In that article, we looked at situations where the goals of the actor were primarily aligned with those of the principal stakeholder, and therefore the business. We discussed using the grid to inspire solutions that removed the conflicts. Two birds with one stone would be the easiest way to summarize.

What about when goals are implicitly in conflict – or even diametrically opposed? We’ve worked on systems where that is exactly the case. Consider a two-tier sales model, and products / websites designed for the people in the second tier.

Two-Tier Sales Models

Here are some common business models where the users have goals that are directly in opposition to the goals of the principal stakeholders.

• Automotive Manufacturer and Dealer. A website, designed by a car manufacturer, to allow dealers to improve their ability to sell cars. The automotive manufacturer makes money by selling cars to the end customer, and the dealer makes money by causing the sale to happen. The manufacturer is compensated by profits on the sale, whereas the dealer is compensated simply by making the sale. In other words, the dealer is motivated to lower the price because a sale at any price generates the same profit for the dealer. The manufacturer is motivated to raise the price because fewer sales at a higher margin generate the same profit, but higher ROA (return on assets) than more sales at a lower margin. This is an implicit conflict.
• Book Publisher and Book Retailer. A book publisher sets a price for sale to the retailer, and a suggested (list) price to the end customer. The retailer will typically pay a negotiated discount, in the form of a percentage off list, for the book. Any books not sold by the retailer are returned to the publisher or (usually) destroyed. The retailer does not have to pay for books that are not sold. The retailer has limited shelf-space, and makes money based upon a combination of the size of the discount and the number of books that are sold, regardless of publisher. The retailer is therefore incented to increase the discount, and thus increase profits for a particular title – and the publisher is incented to reduce the discount, and thus increase the profits for a particular title. Since the list prices are fixed, the retailer negotiates the discount, and as leverage over the publisher, gets to influence the placement of the book within the store. Each publisher is forced to compete with other publishers. These goals are also in conflict.
• High-Tech Equipment Manufacturer and Value-Added Reseller. A value-added reseller (VAR) provides solutions to end customers. They do this by combining products from manufacturers, and adding value – in the form of planning, custom engineering, installation services, etc. The VAR will negotiate a price with the end customer for a solution. The typical VAR sells products from multiple high-tech companies. The VAR will negotiate with the manufacturers to get the lowest possible prices from the manufacturers. The manufacturer wants the end-customer price (when it includes their equipment) to be as low as possible – to increase the number of sales that are made. The manufacturer gets the same profit (based on the price to the VAR) regardless of the end-customer price. Every dollar that the end-customer saves, the VAR loses in profit. The VAR makes profit on the difference between price-to-the-VAR and the final selling price. The VAR will also want to drive down the price it pays to the manufacturer. Reducing the price-to-the-VAR transfers profits directly from the manufacturer to the VAR. These goals are diametrically opposed.

When goals are in conflict like this, we can’t just relegate the “end user” goals to design inputs – we have to incorporate them into the prioritization process. In these examples, the “end users” are the VAR, the book retailer, and the automotive dealer.

Imagine that the high-tech equipment manufacturer sells networking hardware. The manufacturer has an internal (direct) sales force, as well as the VAR channel (indirect) sales force. The manufacturer identifies tools that help sales reps to close deals. Large purchases are commonly made at as much as 80% off list price. The list prices are irrelevant – they are not even guidelines in many cases. So the manufacturer develops tools to help sales reps know what prices to set for end customers.

Internal sales reps have generally well-aligned goals with principal stakeholders. The sales rep will be compensated based on a combination of revenue (sales) and margin (profits). This compensation model creates implicit alignment between the user goals (get a bonus) and the company goals (make profitable sales).

One approach to helping the sales rep close profitable deals is to share an indication of the profitability of the deal with the rep. The rep can (will!) set a price that is designed to maximize the sales rep’s bonus. If the compensation system is well designed – this is also optimal for the business, and therefore the principal stakeholder. The sales rep will get increased compensation for raising the price (and still closing the deal) by getting a percentage of the increase in margin. The sales rep is motivated to increase the manufacturer’s profits.

The VAR, however, should not know the profitability of a particular transaction. The VAR is motivated to lower the price (the price to the VAR) as much as possible – even at the expense of losing a “profitability bonus.” If the VAR is given a percentage of the increased margin (to the manufacturer) it comes at the cost of losing 100% of the change in price to the VAR. Remember, the VAR makes money on the difference between the price from the manufacturer, and the price to the customer.

Depending on the manufacturer’s business model, this could lower the priority of a margin-inspection capability, or dramatically raise the priority of features that restrict margin-viewing to only internal sales reps.

Also remember that the manufacturer and the VAR also have aligned goals – like closing the deal. And the VAR can close the deal with the manufacturer, or with products from a competitor. So the manufacturer is incented to provide valuable functionality to the VAR. The manufacturer cannot ignore the VAR’s needs, or the VAR will work with a competitor and the manufacturer will lose all the business.

You have to determine the value to the principal stakeholder of the value of the feature to the end user. In other words – even if a feature “hurts” the manufacturer, is the pain “worth it” because of the value of the relationship and expected future business that the company will do with the VAR?

Once the value of all of the functionality is identified, you can prioritize it. An enlightening way to visualize this value is with a value-delivery matrix. Thanks again to Roger Burlton for inspiring our approach to prioritizing based upon a combination of pain and gain.

Netlife Research (company website in Norwegian) has done it again. Their 2008 Bad Usability Calendar is here and it is great. So great that it is hard to pick a favorite. Download it here. 2007 has more great examples.

[Note: This is a short post- just got back from the Velvet Revolver concert at Stubb's. Living in Austin rocks!]

You want your software to be used, not to sit on the shelf. You can’t achieve the ROI of your software if people don’t use it. And you can’t achieve the ROI of your software by forcing people to use it either.  Some will fail to achieve the benefits, and others will delay using it or refuse to use it entirely.  You have to make them want to use it, and you have to design the software for the users who must use it.  Otherwise, you won’t achieve the ROI.

The ROI of User Adoption

The image above is great – it shows one person choosing to use the software, and one person choosing to do the work by hand instead. It is a classic user adoption problem. The good thing about it is that you can talk in concrete terms about the goals of your software, and how user adoption plays a role.

Let’s start with a definition of return on investment (and read the full article for more details and examples)

ROI is the acronym for return on investment. Another way to think of it is “How much profit will we make if we invest in this project?” Profit is revenue minus costs. Technically, the question should be “How much profit will we make, relative to our investment, if we invest in this project?”

Definition of ROI – Return on Investment

OK, with a definition of ROI, you have to develop a model for return, and one for investment. The model for investment is built by looking at the cost of developing the software. That defines how much it costs. Assume it costs $1,000,000 to develop and deploy the solution.

The model for return is a little more complicated.

For our user adoption example, assume that the software saves the company $1 every time it is used, relative to whatever the company does in the absence of the software. There are several ways to calculate the ROI of design, we’re picking one Assume we have 1000 users, each of whom would use (or not use) the software 10 times per day. That is a potential savings of $10,000 per day. Call it $2,000,000 per year once all the users are using it all the time.

Unfortunately, many people stop there. They see a 6-month payback period, they fund the project, and they stop thinking about it – the money is already in the bank. But you’re not going to do that.

Imagine that this is your project. You write perfect requirements, you have an excellent development team, and the software works perfectly – saving a dollar every time it is used. The problem is, only 100 people are willing to use the software. Now, instead of a 6-month payback period, it takes five years to recover the cost of developing the software.

The False Promise of the Mandate

“Not a problem for us,” you say. “Everyone must use the software. Or they’re fired!”

Many corporate IT departments work this way. The business tells them what they need (“Save us a dollar!”), and the IT guys gather requirements, develop a perfect application and confirm that it saves a dollar every time someone uses it. And then they mandate that everyone use it. Simple. 6-month payback.

There are a couple problems with this approach.

1. Not everyone uses it right away, even if everyone is required to use it.
2. Not everyone uses it effectively, even when required to use it.

The explanation is a little less obvious, but the brutal reality of it is just as true.

Delayed User Adoption

You issue a mandate – all 1000 users will use the software! The users are conveniently spread out into 10 business units, each with 100 users. One of those business units will go first. And your software is a disaster (we’ll get to #2 in a minute). The plan was to pilot the software with the first 100 users, then roll it out a month later to the other 900. No problem – 7 month payback, and less risk to the business.

The problem is that the people running the business units – and thereby managing the users – are smart too. They see the train wreck that Tom’s department became, and they refuse to use the software. They have their own financial targets, and they don’t want to jeopardize them. These business unit directors, if they lack the power, will escalate, until their demands are heard.

What’s the first place in the hierarchy where there’s a common manager? Is it the CEO? Maybe you’re lucky, and it is only a couple VP’s arguing about it. What conclusion do you think they will reach when presented with the following arguments:

• If we use the software, we will miss our numbers – it slows people down. And that will cost you $X.
• But we want them to use the software – they have to. If they were smarter, it would save money

Even if the argument isn’t accurate – who will win? Best case, your roll-out is delayed. Worst case, it is killed before it ever gets a chance to work.

Ineffective User Adoption

It turns out, after watching the pilot group, that expert users can in fact save a dollar every time. Competent users break even – there’s no savings for them. And beginners were better off with the old solution. It was less efficient, but the new way is so hard that they can’t do it well.

You’ve mandated that people use the software. Since they didn’t otherwise want to use it, you had to spend a bunch of energy (and money, in opportunity cost) to make it happen. The rollout was delayed, and at the end of the day, you break even. Most users have no savings, and the few that do are offset by those that caused an increase in cost.

Mandating user adoption creates a false sense of confidence, and does not assure ROI – it only assures adoption. We’ll say that again, to let it sink in.

Mandating user adoption does not assure ROI – it only assures adoption.

So what can you do?

Measuring User Adoption

As product managers who care about user adoption, our first thought is “measure it!” The biggest challenge is in measuring the right thing.

We can’t just measure how many clicks, because while that might correlate with ease of use – it does not necessarily cause increased user adoption.

To stay aligned with the ROI model, we have to measure, directly, that user adoption is meeting the forecast used to create the ROI model. The only measurement that is assured to be an accurate reflection of user adoption is measurement of user adoption directly.

Prevent Mistakes, Then Monitor Them

If we measure poor user adoption after it has proven to be inadequate, that doesn’t help very much. It is generally accepted that a good design leads to higher user adoption rates. In the case of the “false mandate,” good design yields faster roll-outs and allows more users to be more effective with the software.

There are also other reasons to invest in good design.

The right solution is to invest in good design – targeting competent users (not novice users) to achieve the desired ROI. A good design is easier to learn too. You can track the rate of improvement among users.

Here’s a chart from an article that goes into learning curves in more depth:

[larger image]

The graph shows that for a weekly frequency, after 16 weeks, the task time has reduced from 300 seconds to 100 seconds. With a daily frequency, the task time is even lower – about 60 seconds. This graph shows nothing other than converting the academic learning curve graph into one that incorporates calendar time and frequency of occurance.

Note the odd, vertical drops in task time during week 1. This is just a manifestation of looking at a weekly time scale. For a task that happens once per hour, the user will rack up 40 repetitions during the first week. From a decision-making standpoint, you don’t have time to react to that rapid rate of learning, so it is ok that it is collapsed on this graph.

Software Usability and Learning Curves

Thanks, now go hire some designers and achieve your ROI!

No matter how good your quality process is, you are introducing bugs. This article reviews the places where bugs are introduced in the software development process (from stakeholders to users), and reviews ways to address those bugs.

One way to identify the sources of bugs is by listening to how people tell you about the bugs.

1. “That is not what I want”
2. “That is not what I said”
3. “That is not what I meant”
4. “That is not how it is supposed to work”
5. “That is not what they meant”
6. “That is not what I want it to do”

Visually, you can think of the development process (regardless of methodology) like the following flow

Each of the six sources of bugs is labeled in the above flow. The article, Where Bugs Come From, goes into a detailed explanation of each source of bugs. This article looks at what you can do to improve the ability of your process to reduce these bugs. I have been a stakeholder, a product manager, a business analyst, a developer and a tester. And I’ve been guilty of all of these offenses at one time or another. When I say you, I also mean me. So chill.

“That is not what I want”

Stakeholders introduce bugs into the process by being unable to define what they want. Some agile methodology proponents argue that as a stakeholder, you can’t know what you want until you have something you don’t want in front of you. In other words, those agilists are giving up on trying to prevent these errors – they claim it is futile.

You can change your mind. The team could be dealing with “That is not what I want now” – and change is something that can not be prevented – although the impact of change can be mitigated. Iterative development and course-correction through getting immediate feedback is one way to do that. Prototyping is another method of course correction. Prototypes can also be used to elicit implicit requirements.

But that still avoids trying to directly address the issue of you not effectively describing what you actually want. There are ways to improve your team’s engagement with you, so that you actually say what you want. First, an emphasis on valuable requirements drives critical thinking about why you need a particular capability or feature. One approach to achieve valuable requirements is to question their justification.

The key to making this work is to make sure you are approving your requirements. And watch out for the anti-patterns in the article.

“That is not what I said”

Business analysts, product managers, and requirements analysts introduce bugs into the process through misunderstanding the stakeholders. Using the techniques above to stay on top of changes in stakeholder needs is not enough. Even when you effectively engage your stakeholders so that they are self-aware and can articulate exactly what they need and why, you still have a problem.

You can improve how you listen, and you can improve how you document what you heard. Make sure that you write complete, consistent and unambiguous requirements. While you’re at it – make sure you are writing your requirements correctly.

“That is not what I meant”

Developers can misinterpret requirements. While there are situations where a requirement is adequately expressed, but the developer is not capable of understanding, that is not the norm. Developers are smart. The onus is on the person writing requirements to make sure that they are written for the audience. So, even though the source of these bugs appears to be the developers, it is actually your documentation. If your documents need to span cultural chasms, then that’s your reality.

Make sure you write your requirements unambiguously, so that they are not misinterpreted. Make sure that they are feasible requirements, and that they do not specify design. The requirements also need to be concise without being overly terse.

“That is not how it is supposed to work”"

Developers should test what they create. Given an understanding of what you believe the code is supposed to do, you should test that it actually does it. This is the stereotypical bug – we asked for X, and it doesn’t work. Continuous integration is the most effective technique for addressing bugs that are introduced at this point in the process.

In more complex situations, you can apply more advanced techniques to assure that your testing is complete.

“That is not what they meant”

The testing team can also misinterpret requirements. This is a source of false positive bugs – even when the developers properly interpret the requirements, a bug may be lodged against their code when there is no bug. That’s why we have the ability to mark issues as “not a bug” when closing them. Apply the same rules for well-written requirements and you will also address this issue. Further, make sure that your requirements can be measurable, or verifiable. We recently wrote more about why you should make requirements testable.

“That is not what I want it to do”

User education is the key here. Users should be encouraged to provide feedback about your product. You don’t want to discourage feedback, you want to minimize the number of false positives. False positives come from users misinterpreting how the application is behaving – a sign of poor design. False positives also come from users not understanding what the application can (or should) do.

You can address design issues by focusing on design with the user in mind. And you can educate the users through a combination of training and with a novel approach to documentation. Document the product in terms of what users are trying to accomplish, in the context of how they are trying to do it.

Conclusion

There are a lot of people involved in the software development lifecycle. As people, we introduce bugs. As informed software professionals, we also know how to address many of them.



]]> http://tynerblain.com/blog/2008/01/14/you-are-creating-bugs/feed/ 12 http://tynerblain.com/blog/2007/12/20/global-actor-hierarchies-and-personas/ http://tynerblain.com/blog/2007/12/20/global-actor-hierarchies-and-personas/#comments Fri, 21 Dec 2007 04:34:01 +0000 Scott Sehlhorst http://tynerblain.com/blog/2007/12/20/global-actor-hierarchies-and-personas/



We use actor hierarchies to organize the different users of a system. Different people play different roles, and thus do different jobs. We use different actors to identify and organize those people. When deploying a system globally, we usually discover people that do the same jobs, but do them differently. Incorporating the notion of personas lets us deal with this.

Why Actor Hierarchies?

User-centered, or user-centric design is an approach to developing software that focuses, as the name implies, on the users of the system. When designing the solution – and more importantly, when defining the requirements for the system, special attention is placed on the users of the system. More specifically, center stage is devoted to developing software that helps the users accomplish their goals.

From a requirements perspective, the key is to acknowledge that there are different types of users, doing different types of work with the software. It is reasonable to conclude that they will have different needs. An actor hierarchy provides a way to organize the different needs of the people doing different jobs. [Note: These are the same actors that are identified in use cases.]

A Simple Actor Hierarchy

Let’s start by creating a simple actor hierarchy. We’ll build on this incrementally as we add more complexity to our user base. We’ll look at a hierarchy for sales representatives (reps) for a company that manufactures goods for sale to other businesses. Each business to which the company sells products is known as an account.



The hierarchy represents an approach to classification of sales reps. Each successive level of the hierarchy is more specific than the previous level. All employees who sell products are considered to be sales reps (the top level in the hierarchy). When the company sells products to its accounts, it is not always obvious which products should be sold. In those cases, an engineer is brought in to help determine the exact need, and suggest the right products to sell. This engineer is known as a technical sales rep and the technical sales rep works with an account rep to determine the right products and terms to close the deal. The account rep is responsible for making the individual sale, and occasionally gets help from the technical sales rep to do it.

The hierarchy shows that all sales reps are either technical reps or account reps. If an employee is a sales rep, then he must be one of the two choices.

Account reps are further divided into two groups – representatives who work with large accounts, and reps who work with small accounts. The company currently considers accounts that make more than $500,000 in purchases per year to be large accounts, and the rest are small accounts. The main difference in behavior is that a large account rep works with only one customer – and in fact, for really large accounts, there may be multiple representatives for a single account. Small account reps, however, each work with more than one account, maybe even dozens of accounts.

You could argue, in the abstract, that large and small account reps should not be treated as different types of actors – because they all manage relationships with customers and sell products. However, there is enough of a difference (in our hypothetical example) between the jobs of the two types of reps that it is worth treating them as separate actors.



In our example, the large account reps are focused on account retention, and maintaining relationships with a single customer. These reps are focused on account retention and are usually managing the upgrade cycle for their customers. The upgrade cycle is when a customer replaces their existing, worn out or obsolete equipment with new equipment. Small account reps might be working with really large potential customers who currently buy from the competition. These small account reps are focused on acquisition and growth. They are also trying to manage and create relationships in an attempt to grow the customers until they become large accounts. Small account reps are also managing multiple customers, and have additional activities associated with managing that complexity.

Global User Base

The situation above describes how the company (or if you have a product management hat on – the companies in the target market segment) organizes their sales force in one region. What do you do when the the company has operations around the world? If all global operations are the same, then you don’t do anything. It is a rare company that works this way. Most global companies have grown their regional businesses (where a region might be Europe or Asia, or it might be China or India) independently. This could be do to growth by acquisition, or the parallel growth of separate regions managed separately.

In different regions, companies often have different business models. They also usually deal with customers in different ways. These differences may manifest from varied local business practices, different characteristics of the customers, or different levels of maturity in the market for the products being sold. There are also differences in regional regulations, policies, and laws – although you should generally manage those as variations in business rules, not business practices. We’ll side-step that layer of complexity for the purposes of this article. Pretend that the same laws, policies, and regulations are in effect in all of the regions.

The same general actor hierarchy can serve to represent the sales reps in multiple regions, with some modifications to account for the differences. Consider for our example, that the company sells products both in North America and in Latin America. The actor hierarchy above describes how the sales team in North America. The company’s operations are a little bit different in Latin America (for our example).

Latin American operations are relatively new to the company. Where the company has a well established market in North America, it has relatively low market share but very rapid growth in Latin America. As a result of the way the company’s business has evolved in the two different regions, there are two distinct differences between the two regions in the way that sales reps are organized.

1. Technical Sales Reps. In North America, technical sales reps (TSRs) are salaried employees, who participate in a profit sharing program. In Latin America, the technical sales reps have a base salary, but sales-commissions make up a large portion of their compensation. The North American TSRs are therefore incented to propose solutions that are highly profitable (thus increasing their profit sharing bonuses). The Latin American TSRs are incented to propose solutions that may sacrifice profitability in exchange for increased revenue (thus increasing their commissions) .
2. Small Account Reps. In North America, small account reps, like all account reps, are employees of the company, and work on commission. In Latin America, small account reps are not employees of the company. Small account reps are third-party sales people. The model is very similar to independent insurance agents. The small account reps, when trying to bid on a job, can propose solutions from the company or from one of the competitors. The third party small account rep also gets to set the price that his customer will pay for the products, and negotiate the price that the company will accept – pocketing the difference.

In both situations, the user populations are different, and will behave differently. The TSRs in both regions do fundamentally the same work – but with different motivations. The Small account reps in Latin America, are markedly different from the ones in North America – in motivation, relationship with the company, and the work that they do.

Using Personas To Express Regional Differences

Personas provide an effective way to describe archetypes of classes of users. Persona development is a key component of the interaction design process, but it can be leveraged very effectively in a traditional requirements process as well. Each persona represents a homogeneous population of users. In other words, the collection of users that are in a role represented by a single actor, and who have similar personal goals. By defining personas, we can manage requirements in the context of those personal goals. One actor in the actor hierarchy may have multiple personas associated with it. This makes sense when there are groups of individuals with different personal goals.

The most common example, and one that is almost always present, is built around the notion of user experience. You can classify users of a product as novice, competent, or expert users. Novice users usually become competent users but rarely become expert users. Those people who become expert users are usually a different breed. Professional drivers become experts, where casual drivers rarely exceed competence.

These two user groups are made up of people with different goals, who use cars differently (even though they both drive, they drive differently, and with differing expectations and needs). It is critical to separate user groups that are different, and treat them differently, to avoid the elastic user problem.

The two different technical sales rep (TSR) populations – in Latin America and North America – would be best represented as two different personas. The TSRs in both regions do fundamentally the same work – provide technical support to an account rep, to help in closing a sale. The North American TSR is profit motivated, where the Latin American TSR is motivated by increasing revenue. These different goals make it very unlikely that the same persona would accurately describe the TSR population in both regions. There are other factors that drive the identification of multiple personas for the same role (in the actor hierarchy), but to simplify, we are focusing just on the regional dynamic.

Since the TSRs in both North and Latin America are performing the same job, it does not make sense to treat them as having different roles in the actor hierarchy.

Using Additional Roles in the Hierarchy to Express Regional Differences

In situations like with the Small Account Reps in North and Latin America, it is better to create distinct roles in the actor hierarchy to represent them.



The North American Small Account Rep works on commission and is an employee of the company. This rep is focused on closing the deal, but only with products from the company. This rep also may work with other reps to help them close deals – sharing intelligence about market pricing, helping with product selection, etc. There are elements of teamwork and collaboration present because all of the reps are “on the same team.”

In Latin America, the Small Account Rep is also focused on closing the deal. But this rep does not care who’s products are used to close the deal. The rep will work to get the lowest possible price from the company, in order to maximize the difference between the company’s price (to the rep) and the rep’s price (to the customer). 100% of this difference goes to the rep, so even if the rep is handsomely rewarded or commissioned based on transaction margin, the rep will always make more money by negotiating a lower price from the company. The rep also has additional activities associated with getting quotes from the competition and managing relationships with the company’s competitors.

Not only are the jobs different, but the company will treat these two reps very differently. For example, the company would be likely to share cost or margin information with the North American rep. That information will help the rep know how low he can price products for the customer without jeopardizing the company’s profitability. The company knows that the rep wants to close the deal at the highest possible price, to increase margins and therefore increase the rep’s compensation.

The company will not want to share cost information with the Latin American rep. The company is still motivated to get the most profitable deals, but the company realizes that the rep is trying to get the lowest possible price – not the highest possible price. This means that the company needs to provide low enough pricing for the rep to close the deal, and low enough pricing that the rep will resell the company’s products instead of products from the competition.

Summary

Actor hierarchies provide a way to differentiate groups of people who have different roles. Personas provide a way to describe groups of people within a role who share motivation and characteristics. The combination of the two allows for clear, yet flexible expression of requirements – both from a business perspective, and from a user perspective.

When dealing with user populations in different regions of the world, it makes sense to extend the actor hierarchy when the jobs, and business requirements of the user populations vary. When the users are doing the same job, but with different motivations, using personas to describe the differences makes more sense.



]]> http://tynerblain.com/blog/2007/12/20/global-actor-hierarchies-and-personas/feed/ 2 http://tynerblain.com/blog/2007/10/11/stakeholder-goals/ http://tynerblain.com/blog/2007/10/11/stakeholder-goals/#comments Fri, 12 Oct 2007 04:24:01 +0000 Scott Sehlhorst http://tynerblain.com/blog/2007/10/11/stakeholder-goals/


A couple weeks ago we wrote about Outside-in Software Development, by Carl Kessler and John Sweitzer. One of their ideas about stakeholders and goals has got us thinking about traceability.

Types of Stakeholders

Carl and John present a framework for understanding stakeholders that groups them into four categories – principals, end users, partners, and insiders.

• Principals - the business sponsors for a product or project. These folks have business goals they are trying to achieve. Your mission is to understand those goals. You also need to recognize that they may be moving targets, and be prepared to adapt to those changes. And we need to understand why they want what they want.
• End Users – the people who ultimately interact with the software. The best software development processes are focused on satisfying the end users. And a lot of authors are out there teaching us how to do that, although none perhaps as visibly as Alan Cooper.
• Partners - people who are neither principals nor end users. These are the folks who don’t directly benefit from your product, don’t use it directly, and didn’t help create it. But they are responsible for helping your product live, thrive, and interact with other products. Think IT people, keeping your system running, and keeping other systems talking to it.
• Insiders - the people who define the product. Your (internal) team. Not only product managers and developers, but business folks, marketers, testers, support staff, finance, etc. The people creating a product influence what ends up being created. And they are stakeholders in a narrow scope – they are affected by the creation and ongoing support of your product.

Outside-In Software Development: First Look

In the book, they go into more detail on the different types of stakeholders. One interesting thing that jumped out at me is something the authors mentioned about competing agendas for different stakeholders.

Support Or Not

Carl and John mention that end users have goals that usually, but not necessarily, are aligned with the goals of the principals. This makes sense – someone responsible for the ROI of an improved process would likely have some aligned goals with the person who uses the software within that process. Users are represented with personas, and those personas articulate goals – both personal and practical. This is one are of stakeholder goals. The user’s practical goals are (or should be) aligned with the goals of the principal responsible for the results of what the user does.

Tracing Among Goals

We use tracing to represent the supporting relationships between requirements artifacts.



The diagram above shows the general relationships between different types of artifacts. The following diagram shows how this tracing can incorporate the needs of users.



The top portion of this diagram accentuates that there is tracing within a requirement type. That is – goals can support other goals. From the perspective of stakeholder analysis, Carl and John describe it as alignment.

When we think about traceability, a good way to represent these aligned goals is through subordinate “supporting” relationships and traceability. Carl and John mention that the challenge is to identify conflicting goals and adjust prioritization accordingly (not all stakeholders are created equal!). This is great advice that resonates. The authors also point out that you need to be careful to not ignore a user goal that is critical to the achievement of a principal’s goal.
By tracking principal goals as corporate goals, and end user goals as practical goals, we can identify when goals are aligned (and supportive). We can also identify when goals are unaligned. This is exactly the visibility we need to address the two key elements the authors suggest we look at. Rewording into representation-centric maxims:

• Don’t ignore persona practical goals that support principal (corporate) goals.
• Question persona practical goals that are not aligned with principal (corporate) goals.

Conclusion

Outside-in Software Development continues to be a good read with good ideas. Understanding the relationships between the goals of principal stakeholders and end user stakeholders is one of those good ideas.

Using traceability to manage and communicate our understanding of those relationships will help us to make good decisions about supporting and prioritizing those goals.



]]> http://tynerblain.com/blog/2007/10/11/stakeholder-goals/feed/ 5 http://tynerblain.com/blog/2007/08/27/foundation-series-heuristic-evaluation/ http://tynerblain.com/blog/2007/08/27/foundation-series-heuristic-evaluation/#comments Tue, 28 Aug 2007 03:26:01 +0000 Scott Sehlhorst http://tynerblain.com/blog/2007/08/27/foundation-series-heuristic-evaluation/



A heuristic evaluation (or heuristic analysis) is a quick, low-cost usability analysis of the design of a user interface. Pareto’s rule tells us that we can get 80% of the results from 20% of the effort. And that’s where discount usability tests like a heuristic evaluation come in to play. Formal, and more detailed usability studies yield better results – but cost more and take more time. A small investment can pay off big with a heuristic evaluation.

Heuristic Evaluation

The Nielsen Norman Group defines heuristic evaluation as follows:

Heuristic evaluation is the most popular of the usability inspection methods. Heuristic evaluation is done as a systematic inspection of a user interface design for usability. The goal of heuristic evaluation is to find the usability problems in the design so that they can be attended to as part of an iterative design process. Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognized usability principles (the “heuristics”).

Jakob Nielsen

Relative Usefulness of Usability Methods

In another article of Nielsen’s, we see the results of surveys about different usability methods and their usefulness. One of the figures from that article shows this stunning graph.



[source: Technology Transfer of Heuristic Evaluation and Usability Inspection, Jakob Nielsen]

While user testing is the clear winner – both in usefulness and adoption, heuristic evaluation is a close second. The two of them are reportedly much more effective than all of the other tests.

Ten Usability Heuristics

Nielsen also shares with us a list of ten usability heuristics, or general principles to guide user interface design. His list goes into more detail on each of the ten heuristics, essentially answering the following questions. You can think of it as a checklist.

1. Does the system provide information about its status? ["Please wait while the system is updating"]
2. Does the system use terms and language that are familiar to the user?
3. Can mistakes easily be undone? ["Undo" and "How do I get back to where I was from here?"]
4. Does the system use controls (buttons, links, words) to enable actions consistently ["Yes" vs. "OK" vs. "Apply"]
5. Does the system help prevent common user errors?
6. Is it easy for users to see what they can do, versus being forced to remember what they can do?
7. Are there ways for expert users to be more efficient than novice users?
8. Are users forced to filter out irrelevant information (minimalist design)?
9. Do error messages help users to resolve the errors?
10. Is the documentation searchable, task-centric, and precise with “how to” steps?

These are fantastic questions to answer (and there’s a right answer for each one). And they also represent very easy to understand ideas – which can be very helpful when trying to explain it to someone who thinks that HCI folks just make applications “sexy.”

Summary

Really bad user interface mistakes can be avoided quickly with heuristic evaluations. This is the best bang for the buck, when determining how to invest in user experience efforts.



]]> http://tynerblain.com/blog/2007/08/27/foundation-series-heuristic-evaluation/feed/ 1 http://tynerblain.com/blog/2007/07/25/interface-design-visualization-methods/ http://tynerblain.com/blog/2007/07/25/interface-design-visualization-methods/#comments Thu, 26 Jul 2007 02:17:56 +0000 Scott Sehlhorst http://tynerblain.com/blog/2007/07/25/interface-design-visualization-methods/



Visualizing complex data can be very difficult. There are almost as many ways to visualize data as there are data to visualize. The Ralph Lengler and Martin J. Eppler at the Visual Literacy Organization collect many of them for us in a periodic table.


References

Hat tip to Kevin Kelly for sharing this with us in his Cool Tools blog. The periodic table is available at the visual-literacy.org website. Hovering over each element provides an example of the type of visualization.

Different Types of Visualization

One of the biggest challenges in visualization of complex information is in selecting a means to visualize it. Many approaches simply don’t fit the data – they have too few, or too many variables. For example, a pie chart is great for showing percentages (37% of new iPhone purchasers switched carriers, 5% had no service previously, 40% were existing AT&T customers, 18% did not respond) or relative proportions. The pie chart is not effective at showing trends or changes in the data (last year 20%…). You could have multiple pie charts, but there are more effective ways to show the trends in the data.

The periodic table that the Visual Literacy folks put together would otherwise be overwhelming – each element represents a way to visualize information. When you have something to visualize, it would be difficult and tedious to hover over each element, until you saw something inspiring. So they’ve organized it into six different categories of visualization.

• Data Visualization – line chart, pie chart, scatterplot, etc.
• Information Visualization – radar chart, tree map, data flow diagram, etc.
• Concept Visualization – mind map, layer chart, pyramid technique, etc.
• Strategy Visualization – supply & demand curve, failure tree, life cycle diagram, etc.
• Metaphor Visualization – tree, funnel, bridge, etc.
• Compound Visualization – cartoon, knowledge map, info-mural, etc.

There is also an overlay of text color (blue versus black), where blue elements (like the timeline) represent process visualizations, and black elements (like the venn diagram) represent structure visualizations.

There are also symbols to indicate if the visualization technique is suited to details, overviews or both. And more symbols to indicate if the diagram is intended to cause viewers to come up with new answers (e.g. solutions to a problem), or to indicate that the diagram is designed to clarify insights found in complex data (like the stakeholder rating map).

Conclusion

There are some very cool techniques presented. What would make the site even better would be if each element were a link to a page about that particular type of visualization. Roll-over graphics are handy for easily comparing different techniques – hopefully the authors will add detailed pages for the techniques. Of course, that is a huge job. If they added one per week, it would take two years!

Check them out – they will inspire thoughts when you’re really stuck.



]]> http://tynerblain.com/blog/2007/07/25/interface-design-visualization-methods/feed/ 0 http://tynerblain.com/blog/2007/07/02/humane-design/ http://tynerblain.com/blog/2007/07/02/humane-design/#comments Tue, 03 Jul 2007 04:17:22 +0000 Scott Sehlhorst http://tynerblain.com/blog/2007/07/02/humane-design/



There are at least 7 ideals to keep in mind when designing a user interface. Shmula tells us about them.

In his article on humane interface design, shmula lists and describes seven ideals. After a long day of travel, I don’t have anything to add, so we’re just going to suggest you read his article. Here are the seven ideals that are covered in more depth:

1. It’s not your fault.
2. Simple things should stay simple.
3. Fewer choices mean fewer worries.
4. Your data is sacred.
5. Your train of thought is sacred.
6. Good interfaces create good habits.
7. Modes cause misery.

Check it out.



]]> http://tynerblain.com/blog/2007/07/02/humane-design/feed/ 0