Why Do Products Fail? – Forgetting that Users Learn

Next up in the series on the root causes of product failure – products that fail because you have ignored the user’s level of experience.  The first time someone uses your product, they don’t know anything about it.  Did you design your interfaces for new users?  After they’ve used it for a while, they get pretty good at using it.  How much do you think they like being forced to take baby steps through a guided wizard now?

Why Do Products Fail?

Your product launched a year ago.  People raved about how easy it is to learn to use your product.  Someone posted a video of a toddler using it, it kicked off a meme, you got a nice bump in downloads, and you were thrilled.  Now, people are complaining about how impossible it is to actually get anything done with your product.  And your main competitor is making hay with their message of cost-effectiveness and efficiency.  Sales have dried up to a trickle.  What the heck?!  You loosen your collar and walk in to explain to your investors why they will be lucky to get half their investment back – much less the ten-bagger your confidently forecast for them eleven months ago.

It sure is good you’re performing this thought experiment before you even build the product, instead of running the gauntlet a year after it launched.

There are many reasons a product can fail.  One way to think about your responsibilities as a product manager is to prevent all of them.

[larger version]

In this series, we are exploring as many of them as we can.  In your exploration of the reasons products fail, you started by making sure you’re solving the right problems.

[larger image]

As a firm believer in being user-centric, and making sure you’re delivering value to your users, you’ve doggedly pursued exactly what that means.

[larger image]

You’re confident that

What you failed to do was understand that your users change over time.

Kathy Sierra’s Canyon of Pain

I wish Kathy Sierra was still writing online.  I first came across her canyon of pain model in 2007.

Bigcanyon

On the other extreme is Apple’s iMovie. It gives you almost no control, but the payoff is high right out of the shrinkwrap. It exceeds my expectations of pain-to-payoff. But pretty quickly, anyone who gets into iMovie–and is bitten by the movie-making bug–starts wanting things that iMovie doesn’t let you control. So… Apple says, “not to worry — we have Final Cut Express HD for just $299″. The problem is, the learning curve jump from iMovie to Final Cut Express is DRASTIC. There needs to be something in the middle, to smooth that transition.

Kathy Sierra, How much control should our users have?

While she uses the above example to describe a market segmentation opportunity, it describes not just the difference between users, but the changes a user goes through over time.  Taking an aerial view of Kathy’s Canyon it would look like this:

Most users are competent users.  It makes sense – you start out as a new user, until you develop competence, and possibly develop expertise (although few become experts).   How many users will use your product for long enough- and invest themselves enough in learning to use your product to truly develop expertise?

Conceptually, you need to build a bridge across that canyon, for your product to meet the needs of your users as they develop confidence (and for the few who become experts).

Learning Curves

This is an artful dance – designing for the competent users – and your personas are likely to represent competent users, since competent users will be focusing on using your product to solve their problems, not trying to solve their problems with using your product.  With your focus on making sure you help your users solve their problems, you may overlook the problem that you introduce – learning how to use your product.  So, you better make sure you understand the nature of how people develop competence and expertise.

Malcolm Gladwell, in Outliers, builds on Dr. Ericsson’s analysis that it takes 10,000 hours of doing something to become an expert at it (not everyone agrees).

Defining Competence

The first step to measuring competency is to define the model.  I am proposing a definition in this article that I suspect will yield insights (to help us manage our products). I was unable to find any quantified definitions of competence, when researching it as part of a client engagement.  If you have, or know of a model, please share it in the discussion below this article.

  • A competent user is someone who learns to perform a task in half the time it initially took them.
  • An expert user is someone who can complete a task in 10% of the initial time.

This definition is guided by an expectation that Alan Cooper’s premise about perpetually intermediate users is true.  Being a novice user is a very transient state, and becoming an expert is very infrequent.  The goal of the definition is to be able to segment your users and make well-informed design decisions.

Modeling User Competency

Building on these definitions (and the hypothesis that Cooper is correct), you can model the level of difficulty of task-learning that best represents your product (see the article) to get insight into how rapidly your users will evolve their competency over time.  It can impact your modeling of ROI-realization, but more (?) importantly, it gives you insight into understanding how quickly your users will tire of the hold-my-hand-wizards you coded to guide them through your product.  As an example, their (measurable) improvement may look like the following:

[larger image]

Of course people are rarely chained to their desks and using your product continuously.  Even call-center employees get 5 to 8 minute breaks every hour.  So you should develop some insight into how quickly ( in hours / days / weeks) your users develop competence.

The following graph shows how an 80% learning curve overlays a calendar for tasks that happen daily, weekly, and hourly.

[larger image]

The graph shows that for a weekly frequency, after 16 weeks, the task time has reduced from 300 seconds to 100 seconds. With a daily frequency, the task time is even lower – about 60 seconds. This graph shows nothing other than converting the academic learning curve graph into one that incorporates calendar time and frequency of occurrence.

Software Usability and Learning Curves

Part of the catch-22 of developing products is that by trying to be all things to people in all-levels of competence, you may try and just add features.  Over time, this can really add up.  So, you have to figure out the right amount of capability to build into your product.  You can play a couple tricks, whipping out your handy Kano analysis skills again, and prioritizing capabilities and features with their characteristics in mind.

Consider the more is better requirements. Think of them in two categories – user interaction improvements, and application performance improvements.

User interaction improvements remove complexity, and make software easier to use. This results in more user happiness from a given feature, and also allows us to implement more features at a given level of happiness (appeasing salespeople).

users

Application performance improvements don’t create as dramatic of a shift (they don’t make the application easier to use). They do, make it more enjoyable for a given feature set – shifting the curve up.

apps

Summary

Failing to take into account the learning journey that your target users go through is just a more nuanced version of the elastic user problem.  The easiest way I’ve found – so far – is to treat “learn how to use your product to achieve their (other) goals” as a user goal, and create a distinct persona that represents your target user while they are a novice.

By analogy, one persona for the caterpillar, one for the butterfly.

If you know of a better way to deal with this as a product manager, please chime in below – this solution is a bit more clunky than I would prefer to use.

67 thoughts on “Why Do Products Fail? – Forgetting that Users Learn

  1. Pingback: Jessica Marie
  2. Pingback: garryts
  3. Pingback: Jose A. Briones
  4. Pingback: Jose Baldaia
  5. Pingback: Darrin Johnson
  6. Pingback: Eric Kavanagh
  7. Pingback: Yama
  8. Pingback: Robert Sfeir
  9. Pingback: Vasily M. Komarov
  10. Pingback: Alltop Agile
  11. Pingback: Agnieszka Walorska
  12. Pingback: rmeziani
  13. Pingback: Avinash Raghava
  14. Pingback: Sreekanth MM
  15. Pingback: Teppo Heikurinen
  16. Pingback: Sharad Bhushan K
  17. Pingback: stevebuccola
  18. Scott,

    Excellent article!

    Your stuff is consistently insightful and useful. This article was especially fun for me to read because it also happens to be on a topic I’m thinking a lot about at the moment (e.g., “how do we keep our tool easy to use while adding new features and helping users (i) learn how to use the new features, and (ii) gain more subject matter expertise (“outside of our tool”) so that they will be able to take full advantage of our tool).

    I’m sending out this link now to the team as food for thought. Thanks again.

    Justin

  19. Pingback: lisaw1
  20. Pingback: Josh Lehman
  21. Pingback: EdDale
  22. Our product represents an interesting problem. We have as one of our main differentiators that our time to proficiency is lower as we are building upon the well used AutoCAD platform. This means that our base user persona is one who is well used to advanced usage via automation, scripting, and high level mathematics, but at the same time is unaware of basic concepts of our products (such as how our information model ties drawings together into a cohesive whole via a database (sort of).

    The most useful way I’ve been able to come up with to conceptualize this while trying to deal with this disparity is to do exactly as you’ve done, have multiple user persona’s representing the various states of learning with our product and it’s companion products. Now, I’m curious why you say it’s clunky however, because it seems to tie in really well with the way I’m organizing our feature development. Namely that I develop three paths through any given feature:
    1. The wizard based approach
    2. The competent user approach (multiple selections within the product, shortcutting and mostly removing the wizard)
    3. The expert user approach (automated scripting, macros, and skipping the wizard entirely)

    Now it may be a quirk of our product that we get #3 for free when we develop #1 and #2, but nonetheless it’s been a reasonably successful way of thinking about it.

    Regardless of response, this is a very well written article, and I continually look forward to your posts.

    1. Eban,

      Thanks so much, and welcome to Tyner Blain!

      I really appreciate you sharing your experiences with our readers – and they do mirror my own!

      The reason I say “clunky” is that creating three different personas to represent the same person (at different stages of learning) in the same role violates (or at least bends) the DRY-rule (don’t repeat yourself).

      As a person progresses through the learning stages, they will have an evolving set of transient (learning, and possibly task-accomplishment) goals. However, they will have a consistent context, environment, and other identical not-related-to-this-product goals. So it is a bit clunky.

      What might be more elegant would be a way to incorporate the notion of a user persona that evolves over time. But that introduces different problems, and I suspect the added complexity of adding this dimension would be more work than the repetition.

      One thing I have done is show how one persona “becomes” another one, for a product where it was not just learning to use the product, but capturing how the user’s environment and perspective evolved over time.

      A good example is when a knowledge worker is moving from high school to college to the work force. Different social dynamics, expectations, and responsibilities all come into play. An individual user starts out as mapping to the high schooler persona, but as that user “grows”, she starts mapping better to the college student persona, etc.

      Ultimately, a weakness of using personas as approximations (the benefit of the focus they provide outweighs the weaknesses of the generalization, imho). And the notion of learning over time also “pokes” at the model.

      So, net-net, “clunky,” but better than not doing it, and the best solution I’ve found so far.

  23. Pingback: Justin Hunter
  24. Pingback: UX Feeder
  25. Pingback: Sandi Jones
  26. Pingback: Anna Baik
  27. Pingback: Kano
  28. Pingback: Veronica Figarella
  29. Pingback: David Crow
  30. Pingback: Ellen Gottesdiener
  31. Pingback: Ellen Gottesdiener
  32. Pingback: Alidad
  33. Pingback: UXfeeds
  34. Pingback: UX Factory
  35. Pingback: Harrison
  36. Pingback: Phil Bonhard
  37. Pingback: natraj k
  38. Pingback: Francois Wessels
  39. Pingback: Joni Hoadley
  40. Pingback: MethodGroup
  41. Pingback: MATRIX
  42. Pingback: Kate Valdes
  43. Pingback: sorgoz
  44. Pingback: ExecutiveBrief
  45. Pingback: Rich Mironov

Leave a Reply

Your email address will not be published. Required fields are marked *