Market Problems – Comparing Products Part 3

Comparing products without an understanding of the important market problems by which to compare the products is a waste of time.  This is the third in a series on comparing products - jump back to the introduction if you haven’t already read the previous articles.  Go ahead, we’ll wait, then come back.

Overall Product Comparison Process

This is a relatively long series.  Each article will start with a recap of the overall process.

Getting useful information from comparing products requires you to:

1. Introduction & Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems they care about solving. (This article)
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to the others.

Too many product comparisons are actually mislabeled positioning papers, highlighting differences, and putting one product in the best possible light.  An honest product comparison must compare products in the context of problems that customers are willing to pay to solve.  Those are the important market problems.

Important Market Problems

Probably everyone reading this has heard the Henry Ford quip about how if he had listened to his customers, he would have built faster horses.  A great marketing sound-bite, but not actually true.  The market problem that cars initially addressed wasn’t “get there faster”, the market problem was “spent fuel” disposal (for horses) in major cities – 2.5 million pounds per day in New York City (hat tip to Super Freakonomics for the first reference I saw).  If cars had generated as much manure as horses, do you think they would have been as successful?

The first step in identifying the important market problems is to identify a set of market problems that might be important. Then you can form an opinion about which ones are important, based on which problems are important to the potential customers in the markets on which you are focusing.  Identifying problems is an unconstrained activity, like collecting seashells on a beach.  There will always be more shells.

You will find shells that aren’t very pretty or are broken.  You will find some shells that are “perfect”, and there will always be some shells that exist, that are “perfect” that you won’t find.  The tough part is knowing when to stop looking for shells and move forward with the ones that you’ve already found.  You have to decide when your shell collection is “good enough.”  You can’t have a collection that includes all of the seashells, you don’t want a meager collection with too few shells, and you want the best shells you can find to be the ones you display on your coffee table.

This is a good example of the debate between big up-front design (BUFD) & big up-front requirements (BUFR) and incremental product development.  In The Design of Design, Frederick Brooks provides a great discussion about this debate between rationalism and empiricism.  The best approach I’ve found to date is to use a combination of agile and waterfall techniques to strike a balance between good enough and right now.

The best approach that I’ve found to do this to combine incremental development techniques to the activities of product management, combined with timeboxing to impose practical, artificial constraints on the process.  An agile product management process looks roughly like the following:

[larger version]

The steps of market research, analysis, and synthesis are the activities you perform.  Data is what you collect, and insights are what you form.  Analysis of data also leads you to perform more research.  Synthesis of those insights into a cohesive vision for your product will lead you back to more analysis and more research.  Note that this process has no ending – it goes on forever – or for as long as you are trying to solve these particular market problems.

As an example – in the previous article we identified a set of personas for our Kindle Fire example:

For this series of articles, I will pretend that three personas were developed, in order to demonstrate this approach to comparing products:

1. Hi-Tech Prosumers – people who get excited about convergence, highly value additional multimedia capabilities, live a multi-device existence, and are acutely aware of competitive products.
2. Typical Amazon Kindle Users – people who placed high value on the reading experience, the convenience of being able to easily get new content, and the absence of a subscription fee.
3. Basic Consumers – people who read primarily on paper media, have recently started consuming multimedia on their computers, and anticipate becoming part of the “post pc era.”

I will also invent additional data as we explore the rest of the product comparison process.
Comparing Products Part 2 – Who Are Your Customers?

However, defining requirements for those personas risks creating a product that only satisfies the basest of goals – the lowest common denominators.  These examples don’t take into account the context of use – e.g. why is someone reading on their device? – and thus we are at risk of not creating the right product for any of them.  Even introducing two simple contexts

• Education - Reading for work or school (directed study) or to gain knowledge about a topic (undirected study)
• Entertainment - Reading for pleasure, the literary equivalent of watching a movie

You’re probably not going to want to take notes on the uses of foreshadowing and allegory when reading for entertainment.  It may be critically important to annotate key concepts and ideas when reading for education.

This single distinction should be clanging a bell in your head – it is very easy for one product to be better than another in one context of usage than in the other.  Looking at the three example personas from the previous article, there is not a good way to say “educational use is more important” for one group of people than for another.  If we iterate and refine our list of personas it will lead to a better comparison of products.

1. Tina - A hi-tech prosumer who is using the device to get smarter about the latest trends in her industry
2. Tim - A hi-tech prosumer who is using the device to enjoy niche fiction content, particularly comics, e-zines and self-published works
3. Kenny - A typical kindle user who is using the device for his work in the finance space, studying proposals and business plans, etc
4. Karla - A typical kindle user and voracious reader who is using the device to eliminate the large pile of books on her nightstand
5. Chris - A basic consumer who would is studying business in college
6. Christina - A basic consumer who is in a book club, and who is always reading the latest best seller

Identifying Possibly Important Problems

Now that we have a (better) idea of the customers for whom we are solving market problems, we need to create a list of candidate problems that might be worth solving.  I am not an expert on user research – so there may very well be a better way to do this, and hopefully someone will share that with us in the comments section – but here’s how I’ve learned to do it.  It is at least “good enough.”  The approach I’ve used is a 3-step process.

1. Seed the Cloud – Start with what you think you know already
2. Refine the List – Validate and enhance that list through qualitative interviews
3. Empirical Validation – Assess the relative importance of those problems through quantitative research

Seeding the Cloud

I like to start by capturing my initial ideas in a mindmap or concept map – an ad hoc, semi-structured view of ideas.

[larger image]

I will then drive some brainstorming (more on brainstorming) or idea-seeding or Innovation Games exercises to develop a refined list of market problems that might be worth solving.  These processes may lead to other artifacts and usually result in updates to my mindmap.  These exercises bring others into the exploration, so that it isn’t The World According to Scott – but it is still the world as seen “from the inside.”

Refine The List

What we’ve done at this point, while creating a set of questions worth asking, possibly, is still heavily biased as an inside-out view of the market.  It may be informed by previous research and synthesis of ideas, but it still reflects what we think about the market, not what customers in the market thinks about their problems.

The next step is to gather qualitative information from customers and potential customers that represent your target personas.

Anecdotally, this is the same process for driving outside-in innovation from a product management perspective.

Mostly-open-ended conversations with people that are reflective of your personas is how I transform from an inside-out view to an outside-in view.  I’m not asking people “what they want” (remember the faster horse), I’m getting validation that the problems they face are the ones I think I understand (waste disposal).  This usually uncovers additional problems I have not thought of, and discounts problems I may have thought were important.  Ethnographic research provides this type of information – and when I’m working with a team that does this work, I utilize it.  Usually, I’m not, so I will find representative users and ask them questions, or observe their behavior.  Essentially, I’m playing the 80/20 game – some insight is better than no insight.

Empirical Validation

At this stage, you still only have a list of possibly important market problems, but now you have much more confidence that the list is valid and mostly complete.  You also have some qualitative insights into the relative importance of problems to be solved (maybe you did a card-sorting exercise, or created interrelation digraphs, or played the 20-20 Vision game, to get those inputs).

Now you can gather empirical data, to gain some statistical significance to your analysis.  You can do surveys that help you characterize and profile user behavior.  You can even do analysis of social networks, to see which problems resonate and why particular products get good (or bad) word of mouth.

Amplified Analytics has a presentation on slideshare showing how they can mine statistically significant information about features in the context of products. [Hat tip to Gregory, for the pointer to this].

I would suggest using a similar approach to try and understand which problems people face (and solve) versus which features they prefer (as solutions).  But the same applies.  I’ve had good results creating surveys and using Google Docs to aggregate results that allow me to characterize problems, (reported) behavioral patterns – like frequency of use, and (reported) relative importance of solving particular problems (like performance times for specific actions), and quantifying data-characteristics (e.g. how many emails someone sends, and to how many different people, in a given day).

Example Problem List

Given that we’re not actually doing a rigorous (and free) analysis of the Kindle, I get to “pretend” that I did research following the examples above, resulting in the following list of market problems, that are of (at least some) interest to our target personas.  We’ll use this list moving forward in the rest of the series.

1. Be able to read content in multiple physical environments / on multiple devices, and not lose my place in the book.
2. Be able to annotate / highlight what I’m reading for future review.
3. Be able to have conversations with other people who are reading what I’m reading.
4. Make it easier for me to find other content that I would like to read.
5. Be able to subscribe to magazines / newspapers / blogs / serial publications.
6. Be able to read what people I trust are reading.

Summary

It is important to be creating a product with an identity.  That means knowing for whom the product is designed, and for which uses.  You have to know who your users are, and which problems they want to solve.  If you’re going to compare products, you should compare them based on their suitability as solutions to these important problems.

Recapping the overall flow of this series of articles on product comparison

Getting useful information from comparing products requires you to:

1. Introduction and Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems your customers care about solving. (This article)
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers.

With this information, you can create a point of view about how your product compares to other products.

Attributions

• Thanks Stéphane Vandenwyngaert for the Model T photo.
• Thanks michaelaw for the smiling coffee drinker photo.
• Thanks Dora Pete for the self-portrait photo.
• Thanks Benjamin Earwicker for the spontaneous photo.