Tally the Score – Comparing Products Part 8

After building an understanding of which problems are important to your each customer you want to serve, and rating each competitive product , you’re ready to tally the scores and see how your product compares with your competition. This tells you if you’re likely to crush it, and if not, lets you know where you should invest later. This series on comparing products starts here if you need to get caught up.

And now, on to the finale…

Overall Product Comparison Process

This is a relatively long series. Each article will start with a recap of the overall process.

Getting useful information from comparing products requires you to:

1. Introduction & Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems they care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers. (This Article)

With this information, you can create a point of view about how your product compares to the others.

What You Know

During the series, you’ve pulled together a lot of market information, and probably had several insights. [Reminder so that no one accidentally misinterprets this article – this is a hypothetical analysis with fake data, intended to teach how you do this – it is not an analysis of the Kindle Fire, specifically] Along the way you’ve determined

Who your target customers are, in order to make sure you design your product for the right people, instead of trying to design for everyone (which is the same as designing for no one).

1. Tina – A hi-tech prosumer who is using the device to get smarter about the latest trends in her industry
2. Tim – A hi-tech prosumer who is using the device to enjoy niche fiction content, particularly comics, e-zines and self-published works
3. Kenny – A typical kindle user who is using the device for his work in the finance space, studying proposals and business plans, etc
4. Karla – A typical kindle user and voracious reader who is using the device to eliminate the large pile of books on her nightstand
5. Chris – A basic consumer who would is studying business in college
6. Christina – A basic consumer who is in a book club, and who is always reading the latest best seller

Comparing Products Part 3 – Market Problems

How important each of those customers is to your strategy, to give you a means of determining the relative importance of each solution.

[larger image]

[larger Blue Ocean image]
[…]
Important Customers – Comparing Products Part 5

Which problems those customers care about solving, to minimize the amount of effort you put into capabilities that don’t effect the purchase decision for those customers.

The importance of each of those problems to each of those customers, so that you know which problem to solve first (or best) for each customer.

1. Read Anywhere – Be able to read content in multiple physical environments / on multiple devices, and not lose my place in the book.
2. Annotate – Be able to annotate / highlight what I’m reading for future review.
3. Talk About It – Be able to have conversations with other people who are reading what I’m reading.
4. Find More to Read – Make it easier for me to find other content that I would like to read.
5. Subscribe – Be able to subscribe to magazines / newspapers / blogs / serial publications.
6. More From My Network – Be able to read what people I trust are reading.

[larger image]
Comparing Products Part 3 – Market Problems

Your goal is to find the right level of abstraction at which to determine who your competition is. If we consider Tim, the hi-tech prosumer who consumes niche content, we would see that the decomposition of his high level goals looks like the following:

[larger image]

Know Your Competition – Comparing Products Part 6

5. How effectively each competitive product solves each of those problems, so that you know who your customers consider to be your competition.

Scoring All of the Capabilities for All of the Products

Applying the same process (determine the nature of each capability, determine the criteria for each capability, assign a score to each product for each capability) will result in something that looks like the following [manufactured to illustrate the concepts] data:

[larger image]
Rating Your Competition – Comparing Products Part 7

Now you’re ready to combine this information to see how your product stacks up.

Rolling Up Scores – The Mathy Bits

First, don’t worry – there is a little bit of math here, but it is baked into the process so that you only have to worry about it once or twice, and then it just helps instead of getting in the way. The math serves to answer several subordinate questions along the way to answering the questions that lead up to “which is the better product, and by how much?”

Cognitive Bias in Customers, but not Robots

For each customer, you know how important each problem is to solve. You know how effectively each product solves each of those problems. If your customers were robots, this would be easy – each customer would rationally compute the utility they get from each capability from each product, and optimize their overall utility by picking the product that maximizes this value. You could just multiply the scores and get the answer. Here’s one reason why that won’t work (there are others).

We saw with Kano Analysis of realistic more-is-better capabilities, people get a non-linear sense of satisfaction from solving problems, which we reflected in our [made up] data by mapping of the scores (from 1 through 9) against the overall curve for each individual capability.

[larger image]
Rating Your Competition – Comparing Products – Part 7

This approach normalizes the scores within the pseudo-math for comparison. Don’t lose sight of the fact that it is probably harder to improve your product from a “6” to a “7” for any one score than it is to improve it from a “2” to a “3.” What this approach does embody, is that improving a single capability from a “7” to a “9” is more valuable than moving from a “5” to a “7.”

Because your customers are not robots, they will have cognitive biases that cause them to overweight (mentally) their personal “most important problem” relative to the “less important problems” when determining their overall score for a product that solves all of the problems. In other words, a customer will overweight the “most important capabilities” and underweight the “still important” capabilities, from a purely rational utility-maximization perspective.

It is more important to your customer to be a little bit better at solving their most important problem than it is to be a little bit better at solving their least important problem.

This is true both for robots and customers, and the “just multiply” approach captures this relative importance. However, when you include the cognitive biases of people, you have to go one step further.

It is more important to your customer to improve your solution to their most important problem than it is to improve your solution by twice as much, to a problem that is half as important.

What the pseudo-math does is change the process from prioritizing by the user’s (robot’s) utility, to comparing products based on the buyer’s perceived utility at the time of the purchasing decision – incorporating the cognitive biases into the way the scores are calculated.

As with any numbers-heavy technique, you run the very real risk of creating your own sense of false precision. This entire exercise is the formulation of a model (in Lean terms, a testable hypothesis) of how to compare competing products. If you don’t trust (or if you disprove) the approach that I use as a starting point, then change it to reflect what you think (or what you find).

To incorporate this bias that overemphasizes the solution of the most important problems, I square the values of relative importance of problems (to users). The original data rates the importance of solving problems on a scale from 1 to 5.

[larger image]

When squared, the values become:

[larger image]

I chose to use squaring of the values because it intuitively feels right, when using a 1-5 scale for the (actual) weightings. There may be a better formula, and the ideal adjustment may be different for each product domain, or even for each persona within that domain. My gut tells me that if you invest the time in determining the “right” way to account for biases, you will have lost the opportunity to compete in your market, although you will have gained the opportunity to publish a useful psychology research paper. Use squaring, ignore cognitive bias, or use some other calculation that passes your “sniff test” – it’s up to you. This is another example of why the Art of Product Management is an art.

Scoring Each Product for Each Customer

When you multiply the scores for each product (for each capability) with the relative importance (of each capability) for each persona, you end up with a table of values that reflect the perceived score for each product for each persona.

For Tina, that table looks like the following

[larger image]

For Kenny, that table looks like the following:

[larger image]

Notice that the ranges of values (for Total score) are different for Kenny and for Tina. That’s because Kenny cares about fewer product capabilities. The “Max Possible” column has been added to each table, to reflect the maximum score that each persona would place for any given capability, for any product that scored a “9” on that capability. The total-per-product scores for each persona are then reflected as a percentage of the maximum possible score for each persona. This allows you to manage the comparison across customers effectively.

The table showing the final score per-product, per-customer looks like the following:

[larger image]

What you can see with this view is that none of the products is ideal for any of the personas. All of the products are pretty good for Kenny, and none of them is very good for Christina, although the iPad 2 is noticeably better than any of the others for Christina.

You can also look at this the other way (columns-first, rows-second), and see that the Nook Tablet is better suited to Kenny than anyone else. If Barnes & Noble’s strategy involved a focus on Kenny, that would make sense for them. Choosing Kenny may have been a bad idea, however, since the Nook [in our made-up data] doesn’t do a better job – for Kenny – than the other products.

This is the view that is most useful in making product decisions moving forward – making this product comparison process useful to you. You can use it as a starting point for doing what-if analysis. You can see what you need to improve to improve your product (relative to the competition) for any particular persona. You can hypothesize about what your competitors might do to improve their products, and what competitive responses you might need. You can determine if you have a good opportunity to differentiate your product – for a particular persona – or if that part of the market is too crowded.

In practice, you can use this view to (a) assess the viability of a particular strategy (that involves “winning” with a particular persona), and (b) perform sensitivity analysis for introducing new (or improved) capabilities. To do the sensitivity analysis, start by asking “how much will this improve the rating (for each persona) for the capability?” Then see what the impact is (in this table) of changing your product’s score(s). Intuitively, this is asking the question – “Would improving (or adding) capability X turn the dial in the market?”

You can also use this framework to imagine the future positioning impacts of anticipated or likely improvements in competitive products. This gives you a mechanism for organizing a strategy of competing with what your competitors will be doing (by the time your product launches), instead of trying to compete with where your competitors used to be.

While this is the most useful view for you – informing your individual decisions – you probably need to roll it up to a crisp answer for your boss – answering questions like:

• Which product is best?
• How far ahead (or behind) are we?
• How much do we need to invest to “win”?
• Should we pivot, or should we persist?

Scoring Each Product Across Customers

Earlier in the process, you established a measure of relative importance of each customer, for executing a given strategy with your product.

Using the numbers for the Blue Ocean Strategy, we found [manufactured] the following relative importance of each customer:

[larger image]

When you multiply each per-product, per-customer score by the relative importance of each customer (in a blue ocean strategy), you get a table that looks like the following:

[larger image]

For the Blue Ocean Strategy, the iPad2 and the Kindle Touch are “better” products than all the others, whereas the PC is by far the worst.

If you were to use the Red Ocean Strategy, where we found [created] the following relative importance of each customer:

[larger image]

When you multiple each per-product, per-customer score by the relative importance of each customer (in a red ocean strategy), you get a table that looks like the following:

[larger image]

In the Red Ocean Strategy, the iPad2 is the clear winner, with only the Kindle Touch threatening them.

You can tell your boss, as the product manager for the Kindle Fire:

• We have to pivot away from the Red Ocean Strategy, to the Blue Ocean strategy, if we want to stand a chance.
• We’re still in 3rd place with the Blue Ocean Strategy, but the key to winning it is to win for Tina, where’ we’re pretty much tied.
• The way for us to win with Tina is to exploit the iPad2’s blind spot – finding more to read.
• Here’s my proposal for how we tackle the Finding More to Read user goal for Tina. In 3 months, we’ll have a better set of features, and 6 months after that the ecosystem around those features will result in us having the better product.

Summary

This is a process for bringing some order, structure, and testability to the process of comparing products – as a tool for informing better future product decisions.

Recapping the overall flow of this series of articles on product comparison

Getting useful information from comparing products requires you to:

1. Introduction and Overview (so that the step-numbers align with the article numbers)
2. Identify your customers.
3. Articulate the problems your customers care about solving.
4. Determine how important solving each problem is, relative to the other problems, for your customers.
5. Characterize how important it is for you to solve the problems of each group of customers.
6. Discover which (competitive) products your customers consider to be your competition.
7. Assess how effectively each competitive product solves each important problem.
8. Assess how effectively each competitive product solves each important problem, for each important group of customers. (This Article)

With this information, you can create a point of view about how your product compares to other products.

This model has evolved quite a bit since I first used an early version of it in 2009. Now that you’ve looked at it too – how would you improve it? When you use it, let me know what you find to be the strengths and weaknesses of this approach.

And thanks for sticking with me over the last 9 weeks, 8 articles, and 16,751 words. Please let me know what you think, and share this with your colleagues if you think they would value it.