Estimating the “value” of features is a waste of time. I was in a JAD session once where people argued about if the annoying beeping (audible on the conference line) was a smoke alarm or a fire alarm. Yes, you can get to an answer, but so what?! The important thing is to solve the problem.
Solutions Versus Features
Everyone on that conference call had an immediate and visceral appreciation of the value of making the beeping stop. That’s the power of solving a problem. The methods of solving the problem – mute the offender, replace the battery, throw the alarm out the window – do not have implicit value. They have an indirect value, in an “end justifies the means” kind of way. But not direct value.
The same sort of thing applies when talking about prioritizing features. Eric Krock (@voximate) just wrote a really good article, Per-Feature ROI Is (Usually) a Stupid Waste of Time, where he does two great things, and (barely) missed an opportunity for a hat trick.
The first great thing Eric did was look at the challenges of determining relative (ordinal or cardinal) value of “several things.” He points out several real world challenges:
- When you have a product with several things already and you want to determine the value of yet another thing – how do you allocate a portion of future revenue to the new thing versus the things you already have?
- When thing A and thing B have to be delivered together, to realize value, how do you prioritize things A & B? Relative to each other?
- The opportunity cost of having your product manager do a valuation exercise on a bunch of things is high. She could be doing more valuable things.
- You won’t perform a retrospective on the accuracy of your valuation. So you won’t know if it was a waste of time, and you won’t get better at future exercises.
The second great thing Eric did was reference a Tyner Blain article from early 2007 on measuring the costs of features. I mean “great” on three levels.
- As a joke (for folks who don’t know me, figured I’d mention that I’m kidding, just in case you get the wrong idea).
- There is some good stuff in that earlier costing article about allocation of fixed and variable costs (with a handy reminder.
- Eric’s article gives me an opportunity to shudder at the language I was using in 2007, see how much some of my thinking has evolved in four years, and improve a bit of it here and now.
What Eric slightly missed is the same thing I completely missed in 2007 – features don’t have inherent value. Solutions to problems do have value. He only slightly missed it because he got the problem manifestation right – it takes a lot of effort, for little reward, to spend time thinking about what features are worth. I also missed the opportunity in an article looking at utility curves as an approach to estimating benefits, written two days after the one on cost allocation. We were both so close!
People don’t buy features. They buy solutions.
Valuing Solutions Instead of Features
Estimating the value of solutions addresses a lot of the real problems that Eric calls out. It also has a side benefit of keeping your perspective outside-in versus inside-out. Or as others often say, it keeps you “market driven.”
Anything that you’re doing, as a product manager, that has you focused on understanding your market and your customers and their problems is a good thing. It may even be the most important thing. I would contend that it eliminates objection 3 – the opportunity cost of estimating the value of solutions is minimal or zero. There may be activities with more urgency, but off the top of my head, none that are more important, for a product manager.
Comment if I’m missing something (it’s late and I just got home from another week on the road).
The way I approach determining the value of a solution is by developing a point of view about how much incremental profit I will get when my product starts solving this additional problem. Revenue can increase from additional sales, or from the ability to increase prices. Cost can increase if new marketing and other operations (launches, PR campaigns, etc) are required to realize the incremental revenue.
I start with a customer-centric market model.
A given solution, or improved solution (as in “solves the problem better,” or “solves more of the problem”) – which only applies to some problems – is interesting to some customers, in some market segments.
A solution has value when it brings in incremental customers, in a targeted market segment. It also has value when it reduces or prevents erosion of your current customer base (in a SaaS or maintenance-revenue model) to competitive solutions.
The time you spend thinking about buyer and user personas, the problems they care about, and the nature of those problems (which varies by persona) is not time wasted – or even spent “at the cost of doing something else.”
To make this useful, you have to have a forecast – without solution A, we will sell X; with solution A we will sell Y (and to whom). A good product manager will be looking at sales, and will be able to reconcile the sales with the projections. That helps with objection 4 (but doesn’t completely address it – you don’t know if your projections were accurate, so you can’t really know if your estimation is accurate).
This also helps you deal with challenge #1. You’ve got a model that says “the current product works great for high school students, but not college students, because they also have problem A, which they solve today by…” Your intention is to create solution A, making your product viable to college students. Allocate the incremental profits from college-student sales to solution A.
My approach to challenge #2 is a little more tactical.
Coupled Solutions
There are a couple ways that Eric’s “must deliver A and B” scenario are interesting, when looking at the value of solutions.
Scenario 1: Solution A solves part of problem X for persona M. Solution B solves part of problem X for persona M. Combined, they solve more of problem X for persona M.
This makes sense for “more is better” problems – where “more” solution yields “more” value.
In this case, I have a forecast (the more time I spend on it, the better it will be) that maps incremental sales to improved solutions. The “first” solution to be released will have more value than the second. If they are being released together, then I don’t care about the allocation – I combine them.
Scenario 2: If, however, the two solutions are valuable to different personas, then I treat them separately – even if they solve “the same problem,” it is not the same problem (for the same person).
Conclusion
Prioritization by “Bang For the Buck” is worth doing. Just make sure you are prioritizing solutions, not features.
Also note: this article talked about valuation – what you do with that valuation, prioritizing by market, can be trickier.
Great post, Scott. Some of us in the product management community have beaten the “problems, not features” drum (e.g. here, but it never seems to sink in.
This blog entry brings an associated issue squarely into view: the false comfort of quantification. When we are rather scientifically able to quantify factors that going into decisions, we tend to feel comfortable that our decisions are better informed. In many cases we merely spent a lot of effort quantifying factors that really aren’t essential to solving the underlying problem.
Hey Roger, thanks very much!
The notion of working in a world of “false precision” has been on my radar ever since I started doing estimation of work-effort as a developer. The way to address false precision in the “cost estimation” world is to provide not a number, but a distribution of probabilistic values – a PERT estimate.
I suspect the same approach would make sense for valuation as well. Pick the minimum (5% or less chance of lower-than-this-value) and max (5% or less chance of higher-than-this-value), as well as the “most likely” value.
I’ll have to think about that some. My guess is that this hasn’t taken off because the retrospective allocation of value just doesn’t get calculated by people. So you don’t know if the estimate of value is worthwhile.
Splitting my responses by topic (now that threaded comments are available).
[segue] Note that the feature “threaded comments” is what enables the new value – “lines of conversation.” [/segue]
Luke Hohmann made an interesting comment on your (linked in your comment) article, about the need to sometimes think about features, and sometimes think about problems/solutions.
I think that makes a lot of sense when you think about customers as both buyers and users.
Users, as I discussed above, unquestionably value solutions and not features.
However, buyers sometimes make purchase decisions based on features, not solutions. The reason they do that is that buyer personas are making purchase decisions based on a mental model of potentially realizable value. They aren’t the experts (to your point of “usability is important – is a wizard the way to get there?”), so they really shouldn’t be basing their decision on the feature (or the checklist) – but they do.
I think I should rethink / follow-up this article with something titled:
Buyers may buy features, but users value solutions.
Thoughts?
Buyers have any number of factors that influence their buying decisions. In many cases, the factors have little to do with the product. Remember Sharon Drew Morgen’s Buying Facilitation(R)? We will never know what unique set of factors lies inside a buyer’s system. Trying to guess the factors and then quantify product decision points accordingly seems like a fool’s errand.
Let’s say we determine that buyers at a particular time are more likely to favor a particular set of features. Basing a product decision on to any substantial degree on this fact is risky. You risk buyer perceptions of features changing at any moment, and you risk pulling your product towards an incoherent positioning and value proposition.
In the past, I’ve quantified the relative appeal of features using conjoint analysis. One thing that choice-based conjoint analysis has going for it is that it in some sense forces respondents to get out of an idealistic or hypothetical mindset and into a decision-making mindset. (Buyers don’t sit around assigning priorities to features in real life; they make purchasing choices, from which choice-based conjoint analysis can help us determine the relative priorities.) However, it still would be much more interesting and revealing to frame conjoint analysis around problems instead of features.
Hey Roger, great points!
I think we can tease out the “how do I sell in B2B” stuff, like SDM’s “how do I help the buyer be able to buy?” and set it aside. I agree that it may be the critical element to closing a sale, but I also think that facilitation of closing a (single) sale is outside the purview of product managers. Same logic as “don’t be demo boy.”
The decision that buyers face is “do I want to buy?” and the decision that users face is “do I want to recommend this to people who trust me?”
The user’s value realization (including usability and other factors that increase the user’s utility) is the key to addressing the word-of-mouth aspects of a go-to-market strategy. That’s they key to what I think of as “ethical product management” – creating a product that provides tangible value. And that’s where my brain was when writing this.
It is fair to remember that the buyer persona has a mental model of how value will be realized by the user(s) for whom the buyer is making a purchasing decision. In B2C, the buyer is usually the user – and the customer (buyer-user) has to deal with the transition from “does this meet my expectations?” to “did this meet my expectations?” – involving a change in expectations.
I think you’re also spot-on that buyers are fickle – and that their mental models of perceivable value are therefore ephemeral. I’m inclined to focus on positioning as the primary way to address this. Not “chasing the market” with possibly valueless features, but rather, focusing on communication.
Feature-centricity is a convenient, even if flawed, short-hand for buyers to articulate their mental models. To your point, they aren’t experts, so the models may be flawed. Communication is the way to address this – helping them improve their mental models, to better reflect user persona’s value models.
I agree with you that problem-centric segmentation / choice-modeling will lead to better insights about the market. Like you, I haven’t seen it out there.
Love these discussions, thanks!
I’m not sure the distinction between B2B and B2C is very relevant here. It’s certainly not that relevant for Sharon Drew Morgen’s Buying Facilitation(R). A number of her examples actually focus on purchasing decisions by individual consumers:
“You don’t need data about the new gym until you have already decided to lose weight, get fit, get up at 3:00 a.m to get to the gym before work, etc. It’s not about the gym.”
“How would you know if it were time to reconsider your hairstyle?”
Whether B2B or B2C, the same systems and change management issues exist, and they are often outside the realm of both features and prospect problems. Furthermore, the notion that in the B2C case the user and the buyer are usually the same rests on a common dubious premise – that there is such a thing as the buyer. Even in the case of an individual purchasing a gym membership, behind the scenes there’s a significant other, children, employers, and colleagues who all influence the decision. It’s really quite similar to the B2B situation where the decision maker faces all sorts of internal buy-in issues.
Now, while perhaps the B2B/B2C distinction isn’t particularly relevant, perhaps the influencers issue isn’t all that relevant, either. You can view the buyer and the user as abstract composites of all the influencers. Then any feature comparisons are really sampling the population of these composites. But it’s hard to get a composite to fill out a survey. That’s why choice modeling – or, more precisely, forcing respondents to make “real” choices as they’d have to make when dealing with all the influencers – is so important.