There’s a buzz going around about the conflict and collaboration between product managers and user experience professionals. It started with a pair of articles co-written by Jeff Lash and Chris Baum. In short, with a user-centric view of products, both roles are responsible for the success of the user-interactions. Who makes the decisions?
Ultimately, since product managers are responsible for the overall success of the product, they are the final arbiters of what the product should do. A good market-focused product manager understands the market context and customer needs and makes appropriate decisions about features and functionality based on first-hand experience and all available research.
Jeff and Chris make a really good point, but note that they do not propose delegating decision-making to the UX professional. While this statement is certainly true, it reads as if there is an opportunity to delegate more.
Roger Cauvin wrote an excellent summary of the conflict and proposes a resolution that involves more delegation.
“A product manager frames the usability metrics… An information architect designs the user interactions and interfaces that will satisfy these usability metrics.”
We agree with Roger (mostly*).
Don’t Do Someone Else’s Job
They won’t do yours.
Very few product managers would argue about “back end” architecture decisions. The decision that the implementation team makes (say relational database versus flat files) is intuitively not a “Why” decision but a “How” decision. Why do product managers then get involved in debates about user interface decisions?
Do you argue for things like:
- The user must be able to order their item with no more than two clicks.
- The UI can not use a CAPTCHA to validate that users are actual people.
- The information must be presented in a tree control.
These are implementation and design choices. Not requirements. They may be classified as constraints, because someone (the customer) mandates it – perhaps you are writing custom software for a company that enforces a style guide or other interface-guidelines. But they are not requirements. If they come from the customers, they are constraints – if they come from you (as a product manager or business analyst), they are “design ideas.”
Usability Requirements
This is a very difficult thing to do. Here’s how we would approach the examples above. Comment below with alternative suggestions, there are probably better ways.
- The ordering process funnel (from decision to purchase until purchase has been executed) must have no more than 10% fallout (customers who initiate, but fail to complete an order).
- (A) The user interface must prevent access by bots (automated external systems). (B) The user interface must meet accessibility guideline [insert spec here] for visually impaired users.
- The hierarchical nature of the information must be presented to the user.
*This is the one area where we disagreed a little bit with Roger above. We completely agree with his view on responsibility, we just disagree on his (referenced) approach to documenting usability metrics.
Comparing Approaches
In the first example, we replace “two clicks” with a fallout metric. This is more aligned with our actual goal (high conversion), without specifying design. If the design had ten clicks, but they were compelling enough that people were not giving up, we would be fine.
In the second example, we specify accessibility criteria for visually impaired users (technically a constraint) and the requirement that automated systems not have access to the system. There are “audio captchas” that could work, and “interrogation systems” (pick the picture of a dog, what is 8 divided by 2, etc) that would not meet our true requirement.
The third example is more obvious – we require that hierarchical information be communicated. We don’t specify that it be in a tree.
Summary
The product manager should be just as “tapped in” to the customers as the UX professionals on the team. That doesn’t, however, mean that the product manager is as good at making user experience decisions. The product manager gets a veto vote, but should be very cautious in using it. Likewise, the product manager can veto the choice to use AJAX vs. Flash. That doesn’t mean he should.
Roger’s approach of delegating responsibility for implementation decisions (including interface design) is definitely the right one. The challenge is to define those requirements without specifying design or otherwise being arbitrary (like specifying a maximum number of clicks).
Also – even if you aren’t interested in the user-experience side of product management, the article starts with another great description of the product manager as the “president of the product.” It’s a good read.
I like the discussion of which metrics best address usability problems. The real challenge is to identify which problems the product will solve. It was perceptive of you to identify high user fallout as a potential problem.
The problem is
“More than 10% of users initiate but fail to complete an order.”
So the requirement is, as you wrote
“The ordering process funnel (from decision to purchase until purchase has been executed) must have no more than 10% fallout (customers who initiate, but fail to complete an order).”
But what are the other usability problems, and how would you measure them? Here are some possible usability problems:
1. Frustration due to expenditure of time. Users complete their orders (fallout rate is 0% or otherwise acceptible) but they end up very frustrated due to how long it took them, so they tell their friends how negative the experience was and don’t ever return to the site.
2. Frustration due to effort. Again, users complete their orders (maybe even in a timely fashion), but they end up very frustrated due to all the “work” they had to perform. So again they spread negative word of mouth and aren’t repeat customers.
By the way, I have never advocated number of clicks per se as a metric for usability.
The metric I have referenced is “user gestures”. User gestures is a generic, technology-agnostic term for a unit of effort that the user expends. Unfortunately, it has the drawback of not being clear precisely what qualifies as a “unit of effort” (e.g. is a control-key combination two user gestures or just one?).
But the point behind that metric is to measure user effort. I am very much interested in other proposals for measuring user effort.
Hey Roger, thanks for commenting and being a regular here!
Usability Sells Software. You’re exactly right.
The challenge is that we’re trying to write requirements around something that is opinion-based. If we say “Users must not be frustrated” we create ambiguity. If we say “two clicks, no more” we specify design.
Which is the lesser of two evils?
I don’t know the answer yet, but I’m inclined to think that we should go with ambiguity and rely on the UX pros to give us a great design. It is like specifying a font family. Which is better – serif or sans-serif (or wingdings!)?
I can’t tell you which is better for usability, but I’m sure one of them is distinctly better for a particular user interface. The decision needs to be made, and it should be made by someone with experience in the field. But it is a design decision, so it isn’t clear to me how to express a requirement that guides the decision in the desired direction.
Generalizing this issue, when working on a custom enterprise application, I’ve made sure that the customers have the opportunity to provide design feedback as well as requirements feedback in order to keep the concepts separate.
Definitely more to do on this front – I’d love to hear how you and other readers have tackled this.
Fair point about “number of clicks” – but we still have a problem – what is the appropriate amount of user effort? I think the challenge is in quantifying the impact of 4 gestures versus 5.
Would love to hear other’s inputs too!
Hi Roger,
Sounds like you’re talking about predicting user satisfaction (or frustration) during product planning/design stage. I do not think it
is possible. Even with very experienced UX help you can only
hope that you minimized the user frustration to some degree but not
eliminated it completely.
I believe you wont be able to solve the issue completely without
some kind of a feedback loop. Let your users try the product and
then ask them what they think. (Ask them to rate their satisfaction
or fill out a very simple survey after each transaction.
Or let UX people talk to the users and gather the feedback).
Then immediately incorporate the results of the surveys into your
next product plan.
May be I am missing something. It is different when you’re trying to release your very first version of a product (no users yet, only UX expert estimates) vs. releasing an incremental version (real users, real feedback).
For the very new product you would usually do a beta programm to generate that missing user feedback. Look at Google with all their ‘beta’, ‘by invitation only’ services :).
-Alexey
Roger fell victim to our spam filter – here’s what he sent in email:
Scott,
For some reason, two comments that I tried to post to Tyner Blain have not gone through. If they are somehow queued up awaiting your approval, please post only the last one, which should read:
“Fair point about “number of clicks†– but we still have a problem – what is the appropriate amount of user effort? I think the challenge is in quantifying the impact of 4 gestures versus 5.”
A couple of years ago, I wrote a blog entry about this issue. Identifying the metric is arguably more important than defining particular limits on the metric. Even placing a specific limit the fallout rate is somewhat arbitrary.
“The challenge is that we’re trying to write requirements around something that is opinion-based. If we say ‘Users must not be frustrated’ we create ambiguity. If we say ‘two clicks, no more’ we specify design.”
In theory, we can measure user frustration in terms of neurorequirements. It’s not very scientific, but we could even measure user frustration by asking them to indicate how frustrated they are on a scale of 1 to 10.
But we may not want to commit to solving the user frustration problem (just as we wouldn’t want to commit to achieving a certain ROI). That’s why the appropriate level may be measuring the causes of user frustration: duration and effort.
A quibble: a specification limiting the number of mouse clicks is not necessarily design. It doesn’t mandate or even necessarily presume that mouse clicks will be in the user interface at all. But it would likely be an incomplete requirement, as it leaves out all sorts of other user gestures that could cause frustration.
Roger
Glad that the articles that Chris and I wrote have sparked so much good discussion and thinking. One of the goals with the articles was to highlight the close relationship between UX and PM, and this discussion here seems to be getting to the heart of it.
This hits particularly close to home because I’m working on a project right now and the UX team and I are struggling with a particularly thorny design problem. The product requirements are clear and based on good solid research about our users and customers. We all agree on what we want the design to let people accomplish, it’s just that we’re having problems coming up with a good design to do that. We’ve done a lot of usability testing, gone through multiple iterations, tweaked various aspects, and we’re getting closer, but we’re still not there.
The “metrics” approach would say that I provide the requirements for my UX team — e.g. 80% of users must be able to accomplish this task successfully — and then they go off and come up with a design that meets that requirement and we implement it. In reality, we’ve all been working closely together on this, trying different ideas and discussing various approaches. We’ve asked others for input as well (developers, business analytsts, designers from other teams), just because we don’t have something that we feel comfortable with and that works well enough for users.
Ultimately, I agree that “Roger’s approach of delegating responsibility for implementation decisions (including interface design) is definitely the right one,” and in most cases that works very well, assuming you have good UX designers and a PM who understands this. In this particular situation I’m dealing with, we have had to take a more collaborative approach to come up with a successful implementation because it’s a much harder problem to solve.
Depending on what we come up with, this may be a case where I have “a veto vote, but should be very cautious in using it,” though I’d like to think that we still have a chance to come up with something that everyone — me as product manager, our UX designers, and our customers/users — is happy with.