Design-Free requirements are important for two reasons, and hard for two other reasons.

Design-free requirements are hard because you “know what you want” when you should be documenting “why you want it.”  Writing design-free requirements can be hard when you don’t trust your development team to “do the right thing” even though it is not your job to design the solution.

Design-Free Requirements – Revisiting

It has been three years since I wrote Writing Design-Free Requirements as part of The Big Ten Rules of Writing Requirements.  In that time, agile development practices have moved from being an esoteric development methodology to being the topic on the tips of everyone’s tongues as executives and organizations try to either (1) get the benefits of the state of the art in software-development process or (2) do something shiny and fashionable.

The previous article centered on elements of designs within MRD, PRD, and SRS artifacts.  A regular alphabet soup of artifacts.  Honoring the values behind the agile manifesto encourages us to emphasize working software over comprehensive documentation.  In that light, we’ll take a “what is needed” approach to talking about how requirements, design, and implementation are all needed – rather than issue an edict about where that information should be captured.

Agile Development Inputs

When creating software, someone needs to know:

• What problems are being solved, and how important (valuable) they are to be solved.
• Who has the problems and who is using the software to (help) solve those problems.
• What constraints limit the space that defines the universe of possible viable solutions.
• What acceptance criteria define if the delivered solution will be acceptable.

Subject to those inputs, someone needs to make design decisions:

• User experience design - what interactions will a user of our solution love?
• Program design – how (in a nuts and bolts way) will our solution function?

When talking about agile and talking about design, we should take a look at how Kent Beck and Alan Cooper, as respective though leaders in each space, view this.

Cooper doesn’t talk much about creating the requirements to support the daily use scenarios – he proposes moving directly into design of the solution. This differs from the more traditional technique of writing functional requirements to support use cases. Cooper also breaks down design into two components – program design and interaction design. Program design is everything you don’t see. Interaction design is everything you do see.

[...]

Cooper argues that designing the interaction should happen before any code is written. He uses a construction analogy to drive home his perspective – you can’t pour the concrete before you build the forms. Kent Beck, founder of the XP programming philosophy disagrees with the premise. Beck believes that the cost of changing software is low, and the imagery Cooper uses is hyperbole. We touch on, and link to that debate in this post.

Interaction Design Process Overview

I don’t believe that Beck is arguing for “don’t do design” – I believe he is arguing for “don’t do big upfront design that would delay implementation.”  He’s championing the agile values that emphasize creating working software and responding to change.  I can imagine him saying “no one will buy a design, they want a solution.”  Cooper’s point is that create a solution without first understanding how someone will use it, you can’t create a great product.

Program design has many hidden impacts – cost of maintenance, cost to change, and the performance of the solution.  You can have a design (or architecture) that makes everything easier to do in the future – at the cost of delaying the delivery of anything.  Or you can have a design that minimizes the time to deliver the first thing, while increasing the cost to deliver any particular next thing.  Or you can design a solution that falls somewhere between those extremes.

If you say “we don’t do design” you’re lying.  Every solution includes design.  Your team’s design process may be “big and upfront” or it could be a couple sketches on a whiteboard, or some email exchanges.  You may create storyboards and wireframes.  Or design may be the thing that happens right before the programmer’s fingers strike the keys.  Design may be explicit or it may be implicit – but it never “doesn’t happen.”

In the movie Amadeus, Salieri is astounded that there are no rough drafts of Mozart’s compositions.  That’s because Mozart did the design in his head, not because he didn’t do design.  I’ve worked with programmers who’s “implicit designs” were great, but that is the exception, not the rule.

Who Owns Design?

Since design happens sometime before fingers strike the keyboard, the real question is – “who owns the design?”

• You have a product manager developing an understanding of market problems, and prioritizing the problems that should be solved with your product.
• You may have a product owner managing the backlog and clarifying those requirements (problems) for the development team.
• You may have an interface (or interaction) designer or team of designers who are broadly responsible for “how users interact with our products.”
• You may have an architect or lead engineer who is responsible for the “big picture design” of how your product works.
• You have developers and testers who are responsible for delivering your product. [Note: coding without testing is like typing code without compiling - maybe it works, but probably not.]

You may not have distinct individuals with each of these titles – every small team I’ve worked with has people wearing multiple hats.  This is especially true when you have an agile team full of specializing generalists – any given story (or task) may have a different architect and different implementer.  I’ve only worked with one company where the architects are NOT part of the development team.  If your team is set up that way, please comment below – I had never seen that before, and I’m not convinced that it is the best way to organize – what have your experiences been?

Generally, “program design” is clearly owned by the development team – and product managers (and product owners) know better than to specify program design in their requirements.  Neither the lead engineer nor the product manager believes that the product manager is a better programmer – so the product manager better not be specifying program design and mislabeling it as requirements.  Coincidentally, Steve Johnson, at Pragmatic Marketing has a post running right now with a bit of a quiz – is this a req(uirment) or a spec (design)?

Where the line is more blurred is around interface and/or interaction design.  Some development teams have interface designers as part of the team.  Some companies organize with interface design as a “shared service” within the company.  Either approach can be the “right one” – it depends on too many details to make a sweeping generalization.  When the designers are members of the development team, the solution (from a product management perspective) is the same – the “team” is responsible for all design.  When the designers are not part of the development team, the developers have to consume two sets of guidance – “solve this problem” from product management, and “the solution needs to look like / act like this” from the designers.

Collaboration between product management and user experience people is the ideal solution.  The “requirements” and “design” inputs to the development team are comprehensive and consistent.

Design-Free Requirements

There are benefits – especially when being agile and minimizing documentation – to delivering requirements and design at the same time.  Just don’t do it by embedding design constraints within the requirements.

• When people on your team misinterpret design as requirements, they are unnecessarily constrained.
• As a product manager – are you the best designer on your team?  If you’re busy designing, who’s product managing?

This is trickiest when writing use cases – sequencing a set of interactions is interaction design.  That is one benefit of writing a user story instead of a use case.  An approach that has worked well for me with multiple teams is to deliver user stories (requirements) combined with storyboards (interaction design) and wireframes (interface design).  When details are needed (usually when “changing” versus “creating” an interface), screenshots can replace wireframes.  When business processes are complicated, process flows can replace storyboards.

Just don’t embed the designs within the requirements.

Your company is building out a toolkit to support third-party developers.  You’ll need a bunch of different types of widgets – combo-boxes, text entry fields, domain-specific controls, etc.  You’ve got a long list of desired controls from your customers.  You’re agile.  What do you build first?

Agile In A Soundbite

Being agile is about delivering incremental value, quickly, getting feedback, and then delivering more incremental value.  Repeat until “done.”  Good agile adds a qualifier – do the most valuable thing quickly, get feedback, do the most valuable thing (that has not already been done) quickly. Better agile optimizes the rate at which you deliver value by taking into account both benefit and cost.  Great agile overlays a focus on getting better at doing all of these things while you do them – becoming a learning organization.

Boiling the Ocean

A product manager I was coaching was faced with a challenge commonly referred to as boiling the ocean. His team was tasked with solving a market problem, and they were constrained to doing it with a service-oriented architecture that exposed a set of widgets (user interface controls) that customers could easily integrate into their products.  This approach was designed to provide competitive differentiation by reducing the time and cost to deploy solutions that allowed customers to integrate his product into their existing platforms.

In initial conversations with customers, technologists, and architects, this product manager quickly amassed a list of desired widgets (controls) and scenarios (stories) in which they could be used.  The product manager’s team had recently switched to an agile development methodology.  The internal stakeholders, not yet accustomed to agile development, wanted “all of the widgets,” now.

This product manager was able to convince them that the team would deliver the widgets incrementally, following agile principles.

His question was – How do I sequence the widgets in the backlog?

Defining Widget Priority

The product manager had a list of widgets, combo-boxes, data-grids, text fields, radio buttons, etc., and for each widget, he had a real-world scenario showing how the widget could be used.  Most of the scenarios involved customers using multiple widgets.  He wondered if he should do some sort of analysis that detailed the frequency of use of each widget.  ”This widget is used in 7 scenarios, so it should be done first; this widget is used in 5 scenarios…”

There was a strong temptation to add several “Create a widget” tasks to his backlog.  It would be easy for his development team to estimate the effort required to deliver each widget.  His team wanted to deliver incrementally, and “one widget at a time” felt like logical, discrete chunks of work to them.  They could easily sink their teeth into estimating, building, and testing each widget.

A quick reminder of a main tenet of agile, delivering incremental value, illuminated the flaw in this approach.  This approach would have been an inside-out prioritization, when value delivery requires an outside-in perspective. (See the ‘recommended reading’ suggestions on this page to check out Kessler and Sweitzer’s great book.)

If the team used this approach, after the first widget was complete, they would be able to deliver exactly 1 task, and 0 stories.  Development teams don’t deliver tasks, however.  The team’s customers would not be able to get incremental value from having a single widget.  The team would have delivered seven incomplete stories – so they would have delivered no stories at all!

We reworked his approach as follows:

1. Assess a value for each story, making sure that each story would enable his customers to accomplish something valuable.
2. Engage their user interface / user experience designer to design a solution for the most valuable story.  This design was constrained to use widgets in the user interface.  The suggested way to communicate this design was with a storyboard and low-fidelity wireframes.
3. Identify the widgets needed to be built to deliver that story, using the designer’s design.
4. Deliver the story to the development team, including the storyboard, wireframes, and list of widgets to be used as acceptance criteria.
5. Get the development team to size (estimate) the effort to complete each story.
6. Where stories were too big (epics), collaborate to identify good ways to break up the stories into manageable chunks.
7. Repeat steps 2 through 6 until the amount of identified effort was likely to fill the release (keep the dev team busy delivering value).
8. As the team delivers each story, get feedback from the market, revisit the prioritization, and revise the “next” story.

What we discovered was that a scenario like the following (modified for this article) played out:

• The first story required two widgets.  As no widgets existed, both had to be built.
• The second story required three widgets, but two of them had been built to support the first story – only one incremental widget had to be created.
• The third story used one of the widgets from the first story, but required additional behaviors.  The original widget was modified to provide incremental capabilities (and value).

A Note On Team Structure

Astute readers will notice that there was a design step “inside the requirements process” – before the stories were delivered to the development team.  Technically, that is true, for the way this particular company was organized.  The user experience designer was not a member of the scrum team, but rather, an external consultant who supported multiple teams.  The product manager engaged the designer prior to engaging the development teams.

This just as easily could have happened as follows:

1. Product manager identifies, values, and prioritizes stories to be added to the backlog.
2. Development team, as part of sizing the stories, engages their user experience designer to select the appropriate widgets, and those design decisions inform the estimation process.
3. Everything else is the same.

This procedural variation does not have design “inside the requirements process.”  The same people do the same things, in the same sequence in this scenario.  The only difference is that the interface / interaction designer is a member of the development team.  That would be ideal, but that was not how the company was organized.

In this particular example, having the product manager collaborate with the UX designer made the most sense.  It introduced less complexity for the product manager to accept responsibility for this activity than it would have for the development team (who was still new to using an agile delivery cadence) to do it.

Conclusion

The key ideas at play here:

• Focus on realizable value to the customer (outside-in development), not tractable tasks (inside-out development).
• Collaboration with a UX professional is key to driving “interface requirements.”
• Deliver frequently and valuably, get feedback (learn), and incorporate that knowledge into whatever is next.

Perpetually intermediate (competent) users.  Users who briefly exist as novice users and never become experts. Most of your users are competent, and you should design for them.  Competent users have different needs and different expectations than novice or expert users.  How do you know your user’s competency levels, so you can design for them?

User Competency

User competency is a concept I first read about in Alan Cooper’s The Inmates Are Running The Asylum.  Cooper’s contention is that the level of expertise of your users follows a bell curve, or normal distribution.

When we’re designing software we need to keep in mind that most of our users will be competent – neither experts nor beginners. Alan Cooper’s studies tell us that user skill levels follow a bell curve. He talks about competent users as perpetual intermediates. Some users drop out of the bell curve when they stop using our software. The rare user becomes an expert. Most users only learn enough to get their real job done.
Competent Users and Software Requirements

Cooper contends that the combination of learning curves and natural user tendencies to stop learning or abandon a software application are the sources of this distribution of user competency.

Experience Curves

Experience curves represent the diminishing costs over time of manufacturing something repeatedly.  The process of manufacturing something gets more efficient as you get smarter about the process.  The process of using software is at least analogous to, if not a specific example of a manufacturing process.  If you’re using an email application, you are manufacturing email messages.  In a CRM system, you are manufacturing contacts, or contact reports, etc.

By treating any “using the software to do something” interactions as a process, you can measure the cost (how long it takes) of the user’s interactions.  Applying the math behind experience curves, you can predict the reduction in cost (to your users) over time, for any set of interactions.  Experience curves take into account that some processes are inherently more learnable than others.  This property of learnability is reflected as an efficiency coefficient – how efficiently someone can learn ways to reduce the cost (time) needed to perform the interactions.

This gives us an approach to quantitatively model user competency.  Having a definition allows us to model competence.  And measuring competence allows us to manage product design in the context of user competency - designing for competent users.

Defining Competence

The first step to measuring competency is to define the model.  I am proposing a definition in this article that I suspect will yield insights (to help us manage our products). I was unable to find any quantified definitions of competence, when researching it as part of a client engagement.  If you have, or know of a model, please share it in the discussion below this article.

• A competent user is someone who learns to perform a task in half the time it initially took them.
• An expert user is someone who can complete a task in 10% of the initial time.

This definition is guided by an expectation that Alan Cooper’s premise about perpetually intermediate users is true.  Being a novice user is a very transient state, and becoming an expert is very infrequent.  The goal of the definition is to be able to segment your users and make well-informed design decisions.

A Proposal, Not a Doctrine

This is a proposal that doubling performance reflects competence, and achieving a ten-times improvement represents expertise.  It may be that some different measure of performance improvement more accurately reflects competence and expertise.  We have to test it to know.

The experience curve is defined mathematically by Henderson’s Law.  It states that the time to complete a task is a function of the number of times you have previously done that task, adjusted by the “elasticity” of the cost of that task.  In other words, some tasks are easier to improve than others.  If you populate a table with the results of applying Henderson’s Law, you get the following:

[larger image]

• Each row in the table represents the Nth repetition of a task.
• Each column represents how easy that task is to learn – progressing from “hard to improve” on the left, to “easy to improve” on the right.
• Each cell in the table represents how long it would take to perform the Nth repetition of the task, as a function of how easy it is to improve your performance at the task.

The above definitions represent what experience curves predict mathematically, when the task initially takes 60 seconds.  The following definitions reflect the proposed user competency model:

• A user is a novice user until she has learned enough to cut the time-on-task in half.  These cells have a white background (and are in the upper left area of the table).
• A user is a competent user when the time needed to complete the task is between one half and one tenth of the initial time.  These cells are shown with a yellow background (and are in the central area of the table).
• A user is an expert user when she can complete the task in less than one tenth of the time required to initially complete the task.  Theses cells are shown with a red background (and are in the lower right area of the table).

In the absence of empirical data, I used my intuition to suggest that a representative experience curve for a typical task performed in software would be one with an “elasticity” of 50%.  For a task with those learning characteristics:

• A novice user would cut the needed time in half on the 4th repetition of the task, and would be considered to be a competent user.
• A competent user would further reduce the time needed to one tenth of the initial time on the 100th repetition of the task, and would be considered an expert user.
• The 50% elasticity column is surrounded by a black border, and the number of repetitions required to advance to competent or expert status is also highlighted with a black border.

Inferring Competence

Since different processes have different learning characteristics, you have to figure out how easy it is for your users to improve at your processes.  To do that, you have to study (or at least measure) your users’ interactions with your software.  In the 50% curve highlighted above, a user is capable of cutting their time-on-task in half by the fourth time they perform the task.

If data from your initial testing (or measurement) reveals this to be true, then you have selected the correct curve (the correct column in the table).  If it takes more or less time to reach this level of improvement, shift to the left or right to find the appropriate curve.  If software-interactions are reasonable analogs to manufacturing processes, then the experience curve projects an expected rate of improvement on task.

The following graph isolates the time-on-task data for a user who is learning to improve when repeating a task (process) that matches the 50% elasticity curve.

[larger image]

Note that the number of repetitions of the task (the X axis) is represented as a logarithmic scale.  The data points along the curve correspond to the cell values in the table above (for the 50% column).

Shades of Gray

One nice thing about this quantitative approach to inferring competency by measuring usage is that your measurements are per-process.  Users are not “purely novice” or “purely expert” – they can be experts at some processes, while remaining neophytes at other.  There is also awareness, for any particular process, of “how much competency” a user has.  This allows you to refine your assumptions of the steepness of the learning curve, and of the thresholds (doubled performance and ten-times performance improvements).

Improvement Over Time

Any particular learning curve can be considered relative to calendar time, to see how quickly a user will progress along the curve (as a function of frequency of use).  This can be useful for determining the ROI of improvement in a particular process.

The following graph shows how an 80% learning curve overlays a calendar for tasks that happen daily, weekly, and hourly.

[larger image]

The graph shows that for a weekly frequency, after 16 weeks, the task time has reduced from 300 seconds to 100 seconds. With a daily frequency, the task time is even lower – about 60 seconds. This graph shows nothing other than converting the academic learning curve graph into one that incorporates calendar time and frequency of occurrence.

Software Usability and Learning Curves

One approach to inferring user competency would be to measure how long a user has been using your software.  The variation in how frequently different users perform the same task will introduce an error into that inference.  You can avoid introducing that error into your modeling by counting the number of times a user has performed a task.

Applying the User Competency Model

The advice in previous articles, and from Cooper’s book, and from this great article on the  Coding Horror site, encourages us to focus on the competent users.

I’m working with a client who needs to prioritize a set of capabilities and establish design principles for a product.  We will incorporate this user competency model as part of our analysis.

Hopefully we’ll have an opportunity to collect data to validate and / or refine the model.  I’m proposing that we first gain some insight into the which users (novice, competent, expert) drive the most revenue and profit from use of the product – to establish the importance of each category of user.

For this product, I suspect that we will find many more novice users than a normal distribution would predict.  If that is true, the next question will be to understand if we are dealing with a normal behavioral dynamic, or if characteristics of the current product “force” novice users to abandon it before they achieve competence.

Either way, we will have a framework for prioritizing the goals of the novice, competent, and expert users.

How would you apply a model like this to improving your product?

When defining requirements, you always start in the context of a goal – either a user goal or a corporate goal.  You need to be aware of both.  Having a positive user experience is important, and requires a user-centered understanding.  Achieving your corporate goals might be in conflict with some user goals.

User Goals

A user centered, or user-centric approach to developing software is one where you start by identifying the key persona in your target market.  For each of those personas, you identify their most important goals.  You then identify the user stories or use cases that represent the things these people do in order to achieve their goals.  These “things” are manifested as practical goals.  For these activities, you design user interfaces (process, layout, architecture, look and feel, etc) that create customer delight for those users, doing those things.  This is how your product exceeds the suck-threshold (good enough that it doesn’t suck to use your product).

Corporate Goals

There’s a reason your company is creating or improving a product.  It may be to gain market share, or improve profitability, or increase sales.  Whatever it is, there’s a corporate goal that your decisions should be supporting.  The (subset of) corporate goals that are relevant to your product could be communicated in a vision and scope document, that constrains the domain of problems to be solved, and provides nuanced insights into viable approaches to solving them (example vision and scope).

Corporate goals are achieved when customers do things with the products.  These “things” are manifested as practical goals.  For those activities, you document what should be accomplishable and why.

Alignment and Conflict of Goals

Notice that both the user goals and corporate goals manifest in terms of people doing things.  When the person (user) wants to do the same things that your company wants them to do, you’re in alignment.  When the user goals and corporate goals suggest different activities, you’re in conflict.

You can see visually that these worlds collide at the “practical goals” stage  in the following diagram, from an article on combining interaction design and structured requirements principles.

It may be great – when the user and corporate goals are in alignment, the practical goal and associated scenario (activity) are easily defined.  What about when the goals are in conflict?

Vending Machine Example

Last Thursday morning, as I interacted with a vending machine, the idea for this article was dispensed along with my Diet Mountain Dew.  Yes, I know that’s odd.  Move on.

It occured to me that the process of buying a cold, caffeinated beverage can be both aligned and unaligned from the perspective of user and corporate goals.  If you imagine writing a use case for purchasing a beverage from the vending machine, your most important scenario is the one where a person wants to purchase a beverage that is available in the machine.

• User Goal: Get refreshed and vitalized.
• Corporate Goal: Sell a beverage.

I want to buy a Diet Mountain Dew, as a means to realize my personal goal.  The owner of the vending machine wants to realize her corporate goal of selling me the beverage I want to buy.  We’re in alignment.  Our requirements definitions and design decisions will be pretty easy – everything that increases the likelihood of and improves the experience of purchasing that beverage is good.  I put my money in the machine and push the button.

Then it occurs to me – there’s a situation where my personal goals are clearly not aligned with the corporate goals of the vending machine owner.  When there is no Diet Mountain Dew available to be purchased.

Consider the following procedure I’m forced to endure:

1. I view the available beverages that this machine dispenses – Diet Mountain Dew is one of them.
2. I put my money in the machine.
3. I push the button for Diet Mountain Dew.
4. The tiny display on the machine presents a message in taunting, scrolling red LED lights: SOLD OUT.

At this point, I’m faced with a choice – select a less desireable beverage, or request my money back and try to satisfy my user goal in some other way.

Consider an alternate procedure:

1. I view the available beverages that this machine dispenses – Diet Mountain Dew is one of them.
2. I view the ‘current availability’ of Diet Mountain Dew and see that this machine is currently sold out of Diet Mountain Dew.

At this point, I’m faced with a choice – select a less desireable beverage, or request my money back and try to satisfy my user goal in some other way.

The differences in these two possible procedures indicate that there is a conflict between the corporate goal and the user goal.  With the first procedure, the “Sell a beverage” goal is given more importance, because it makes me more likely to purchase a beverage that I don’t particularly want.  My utility is potentially sacrificed, while the owner of the vending machine is just as satisfied.

By failing to tell me that I can’t get what I want, until I’m further invested in the process, I am more likely to purchase something else.  That purchase creates just as much value for the owner of the vending machine, but less value for me.

If the vending machine owner had allowed me to use the second procedure, it would demonstrate that the  owner believed the improved customer experience, while risking the loss of this sale, would encourage me to purchase more over time.  The owner would have been prioritizing the requirements and design that supported the user goal ahead of those supporting the corporate goal – in hopes that my user loyalty would result in better word of mouth (more business from other people) and more business from me over time.

Generalizing User and Corporate Goal Conflicts

In this vending machine example, the annoying procedure is probably the right one – I’m going to treat the “point of use vending of cold, caffeinated beverages” as a commodity product.  This market has a very low barrier to entry, and relatively small problem being solved.  I’m not going to tell my friends about how much I love the vending machines from Joey’s Vending and encourage them to go out of their way to find and use those machines.  Nor will I purchase more Diet Mountain Dew because the emotional cost of the occasional “sold out” scenario is trivially smaller with Joey’s machines.  I would, however, be more likely to buy a Diet Pepsi when I’ve already invested time in the process and put my money in the machine – especially when the alternative is to find a water fountain.  I’m emotionally invested at this point.

However, this experience did jump out at me as one that is worth explicit consideration when defining requirements and designing solutions.  It does come up regularly.

The new iPhone 3GS hardware allows tethering (using the phone as an internet-connection device for your laptop).  ATT (the exclusive carrier for iPhones in the USA) is indicating that they will charge customers an extra monthly fee for this capability, when they eventually allow it on their network.  The user already has a data plan, and the traffic that ATT would have to carry is just “data.”  The user just wants it to work.  ATT, however, may want more money, or may want to limit data traffic on its network – perhaps to improve the quality of service to all customers.  This could be ATT’s corporate goal of “provide better service quality” conflicting with the user’s “increase the flexibilty of how I use my data plan.”  The former would likely influence customer satisfaction (and abandonment) over time, while the latter would influence customer acquisition rates.  Another conflict in goals.

Think about how different goals can lead to conflicting solutions, requirements, and designs.  And tell me when you’re out of Diet Mountain Dew before I’m emotionally invested in the purchase.

Ever scratch your head and wonder why you can use your favorite application for free?  How can a business actually make money (and stay in business) when they offer their product for free?  This article looks at the freemium business model, to see when it makes sense for a company to offer it.  The freemium model is one where the company offers two (or more) versions of a product.  The basic version is free to use.  You have to pay for the premium version.  The goal of this article is to answer the product management question, “Should you create a freemium business model?”

Economics of a Freemium Business Model

One way to look at the freemium business model is to look at it per user.  A user will either use the free (basic) version, or for-a-fee (premium) version.  

By definition, a freemium model is where one user is faced with a choice – do I use it for free, or do I use it for a fee?  

We will look at how to encourage users to move from the free version to the for-a-fee version later.  For now, we’ll just look at the impact of that choice.

Billing Peter to Pay for Paul (Freemium)

Every free user gets benefits, for free.  Every for-a-fee user pays for the benefits of the product.  The customers who pay for your product also cover the costs you incur when providing the service for free to other customers.

As a company, you have to look at your aggregate user base to analyze the economics.  Some percentage of your users pay for a product (premium), and another percentage do not (free).  What makes this interesting is asking the question – “What percentage of your users will pay when a free version is available?”  Basecamp, from 37signals just celebrated its 5th anniversary, and serves as a good illustrative example.  Note that 37signals expressly does not share this information, so we have to speculate.

• In a 2006 interview with Ryan Carson for ThinkVitamin; Jason Fried, owner of 37signals, indicated that it was “more than 0.87%” – we’ll call that 1%.
• In a 2009 interview with Brad Spirrison for MidWestBusiness.com; Fried indicated that 90% of revenue comes from subscriptions to web applications.  Spirrison points out that 37signals earns $40,000 monthly from their job board – so we’ll estimate $360,000 per month from subscriptions.  We can sanity-check our 1% estimate.  Fees for 37signals products range from $24 to $149 per month.  If the average paying user pays $36 per month, then there would be 10,000 paying customers – 1% of a million.  We could tweak the numbers (the average might be lower, there may be more than a million users, etc).  But this data is consistent with a 1% conversion rate.
• Jed Christiansen did an analysis about a year ago where he estimated ~ $5,000,000 per year, with numbers very consistent with the Spirrison interview.  Jed built his estimates up from the usage stats that 37signals reported (links at Jed’s article), and some assumptions for converting from usage to number-of-users.  His estimates would put conversion somewhere around 0.5% to 1%.  He provides a spreadsheet of the model too, if you want to tinker with it.

This feels reasonable – 100 free users for every paying user.  Even if that number is wrong, the rest of this article holds true, but it sometimes helps to have a number to think about.

The Left Hand Doesn’t Know What the Right Hand is Paying (Not Freemium)

This is the situation where a user gets a product or service for free, and a different user gets a different product or service for free.   Technically, this is not a freemium model – the same user does not choose between the two options.  User A chooses between free-product-A and not using anything.  User B chooses between for-a-fee-product-B and not purchasing anything.  User A has no interest in product B and vice-versa.  A company may offer a free product or service using this business model, and it may make sense – but it is not freemium, because the same user is not choosing between the two different products.  A company may also use a freemium business model, but augment it with this business model.  The following examples are examples of this model, listed here to avoid miscommunication:

• A company can offer a product for free to (primary) users, then charge advertisers (secondary users) to display ads to the primary users.
• A conference may offer the opportunity to speak/present (for free) to lecturers, then charge attendees to listen to the lectures.
• A government may offer waivers on corporate or property taxes to a company to build a new facility, then levee payroll taxes against the employees for the priveledge of working there.
• A shopping mall may host free events (like christmas pageants) to the general public, then charge the retailers for rental space in the mall.

In each of these scenarios, the users who get the free product or service are not choosing it relative to the paid product or service.  Different users are targeted for each.  The first model (advertizing supported products) is easy for everyone to identify, but all of the examples share a commonality.

Freemium Product Costs and Prices

Isolating the freemium business model from other revenue-generating opportunities, you can see that finding a profitable model can be tough – you have to control costs and set prices correctly.  Assuming our data from above is representative (and I don’t know if it is), if 1% of customers are paying customers, then each paying customer has to cover the costs of 100 customers to have the possibility of being profitable.

Quick Cost-Model Refresher

From a management accounting standpoint, for a given product, there are two types of costs.  

• Fixed Costs – these costs are the same for the company, no matter how many users there are – additional (incremental) users add no incremental costs.
• Variable Costs – these costs are the same per user – incremental users add incremental costs.
• Total Costs – the sum of fixed and variable costs.

There are also two important ways to look at profitability – overall, and per product sale.

• Total Profit – The sum of all product sales minus the total costs to make and sell the product (including overhead).
• Contribution Margin – The difference between product (or service) revenue and the variable costs to make and sell the product.

When the total revenue from product sales exceeds the total costs to make and sell that product, the product is profitable.  From a decision-making standpoint, the contribution margin needs to be positive. The number of products that need to be sold for the company to be profitable is the fixed costs divided by the contribution margin.  Here’s an example (using a software as a service pricing model):

• Your business has $10,000 per month in fixed costs.  
• Your product has a variable cost of $0.10 per month per user.
• 1% of your subscribers pay for their subscription, 99% subscribe to the free version.
• You price your product at $20 per month per user (per unit subscribed).

That looks like a hell of profitable product – some people will pay $20 for something that costs you a dime.  But looks are deceiving.  You have to cover the costs of the free subscribers, and you have to cover the fixed costs of making and selling your product.

 [click for larger image]

You have to get to 100,000 subscribers (1,000 paying customers) just to break even.  This takes much longer than you would expect when selling dimes for $20!  Even a 25% per month growth rate can’t help you early on.

 [click for larger image]

The exponential growth does start to compound, but it delays break-even.

The reason this happens is that you have to pay for 100 free-account subscribers with the revenue from each paying customer.  The contribution margin is the key here, and three things have to be true or you shouldn’t have a freemium business model.

1. You have to have a contribution margin that is positive, when taking into account the ratio of free users to for-a-fee users.
2. You have to have a sufficiently large user base (number of users – more precisely, paying customers) to cover your fixed costs.
3. You have to lower your costs (if (1) is not positive or high enough) and grow your user base (if (2) is not large enough) fast enough to get profitable before you run out of funding.

Growing Your Customer Base

There are a two ways to grow your customer base – traditional marketing to grow your customer base, or word of mouth marketing to grow your customer base.  If you’re relying on word of mouth marketing, there are two different dynamics that drive word of mouth [thanks to Jonathan Berkowitz of Thinktiv.com for this insight!] – altruistic and selfish.  

Altruistic – This product helps me, it will help you too.  You should use it.

Selfish – It helps me if you start using this product.  You should use it.

Discussion of the two different word-of-mouth patterns will have to wait for the next article.  I’ll update this one with a link to that one when it is written.

[Update: Viral Product Management is now posted!  Thanks all for the great attention to this one so far.]

You want your software to be used, not to sit on the shelf. You can’t achieve the ROI of your software if people don’t use it. And you can’t achieve the ROI of your software by forcing people to use it either.  Some will fail to achieve the benefits, and others will delay using it or refuse to use it entirely.  You have to make them want to use it, and you have to design the software for the users who must use it.  Otherwise, you won’t achieve the ROI.

The ROI of User Adoption

The image above is great – it shows one person choosing to use the software, and one person choosing to do the work by hand instead. It is a classic user adoption problem. The good thing about it is that you can talk in concrete terms about the goals of your software, and how user adoption plays a role.

Let’s start with a definition of return on investment (and read the full article for more details and examples)

ROI is the acronym for return on investment. Another way to think of it is “How much profit will we make if we invest in this project?” Profit is revenue minus costs. Technically, the question should be “How much profit will we make, relative to our investment, if we invest in this project?”

Definition of ROI – Return on Investment

OK, with a definition of ROI, you have to develop a model for return, and one for investment. The model for investment is built by looking at the cost of developing the software. That defines how much it costs. Assume it costs $1,000,000 to develop and deploy the solution.

The model for return is a little more complicated.

For our user adoption example, assume that the software saves the company $1 every time it is used, relative to whatever the company does in the absence of the software. There are several ways to calculate the ROI of design, we’re picking one Assume we have 1000 users, each of whom would use (or not use) the software 10 times per day. That is a potential savings of $10,000 per day. Call it $2,000,000 per year once all the users are using it all the time.

Unfortunately, many people stop there. They see a 6-month payback period, they fund the project, and they stop thinking about it – the money is already in the bank. But you’re not going to do that.

Imagine that this is your project. You write perfect requirements, you have an excellent development team, and the software works perfectly – saving a dollar every time it is used. The problem is, only 100 people are willing to use the software. Now, instead of a 6-month payback period, it takes five years to recover the cost of developing the software.

The False Promise of the Mandate

“Not a problem for us,” you say. “Everyone must use the software. Or they’re fired!”

Many corporate IT departments work this way. The business tells them what they need (“Save us a dollar!”), and the IT guys gather requirements, develop a perfect application and confirm that it saves a dollar every time someone uses it. And then they mandate that everyone use it. Simple. 6-month payback.

There are a couple problems with this approach.

1. Not everyone uses it right away, even if everyone is required to use it.
2. Not everyone uses it effectively, even when required to use it.

The explanation is a little less obvious, but the brutal reality of it is just as true.

Delayed User Adoption

You issue a mandate – all 1000 users will use the software! The users are conveniently spread out into 10 business units, each with 100 users. One of those business units will go first. And your software is a disaster (we’ll get to #2 in a minute). The plan was to pilot the software with the first 100 users, then roll it out a month later to the other 900. No problem – 7 month payback, and less risk to the business.

The problem is that the people running the business units – and thereby managing the users – are smart too. They see the train wreck that Tom’s department became, and they refuse to use the software. They have their own financial targets, and they don’t want to jeopardize them. These business unit directors, if they lack the power, will escalate, until their demands are heard.

What’s the first place in the hierarchy where there’s a common manager? Is it the CEO? Maybe you’re lucky, and it is only a couple VP’s arguing about it. What conclusion do you think they will reach when presented with the following arguments:

• If we use the software, we will miss our numbers – it slows people down. And that will cost you $X.
• But we want them to use the software – they have to. If they were smarter, it would save money

Even if the argument isn’t accurate – who will win? Best case, your roll-out is delayed. Worst case, it is killed before it ever gets a chance to work.

Ineffective User Adoption

It turns out, after watching the pilot group, that expert users can in fact save a dollar every time. Competent users break even – there’s no savings for them. And beginners were better off with the old solution. It was less efficient, but the new way is so hard that they can’t do it well.

You’ve mandated that people use the software. Since they didn’t otherwise want to use it, you had to spend a bunch of energy (and money, in opportunity cost) to make it happen. The rollout was delayed, and at the end of the day, you break even. Most users have no savings, and the few that do are offset by those that caused an increase in cost.

Mandating user adoption creates a false sense of confidence, and does not assure ROI – it only assures adoption. We’ll say that again, to let it sink in.

Mandating user adoption does not assure ROI – it only assures adoption.

So what can you do?

Measuring User Adoption

As product managers who care about user adoption, our first thought is “measure it!” The biggest challenge is in measuring the right thing.

We can’t just measure how many clicks, because while that might correlate with ease of use – it does not necessarily cause increased user adoption.

To stay aligned with the ROI model, we have to measure, directly, that user adoption is meeting the forecast used to create the ROI model. The only measurement that is assured to be an accurate reflection of user adoption is measurement of user adoption directly.

Prevent Mistakes, Then Monitor Them

If we measure poor user adoption after it has proven to be inadequate, that doesn’t help very much. It is generally accepted that a good design leads to higher user adoption rates. In the case of the “false mandate,” good design yields faster roll-outs and allows more users to be more effective with the software.

There are also other reasons to invest in good design.

The right solution is to invest in good design – targeting competent users (not novice users) to achieve the desired ROI. A good design is easier to learn too. You can track the rate of improvement among users.

Here’s a chart from an article that goes into learning curves in more depth:

[larger image]

The graph shows that for a weekly frequency, after 16 weeks, the task time has reduced from 300 seconds to 100 seconds. With a daily frequency, the task time is even lower – about 60 seconds. This graph shows nothing other than converting the academic learning curve graph into one that incorporates calendar time and frequency of occurance.

Note the odd, vertical drops in task time during week 1. This is just a manifestation of looking at a weekly time scale. For a task that happens once per hour, the user will rack up 40 repetitions during the first week. From a decision-making standpoint, you don’t have time to react to that rapid rate of learning, so it is ok that it is collapsed on this graph.

Software Usability and Learning Curves

Thanks, now go hire some designers and achieve your ROI!

We use actor hierarchies to organize the different users of a system. Different people play different roles, and thus do different jobs. We use different actors to identify and organize those people. When deploying a system globally, we usually discover people that do the same jobs, but do them differently. Incorporating the notion of personas lets us deal with this.

Why Actor Hierarchies?

User-centered, or user-centric design is an approach to developing software that focuses, as the name implies, on the users of the system. When designing the solution – and more importantly, when defining the requirements for the system, special attention is placed on the users of the system. More specifically, center stage is devoted to developing software that helps the users accomplish their goals.

From a requirements perspective, the key is to acknowledge that there are different types of users, doing different types of work with the software. It is reasonable to conclude that they will have different needs. An actor hierarchy provides a way to organize the different needs of the people doing different jobs. [Note: These are the same actors that are identified in use cases.]

A Simple Actor Hierarchy

Let’s start by creating a simple actor hierarchy. We’ll build on this incrementally as we add more complexity to our user base. We’ll look at a hierarchy for sales representatives (reps) for a company that manufactures goods for sale to other businesses. Each business to which the company sells products is known as an account.

The hierarchy represents an approach to classification of sales reps. Each successive level of the hierarchy is more specific than the previous level. All employees who sell products are considered to be sales reps (the top level in the hierarchy). When the company sells products to its accounts, it is not always obvious which products should be sold. In those cases, an engineer is brought in to help determine the exact need, and suggest the right products to sell. This engineer is known as a technical sales rep and the technical sales rep works with an account rep to determine the right products and terms to close the deal. The account rep is responsible for making the individual sale, and occasionally gets help from the technical sales rep to do it.

The hierarchy shows that all sales reps are either technical reps or account reps. If an employee is a sales rep, then he must be one of the two choices.

Account reps are further divided into two groups – representatives who work with large accounts, and reps who work with small accounts. The company currently considers accounts that make more than $500,000 in purchases per year to be large accounts, and the rest are small accounts. The main difference in behavior is that a large account rep works with only one customer – and in fact, for really large accounts, there may be multiple representatives for a single account. Small account reps, however, each work with more than one account, maybe even dozens of accounts.

You could argue, in the abstract, that large and small account reps should not be treated as different types of actors – because they all manage relationships with customers and sell products. However, there is enough of a difference (in our hypothetical example) between the jobs of the two types of reps that it is worth treating them as separate actors.

In our example, the large account reps are focused on account retention, and maintaining relationships with a single customer. These reps are focused on account retention and are usually managing the upgrade cycle for their customers. The upgrade cycle is when a customer replaces their existing, worn out or obsolete equipment with new equipment. Small account reps might be working with really large potential customers who currently buy from the competition. These small account reps are focused on acquisition and growth. They are also trying to manage and create relationships in an attempt to grow the customers until they become large accounts. Small account reps are also managing multiple customers, and have additional activities associated with managing that complexity.

Global User Base

The situation above describes how the company (or if you have a product management hat on – the companies in the target market segment) organizes their sales force in one region. What do you do when the the company has operations around the world? If all global operations are the same, then you don’t do anything. It is a rare company that works this way. Most global companies have grown their regional businesses (where a region might be Europe or Asia, or it might be China or India) independently. This could be do to growth by acquisition, or the parallel growth of separate regions managed separately.

In different regions, companies often have different business models. They also usually deal with customers in different ways. These differences may manifest from varied local business practices, different characteristics of the customers, or different levels of maturity in the market for the products being sold. There are also differences in regional regulations, policies, and laws – although you should generally manage those as variations in business rules, not business practices. We’ll side-step that layer of complexity for the purposes of this article. Pretend that the same laws, policies, and regulations are in effect in all of the regions.

The same general actor hierarchy can serve to represent the sales reps in multiple regions, with some modifications to account for the differences. Consider for our example, that the company sells products both in North America and in Latin America. The actor hierarchy above describes how the sales team in North America. The company’s operations are a little bit different in Latin America (for our example).

Latin American operations are relatively new to the company. Where the company has a well established market in North America, it has relatively low market share but very rapid growth in Latin America. As a result of the way the company’s business has evolved in the two different regions, there are two distinct differences between the two regions in the way that sales reps are organized.

1. Technical Sales Reps. In North America, technical sales reps (TSRs) are salaried employees, who participate in a profit sharing program. In Latin America, the technical sales reps have a base salary, but sales-commissions make up a large portion of their compensation. The North American TSRs are therefore incented to propose solutions that are highly profitable (thus increasing their profit sharing bonuses). The Latin American TSRs are incented to propose solutions that may sacrifice profitability in exchange for increased revenue (thus increasing their commissions) .
2. Small Account Reps. In North America, small account reps, like all account reps, are employees of the company, and work on commission. In Latin America, small account reps are not employees of the company. Small account reps are third-party sales people. The model is very similar to independent insurance agents. The small account reps, when trying to bid on a job, can propose solutions from the company or from one of the competitors. The third party small account rep also gets to set the price that his customer will pay for the products, and negotiate the price that the company will accept – pocketing the difference.

In both situations, the user populations are different, and will behave differently. The TSRs in both regions do fundamentally the same work – but with different motivations. The Small account reps in Latin America, are markedly different from the ones in North America – in motivation, relationship with the company, and the work that they do.

Using Personas To Express Regional Differences

Personas provide an effective way to describe archetypes of classes of users. Persona development is a key component of the interaction design process, but it can be leveraged very effectively in a traditional requirements process as well. Each persona represents a homogeneous population of users. In other words, the collection of users that are in a role represented by a single actor, and who have similar personal goals. By defining personas, we can manage requirements in the context of those personal goals. One actor in the actor hierarchy may have multiple personas associated with it. This makes sense when there are groups of individuals with different personal goals.

The most common example, and one that is almost always present, is built around the notion of user experience. You can classify users of a product as novice, competent, or expert users. Novice users usually become competent users but rarely become expert users. Those people who become expert users are usually a different breed. Professional drivers become experts, where casual drivers rarely exceed competence.

These two user groups are made up of people with different goals, who use cars differently (even though they both drive, they drive differently, and with differing expectations and needs). It is critical to separate user groups that are different, and treat them differently, to avoid the elastic user problem.

The two different technical sales rep (TSR) populations – in Latin America and North America – would be best represented as two different personas. The TSRs in both regions do fundamentally the same work – provide technical support to an account rep, to help in closing a sale. The North American TSR is profit motivated, where the Latin American TSR is motivated by increasing revenue. These different goals make it very unlikely that the same persona would accurately describe the TSR population in both regions. There are other factors that drive the identification of multiple personas for the same role (in the actor hierarchy), but to simplify, we are focusing just on the regional dynamic.

Since the TSRs in both North and Latin America are performing the same job, it does not make sense to treat them as having different roles in the actor hierarchy.

Using Additional Roles in the Hierarchy to Express Regional Differences

In situations like with the Small Account Reps in North and Latin America, it is better to create distinct roles in the actor hierarchy to represent them.

The North American Small Account Rep works on commission and is an employee of the company. This rep is focused on closing the deal, but only with products from the company. This rep also may work with other reps to help them close deals – sharing intelligence about market pricing, helping with product selection, etc. There are elements of teamwork and collaboration present because all of the reps are “on the same team.”

In Latin America, the Small Account Rep is also focused on closing the deal. But this rep does not care who’s products are used to close the deal. The rep will work to get the lowest possible price from the company, in order to maximize the difference between the company’s price (to the rep) and the rep’s price (to the customer). 100% of this difference goes to the rep, so even if the rep is handsomely rewarded or commissioned based on transaction margin, the rep will always make more money by negotiating a lower price from the company. The rep also has additional activities associated with getting quotes from the competition and managing relationships with the company’s competitors.

Not only are the jobs different, but the company will treat these two reps very differently. For example, the company would be likely to share cost or margin information with the North American rep. That information will help the rep know how low he can price products for the customer without jeopardizing the company’s profitability. The company knows that the rep wants to close the deal at the highest possible price, to increase margins and therefore increase the rep’s compensation.

The company will not want to share cost information with the Latin American rep. The company is still motivated to get the most profitable deals, but the company realizes that the rep is trying to get the lowest possible price – not the highest possible price. This means that the company needs to provide low enough pricing for the rep to close the deal, and low enough pricing that the rep will resell the company’s products instead of products from the competition.

Summary

Actor hierarchies provide a way to differentiate groups of people who have different roles. Personas provide a way to describe groups of people within a role who share motivation and characteristics. The combination of the two allows for clear, yet flexible expression of requirements – both from a business perspective, and from a user perspective.

When dealing with user populations in different regions of the world, it makes sense to extend the actor hierarchy when the jobs, and business requirements of the user populations vary. When the users are doing the same job, but with different motivations, using personas to describe the differences makes more sense.

Visualizing complex data can be very difficult. There are almost as many ways to visualize data as there are data to visualize. The Ralph Lengler and Martin J. Eppler at the Visual Literacy Organization collect many of them for us in a periodic table.

References

Hat tip to Kevin Kelly for sharing this with us in his Cool Tools blog. The periodic table is available at the visual-literacy.org website. Hovering over each element provides an example of the type of visualization.

Different Types of Visualization

One of the biggest challenges in visualization of complex information is in selecting a means to visualize it. Many approaches simply don’t fit the data – they have too few, or too many variables. For example, a pie chart is great for showing percentages (37% of new iPhone purchasers switched carriers, 5% had no service previously, 40% were existing AT&T customers, 18% did not respond) or relative proportions. The pie chart is not effective at showing trends or changes in the data (last year 20%…). You could have multiple pie charts, but there are more effective ways to show the trends in the data.

The periodic table that the Visual Literacy folks put together would otherwise be overwhelming – each element represents a way to visualize information. When you have something to visualize, it would be difficult and tedious to hover over each element, until you saw something inspiring. So they’ve organized it into six different categories of visualization.

• Data Visualization – line chart, pie chart, scatterplot, etc.
• Information Visualization – radar chart, tree map, data flow diagram, etc.
• Concept Visualization – mind map, layer chart, pyramid technique, etc.
• Strategy Visualization – supply & demand curve, failure tree, life cycle diagram, etc.
• Metaphor Visualization – tree, funnel, bridge, etc.
• Compound Visualization – cartoon, knowledge map, info-mural, etc.

There is also an overlay of text color (blue versus black), where blue elements (like the timeline) represent process visualizations, and black elements (like the venn diagram) represent structure visualizations.

There are also symbols to indicate if the visualization technique is suited to details, overviews or both. And more symbols to indicate if the diagram is intended to cause viewers to come up with new answers (e.g. solutions to a problem), or to indicate that the diagram is designed to clarify insights found in complex data (like the stakeholder rating map).

Conclusion

There are some very cool techniques presented. What would make the site even better would be if each element were a link to a page about that particular type of visualization. Roll-over graphics are handy for easily comparing different techniques – hopefully the authors will add detailed pages for the techniques. Of course, that is a huge job. If they added one per week, it would take two years!

Check them out – they will inspire thoughts when you’re really stuck.