Welcome to product management! Over the the better part of three months, Adrienne Tan at brainmates, product management people, put together a series of posts with ten tips for new product managers. Check out our article for a quick summary, and links to all the articles at brainmates.

Ten Tips for New Product Managers

Thanks to Adrienne and the folks at brainmates for putting together this list! The commentary is ours – there is a full article on each tip at their site. Adrienne’s tips are not just about being new to product management, but also about taking over an existing product management position.

1. Speak to People and Listen to What They Say

Find out who all the players are within your organization, who interact with, rely upon, are responsible for, and help create your product. Talk to them. Find out the good, the bad, and the ugly. Your goal at this point is to get an overview. Remember, when joining a new team, it is always best to listen first and absorb, and only then should you speak.

2. Use Your New Product

This is both important and surprisingly overlooked. On a project with one of our first clients, I installed the application on my first visit to the client site. A couple months later, an irate director was screaming on a conference call when none of the people on her team, except the consultant, had even bothered to install the application – including her developers.

Adrienne also makes a good point about not picking up other people’s baggage – “we’ve always done it that way”, “it’s never been a priority”, etc. Do your own research, form your own opinions.

3. Review Your Key Performance Indicators

Knowing what to measure as a product manager can be very very difficult. Determining what to measure about your product can be a lot easier. Make sure you know what is important about your product – what contributes to your strategy, how effectively are you generating or increasing ROI?

With an understanding of the business impact your product is having, you can make informed decisions about your road map – and invest where you need to, and not where you don’t.

4. Understand What Your Market is Saying About Your Product

Understanding your market is the key to developing a market-driven solution. Avoid the tyranny of one (I think I heard that at a Pragmatic Marketing seminar). Don’t just listen to the loudest customer, and don’t rely only on your perspective. Get tapped into your market and stay there. As Adrienne mentions, this involves not only understanding the people who buy your product, but the people who don’t buy your product. Win-loss analyses can help with this.

5. Locate and Review All Documentation About Your Product

Review the MRD and PRD for your product, if they exist. If they don’t exist, create them. Different teams will apply different levels of formality and rigor to these documents. The point is – you are codifying your vision, which allows you to communicate it, and help it evolve.

6. Be the Best

Adrienne provides several good tips for being the best at what you do. Our approach is simply that the way to become the best is to become better. Most career advice books will sometimes tell you to improve your weakest characteristics (eliminating the excuse for people to pass you over for a promotion). Another approach is to improve your strongest characteristic – further differentiating yourself.

As product managers, especially at smaller companies, we are often asked to do “pretty much everything.” This means we need to have at least some skill or knowledge in many areas. If you’re missing key skills, gain them one at a time. Once you have them all (at some level), use your product management skills to prioritize – focus on the one that will have the most impact.

For me, about half a million words ago, it was getting better at writing. There is still room for improvement, but other areas will have more impact, so I focus on them.

7. Review Your Predecessor’s Task List

When you walk into a new job, there’s usually a backlog of things that need to get done. If you’re lucky, the previous product manager has all of the short term needs addressed, and her task list is about defining a new market, or developing a strategic initiative. More likely, there’s a demo that needs to be built, an irate customer who didn’t get his feature in the latest release, and a development team that has implemented everything that they were told to do before you arrived.

Work the task list. Prioritize and execute. Remember, you’ve got the context you need from talking to people and understanding your market. Use it.

8. Plan to Update Your Skills

All of the standard professional improvement activities are as true for product managers as for anyone else. Adrienne suggests several to think about.

9. Enjoy Yourself

Aside from the importance of leading by demonstrating the honest enthusiasm that comes from having fun – it is important that you have fun too. I have a friend who is an outstanding consultant. He’s the best person I’ve seen at establishing peer level relationships with clients. And he’s smart. He took a role as a product manager, and hated it. He was also self-aware, and got out of that role in a couple months. Turns out, as much as he values having good requirements, he hates writing them. He’s a successful technical consultant again. He knew what most people fail to realize – if you don’t enjoy a job, don’t do it.

10. Take Steps to Be a New Product Manager Again

It is easy to get caught up in the execution mode of a product manager. You’re being asked to do too much. You can barely make time to do all of the strategic stuff you need to do, with all of the short-term demands that others place on you. Just don’t forget to allocate some time to the items in this list. You need an opportunity to improve – your competitors are.

Conclusion

Pay attention to what Adrienne has to say – it’s good stuff. And just because you’re already a product manager doesn’t mean you shouldn’t do these things.

In a slight segue from our agile project articles, here are five tips that may help other first time Ruby on Rails / Capistrano deployments. Even with the great resources available on the internet, there were some unexpected and obscure hurdles for a new-to-rails developer to get a site up and running. While Ruby favors convention over configuration, not all hosts are set up with the same conventions – so there isn’t much help available for the really weird problems.

First, let me say that I heart Capistrano. And after less than two weeks using Ruby and Ruby on Rails, I am very excited. That “two weeks” bit is important – experienced people don’t tend to run into the problems that I did – or they at least don’t tend to write about them enough.

Capistrano (Cap) is a tool for deploying software to (one or more) servers while they are running, that allows you to do the updates with minimal overhead – it synchronizes steps, restarts servers, and automates “everything.” An agile development project will deploy software very frequently – so completely automating that process is a great thing. Cap can deploy software that isn’t Ruby on Rails, and it can use a source code control system that isn’t subversion. If you’re doing that stuff, this article probably isn’t for you, but Cap probably still is.

There are several good tutorials and explanations of what to do, in what sequence. And the process is actually very simple. The challenge comes from tutorials that say “type X, then type Y.” Well, what if “X” throws an error? Then what? Occasionally an article would explain why I needed to type X. But most articles glossed over those details. Perhaps X normally doesn’t fail.

But the devil is in the details, and trying to find the exorcism rituals with Google was tedious and occasionally fruitless.
The “solutions” presented below may not give you complete answers. Think of them as clues to help you Google for more expert advice. Like a software development scavenger hunt.

That said, here are the first five “Oh Crap!” and “Oh No!” challenges I faced while attempting to deploy my first prototype with Capistrano.

First Five Capistrano Tips

1. Imitation is More Than a Form of Flattery

Site5 is my web-host, and they host Ruby on Rails sites, as well as subversion repositories. Last week I installed Ruby on Rails and subversion, and created an application prototype. Over the weekend, I attempted to deploy it, and failed. One likely source of problems – Site5 was running different versions of Ruby and Rails than I was. Running different versions of Rails is fine – because you can “freeze” your version of rails within your application. Running a different version of Ruby was possibly not so smart. Note – this tip is for deploying into a shared server environment. If you control the server, you still want to match versions, but you have the choice to change either end.

Earlier this week I rebuilt my environment to use the same version of Ruby as my host.

2. Shebang Nabit!

Most Ruby on Rails developers apparently use a version of Unix for their development environments. Apple OS-X and Linux are both versions of Unix. I am using Windows XP. My Site5 server is running Linux. Every third article warned me to not try and deploy to a server running Windows, so, “whew!”

At the top of a few important files in your code is a “shebang” line – #! c:\dev\ruby\bin in my case. This line tells the operating system where to find the Ruby executable. This allows you to issue commands without typing “ruby” first.

The location of the Ruby files on your dev machine is irrelevant. You need to change this so that the server can find Ruby without your help. That probably means changing the line to #! /usr/bin/ruby. You can find out exactly what you have to change it to by typing whereis ruby into a command prompt on the server. I used putty to create a bash shell command prompt. You may have to request access to get this (I did) – but this part is covered in many tutorials.

3. Is The Computer Plugged In?

In the early days of tech support, many a laugh was had among the IT people when resolving a user’s problem amounted to turning on the computer, or telling them not to stick floppy disks to the side of the desk with a magnet. Ah, the golden years.

I got myself into a bad spot, trying to roll-back my software versions to match my host. Suddenly, my dev environment didn’t work any more either. The error messages didn’t help me to google the solution, so my problem must have been way out in the long tail of stupid mistakes. My theory? I must have misused subversion, and created a mixture of old and new files.

The reality? My dev environment was not working because my local mySQL and Apache servers were not running. When I turned them on, it worked. Pebkac (problem exists between keyboard and chair). So, make sure they are running in the future. Also – I received a helpful suggestion that WEBrick (the Ruby-based development-environment web server that is installed by default with Ruby) will occasionally throw inexplicable errors. Well, I’m sure the errors make sense, but as a n00b (that’s l33t (elite) speak for plebe), they would certainly be inexplicable to me. Forewarned is forearmed.

4. Ask for Permission, Not Foregiveness

Later on, Capistrano was doing everything great, apparently. I checked code into subversion, typed cap deploy on my dev machine, and magically, the latest version of my code was now on my server, ready to run. Except it wouldn’t run.

File permissions is something that winxp mostly takes care of for you – you never have to think about it. To most users, “permissions” invokes images of someone password protecting an excel spreadsheet and emailing it to you. Well, permissions matter a lot on a server. They define who can do what with which file or directory. Who is an interesting concept to a server. I have a user account that I use to install my appication. Capistrano installs files for me, as me. But Apache (the web server) may not be running as me, since this is a shared server.

There’s a file called dispatch.fcgi that Cap has deployed for me, that Apache needs to interact with. chmod 755 dispatch.fcgi takes care of that problem.

5. Equivalence is NOT Equality

With the help of some awesome guys at AustinOnRails.org, we took a very instructive debugging approach – manually doing on the server, one step at a time, what Capistrano was going to do for me later. Eliminating variables and issues step by step.

One thing Capistrano does is get the latest source code out of subversion and put it where it belongs on the server. OK, easy to do. svn co file:///.... is the command to get the latest version of source from a subversion repository on the same machine. I was typing the command through my bash shell (running on the server), so that made sense. And it worked. Once we had everything running, we were ready to let Cap do its magic and automate the steps we confirmed manually.

Well, Capistrano runs from your local dev machine. And it uses the command svn co svn+ssh://... to push data from subversion to the server. And it appears to work. But it didn’t actually work.

It turns out that getting the same source, from the same repository, and putting it in the same place, is equivalent but not equal. Subversion is smart. And when we used the “file” accessor to create a working copy of the source the first time, it knew that the “svn+ssh” (tunnelling) accessor for the same stuff, to go in the same place, is actually not the same. So we didn’t get any updates.

Most people probably never have to do the manual steps that I did. So they don’t run into this problem. The solution – delete everything. Scary. Only delete the files and directories that Capistrano created. And don’t blame me when you delete something you shouldn’t. But delete is what I did. Your mileage may vary. Then run cap setup, and if that works, cap deploy. And it will pull down the latest code from subversion. Then change something, check it in, and run cap deploy again. It will get the changes.

Conclusion

Seriously. Love this stuff. Wish it had been as easy to deploy the app (as a new-to-deployment developer) as it was to create the app (as a new-to-ruby developer). I hope google finds this, and puts it somewhere where it helped you. Let me know if it did.And experts – please correct any boneheaded things I said above, or provide some links to great tutorials to help people on their scavenger hunts.

Active listening is about more than gaining understanding. Active listening is about giving. Giving assurance that you understand someone’s needs. Giving confidence that you will address those needs. Giving feedback and acknowledgment that someone’s input is valuable. If you haven’t tried active listening, you may think it is a passive, receptive activity. Here are ten active listening skills that will help you, your customers and your team.

What is Active Listening?

McGraw-Hill’s accurate yet insufficient definition of active listening is “Giving undivided attention to a speaker in a genuine effort to understand the speaker’s point of view.” That’s fine, but it doesn’t tell you why you give undivided attention, and it doesn’t tell you how. And active listening is much more that paying attention really well. That implies a one-directional communication. Active listening is bidirectional. This definition doesn’t tell you what you’re giving to the speaker and that’s the most important part.

Why Practice Active Listening Skills?

There are plenty of generic reasons to be an active listener. Dale Carnegie swears that active listening is the key to making a great first impression (he’s right). Dr. John Gray promises to improve your cross-gender relationships with active listening that gets to the heart of the unimaginable motivations of the opposite sex [tongue in cheek].

As a business analyst or product manager, you have some very specific goals and challenges. Joy at Seilevel recently wrote about the challenge of convincing people that documenting requirements is important. She looks at the motivation of the naysayers, and recognizes that the reasons may not be logical, or they may not have the data. This is a great example of the kinds of situations, beyond gathering requirements, where active listening skills can help.

The Center for Rural Studies [yeah, I know, it is odd] provides a great list of ten active listening skills. We’re re-purposing that list here to

• Make you more successful.
• Make your products better.
• Make your customers happier.

Ten Active Listening Skills

1. Acknowledging. You send cues to the speaker that acknowledge that you are hearing them. You have to demonstrate that you understand the ideas. Make eye contact and dilate your pupils, raise your eyebrows, nod your head. This is more than just acknowledging that you here the words (and be aware of different cues in different cultures). You have to acknowledge the ideas to be effective. When you don’t understand something, you can make the “I don’t get it” face.
2. Restating. The best way to overcome missed signals in the non-verbal attends that represent acknowledgment is by restating what you just heard. The key to restatement is not reiteration, but paraphrasing. You demonstrate that you’ve absorbed a concept by rewording it. If you missed a key concept, your reworded restatement should make it obvious to the speaker that you missed the idea. In an ideal world, she will be practicing active listening skills too – and will restate your restatement, providing another means for you to grasp the idea.
3. Reflecting. Imitation is the most sincere form of flattery. Subliminal imitation is sort of what reflecting does. You pick up on the body language and emotions of the speaker, and reflect them back at her. Several “how to get ahead” management books will tell you to emulate your boss. This is because we are all pre-wired for a little bit of xenophobia. We tend to like people who are “like us.” More importantly – by demonstrating that you are developing the same reactions as the speaker, you are affirming that you have reached the same understanding.
4. Interpreting. Ah, the psychiatrist’s trick. “I see from your uncontrollable twitch that this has upset you. Tell me why…” This is generally focused on being empathetic, and encouraging people to talk more. However, this is also where you can ask for clarifications, to make sure you understand what your speaker means. “Are you saying you need ‘drag and drop’ because you need an easy to use interface, or because people will be using a tablet pc with no keyboard?
5. Summarizing. When you ask a good open ended question, you will get a relatively long response. When applying active listening skills to requirements gathering, you often want to let your speaker go off-topic a little bit, as it helps identify other requirements. Make notes of those other topics for followup, then summarize the parts of the answer that addressed your initial question. You’re reinforcing for the speaker that you understood why they said what they said, and that you didn’t muddy the waters with the other information they gave you.
6. Probing. People often summarize in order to communicate effectively. Developers will use design patterns to allow them to describe detailed software implementations in a word or two. People in general will use symbols as replacements for compound ideas. Ask a clarifying question or two to assure the speaker (and yourself) that you understood what they intended you to understand. This can also help you with credibility with the speaker, as it demonstrates some knowledge of or comfort in their domain.
7. Giving Feedback. By sharing your opinions about particular ideas, you create a collaborative bond with the speaker. You should focus on affirmation of their insights or ideas, instead of criticisms. If you tell someone that their question is stupid, you encourage them to shut down and shut up. Listen to speakers or panelists in a Q&A session – they regularly start their answer with “That’s a great question.” The really good ones will say “That’s a great question, because…” Without the because, the feedback can start to sound like a pre-programmed platitude. Quickly snap off half a dozen “Great Question. Here’s my answer.” answers without the rationalization, and people will tune it out as noise. If you’re struggling for responses, use anecdotes. “That’s a great question, I had a client who never asked it – and here’s how the disaster unfolded…”
8. Supporting. Validation of the speaker’s ideas and concerns is important. By supporting their worries as being valid, and ultimately resolving those worries, you create loyal customers. Dell recently launched their Idea Storm website to elicit exactly this kind of feedback (among others). If they add their voices to the mix, providing support for the ideas that people present, they will get more and better ideas. If they follow-up, they can demonstrate a clear cause-and-effect for their customers. “You said it. We did it.” That would generate some loyalty!
9. Checking Perceptions. When you’re actively listening to someone, in addition to getting data, you are forming impressions and perceptions. You need to check the validity of those perceptions with the speaker. When gathering requirements, this often leads to identification of the true requirements, and even implicit requirements. You’re also letting the speaker know that you “get it.” This is a great opportunity to double up on the supporting and feedback active listening skills with responses like “I think that is a great requirement, because it will prevent incorrect orders from being shipped, and that will reduce field-servicing costs. Or was there a different benefit you had in mind?”
10. Being Quiet. Interviewers use this technique all the time. Silence can make people uncomfortable, so they tend to fill the void – the only way they can, by talking more. While this is effective for confrontational interviews, there are more positive and enabling reason to do be quite. First, you need to temper your exuberance to provide feedback, support and summaries, so that you appear to be listening and not talking. You’re meeting with someone so that you can understand their perspective – not so that they can understand yours. Second, you want to give people time to think. If you are creating a “take all the time you need,” positive, supporting silence, they will use that time effectively. The attends and emotional affirmations you’re providing are what make it supporting and not interrogative.
11. [Bonus] Extension. This is a variation on the restating technique. Many people you interview will be providing you with data from a discrete perspective. When gathering requirements, you have to find a way to abstract that information into market requirements. People also often talk in terms of their existing tools and processes. You may be getting great information, but it may be overwhelmed in implementation details, either about how they do their job, or how they envision the future system to be. A great way to validate that you’re generalizing the salient parts of their ideas is to extend the ideas. “I need to be able to sort the accounts receivable list by name, even though Joan sorts them by outstanding amount” can be combined with other requirements and extended to “Each user shall be able to organize outputs by field in the UI.” And you just discovered that Joan is a stakeholder who uses the same functionality for a completely different purpose.

Conclusion

Active listening skills are critical to communication, and more importantly, collaboration. Requirements gathering – for product managers AND business analysts, is the essence of collaboration. It isn’t input only. Use your active listening skills to make the conversation bi-directional, and you will get better information. You’ll also have better products, and happier customers.

[Update 2007/02/12: Welcome TamsPalm readers from Carnival of the Capitalists. We hope you like what you find and stick around. Let us know what you think!]

Return on investment calculation is critical to using ROI for prioritizing requirements. We’ve discussed how to forecast return on investment by estimating costs and predicting benefits. Here are five tips to help you when calculating return on investment.

Summary of ROI Tips

The following ROI calculation tips are detailed below:

1. Recognize the Risks
2. Discount Future Cash Flows
3. Separate Sales From Expenses
4. Overcome Ozymandias Syndrome
5. Ignore Infinite Elvises

Here are the details…

1. Recognize the Risks

You can’t predict the future with certainty. You have to account for the range of possible returns on our investment, or the risks to those returns. One way to account for risk is to assign a set of probabilities to the possible outcomes, and determine the expected value of the return. If you believe there is a 50% chance that your investment will return $1,000 and a 50% chance that it will return $2,000, then the expected value of your investment is $1,500.

Another approach would be to use PERT estimation to calculate your expected return. To use PERT, you identify the worst case, most likely case, and best case returns. You may not have the data you need to accurately calculate a return with a PERT estimate. If you do not have the data, you should use a simple expected value calculation.

The PERT estimate is a representation of a beta distribution. The math is pretty complex, and applying it to a single estimate can give us a false sense of precision. Remember that the numbers we put together in the first place are our best guesses, so doing more precise math on rough estimates doesn’t give us a more precise PERT estimate, just more math.

The beta distribution is very similar to a normal distribution (the familiar bell curve), when it is balanced. This similarity is used to simplify the application of PERT. The standard simplification in using PERT is to treat estimates as if they represented the following probabilities:

• Optimistic: 5% of the time we will complete our task in less than the optimistic time estimate.
• Likely: 50% of the time we will complete our task in less time than the likely time estimate.
• Pessimistic: 95% of the time we will complete our task in less than the pessimistic time estimate.

Foundation Series: Basic PERT Estimate Tutorial

The discussion above applies to calculating the returns. We also need to calculate the costs in order to determine ROI.

Costs are usually represented by a combination of known expenses (investments, purchases, etc) and predicted expenses (labor to create something, relative labor increases, etc). There may also be probabilities (risks) associated with your cost forecasts. You should use the same approach for predicting costs as you do for predicting returns. It is more likely that you will be able to use PERT estimation for labor cost estimates.

Our costs can also be a combination of fixed costs and variable costs. We have to take that into account as well, and again, adjust for risk.

2. Discount Future Cash Flows

A dollar a year from now is worth less than a dollar today. Think about it – if I offered you $100 today in exchange for $100 a year from now, you would take it. But if we reverse the exchange, you’d be foolish to take it. Economists call this discounting future cash flow. To determine today’s value of tomorrow’s dollar, you compute the net present value (NPV) of that dollar. A future dollar is discounted by the interest rate that you could earn if you chose to invest your money (risk free) today. Check out our net present value tutorial for more details.

3. Separate Sales From Expenses

Sales dollars and expense dollars are apples and oranges.

At first glance, the following two investments might look to be equivalent:

1. Spend $10,000 to increase sales revenue by $50,000.
2. Spend $10,000 to cut costs by $50,000.

Sales dollars represent top-line revenue (money coming in the door), and expense dollars represent bottom-line earnings (money going out the door). What many people overlook is that sales dollars do not equal profit dollars. Profit dollars are usually called margin dollars and are equivalent to expense dollars when calculating return on investment.

Determining Margin Dollars

When you sell a product you have costs associated with that sale. Those costs can be separated into fixed costs and variable costs. The fixed costs are fixed – an increase in sales will not change them. But variable costs will increase with increasing sales. For every new dollar in sales revenue you get, you also incur additional variable costs. The difference between sales dollars and variable costs is called gross margin dollars.

An increase in sales of $1,000 with a corresponding increase in variable costs of $800 would yield a gross margin increase of $200. The $200 is your increase in profits – and this is the amount you should use in your ROI calculations.

Determining Expense Dollars

Your investment might be a cost reduction. If so, every dollar you save flows through to the bottom line as a dollar of profit. These are known as net margin dollars.

When an investment will both increase sales and reduce costs, you can’t add the numbers together – you can only add the profit dollars together. The same holds true if your costs are going up (say your investment is to hire an additional salesman). You have to calculate your return on investment based on the increase in gross margin dollars relative to the increase in expenses.

4. Overcome Ozymandias Syndrome

[Photo taken by Hajor, December 2001. Released under cc-by-sa and/or GFDL]

Nothing lasts forever. You may have a proposal that will reduce costs by $100,000 per year. What is the total savings? $100,000, $500,000, infinity? For how many years are you predicting that the savings will be realized?

I always think of this quote from the poem Ozymandias:

[...]

Look on my works. Ye Mighty, and despair!’
Nothing beside remains. Round the decay
Of that colossal wreck, boundless and bare
The lone and level sands stretch far away.

Percy Bysshe Shelley

Nothing lasts forever. There are three time limits that you can apply to constrain your ROI calculations.

• Limit by Uncertainty
• Limit by Financial Impact
• Limit by Compelling Event

Limit by Uncertainty

You can’t predict the future. Even with good estimates for potential savings, growth and costs, you can’t predict outside factors that could make your investment irrelevant. Netscape couldn’t predict that Microsoft would offer Internet Explorer for free. Our younger readers should know that browsers used to cost money. Microsoft’s action all but obliterated the business model for Netscape. The company certainly imploded.

In software, three years is about as long as you can conservatively plan on relevance. The IRS guidelines allow for depreciation of a software purchase for either three or five years. This makes for a good sanity check – in theory, your capital purchases should be completely depreciated at the time that they have no value.

Limit By Financial Impact

When you calculate net present value, you applied a discount rate to determine the value today of a cash flow stream from the future. When calculating NPV, you see that a dollar next year is worth less than a dollar this year. Similarly, a dollar two years from now is worth less than a dollar next year. On some future date, that future dollar will be immaterial in today dollars.

There isn’t a firm definition of “material financial impact”, but 10% feels like a reasonable number to us. In other words, the first year in which a future dollar is worth $0.10 or less is the first year to ignore. If your discount rate is 10%, that would be year 25. If your discount rate is 20%, you would look no more than 12 years into the future.

In software development, the relevance factor will always constrain us first. In industries with longer time horizons (like transcontinental shipping, or toll-road building), the financial limit is more likely to be the more constraining item.

Limit By Compelling Event

A compelling event would be a change in strategy, the elimination of the need for the product you’re evaluating, or another future improvement that will be implemented. In short, anything that drives obsolescence.

5. Ignore Infinite Elvises

Here’s a quote from 1994:

When Elvis Presley died in 1977 there were 37 Elvis impersonators in the world. Today [Ed: 1994] there are 48,000. If the current trend continues, by the year 2010, one out of every three people in the world will be an Elvis impersonator.

We’re running out of time.

There is a good point in here – there is huge danger in extrapolation of trends. Even when you think you understand the underlying causes of growth, it is unreasonable to expect the trend to continue. Venture capitalists regularly rant and/or joke about people who project infinite Elvis growth, almost as much as they complain about people who can’t support why they will get “just 1% of a zillion dollar market.”

Summary

You can’t over-estimate the importance of return on investment calculations when making decisions. Many of our articles touch on ROI, even when we’re writing about seemingly unrelated topics. In addition to those articles, remember these 5 tips for improving your ability to calculate return on investment:

1. Recognize the Risks
2. Discount Future Cash Flows
3. Separate Sales From Expenses
4. Overcome Ozymandias Syndrome
5. Ignore Infinite Elvises

The first step in writing the use cases for a project is to define the scope of the project. One way to do that is to list the use case names that define all of the user goals that are in scope. To do that, you need to know how to write good use case names. Good use case names also serve as a great reference and provide context and understanding throughout the life of the project.

Goals of Use Case Naming

Use case names are also known as use case titles. When creating names, we have a set of goals:

• Clearly indicate the user goal represented by the use case.
• Avoid specifying the design of the system.
• Make people want to read the use case, not dread reading it.
• Allow for evolution of use cases across releases.
• Define the scope of the project.
• Write consistently

Common Use Case Mistakes

We identified the top ten use case mistakes in a couple of articles about a year ago. They still hold true today:

From Top Five Use Case Blunders:

• Inconsistency
• Incorrectness
• Wrong Priorities
• Implementation Cues
• Broken Traceability

From Top Ten Use Case Mistakes:

• Unanticipated Error Conditions
• Overlooking System Responses
• Undefined Actors
• Impractical Use Cases
• Out of Scope Use Cases

Writing good use case names will help avoid errors in consistency, implementation cues, scope management, and traceability. They will also help us make people want to read the use cases. Think of the use case name as the headline of a magazine article – does it make you want to read it, or avoid it?

Good use case names also serve as reminders of what a particular use case does. Weeks after we’ve written a use case, a quick scan of the title will remind us of what the use case represents. On a large project with dozens of use cases, this is invaluable.

Tips For Writing Good Use Case Names

Here are the best practices we’ve adopted, and some we’ve collected from around the internet.

• Good Use Case Names Reflect User Goals. A good use case name reflects the goal of the user (or external system). A name like “Process Invoices” doesn’t tell us what’s being done – is it collections, organization, auditing, or some other function? A more insightful name would be “Collect Late Payments From Customers.” The goal in this example is to collect payments from delinquent customers. The second name does a much better job of defining what the user is trying to do when they perform the use case.
• Good Use Case Names are As Short As Possible. Some people suggest 5 words, or even two words. There are just too many examples that make setting specific word-count limits impractical. In the previous example, “Collect Late Payments From Customers,” which words would you remove without losing meaning? This name is as short as we can make it without losing clarity. This short name is better than “Collect Late Payments From Customers Who Are Past-Due.”
• Good Use Case Names Use Meaningful Verbs. Usually people will suggest that we should prefer strong verbs to weak verbs. That is effective advice for general writing. For writing use cases, we can be more specific. A meaningless verb is one that, while indicating action, does not specify the action with enough detail. “Process the Order” can be improved with a more meaningful verb. “Validate the Ordered Items” makes it much more clear what the user is trying to achieve.
• Good Use Case Names Use An Active Voice. A call to action is a hallmark of good writing. Using an active voice will inspire action more than a passive voice. “Calculate Profitability” is more inspiring than “Profitability is Calculated.”
• Good Use Case Names Use The Present Tense. “Create New Account” is in the present tense. “New Account Was Created” is in the past tense. The present tense implies what the user is trying to do, not something that has already been done.
• Good Use Case Names Don’t Identify The Actor. Some people prefer to name the actor in the use case, because it is more specific. We like the idea of using evolutionary use cases to manage the delivery of functionality across releases. When we do this, we are often releasing the first version of the use case for one actor, and the next version for another actor. For example, “Rank Employee Performance” might be our use case. In the first release, we want to enable the functionality for supervisors – who can rank their direct employees. In the second release, we want to add the ability for managers to rank the employees that report to multiple supervisors. We prefer having two versions of the same use case over having two use cases (Rank Direct/Indirect Employee Performance).
• Good Use Case Names Are Consistent. We should always apply the same set of rules across all of our use case names. Inconsistent application of the names will create a sense of discord for our readers. Consistent names will make it more comfortable for readers, and provide a sense of cohesion for the overall project.

Other References

• Use Case Analysis
• Java Blackbelt on Use Case Diagrams
• The Coad Letter, Use Case Do’s and Don’ts

Summary

Good use case names take very little effort once we are used to writing them with a consistent style, following the tips listed above. Good names also provide benefits down the road when reviewing and reading the use cases. Good names are short, clear, and stylish. They also make us want to read the use cases, and easily jog our memories about the user’s goals.

Marketing is as foreign to most software developers as flying is to fish. We’ve found a list of ten truths of marketing, and we’re secretly sharing them with the developers who hang out here. Shhh. Don’t tell anyone in marketing.

Marketing 101

John Dodds wrote Marketing 101 For Geeks, where he shares 10 observations about marketing that might make sense to geeks and coders.

Here’s John’s list with our comments:

1. Marketing is not a department. A great way to segue into the conversation – as an engineer, the first visual I always have of a marketing department is the one from Dilbert (Scott Adams draws marketing people as if they are at a perpetual cocktail party).
2. Marketing is a conversation.* This is hard for developers. Conversation requires two-way communication. That’s a truth. But good marketing pre-empts questions and answers them. Imagine the reader having a conversation with your copy (marketing materials): “I wonder what this is?” “oh.” “I wonder how we could use that?” “oh. cool.” “where can I get it?”
3. Simplicity does not negate complexity. A clear, easy to understand message is what coders might call “incomplete,” “over-simplifying,” or “simplicistic.” The secret that marketers keep to themselves is that this clear message is what opens the door – making it possible for customers to (eventually) understand and appreciate the power of a product that might be described with greater complexity.
4. Think what? not how?. As cool as it might be that your search engine uses a trie data structure, what potential customers care about is the fact that you can search a billion documents in a tenth of a second. This secret seems to be the reverse of a simple definition of geek – “someone who cares about how it works more than what it does.”
5. Think will not can. Featuritis is the condition of having too many features. Even the swiss army knife eventually became too large to slip in your pocket. We have to focus on what users need to do, and not everything that could possibly be done.
6. Only you RTFM. Think about the obvious ways to use a product. Intuititive user interfaces have affordances. They don’t require people to read the manual. And the manual should be written to help people accomplish their goals- not as a description of the functionality.
7. Technical support is marketing. Every touch-point with a customer is a marketing opportunity. Remember, we market not just by purchasing ads and putting up booths at conventions. We market by word of mouth.
8. You’re not marketing to people who hate marketing. Remember the disdain you had when you started reading this list? Well, we’re not marketing to people who hate marketers. People want to know how to solve their own problems. They want to know how they can use our products to help. And they like the people who tell them.
9. You’re not marketing to people who hate technology products. The people who get our message are the ones who are technology-agnostic (see #4 above). They neither love nor hate the product. But they love solutions.
10. Marketing Demystifies. Remember the conversation from #2? As the conversation progresses, we enlighten our customers, and eventualy they develop an understanding of what they can do with our product. And from this, they develop a desire to buy our product.

*John’s original point #2 was really an anti-jargon point. We thought the conversational part of his point should be stressed instead.

Conclusion

Don’t let them know, but we’re on our way to understanding how this stuff works.

The BABoK (Business Analyst Body of Knowledge) lists 10 techniques for gathering requirements. Here’s an overview of each one. For more details, check out the latest Guide to the BABoK.

1. Brainstorming
2. Document Analysis
3. Focus Group
4. Interface Analysis
5. Interview
6. Observation
7. Prototyping
8. Requirements Workshop
9. Reverse Engineering
10. Survey

1. Brainstorming

Brainstorming is used in requirements elicitation to get as many ideas as possible from a group of people. Generally used to identify possible solutions to problems, and clarify details of opportunities. Brainstorming casts a wide net, identifying many different possibilities. Prioritization of those possibilities is important to finding the needles in the haystack.

2. Document Analysis

Reviewing the documentation of an existing system can help when creating AS-IS process documents, as well as driving gap analysis for scoping of migration projects. In an ideal world, we would even be reviewing the requirements that drove creation of the existing system – a starting point for documenting current requirements. Nuggets of information are often buried in existing documents that help us ask questions as part of validating requirement completeness.

3. Focus Group

A focus group is a gathering of people who are representative of the users or customers of a product to get feedback. The feedback can be gathered about needs / opportunities / problems to identify requirements, or can be gathered to validate and refine already elicited requirements. This form of market research is distinct from brainstorming in that it is a managed process with specific participants. There is danger in “following the crowd”, and some people believe focus groups are at best ineffective. One risk is that we end up with the lowest common denominator features.

4. Interface Analysis

Interfaces for a software product can be human or machine. Integration with external systems and devices is just another interface. User centric design approaches are very effective at making sure that we create usable software. Interface analysis – reviewing the touch points with other external systems – is important to make sure we don’t overlook requirements that aren’t immediately visible to users.

5. Interview

Interviews of stakeholders and users are critical to creating the great software. Without understanding the goals and expectations of the users and stakeholders, we are very unlikely to satisfy them. We also have to recognize the perspective of each interviewee, so that we can properly weigh and address their inputs. Like a great reporter, listening is the skill that helps a great analyst to get more value from an interview than an average analyst.

6. Observation

The study of users in their natural habitats is what observation is about. By observing users, an analyst can identify a process flow, awkward steps, pain points and opportunities for improvement. Observation can be passive or active (asking questions while observing). Passive observation is better for getting feedback on a prototype (to refine requirements), where active observation is more effective at getting an understanding of an existing business process. Either approach can be used to uncover implicit requirements that otherwise might go overlooked.

7. Prototyping

Prototypes can be very effective at gathering feedback. Low fidelity prototypes can be used as an active listening tool. Often, when people can not articulate a particular need in the abstract, they can quickly assess if a design approach would address the need. Prototypes are most efficiently done with quick sketches of interfaces and storyboards. Prototypes are even being used as the “official requirements” in some situations.

8. Requirements Workshop

More commonly known as a joint application design (JAD) session, workshops can be very effective for gathering requirements. More structured than a brainstorming session, involved parties collaborate to document requirements. One way to capture the collaboration is with creation of domain-model artifacts (like static diagrams, activity diagrams). A workshop will be more effective with two analysts than with one, where a facilitator and a scribe work together.

9. Reverse Engineering

Is this a starting point or a last resort? When a migration project does not have access to sufficient documentation of the existing system, reverse engineering will identify what the system does. It will not identify what the system should do, and will not identify when the system does the wrong thing.

10. Survey

When collecting information from many people – too many to interview with budget and time constraints – a survey or questionnaire can be used. The survey can force users to select from choices, rate something (“Agree Strongly, Agree…”), or have open ended questions allowing free-form responses. Survey design is hard – questions can bias the respondents. Don’t assume that you can create a survey on your own, and get meaningful insight from the results. I would expect that a well designed survey would provide qualitative guidance for characterizing the market. It should not be used for prioritization of features or requirements.

Seilevel has a post that presents the top 10 signs that you should not pursue a career writing requirements, check it out.  Thanks Joy for the great article!
Our favorites:

#10 You cannot quickly understand new concepts

#9 You don’t have the patience to deal with customers

#4 You cannot form a mental model of all the pieces

Our take

Writing requirements is much more than taking dictation.  To develop great software, you have to develop an understanding of the needs of the customer.  From those needs, you have to synthesize a solution approach.  And you have to communicate that approach, both with customers (to validate it) and with the engineering team.  All of Joy’s entries (except the bonus #0 item) support this general framework.

We agree with Roger’s comments on the post that agile processes are not mutually exclusive to writing requirements.  Charles posted recently about the new product manager – implying that an agile product manager is different than a non-agile product manager.

There are many different agile processes, which use differing amounts of up-front planning, and differing formats for documentation.  Feature-driven development (FDD) does high level planning to understand the general approach of the product.  Details are then defined incrementally.  Incremental development works best when the most important stuff is worked on first.  This doesn’t preclude the need to communicate with customers and developers.  The fact that this communication happens incrementally doesn’t make documentation irrelevant.
The conversation about item #0 continues on Seilevel’s discussion forum, join in there, or add your thoughts here.

If This Be Treason, Make the Most Of IT! (Patrick Henry)

The customer is always right, except when he is wrong. When we have bad customers, we should fire them. Declare today as Customer Independence Day, where we declare our independence from bad customers.

The Loophole

While the customer is always right, there’s a big loophole – when the customer is wrong, he should stop being our customer.

Hawkins on Firing Customers

Christopher Hawkins provides a list of 11 customer archetypes who should be fired. Having them as customers is just bad business. Christopher provides a lot more detail, and here is his list of abusive client types:

1. The disillusioned
2. The suspicious
3. The chiseler
4. The bully
5. The something-for-nothing
6. The slow-pay
7. The flake
8. The liar
9. The blackmailer
10. The money pit
11. The clinger

Our Take

We agree that people who exhibit these traits tend to make bad customers, friends, bosses, etc. We should always strive to resolve conflicts or change the behavior of our clients when it is unacceptable. Only when all else fails should we abandon a customer because of a personality problem.

There are times when we have great relationships with our customers, and we simply become unaligned. We should fire those customers too. Perhaps the client’s strategy is changing, and it isn’t one we want to support. Perhaps the customer’s needs are such that they ask us to perform work that we choose not to do. Perhaps our direction has changed, or is evolving, away from an existing customer’s business needs.
We will dilute our efforts if we try and be all things to all people.

As a small company starting out, it can be very hard to walk away from a bad “opportunity” with no paying alternative in sight. Someone once said “the hard thing to do is often the right thing to do.” It could be for you. It has been for us in the past.

Be Professional

We’ve decided to fire customer X. We don’t want to dump a shipment of tea in the harbor. Customers have memories, and they also have friends. Even if we never want to work for customer X again, we don’t want a bad reputation.

Some things to do when separating from a customer:

• Be courteous and polite.
• Provide ample lead-time. Two weeks is a minimum, even if the customer wouldn’t have provided it to you. Some situations require more lead time.
• Review your documentation and update it as needed. Presumably, someone else will take the post we decline (or abandon). Make sure they have all the information they need to know everything you did.
• Be proactive about knowledge transfer. Don’t wait for the client to ask for knowledge transfer – actively contribute to the plan, and drive it forward.
• Reccommend alternatives. If you can propose your own replacement (and vouch for her) – even better.

- -
*(based on The Spirit of ‘76 by Archibald Willard 1836-1918)

Michael on Product Management and Marketing has posted the first in his series of product management commandments – Know Thy Customer. He provides five tips on how to know your customer better. We extend his idea to include understanding our customers’ markets, and provide more tips. By analogy, this is the difference between a detective who studies a criminal and a profiler who seeks to understand a class of criminals.

Why Must We Know Our Customers?

As Michael points out, the key trait of an effective product manager is one who knows her customers. As product managers, we have to understand our customers’ problems and convert them into requirements. From those requirements we define solutions, represented as products. Without a good understanding of our customer, we can’t identify the most important problems, and therefore can’t create the most valuable solutions.

Michael suggests five ways to know our customers better:

1. Listen to customers.
2. Usability tests.
3. Follow-me-home.
4. Walk-a-mile.
5. Customer surveys and focus groups.

Michael’s article

Know Thy Customer’s Market

One of many points that Pragmatic Marketing makes in their training is that product managers need to focus on multiple customers, not individual customers. They replace the “become a customer expert” goal with “become a market expert.” This helps drive their point home. We think we need to know our customer’s market, and not just our customer.

Understanding the market our customer requires us to know not only our customer, but our customer’s competition. Knowing our customer helps us understand the important and valuable problems. Knowing her competition’s problems helps us know which problems have value for multiple customers. One-customer problems require custom solutions. We want to maximize the time we spend on creating reusable valuable product features.

Michael definitely uses the plural when talking about understanding customers. We are taking this one step further, to include understanding of potential customers.

Things to Know About Our Customer’s Markets

1. The competitive landscape. Are there few or many companies in the space? Is it an oligopoly, monopoly, or free market?
2. How does our customer compete? What are the factors that our customer believes make them (or will make them) competitive? Do they compete on price or features? Do they rely on existing customers or new customers for their growth?
3. Is the market growing, shrinking or stagnating? Is our customer fighting for a larger slice of a shrinking pie?
4. What might disrupt this market? Our customer’s market is at risk of being obviated or obliterated at any time. Our customer may or may not accept this reality. At a minimum, we will understand our customers better by understanding their risks.

In addition to learning about potential customers, we also get a deeper understanding of our current customers by studying their markets.

More Sources of Customer Information

In addition to Michael’s five tips, we would suggest the following for each major player in the market:

1. Read the annual report. In addition to market data, we will find out what the CEO believes to be the most important objectives of our customer. This will help with prioritization of requirements.
2. Understand the users. Creating personas to characterize the users of our software at our customer helps us make better design decisions.
3. Diagram the political power in the organization. Politicians hold the trump cards for all software projects. By understanding who the real decision makers and influencers are, we can markedly improve our prospects for successful software sales. [Note: This applies to enterprise software more so than shrink-wrapped software.]

Conclusion

It is neccessary, but not sufficient to understand a customer. It is far better to understand the market of customers.

Why do we write use cases?

We write use cases for the same reasons that people use our software – to achieve goals. In our case, we want to assure that we are creating the right software. By looking at this high level goal in more detail, we can make decisions that drive the best possible use case creation. Let’s apply our product management skills to writing better use cases by writing an MRD for use cases.

This article can be used as a guide to develop a process for defining, documenting and gathering use cases. It can be used to define a template for use cases, and it can be used to define specifications for a use case management system. We will start with a market analysis and a vision statement, and then create our market requirements.

Market Analysis

• 69% of software projects failed or were delayed in 2003 according to the Standish Group’s latest study.(1994 study, 2001 study)
• 40% to 60% of problems were attributed to bad requirements.
• Current investments are focused on coding efficiency not requirement quality.
• Incremental delivery is becoming more common as teams adopt different forms of agile processes. This increases the frequency of requirements changes during projects.

Bad requirements are further detailed as the following:

• Requirements that are overlooked cause us to fail to meet expectations and fail to deliver value.
• Requirements that are incorrect cause us to incorrectly address problems and fail to deliver value.
• Requirements that are poorly communicated cause us to implement incorrectly, failing to address problems and deliver value.
• Requirements that are low-value cause us to spend time and money on problems that don’t maximize value.

Through the course of any long term project, requirements will change. This happens more when we use iteration and prototyping to accelerate stakeholder feedback cycles. But that’s a good thing, because the changes result in better requirements. Agile development methodologies like feature driven development supercharge this phenomenon.

Vision Statement

We will improve our ability to write and manage use cases so that we may maximize their impact on

• Writing and maintaining great requirements
• to improve our ability to deliver the right functionality
• and ultimately achieve software product success

Market Requirements

With an understanding of the market problems and a guiding vision, we will document the market requirements for writing better use cases. The market requirements have an explicit scope – they specify which and how much of the market problems we intend to address.

When we use the phrase ‘our use cases‘, we are really saying ‘our use cases and our approach to managing the use cases.’ We’re using shorthand to improve the readability.

Prioritization of the requirements is denoted with (H) (M) or (L) prepending the requirement, representing high, medium and low priority requirements, respectively.

Requirements that are overlooked

• 1. (M) Our use cases must support validation of requirement completeness. [Update: Detailed article on this goal.]

Requirements that are incorrect

• 2. (L) Our use cases must support validation of requirement correctness. [Update: Detailed article on this goal.]

Requirements that are poorly communicated

• 3. (H) Our use cases must improve communication with stakeholders about the intended content of the system. [Update: Detailed article on this goal.]
• 4. (H) Our use cases must improve communication with the implementation team (dev + test) about the intended content of the system. [Update: Detailed article on this goal.]
• 5. (M) Our use cases must improve communication about the release-planning (delivery schedule) for the system to both stakeholders and implementers. [Update: Detailed article on this goal.]
• 6. (H) Our use cases must improve communication of the scope of proposed requirements changes to developers. [Update: Detailed article on this goal.]
• 7. (L) Our use cases must improve communication of the impact of requirements change proposals to stakeholders. [Update: Same detailed article as #6 for this goal.]

Requirements that are low-value

• 8. (L) Our use cases must support prioritization of requirements across releases.

Conclusion

These goals define why we write use cases as part of software development. We do it to improve our ability to write the right software to solve our customer’s problems. We also write use cases to help us manage requirements changes and set delivery expectations with our stakeholders.

Pragmatic Marketing has a training seminar called Requirements That Work. In support of that, they provide a list of 8 characteristics of good requirements. We change one and add two more to round it out to The Big Ten Rules. Combine this with Michael’s ten tips for writing MRDs, and we’ve got a good handle on how to create a great MRD.

Pragmatic’s List (1-8) + Two Three Four More

1. Necessary Valuable
2. Concise
3. Design Free
4. Attainable
5. Complete
6. Consistent
7. Unambiguous
8. Verifiable
9. Atomic
10. Passionate
11. Correct
12. Stylish

Looking at each rule of writing good requirements…

1. Valuable

Updated for 2009

Writing valuable requirements is important.  It doesn’t matter how well your teams execute if they are off building the wrong products / capabilities / features.  The right products and capabilities are the ones that have relevant value.

• Valuable requirements solve problems in your market.
• Valuable requirements support your business strategy.
• Valuable requirements solve problems for your users.
• Valuable requirements meet your buyers’ criteria.
• Valuable requirements don’t over-solve the problems.

Valuable Requirements

Pragmatic uses necessary as a criteria of good requirements. We believe that valuable requirements are good requirements. When prioritizing requirements, we should do the must-have requirements first. But other valuable requirements are critically important – even if they aren’t mandatory. Prioritization and release scheduling should stress necessary requirements first, and then the most valuable requirements next.

Requirements that can differentiate our product from the competition are by definition not necessary – or the competition would have done it already.

[Update: Our detailed article on Writing Valuable Requirements ]

2. Concise

Updated for 2009

Concise requirements give your team a useful, easy to read and easy to change understanding of what must be done.  Great requirements exist to do three things:

1. Identify the problems that need to be solved.
2. Explain why those problems are worth solving.
3. Define when those problems are solved.

Concise Requirements

Easy to read and understand. If only it were that easy. For whom is it easy to read? A market requirements document (MRD) is written for several different people on the team. It provides a vision of what problems our product solves. It provides clarification to the implementation team. It also sets expectations with stakeholders. Different people on the team have different domains of expertise – we have to write requirements that are easily understood by all of them.

[Update: Our detailed article on Writing Concise Requirements]

3. Design Free

Updated for 2009

Design-Free requirements are important for two reasons, and hard for two other reasons.

Design-free requirements are hard because you “know what you want” when you should be documenting “why you want it.” Writing design-free requirements can be hard when you don’t trust your development team to “do the right thing” even though it is not your job to design the solution.

Design-Free Requirements

Generally, a requirement should not specifiy any of the implementation choices. From a product manager’s perspective the requirement is the ‘what’ and the spec is the ‘how’. To a system designer or architect or lead developer, the requirement serves as a ‘why’.

[Update: Our detailed article on Writing Design-Free Requirements]

4. Attainable

Updated for 2009

Unless you live in a world filled with unicorns and rainbows, writing realistic requirements is critical. When you set unattainable goals, the best result you can hope for is a frustrated engineering team. Write requirements that are attainable, and your team will surprise you with what they can achieve.

Attainable Requirements

The requirement must be realistically achievable. Barbara and Steve make great points about understanding the cost of implementing something as expressed in the requirements. As we pointed out in our thoughts about good requirements management, there is an optimal tradeoff between costs and benefits for any given company or project. We can formally approach this using the techniques identified for more is better requirements using a Kano framework. In short, the investment must have an ROI that exceeds the opportunity costs by the hurdle rate.

Looking at cost-benefit tradeoffs also supports the argument that valuable should replace necessary.

[Update: Our detailed article on Writing Attainable Requirements]

5. Complete

Simply put, if the requirement is implented as written, the market need is completely addressed. No additional requirements are required. When writing a specification, we may use decomposition to break individual requirements into more manageable, less abstract criteria.

[Update: Our detailed article on Writing Complete Requirements]

6. Consistent

Pragmatic highlights that the requirement must be logically consistent with the other requirements in the document – no overlaps, no contradictions, no duplications. This is certainly the most important point of consistency.

There is also benefit to consistent writing in an MRD. We can use templates to provide a consistent framework, but more importantly the prose needs to be consistent. This consistency makes it easier on the readers.

[Update: Our detailed aricle on Writing Consistent Requirements]

7. Unambiguous

A great requirement has a single interpretation. A good requirement has a single reasonable interpretation. As part of our development process, we will use listening skills like active listening to make sure that our engineering team understands the requirement we intended to write. The better the requirements, the less expensive and risky this communication process will be. Writing unambiguously is critically important when using outsourcing models that limit our interactions with other team members.

[Update: Our detailed article on Writing Unambiguous Requirements]

8. Verifiable

We use a process that starts with market requirements, and then decomposes them into software requirement specifications. the market requirements must be written in a way that we can verify that the associated requirements specification will meet the market need.

[Update: Our detailed article on Writing Verifiable Requirements]

9. Atomic

Every requirement should be a single requirement. If we can say “Half of this requirement is implemented” then this needs to be two or more requirements. If a requirement read “Sales reps can manage their client list and generate custom reports” it expresses two atomic ideas (list management and report generation). Those ideas need to be separated

[Update: Our detailed article on Writing Atomic Requirements]

10. Passionate

Nothing great has been born from complacency, lethargy or mediocrity. When we are defining requirements, we must be passionate about what we’re doing. If we’re just going through the motions, it shows up in the writing. If we aren’t excited about a requirement, the problem is either with us or with the requirement. Either way, it won’t inspire the rest of the team to do something great.

[Update: Our detailed article on Writing Passionate Requirements]

11. Correct

[Update: Added 30 Oct 2006]

In addition to all of the analyses above, a requirement must actually further the objective it supports, and must be neccessary to meeting that objective, given a particular approach to solving the problem.

Our detailed article on Writing Correct Requirements.
12. Stylish

[Update: Added 5 Jan 2007]

Style can also be the difference between a well-crafted requirement and one that is hard to read. And style, applied consistently across requirements makes it easier to view them as a whole, and identify gaps and inconsistencies. This ease of comprehension matters when trying to achieve correct, consistent, complete requirements.

Our detailed article on Writing Stylish Requirements.

Summary

There isn’t really anything to summarize – this article is one big summary.

I would like to take a moment to thank everyone who has been subscribing, reading, commenting, and sharing Tyner Blain. We started this site six months ago and I am continually surprised, flattered, and thankful for all of the readers and support we have.

Thank you very much!

Scott

Michael wrote five (and another five) tips on writing a market requirements document (MRD). Michael has written a good set of tips with detailed explanations and anecdotes. We’ll extend the conversation here…

Michael’s list:

1. Write From User’s Perspective
2. Use Screen Shots
3. Write Using Simple Language
4. Use Templates – But With Care
5. Prioritize Requirements
6. Specify What & Why – But Not How
7. Cover Non-Functional Requirements
8. Review & Update
9. Define Target Market & Positioning
10. Include a Glossary

We have re-organized these tips into three general areas of guidance and provide our thoughts.

1. Good communication (tips: 1,2,3,4,10)
2. Good requirements (tips: 5,6,7,9)
3. Good requirements management (tips: 8)

Good communication

The purpose of the document is to communicate. Writing for our audience is essential to communication. Using templates to create a predictable structure and flow makes the content easier to absorb. Using clear, unambiguous language (no contrived terms or jargon), makes the ideas easier to grasp and remember. Providing a glossary of terms to provide domain context for the readers is essential for many projects.

A software developer who is an expert at load-balancing a web server may not have any idea how to calculate incremental margin for sales beyond forecast. “Incremental margin” may seem like jargon, but it is a standard term (within the management accounting domain). Use that as your rule of thumb – is the term company-specific, or would anyone in the domain understand it. If the former, its jargon. The latter should have a glossary entry.

A picture is worth a thousand words, certainly. It isn’t clear that using screenshots or mockups is the best way to capture requirements in an MRD. Because they are so powerful, it is impossible to get the intent, without being influenced by the form. This is an area where a lot of smart people agree to disagree. We’ve written about the dangers of using screenshots or other implementation cues that can be interpreted as ‘how’. As Michael points out, often ‘how’ is the appropriate way to communicate with other members of the team – it depends on the team. Barbara Nelson, a product management instructor for Pragmatic Marketing, stresses that the product management role is strategic, and getting into details isn’t. It may be required, but if the product manager is doing it, who’s doing the strategic work?

On the flip side, Alan Cooper is promoting Interaction Design. In an interaction design process someone is responsible at a level analogous to an MRD for determining both what and how. This is basically an interpretation that the how is an element of the what, not just an implementation detail. Combining interaction design with classical structured requirements might look like this amalgam.

Good requirements

Prioritization is the most important element in the document (other than the ideas being prioritized). One goal of an MRD is to communicate the vision of the software. Understanding what and why, with the context of which is more important, is the mechanism of communication to the engineering team. This message provides the framework in which they operate.

MRDs are also the right focal point for driving much of the outbound communication. The vision in an MRD identifies which white papers should be created. The (market) positioning provides guidance to sales people in individual account positioning. (Sales) demos should highlight the most valuable capabilities of the product. Roadmaps and the multi-release schedule are driven by prioritization. Schedules may be managed in timeboxes and delivered in use-case-sized chunks, but the driving prioritization is in the MRD.

Good requirements management

MRDs are not carved in stone and aren’t found in a cave high in the desert. They are not an end, they are a means. A means to communicate a vision. One thing that the Agile proponents clearly have right is that change happens. The market is a moving target. A vision for dominating that market better move with it. And the resulting MRD is going to change.

Product Managers are also not infallible. Left to our own devices, we will write some horrible requirements. We require feedback from the implementation teams in order to write great requirements. A great example is documenting what Kano would define as a “more is better” requirement.

The optimal point is not the revenue-maximizing feature, it is the profit-maximizing feature. We’re driven by ROI. We need an understanding of the cost function to combine with the price function to result in a profit function. We must solicit and respond to that feedback from our engineers.

This is the level of execution expertise that can differentiate our teams, and our products.

Summary

An MRD is critical to capturing product strategy information. Capturing the right information is critical. The goals of capturing that information are to disseminate the ideas, and for the team to collaborate on them. Techniques that make it easier for our target audience to read the MRD are important. And having a good approach to managing the document as it evolves is what can set us apart from other teams, or make us more competitive.

Martin Fowler has identified the key process elements of making Continuous Integration work. You could even argue that they are the elements that define Continuous Integration (done correctly). We include his list and our thoughts below:

1. Maintain a Single Source Repository
2. Automate the Build
3. Make Your Build Self-Testing
4. Everyone Commits Every Day
5. Every Commit Should Build the Mainline on an Integration Machine
6. Keep the Build Fast
7. Test in a Clone of the Production Environment
8. Make it Easy for Anyone to Get the Latest Executable
9. Everyone can see what’s happening
10. Automate Deployment

For background information, check out the Foundation Series on Continuous Integration article.

1. Maintain a Single Source Repository

The smartest people I know use subversion when they have been able to make the choice themselves. Aside from being open source, it provides two key differentiated benefits relative to “everything else”.

• Atomic commits: The ability to check in everything or nothing, so you don’t risk breaking the build with a partial check-in.
• Overall project versioning: Allows you to track changes in source file directory hierarchies, file renaming, etc. Each version is of the entire project, not of a single file.

2. Automate the Build

Fowler sums it up perfectly:

“…anyone should be able to bring in a virgin machine, check the sources out of the repository, issue a single command, and have a running system on their machine”

3. Make Your Build Self-Testing

Include automated testing as part of the build.

4. Everyone Commits Every Day

More frequently when possible. This is the minimum.

5. Every Commit Should Build the Mainline on an Integration Machine

A seperate, dedicated machine does a daily (or more frequent) build and full test suite run autonomously. The “build and test” model above relies on people to kick off the build when they commit. A scheduled task on a seperate machine provides a safety net for human error (oversight). Alternately, companies like Calavista can make this foolproof by automatically triggering an automated build as part of every commit. With Calavista’s devEdge, if the developer “commits”, what really happens is that the automated build/test cycle is triggered with his new code, and it gets promoted only if all the tests pass.

6. Keep the Build Fast

Tests can take a long time, builds should only take ten minutes. Fowler suggests a strategy of staged builds to address the 10-minute threshold. Run unit tests against the 10-minute build, and run the full suite in parallel or series.

Another option is to use statistical sampling of tests to get a “10 minute answer” while the full suite is kicked off in parallel.

7. Test in a Clone of the Production Environment

Eliminate even more variables. Make sure the tests are running against a clone of the production environment. Teams that are pushing the envelope today use virtual machines (VMs) to quickly create cloned production environments, install the software and run the tests.

8. Make it Easy for Anyone to Get the Latest Executable

Make sure everyone knows where the latest build can be found. Probably a good idea to keep recent builds in the same place too, in case a problem sneaks through the process temporarily (like a memory leak or other obscure, not-yet-tested situation).

9. Everyone can see what’s happening

Visibility! eMail the team when builds start/finish, including success/failure information. Put a rubber chicken on the desk of the person currently running the build (don’t ask – just read Fowler’s post). Ring a desk bell when the build passes. Have fun with it.

10. Automate Deployment

Make deployment into production as easy as running the build. Since tests are run against a production clone, and are already automated, this presents minor incremental effort.

Conclusion

Martin presents a great list. In addition to the above, we would suggest

• Generate test-results documents per requirement. For each build, identify which requirements pass, fail, or are untested. The most relevant information for communicating outside of the team is status of previous requirements (did our regression tests pass?) and current requirements (are we almost done with this timebox?).

Most guides to writing an executive summary miss the key point: The job of the executive summary is to sell, not to describe.

This from Guy Kawasaki’s recent post, The Art of the Executive Summary. Guy’s article is structured towards pitching an idea to a potential investor. We’re going to apply the same rationale to the communication that is key to successful product development – communication from the team, to stakeholders and sponsors.

Two types of communication – tactical and strategic

For this post, we’ll assume that we are part of a team delivering a new software product for our company. We have users, marketing, product management, development, quality, and SMEs (subject matter experts). We are all working together to deliver great software. We communicate with each other as part of executing on our tasks. This is tactical execution.

James Shore posted Two Kinds of Documentation last month, where he presents an Agile perspective on documentation. The two types of documentation he identifies are ‘get work done’ documentation and ‘enable future work’ documentation. The purpose of these documents (or more precisely, the communication they represent) is to help our team execute either now or in the future. Internal communication is tactical communication.

We also communicate with people outside of our team. We communicate to set expectations with customers, users, and clients.We communicate with sponsors, customers, and others who fund our software development. Without these channels of strategic communication, we won’t have a project, or worse, won’t have a customer when we’re done. External communication is strategic communication.

Tailoring strategic communication

Guy’s quote is very insightful – there is only one reason for presenting to the executive – to get funding. Why? Because the executive has only one reason for listening to us – to decide if we should get funding. All of our strategic communication should focus on one goal. – The goal of our audience.

Here are three tips on providing targeted communication to people external to the team.

1. It’s the economy, stupid! When President Clinton ran for office the first time, this was one of his slogans. He capitalized on the fact that his opponent was busy talking about what his opponent thought was important. Then Governor Clinton talked about what his audience thought was important. This is the hardest thing to remember, especially when we’re passionate about what we’re creating. We need to run our communication through the “so what” filter. If it isn’t important to our audience, don’t make them listen to it (or read it).
2. No habla Ingles. Once we identify what our audience cares about, we have to make sure that we can communicate that information in a language that they understand. In one of our earliest posts, Intimate Domains, we talk about how people are so rooted in their areas of expertise that they almost speak different languages. If we can’t modulate our signal to a band-passed frequency*, we might as well be speaking jargon gibberish.
3. Brevity. Clear. Concise. Contextual.

[Update 25 Apr 06]

We have added a post, Targeted Communication – Status Reporting as a detailed example of targeted communication.
- -

*If we can’t get the message across in terms our listener understands,...



]]> http://tynerblain.com/blog/2006/04/03/targeted-communication-three-tips/feed/ 0 http://tynerblain.com/blog/2006/03/06/top-ten-tips-for-preventing-innovation/ http://tynerblain.com/blog/2006/03/06/top-ten-tips-for-preventing-innovation/#comments Tue, 07 Mar 2006 04:14:51 +0000 Scott Sehlhorst http://tynerblain.com/blog/2006/03/06/top-ten-tips-for-preventing-innovation/

At a recent presentation in Austin by Seilevel about the goals and methods of requirements gathering, a member of the audience asked “What can we do with our requirements to assure innovation?” That’s a tough question with an easy answer – nothing.

What if the question had been “What can we do to prevent innovation?” That’s a better question with a lot of answers.

Struggling with too much innovation?

Yes, people have been innovating since fire and the wheel it’s a curse we’ve inherited. In modern times, much of that innovation has happened inside companies. 3M had the post-it note, Lockheed had the skunkworks that created the SR71. Google allows their employees to dedicate 20% of their time to whatever interests them – and Google’s employees innovate a lot.

Most companies do a good job of providing incremental improvements to existing products and processes. What are those few who struggle with innovation doing wrong?

Companies with track records of innovation have flawed processes.

• They fail to screen out likely innovaters in their hiring process.
• They mismanage their employees, who end up innovating when they should be towing the line.
• They inadvertantly reward innovation instead of mediocrity with recognition and compensation.
• They create opportunities to innovate and their employees drive Mack trucks through these loopholes.

Here is some guidance about how to fix those problems:

Top ten tips for preventing innovation

1. Hire employees looking for safety in their roles. Innovation happens when people stretch outside their comfort zones – don’t let them stretch! Find people who primarily want security and a nine-to-five role, stay away from those troublemakers who want to “change the world.”
2. Hire incompetent employees. What better way to prevent innovation than to have people who have to focus just to do the bare minimum? For extra safety, try and find someone who can take credit for other people’s work and hide their own incompetence – these people are easier to promote, which will become important later. If we are forced to hire someone who is competent, it’s critical that we make sure that they only have one area of expertise. People with more than one area of expertise, switch-hitters, just cause trouble by talking to people on other teams.
3. Keep salaries below the 75th percentile. Innovators know their value – and when they aren’t applying for jobs with intrinsic utility to them, they are commanding higher salaries. If we keep our salaries low, there’s much less risk of one of these innovators sneaking into our organization. As a bonus, we’ll save a fortune!
4. Read The Ten Faces of Innovation by Tom Kelley of IDEO. He focuses on the types of people and organizational behavior that encourage innovation. The writing style is very clever – Mr. Kelley writes as if he were trying to encourage innovation – what a riot! He identifies ten personas that contribute to innovation. Put those ten faces on the wall in HR like an FBI most-wanted poster and coach HR to screen those people out.
5. Treat employees like garbage. Yell at them. Whenever possible, call them at midnight to yell at them some more. They work for us. If they get uppity, make them work on the weekends. Make them dig holes and fill them back up again. Threaten them – especially when they need the job. If you can’t yell, at least be condescending in public forums. Remember we are smarter than they are. Punks.
6. Reward conservative and marginal successes. The old rule of thumb for office politics was “It takes ten good projects to recover from one bad project.” Stick to it! If we punish people for mistakes when they ’swing for the fences’, and reward them for marginal and safe projects, they will quickly get the idea. This is the most subtle of all the tips – but don’t worry – people will figure out the reward system and shy away from those risky projects. This technique has the added benefit of propogating itself up and down the management hierarchy. Many organizations get lucky, and do this one accidentally. Wish we were all so lucky!
7. Micromanage. We’ve been promoted because we understand their jobs so well that we could do them in our sleep. Whatever those pesky little people think, it’s wrong. We know what we want, we know how we want it (not like that, you fool!). Every day we should make sure they do things exactly like we want. Even things like using the right font in their emails can be important. If anything slips thru unmanaged, then we aren’t doing our jobs. Of course, if we have a good boss, he’ll tell us exactly how to manage them.
8. Only create customer-requested features. Let our customers tell us what to do. Lucky for us – customers don’t have big ideas, they keep us focused on what we’re doing. Don’t let them whine about their other problems – that’s not why we’re talking to them. We just want to know if they like the idea of animated buttons on all the dialogs. Stay away from the unhappy customers – if we aren’t getting the job done now, well, we don’t really care what they say (they are existing customers, we need new customers). We’re here to solve our problems. Oh – and don’t second guess the customer. If they say they want the menu items in alphabetical order, well, that’s what they want. The customer is always right. If Henry Ford had listened, think of how fast horses would be today. Even better, appoint a user-representative, then we don’t have to talk to the customers at all.
9. Make performance reviews easy. Create some easy-to-measure metrics (like # of sick-days taken, # of powerpoint slides created, # of meetings attended), and use those for performance reviews. People always gravitate toward the metric. We can run the reviews with a minimum of effort, giving us more time to tell them how to do their jobs. Just an hour a year. Some managers can give feedback in 15 minutes.
10. Build a kingdom. When we have information, that means we have power. With that power, we can grow our organization. The more people we have, the more important we are. We need to make sure that those other teams don’t get our information. They might apply it in ways that we didn’t intend. While we’re at it – make sure our people don’t find out what we know. Not only will it protect us from them, but it will keep them from accidentally discovering a more important problem, or an alternate way to apply what they already know to a new problem domain.



]]> http://tynerblain.com/blog/2006/03/06/top-ten-tips-for-preventing-innovation/feed/ 26 http://tynerblain.com/blog/2006/02/08/software-testing-series-organizing-a-test-suite-with-tags-part-two/ http://tynerblain.com/blog/2006/02/08/software-testing-series-organizing-a-test-suite-with-tags-part-two/#comments Wed, 08 Feb 2006 13:46:41 +0000 Scott Sehlhorst http://tynerblain.com/blog/2006/02/08/software-testing-series-organizing-a-test-suite-with-tags-part-two/

Organizing a test suite with tags (part 2)

This is the second in a three-part post about using tags as a means to organize an automated test suite.

Part 2 of this post can be read as a standalone article. If it were, it would be titled Top five problems with test automation suites. If you’re only reading this post and not parts 1 and 3, pretend that this is the title.

• In part one of this post we developed an understanding of tagging as a technology, including it’s pros and cons.
• In part two of this post we will define the top five opportunities and problems inherent in the organization of automated test suites.
• In part three of this post we will explore an approach to combining tagging with test suite organization.

What are the problem areas inherent in managing automated tests?

We start with identification of the problems or opportunities, before defining what the requirements will be. This is the same process we discussed in From MRD to PRD, applied to the test-automation space. The following are the top five problem areas we can identify about test automation suites.

1. Maintaining the suite becomes too expensive. Once we have a suite in place, we have to maintain it. As the size of the suite grows, the amount of maintenance of existing test grows. It grows in proportion to the number of tests in the suite and the rate of change of the underlying software being tested. As the amount of maintenance grows, so does its expense.
2. Developers will never start using the suite. Change is bad. Well, for many people it is. Asking someone with a full time, salaried job to take on additional responsibilities has to be done correctly. There is absolutely a risk that people won’t start using the suite. Since this project is focusing on iterative development of an already deployed tool, already in use, this problem really doesn’t apply.
3. Developers will stop using the suite. Developers avoid tedium. They’re smart. They want to avoid unneccesary work, menial work, and irrelevant work. If the developers perceive the test suite in any of these ways, we’re doomed – they will stop using it.
4. Not testing the right stuff. A test suite that doesn’t test the right areas of the software is worse than not having one at all – because it gives you a false sense of confidence.
5. Test suite becomes less effective over time. An initially effective suite can grow less effective over time as the underlying software changes. Individual tests become irrelevant as they become impossible to reproduce with the application – perhaps the user interface has changed. If test design was linked to the heaviest usage patterns, and those patterns change, then coverage of the new heavy usage parts of the suite will be reduced – and the effectiveness of the suite will be reduced.

Which problems should we address with software?

With limited resources, we need to make sure that we focus our software efforts on those problems where software can have the most impact on solving the problem. We’ll start by identifying

1. Maintaining the suite becomes too expensive. There are three approaches to solving this problem – reduce the required maintenance, make the required maintenance more efficient, and reduce the cost of the labor that maintains the solution. Labor cost reductions may very well be the most effective general way to solve this problem, but given the real world project constraints for the project behind this post, we aren’t exploring that option. This is a candidate for the software solution.
2. Developers will never start using the suite. Make them want to use it, or make them use it. We believe you want to make them want to use it – both by evangelizing the benefits and by quickly crossing the suck threshold so that users get positive feedback. For this project, we have taken that approach, although it’s true that there is also a mandate from the dev team’s managers that we must make sure they use it. With process and education approaches that have proven effective, this is not a target of the current software solution.
3. Developers will stop using the suite. The looming mandate will assure that developers won’t go AWOL on the suite. But if they can present a compelling reason to their managers, there is a risk that they will decide to stop using it. This is a candidate for the software solution.
4. Not testing the right stuff. Test suite planning is a science unto itself. We will keep in mind “ways to make test suite planning easier” as a candidate for the software solution, but we aren’t otherwise targeting this for the current software solution.
5. Test suite becomes less effective over time. Tests can grow irrelevant over time when the software they test is constantly changing (as in this project). This problem has been addressed to a large extent by using whitebox unit tests in the test suite. We are not targeting this as part of the current software solution.

Reminder

• In part one of this post we developed an understanding of tagging as a technology, including it’s pros and cons.
• In part two of this post we defined the top five opportunities and problems inherent in the organization of automated test suites.
• In part three of this post we will explore an approach to combining tagging with test suite organization.

- – -

Check out the index of software testing series posts for more articles.



]]> http://tynerblain.com/blog/2006/02/08/software-testing-series-organizing-a-test-suite-with-tags-part-two/feed/ 0 http://tynerblain.com/blog/2006/01/30/five-measures-of-product-manager-performance/ http://tynerblain.com/blog/2006/01/30/five-measures-of-product-manager-performance/#comments Tue, 31 Jan 2006 05:10:10 +0000 Scott Sehlhorst http://tynerblain.com/blog/2006/01/30/five-measures-of-product-manager-performance/

Joy posted a really good article last week at Seilevel’s requirements defined blog, Measuring product manager performance on internal system products. Her post is a followup to an extensive and heated debate that happened last fall on the Austin PMM forum. It’s a great forum to subscribe to – a lot of experienced people with strong opinions and steamer trunks full of anecdotal data – and they don’t all agree. You get to see the coin from all three sides* with this group – it’s awesome.

Five Reasonable Metrics for Product Managers

She puts together a summary of five reasonable, measurable metrics for Product Managers

1. Track scope creep beyond a baseline of the requirements by counting new requirements added (and outside the original project scope).
2. Track errors of omission by counting new requirements added after the baseline (but within the original project scope).
3. Track errors of ambiguity/incorrectness by counting changes to requirements after they have been baselined.
4. Measure user adoption or satisfaction, presumably relative to an initial goal.
5. Measure the “missed ROI“ of the delivered project, relative to the initial plan.

There is good detail behind the sources of the measurements and methods of measurement. Joy also recognizes that they are imperfect metrics. I love the discussion that she has started, and hope it extends beyond this post as well. If you have a blog, post another followup to this one, and link back to both of our posts.

People perform to the metric

Measuring performance quantitatively is extremely hard. It can be very easy to make measurements, but it is very hard to know what to measure. The only thing worse than not having any clear objectives for an employee is having the wrong metrics. At the end of a review period (or during a project debrief), you want to be able to tell someone more than “exceeds/meets/fails-to-meet expectations.”

The problem then becomes defining a metric for measuring performance. Even worse than not having a metric is reaching the end of a review period and finding that someone excelled against the metrics, but failed against the objectives of the company. This is a classical organizational design problem, not just a “how do I measure product managers” problem. We will explore the larger problem in general, but use the PdM in specific for examples.

I contend that people’s performance is at best influenced by, and at worst strictly designed for meeting metrics. The philosophy and approach below is built upon that premise.

Looking at the bigger picture


On The West Wing, Leo is coached before a press briefing to respond with “I don’t accept the premise of your question.” when asked a lose-lose question. We should take a step back and explore the premise, instead of blindly critiquing the actionable ideas Joy presents.



Thinking outside the box

Since we talk about requirements definition a lot here, let’s approach the definition of a performance measurement system as a requirements elicitation exercise.

Why do you measure someone’s performance?

First identify the problem/opportunity of increasing the success of the company. It is reasonable to believe that people impact the success of your company through their performance.

• As a company, you have a goal. This could be philanthropic, profit motivated, or be based on any other company vision. Assume for the sake of this exercise that your company is profit motivated.  Your company also has a set of implicit and explicit values – Google includes “do no evil” in their list of values. Think of these as constraints on what your employees do and how they do it.
• People’s performance in their role either advances or hinders your company in reaching its goal. You therefore want to put systems in place that maximize people’s aligned performance with your company’s goals. You will also dedicate internal resources (a cost) to create and maintain this system.
• You want to maximize the long term, positive impact that a person’s performance has on your company goals.

What are the elements of your company’s goals that you want to impact?

What are those things that you want to achieve as a company, which you want to achieve through individual performance? For this exercise, let’s assume that you believe that profits are best achieved through long term successful product management relationships with your clients.

Based on that, and focusing on product managers, what are the traits or actions you want to encourage to support your goals?  Determining those aspects of your company objectives that are best addressed through individual performance is an ideation process.

• You want your projects to succeed. Successful projects will occasionally lead to repeat business, and failed projects will cause you to lose long term clients by weakening your relationships.  Successful projects  will differentiate you from your competitors.
• You want to improve project ROI for your customers.  By providing your customers with more cost-effective projects, you will gain additional business, and also make existing business more profitable.
• You want relationships with decision makers. A bad reputation will stop the best sales cycle on a dime. A good reputation can open doors, uncover opportunities, and serve as a tie-breaker in competitive bids.  Having a good reputation will help you close more deals, and close them more profitably.

What are the measurements you should use to assess performance?

You are crafting a system of measurement, designed to achieve the subset of goals that you believe are best accomplished through individual performance. As such, you should review some design considerations.

• You need to be aware of the danger of false precision. Here’s an example – in a 3 month period, one product manager wrote 100 requirements that did not change after the baseline. Another product manager wrote 50. Who is the better product manager? What about developers – is the developer who writes more lines of code the better developer? What about the one who closes the most issues? These metrics do not give you useful information without context. They are easy to measure, but hard to show the relevance of the measurement to the stated objectives. In fact, the best thing that can be said about them is that they are easy to measure.
• You need to avoid destructive (but instinctive) human behavior. Another example – you penalize product managers for changing requirements after the baseline – specifically based on %changed data. Your developers find the spec to be ambiguous, so they ask questions to get clarification. The product manager answers the questions instead of updating the spec.  Your product manager ends up wasting a lot of time explaining, but avoids getting dinged on an easily measurable metric.  By aligning behavior with the metric, this product manager becomes less efficient (by repeating the same explanation over and over), while simultaneously improving his “score.”
• You need to avoid performance metrics that can not be controlled by the employee. It can be demotivating to employees to know that their efforts to affect their performance reviews are futile. In the worst case, the employee will give up. In the best case, the employee will do what is right in spite of the metric – and you will fail to equitably reward the employee. There is an entire industry that defines compensation systems using quantified metrics. The stock market uses the same approach.
• You want to give feedback regularly.  Presenting an employee with feedback and suggestions for improvement will result both in more job satisfaction and in ever-increasing impact from the employee. This is not best done during a performance review (although it should happen then). Feedback should be given repeatedly during the performance period – to give the employee an opportunity to course-correct along the way. This would be agile performance-management. Waiting until the performance review to surprise someone with good or bad news is the waterfall performance management method.

OK, sounds impossible – what do I do?

A fomulaic appraisal system should not be used.

It is inefficient because the metrics can mislead you about who is doing a great job and who isn’t.  Metrics will stimulate sub-optimal behavior in employees, ultimately subverting your company objectives to some degree.

The right way to assess performance is to qualitatively assess the results of the work done by the employee.  Talk to your contacts at the client to assess the impact the employee has had on the relationship.  Have the employee submit periodic status reports with both anecdotes and progress updates. Review the success of the project, and identify the context – were the results in spite of or because of the efforts of the employee?  Ask the consumers of his artifacts (requirements documents) for their opinions on the quality of the documents – and develop an understanding of their capabilities to judge. Through the course of the review period, interview the employee and ask questions about the content of the status report. By understanding what’s working and what isn’t, what’s easy and what’s hard, and what’s on time and what’s late; you can apply your own expert opinion about an employee’s performance – and share it with the employee right then. This also then gives you the ability to assess the employee’s improvement relative to the feedback he has received.

And another thing

Another way to assess performance is to identify goals at the beginning of a review period, and track progress towards those goals. These high level abstractions are easy to align with company objectives, and can be easy to measure (did you sell an extra million dollars in services to your account this quarter?). This approach can be very effective when evaluating adaptive high-performing employees with the latitude to operate in a loose organizational structure to “do whatever it takes” to get the job done. This can be done in conjunction with the other qualitative assessments outlined above.

Align bonuses with goal-based measurement of final results (not interim steps), and manage salary via qualitative assessments. Note that this approach is only effective when reviewing empowered, typically professional, employees.

[Update: 2009.08.31: Edited the above article for style, and struck through the final piece of advice.  Watch Dan Pink's fantastic TED talk on the science of motivation.  Based on his research, you should not reward product managers for performance.  Honestly, I don't know what the right answer is - I'm conflicted, as Dan's ideas (and facts) resonate with me, but I also can't shake my belief in a reward system.]
*heads, tails, and the edge make up the three sides of a coin



]]> http://tynerblain.com/blog/2006/01/30/five-measures-of-product-manager-performance/feed/ 1 http://tynerblain.com/blog/2006/01/26/top-ten-use-case-mistakes/ http://tynerblain.com/blog/2006/01/26/top-ten-use-case-mistakes/#comments Thu, 26 Jan 2006 11:56:09 +0000 Scott Sehlhorst http://tynerblain.com/blog/2006/01/26/top-ten-use-case-mistakes/
The top ten use case mistakes

We’re reiterating the top five use case mistakes from Top five use case blunders and adding five more. For details on the first five, go back to that post.

There’s also a poll at the end of this post – vote for the worst mistake.

1. Inconsistency.
2. Incorrectness.
3. Wrong priorities.
4. Implementation cues.
5. Broken traceability.
6. Unanticipated error conditions. The error conditions are explicitly called out in a formal use case as exception courses. When we fail to think about how things can go wrong, we take a bad situation (an error) and make it worse by leaving our users with no reasonable way to deal with the errors.
7. Overlooking system responses. When people use computers, the computers respond. It is a cause and effect relationship – and ideally one that is predictable and comfortable to the user. Reading a use case should be like watching a tennis match, with activities being performed alternately by the user and the system. “The user does X, the system does Y, the user does Z…”
8. Undefined actors. Novice and expert users have different ways of using an application. Different design tradeoffs will be made to accomodate for these users. Understanding the domain of the user can also be important. Imagine a calculator application – the use case of “get a quick answer to a calculation while doing something else” will be very different for a loan application officer than it will be for a research scientist.
9. Impractical use cases. We have to remember to validate with our developers that they can implement the use cases, given the current project constraints. As a former co-worker is fond of saying – “It’s software – we can do anything” which is true. But considering the skills of the currently staffed team, the budget and timeline for the project, and the relative priority of the use case is prudent.
10. Out of scope use cases. If we don’t define the system boundaries, or scope of our effort, we risk wasting a lot of time and money documenting irrelevant processes. Starting with the specious argument – although our user has to drive to the office in order to perform her job, we don’t include her commute in the scope of our solution. An online fantasy sports league application would certainly include a use case for picking players for individual teams – it may or may not include researching player-statistics. Knowing where the boundary is will prevent us from defining and building undesired or unnecessary functionality.

More discussion on common use case mistakes

I liked this article by Susan Lily on use case pitfalls. Susan goes into more detail on out of scope use cases(#10 above), where she talks about defining the system boundary in UML use case diagrams as a means of helping to avoid out of scope use cases. She also encourages using a standard template for use cases (Inconsistency – #1) and proposes a minimum set of criteria for creating your own templates. She provides a good argument against CRUD use cases – in a nutshell, they do not represent primary user goals (but rather tertiary goals).

At one point she proposes a compromise of including low-fidelity screen mockups in use cases as a means to make them easier to understand and more efficient to communicate. I disagree with her here – this is at best a slippery slope, and more likely the use case equivalent of my requirements documentation mistake. Because images can be so powerful – even the simplest screen design elements will provide design guidance (Implementation cues - #4) to the developers – IMHO, it is unavoidable.

–

We’ve added a new feature to Tyner Blain – polls on individual posts! We’re going back and adding polls to the most popular posts, and including them in many of the new ones. Each poll can have up to 7 entries – if an item isn’t displayed, hover over the up or down arrows and the list will scroll. If the text for an entry appears truncated, hover over it with the mouse and the text will scroll. Vote early and vote often, and thanks for your vote!
–

Poll: The worst use case mistake is

If you selected ‘Other – not on the list’ please add a comment and tell us why!



]]> http://tynerblain.com/blog/2006/01/26/top-ten-use-case-mistakes/feed/ 5 http://tynerblain.com/blog/2006/01/23/top-five-presentation-tips/ http://tynerblain.com/blog/2006/01/23/top-five-presentation-tips/#comments Mon, 23 Jan 2006 19:26:02 +0000 Scott Sehlhorst http://tynerblain.com/blog/2006/01/23/top-five-presentation-tips/

From Start to End has a great post, Some tips on presentations. Very little we can add here – check it out.

Our top five presentation tips (our first four picks are from the list behind the link)

1. Know your audience. A key preparation – you have to have a goal for a presentation. Are you convincing, educating or inspiring people? What do those people care about (and what do they already know?)? Also – do you actually know the people in the audience?
2. Revise and rewrite. Editing is the best thing ever. When we first put ideas down, it’s generally from our point of view. Validate that the content is targeted at the audience.
3. Minimize the text on the slide. Eyecharts distract from the presenter. People read ahead – the slide content should provide cues for you to speak, and for your audience to remember. If we need a bunch of text to support our point, we include it in a handout.
4. One idea per slide. Focus!
5. Include supporting slides. We’re already simplifying the content we present to maximize the impact of the ideas, which means that there is more content somewhere, but we haven’t shown it. Often someone in the audience (generally interested person, micro-manager, dude-trying-to-look-smart) will ask drill down questions – “Where did you get that data?” “Isn’t that diagram overly simplified?”. Adding those supporting slides (created in previous presentations, or prior to revision) after a blank slide (with the title “End of presentation”) to the deck. Don’t plan on showing these slides, just have them at the ready.

The best advice I know about preparing content for a presentation: Plan the formal part of the presentation to share 2/3 of what you want to tell the audience. Draw that last third out through engaging conversation and informal asides during the formal presentation.



]]> http://tynerblain.com/blog/2006/01/23/top-five-presentation-tips/feed/ 2