The ideal agile team is made up of specializing generalists – but what does that really mean?  The goal isn’t to prevent functional silos of expertise, it is to allow people to cover for each other.

Great Conversation

Elena Yatzeck (@eyatzeck) posted a comment on an earlier article about agile maturity models.

In terms of refinement, I’m thinking a lot these days about “staffing the engineering team correctly.” I’m not sure I agree in practice that you can or should try to staff all teams with “specializing generalists,” or at least not as taken to an extreme. (If you’ll forgive the self-promotion, I talked more about this here: http://pagilista.blogspot.com/2012/01/no-blender-zone-cross-functional-doesnt.html.)

I’ll not only “forgive” the promotion, I’ll re-promote it.  Good stuff.

When re-reading the maturity-model article, this snippet popped out at me:

People over process is the right emphasis. If you can’t find people that are “good enough” you might as well go home. Doesn’t matter how agile you are if you don’t have the horsepower. You also need people who are excited to “do agile” – they like to communicate, they enjoy the project and team dynamics of an agile process. You’re also better off with specializing generalists – ideally, every member of the team can do any work that is needed. This is an efficiency play – you risk introducing bottlenecks when you have a specialist who is the “only one” who can do particular types of work – because you will not have a consistent mix of types of work from release to release.
Agile Maturity Model

Thirty months later, my experiences have increased my conviction that this is true – and have realized that the way I wrote the quote above fails to provide a key clarification.

Following that link to an (even earlier) article on specializing generalists, brings the following (emphasis added):

The idea of specializing generalists is easiest to grasp by first saying what it is not. It is not staffing a team with a database expert, a user interface coder, a SOA (service oriented architecture) guru and an architect. With four specialists, each development task has an obvious owner. Database changes and refactoring go to the database expert. Reworking the UI goes into the queue for the AJAX hotshot. The problem is that this approach is only efficient when each team member is equally loaded with work. Since an agile team is continuously reprioritizing their work based on repeated feedback cycles as part of each release, this doesn’t work. The team will never face a situation where the (for example) four most important things to do are one item for each specialist. You can very easily have a release where all of the most important tasks are focused on the user interface. So all of the non-interface-experts are either working on lower-priority tasks, or even worse – they are idle. And you delay the most important work until the specialist can get to it.

By staffing a team with people who have an area of expertise, but can do anything, you can maximize the value of each delivery cycle. In our example, where all of the tasks for a release are UI tasks, they can be interchangeably assigned to any of the developers. The UI expert may suggest an implementation approach, do code reviews, or provide guidance to all the other developers. But every developer (including the database guy) can sling code effectively to get the job done. Specializing generalists.

This is very effective for making the “development engine” a black-box. Feed it the highest priority stuff, and it all gets done. We can take that approach to the next level. Designers can implement, project managers can design test plans, and yes, product managers can specify design. Twitch. Back up a sentence and read it again.

Specifying design is not the job of the product manager. True. Very true. Emphatically true. But specifying design can be what a specializing generalist does, even when that person is also responsible for defining market needs.
Specializing Generalists 2008

Elena’s article identifies a common misconception – that “specializing generalist” is a fancy way of saying “a bunch of people who can all do everything:

It’s a seductively simple fallacy of division to interpret the concept of “cross functional” team to mean a “collection of cross-functional individuals.” New agilists are quick to apologize that “we still have functional silos here” as though it would be much better if everyone could do all the same things. Grab some equally skilled poly-functional people, have them all take turns doing all of the jobs as needed, and you’ll all laugh your way to on-time, high-quality, and valuable working software.

Not so fast!

The power of an effective agile team, like the power of any other effective team, doesn’t come from its homogeneity, but from its ability to harness its diversity.
No Blender Zone: Cross Functional Doesn’t Mean Homogenous

Elena goes on to say (emphasis mine)

Team members shouldn’t attempt to Harrison Bergeron themselves into a mish-mash of mediocre (but working!) software. Someone needs to facilitate the stakeholders into some sensible semblance of a business case. Someone needs to build functional test suites that mercilessly beat on the code to prevent it from breaking in production. Neither of these are exactly the same skills it takes to gradually evolve the design of a complex system in modules of 100 lines of code or less. If people want to try new things, that’s great, but it needs to be with the realization that other jobs on the team are actual professions with skills and the need for experience in order to excel.

I completely agree.

Specializing Generalist

Specializing Generalist.

• Not a specialist.
• Not a generalist.

You need best of breed team members who specialize in areas of experise – “actual professions with skills,” as Elena puts it.  Without people who excel in the needed areas, you end up with a mediocre product.  How many times have you gone to the store and asked for the “middle of the pack” product?

That’s not even table stakes anymore.  Just the ability to create “something” isn’t interesting in the market, and isn’t interesting to the members of the team.  How many times have you heard someone brag “I love my job, I’m a cog in the machine?”  You have to have people who specialize in all of the needed areas (interface design, market insight, coding, quality, etc) in order to create a viable product.

If you staff your team with (only) generalists you will fail.

Pure generalists cannot create a product that is “good enough” – because they aren’t good enough at the creating the parts, from which the product is the sum.  You have to have people who specialize in creating great “parts” of the solution.  That’s what you need to have a shot at creating a great product.  But it isn’t really enough.  The problem is in how you define “great.”  Great means that customers buy it, users love it, and your competition is knocked back on their heels by it. Everyone agrees on this, but most people miss one thing.

Your market is changing – you also have to be fast.  You can’t solve the right problems if you aren’t fast, because the problems that are “right” are constantly changing – your market is a moving target.

Specialists, as individuals, are capable of creating great “parts” in their silos, and those parts all add up to a “great” product, so what’s the problem?  The problem is that collectively, by the time the specialists are done, they are no longer solving the right problem.

If you staff your team with (pure) specialists you will fail.

The most important tasks for the team, in any given sprint, will not balance into a perfectly allocated workload, where each “part” is worked on by each specialist, where no one is idle, and no one is a bottleneck.  It just doesn’t happen.  I haven’t seen it in 15 years in the software world, or in my prior decade as a mechanical engineer.

When one specialist is waiting for something important, she isn’t idle, she’s just working on something that is by definition not as important.  OK, you’re minimizing the damage – but you’re still taking damage.  When another specialist is the bottleneck, you lose.  Nothing magical to do here.

If you staff your team with specializing generalists you may succeed.

The work that piles up in any one specialized silo is of varying degrees of complexity.  The “UI specialist” may be backed up with a bunch of CSS tweaks, some straightforward AJAX calls to write, and a gnarly refactoring of the model-view-controller model to adapt to changing understanding of market needs.  No one can solve the MVC problem without specialized skills – but with guidance from the UI expert, one of  the other team members can handle the AJAX calls and CSS updates.  Extend this same model across other aspects of the product.  Your database expert may be needed to optimize query performance or resolve locking problems, but other members of the team could make straightforward schema changes.

It is the collective ability of the team to optimize what they collectively work on that accelerates the team’s delivery of the most important capabilities.

You have to have people who specialize, in order to optimize individual performance.  But your team needs to be built with specializing generalists in order to optimize for team performance.

T-Shaped People

From an HR perspective, I was taught about “T-Shaped People” – people who have breadth and depth of skills.

• Specialists are “I-Shaped People” – people who have depth of expertise, without breadth
• Generalists are “Minus-Shaped People” – people who have a breadth of skills, but no depth of expertise.
• Specializing Generalists are “T-Shaped People” – people who have depth of expertise in one area, combined with a breadth of skills across many areas.

These are the people you’re going for.

Thanks Elena for re-invigorating the discussion!

Should product managers get certifications? Ask a Good Product Manager asked us to answer.

Is There Value in a Product Management Certification?

In a recent question / article at Ask a Good Product Manager, an MBA student with a background in engineering and an interest in product management asked:

Among the certifications from organizations like Pragmatic Marketing, AIPMM, 280 Group and others, what are the aspects I should look for in deciding about product management certification?

Should I get product management certification?

For my complete answer, please check out the response on Jeff’s blog.  And if you’re new to it, sit back with a cup of joe, there’s a big pile of great questions and answers there.

I re-framed the question to match the title of the article – “Should I get product management certification?”

No and Yes and No

[ Thanks Wendy for the photo]

In my response to the question at Jeff’s blog, I pretty much start and end with the “no” side of the argument.

When I’m interviewing a product manager candidate, I don’t care if he or she has any certifications. I care a little bit about what they know (what skills do they have), and a lot about what they will be able to learn.

[...]

I’ll also add that I haven’t heard anyone I’ve ever worked with express that they “care about” certifications for product managers.

I mixed some “yes” into the middle:

Personally, I have the Pragmatic Marketing “practical product management” certification, which I believe is useful shorthand for “think strategically” and is a primer for discussion, but otherwise does not provide value. Their practical product management training is to this day the best single training class I’ve attended in any topic. I would place significant value on a product manager having the perspective that Pragmatic espouses, and being able to demonstrate their ability to apply it. Having the associated piece of paper is secondary.

I’ll add that someone who has attended Pragmatic Marketing’s training probably gets the benefit of the doubt.  I still “trust, but verify” their perspectives when interviewing a candidate or deciding to rely on someone to make a product great.

What, Then, Should You Do?

Is platitude format: Get good at product management.

What does that mean?

It means you need to learn how to manage your time and focus on strategic activities that add a ton of value.  It means you need to make sure you aren’t spending time doing other people’s work (because they won’t be doing yours).

Strategic activities?

• Understand your market and market segment – and understand the problems your customers face.
• Determine the value of solving your customer’s biggest problems.
• Learn how to really listen, and become a great communicator.
• Include the cost of solutions when prioritizing them by ROI.
• Create product roadmaps that communicate the vision of your product.
• Learn how to lead, especially people you aren’t managing.

A certification won’t help you develop those skills.  And to date, I don’t know of one that vouches for them either.

The director of product management at Mozilla (makers of firefox) has started a blog, Sherman’s Blog, about how Mozilla approaches product management. From the first post on the Role of Product Management by Sherman (Dickman), I think this blog is likely to be one to watch.

Mozilla’s approach seems to focus mostly on inbound product management activities (determining what to build) based on the activities they identify. They have a plan to create an MRD for firefox 3.

A Couple Quotes On Requirements Gathering -

If you ask a panel of home cooks questions about how to build a better oven, they’ll tell you. But if you ask them about the challenges of feeding a family each night, you might receive a completely different set of answers — answers that might suggest a new product opportunity (such as the microwave oven).

For example, Nike golf shirts now have SPF 30 protection. What types of questions did they ask consumers to come up with that idea?

One of the big challenges in writing valuable requirements is in identifying the value. We presented tips on interviewing to uncover requirements in January. Sherman provides a specific set of three questions that help maintain the focus on what is valuable and why. Check out their elicitation formula and see if you like this new entrant as much as I do.

Joy posted a really good article last week at Seilevel’s requirements defined blog, Measuring product manager performance on internal system products. Her post is a followup to an extensive and heated debate that happened last fall on the Austin PMM forum. It’s a great forum to subscribe to – a lot of experienced people with strong opinions and steamer trunks full of anecdotal data – and they don’t all agree. You get to see the coin from all three sides* with this group – it’s awesome.

Five Reasonable Metrics for Product Managers

She puts together a summary of five reasonable, measurable metrics for Product Managers

1. Track scope creep beyond a baseline of the requirements by counting new requirements added (and outside the original project scope).
2. Track errors of omission by counting new requirements added after the baseline (but within the original project scope).
3. Track errors of ambiguity/incorrectness by counting changes to requirements after they have been baselined.
4. Measure user adoption or satisfaction, presumably relative to an initial goal.
5. Measure the “missed ROI“ of the delivered project, relative to the initial plan.

There is good detail behind the sources of the measurements and methods of measurement. Joy also recognizes that they are imperfect metrics. I love the discussion that she has started, and hope it extends beyond this post as well. If you have a blog, post another followup to this one, and link back to both of our posts.

People perform to the metric

Measuring performance quantitatively is extremely hard. It can be very easy to make measurements, but it is very hard to know what to measure. The only thing worse than not having any clear objectives for an employee is having the wrong metrics. At the end of a review period (or during a project debrief), you want to be able to tell someone more than “exceeds/meets/fails-to-meet expectations.”

The problem then becomes defining a metric for measuring performance. Even worse than not having a metric is reaching the end of a review period and finding that someone excelled against the metrics, but failed against the objectives of the company. This is a classical organizational design problem, not just a “how do I measure product managers” problem. We will explore the larger problem in general, but use the PdM in specific for examples.

I contend that people’s performance is at best influenced by, and at worst strictly designed for meeting metrics. The philosophy and approach below is built upon that premise.

Looking at the bigger picture

On The West Wing, Leo is coached before a press briefing to respond with “I don’t accept the premise of your question.” when asked a lose-lose question. We should take a step back and explore the premise, instead of blindly critiquing the actionable ideas Joy presents.

Thinking outside the box

Since we talk about requirements definition a lot here, let’s approach the definition of a performance measurement system as a requirements elicitation exercise.

Why do you measure someone’s performance?

First identify the problem/opportunity of increasing the success of the company. It is reasonable to believe that people impact the success of your company through their performance.

• As a company, you have a goal. This could be philanthropic, profit motivated, or be based on any other company vision. Assume for the sake of this exercise that your company is profit motivated.  Your company also has a set of implicit and explicit values – Google includes “do no evil” in their list of values. Think of these as constraints on what your employees do and how they do it.
• People’s performance in their role either advances or hinders your company in reaching its goal. You therefore want to put systems in place that maximize people’s aligned performance with your company’s goals. You will also dedicate internal resources (a cost) to create and maintain this system.
• You want to maximize the long term, positive impact that a person’s performance has on your company goals.

What are the elements of your company’s goals that you want to impact?

What are those things that you want to achieve as a company, which you want to achieve through individual performance? For this exercise, let’s assume that you believe that profits are best achieved through long term successful product management relationships with your clients.

Based on that, and focusing on product managers, what are the traits or actions you want to encourage to support your goals?  Determining those aspects of your company objectives that are best addressed through individual performance is an ideation process.

• You want your projects to succeed. Successful projects will occasionally lead to repeat business, and failed projects will cause you to lose long term clients by weakening your relationships.  Successful projects  will differentiate you from your competitors.
• You want to improve project ROI for your customers.  By providing your customers with more cost-effective projects, you will gain additional business, and also make existing business more profitable.
• You want relationships with decision makers. A bad reputation will stop the best sales cycle on a dime. A good reputation can open doors, uncover opportunities, and serve as a tie-breaker in competitive bids.  Having a good reputation will help you close more deals, and close them more profitably.

What are the measurements you should use to assess performance?

You are crafting a system of measurement, designed to achieve the subset of goals that you believe are best accomplished through individual performance. As such, you should review some design considerations.

• You need to be aware of the danger of false precision. Here’s an example – in a 3 month period, one product manager wrote 100 requirements that did not change after the baseline. Another product manager wrote 50. Who is the better product manager? What about developers – is the developer who writes more lines of code the better developer? What about the one who closes the most issues? These metrics do not give you useful information without context. They are easy to measure, but hard to show the relevance of the measurement to the stated objectives. In fact, the best thing that can be said about them is that they are easy to measure.
• You need to avoid destructive (but instinctive) human behavior. Another example – you penalize product managers for changing requirements after the baseline – specifically based on %changed data. Your developers find the spec to be ambiguous, so they ask questions to get clarification. The product manager answers the questions instead of updating the spec.  Your product manager ends up wasting a lot of time explaining, but avoids getting dinged on an easily measurable metric.  By aligning behavior with the metric, this product manager becomes less efficient (by repeating the same explanation over and over), while simultaneously improving his “score.”
• You need to avoid performance metrics that can not be controlled by the employee. It can be demotivating to employees to know that their efforts to affect their performance reviews are futile. In the worst case, the employee will give up. In the best case, the employee will do what is right in spite of the metric – and you will fail to equitably reward the employee. There is an entire industry that defines compensation systems using quantified metrics. The stock market uses the same approach.
• You want to give feedback regularly.  Presenting an employee with feedback and suggestions for improvement will result both in more job satisfaction and in ever-increasing impact from the employee. This is not best done during a performance review (although it should happen then). Feedback should be given repeatedly during the performance period – to give the employee an opportunity to course-correct along the way. This would be agile performance-management. Waiting until the performance review to surprise someone with good or bad news is the waterfall performance management method.

OK, sounds impossible – what do I do?

A fomulaic appraisal system should not be used.

It is inefficient because the metrics can mislead you about who is doing a great job and who isn’t.  Metrics will stimulate sub-optimal behavior in employees, ultimately subverting your company objectives to some degree.

The right way to assess performance is to qualitatively assess the results of the work done by the employee.  Talk to your contacts at the client to assess the impact the employee has had on the relationship.  Have the employee submit periodic status reports with both anecdotes and progress updates. Review the success of the project, and identify the context – were the results in spite of or because of the efforts of the employee?  Ask the consumers of his artifacts (requirements documents) for their opinions on the quality of the documents – and develop an understanding of their capabilities to judge. Through the course of the review period, interview the employee and ask questions about the content of the status report. By understanding what’s working and what isn’t, what’s easy and what’s hard, and what’s on time and what’s late; you can apply your own expert opinion about an employee’s performance – and share it with the employee right then. This also then gives you the ability to assess the employee’s improvement relative to the feedback he has received.

And another thing

Another way to assess performance is to identify goals at the beginning of a review period, and track progress towards those goals. These high level abstractions are easy to align with company objectives, and can be easy to measure (did you sell an extra million dollars in services to your account this quarter?). This approach can be very effective when evaluating adaptive high-performing employees with the latitude to operate in a loose organizational structure to “do whatever it takes” to get the job done. This can be done in conjunction with the other qualitative assessments outlined above.

Align bonuses with goal-based measurement of final results (not interim steps), and manage salary via qualitative assessments. Note that this approach is only effective when reviewing empowered, typically professional, employees.

[Update: 2009.08.31: Edited the above article for style, and struck through the final piece of advice.  Watch Dan Pink's fantastic TED talk on the science of motivation.  Based on his research, you should not reward product managers for performance.  Honestly, I don't know what the right answer is - I'm conflicted, as Dan's ideas (and facts) resonate with me, but I also can't shake my belief in a reward system.]
*heads, tails, and the edge make up the three sides of a coin