The ideal agile team is made up of specializing generalists – but what does that really mean?  The goal isn’t to prevent functional silos of expertise, it is to allow people to cover for each other.

Great Conversation

Elena Yatzeck (@eyatzeck) posted a comment on an earlier article about agile maturity models.

In terms of refinement, I’m thinking a lot these days about “staffing the engineering team correctly.” I’m not sure I agree in practice that you can or should try to staff all teams with “specializing generalists,” or at least not as taken to an extreme. (If you’ll forgive the self-promotion, I talked more about this here: http://pagilista.blogspot.com/2012/01/no-blender-zone-cross-functional-doesnt.html.)

I’ll not only “forgive” the promotion, I’ll re-promote it.  Good stuff.

When re-reading the maturity-model article, this snippet popped out at me:

People over process is the right emphasis. If you can’t find people that are “good enough” you might as well go home. Doesn’t matter how agile you are if you don’t have the horsepower. You also need people who are excited to “do agile” – they like to communicate, they enjoy the project and team dynamics of an agile process. You’re also better off with specializing generalists – ideally, every member of the team can do any work that is needed. This is an efficiency play – you risk introducing bottlenecks when you have a specialist who is the “only one” who can do particular types of work – because you will not have a consistent mix of types of work from release to release.
Agile Maturity Model

Thirty months later, my experiences have increased my conviction that this is true – and have realized that the way I wrote the quote above fails to provide a key clarification.

Following that link to an (even earlier) article on specializing generalists, brings the following (emphasis added):

The idea of specializing generalists is easiest to grasp by first saying what it is not. It is not staffing a team with a database expert, a user interface coder, a SOA (service oriented architecture) guru and an architect. With four specialists, each development task has an obvious owner. Database changes and refactoring go to the database expert. Reworking the UI goes into the queue for the AJAX hotshot. The problem is that this approach is only efficient when each team member is equally loaded with work. Since an agile team is continuously reprioritizing their work based on repeated feedback cycles as part of each release, this doesn’t work. The team will never face a situation where the (for example) four most important things to do are one item for each specialist. You can very easily have a release where all of the most important tasks are focused on the user interface. So all of the non-interface-experts are either working on lower-priority tasks, or even worse – they are idle. And you delay the most important work until the specialist can get to it.

By staffing a team with people who have an area of expertise, but can do anything, you can maximize the value of each delivery cycle. In our example, where all of the tasks for a release are UI tasks, they can be interchangeably assigned to any of the developers. The UI expert may suggest an implementation approach, do code reviews, or provide guidance to all the other developers. But every developer (including the database guy) can sling code effectively to get the job done. Specializing generalists.

This is very effective for making the “development engine” a black-box. Feed it the highest priority stuff, and it all gets done. We can take that approach to the next level. Designers can implement, project managers can design test plans, and yes, product managers can specify design. Twitch. Back up a sentence and read it again.

Specifying design is not the job of the product manager. True. Very true. Emphatically true. But specifying design can be what a specializing generalist does, even when that person is also responsible for defining market needs.
Specializing Generalists 2008

Elena’s article identifies a common misconception – that “specializing generalist” is a fancy way of saying “a bunch of people who can all do everything:

It’s a seductively simple fallacy of division to interpret the concept of “cross functional” team to mean a “collection of cross-functional individuals.” New agilists are quick to apologize that “we still have functional silos here” as though it would be much better if everyone could do all the same things. Grab some equally skilled poly-functional people, have them all take turns doing all of the jobs as needed, and you’ll all laugh your way to on-time, high-quality, and valuable working software.

Not so fast!

The power of an effective agile team, like the power of any other effective team, doesn’t come from its homogeneity, but from its ability to harness its diversity.
No Blender Zone: Cross Functional Doesn’t Mean Homogenous

Elena goes on to say (emphasis mine)

Team members shouldn’t attempt to Harrison Bergeron themselves into a mish-mash of mediocre (but working!) software. Someone needs to facilitate the stakeholders into some sensible semblance of a business case. Someone needs to build functional test suites that mercilessly beat on the code to prevent it from breaking in production. Neither of these are exactly the same skills it takes to gradually evolve the design of a complex system in modules of 100 lines of code or less. If people want to try new things, that’s great, but it needs to be with the realization that other jobs on the team are actual professions with skills and the need for experience in order to excel.

I completely agree.

Specializing Generalist

Specializing Generalist.

• Not a specialist.
• Not a generalist.

You need best of breed team members who specialize in areas of experise – “actual professions with skills,” as Elena puts it.  Without people who excel in the needed areas, you end up with a mediocre product.  How many times have you gone to the store and asked for the “middle of the pack” product?

That’s not even table stakes anymore.  Just the ability to create “something” isn’t interesting in the market, and isn’t interesting to the members of the team.  How many times have you heard someone brag “I love my job, I’m a cog in the machine?”  You have to have people who specialize in all of the needed areas (interface design, market insight, coding, quality, etc) in order to create a viable product.

If you staff your team with (only) generalists you will fail.

Pure generalists cannot create a product that is “good enough” – because they aren’t good enough at the creating the parts, from which the product is the sum.  You have to have people who specialize in creating great “parts” of the solution.  That’s what you need to have a shot at creating a great product.  But it isn’t really enough.  The problem is in how you define “great.”  Great means that customers buy it, users love it, and your competition is knocked back on their heels by it. Everyone agrees on this, but most people miss one thing.

Your market is changing – you also have to be fast.  You can’t solve the right problems if you aren’t fast, because the problems that are “right” are constantly changing – your market is a moving target.

Specialists, as individuals, are capable of creating great “parts” in their silos, and those parts all add up to a “great” product, so what’s the problem?  The problem is that collectively, by the time the specialists are done, they are no longer solving the right problem.

If you staff your team with (pure) specialists you will fail.

The most important tasks for the team, in any given sprint, will not balance into a perfectly allocated workload, where each “part” is worked on by each specialist, where no one is idle, and no one is a bottleneck.  It just doesn’t happen.  I haven’t seen it in 15 years in the software world, or in my prior decade as a mechanical engineer.

When one specialist is waiting for something important, she isn’t idle, she’s just working on something that is by definition not as important.  OK, you’re minimizing the damage – but you’re still taking damage.  When another specialist is the bottleneck, you lose.  Nothing magical to do here.

If you staff your team with specializing generalists you may succeed.

The work that piles up in any one specialized silo is of varying degrees of complexity.  The “UI specialist” may be backed up with a bunch of CSS tweaks, some straightforward AJAX calls to write, and a gnarly refactoring of the model-view-controller model to adapt to changing understanding of market needs.  No one can solve the MVC problem without specialized skills – but with guidance from the UI expert, one of  the other team members can handle the AJAX calls and CSS updates.  Extend this same model across other aspects of the product.  Your database expert may be needed to optimize query performance or resolve locking problems, but other members of the team could make straightforward schema changes.

It is the collective ability of the team to optimize what they collectively work on that accelerates the team’s delivery of the most important capabilities.

You have to have people who specialize, in order to optimize individual performance.  But your team needs to be built with specializing generalists in order to optimize for team performance.

T-Shaped People

From an HR perspective, I was taught about “T-Shaped People” – people who have breadth and depth of skills.

• Specialists are “I-Shaped People” – people who have depth of expertise, without breadth
• Generalists are “Minus-Shaped People” – people who have a breadth of skills, but no depth of expertise.
• Specializing Generalists are “T-Shaped People” – people who have depth of expertise in one area, combined with a breadth of skills across many areas.

These are the people you’re going for.

Thanks Elena for re-invigorating the discussion!

Your boss wants a commitment.  You want to offer a prediction.  Agile, you say, only allows you to estimate and predict – not to commit.  ”Horse-hockey!” your boss exclaims, “I want one throat to choke, and it will be yours if you don’t make a commitment and meet it.”  There’s a way to keep yourself off the corporate gallows – estimate, predict, and commit – using agile principles.

This is an article about agile product management and release planning.

Change and Uncertainty

In the dark ages before your team became agile, you would make estimates and commitments.  You never exactly met your commitments, and no one really noticed.  That was how the game was played.  You made a commitment, everyone knew it would be wrong, but they expected it anyway.  Maybe your boss handicapped your commitment, removing scope, lowering expectations, padding the schedule.  Heck, that’s been the recipe for success since they planned the pyramids.

It makes sense.

1. Your early estimates are wrong. When you add them up, the total will be wrong.  If you do PERT estimation, the law of large numbers will help you in aggregate.  But you’ll still be wrong.
2. The outside demands on, and availability of, your people will change. Unplanned sick time, attrition, levels of commitment over time, lots of “people stuff” is really unknown.
3. The needs of your customers will change. Markets evolve over time.  You get smarter, your competitors get better, your customer’s expectations change.

Agile processes are designed to help you deliver what your customer actually needs, not what was originally asked for. Contrast the two worlds.

In the old world, you would commit to delivering a couple pyramids. After spending double your budget, with double the project duration, you would have delivered one pyramid. When you deliver it, you find out that sphinxes are all the rage. Oops.

Your team changed to agile, so that you could deliver the sphinx. But your Pharaoh still wants a commitment to deliver a couple pyramids (the smart ones will be expecting to get just one).  You can stay true to agile, and still mollify your boss’ need to have a commitment, if you take advantage of the first-principles of why agile estimation works.

Estimation

A commitment is a factual prediction of the future.  ”This will take two weeks.”  Nobody is prescient.

A factual prediction has to be nuanced.  ”I expect* this will take no more than two weeks.”

*in reality, this is shorthand for a mathematical prediction, such as “I expect, with 95% confidence, that this will take no more than two weeks.”

Few non-scientist, non-engineers, non-mathematicians understand that 95% confidence has a precise meaning.  People usually interpret it to mean “a 5% chance that it will take more than two weeks.”  What it really means is that if this exact same task were performed twenty thousand times (in a hypothetical world, of course), then nineteen thousand of those times, it would be completed in under two weeks – do you feel lucky?

To make a statement like this, you actually have to create a PERT estimate – identifying the best-case, worst-case, and most-likely case for how long a task will take.

Unfortunately, we’re rarely asked to make a commitment about a single task – but rather a large collection of tasks – well-defined, ill-defined, and undefined.

You can combine PERT estimates for the individual tasks, resulting in an overall estimate of the collection of tasks.

The beauty of this approach is that the central limit theorem, and the law of large numbers, work to help you estimate a collection of tasks – you can actually provide better estimates of a group of tasks than a single task.  This obviously helps with the well-defined tasks that you know about at the start of the project.  This even helps with the ill-defined tasks.  Rationalists will argue that the key, then, is to do more up-front research to discover the undefined tasks – and then we’re set.  As Frederick Brooks (Mythical Man-Month) points out in The Design of Design, this debate has been going on since Descartes and Locke.  It is not a new idea.

Big Up-Front Design and Requirements (BUFD & BUFR) hasn’t worked particularly well, so far.

Don’t throw out the baby with the bath-water, however.  The math of estimation is still important and useful.  Even if empiricism is not the silver bullet.

Prediction

Estimation is a form of prediction.  Even agile teams do it.  In Scrum, you estimate a collection of user stories – in story points that represent complexity, and you predict how many points the team can complete in this sprint.  Note the time factor.  If you’re working a two-week sprint, there is very little risk of changes in staffing during a two-week period.  There’s also very little risk that your market will change significantly in two weeks – and if it does, what are the odds that you will notice and materially change your requirements in two weeks?

Visually, let’s take that PERT estimate and turn it sideways – so we can introduce the dimension of time.  Imagine you estimated all of the tasks (well-defined, ill-defined, and a guess about the undefined), as if they were all to happen in the first sprint.  Ignore inter-task dependencies, and pretend you had unlimited resources and the ability to perform all tasks in parallel.

The graph above shows the aggregate estimate – the circle is your best prediction, with error bars representing your confidence interval in the estimate.  If you were using PERT estimates, these could represent that 5% and 95% confidence lines.  Subjectively pick something based on your team’s experience in the domain and your confidence in your guesses (about the undefined tasks).

We need a segue into the “best of waterfall” approach to estimating projects, to steal and invert a good idea.

The Cone of Uncertainty

The folks at Construx have published a nice explanation of the cone of uncertainty – an adaptation of an idea from Steven McConnell’s Software Estimation: Demystifying The Black Art (2006).  That article uses his imagery with permission – so please go look at it there.  The idea is that as the project becomes better defined (e.g. during the project), the amount of uncertainty is reduced.

The findings show that initial estimates are off by 400% (either low by a factor of 4 or high by a factor of 4)!  Even after “nailing down” requirements, estimates are still off by 30% to 50%!

As bad as that sounds, it is actually worse.  This is a prediction for the original project (delivering pyramids).  Not only are your estimates wrong – but they are bad estimates for delivering the wrong product.

But – the core idea is sound – the further into the future you have to execute, the greater the mistakes in your estimate.

Taking that concept, and applying it to our diagram, we get the following:

The further into the future you are trying to predict, the less accuracy you have in your prediction.  This reduction in accuracy is reflected as a widening of the confidence bands for your estimate.

• A couple sprints’ worth of work is not much different than one sprint – so your estimation range is not much changed.
• An entire release of sprints (say 6 to 10 sprints) has much more opportunity for the unknown to rear its head.

Now, your prediction is (probably) unusably vague and imprecise.  ”This set of tasks will take X plus or minus a factor of two.”

That’s the reality.

Note: This has always been the reality.  People have historically reduced this “risk to timing” by hiding the “risk of change” aspects – and waterfall processes encourage you to deliver the wrong thing, as close to on-time as possible.

That’s not what we want to do, however.

We still want to deliver the (not-yet-defined) right product, as efficiently as possible.  That’s the goal of agile.  (For folks who haven’t been here at Tyner Blain for long – “right” includes both value and quality).

Refinement

Because we’re agile, and we’re willing to “get smarter” about our product over time, we have an opportunity to improve.  Because of the nature of compounding estimates and the cone of uncertainty, our uncertainty gets smaller over time.

Let’s remove our artificial simplification that we could do everything “right now” and look at what we think we know right now, about the end of the release.

Our ability to predict the amount of effort (for today’s definition of the product) at the end of the release is not very good.

Our ability to predict (today’s definition of the product) one sprint into the future is much better.

After completing the first sprint, we are a little bit smarter – the ill-defined tasks are better defined.  Maybe some of the undefined tasks are now ill-defined.  The same cone of uncertainty is now a little bit smaller – we are a little bit smarter, and the time horizon of the release date is a little bit closer.

The trend continues – each sprint gets us closer to the release date, and with each sprint (assuming we get feedback from our customers, and continue to study our markets) we get a little bit smarter.  We also get better at predicting the team’s velocity (how much “product” they can deliver during each sprint).

Commitment

Your boss still wants a commitment, however.  And that’s where we get to change the way we look at this (again).

The above diagrams all display how we converge on an estimate for a stable body of work.  However, we know that the body of work is constantly changing.

Backlog! [you say]

Yes!  The backlog.  The backlog is an ordered, prioritized list of user stories and bugs.  I was talking with Luke Hohmann of Innovation Games last month, and one of the most popular online Innovation Games is now the one they created based on prioritizing by bang for the buck.  Play it today online (for free!).  How cool is that?

The backlog represents the work the team is going to do – in the order in which the team is going to do it.  Over time, as we get smarter, we will add and remove items from the backlog – because we discover new capabilities that are important, and because we learn that some things aren’t worth doing.  We will even re-order the backlog as we recognize shifting priorities in the markets (or in our changing strategy).

As this happens, it turns out that the items at the top of the list are least likely to get displaced, and therefore most likely to still be part of the product by the time we get to the release.

Instead of thinking about uncertainty in terms of how long it takes, think about uncertainty in terms of how much we complete in a fixed amount of time. In agile, generally, we apply a timebox approach to determining what gets built.

Now, uncertainty, instead of manifesting as “when do we finish?” becomes “what will we finish?”

Your boss is rational.  She appreciates the constraints, she just wants to know what you can commit.  Every boss I’ve worked with has been willing (sometimes only after much discussion) to treat this uncertainty in terms of what instead of when.  They acknowledge that they need to translate (usually for their boss) into a “fixed” commitment.

The solution: commit to a subset of what you predict you can complete.

At the start of the release, you may have 500 points worth of stories.  Based on your team’s expected velocity, and the number of sprints in the release, you predict that you can complete 320 points worth of stories (5 people on the team, a team velocity of 40 points per sprint, and 8 sprints in the release).  Starting at the top of the backlog and working down, draw a cut-line at the last story you can complete (when you reach 320 points).  This is your prediction.

Now the commitment part.  You’ll have to figure out what you’re comfortable with.  Maybe for 8 sprints (say, 16 weeks into the future), you may only be comfortable committing to half that amount – 160 points.  Go back to the top of the backlog, and count down until you reach 160 points.  Everything above the line is what you commit to delivering.

Maybe you are comfortable committing to 240 points, maybe only 80.  This is like playing spades.  The more you can commit to, without missing, the better off you are.  Your tolerance for risk is different than mine.

You can also negotiate with your boss.  Commit to 160 points now, and provide an update after every other sprint.  More likely than not, you will be increasing the scope of your commitment with every update.

Mid-project updates of “we can do more” are always better than “we can do less.”  And both are better than end-of-project surprises.  This also allows you to have updates that look like this:

We didn’t know this at the start of the release, but X is really important to our customers – and we will be able to deliver X in addition to what we already committed.  Without slipping the release date.

Conclusion

Making commitments with an agile process is not impossible.  It just needs to be approached differently (if you want to stay true to agile).  The end result: better predictions, more realistic commitments, and the likelihood that each update will be good news instead of bad.

[Update: Changed initial image.  Thanks Dennis for the great photo!]

Does product management slow down your company?

What Causes Your Business to Be Slow?

Paul Young put out the call for the third annual You Might Be A Product Manager… list.  If you are spending your holiday wondering if Jason Calacanis is right, and product management is actually preventing your company from being successful, you might be a product manager.

Facebook’s success — and mistakes — are based on its developer-driven culture, not because Zuckerberg is some evil mastermind.

The Zuckerberg Doctrine: Developers design products with significantly improved speed and functionality compared to product managers and designers, outweighing potential mistakes and drawbacks.

Jason Calacanis, Launch newsletter 002

Sound like link bait?  Maybe, but you’re probably a product manager :).  Jason’s put his money where his mouth is – he changed the way things are done at Mahalo (his human-curated search / answers company).

In under 30 days, we completely overhauled our product-development process, removing everything between the developer and iterating on the product.

We eliminated positions and process. We made it clear the developers were to make the decisions even if those decisions resulted in a developer being 50 percent slower because they were busy *thinking* about the product (as opposed to just transcribing features from the product manager wireframes).

Jason Calacanis, Launch newsletter 002

As you read through the details of the analysis in Mr. Calacanis’ newsletter, you’ll see that his position is not ultimately as generalized as the above quotes appear to be – Mr. Calacanis (who I’ve seen, and to whom I’ve listened for several years, but never met) is talking specifically about startups.  However, he uses AOL, Yahoo, MySpace and Google as his “bad” examples – not exactly startups.

I think this logical flaw may be leading Mr. Calacanis to demonize the wrong bad actors.

ProductCamp Austin 6

If you’re planning to attend ProductCamp Austin 6 – January 15th, 2011 (register here if you haven’t already), then consider attending a session that John Milburn, Roger Cauvin, and I are going to host – discussing this topic.  If you can’t attend, then jump into the conversation here.  John and Roger and I have been talking about this and exchanging some ideas since the newsletter came out – and we really know how much better ideas get when we open the conversation up to the community.  We’re looking forward to doing that at product camp, and I’m writing this to open it up now – so please, chime in!

Implicit Product Decisions

When I was still writing software and leading teams that were writing software, I would occasionally point out that all software is designed – even if someone sits down and just starts typing code.  There is, at the minimum, implicit design happening in the mind of the programmer.  It might not be “good” design, but there is always design.  There is always a method to the madness, just not always a considered method.  The same is true of product management.

• The creation of every feature and capability, in every product, is preceded by the notion that having this capability is a good idea.  That’s what product managers do – decide which capabilities a product should have.
• Eliminating product managers does not eliminate product management.

If “developer led” companies are still doing product management, how then, are they moving faster?  Mr. Calacanis addresses valid points – his “bad examples” do seem to move pretty slowly, and his “good examples” do seem to move faster.  So what’s really different?  John and Roger and I will be framing the discussion at ProductCamp around how to move faster, and not throw the baby out with the bathwater.

I might describe what Mahalo has apparently done as follows:

• Mahalo, with product managers involved in product decisions, was not moving as fast as Mr. Calacanis desired.  So they reorganized so that product managers were no longer involved in the process – in hopes of having a faster process.  Mr. Calacanis indicated that in a trade-off between “better” and “faster,” he would prefer “faster.”

Agility

The core of the agile development philosophy is – “fail fast. learn. improve.”

All of the other stuff in the implementations of agile comes back to this – consider the main ideas in the agile manifesto*

• People over process: empowerment to fail and learn and improve.
• Value working software: learning is experiential, and you can’t fail or improve without shipping.
• Collaboration: You have to understand someone else’s problem before you can solve it.  Too many products emerge from insular and isolated “exploration.”
• Encourage, don’t inhibit change:  If you punish failure you prevent learning. If you prevent that new knowledge from being applied, you make learning irrelevant.

*Agreed – those aren’t the words used in the manifesto.  Same ideas, though.

When you talk about business agility, while the mechanics might be different, the goals are the same.  Fail fast.  Learn.  Improve.

Mr. Calacanis, in my interpretation, is saying that this is exactly what he wants the Mahalo team to do.  I applaud that goal.

Does he have to “kill all the product managers” in order to infuse his company with (business) agility?

Is product management the antithesis of agility?  By definition, product management is still happening – just without product managers.  Maybe the product management process has room for improvement…

Market Driven

As a good product manager, you are market driven.  What does that mean to agility?

• Fail Fast.  Maybe you’re making decisions that delay launches until you know “the right product.”  That would hurt agility.
• Lean. Are you listening to your customers, and learning from them?  Great!
• Improve. Does what you’ve learned lead to trying something different, with a hypothesis that it will be better this time?

OK, so being market driven will help your company learn and improve.  That’s the up-side.  It may also enable (but not cause) some corporate dysfunction. If your organization punishes failure, or is afraid of mis-steps, and you’re market driven, you have already heard this conversation – don’t (schedule | design | release) until we get [feedback X] or [insight Y].

A tight coupling with your market is a powerful tool.  It can be used for good (learning) or evil (avoid failure).

Whew.  Good to know that being market driven is not the source of the problem.  An organization that is afraid of failure is the problem.

Mahalo’s experiment may work – Mr. Calacanis clearly intends to encourage the “fail fast” element.  So, when his developers are doing product management, they will have an opportunity to succeed by being market driven.

Bureaucracy

In fairness, Mr. Calacanis is really only prescribing the “no product managers” approach for startups.  Startups are not particularly bureaucratic.  Perhaps Mahalo was becoming bureaucratic.  It is easy to see the big companies mentioned in the newsletter as being rife with bureaucracy.  Even if you could fail fast, learn, and improve in a world of t-crossing and sign-offs, your definition of “fast” would not match your competitors.

Piloting your company would be like flying a Cessna twin-prop airplane in a world of super-sonic Gulfstream jets.

How are product managers introducing overhead into your product creation process?  What parts of product management are “not worth the delays?”

There are very real “slow things down” activities in the product creation process.

• Some activities can be removed.
• Some activities can be improved.
• Most activities can be done in parallel with the product creation process, eliminating delays.

What Have You Seen?

There are a lot of stories from the trenches out there – what are you seeing?  Have you tackled this already?  How did you make it better.

I hope that our session focuses on helping attendees, in a very real way, make their product creation process more effective, and make their businesses more agile.

Of course, we don’t have to wait until January 15th – we can start talking about it right now…

The maturity model approach to describing organizations and processes comes and goes out of fashion.  It is a repeating framework de jour.  In the game of agile jargon whack-a-mole, the agile maturity model is poking its head up again.

Agile Maturity Model

Ellen Gottesdeiner, author of Requirements by Collaboration, tweeted (@ellengott) a few hours ago with a link to an article proposing a set of maturity models.  She graciously passed on the opportunity to comment about the “agile maturity model” when pointing out her like of the collaboration maturity model.  The article proposed the following as an agile maturity model:

Agile Maturity Model (AMM)

0 – Dormant

1 – Speed: Focusing on being expeditious.

2 – Reactive: Focusing on acting relative to change from the perspective of the moment rather than a longer timeframe.

3 – Responsive: Focusing on acting relative to change from the perspective of the moment balanced with a longer timeframe.

I didn’t find it to be a particularly useful model.  Although descriptive, it won’t help your organization improve.

What Does Maturity Mean?

I wrote a series of articles a couple years ago that explored the CMMI and RMM – the capability maturity model and the requirements maturity model.  These are two different models that use maturity as a concept for articulating different levels, or grades (or enlightenment, or rigor), with respect to organizational behaviors.  At the time, there was both momentum and confusion around the notion of a requirements maturity model.  CMMI is a description of an organization’s rigor in “saying what it does, and doing what it says.”  RMM is an assessment of the level of critical thinking incorporated into the ways an organization is using requirements to develop products.

The problem is that people were incorrectly assuming that an organization “at level 4 CMMI” would approach requirements management at a comparably enlightened level.  The problem is that they are putting entirely unrelated concepts into similarly named classifications.  An organization could be at CMMI 1 and RMM 4 or RMM 2 and CMMI 3.  That series of articles explored what it would mean to be in any of the combinations of “maturity” for those two models.  Check out all the articles in the series if you want to put it all in perspective:

• Foundation Series: CMMI Levels Explained
• What CMMI Level Should We Use?
• CMMI Levels and RMM Introduction
• CMMI Levels and RMM Level 1 – Written Requirements
• CMMI Levels and RMM Level 2 – Organized Requirements
• CMMI Levels and RMM Level 3 – Structured Requirements
• CMMI Levels and RMM Level 4 – Traced Requirements
• CMMI Levels and RMM Level 5 – Integrated Requirements
• Don’t forget to take our One Minute Survey on CMMI and RMM.

Making a Maturity Model Useful

Why bother with a maturity model?  Not so you can “keep score” relative to others – rather, so that you can improve your own organization.  For a maturity model to be useful, you have to be able to do two things.

1. You must be able to determine where your organization is in the model.
2. You must be able to identify actions you can take to “improve” your organization relative to the model.

If you can’t measure maturity, and the model does not provide guidance about how to improve, it’s useless.  One challenge with maturity models is that they risk becoming contextually narrow in their application.  The more concrete a measurement or suggestion becomes, the less extensible it is likely to be.  Ideally, your model would be broadly applicable to many organizations.

With an agile manifesto that emphasizes people over process, it is ironic to consider applying a metric that measures your agile process.  So – goal #1 is immediately undermined.  Goal #2 – improving your organization, is, however, very valuable.

Improving An Agile Process

I’ve worked in several agile environments, both as a practicioner and as someone helping to transform organizations to become agile (or better at agile).  I’ve worked in enterprise software, helping large teams adopt agile processes; with a SaaS consumer product team, and in large IT departments with collections of teams adopting agile processes at varying paces.  I’ve played on both sides of the fence (delivery/QA & product management/ownership).

Going from zero to sixty on agile is a big task.  You can’t solve all of the organizational, practical, and interpersonal challenges at the same time.  Or at least you would be more effective tackling and resolving each issue in sequence.  In fact, you should solve the most important / largest problem first – then move on to the next, and so on.  You can call it agile agile adoption, or you can call it eating the elephant.

One powerful use of a maturity model is highlighting the next-biggest hurdle you have to overcome.  Unfortunately, most communication about maturity models is “look which hurdle we just overcame” – but the focus should be on “what’s next?”

When helping organizations be effective with agile processes, there are some clear “you have to overcome this first” challenges, that if unresolved, make other challenges irrelevant.

If you’re a software developer, I’m sure you’ve experienced the following scenario:

• A bug is reported.
• You fix the bug, and write some tests to prevent it from reoccuring.
• While testing, you discover another bug that was hidden by the first bug.

Fixing that second bug doesn’t do any good until you’ve fixed the first one.

If you’re a product manager or business analyst, think about it in terms of capability enhancements.  Is a faster search important if the current search algorithm is not returning the right results?  Get good results first, then make it faster.

You have to solve the biggest/closest/roadblockiest issue first.  Then move on to the next issue.  That’s how you should use a maturity model – as guidance about “what’s next?”

When assessing / improving your organization’s ability to be agile, you need to be addressing whatever is next.

What’s Next?

Let’s discard the “maturity model” notion and focus on “what’s next?” instead.  And that of course starts with “what’s first?”  Here’s how I’ve presented an agile hierarchy of needs to in the past:

1. Staffing the engineering team correctly.  People over process is the right emphasis.  If you can’t find people that are “good enough” you might as well go home.  Doesn’t matter how agile you are if you don’t have the horsepower.  You also need people who are excited to “do agile” – they like to communicate, they enjoy the project and team dynamics of an agile process.  You’re also better off with specializing generalists – ideally, every member of the team can do any work that is needed.  This is an efficiency play – you risk introducing bottlenecks when you have a specialist who is the “only one” who can do particular types of work – because you will not have a consistent mix of types of work from release to release.
2. Assuring Quality is in your team’s DNA.  Arguably, this is part of what’s first, but there are a lot of teams that get cood at cranking out code, before they get good at cranking out good code.  Continuous integration is the approach you must have.  Test-driven development, spec-driven development, and other testing-integration approaches are extremely important, but are really reflective of different flavors of agile and quality, not different degrees.
3. Reducing overhead in the release process.  There’s a cost associated with releasing a product.  Once you get good at releasing, and are releasing good product, your next focus is on finding the right release cadence.  There are three factors that essentially dictate your release frequency.  The size of atomic deliverables (user stories and use cases, non-functional requirements, etc) your team is creating, the overhead (time and cost) of releasing, and your customer’s capacity to consume the releases.  My anecdotal experience is that teams are usually constrained initially by the cost of releasing – dedicated hours to creating a build, code freezes, etc.  Automating the build process (to reduce both costs and errors introduced while building) is first.  Integrating automated build and automated test processes together is next.
4. Feeding the beast.  Once you have an engineering / delivery team that is operating efficiently, you run into the problem of making sure they have enough important work to do.  There’s pressure on product owners and  product managers to schedule something because it is easy to define, not because it is important.  It is also important that the team understand the context and importance of what they are doing.  So you have to focus on making sure your product management team has enough capacity to keep the engineering team busy on delivering really valuable stuff.  The “right” solution to this depends on how the problem manifests in your organization. Is your product manager spread too thin across multiple products?  Too junior?  Too bogged down in sales support or trade show attendance or or or?
5. Managing stakeholder expectations.  A big challenge in changing an organization to become agile is in resetting and managing the expectations of stakeholders.  Execs are used to having an annual budget exercise where they dedicate X dollars in the pursuit of Y objectives.  We’ll set aside the reality that they’ll only achieve 1/3 of the objectives, at 2X the cost, much later than originally forecasted.  We aren’t looking at the failings of waterfall, we’re looking at the risks of agile.  There’s a ton of stuff here, from doing damage control to reverse perceptions that “people who want to do agile just don’t want to be accountable” or addressing the “we can’t tell you what X dollars gets you” message.  Again – the particular solution is a function of how the problem manifests in your organization.  Once your team is delivering valuable stuff efficiently, you have to make sure your management team is happy and engaged and knows what to expect.
6. Continuously learning from your markets.  This is really what differentiates an agile organization from an organization with an agile development team.  Agile processes do enable you to develop code more efficiently.  If you aren’t taking advantage of the faster development, feedback loops, and ability to change that agile enables, you’re leaving money on the table.  And one of your competitors will pick it up.  There’s a whole community of MBAs that talk about business agility without once thinking about software development.  They’re talking about an organization’s capacity for rapid response to changing market conditions.  It’s what separates the winners from everybody else.  And agile development makes business agility possible.  As long as your team is able to identify and value industry trends, competitive threats, and market opportunities.  When you’re good at this, you have an agile organization.  And you win.

Conclusion

Any agile “maturity model”, irony aside, needs to be structured to help organizations progress through the stages of enlightened operations – continuously providing insights into “what’s next?” for those teams to grow.

I know there are many readers here with years of agile experience – how would you improve the list above, to better provide a framework for “what’s next?”  When that framework is solid, we can do some wordsmithing and repackage it as a maturity model.  Until then, just focus on “what’s next” – that’s all a maturity model should be used for anyway.

Business rules are often hidden in processes as hidden decisions.  Once you discover that hidden decision, how do you communicate the impact of exposing and managing the decision?

Hidden Decision

In our previous article on hidden business rules and enterprise decision management, we looked at a process that had a hidden decision.

First, take a look at the process with the decision still hidden:

Here’s the process (with the decision exposed) from that article:

The decision that was hidden was an implicit decision – “Pick One?” – for which the answer was always “No.”  After making this discovery, and communicating with your team, you then have to decide if you are going to make the decision explicit.  Making the decision explicit will involve defining the business rules for how to make the decision, and (if done in software) implementing the decision, or the ability for a person to make the decision.

Impact of a Decision

The diagrams above allude to the fact that there is money to be made in the process, in the “Continue Automated Process” step.  We need to provide a better understanding of how / when money is made.  A slight expansion of the process flow above (replacing the “Continue Automated Process” with a couple steps, looks like the following:

[click for larger version]

The last step introduces a real-world decision: “Sell products?” and only when the answer is yes does the process “Earn Revenue.”

If you’re analytically minded, you can apply percentages to each branch of each decision, and add everything up to figure out an answer.  That might be reasonable for a simple example like this one, but it can become too complex with a more complicated process.

There’s an easier way.  And since the goal is to communicate the impact to someone, you want an easier way.

Truth Tables and Decisions

A truth tables is a tried and true artifact for describing combined sets of decisions.  You can use it effectively to extract information from a decision-heavy process diagram.  Consider the following application of a truth table to the process above:

[larger version]

Each of the decisions in the process flow has a column.  Each row represents a unique path through the flow.  Each cell contains the result of the decision, for that path.  Note that when a path does not involve a decision, there is no value in the cell.

That provides a comprehensive view of the process, which you could build upon.  However, in this case, you are focusing on the impact of the hidden decision (“Pick One”).  The following paths, from the truth table, involve that decision:

[larger version]

Ignoring the other paths, and re-organizing, you find the following:

[larger version]

50% of the time, when “Many” results are found in the first decision in the process, if you pick one of the results, you will earn revenue.

[larger version]

When you do not pick one, 20% of the time, the customer (or user) abandons the process.  For the users that do not abandon the process, you earn revenue 50% of the time.  Taking both into account, you earn revenue only 40% of the time.

Adding Up the Impacts

When there are “Many” results from search, you can improve the performance of this part of your process by 20% – specifically, you increase revenue by 20%.  But only if you always “Pick One” of the many results. You can extend this analysis to determine the percentage of None/One/Many outputs from the search at the start of the process, and determine an overall impact on the process.  If search returned many results 10% of the time, you can increase revenue (overall) by 1% by always picking one of those results.  That gives you the business justification to calculate ROI on exposing the hidden decision.

The end result is that you have a simple way to communicate the impacts (truth table) in the language of stakeholders (ROI).

A business process is not just a sequence of steps.  A business process is a series of decisions and actions.  Some decisions are obvious and can be actively managed.  Some decisions are hidden, and until you discover them, you can’t manage or improve them.  Here is a real-world example of the discovery of a hidden enterprise decision.

Enterprise Decision Management

Over the past couple of years, I’ve collaborated with and become friends with James Taylor, an enterprise decision management guru.  He and Neil Raden wrote the book, Smart (Enough )Systems , you should get it if you don’t already have it.    It is sitting on my desk right now, and every time I pick it up I have to mark another page with a post-it note so I can find that good idea again later.  If you aren’t convinced, listen to this interview with James about their book.

James and I developed a presentation, Getting It Right: Rules and Requirements in Software last fall for the International Business Rules forum.  Below is a slideshare of a shortened version of the presentation, delivered to the Austin IIBA this spring.

A large portion of that talk was dedicated to discovering hidden business rules within use cases.  The reason we dedicated so much time to rules discovery is that the first step to managing decisions across your enterprise is to discover and isolate those decisions!  In this article, we will look at an example of a decision hidden within a simple business process.

A Simple Business Process

The following is a simple real-world business process.  The details of the steps in the process have been modified to clarify this example.

The process flow example above depicts an automated process that involves searching internal systems for an existing customer (record).  There are three possible outcomes from the search, shown above from left to right: no results are returned from the search, a single result is returned, or multiple results are returned.  When no results are returned, a new customer (record) is created, and the automated process continues.  When a single result is returned, that customer is used and the process continues.  When multiple results are returned, a new customer (record) is created and used, but the automated process is interrupted and a manual intervention is required before the process can continue.

This third path confused me when it was initially presented to me this way by the subject matter expert who explained the process to me. The business creates a new customer (record) even when multiple results are found because they don’t have time to determine which result is the right one, within the automated process.  They would rather create a duplicate customer (record), continue with the automated process, and clean up the mess later.  That’s much better than losing a sale (a very real risk because of delays due to manual oversight.

As part of this elicitation session, I applied a variation on active listening by talking to the diagram above (on a whiteboard) and enhancing it.

Impacts and Decisions

One technique that is effective in reaching agreement, generally, is to inspire the other person to “have your idea” so that it becomes their idea.  Once I understood the process above, I realized that there was an implicit decision in the process.  Instead of making the statement – “you have a hidden decision in your process” and then hoping the business stakeholder (also in the room) would care, I chose to start with the impact.  Presenting the impact of a hidden decision first might expose that decision and trigger acknowledgment of the hidden decision by the stakeholders.  The following diagram shows how I first modified the flow on the whiteboard.

The subject matter experts had told me that when there is a manual intervention in the otherwise automated process, the process is sometimes abandoned.  Imagine customers walking away from an online sale because (for some reason) the sale cannot be completed without an unexpected delay.  I represented this with the “Abandon Process?” decision in the flow.  This decision was never drawn before, because it is the customer’s decision, and it can not be controlled by the business.

I also pointed out that the business makes money only through the last step in the process, “Continue Automated Process.”  My goal was to accentuate that this abandonment decision has a very real value.  The stakeholder and subject matter experts agreed – this was something they had told me previously.  But they had never drawn it directly within their process.

As it turns out, this did not trigger any additional insights.  It did however, set the stage for describing the relevance of the hidden decision.

The Hidden Decision

After setting the stage, I updated the process flow diagram to expose the hidden decision.

The hidden decision is the decision of picking one of the many search results (top right corner of the diagram).  Your first response may be that the business is not picking any of the results.  My response is that choosing none of the provided results is a choice, just as picking any one of them would be.

This view of the process highlights that if the business could automatically choose a “best” result when searching returns multiple results, the business would avoid the risk of abandoned processes due to manual oversight.

With the current process, the business stakeholder acknowledged that the hidden business rule is ” pick none of the results 100% of the time”.  With current project schedules, there is nothing that can be done about that in time for the immediate software release (of the software that embodies this process).

Before our meeting was over, the business stakeholder added determination of future-state business rules to define the “how do we pick one?” decision to the deliverables for the next release.

A Smart (Enough) Decision

James and Neil (I would guess) will tell you that ideally, the decision (should we pick one, and which one should we pick) should be informed by the process.  A feedback loop can be created where we identify the customers (the people who might abandon the process), and determine statistically which ones are most likely to abandon the process.  For those people, it is especially important that we be able to “pick one” from a set of search results, thus completely avoiding the opportunity to abandon the process.  For those people who are unlikely to abandon the process, the “pick one” decision is less important.

If it were easy to “pick one”, we would consider changing the way search is handled to never return multiple results.  In this real world example, that is not an option (I’m intentionally not sharing the reasons, to avoid disclosing too much).  The only option in this case is for the business to respond intelligently (enough) when presented with multiple search results.

Conclusion

This process flow example has two primary outcomes – generate profits, or don’t generate profits.  The first modification of the process flow makes that possibility explicit.  That allows for an analysis to determine if there are any hidden decisions in the flow (there are) that impact the outcomes of the flow.  There may be other hidden decisions, but they do not impact the profitability of this process, so they were not explored.

Product managers are often described as the hub or center of a software development organization. Saeed Khan takes umbrage with this under-appreciative image in an awesome article about information flow, product managers, and the SDLC.

Hat Tip

I love that the community of great people writing about product management and business analysis has grown so richly over the last couple of years. Yet again we get to thank someone else for pointing us to a great article. Steve Johnson (vote for him) points us to the On Product Management blog in a recent article. They’ve quietly amassed over 50 articles in the last three months, and we, thanks to Steve, found one we loved today.

Information Flow

In one of a series of articles on being a great product manager, Saeed discusses the importance of the product manager not only in creating information, but in getting the right information to the people who need it. Here’s how you know you want to read the article. First, scan it. Then click on the two diagrams. The first one shows the network of roles in a typical large software company, showing the more significant information flows between them. The second diagram shows a detailed heatmap of the amount of information flowing through any particular role as a function of where the product is within the SDLC.

As soon as you see these two diagrams, you will immediately walk away with the appreciation that Saeed speaks with perspective, experience, and insight. Now you know you want to read the rest of the article in depth.

Enabling, Not Documenting

One thing I really like about Saeed’s article is that he positions part of the role of the product manager as enabling other people on the team to be more effective. Product managers don’t make all the decisions, don’t do the product implementation, and don’t do the tactical selling activities. Other people do. What product managers do is enable them to do it better, by providing them with the right information.

In another good article from the same blog, Alan tells about the tragic demo from a company pitching software to product managers – and discussing the features, not the value. Unlike in that demo, Saeed is focused on the value of information flow within an organization. He doesn’t talk about the artifacts that are created (if any), he focuses, particularly with the heatmap, on how different roles are involved in different stages of the SDLC.

The article makes good points, and the heatmap drives home that there’s rigor behind the assertions. And as the first article I read from this blog, I’m extremely excited to read more. I would like to see (and will now have to think about) how different roles need information with an agile product development lifecycle. These are the types of inward focused assessments that help us get better at product management, and help our teams achieve greater software product success.

Today we recorded an interview with James Taylor, co-author of Smart (Enough) Systems, How To Deliver Competitive Advantage by Automating Hidden Decisions. This book, written by James Taylor with Neil Raden comes out on Jun 29th (2007), and is available for pre-order from Amazon today. Our interview covers many of the topics in their book, with a focus on the ideas inside and the benefits you can get from applying them, in just under an hour.

Smart (Enough) Systems

James and Neil introduce a pretty compelling concept – that paying attention to small decisions can have as large an impact for companies as paying attention to large decisions. All successful companies invest in strategic decision making – the big impact, infrequent decisions. But most companies overlook the benefits of investing cost-effectively in the small decisions that happen thousands or millions or even billions of times a day.

In our interview, James gives us some insight into exactly how that can happen, how to approach these small decisions, and how to manage them.

Interview With James Taylor

You can download our interview with James Taylor in mp3 format (18 MB, 52 minutes) and listen at any time. As this is my first recorded interview, please accept my apologies for the “ums and ahs.” James does 99% of the speaking, so it still turned out ok – he did a fantastic job.

James’ book is an excellent read, and eye-opening, revealing what should be obvious but remains hidden about improving our businesses. Every action and interaction a company makes is the result of a decision. Making those decisions accurately, quickly, and cost-effectively is critical to every business. Being able to adapt those decisions (and their resulting processes and customer-interactions) rapidly is the key to success in a world that is changing faster than ever. The flow of ideas is straightforward, and each new idea builds upon the previous ones, allowing readers to quickly conceptualize a better way to run their business. The anecdotes and real-world examples also solidify the concepts, and almost force the reader to think about how to apply it to their own business.

I personally recommend the book, and thank James for the opportunity to read it and speak with him about it.

More Resources on Smart (Enough) Systems

Near the end of our interview, James mentions the web site for the book. The book is available from Amazon now, and can be ordered through the link on the main page. They also have an RSS feed with news on the book, and will be launching a wiki for the book very soon as well.

This approach to integrating business and IT systems is also known as enterprise decision management, or EDM. You can read more about EDM at James’ EDM Blog.

Please take one minute to answer the following two questions, because people need to know how their process maturity compares with everyone else. You’ll be answering two easy questions -

• What is your CMMI level?
• What is your RMM level?

Background

We just completed a series of six articles about CMMI levels and RMM levels. We discussed how the two frameworks can be connected, and explored the reasons for trying to reach the next level on either scale.

Question 1

[poll=2]

Question 2

[poll=3]

Thanks for taking the survey, and if you have any thoughts about the results, just comment here.

Background

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure. In this article, we will look at the definition of RMM level 5. We also look at the mapping from RMM level 5 to various CMMI levels.

(larger version)

In the previous article in the series, we looked at how RMM level 4 – traced requirements processes map to CMMI levels.

RMM Level 5 – Integrated Requirements

In RMM level 4 we discussed the benefits of establishing traceability within the scope of the requirements realm. This traceability helped us with tracking the impact of changes and validation of requirements, and it helped us with reporting and management functions that improved our insight and lessened our efforts.

RMM level 5 extends this traceability concept beyond the requirements realm. We can extend tracing into development, testing and documentation, as well as project management.

Delivery Scheduling

Each delivery of our software includes a set amount of functionality. We prefer using a timebox approach for planning those deliveries. We also suggest using use cases as the “pieces” of functionality that are delivered. We also have proposed an approach to managing changes to the delivery schedule. This overall process is a form of integration of requirements, and is dependent upon having organized, structured, traced requirements. It also requires us to create linkages between implementation elements/effort and the requirements that are supported.

Quality Assurance

There are two elements of integration between QA and requirements. The first is in defining test cases based on use cases. Each use case drives the creation of one or more test cases that are used to validate that the delivered software meets the requirements.

The second element is in tracking and management of bugs. We have a triage process for determining how to address bugs. Those bugs can come from anywhere – and sometimes they come from incorrect requirements. Establishing relationships between requirements and bugs allows us to manage the defect resolution process more effectively.

Documentation

When we write documentation, our goal is to have the reader know how to accomplish something that they want to do. When we develop software, we define the things that users will be able to do. It only makes sense to structure documentation around use cases.

Mapping CMMI Levels to RMM Level 5

In our diagram, we show the following mappings for RMM level 5:

• CMMI level 0 – No Entry
• CMMI level 1 – No Entry
• CMMI level 2 – No Entry
• CMMI level 3 – Requirements Should Be Integrated
• CMMI level 4 – Requirements Should Be Integrated
• CMMI level 5 – Requirements Should Be Integrated

For CMMI Level 0 through CMMI Level 2 – when our process is unmanaged, and unstructured, integration doesn’t matter. Requirements management software will not solve our problems.

For CMMI Level 3 through CMMI Level 5 – Being able to quantify the performance of our process, and improve our process based on that feedback both require an element of instrumentation and insight into our techniques and tools. Attempting to do that meaningfully without additional structure and traceability will provide limited benefit. If we’re investing this level of effort in our requirements process, we should extend it to include our overal software development process by integrating to other systems.

From a CMMI Level Perspective

The previous analysis basically looked at the “RMM level 5″ column in our grid, and inspected the relative decisions for each “CMMI level” row. Now we will look at it by reviewing the CMMI levels, and seeing how they map to the RMM level.

A quick review of the same chart (so you don’t have to scroll up and down):

(larger version)

At CMMI level 1 through CMMI level 3, we don’t address integration with other systems. We want to focus on getting a cohesive and powerful requirements management process in place before we look at integrating it to our other processes.

At CMMI levels 4 and 5 we are measuring and improving on our process. We require traceability as a key component to our quantified analysis and instrumentation. We should be extending that traceability beyond the requirements realm to maximize the value of what we are doing.

Summary

• RMM level 5 specifies that requirements documents are organized, structured, and traceable – both to each other and to other systems.
• Once we reach a company-standard process (CMMI level 3) we should be at RMM level 5, but we must have reached RMM level 2.
• RMM level 5 is never required to achieve any of the CMMI levels – but it is valuable and encouraged.

Don’t forget to take our One Minute Survey on CMMI and RMM Levels.

Background

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure. In this article, we will look at the definition of RMM level 4. W also look at the mapping from RMM level 4 to various CMMI levels.

(larger version)

In the previous article in the series, we looked at how RMM level 3 – structured requirements processes map to CMMI levels.

RMM Level 4 – Traced Requirements

RMM level 4 builds upon the previous three levels – structured, organized, and documented requirements. With an organization of structured requirements, we can overlay the notion of tracing between requirements.

Consider the structured requirements approach we adapted from Karl Wiegers.

In this structured approach, there is a notion of dependence.

• Goals depend upon use cases in order to be achieved.
• Goals depend upon non-functional requirements to define their achievability.
• Use cases depend upon functional requirements to enable their execution.
• Use cases depend upon non-functional requirements to characterize their effectiveness.
• Functional requirements depend upon designs, which depend upon implementation.

This structure of dependency represents the real-world reliance of one artifact on another. In an ongoing software development project, we can be making changes to any of these elements. Those changes can impact other elements.

As an example, we could change a use case. The goal that depends upon that use case might be affected. Our changes may affect the functional and non-functional requirements upon which the use case depends.

Traceability allows us to say this use case relies on those requirements. It represents relationships between specific artifacts. We can use traceability to reduce the effort (and errors) associated with propogating changes through the dependency network.

We can also use traceability to enable interesting aggregations of reporting information. For example, we could identify the percentage of completion of a given use case – by looking at the percentage completion of all implementation elements that support all design elements upon which the use case depends. Other analogous relationships can be created to meet other reporting objectives.

We can also use traceability to validate completeness (IBM uses the word “coverage” in their article) of our specification. We can review a goal, and ask the question: “Are all of the use cases required to achieve this goal defined?” We can also validate in the other direction: “Are all of these use cases required to achieve that goal?” We covered this specific example in our article, Completeness Validation With Use Cases. This also applies to the completeness validation of other artifacts in the requirements hierarchy.

Mapping CMMI Levels to RMM Level 4

In our diagram, we show the following mappings for RMM level 4:

• CMMI level 0 – No Entry
• CMMI level 1 – No Entry
• CMMI level 2 – Requirements Should Be Traced
• CMMI level 3 – Requirements Should Be Traced
• CMMI level 4 – Requirements Must Be Traced
• CMMI level 5 – Requirements Must Be Traced

For CMMI Level 0 and CMMI Level 1 – when our process is unmanaged, and unstructured, traceability does not provide value – it creates confusion.
For CMMI Level 2 and CMMI Level 3 - A valuable process must include organization of documented of requirements. Those documents should also be structured and traced.
For CMMI Level 4 and CMMI Level 5 – Being able to quantify the performance of our process, and improve our process based on that feedback both require an element of instrumentation and insight into our techniques and tools. Attempting to do that meaningfully without additional structure and traceability will provide limited benefit.

From a CMMI Level Perspective

The previous analysis basically looked at the “RMM level 4″ column in our grid, and inspected the relative decisions for each “CMMI level” row. Now we will look at it by reviewing the CMMI levels, and seeing how they map to the RMM level.

A quick review of the same chart (so you don’t have to scroll up and down):

(larger version)

At CMMI level 1, we don’t address traceability We would focus on reaching CMMI level 2 before reaching RMM level 4.

At CMMI level 2 and CMMI level 3, we require that the documentation be organized. A managed process without some form of organization and consistent documentation is a poorly managed process. We also suggest that an RMM level of at least 3, and ideally 4 be adopted.

At CMMI levels 4 and 5 we are measuring and improving on our process. We require traceability as a key component to our quantified analysis and instrumentation.

Summary

• RMM level 4 specifies that requirements documents are organized, structured, and traceable.
• CMMI level 2 specifies that there is a managed process – in our case, one for managing requirements, and it should involve structure and traceability as components that simplify that management.
• A process must be at RMM level 4 before it can reach CMMI level 4.

Check out the next article, CMMI Levels and RMM Level 5 or  take our One Minute Survey on CMMI and RMM Levels.

Background

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure. In this article, we will look at the definition of RMM level 3. We also question the language used and reinterpret some of what IBM suggests. Finally, we look at the mapping from RMM level 3 to various CMMI levels.

(larger version)

In the previous article in the series, we looked at how RMM level 2 – organized requirements documents processes map to CMMI levels.

RMM Level 3 – Structured Requirements

RMM level 1 requires us to document our requirements. RMM level 2 requires us to organize that documentation and use consistent formatting for the documents. RMM level 3 introduces the concept of using structured requirements, as well as the idea of requirements having attributes. The first notion relates to the relationships between requirements, and the second is a way of apply structure to the requirements so that we can reason about them more effectively.

Structured Requirements

The first thing that the IBM team identifies is the need to identify different types of requirements. Avoid all of the naming bugaboos, and consider the notion of identifying different structures of artifacts in the software development process. We have a series of elements of information that we need to understand and articulate in order to travel from an identified market need to a delivered software product.

There are many different approaches to documenting requirements. We all struggle to agree on particular naming conventions. We use different requirements documents to represent different parts of the flow.

In Alphabet Soup – Requirements Documents, we use the following diagram to try and summarize the stages of decomposition.

This is the first level of decomposition – requirements (MRD or PRD) versus specification (SRS or FRS).

There’s another level of detail in the structuring of requirements, built on the work that Karl Wiegers has done. It looks at the artifacts in more detail – and I believe is what the IBM team had in mind when they defined RMM level 3. Here’s the version of the diagram that we developed in our article on non-functional requirements, and then reference as part of our introduction to structured requirements. You can read more about this approach in those articles.

We’ve also done some exploration of how to marry interaction design with structured requirements. The approach of starting with a user-centric perspective has a lot of benefits, and we believe there is a way to combine those benefits with the benefits inherent in a structured approach to requirements documentation. Here’s the diagram we created that shows how we adapt our structured approach to an interaction design context.

Regardless of the approach you take, the element that is relevant to having an RMM level 3 requirements process is the notion that different documents represent different types of requirements/constraints/designs.

Requirement Structures

The IBM team also talks about having attributes as part of requirements. Unfortunately, this is a little bit of the “if you have a hammer, everything looks like a nail” syndrome. What their suggestion implies is

1. You have a notion of objects, and you use objects to represent requirements artifacts.
2. You apply the concept of attributes to structure the elements of information within those artifacts.
3. You have some means (human or machine) to reason about those attributes in a way that provides distinct value relative to reasoning about the artifacts.

Their approach is unfortunate, if only because it appears presumptive, and perhaps biased. I believe we can restructure their language into something design-agnostic that achieves the same objectives.

We propose that there are two relevant benefits that could be addressed with the attributes-approach they suggest:

1. Being able to manage and reason about the meta-data of a requirement artifact has value. Meta-data are pieces of data that describe the data. For example, who is the author of the document? When was it last edited? What is its priority? To which other requirements is it related? Being able to track, edit, and view this information allows us to make decisions about how and when to use the document. It helps us plan activities and investments that are looking at the process as a whole – combining information about all of the requirements to make high level decisions.
2. Structuring information within a requirement artifact has value. Artifacts can be free-form text. That text can be organized into sections and lists and tables. That type of organization is helpful to humans who read it. It allows us to organize our content so that it is easier to read the requirements. Most good business writing has these elements – where the organization of information is suited to the content and its intended use. To be at RMM level 3, we must also be at RMM level 2, which requires consistent formatting. Combining that consistency with structure makes it easier for people to read (and saves time when writing) requirements. There is also benefit to using a structure that can be read by machines as well as humans. When information has structure, it introduces the possibility of machine-reasoning, just as it improves human-consumption. While machine-reasoning about elements of requirements documents is not a criterion of achieving RMM level 3, the IBM article implies that this benefit exists. And it does exist. Without going off on a tangent, we can at least easily envision the generation of a report based upon the status of all requirements scheduled for a given release. This report can be created without formally structuring the information, but it is easier to create when we can reason about the structure of the information.

I think this is exactly what IBM intended, and they just used an unfortunately symbolic word – attributes. The same criticism has been applied to much of our writing about the way we use the word requirement.

Mapping CMMI Levels to RMM Level 3

In our diagram, we show the following mappings for RMM level 3:

• CMMI level 0 – No Entry
• CMMI level 1 – Requirements Should Be Structured
• CMMI level 2 – Requirements Should Be Structured
• CMMI level 3 – Requirements Should Be Structured
• CMMI level 4 – Requirements Must Be Structured
• CMMI level 5 – Requirements Must Be Structured

For CMMI Level 0 – when our process is so ad-hoc that documentation of requirements is questionable, discussions about how we organize and structure the requirements documents are irrelevant.

For CMMI Level 1 through CMMI Level 3 - A valuable process must include documentation of requirements. Those documents really should be organized and structured. Structure is essentially organization at the next level of detail, and it is worth doing.

For CMMI Level 4 and CMMI Level 5 – Being able to quantify the performance of our process, and improve our process based on that feedback both require an element of instrumentation and insight into our techniques and tools. Attempting to do that meaningfully without additional structure will provide limited benefit.

From a CMMI Level Perspective

The previous analysis basically looked at the “RMM level 3″ column in our grid, and inspected the relative decisions for each “CMMI level” row. Now we will look at it by reviewing the CMMI levels, and seeing how they map to the RMM level.

A quick review of the same chart (so you don’t have to scroll up and down):

(larger version)

At CMMI level 1, we require that requirements be written. We suggest that they be organized and structured.

At CMMI level 2 and CMMI level 3, we require that the documentation be organized. A managed process without some form of organization and consistent documentation is a poorly managed process. We also suggest that an RMM level of at least 3, and ideally 4 be adopted.

At CMMI levels 4 and 5 we are measuring and improving on our process. We’ll address the higher CMMI levels in more detail as this series of articles continues.

Summary

• RMM level 3 specifies that requirements documents are organized, and structured.
• CMMI level 2 specifies that there is a managed process – in our case, one for managing requirements.
• A process must be at RMM level 2 and should be at level 3 or 4 to be at CMMI level 2 or CMMI level 3.
• A process should be at RMM level 3 if it is at CMMI level 1.
• A process should be at RMM level 4 if it is at CMMI level 2.

Note that this implies that we would spend the extra effort to get to CMMI 3 before we would try and reach RMM level 5.

Check out the next article, CMMI Levels and RMM Level 4 or take our One Minute Survey on CMMI and RMM Levels.

Background

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure. In this article, we will look at the definition of RMM level 2. We also cover the tradeoffs and benefits of the practices it requires. Finally, we look at the mapping from RMM level 2 to various CMMI levels.

(larger version)

In the previous article in the series, we looked at how RMM level 1 – written requirements processes map to CMMI levels.

RMM Level 2 – Organized Requirements

RMM level 1 requires us to document our requirements, but doesn’t talk about how we document them. We can use emails, store information in databases, spreadsheets, and screenshots. But without an over-arching organization. In RMM level 2, we have to organize our requirements documents, we have to use consistent formatting, and we have to deal with administrative issues like security and version control.

The Case For Organizing Requirements

There are two main drivers for organizing our requirements:

• The need to consume those requirements.
• The need to change those requirements over time.

Consuming Requirements

Requirements are written not just to organize our thoughts, but to provide direction for the team. They are a form of targeted communication. The set the scope for software delivery, provide guidance in making prioritization decisions, and provide insight into what we will deliver – helping manage the expecations of our customers.

Organizing our requirements makes it easier to consume them. When we ask people to review our requirements, they will have more confidence, and experience less frustration, if they are consistently looking for documents in the same location.

This location could be a document repository, a shared drive on the network, a website, a portal site, or even organized in a file cabinet (?!). The point is that they are always in the same place. As the quantity of our requirements grows, they should also be organized in a logical way within that location. At RMM level 2, any organization is valid – as long as it is consistent, it will provide value.

Changing Requirements Over Time

While documenting the requirements provides benefit, the disorganization comes at a cost. Requirements change over time. Our requirements documentation should change over time as well.

The biggest complaint with waterfall projects is that our understanding of the requirements does not change over time. Requirements are a moving target. With a waterfall project, we define a set of requirements, and then kick off the project – sort of a fire and forget model. Months or years later, we deliver a product – and it will probably match our documented requirements. While we were happily developing against the requirements we documented, the actual requirements have changed. There’s a very high risk that what we deliver will not meet the evolved needs.

• The Standish group reports that over 80% of projects are unsuccessful
  either because they are over budget, late, missing function, or a
  combination. (http://www.standishgroup.com/sample_research/chaos_1994_1.php)
• 53 percent of projects cost 189 percent of their original estimate.
• Average schedule overruns may be as high as 100%
• Over 40% to 60% of defects in software are caused by poor requirements
  definition.
• About One-Quarter of all projects are cancelled.
• Customers do not use 20% of their product features.

Why We Should Invest in Requirements Management

While poorly documented requirements are certainly a factor in the statistics above, another factor is no-longer-relevant requirements. These likely play out as features that are missing, features that are not used, and project cancellation (due to lack of relevance, or lack of ROI).

When we don’t organize our requirements, then changing them becomes more expensive – we have to find them, modify them, and notify people that they’ve been changed. It also becomes difficult to know if this document is the latest document. Organization addresses this problem.

Why Avoid Organization?

Organization does come at a cost. We have to spend the time (and possibly money) to set up the repository. We have to spend time to determine how we want to organize our requirements. And we have to spend some time putting the documents into our organized repository.

We identified the benefits of getting data from an organized location. What if we don’t do that very often? If our requirements approvers never have to review the documents, then they don’t benefit from the effort we spent organizing the documents. Perhaps we just have a meeting where get verbal approvals, or route them all with an email (Microsoft Outlook lets you put voting buttons on emails) for approval.

If we’re using a waterfall style project where we document the requirements once, and never change them, then each person on the implementation team can just print out a copy and refer to it when they need it. Again, no benefit from organization.

We all recognize the costs of both of these approaches, but they do avoid a little bit of busy work. It’s possible, however unwise, that some teams will take this approach, and thereby not benefit from organization. Those teams might operate best at RMM level 1.

The Case For Consistently Formatting Requirements

By using consistent formatting, we make it easier for someone to read multiple requirements. They can more easily compare and contrast the documented requirements. They don’t have to spend cycles re-learning how to read each requirements document. Once they become familiar with the format, they can ignore it, and spend time on the content of the document.

When we talk about consistent requirements, we are generally talking about the logical consistency of the statements within and across requirements – but the consistency of formatting also has value. This formatting consistency is what RMM level 2 requires.

Avoiding Consistency

We save some time in training by not requiring people to write consistently. However, the time we save is probably completely absorbed by the time people spend thinking about how to structure the requirements while writing them. And we lose the benefits that come from reading requirements that use a consistent format.

Requirements Administration

The final element identified in RMM level 2 is the administrative perspective. A focus on security, access, and version control is what the IBM team identifies as the relevant administrative issues.

Security and access are identified as elements that engender trust in the documentation. We may be too agile or too trusting, but we don’t see those factors as being particularly relevant to trust. They are certainly valuable when it comes to protecting against unwanted distribution of the information – but we are not generally concerned with people modifying the documents in unacceptable ways. We’ll grudgingly admit that it is possible that a developer will open a requirements document and delete a requirement that he feels is inappropriate, or rewrite it so that it matches his implementation. We just don’t think that it is a practical concern.

Version control, however, is very important. The biggest trust issue we have is in being able to trust that we are reviewing the latest version of a requirements document. Version control provides us with that benefit. It also allows us to undo any untoward modifications of the document. At a minimum, version control should consist of the persistence of previous versions of files. This can be handled by using unique names for each version of the file, by storing copies of the file on a regular basis as backups, or by using version control.

Subversion is the best version control system (VCS) we know of. If implementing a new VCS, we suggest using subversion. It is open source, easy to administer, and best-of-breed.

Mapping CMMI Levels to RMM Level 2

In our diagram, we show the following mappings for RMM Level 2:

• CMMI level 0 – No Entry
• CMMI level 1 – Requirements Should Be Organized
• CMMI level 2 – Requirements Must Be Organized
• CMMI level 3 – Requirements Must Be Organized
• CMMI level 4 – Requirements Must Be Organized
• CMMI level 5 – Requirements Must Be Organized

For CMMI Level 0 – when our process is so ad-hoc that documentation of requirements is questionable, discussions about how we organize the requirements documents are irrelevant. We’re talking about icing and candles when we don’t even know if we have a cake.

For CMMI Level 1 – A valuable process must include documentation of requirements. Those documents really should be organized. The benefits of versioning alone should make this an easy decision. Placing the documents in known locations, and having them be written in a consistent format is valuable too.

For CMMI Level 2 and higher – When we talk about a managed process, we are talking about bringing order to the chaos. Centralizing the requirements in a repository, versioning the documents, and using consistent formatting all bring order.

Imagine a managed requirements process that does everything with the exception of applying consistent formatting to our documents. Perhaps we have various authors of our requirements documents, and they write inconsistently. There’s value in doing all of this, but it would be CMMI level 2, RMM level 1. Only with all three elements (consistent location, consistent formatting, and versioned documents) would the process be both CMMI level 2 and RMM level 2.

We would definitely focus on moving from RMM level 1 to RMM level 2 before we would try and standardize our process across our company. That standardization would be the move from CMMI level 2 to CMMI level 3. Based on that perspective, we believe that an RMM level 2 process rating is a mandatory element of all CMMI levels above CMMI level 1.

From a CMMI Level Perspective

The previous analysis basically looked at the “RMM level 2″ column in our grid, and inspected the relative decisions for each “CMMI level” row. Now we will look at it by reviewing the CMMI levels, and seeing how they map to the RMM level.

A quick review of the same chart (so you don’t have to scroll up and down):

(larger version)

At CMMI level 1, we require that requirements be written. We suggest that they be organized and structured.

At CMMI level 2, we require that the documentation be organized. A managed process without some form of organization and consistent documentation is a poorly managed process.

At CMMI level 3, we are standardizing our approach across our company. And at CMMI levels 4 and 5 we are measuring and improving on our process. We’ll address the higher CMMI levels in more detail as this series of articles continues.

Summary

• RMM level 2 specifies that requirements documents are organized.
• CMMI level 2 specifies that there is a managed process – in our case, one for managing requirements.
• A process must be at RMM level 2 to be at CMMI level 2.
• A process should be at RMM level 3 if it is at CMMI level 1.

Note that this implies that we would spend the extra effort to get to CMMI 3 before we would try and reach RMM level 5.

Check out the next article, CMMI Levels and RMM Level 3 or  take our One Minute Survey on CMMI and RMM Levels.

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure. In this article, we will look at the definition of RMM level 1. We also cover the tradeoffs and benefits of the practices it requires. Finally, we look at the mapping from RMM level 1 to various CMMI levels.

Background

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure.

(larger version)

RMM Level 1 – Written Requirements

Level 1 of the requirements maturity model is defined at a high level as simply having written requirements. IBM defines written requirements as persistent documentation. They point out that post-it notes and whiteboards don’t count. Email discussions, word documents, spreadsheets and presentations all count.

IBM presents an argument of tradeoffs – as long as the cost of documenting requirements is exceeded by the benefits, it makes sense to write the requirements. They point out three benefits of having written requirements:

1. A contract is explicit, or implicit in the requirements. The documented requirements can be used to manage the customer’s expectations, and can also be used to validate that what was promised was delivered.
2. Clear direction for the implementation team can be found in the requirements documents.
3. New team members can rely on the documented requirements as a means to get up to speed.

While we strongly agree with the first two points – we think the third one is a bit of a stretch. While having requirements documentation does help people get up to speed, it isn’t a first-order benefit. Videotape a couple 1-hr presentations. One presentation discussing the goals of the project, and one discussing (and whiteboarding) the architectural approach of the solution. Put these on the server and let new people watch them. Much more cost-effective at helping people get up to speed. [Note - I'm pretty sure that I heard Alistair Cockburn suggest this approach or something like it in a podcast interview, so the credit for the idea is his, not ours.]

We would also add that documenting requirements is all but worthless if we don’t use the documents as tools to support conversation with our team members. Incremental delivery is a process that is dependent upon feedback. We must get feedback from stakeholders, and from the implementation team.

Stakeholders will verify the correctness and validate the completeness of the requirements.

The implementation team will provide feedback about the clarity, verifiability, and feasibility of the requirements as written.
Requirements need to be written to support verification. The QA team and stakeholders are responsible for verifying that what was delivered is what was expected. Technically, the delivery must match the requirements – but the requirements should match the expectations of the customer.

One Step Above Chaos

I like that the IBM guys name level zero as “Chaos.” I’ve worked as a developer on projects without requirements. It is chaos. There’s a reason we write requirements. They set expectations. And theres a reason why we review and approve the requirements. It’s essentially a form of structured active listening.

Mapping to the CMMI Levels

In our diagram, we show the following mappings for RMM Level 1:

• CMMI level 0 – Requirements Should Be Written
• CMMI level 1 – Requirements Must Be Written
• CMMI level 2 – Requirements Must Be Written
• CMMI level 3 – Requirements Must Be Written
• CMMI level 4 – Requirements Must Be Written
• CMMI level 5 – Requirements Must Be Written

For CMMI level 0 – even if we don’t have a formal process, we really should be writing our requirements – and using those documents to manage expectations, provide feedback (that we’re doing the right stuff), and scope and focus our efforts.

For CMMI levels 1 and higher – all of the measured CMMI levels require that we have a defined process. Even with the disorganization of a team operating at CMMI level 1, we still need to have a process defined. And a requirements management process that doesn’t involve documenting the requirements isn’t worth very much at all.

Note that documentation might be in the form of prototypes, wireframes, and JAD Session notes. No one is saying that they have to be documented in any particular way. In fact, at RMM level 1, they aren’t in a consistent format, and don’t use a structured requirements framework. Consistent formatting is an element of RMM level 2. And RMM level 3 is focused on structured requirements.

The requirements documents may be scattered through a series of email debates, collaborative databases, and files on network share drives. That’s fine for RMM level 1 – in fact, it is part of the definition of RMM level 1. Organized requirements are a characteristic of RMM level 2.

Remember – CMMI Levels only represent how a process is implemented – they don’t characterize the effectiveness of any one process.

From a CMMI Level Perspective

The previous analysis basically looked at the “RMM level 1″ column in our grid, and inspected the relative decisions for each “CMMI level” row. Now we will look at it by reviewing the CMMI levels, and seeing how they map to the RMM level.

A quick review of the same chart (so you don’t have to scroll up and down):

(larger version)

At CMMI level 1, we require that requirements be written. We suggest that they be organized and structured.

At CMMI level 2, we require that the documentation be organized. A managed process without some form of organization and consistent documentation is a poorly managed process.

At CMMI level 3, we are standardizing our approach across our company. And at CMMI levels 4 and 5 we are measuring and improving on our process. We’ll address the higher CMMI levels in more detail as this series of articles continues.

Summary

• RMM level 1 specifies that requirements are documented.
• CMMI .evel 1 specifies that there is a process – in our case, one for managing requirements.
• A process must be at RMM level 1 to be at CMMI level 1.
• A process should be at RMM level 2 or 3 if it is at CMMI level 1.

Note that this implies that we would spend the extra effort to get to CMMI 2 before we would try and reach RMM level 4.

Check out the next article, CMMI Levels and RMM Level 2 or  take our One Minute Survey on CMMI and RMM Levels.

Welcome Readers of the Carnival of Enterprise Architecture! We hope you enjoy this series of articles!

CMMI (Capability Maturity Model Integration) is a description of the level of enlightenment of a process. It is essentially a measure of the quality and capability of a process. There are five categories, into one of which every process will fall. IBM took a similar approach to defining the requirements management process. In this series of posts, we will marry the two frameworks.

Background on CMMI Levels

We wrote an introduction to CMMI levels last March. In our article, we identified that there are five CMMI levels. Technically, there are six CMMI levels, when you include level zero. Level 0 is “undefined” by the CMMI, and represents an ad hoc process, or a lack of process.

CMMI Levels

• CMMI Level 0. Undefined. No real process.
• CMMI Level 1. Performed. A process is defined, but disorganized.
• CMMI Level 2. Managed. A defined process is managed.
• CMMI Level 3. Defined. A managed process is standardized across the company.
• CMMI Level 4. Quantitatively Measured. The performance of a standardized process is measured.
• CMMI Level 5. Optimizing. Performance measurement is used as a feedback loop to improve the process.

Take CMMI Levels With A Grain of Salt

Just knowing the CMMI Level of a process is not enough to know if the process is any good. By the same token, choosing a particular CMMI level, and meeting the technical requirements of that level are not enough to assure a good process.

Backgroundon RMM Levels

The folks at IBM wrote an article in 2003, where they defined five levels of maturity for requirements management processes. All five of the requirements management maturity (RMM) levels all build on the previous level, with increasing capability.

• RMM Level 0. Chaos. No persistent documentation of requirements.
• RMM Level 1. Written Requirements. Writing requirements documents (not emails and whiteboards).
• RMM Level 2. Organized Requirements. Colocation, versioning, consistent formatting.
• RMM Level 3. Structured Requirements. Defining types of requirements and their relationships.
• RMM Level 4. Traced Requirements. Explicitly mapping the support-network of requirements.
• RMM Level 5. Integrated Requirements. Integrating with the development environment and change management.

What IBM Didn’t Do

They didn’t map their framework back into the CMMI framework (known as CMM at the time) except for the following comment in the introduction of their article:

Those familiar with the CMM (Capability Maturity Model) from the Software Engineering Institute (SEI) will note some similarities to our parallel model, which has no direct relationship to the CMM save one: Achieving Level Five of the RMM will assuredly help an organization get to at least Level Three of the CMM.

IBM put together a great framework for describing elements of increasingly capable requirements management processes.

That is what the SEI tried to do when they developed the CMMI. Why couldn’t the IBM team just map their framework into the CMMI framework?

The problem is there is a mismatch between the two frameworks.

• The RMM framework describes steps and elements of a requirements management process. Each step adds a level of capability to the process. It might be more aptly named the requirements management capability framework.
• The CMMI framework describes the strategic capabilities (maturity) of how a process is applied, without assessing the tactical capabilities of the process itself.

The SEI recognized that the analysis of the tactical capabilities of any process would be different for every process, and left it to others to perform that work. This is almost what the IBM team did. We’re going to take a crack at it here.

Mapping RMM Levels to CMMI Levels

This is the first in a series of articles that will present a mapping of RMM levels to CMMI levels. We like using CMMI as a means to evaluate our internal processes, notwithstanding the challenges we mentioned earlier. We also like the framework that IBM presented for describing requirements management processes.

Shoot First, Ask Questions Later

There’s a lot more to write about this than we can put into a single article. We’re going to tackle this as a series. Even so, we put together an initial draft of how we think this will ultimately work out. We’ll share that here now. But we reserve the right to fix it when we find problems as we (and you!) put more effort into it.

(larger version)

Articles In This Series

• Foundation Series: CMMI Levels Explained
• What CMMI Level Should We Use?
• CMMI Levels and RMM Introduction
• CMMI Levels and RMM Level 1 – Written Requirements
• CMMI Levels and RMM Level 2 – Organized Requirements
• CMMI Levels and RMM Level 3 – Structured Requirements
• CMMI Levels and RMM Level 4 – Traced Requirements
• CMMI Levels and RMM Level 5 – Integrated Requirements
• Don’t forget to take our One Minute Survey on CMMI and RMM.

Code Debt is the debt we incur when we write sloppy code. We might do this to rush something out the door, with the plan to refactor later. Agile methodologies focus on delivering functionality quickly. They also invoke a mantra of refactoring – “make it better next release.” This can create pressure to “get it done” that overwhelms the objective of “get it done right.” Taking on code debt like this is about as smart as using one credit card to pay off another one.

Using Timeboxes to Visualize Pressure

In our timeboxing tutorial, we introduced a framework for managing releases and the amount of work we do within each release. We can apply the same framework to visualize the pressure that makes us consider borrowing against our future to take on a code debt.
Here’s a quick summary of the concepts from that article.

Each unit of functionality that we deliver in a product or release is made up of two elements – the time it takes to make it, and the time it takes to make it right. We can think of this as function and quality respectively. We;ve drawn them as puzzle pieces to show that they are (or at least should be) locked together. When a developer gives an estimate, it should be for the combination of function and quality – not quality alone.

A timebox represents the amount of time and money we allocate for a given release. It basically determines the size of the box that we can fill up with our units.

In our article, we talked about the options we have to try and get more “stuff” delivered in a single timebox. One obviously bad option is to decouple the quality from the functionality in order to deliver more functionality.

Several people essentially said “No one would ever plan on bad quality – that’s a bad suggestion!”

We agree – it is a bad idea. We were merely pointing out that it is something people consider. Especially managers who’ve never read The Tipping Point, and don’t know about “broken windows.” “Broken windows” in this case is a reference to the downward pressure that is applied to all future development efforts by forcing developers to work on a code base that has a lot of “low quality code.” The idea is that if we skip the quality part enough times, we will eventually stop caring at all.

We also agree that rational people won’t make a habit out of using this approach. However, there is another way we could find ourselves in this situation.

What if our estimates were wrong? In the middle of the timebox, we suddenly find ourselves without enough time / people to finish.

In the highlighted region of the diagram, we can see that the feature on the left took longer than we expected – and it pushed out the red/orange feature. There isn’t enough room in our timebox to get it all done.

Guilty

We’ve even suggested that maybe the right thing to do is to take a loan against ourselves and incur a code debt.

There are times when incurring a temporary code-debt is pragmatically more useful than delaying a release or a feature in order to polish the code.

Software Product Delivery – 20 Rules?

I’ve also argued against code-debt as a long term strategy, in the comments on another post.

I definitely agree that code-debt builds over time and gets more and more expensive. Plus there’s the risk of the whole ‘broken windows’ problem that Malcolm Gladwell outlines in The Tipping Point. I visualize it like the graph for a cascading diode – you can build up more and more code-debt (or design-debt, as you describe in the link), with its associated performance penalty, until you reach the ‘critical voltage’ and flood the diode. At this point, it becomes impractical to refactor – rewriting becomes more cost effective.

The Agile Dragon

So, we’ve stated that it might be a pragmatically good idea in the short run, and definitely a bad idea in the long run. But how do we crystalize the message that it is risky? With another good analogy.

Another Good Analogy

We owe our thanks to Mishkin for this extension of the “code debt” analogy that brings perfect clarity to the issue.

Last night I was thinking more about the analogy of technical debt. In this analogy, design and quality flaws in a team’s work become a “debt” that must eventually be paid back.

[...]

In other words, every time someone asks a team to let quality slide, they are asking the team (and the organization) to take on debt with an unknown interest rate. Which is lunacy.

Technical Debt, Mishkin Berteig

Thanks Mishkin!

Conclusion

Technical debt is very risky. Maybe it is the right call in the short run (but assume it isn’t). It is never the right call in the long run.

Johanna Rothman recently wrote an article with a poignant introduction: “A project team focuses on an interim milestone, works like the devil to meet that milestone. They meet the milestone, look up, and realize they’re not at the end of the project–they still have to finish the darn thing. They’re living the Crossing the Desert syndrome.” Fixing it isn’t enough – how do we prevent it from happening?

Unrealistic Scheduling

Johanna points out that when a team has to put in overtime (what we might call heroic efforts) to achieve an interim milestone in a project, the remaining project schedule is suspect. Johanna reccommends revisiting the remaining schedule, and we agree.

Recovery

Johanna also highlights the way to recover – give the people on the project a break. Have them move back to 40 hour weeks if they were working overtime. Force them to take the weekends off if they weren’t already. In short, restore a rational schedule. If people worked through vacations or holidays, require them to take the time off.

Its been said that it takes two weeks of down time to recover from work-burnout. Extending the desert analogy that Johanna credits to Jack Nevison of Oak Associates, we’ve reached the oasis in the middle of the desert, and we need to stay there long enough to rest and rehydrate.

Unfortunately, we’re still in the middle of the desert, and the rush to reach the oasis presumably had merit. If a project schedule is so intense that we have created the need for recovery in the middle of the project, the likelihood of having two weeks to recover is low. Whatever drove us to push to reach the oasis is still driving us, so resting for too long will only make it worse. And we’re still in the middle of the desert. We need to focus on prevention.

Prevention

We can’t always prevent the impetus to work long hours on a project. We can manage our execution in a way that reduces the chances of feeling like we’re crossing a desert.

• Improved estimation of tasks. Entire books are written about this topic, we won’t try and summarize ways to do this in a single blog post.
• Realistic effort allocation. When scheduling how many hours a day a developer can be “on task”, our experience has been that 5 to 6 hours is the maximum (when working full time or a little more). For requirements work, this might even be a little bit aggressive.
• Writing verifiable requirements. We need requirements that specify (or at least allow us to identify) when we’re done. Scope creep isn’t always implementing more features, it can be over-implementing features. With test-driven development (TDD) processes, the tests are written first (they fail), and the developer writes code until the tests pass. Once the tests pass, the code is done. Doing this requires us to write testable requirements. Practically speaking, this may only be realistic when using a continuous integration approach to code development.
• Managing smaller work chunks. Timeboxing (more details) is very effective for controlling the size of iterations. Iterations are important not only for the feedback cycle, but because they reduce the size of the “desert patches.”
• Feedback into the estimation cycle. Timeboxes become even more effective when we keep track of our estimation accuracy, and refine those estimates on the fly to help in replanning future releases. This is a key step in the maturation of a process or company from a CMMI perspective.
• Better communication of release content. Planning releases based on use cases / use case versions is a great way to target communication for external (to the team) stakeholders. It can also be really effective for helping people avoid the desert-crossing syndrome. Essentially you’re providing a map (“The next watering hole is just over that sand dune”) that helps keep efforts in context. One reason the desert image is so powerful is that we imagine being lost in a sea of sand – the hopelessness is a function of both the reality and the perception. Better communication helps address perceptions, while other elements help with the reality.

Conclusion

Overworking the team is bad. Burning them out in the middle of the project is worse. Prevention is the solution, through iterative development, better communication, and improved estimation/planning skills.

Rishikesh Tembe shared twenty rules for software product delivery last month. His rules are from the perspective of a former software developer. Some we like. Some, not so much.

We Like

Rishikesh has a short post with 20 rules. Among those rules are some concepts that we feel like expanding upon:

8. Do the riskiest part of the project first.

This is a great idea (for developers and project managers) for a couple reasons. Risk may be a function of technical feasibility or customer accceptance/value. Risk may even be an artifact of how we go about developing our product – for example, offshoring introduces risks. Risk is generally a nebulous, but bad thing to have in a project. If a problem is known, a solution can be identified, addressed, and implemented.

Risks represent the unknown unknowns, or those problems that we don’t understand well enough to address them. Big risks can be very scary – they jeopardize the ROI of our project. Lack of user adoption can affect the expected value to our customers, which will directly or indirectly affect our bottom line.

Note: This is not meant to imply that prioritization of requirements by release should be subservient to risk-mitigation. The most important stuff should always be done first. Within a particular release, address the riskiest issues first. Prioritize first. Mitigate second.

10. Make sure you’re in total control of your toolset and improve it systematically

This alludes to the general benefits of operating at a higher CMMI level. As a CMMI level five organization, we would quantitatively measure and improve not only our tools but our processes.

16. Build regression testing into the build process.

We’ve talked about continuous integration in the past. In our opinion, no other approach should be used to develop software.

Not So Much

11. Do not take the clients’ deadlines literally – first accept the project, then renegotiate the deadline.

Clients have deadlines for reasons. They may be market-driven, or driven by internal politics or budget cycles. While it is possible that a deadline is arbitrary, it is usually associated with a compelling event of some sort. Any discussion of deadlines should be approached in the same way we manage scope creep as a relationship-building exercise.

Project constraints, such as budgets and deadlines, are things that should be addressed collaboratively. Our clients are our partners, not our opposition.

13. Document the interfaces perfectly, but don’t document code (see next point).
14. Be fanatical about the readability of code.

Absolutes are rarely rational end-points, although presenting goals as absolutes can be effective in motivating directional change in an organization (with no expectation of actually achieving the goal). More on that some other time.

Broken Windows, as Gladwell describes them, are absolutely detractors from any environment, and serve to degrade the performance of those who operate in the environment. Code readability, micro kitchens, flexible schedules. There are many soft ROI elements that make up the environment of a software developer. Readability of code, like readability of requirements, is important. A friend of mine used to joke that perl is a “write-only” language. Notwithstanding his joke, code needs to be readable. Even in perl.

Perl serves as a great example – elegance of code does not always coincide with syntactic simplicity. Avoiding commenting of the code is just a bad idea. People read code. Make it readable.

There are times when incurring a temporary code-debt is pragmatically more useful than delaying a release or a feature in order to polish the code.

Conclusion

There are some good ideas and some bad ones in the list. Most of them are thought provoking, regardless. We may eventually find a list of absolute rules we should all follow, and it would overlap with this list somewhat. But for now, we’re still looking.

“I believe the hard part of building software to be the specification, design, and testing of this conceptual construct,[...] If this is true, building software will always be hard. There is inherently no silver bullet.”
Frederick P. Brooks, Jr. Computer Magazine, Apr 1987

Hat Tip To Joel

Joel Spolsky wrote an article, Lego Programming, about how mainstream media gets it wrong. And he ended the article with an interesting quote:

None of them believed Frederick P. Brooks, in 1987: “Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any—no inventions that will do for software productivity, reliability, and simplicity what electronics, transistors, and large-scale integration did for computer hardware[...]”

Spolsky quoting Brooks

I used to have a department manager who regularly asked for silver bullets – not metaphorically, literally. So I just had to check out Mr. Brooks’ article.

Brooks in 1987

Any article about software that still rings true after almost 20 years is amazing. Brooks starts us off right away with his conjecture that the big problem of complexity in software will never be solved. He proposes that an order of magnitude improvement might be found, but that the problem will never go away. Never? We still have the problem today.

Brooks identifies elements of the essence of software that make it forever a were-wolf (no silver bullet).

• Complexity. Brooks cites the non-linear increase in complexity of software with size of software. Even with OO constructs and abstraction layers, this is still true today. With a decade of enterprise software experience, I will vouch for this, in terms of complexity versus “size of the software.” The recent thread on Vista’s shutdown options is another good example – not because there are so many ways to do it, but rather because the team spent over a year implementing it. As to complexity vs. “value of the software”, I’m not certain it is non-linear. Applications are getting larger (generally), but the amount of value they provide is growing much faster. Not cause-and-effect, but correlating. Businesses are re-engineering to utilize software in their processes. Those savings are large, and inferring from the increases in spending on software, are more valuable than in the past. Is achieving “the same value” easier today than twenty years ago? Yes. So much so that we don’t even try to do it – we try to do harder, more valuable stuff.
• Conformity. Brooks argues that there are no guiding principles that cause software engineers to conform (stylistically, representationally, etc). Therefore, they will spend some cycles dealing with “arbitrary complexity.” Fair point.
• Changeability. Software becomes complex because it can. Brooks points out that cars don’t change often, because it isn’t easy to change them. Software has no such barrier. If you’ve ever spent time on a working farm, and seen the myriad of wonderful things that can be accomplished with bailing wire, twine, and duct tape, you know that the impetus to change things is not limited to software. Without the natural barriers enjoyed by manufactured goods, software will be changeable, and therefore hard.
• Invisibility. Brooks argues that because of the representational power of software, we represent constructs that we inherently can not visualize. Some smart physicists at Nicholas Copernicus University agree, and spend considerable effort “simplifying” multi-dimensional data into a framework we can accept.

Brooks discusses several advances in software, and approaches that people hope will be a silver bullet. He posits that none of them have been or will be (as of 1987). We’re approaching two decades later, and none of them have. There are some prescient quotes in his article.

Effective Strategies

Brooks also discusses some effective strategies to change the essence of software complexity and find a silver bullet.

1. Buy Vs. Build

He proposes a very outside-the-box solution – don’t build the software, buy it. Buying customized software is not what Brooks proposes. That would be no more effective than Kanban was (shifting the overhead burden onto suppliers who subsequently raised prices). He proposes using non-customized software, and adapting business processes to those supported by the off-the-shelf software.

2. Requirements Refinement and Rapid Prototyping

His opening paragraph is a very strong statement – and one we agree with:

The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is as difficult as establishing the detailed technical requirements, including all the interfaces to people, to machines, and to other software systems. No other part of the work so cripples the resulting system if done wrong. No other part is more difficult to rectify later.

Brooks goes on to make a statement that seems eerily similar to one I’ve heard Kent Beck say:

I would go a step further and assert that it is really impossible for a client, even working with a software engineer, to specify completely, precisely, and correctly the exact requirements of a modern software product before trying some versions of the product.

From this perspective, Brooks promotes incremental development as the right approach. In 1987!

3. Great Designers

We’ve always said that people trump process. Brooks said it first.

Conclusion

Software is hard. Requirements are harder. Nothing (according to Brooks) will ever change that.

Enough of the debates about requirements and what we call them. Why don’t we just hire great developers and empower them to work directly with the customers?

Lost Garden

Danc, at Lost Garden has an article that asks just this question, and addresses the challenges of directly empowering developers.

A couple quotes to set the tone of their article:

What would happen if the developers possessed a deep understanding of their customers needs and desires? Suddenly, those thousand little decisions aren’t introducing random noise into the product. Instead, they are pushing the product forward.

And…

One philosophy is that we need better specs and those damned monkeys need to implement the specs exactly as designed. Better command and control system and more rigorous process is obviously the trick to success. This tends to generate poor results.

Getting Closer To The Customer

Danc offers these tips to help the developers get closer to their customers:

• Use your own product
• Onsite customers
• Observe customers using your product
• Hire psychologists and ethnologists to study your customers
• Listen to lead users
• Reduce feedback cycle time
• Improve objectivity of results
• Improve clarity of results

The only inconsistent point in his article seems to be the suggestion about using ethnologists. Don’t they present a barrier to developers in the same way that a product manager would by synthesizing and prioritizing “the voice of the customer?”

Freeing developers from a job where they are coding “to spec” will definitely empower the developers. I believe that having someone trained in prioritization and interpretation of needs (a product manager, for example) can help keep the team aligned on the important stuff by providing insight into what is important. This is the same argument for having “experts” interpret customer behaviors.

Check it out.