Please take one minute to answer the following two questions, because people need to know how their process maturity compares with everyone else. You’ll be answering two easy questions -

• What is your CMMI level?
• What is your RMM level?

Background

We just completed a series of six articles about CMMI levels and RMM levels. We discussed how the two frameworks can be connected, and explored the reasons for trying to reach the next level on either scale.

Question 1

[poll=2]

Question 2

[poll=3]

Thanks for taking the survey, and if you have any thoughts about the results, just comment here.

Background

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure. In this article, we will look at the definition of RMM level 5. We also look at the mapping from RMM level 5 to various CMMI levels.

(larger version)

In the previous article in the series, we looked at how RMM level 4 – traced requirements processes map to CMMI levels.

RMM Level 5 – Integrated Requirements

In RMM level 4 we discussed the benefits of establishing traceability within the scope of the requirements realm. This traceability helped us with tracking the impact of changes and validation of requirements, and it helped us with reporting and management functions that improved our insight and lessened our efforts.

RMM level 5 extends this traceability concept beyond the requirements realm. We can extend tracing into development, testing and documentation, as well as project management.

Delivery Scheduling

Each delivery of our software includes a set amount of functionality. We prefer using a timebox approach for planning those deliveries. We also suggest using use cases as the “pieces” of functionality that are delivered. We also have proposed an approach to managing changes to the delivery schedule. This overall process is a form of integration of requirements, and is dependent upon having organized, structured, traced requirements. It also requires us to create linkages between implementation elements/effort and the requirements that are supported.

Quality Assurance

There are two elements of integration between QA and requirements. The first is in defining test cases based on use cases. Each use case drives the creation of one or more test cases that are used to validate that the delivered software meets the requirements.

The second element is in tracking and management of bugs. We have a triage process for determining how to address bugs. Those bugs can come from anywhere – and sometimes they come from incorrect requirements. Establishing relationships between requirements and bugs allows us to manage the defect resolution process more effectively.

Documentation

When we write documentation, our goal is to have the reader know how to accomplish something that they want to do. When we develop software, we define the things that users will be able to do. It only makes sense to structure documentation around use cases.

Mapping CMMI Levels to RMM Level 5

In our diagram, we show the following mappings for RMM level 5:

• CMMI level 0 – No Entry
• CMMI level 1 – No Entry
• CMMI level 2 – No Entry
• CMMI level 3 – Requirements Should Be Integrated
• CMMI level 4 – Requirements Should Be Integrated
• CMMI level 5 – Requirements Should Be Integrated

For CMMI Level 0 through CMMI Level 2 – when our process is unmanaged, and unstructured, integration doesn’t matter. Requirements management software will not solve our problems.

For CMMI Level 3 through CMMI Level 5 – Being able to quantify the performance of our process, and improve our process based on that feedback both require an element of instrumentation and insight into our techniques and tools. Attempting to do that meaningfully without additional structure and traceability will provide limited benefit. If we’re investing this level of effort in our requirements process, we should extend it to include our overal software development process by integrating to other systems.

From a CMMI Level Perspective

The previous analysis basically looked at the “RMM level 5″ column in our grid, and inspected the relative decisions for each “CMMI level” row. Now we will look at it by reviewing the CMMI levels, and seeing how they map to the RMM level.

A quick review of the same chart (so you don’t have to scroll up and down):

(larger version)

At CMMI level 1 through CMMI level 3, we don’t address integration with other systems. We want to focus on getting a cohesive and powerful requirements management process in place before we look at integrating it to our other processes.

At CMMI levels 4 and 5 we are measuring and improving on our process. We require traceability as a key component to our quantified analysis and instrumentation. We should be extending that traceability beyond the requirements realm to maximize the value of what we are doing.

Summary

• RMM level 5 specifies that requirements documents are organized, structured, and traceable – both to each other and to other systems.
• Once we reach a company-standard process (CMMI level 3) we should be at RMM level 5, but we must have reached RMM level 2.
• RMM level 5 is never required to achieve any of the CMMI levels – but it is valuable and encouraged.

Don’t forget to take our One Minute Survey on CMMI and RMM Levels.

Background

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure. In this article, we will look at the definition of RMM level 4. W also look at the mapping from RMM level 4 to various CMMI levels.

(larger version)

In the previous article in the series, we looked at how RMM level 3 – structured requirements processes map to CMMI levels.

RMM Level 4 – Traced Requirements

RMM level 4 builds upon the previous three levels – structured, organized, and documented requirements. With an organization of structured requirements, we can overlay the notion of tracing between requirements.

Consider the structured requirements approach we adapted from Karl Wiegers.

In this structured approach, there is a notion of dependence.

• Goals depend upon use cases in order to be achieved.
• Goals depend upon non-functional requirements to define their achievability.
• Use cases depend upon functional requirements to enable their execution.
• Use cases depend upon non-functional requirements to characterize their effectiveness.
• Functional requirements depend upon designs, which depend upon implementation.

This structure of dependency represents the real-world reliance of one artifact on another. In an ongoing software development project, we can be making changes to any of these elements. Those changes can impact other elements.

As an example, we could change a use case. The goal that depends upon that use case might be affected. Our changes may affect the functional and non-functional requirements upon which the use case depends.

Traceability allows us to say this use case relies on those requirements. It represents relationships between specific artifacts. We can use traceability to reduce the effort (and errors) associated with propogating changes through the dependency network.

We can also use traceability to enable interesting aggregations of reporting information. For example, we could identify the percentage of completion of a given use case – by looking at the percentage completion of all implementation elements that support all design elements upon which the use case depends. Other analogous relationships can be created to meet other reporting objectives.

We can also use traceability to validate completeness (IBM uses the word “coverage” in their article) of our specification. We can review a goal, and ask the question: “Are all of the use cases required to achieve this goal defined?” We can also validate in the other direction: “Are all of these use cases required to achieve that goal?” We covered this specific example in our article, Completeness Validation With Use Cases. This also applies to the completeness validation of other artifacts in the requirements hierarchy.

Mapping CMMI Levels to RMM Level 4

In our diagram, we show the following mappings for RMM level 4:

• CMMI level 0 – No Entry
• CMMI level 1 – No Entry
• CMMI level 2 – Requirements Should Be Traced
• CMMI level 3 – Requirements Should Be Traced
• CMMI level 4 – Requirements Must Be Traced
• CMMI level 5 – Requirements Must Be Traced

For CMMI Level 0 and CMMI Level 1 – when our process is unmanaged, and unstructured, traceability does not provide value – it creates confusion.
For CMMI Level 2 and CMMI Level 3 - A valuable process must include organization of documented of requirements. Those documents should also be structured and traced.
For CMMI Level 4 and CMMI Level 5 – Being able to quantify the performance of our process, and improve our process based on that feedback both require an element of instrumentation and insight into our techniques and tools. Attempting to do that meaningfully without additional structure and traceability will provide limited benefit.

From a CMMI Level Perspective

The previous analysis basically looked at the “RMM level 4″ column in our grid, and inspected the relative decisions for each “CMMI level” row. Now we will look at it by reviewing the CMMI levels, and seeing how they map to the RMM level.

A quick review of the same chart (so you don’t have to scroll up and down):

(larger version)

At CMMI level 1, we don’t address traceability We would focus on reaching CMMI level 2 before reaching RMM level 4.

At CMMI level 2 and CMMI level 3, we require that the documentation be organized. A managed process without some form of organization and consistent documentation is a poorly managed process. We also suggest that an RMM level of at least 3, and ideally 4 be adopted.

At CMMI levels 4 and 5 we are measuring and improving on our process. We require traceability as a key component to our quantified analysis and instrumentation.

Summary

• RMM level 4 specifies that requirements documents are organized, structured, and traceable.
• CMMI level 2 specifies that there is a managed process – in our case, one for managing requirements, and it should involve structure and traceability as components that simplify that management.
• A process must be at RMM level 4 before it can reach CMMI level 4.

Check out the next article, CMMI Levels and RMM Level 5 or  take our One Minute Survey on CMMI and RMM Levels.

Background

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure. In this article, we will look at the definition of RMM level 3. We also question the language used and reinterpret some of what IBM suggests. Finally, we look at the mapping from RMM level 3 to various CMMI levels.

(larger version)

In the previous article in the series, we looked at how RMM level 2 – organized requirements documents processes map to CMMI levels.

RMM Level 3 – Structured Requirements

RMM level 1 requires us to document our requirements. RMM level 2 requires us to organize that documentation and use consistent formatting for the documents. RMM level 3 introduces the concept of using structured requirements, as well as the idea of requirements having attributes. The first notion relates to the relationships between requirements, and the second is a way of apply structure to the requirements so that we can reason about them more effectively.

Structured Requirements

The first thing that the IBM team identifies is the need to identify different types of requirements. Avoid all of the naming bugaboos, and consider the notion of identifying different structures of artifacts in the software development process. We have a series of elements of information that we need to understand and articulate in order to travel from an identified market need to a delivered software product.

There are many different approaches to documenting requirements. We all struggle to agree on particular naming conventions. We use different requirements documents to represent different parts of the flow.

In Alphabet Soup – Requirements Documents, we use the following diagram to try and summarize the stages of decomposition.

This is the first level of decomposition – requirements (MRD or PRD) versus specification (SRS or FRS).

There’s another level of detail in the structuring of requirements, built on the work that Karl Wiegers has done. It looks at the artifacts in more detail – and I believe is what the IBM team had in mind when they defined RMM level 3. Here’s the version of the diagram that we developed in our article on non-functional requirements, and then reference as part of our introduction to structured requirements. You can read more about this approach in those articles.

We’ve also done some exploration of how to marry interaction design with structured requirements. The approach of starting with a user-centric perspective has a lot of benefits, and we believe there is a way to combine those benefits with the benefits inherent in a structured approach to requirements documentation. Here’s the diagram we created that shows how we adapt our structured approach to an interaction design context.

Regardless of the approach you take, the element that is relevant to having an RMM level 3 requirements process is the notion that different documents represent different types of requirements/constraints/designs.

Requirement Structures

The IBM team also talks about having attributes as part of requirements. Unfortunately, this is a little bit of the “if you have a hammer, everything looks like a nail” syndrome. What their suggestion implies is

1. You have a notion of objects, and you use objects to represent requirements artifacts.
2. You apply the concept of attributes to structure the elements of information within those artifacts.
3. You have some means (human or machine) to reason about those attributes in a way that provides distinct value relative to reasoning about the artifacts.

Their approach is unfortunate, if only because it appears presumptive, and perhaps biased. I believe we can restructure their language into something design-agnostic that achieves the same objectives.

We propose that there are two relevant benefits that could be addressed with the attributes-approach they suggest:

1. Being able to manage and reason about the meta-data of a requirement artifact has value. Meta-data are pieces of data that describe the data. For example, who is the author of the document? When was it last edited? What is its priority? To which other requirements is it related? Being able to track, edit, and view this information allows us to make decisions about how and when to use the document. It helps us plan activities and investments that are looking at the process as a whole – combining information about all of the requirements to make high level decisions.
2. Structuring information within a requirement artifact has value. Artifacts can be free-form text. That text can be organized into sections and lists and tables. That type of organization is helpful to humans who read it. It allows us to organize our content so that it is easier to read the requirements. Most good business writing has these elements – where the organization of information is suited to the content and its intended use. To be at RMM level 3, we must also be at RMM level 2, which requires consistent formatting. Combining that consistency with structure makes it easier for people to read (and saves time when writing) requirements. There is also benefit to using a structure that can be read by machines as well as humans. When information has structure, it introduces the possibility of machine-reasoning, just as it improves human-consumption. While machine-reasoning about elements of requirements documents is not a criterion of achieving RMM level 3, the IBM article implies that this benefit exists. And it does exist. Without going off on a tangent, we can at least easily envision the generation of a report based upon the status of all requirements scheduled for a given release. This report can be created without formally structuring the information, but it is easier to create when we can reason about the structure of the information.

I think this is exactly what IBM intended, and they just used an unfortunately symbolic word – attributes. The same criticism has been applied to much of our writing about the way we use the word requirement.

Mapping CMMI Levels to RMM Level 3

In our diagram, we show the following mappings for RMM level 3:

• CMMI level 0 – No Entry
• CMMI level 1 – Requirements Should Be Structured
• CMMI level 2 – Requirements Should Be Structured
• CMMI level 3 – Requirements Should Be Structured
• CMMI level 4 – Requirements Must Be Structured
• CMMI level 5 – Requirements Must Be Structured

For CMMI Level 0 – when our process is so ad-hoc that documentation of requirements is questionable, discussions about how we organize and structure the requirements documents are irrelevant.

For CMMI Level 1 through CMMI Level 3 - A valuable process must include documentation of requirements. Those documents really should be organized and structured. Structure is essentially organization at the next level of detail, and it is worth doing.

For CMMI Level 4 and CMMI Level 5 – Being able to quantify the performance of our process, and improve our process based on that feedback both require an element of instrumentation and insight into our techniques and tools. Attempting to do that meaningfully without additional structure will provide limited benefit.

From a CMMI Level Perspective

The previous analysis basically looked at the “RMM level 3″ column in our grid, and inspected the relative decisions for each “CMMI level” row. Now we will look at it by reviewing the CMMI levels, and seeing how they map to the RMM level.

A quick review of the same chart (so you don’t have to scroll up and down):

(larger version)

At CMMI level 1, we require that requirements be written. We suggest that they be organized and structured.

At CMMI level 2 and CMMI level 3, we require that the documentation be organized. A managed process without some form of organization and consistent documentation is a poorly managed process. We also suggest that an RMM level of at least 3, and ideally 4 be adopted.

At CMMI levels 4 and 5 we are measuring and improving on our process. We’ll address the higher CMMI levels in more detail as this series of articles continues.

Summary

• RMM level 3 specifies that requirements documents are organized, and structured.
• CMMI level 2 specifies that there is a managed process – in our case, one for managing requirements.
• A process must be at RMM level 2 and should be at level 3 or 4 to be at CMMI level 2 or CMMI level 3.
• A process should be at RMM level 3 if it is at CMMI level 1.
• A process should be at RMM level 4 if it is at CMMI level 2.

Note that this implies that we would spend the extra effort to get to CMMI 3 before we would try and reach RMM level 5.

Check out the next article, CMMI Levels and RMM Level 4 or take our One Minute Survey on CMMI and RMM Levels.

Background

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure. In this article, we will look at the definition of RMM level 2. We also cover the tradeoffs and benefits of the practices it requires. Finally, we look at the mapping from RMM level 2 to various CMMI levels.

(larger version)

In the previous article in the series, we looked at how RMM level 1 – written requirements processes map to CMMI levels.

RMM Level 2 – Organized Requirements

RMM level 1 requires us to document our requirements, but doesn’t talk about how we document them. We can use emails, store information in databases, spreadsheets, and screenshots. But without an over-arching organization. In RMM level 2, we have to organize our requirements documents, we have to use consistent formatting, and we have to deal with administrative issues like security and version control.

The Case For Organizing Requirements

There are two main drivers for organizing our requirements:

• The need to consume those requirements.
• The need to change those requirements over time.

Consuming Requirements

Requirements are written not just to organize our thoughts, but to provide direction for the team. They are a form of targeted communication. The set the scope for software delivery, provide guidance in making prioritization decisions, and provide insight into what we will deliver – helping manage the expecations of our customers.

Organizing our requirements makes it easier to consume them. When we ask people to review our requirements, they will have more confidence, and experience less frustration, if they are consistently looking for documents in the same location.

This location could be a document repository, a shared drive on the network, a website, a portal site, or even organized in a file cabinet (?!). The point is that they are always in the same place. As the quantity of our requirements grows, they should also be organized in a logical way within that location. At RMM level 2, any organization is valid – as long as it is consistent, it will provide value.

Changing Requirements Over Time

While documenting the requirements provides benefit, the disorganization comes at a cost. Requirements change over time. Our requirements documentation should change over time as well.

The biggest complaint with waterfall projects is that our understanding of the requirements does not change over time. Requirements are a moving target. With a waterfall project, we define a set of requirements, and then kick off the project – sort of a fire and forget model. Months or years later, we deliver a product – and it will probably match our documented requirements. While we were happily developing against the requirements we documented, the actual requirements have changed. There’s a very high risk that what we deliver will not meet the evolved needs.

• The Standish group reports that over 80% of projects are unsuccessful
  either because they are over budget, late, missing function, or a
  combination. (http://www.standishgroup.com/sample_research/chaos_1994_1.php)
• 53 percent of projects cost 189 percent of their original estimate.
• Average schedule overruns may be as high as 100%
• Over 40% to 60% of defects in software are caused by poor requirements
  definition.
• About One-Quarter of all projects are cancelled.
• Customers do not use 20% of their product features.

Why We Should Invest in Requirements Management

While poorly documented requirements are certainly a factor in the statistics above, another factor is no-longer-relevant requirements. These likely play out as features that are missing, features that are not used, and project cancellation (due to lack of relevance, or lack of ROI).

When we don’t organize our requirements, then changing them becomes more expensive – we have to find them, modify them, and notify people that they’ve been changed. It also becomes difficult to know if this document is the latest document. Organization addresses this problem.

Why Avoid Organization?

Organization does come at a cost. We have to spend the time (and possibly money) to set up the repository. We have to spend time to determine how we want to organize our requirements. And we have to spend some time putting the documents into our organized repository.

We identified the benefits of getting data from an organized location. What if we don’t do that very often? If our requirements approvers never have to review the documents, then they don’t benefit from the effort we spent organizing the documents. Perhaps we just have a meeting where get verbal approvals, or route them all with an email (Microsoft Outlook lets you put voting buttons on emails) for approval.

If we’re using a waterfall style project where we document the requirements once, and never change them, then each person on the implementation team can just print out a copy and refer to it when they need it. Again, no benefit from organization.

We all recognize the costs of both of these approaches, but they do avoid a little bit of busy work. It’s possible, however unwise, that some teams will take this approach, and thereby not benefit from organization. Those teams might operate best at RMM level 1.

The Case For Consistently Formatting Requirements

By using consistent formatting, we make it easier for someone to read multiple requirements. They can more easily compare and contrast the documented requirements. They don’t have to spend cycles re-learning how to read each requirements document. Once they become familiar with the format, they can ignore it, and spend time on the content of the document.

When we talk about consistent requirements, we are generally talking about the logical consistency of the statements within and across requirements – but the consistency of formatting also has value. This formatting consistency is what RMM level 2 requires.

Avoiding Consistency

We save some time in training by not requiring people to write consistently. However, the time we save is probably completely absorbed by the time people spend thinking about how to structure the requirements while writing them. And we lose the benefits that come from reading requirements that use a consistent format.

Requirements Administration

The final element identified in RMM level 2 is the administrative perspective. A focus on security, access, and version control is what the IBM team identifies as the relevant administrative issues.

Security and access are identified as elements that engender trust in the documentation. We may be too agile or too trusting, but we don’t see those factors as being particularly relevant to trust. They are certainly valuable when it comes to protecting against unwanted distribution of the information – but we are not generally concerned with people modifying the documents in unacceptable ways. We’ll grudgingly admit that it is possible that a developer will open a requirements document and delete a requirement that he feels is inappropriate, or rewrite it so that it matches his implementation. We just don’t think that it is a practical concern.

Version control, however, is very important. The biggest trust issue we have is in being able to trust that we are reviewing the latest version of a requirements document. Version control provides us with that benefit. It also allows us to undo any untoward modifications of the document. At a minimum, version control should consist of the persistence of previous versions of files. This can be handled by using unique names for each version of the file, by storing copies of the file on a regular basis as backups, or by using version control.

Subversion is the best version control system (VCS) we know of. If implementing a new VCS, we suggest using subversion. It is open source, easy to administer, and best-of-breed.

Mapping CMMI Levels to RMM Level 2

In our diagram, we show the following mappings for RMM Level 2:

• CMMI level 0 – No Entry
• CMMI level 1 – Requirements Should Be Organized
• CMMI level 2 – Requirements Must Be Organized
• CMMI level 3 – Requirements Must Be Organized
• CMMI level 4 – Requirements Must Be Organized
• CMMI level 5 – Requirements Must Be Organized

For CMMI Level 0 – when our process is so ad-hoc that documentation of requirements is questionable, discussions about how we organize the requirements documents are irrelevant. We’re talking about icing and candles when we don’t even know if we have a cake.

For CMMI Level 1 – A valuable process must include documentation of requirements. Those documents really should be organized. The benefits of versioning alone should make this an easy decision. Placing the documents in known locations, and having them be written in a consistent format is valuable too.

For CMMI Level 2 and higher – When we talk about a managed process, we are talking about bringing order to the chaos. Centralizing the requirements in a repository, versioning the documents, and using consistent formatting all bring order.

Imagine a managed requirements process that does everything with the exception of applying consistent formatting to our documents. Perhaps we have various authors of our requirements documents, and they write inconsistently. There’s value in doing all of this, but it would be CMMI level 2, RMM level 1. Only with all three elements (consistent location, consistent formatting, and versioned documents) would the process be both CMMI level 2 and RMM level 2.

We would definitely focus on moving from RMM level 1 to RMM level 2 before we would try and standardize our process across our company. That standardization would be the move from CMMI level 2 to CMMI level 3. Based on that perspective, we believe that an RMM level 2 process rating is a mandatory element of all CMMI levels above CMMI level 1.

From a CMMI Level Perspective

The previous analysis basically looked at the “RMM level 2″ column in our grid, and inspected the relative decisions for each “CMMI level” row. Now we will look at it by reviewing the CMMI levels, and seeing how they map to the RMM level.

A quick review of the same chart (so you don’t have to scroll up and down):

(larger version)

At CMMI level 1, we require that requirements be written. We suggest that they be organized and structured.

At CMMI level 2, we require that the documentation be organized. A managed process without some form of organization and consistent documentation is a poorly managed process.

At CMMI level 3, we are standardizing our approach across our company. And at CMMI levels 4 and 5 we are measuring and improving on our process. We’ll address the higher CMMI levels in more detail as this series of articles continues.

Summary

• RMM level 2 specifies that requirements documents are organized.
• CMMI level 2 specifies that there is a managed process – in our case, one for managing requirements.
• A process must be at RMM level 2 to be at CMMI level 2.
• A process should be at RMM level 3 if it is at CMMI level 1.

Note that this implies that we would spend the extra effort to get to CMMI 3 before we would try and reach RMM level 5.

Check out the next article, CMMI Levels and RMM Level 3 or  take our One Minute Survey on CMMI and RMM Levels.

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure. In this article, we will look at the definition of RMM level 1. We also cover the tradeoffs and benefits of the practices it requires. Finally, we look at the mapping from RMM level 1 to various CMMI levels.

Background

In our introduction to mapping RMM levels to CMMI levels, we presented background info on CMMI, introduced the IBM article on RMM levels, and posted an initial mapping structure.

(larger version)

RMM Level 1 – Written Requirements

Level 1 of the requirements maturity model is defined at a high level as simply having written requirements. IBM defines written requirements as persistent documentation. They point out that post-it notes and whiteboards don’t count. Email discussions, word documents, spreadsheets and presentations all count.

IBM presents an argument of tradeoffs – as long as the cost of documenting requirements is exceeded by the benefits, it makes sense to write the requirements. They point out three benefits of having written requirements:

1. A contract is explicit, or implicit in the requirements. The documented requirements can be used to manage the customer’s expectations, and can also be used to validate that what was promised was delivered.
2. Clear direction for the implementation team can be found in the requirements documents.
3. New team members can rely on the documented requirements as a means to get up to speed.

While we strongly agree with the first two points – we think the third one is a bit of a stretch. While having requirements documentation does help people get up to speed, it isn’t a first-order benefit. Videotape a couple 1-hr presentations. One presentation discussing the goals of the project, and one discussing (and whiteboarding) the architectural approach of the solution. Put these on the server and let new people watch them. Much more cost-effective at helping people get up to speed. [Note - I'm pretty sure that I heard Alistair Cockburn suggest this approach or something like it in a podcast interview, so the credit for the idea is his, not ours.]

We would also add that documenting requirements is all but worthless if we don’t use the documents as tools to support conversation with our team members. Incremental delivery is a process that is dependent upon feedback. We must get feedback from stakeholders, and from the implementation team.

Stakeholders will verify the correctness and validate the completeness of the requirements.

The implementation team will provide feedback about the clarity, verifiability, and feasibility of the requirements as written.
Requirements need to be written to support verification. The QA team and stakeholders are responsible for verifying that what was delivered is what was expected. Technically, the delivery must match the requirements – but the requirements should match the expectations of the customer.

One Step Above Chaos

I like that the IBM guys name level zero as “Chaos.” I’ve worked as a developer on projects without requirements. It is chaos. There’s a reason we write requirements. They set expectations. And theres a reason why we review and approve the requirements. It’s essentially a form of structured active listening.

Mapping to the CMMI Levels

In our diagram, we show the following mappings for RMM Level 1:

• CMMI level 0 – Requirements Should Be Written
• CMMI level 1 – Requirements Must Be Written
• CMMI level 2 – Requirements Must Be Written
• CMMI level 3 – Requirements Must Be Written
• CMMI level 4 – Requirements Must Be Written
• CMMI level 5 – Requirements Must Be Written

For CMMI level 0 – even if we don’t have a formal process, we really should be writing our requirements – and using those documents to manage expectations, provide feedback (that we’re doing the right stuff), and scope and focus our efforts.

For CMMI levels 1 and higher – all of the measured CMMI levels require that we have a defined process. Even with the disorganization of a team operating at CMMI level 1, we still need to have a process defined. And a requirements management process that doesn’t involve documenting the requirements isn’t worth very much at all.

Note that documentation might be in the form of prototypes, wireframes, and JAD Session notes. No one is saying that they have to be documented in any particular way. In fact, at RMM level 1, they aren’t in a consistent format, and don’t use a structured requirements framework. Consistent formatting is an element of RMM level 2. And RMM level 3 is focused on structured requirements.

The requirements documents may be scattered through a series of email debates, collaborative databases, and files on network share drives. That’s fine for RMM level 1 – in fact, it is part of the definition of RMM level 1. Organized requirements are a characteristic of RMM level 2.

Remember – CMMI Levels only represent how a process is implemented – they don’t characterize the effectiveness of any one process.

From a CMMI Level Perspective

The previous analysis basically looked at the “RMM level 1″ column in our grid, and inspected the relative decisions for each “CMMI level” row. Now we will look at it by reviewing the CMMI levels, and seeing how they map to the RMM level.

A quick review of the same chart (so you don’t have to scroll up and down):

(larger version)

At CMMI level 1, we require that requirements be written. We suggest that they be organized and structured.

At CMMI level 2, we require that the documentation be organized. A managed process without some form of organization and consistent documentation is a poorly managed process.

At CMMI level 3, we are standardizing our approach across our company. And at CMMI levels 4 and 5 we are measuring and improving on our process. We’ll address the higher CMMI levels in more detail as this series of articles continues.

Summary

• RMM level 1 specifies that requirements are documented.
• CMMI .evel 1 specifies that there is a process – in our case, one for managing requirements.
• A process must be at RMM level 1 to be at CMMI level 1.
• A process should be at RMM level 2 or 3 if it is at CMMI level 1.

Note that this implies that we would spend the extra effort to get to CMMI 2 before we would try and reach RMM level 4.

Check out the next article, CMMI Levels and RMM Level 2 or  take our One Minute Survey on CMMI and RMM Levels.

Welcome Readers of the Carnival of Enterprise Architecture! We hope you enjoy this series of articles!

CMMI (Capability Maturity Model Integration) is a description of the level of enlightenment of a process. It is essentially a measure of the quality and capability of a process. There are five categories, into one of which every process will fall. IBM took a similar approach to defining the requirements management process. In this series of posts, we will marry the two frameworks.

Background on CMMI Levels

We wrote an introduction to CMMI levels last March. In our article, we identified that there are five CMMI levels. Technically, there are six CMMI levels, when you include level zero. Level 0 is “undefined” by the CMMI, and represents an ad hoc process, or a lack of process.

CMMI Levels

• CMMI Level 0. Undefined. No real process.
• CMMI Level 1. Performed. A process is defined, but disorganized.
• CMMI Level 2. Managed. A defined process is managed.
• CMMI Level 3. Defined. A managed process is standardized across the company.
• CMMI Level 4. Quantitatively Measured. The performance of a standardized process is measured.
• CMMI Level 5. Optimizing. Performance measurement is used as a feedback loop to improve the process.

Take CMMI Levels With A Grain of Salt

Just knowing the CMMI Level of a process is not enough to know if the process is any good. By the same token, choosing a particular CMMI level, and meeting the technical requirements of that level are not enough to assure a good process.

Backgroundon RMM Levels

The folks at IBM wrote an article in 2003, where they defined five levels of maturity for requirements management processes. All five of the requirements management maturity (RMM) levels all build on the previous level, with increasing capability.

• RMM Level 0. Chaos. No persistent documentation of requirements.
• RMM Level 1. Written Requirements. Writing requirements documents (not emails and whiteboards).
• RMM Level 2. Organized Requirements. Colocation, versioning, consistent formatting.
• RMM Level 3. Structured Requirements. Defining types of requirements and their relationships.
• RMM Level 4. Traced Requirements. Explicitly mapping the support-network of requirements.
• RMM Level 5. Integrated Requirements. Integrating with the development environment and change management.

What IBM Didn’t Do

They didn’t map their framework back into the CMMI framework (known as CMM at the time) except for the following comment in the introduction of their article:

Those familiar with the CMM (Capability Maturity Model) from the Software Engineering Institute (SEI) will note some similarities to our parallel model, which has no direct relationship to the CMM save one: Achieving Level Five of the RMM will assuredly help an organization get to at least Level Three of the CMM.

IBM put together a great framework for describing elements of increasingly capable requirements management processes.

That is what the SEI tried to do when they developed the CMMI. Why couldn’t the IBM team just map their framework into the CMMI framework?

The problem is there is a mismatch between the two frameworks.

• The RMM framework describes steps and elements of a requirements management process. Each step adds a level of capability to the process. It might be more aptly named the requirements management capability framework.
• The CMMI framework describes the strategic capabilities (maturity) of how a process is applied, without assessing the tactical capabilities of the process itself.

The SEI recognized that the analysis of the tactical capabilities of any process would be different for every process, and left it to others to perform that work. This is almost what the IBM team did. We’re going to take a crack at it here.

Mapping RMM Levels to CMMI Levels

This is the first in a series of articles that will present a mapping of RMM levels to CMMI levels. We like using CMMI as a means to evaluate our internal processes, notwithstanding the challenges we mentioned earlier. We also like the framework that IBM presented for describing requirements management processes.

Shoot First, Ask Questions Later

There’s a lot more to write about this than we can put into a single article. We’re going to tackle this as a series. Even so, we put together an initial draft of how we think this will ultimately work out. We’ll share that here now. But we reserve the right to fix it when we find problems as we (and you!) put more effort into it.

(larger version)

Articles In This Series

• Foundation Series: CMMI Levels Explained
• What CMMI Level Should We Use?
• CMMI Levels and RMM Introduction
• CMMI Levels and RMM Level 1 – Written Requirements
• CMMI Levels and RMM Level 2 – Organized Requirements
• CMMI Levels and RMM Level 3 – Structured Requirements
• CMMI Levels and RMM Level 4 – Traced Requirements
• CMMI Levels and RMM Level 5 – Integrated Requirements
• Don’t forget to take our One Minute Survey on CMMI and RMM.

Rishikesh Tembe shared twenty rules for software product delivery last month. His rules are from the perspective of a former software developer. Some we like. Some, not so much.

We Like

Rishikesh has a short post with 20 rules. Among those rules are some concepts that we feel like expanding upon:

8. Do the riskiest part of the project first.

This is a great idea (for developers and project managers) for a couple reasons. Risk may be a function of technical feasibility or customer accceptance/value. Risk may even be an artifact of how we go about developing our product – for example, offshoring introduces risks. Risk is generally a nebulous, but bad thing to have in a project. If a problem is known, a solution can be identified, addressed, and implemented.

Risks represent the unknown unknowns, or those problems that we don’t understand well enough to address them. Big risks can be very scary – they jeopardize the ROI of our project. Lack of user adoption can affect the expected value to our customers, which will directly or indirectly affect our bottom line.

Note: This is not meant to imply that prioritization of requirements by release should be subservient to risk-mitigation. The most important stuff should always be done first. Within a particular release, address the riskiest issues first. Prioritize first. Mitigate second.

10. Make sure you’re in total control of your toolset and improve it systematically

This alludes to the general benefits of operating at a higher CMMI level. As a CMMI level five organization, we would quantitatively measure and improve not only our tools but our processes.

16. Build regression testing into the build process.

We’ve talked about continuous integration in the past. In our opinion, no other approach should be used to develop software.

Not So Much

11. Do not take the clients’ deadlines literally – first accept the project, then renegotiate the deadline.

Clients have deadlines for reasons. They may be market-driven, or driven by internal politics or budget cycles. While it is possible that a deadline is arbitrary, it is usually associated with a compelling event of some sort. Any discussion of deadlines should be approached in the same way we manage scope creep as a relationship-building exercise.

Project constraints, such as budgets and deadlines, are things that should be addressed collaboratively. Our clients are our partners, not our opposition.

13. Document the interfaces perfectly, but don’t document code (see next point).
14. Be fanatical about the readability of code.

Absolutes are rarely rational end-points, although presenting goals as absolutes can be effective in motivating directional change in an organization (with no expectation of actually achieving the goal). More on that some other time.

Broken Windows, as Gladwell describes them, are absolutely detractors from any environment, and serve to degrade the performance of those who operate in the environment. Code readability, micro kitchens, flexible schedules. There are many soft ROI elements that make up the environment of a software developer. Readability of code, like readability of requirements, is important. A friend of mine used to joke that perl is a “write-only” language. Notwithstanding his joke, code needs to be readable. Even in perl.

Perl serves as a great example – elegance of code does not always coincide with syntactic simplicity. Avoiding commenting of the code is just a bad idea. People read code. Make it readable.

There are times when incurring a temporary code-debt is pragmatically more useful than delaying a release or a feature in order to polish the code.

Conclusion

There are some good ideas and some bad ones in the list. Most of them are thought provoking, regardless. We may eventually find a list of absolute rules we should all follow, and it would overlap with this list somewhat. But for now, we’re still looking.

“What CMMI level should we use?” is not the right question, but it is the question most people ask.

The CMMI (Capability Maturity Model Integration) of a software development process is the measure of that process’s capability. The goal of the measurement is to provide an assessment of the capability of a process with respect to creating software. Our foundation series post on CMMI provides background information, while this post focuses on the danger of misusing CMMI ratings.

1. The CMMI measurement is (mostly) a facade.

With the exception of a CMMI level five (Optimizing) process, having a CMMI rating doesn’t mean that the process is good. It means that the process is documented and managed (CMMI level two), standardized within the company (CMMI level three), or quantitatively measured (CMMI level four). Even CMMI level five status doesn’t tell us how good a process is, only that the team is actively focused on improving the process.

Having a documented process doesn’t make it a good process. This is the main flaw. If we documented a process that included steps like “developers create use cases” and “to certify a release, the developer installs the final build on his laptop”, we would qualify for CMMI level two. If we standardize on our poor process, we reach the next CMMI level. And we could measure “lines of code written per hour” and other skew quantifications of activity to achieve CMMI level four.The CMMI measurement isn’t entirely worthless – Carnegie Mellon has a track record of doing really great and smart stuff – CMMI is the best normalized measurement that anyone could come up with that would be one-size-fits-all. The problem is that in order to make the measurement apply to everyone, it has been neutered to the point of not providing very much valuable information.

It is important to know that a company has a process and measures it’s performance. It provides very valuable insight to know when a company is also optimizing that process (CMMI level five).

CMMI alone does not tell us enough about the process.

Which team would we rather have developing software for us – a CMMI level 3 team, or a CMMI level 2 team? We absolutely can not answer without more information. If our conversation with a potential outsourcer goes like this, then we have a problem:

“What is your CMMI level?”

“We operate our business at CMMI level four.”

“You’re hired!”

If however, our conversation goes more like this, we’re in a good place:

“Our technical guys have reviewed your process, and we like it. How long have your people been using it, and can you give us a couple references of companies for whom you’ve used this process?”

“Thank you. We received CMMI level four certification for this process two years ago. Since this is our standard process, all of our reference accounts have benefited from this process – you can contact any of them.”

“You’re hired!”

The key difference is that we’ve actually reviewed the process to determine it’s value. The CMMI rating gives us some assurance that the process is followed rigorously. ISO9000 certification, in the hardware world, suffers from the exact same problem. In a nutshell, ISO9000 requires companies to say what they do, and do what they say. It provides no insight into the value of what the company chooses to do.

2. CMMI ratings create a false sense of security.

It is very tempting for companies to advertise their CMMI level, especially with outsourcing companies, and especially with the global providers. These companies can capitalize on the human instinct – out of sight is out of mind. When companies outsource, they want to be able to “not worry about it”, and CMMI ratings can be engender a false sense of confidence in the outsourcing provider.

In addition to the implicit presumption that a documented process is a good process, it is also easy to assume that people who follow a process are at least competent at what they do. There is no reason to presume this without reviewing the quality of their work. We should always talk to referrals to find out their level of satisfaction with an outsourcer.

When we’re managing our own team, it is easy to fall in the “our process is broken” trap. Very few people will tell you that it was their fault. We’ve not yet heard someone say “I was not smart enough to solve the problem.” or “If I had worked harder, we would have made it.” We have repeatedly heard “The process is broken.” and “I need better tools / a bigger team / more time and budget.”

The process may very well be broken. Eventually, Chicken Little was right. But achieving a CMMI level without fixing the process doesn’t fix the process.

3. Standardized processes can shackle innovators.

Many people thrive on having a structured environment and process in which to work. They actually do better work when given concrete tasks, discrete deliverables, and monitored timelines. Very few innovators work best this way. When creating differentiated products, an innovator may best be served with a differentiated process. As a result, people who tend to gravitate towards standardized processes tend to create standardized (me-too) products.
Many innovative companies, like IDEO or Frog, solve a wide range of problems, from software to electronics, to toothpaste dispensers. A single unified process would be either stifling or irrelevant if all of those teams had to use it.

4. Focusing on the process means not focusing on the product

In a well-known play in American football, the quarterback throws a long pass to a widereceiver who is running down the field. The receiver attempts to catch the ball, avoid a tackle from the nearby defender, and keep running. The receiver will occasionally drop the football, because he is too focused on avoiding the tackle and on running. The commentators will point out that he needs to not think about getting tackled until he actually catches the football. The receiver does not have his eye on the ball, metaphorically speaking.

Driving and rewarding our teams for the CMMI level of the process they follow is like rewarding them for avoiding tackles and running. If this becomes a higher priority than writing great software (catching the pass), then they will do a methodical and rigorous job of following the process, and if we’re lucky, write great software along the way. Our goal is the great software – we need to make sure we are managing our teams with the software as the highest priority.

When teams are focused on writing great software, then a great process can help them. And CMMI can provide some affirmation (but not validation) that they are following a good process.

The right question

The right question is “How good is our process?”

A good process can make a good team very good, and can make a great team invincible. A good process helps an incompetent team by providing us good information about their incompetence. A bad process at best annoys good and great people, but more commonly it dillutes their efforts or even derails their projects. It is important to understand the quality of the process being followed by the team. And investments in improving the process can be worthwhile (subject to the 80/20 rule).

CMMI, unfortunately, can not tell us if the team is competent. It can not tell us if the process is good. It can only tell us that a process is being followed (or measured).

Conclusion

We’ve used the phrase neccessary but not sufficient repeatedly when we describe important elements of software product success. CMMI ratings fall into the same category.

In fairness, a team with a CMMI level five process is actively applying their ongoing analysis (CMMI level 4) to improving their process. This is the one piece of CMMI data from which we are more likely to infer that the process is a good one. However, as the SEI points out themselves in their documentation:

Reaching CMMI level 4 or 5 for a process area is conceptually feasible but may not be economical except, perhaps, in situations where the product domain has become very stable for an extended period of time.

With our focus on great software, we have to prioritize innovation, and more specifically differentiated innovation. By definition, this precludes us being in “stable” product domain.

CMMI ratings are not what drive us.

CMMI is the initialism for Capability Maturity Model Integration.

CMMI is a numeric scale used to “rate” the maturity of a software development process or team. Maturity can be thought of like enlightenment. An immature process is not much different from the old “infinite monkeys” yarn – maybe we get it right, but probably not. A fully matured or enlightened process not only does it right, but improves itself over time.

The Software Engineering Institute (SEI) at Carnegie Mellon (Go Tartans! BSME90) created the CMM model for software engineering in the late 80′s and early 90′s. In an effort to consolidate multiple CMM models for different process areas, the SEI team created the CMMI in 2002. In this post, we will understand what each level represents.

Technically, the name of the model is the Capability Maturity Model Integration for Software Engineering, or SW-CMM, but in practice people just use CMM. The 645 page document can be found on the CMU SEI site.

Five levels of maturity in CMMI

The folks at the SEI created five classifications or levels of process maturity. Every software project can be classified into one of the levels. The levels are progressively harder to achieve, and each builds on the previous level. To be classified in a particular level, a process must meet all of the criteria of that level. If a process meets all the criteria of level 2, and some of the criteria of level 3, then that process is considered to be a level 2 process.

Note that there is a level 0, or incomplete. This level represents the absence or incompleteness of activity. If we are developing software, we are at least at level 1.

1. Performed.
2. Managed.
3. Defined.
4. Quantitatively Managed.
5. Optimizing.

CMMI Level 1: Performed

The lowest possible CMMI level (there is no level 0) represents software development essentially in the absence of a process. Specifically, the SEI defines level 1 as being achieved if the goals of the process are met. If we are trying to write software, and we write software, then we are at level 1. In practice, level 1 processes are the ones described as chaotic and unordered. A team with a level 1 process can easily be dependent upon individual effort for success.

Most small software companies, independent developers, as well as many corporate IT departments operate at level 1. If we can point to projects that were “saved” by a superstar on the team, we are probably operating at CMMI level 1. We love having superstars on our teams – we hate being dependent upon them for survival.

CMMI Level 2: Managed

SEI defines CMMI level two as follows:

A managed process is a performed (capability level 1) process that is also planned and executed in accordance with policy, employs skilled people having adequate resources to produce controlled outputs, involves relevant stakeholders; is monitored, controlled, and reviewed; and is evaluated for adherence to its process description.

Our interpretation: take a CMMI level one process, and add a project manager to coordinate the chaos. The distinction is that at this level, the activities are planned, and then managed according to the plan. No additional insight is applied, just orchestration.

CMMI Level 3: Defined

To achieve CMMI level three, we have to take a process that qualifies for CMMI level two, and institute it as a corporate standard. Then we tailor and apply that standard process to individual projects.

CMMI Level 4: Quantitatively Managed

When we incorporate quantitative measurement or statistical analysis tools as part of our process management process, we are operating at CMMI level four. This level represents “hard” introspection that utilizes quantified data to improve the management of the current project within our defined project.

CMMI Level 5: Optimizing

When we analyze the quantitative data from our projects and apply it to refining our processes for the future, we are operating CMMI level five. This level represents an introspective approach to quantitative project management, combined with a continuous improvement objective.

Articles In This Series

• Foundation Series: CMMI Levels Explained
• What CMMI Level Should We Use?
• CMMI Levels and RMM Introduction
• CMMI Levels and RMM Level 1 – Written Requirements
• CMMI Levels and RMM Level 2 – Organized Requirements
• CMMI Levels and RMM Level 3 – Structured Requirements
• CMMI Levels and RMM Level 4 – Traced Requirements
• CMMI Levels and RMM Level 5 – Integrated Requirements
• Don’t forget to take our One Minute Survey on CMMI and RMM.

- – -

Also check out the index of the Foundation Series posts for other introductory articles.