Gartner research on Agile Requirements Definition and Management (RDM)

Gartner has a research report available for $95, titled Agile Requirements Definition and Management Will Benefit Application Development (report #G00126310 Apr 2005). The report is 7 pages long and makes an interesting read. Gartner makes a set of predictions for 2009 about requirements definition and management (RDM) systems, and the software created with RDM tools. Gartner misattributes several benefits of good process to RDM tools. We give them a 3.5/7 for their analysis – check out the details here.

Here’s the excerpt from their website (AD = application development):

The flexibility with which requirements are gathered and managed shows how disciplined an AD process is. AD organizations with automated requirements definition and management environments will better support change control, gain testing efficiencies and reduce future maintenance burdens.

Gartner Predictions

• The cost of quality for developed systems will drop by 30%
• Maintenance costs will drop by 10%
• User satisfaction will go up from ‘fair’ to ‘good’ for medium and large applications

RDM systems have a meager impact on these predictions – other trends and processes are just as likely to affect them, hence our low rating of the Gartner report. Read on to see where they err.

Reduced cost of quality (RDM score 2/3)

Better requirements lead to fewer bugs. In Where bugs come from, we talk about the introduction of bugs from different sources in the process. The area where RDM can help is with misinterpretation of requirements. A structured system makes it easier to validate the alignment of proposed designs with requirements. RDM score = 1/1.

Incremental delivery processes will likely have a greater impact on cost of quality, as they help us correct mistakes in requirements documentation. This source of bugs (doing the wrong thing right) is higher up in the process, and thus has a larger impact on the bottom line. Gartner touches on this trend as well, but does not try to tease out the potential benefits and isolate them from the “benefits of RDM” analysis they present. RDM score = 0/1.

Traceability in RDM tools can provide large benefits in the cost of quality. To achieve these savings, RDM users must trace their testing to the requirements. This traceability helps uncover gaps in test coverage of the requirements, resulting in fewer bugs released to the field. RDM score 1/1.

Maintenance costs will drop (RDM score 1.5/3)

There are three trends that can drive lower maintenance costs for software. RDM can reasonably play a role in 1.5 of the 3.

1. Improved design and implementation. RDM doesn’t make design better. Incremental construction creates opportunities for refactoring the code. RDM score = 0/1.
2. Fewer gratuitous features. Most software today has too many features. Optimizing the features for a product will reduce the size of the code, and therefore reduce costs. RDM provides traceability of features to top level objectives (ROI), making it easier to identify and descope marginal features. RDM score = 0.5/1.
3. Cheaper labor. Outsourcing can reduce the cost of development labor, but not as much as you might think. Joel Spolsky shows us that 80% of the costs aren’t programmers, so the upper bound on savings is small. Tarun Upadhyay provides some real world data showing that the total cost savings can be between 8% and 38% when replacing 70% of US developers with Indian developers. The extra savings comes from moving some of the abstraction layer (supporting roles and overhead costs) to India as well. RDM makes this possible by providing a means for rigorous asynchronous communication between team members operating in different time zones. RDM score = 1

User satisfaction will rise (RDM score 0/1)

User satisfaction is ultimately increased when software is focused on user task-accomplishment, or goal-achievement. Process approaches like Alan Cooper’s interaction design process make this more likely to happen. Existing RDM products use a structured-requirements representation, most aligned with Karl Wiegers’ framework. With Wiegers, the main tool for capturing user tasks is the formal use case.

While it is possible to combine interaction design and structured requirements processes, none of the current RDM products do this. Our prediction – none of them will make this change in time to impact Gartner’s predictions. As interaction design gains momentum in the product management space, customer satisfaction will definitely go up. But it will be in spite of RDM systems, not because of them. RDM score = 0

Conclusion (RDM score 3.5/7)

The optimism about improved economics of software development and improved levels of user satisfaction is heartening. Unfortunately, Gartner has tied too much of this optimism to the proliferation of RDM systems (they also forecast strong growth in sales of RDM systems). We pointed out that requirements management software will not solve the problems with bad processes.

We are excited to see Gartner focus on RDM and on RAD processes. Executives are very swayed by Gartner opinions. If RDM retools to support interaction design, it could indeed live up to these predictions.