How many people should be involved in requirements elicitation? A question from one of our readers via email.
Hi Scott, in the last months I faced the issue of managing the requirement elicitation phase in an Identity Management project. I have a very simple question. In your opinion how many people should do the interview process against the customer ?
In my experience one person is not enough even if the interview session is electronically recoded. I have observed the two heads are much more better than one since two different points of view are captured from the Voice of the Customer.
I’d be happy to hear your opinion on this.
Thanks for the great question! I’ll turn this into two questions and try and answer them both.
- How many requirements gatherers should gather requirements from the subject matter expert (SME)?
- How many SMEs should be providing requirements to the requirements gatherer?
How Many Requirements Gatherers?
A simple question deserves a simple answer: two if you can afford it, one if you can’t. Feel free to skip the rest of the article. :)
The success of requirements elicitation hinges on our ability to listen effectively. Active listening allows one analyst to get the same information from an answer that two people (who hear things differently) would get. The bigger benefit of adding a second person comes from asking questions. Two people will identify more loopholes and corner cases, will ask questions that uncover unspoken requirements, and otherwise improve the quality of the information gathered. Will they identify twice as much? No. There’s a law of diminishing returns at play. And the greater the skill of the analysts, the more overlap there will be in the questions they each ask. A third analyst won’t add much at all, and will run the risk of getting distracted and not really contributing anyway.
Having two interviews, each by one analyst, each of a single SME, will probably unearth more requirements than if both analysts interviewed a single SME once in a joint session. When the schedule/budget is tight, use this approach for the most efficient requirements gathering. When efficiency is not a driver, then doubling up the analysts will get better data – just not twice as good.
How Many SMEs?
Another valid question is how many SMEs to interview in a single session? I think two is an ideal number. One SME will unconciously gloss over details, that a second SME will catch. The two SMEs can debate contentious points, and correct and help each other. If both SMEs aren’t available for the same meeting, meet with them sequentially. Have the second SME review the requirements that were documented with the first SME.
Again, there is an efficiency issue – do we double up the SMEs (and the cost of them), or do we work with them sequentially? Doubling up is better, but again, not twice as good due to diminishing returns. A third SME is likely to cause the conversations to devolve and become much less effective – even if uncovering more details, the overall process will be less efficient.
Other Factors
There are a couple other factors that have more of an impact than perfectly optimizing the number of attendees. The best way to improve requirements gathering is by increasing the experience/skill of the business analyst. Check out the factors at the end of this article on estimating requirements gathering for more details. Similarly, SME aptitude plays a role.
As analysts, we have to be able to wear a bunch of different hats. We are talking to SMEs about their expert domain. We are validating that our requirements are attainable, by applying our knowledge of the IT world. And we are making sure that we focus on the most valuable requirements by applying some understanding of ROI. Basically, we have to be comfortable operating in all three domains of expertise.
If we find that we need to include more people in our meetings to get coverage across all the domains, that would supercede the “two is better than one, but one is more efficient” arguments from above.
Conclusion
We have to make sure we get coverage of each domain of expertise. One person is more cost-effective than two. Two people will elicit better requirements than one.
Scott, there’s also the issue of larger elicitation meetings, where you might have 4-5 SMEs. In that case, I always recommend one facilitator and one scribe (both of which are analysts) so that the facilitator can focus on keeping the conversation going the the scribe can focus on writing it down and doing some summaries for checkpoints during the meeting.
I was at the Proforma user conference last week where one customer described their requirements elicitation meetings, which always included both a facilitator and a scribe. In this case, the scribe used the Proforma tool directly to model the business processes live in the meeting, which they found to increase accuracy and shorten the requirements documentation process. They also recommend that smaller groups are better, a point with which I agree, although sometimes more than the two SMEs that you recommend can be optimal.
Scott, I would re-examine your assumption that SMEs are the primary source for requirmeents. I have written before that this assumption is suspect; please see my latest thoughts on the subject here and here.
Sandy – thanks again for reading and commenting! The tip about using a scribe and facilitator is a great one. And I can’t overemphasize the value of the scribe being an analyst too. It is important that they be able to contextualize and organize the information as it comes out – stream of conciousness notes are much harder to organize after the fact (and if the scribe isn’t a BA, that’s what they will be).
As to larger meetings – yes, there are other types of meetings – I was targeting the interview specifically in this post. Getting more people in the room changes the dynamic of the session into a different type of session.
Depending on the level of expertise of the SMEs, larger sessions may be required to get a full understanding of a process/requirement. Otherwise, larger meetings will have SMEs “taking turns” telling us stuff – it could be more efficient to split those meetings into smaller more targeted sessions.
Thanks again for reading!
Roger, thanks for reading and commenting!
I guess I should have titled the post “How many people for SME interviews”, and called out that it is only one form of requirements gathering. We also have an article that identifies several types of requirements gathering.
We’ve found that working in analyst pairs for any given domain or functional area to be an excellent way of working. We got the idea from the book, “The Mythical Man Month” which suggested that architects work in pairs. We used the same notion with analysts. Working in pairs gives you the advantages such as those described by the others in interviews, but it also gives you a backup to the lead analyst. The secondary analyst can be a representative in meetings and reporting up, in addition to serving as a sounding board and note taker. The important thing is that the secondary analyst is just an observer/note taker. We don’t load the second analyst with any deliverables – he reviews documents, but isn’t on the hook to give feedback; attends meetings, but just serves as note taker, etc… This minimizes the impact on our resources.
Louie, thanks for commenting, and thanks for reading!
OK, it looks like y’all are ganging up on me about the facilitator/scribe combination. I like the idea very much, and agree that it is effective.
In the article above, I proposed that it is more expensive (and therefore less efficient) to use this approach – that the doubled cost, while yielding markedly improved results, will not yield more than doubled results.
I’m happy to be proven wrong – so – everyone who’s read this far – please chime in: are two analysts more than twice as good as one?
Scott,
I’m not sure you’re asking the right question. What does it mean to get “double the results?” Are we talking about quantity of requirements or quality of requirements? If you’re asking if two analysts could generate twice as many requirement statements as one, then I would say, probably not. If we’re talking about quality of requirements, then how does one measure this to determine if two analysts are better than one?
I think the answer lies in the complexity of the domain under consideration. I wouldn’t necessarily double up analysts on, say, the requirements for User Management. In general, this functional area tends to be straightforward and has identifiable patterns. However, for functional areas or domains that are quite a bit more complex, then the marginal benefits of a second analyst and the concommitant increase in quality of requirements becomes a better value proposition. The costs of screwing up the domain model or data model for a system is much higher than screwing up look/feel in a UI. A second analyst eyeballing a domain model and the web of business rules that accompany it can be invaluable when the business is complex and confusing – especially if one is involved in de-tangling the organic structure of legacy systems.
The question, “is the cost of two analysts worth the marginal increase in quality of requirements delivered?”, can only be answered in context to the risk inherent to the functional area being examined.
Louie,
Great comments and questions! I agree with your second paragraph, and would say that I am asking the right question (in a somewhat Socratic way).
Starting with agreement. Your point about risk-adjusted analysis of cost/benefit is perfectly right. Also, recognizing that some topics are analytically harder to absorb than others is a key insight. That combined with variation in BA skill / horsepower will drive tactical staffing decisions more than anything else. For some BAs, a complex domain is “simple” and for some, a straightforward domain may still be “hard to grasp.”
As to the question I was asking – I intentionally used the phrase “twice as good” because the desired results will vary by project and project objective. My goal was to have people ask “what are the results that matter for my project?”
Quality of requirements might be paramount, given binary choices, but not with a continuum of quality and the law of diminishing returns. There is absolutely a minimum level of quality, below which every incremental effort is best spent improving quality. What about above that level?
Using hypothetical data to make my point: 20 requirements written with “good enough” quality (conceptual accuracy, spelling and grammar mistakes, slightly non-optimal prioritization) will provide “more results” than 10 requirements written with the same conceptual accuracy and perfect spelling, prioritization.
We also have to balance resources across the team – if we have developers starved for work (analyst bottleneck), then the “value” of an additional requirement today (even if it gets rewritten in the future) may exceed the value of perfecting an existing requirement. We may also be constrained by SME/stakeholder availability.
It is a complex balancing act, and requires exactly the kind of analysis you provide in your comments – but that analysis is going to be specific to a given situation.
Given all that, I still think the “summary” answer that I wrote originally is sound:
Based on all the great feedback in the comments, I would say that we all agree that investing in “more analysis” is better, and we should push to staff the double analysts. When we can’t get the staffing we desire, I lean towards the single analyst model, although the facilitator/scribe method does have a lot of benefit as long as both people have the needed skills.
More analysts is better might be fine, but more requirement sources is not. When you get two many people in the room, social processes kick in and people defer to the de facto leader. And, from there, you are already on the road to requirements volitility.
Somewhere in all of this the RA was said to be the reality check for delivering implementable requirements. Efficency comes up in this discussion. We are still using RE techniques that originated when code was expensive. As code has become cheaper, RE techniques have not changed. Given the volitility of requirements gathered via JAD and other group techniques, would it be more efficent to gather and deliver applications at the level of the individual? My guess is yes. Otherwise, the application will be recoded in a few weeks when the next paradigm wins discipline adherents.
David, thanks for reading and commenting!
Great point about having too many people SMEs in the room.
Interesting observation about requirements volatility and the “cost of code.” With incremental processes, I think you may be on to something. On waterfall projects, we still have a very large cost of misdelivery (even if the coding is cheap, the lost ROI is expensive).
I don’t know that I’ve experienced “group-based requirements” as being more volatile than others (doc analysis, SME interview, etc). Anecdotally, I think I’ve seen the most volatility when doing sequential interviews of SMEs with differing perspectives, and by getting those folks in the room at the same time, I’ve been able to reach consensus more quickly. All during the elicitation phase. Can you elaborate on how the group-stuff is more volatile than requirements gathered in other ways?