Cargo Cult Requirements

Photo of Melanesians with bamboo plane

Is your team focusing on the mechanics of creating good software, without understanding the connections from your efforts to your goals? Are you primarily focused on the structure of use cases, the syntax of user stories, and the cardinality of your domain diagrams? All of these things are important to effective communication – but if this is your primary focus, you may be in a cargo-cult environment.

When you first hear about cargo cults – a real thing that happened – you may laugh at the absurdity of it, but when you read the Wikipedia article on cargo cults, and start to understand the context in which the Melanesian people formed the cults, you will stop laughing. The lack of perspective and implied desperation of the people of Melanesia is quite sobering. For folks who don’t want to click through to the Wikipedia article, here’s the short version. During World War II, the Japanese, and later the Americans, delivered supplies to their troops on the Melanesian islands. After the war ended, the cargo shipments stopped. In attempts to get more cargo to fall from the sky, islanders imitated the behaviors of the now-gone military personnel. Placing torches to light runways, carving headsets from wood, even mimicking daily military activities like wearing uniforms and marching in formation – all of these “following the mechanics” activities were performed in hopes of achieving the goal – getting more cargo.

The Melanesian islanders precisely followed the procedures that their “instructors” followed – they were intelligent people, who learned exactly everything that was shown to them. However, they were not told that the rituals of signaling to incoming planes did not cause the planes to fly over and drop cargo – the rituals were only important to getting cargo when the planes were already flying overhead.

Melanesian bamboo plane

A similar thing happens with many teams when it comes to creating software. When developers are following the rituals of good software engineering practices, it does not cause them to create good products. It only causes them to create products effectively. There’s no correlation that assures that they are being asked to build the right product. The developers have no power to make planes drop cargo. Like the Melanesians, they can only march in formation. Without the right requirements, the product will fail.

Development teams realize this, and intellectually appreciate the importance of having someone give them good requirements. Some development teams go out and get requirements themselves – I would argue that another way of saying this is that on some teams, the developers are also playing the role of business analyst or product owner, on a part time basis. One of the responsibilities of the product owner is to “feed” the development team requirements – specifically good requirements.

There’s another problem here, though, that I see far too often. Product owners trapped in a cargo-cult of their own. When product owners (or product managers or business analysts) focus on following a use case template, or writing user stories according to the common sing-song structure, or create UML diagrams with proper syntax, they are just waving torches and hoping that the plane will drop cargo for them. Writing measurable, unambiguous, consistent requirements is necessary – but not sufficient. Grooming the backlog and doing estimation exercises or formulating a cogent SRS (software requirements specification) are the software development equivalent of carving a wooden headset when focusing on the needs of the market doesn’t happen.

Torches (thanks Vince Wingate for the photo)

The most important responsibility of the product manager (or product owner or whomever) is to assure that the team is solving the right problems. If the product manager is not sending the planes, it doesn’t matter how good the team is at lighting the runway – no cargo will be dropped. I’ve seen teams invest years of effort building the wrong products.

  1. Building the wrong features into products to expose capabilities.
  2. Building the wrong capabilities into products for the users to solve their problems.
  3. Building solutions to the wrong problems for their users.
  4. Building solutions to problems for the wrong users.
  5. Building solutions for users in the wrong markets.
  6. Building solutions for users in markets that aren’t aligned with their company’s strategy.

I numbered the list above intentionally. The lower the number, the more frequently I see teams addressing the problem. The higher the number, the bigger the problem. As a guy who “went agile” almost 14 years ago when developing software and leading teams, I see exactly the opposite of what would happen if we prioritized our efforts to build better products correctly. Teams are focused on the things that are easiest to fix (wearing uniforms and creating requirements artifacts) instead of focusing on the things that cause the biggest improvements (scheduling planes and identifying the right problems to solve). But the teams keep expecting cargo to fall from the sky.

Here’s the most valuable hour you can spend today – skip one of those meetings that monopolizes your day and try it.

Pick a feature (or story) from the “must have” section of your requirements document (or the top of your backlog), and walk through the following questions – each time you get a “Yes” you can advance to the next question. When you find a “no”, figure out what you need to do to change your answer to “yes”, and move to the next question. Can you get through all the questions in an hour? If not, where did you get stuck (chime in below)? If you made it through all of them, quickly scan the next few requirements (or stories) and ask yourself if it would be worthwhile for you to do the exercise again. My guess – somewhere along the way, you’ll open Pandora’s box. Imagine that you open the box, and engage your leadership team to actually fix the problems – how awesome will your product be then?

  1. For the story / requirement you picked – who is the user? Specifically, is there a user persona defined? Pick the primary user if there are multiple ones.  A requirement not intended for a specific user is a requirement that has no users.
  2. For this user, do you know what problem are they trying to solve when using the features / capabilities represented by the story? The features may be explicitly defined (e.g. “The system must allow users to make off-site backup copies of their photos.”) or implicitly (e.g. “As a photographer, I want to retain my photos if my device is destroyed.”).  You need to explicitly articulate the problems that those features are intended to help users solve.
  3. Are the capabilities represented in this story the best ones you can build to solve that problem? “Best” is defined as “makes the most sense right now” – you may be iteratively delivering incrementally more-capable solutions – that’s often completely appropriate – be pragmatic, not pedantic here.
  4. Is this problem the most important problem you could be helping your users to solve? It may be the most important problem to your user, or it may be the most important problem to you – perhaps your competition is winning deals because they are much better than you at enabling users to solve this problem. Perspective is key here – a perspective, or at least a hypothesis about the definition of “most important” is what matters.
  5. Do you have a rationale for why you are focusing on this user persona – and within that rationale, is this the right persona? Maybe this persona represents the bulk of your target market. Maybe this persona is a thought-leader who will drive adoption by other users, once you convince this user to love your product. Maybe this is a buyer persona, who you have to satisfy before your product ever gets in a user’s hands. Answering this question requires you to have an intentional approach for getting your product out there, and a definition of the “there” you want your product to be. As a quick litmus test - if you don’t know who the “wrong” personas are, can you know this is the “right” persona?
  6. Have you elected to focus on the right market?  Your persona represents a subset of your market (or your entire market, if you are hyper-focusing).  There’s a rationale for selecting the right market. It combines sizing the potential opportunity, assessing the risks of failure in the market, looking at ease of entry, and understanding the alignment of that selected market (for this product) with the overall company strategy. If you aren’t a single-product start-up that never intends to sell other products, market selection should be done from a portfolio perspective.

OK, so how did you do? If you’ve got “yes” answers all the way through the list of questions, your product might still fail, but at least you’re being intentional.

You’ve avoided the cargo cult requirements anti-pattern.

If you ran into a “no” somewhere along the way, you’re making the same leap of faith that the Melanesians made when they lit the torches.

Landing gear (thanks Henri Sivonen for the photo)

Will you be more successful than they were?

Post to Twitter Post to Facebook

This article was published on Tuesday, June 25th, 2013 at 8:33 am and is filed under Uncategorized.
You can leave a comment on this post

29 Comments

  1. I love this! Wish I’d written it myself! Retweeting and whatnot. Cheers!

  2. Great post. I’d add that the type of problem(s) you outlined above should also be determinative of the type of tool(s) a product manager picks up to address them. For example:

    Type 1 – Personas are probably the right tool
    Type 2 – Requirements are probably the right tool
    Type 3 – A series of prototypes are probably the right tools
    Type 4 – A combination of primary research (field reports, observations, experience studies), and personas, which feed into the roadmap are probably the right tools
    Type 5 – Components of the business plan, and marketing plans, plus personas, are probably the right tools
    Type 6 – The business plan is the right tool

    and I’m going to split out a new type to give you seven, since that’s a mythical number. Type 6 was defined as “Building solutions for users in markets that aren’t aligned with their company’s strategy.” I’m going to define type 7 as “You don’t know what your company strategy is or don’t have one.”

    If you have a type 7 problem, you need to reach for tools that will drive internal alignment around what you as a company want to be when you “grow up.” For example, the Igor-Ansof matrix or variations of it that (there are others).

    • Thanks, Paul!

      You know, I linked back to articles that link back to articles that include links to “how/when/why to use each of those tools.” You’ve undermined my business model by concisely writing in 7 sentences what I wrote in ~70,000 words.

      Seriously, though – thanks! Also, fantastic fantastic point about “you don’t have a strategy!”

  3. Awesome post. I think the overarching point is that answering the question “why are we doing this” is arguably more important than answering the question “what are we doing”. Without having a shared understanding of the answer to the “why” question, the team and the product effort are likely to fail.

    • Thanks, Roger!

      I completely agree – getting good at knowing why you want to build something is more important than getting good at knowing how you want to build something. By an order of magnitude or two.

      • Not to mention that getting good at knowing why, and staying focused on it, is a lot of work!

        I wonder if we all easily slip into designing solutions instead of asking for motivation, because the design piece, the “what,” is so much less slippery. You need to keep testing the “what” back against the “why,” and both are constantly in motion!

        For me the real world description of this is a conversation I had yesterday with a business analyst new to agile development who was feeling sad that he wasn’t invited to join the developers in designing the software. “What do I do to keep busy now?” he asked.

        He had a ton to do! The business people making the requests had been lazy about even asking for specific things, much less explaining what their motivation was, and there was a LOT of work still to do to lay out all the things they were asking for, placing them into a specific and testable operational context, and ask questions about the quantifiable value, the “SMART” goal to be achieved by doing what they had asked for. They had architecture artifacts before they even had an operating model, much less a compelling business case.

        This whole layer of work is invisible in most project teams I coach, because in their waterfall process, the people who feel the consequences of NOT doing it are far, far away when the “requirements” are being written. The teams are locked into a cycle of analysts writing a big “functional spec” and walking away happy, and underpaid testers are blamed months later for a “bad fit” or “defects” in the software. And then everyone is off to do “root cause analysis.” I mean seriously, if the analyst doesn’t understand the motivations in the first place, what are the chances that the software will work to meet the needs?

        Clearly agile helps everyone work together to avoid this mess, and fast iterations help to avoid serious misfires, but I think many of our teams (and maybe even many of us, myself included) prefer the discussions about architecture to the discussions about profit or operational cost, because the architecture is tangible to us and the P&L is not. Even with an agile process, bringing a living business case to the technology table every day is a very strenuous job.

  4. Like Elena I’m sharing the link with my followers (already included it in the last newsletter for subscribers of updates from my website).

    I’m always surprised with how often the “why” is ignored in organizations. Some time ago I was helping a team fix the problems with a big data project, and the first thing I asked was, “what are the goals for this project”? The answer from IT and the internal team couldn’t have been vaguer. “Begin with the end in mind” was definitely not a guideline they were following!

    Interestingly enough, lots of executives who own the budget for my projects frequently don’t know the answer either. I lost the count of the times when I asked what problem they were trying to solve (or what questions they were trying to answer with a dashboard), and the answer was extremely evasive.

    (I secretly enjoy when that happens at the beginning of a project, because it means I can take charge of identifying the real problem to be solved, and come back with a proposal myself: “this is what I think our big picture is, this how we are going to measure success, etc.” ;-)

    • Sweet – three slightly overlapping (I’m guessing) newsletters from three rock stars, all including this article. Instead of “follow Friday”, I think this is “famous Friday.”

      @Adriana – I love it when the problem is ill-defined and I get a chance to help. No “secretly” about it for me. :) Even when I’m working with teams that have crisply defined goals, I start with questioning the relevance of those goals (e.g. are those the most important things to be addressing now, and how does this fit into the bigger picture / longer-term view?).

  5. Adriana, that’s awesome! I agree. It is very fun when you get there in time to get them to have a project that is attempting to do something in particular, and you get to lay it out yourself!

    I had another thought about this topic today, based on my daughter’s recent (unauthorized) wrist tattoos–one reads “How” and the other “Why”. I’m glad she’s so clever and all, but seriously, wrist tattoos? But I digress.

    I’ve always thought the product owner in agile says “what” to build, and the team says “how.” I’m now thinking maybe it’s more that the product says “why” they are willing to pay some money–what need are they feeling? The team still says “how.” But “what” is the part that is always morphing. And of course that matches the way agile handles scope.

    Maybe there’s something to this? Great to see your face on this thread, Adriana!

    • We also have a recent tatoo-addition to the family :). She did all the “how” research before-hand (she knows her mom), and had a great answer to “why” as well (she knows me too, I guess).

      I think of the PO more as having the “why, in detail” responsibility (see response to @Roger), and the “what” responsibility is a solution-design responsibility (e.g. we chose this “what” because of that “why”).

      Some product owners also put on this “define the what” hat – but I would argue that they are filling a vacuum, because “someone” needs to do it, not that it is what they should be doing based on a role definition. It may be that the PO is also great at user-experience, process-design, software-architecture, and the other skills that help you create the right approach to solving a particular problem, but I view that as coincidence. The PO should be focusing on best guiding the “invent a solution” members of the team towards inventing solutions to the right problems. When the PO is part of the “inventors” group, the PO has less time to choose problems.

      None of these are absolutes – it comes down to who is on the team. But in the absence of that information, this is my starting position, from which each team should drift, as appropriate.

  6. Great to see you here too, Elena! I’m lucky to live near Scott so we get to interact in person from time to time :-).

    I’m laughing at your daughter’s unauthorized tattoos — at least she chose to showcase her cleverness in them, heh.

    Your comment about product owners reminds me of a question I have. I keep reading in agile blogs “the product manager is more customer-facing and the product owner more team-facing”. From this statement I conclude that the product manager is the role more focused on “why”, while the product owner is more focused on “what” and “how”. Do you know what’s the rationale behind this terminology? For me it’s curious that “manager” is used to represent the role with a broader span of control than “owner” has.

    (Disclaimer: Scott has shared before the notion that the “how” from a perspective may be the “what” of another. In line with my role, I always approach “why – what – how” from the perspective of building a software solution: why it’s being build, what will be in scope for the solution, how the requirements in scope are going to be implemented.)

  7. I got into a really stupid fight with a colleague at one point about “what is the product owner” versus “what is a product manager,” primarily because he had developed a product manager training he wanted everyone to use. I think there is a business team of some sort which looks at the market, looks at operational use of the software (if it is internal), and/or work with the team. To me, “Product Owner” encompasses the whole she-bang, so I don’t worry too much who calls himself/herself what. In the end, someone has P&L and operations responsibility, and that person (or those people) should be defining “why” money is being spent to software developers, and ensuring that the money is being spent well.

  8. @Adriana, I do agree that, in practice, product managers are more customer facing, and product owners more team facing. It’s possible but hard for one person to fill both roles.

    Regardless, with both roles, it’s essential to focus on the “why”. Great product managers turn to the market to determine, understand, and synthesize the “why”. Great product owners turn to the internal team to facilitate a shared understanding of the “why”.

    To be sure, many product owners also get into product design and even technical specifications. But some product owners manage to stay out of the weeds and concentrate mainly on the already-monumental task of facilitating an understanding of, and alignment around, the “why”.

    • @Roger (and @Adriana),

      I’m in the “both product manager and product owner” are focused on the “why” camp. I also agree, that if you measured time-spent, the PO will spend more time with the team, and the PdM will spend more time “outside the building.”

      Here’s how I described it yesterday in a discussion about org-design and RACI:

      Both PO and PdM are responsible for “why,” the difference is in time-horizon. The PdM is thinking across releases and per-release, the PO is thinking per-release and within a release (per-sprint, if that’s what you do). The PdM is focusing on the market with a broad brush, the PO is focusing on the market in detail.

      Understanding likely future competitive moves, shifts in customer needs, etc – the plate tectonics of continental drift – is the source of insight that drives vision and (vague) definition of what the product needs to be to be successful. Understanding nuances of customer behavior, immediate needs, current competitive positioning, etc – the contours of the land (as it is) – is the source of insight that drives smart decisions about the specifics of what is immediately important.

      Because communication is important, both PO and PdM communicate with both customers and the team building the product. But the proportions are different, given the different focii.

  9. @Roger: my curiosity is just regarding the choice of names (for me, it would make more sense for product manager to be the team facing person and product owner the cusotmer facing, but that could be due to the fact that English is not my native language, so the hierarchy seems to be inverted from the perspective of a Portuguese-speaking person :-).

    Agree that in both roles the focus on the “why” is crucial.

    • Here’s my theory on the names of the product owner and manager role:

      Both are the offspring of organizational encapsulation.

      Product Manager – a term that’s been around for a long time, formed by executives who wanted to delegate responsibility for “the product” to someone who “herds the cats” and “makes it happen.” Executives like the word “manager” so that’s what they used.

      Product Owner – a term that came from the agile, particularly Scrum camp, formed by development-centric folks who encapsulated the responsibility for “[communicating what] the product should be” to the team responsible for making it happen. Developers dislike the word “manager” so they avoided using it.

      Net-net, different points of view drove different names for overlapping roles.

  10. Wow, you know, it just warms my heart when great conversations like this happen – and I wake up to check the blog, finding this. It’s like I walked into the next room at a party I was hosting, to find a great conversation already in progress, and I can jump right in. Thanks for that – gonna be a good day!

    Replies to points / questions above in the branches of the thread. :)

    Also – I’m listening to Code Monkey by Jonathan Coulton right now too (http://www.jonathancoulton.com/wiki/Code_Monkey ). So fun!

    Thanks again!

  11. Great conversation, indeed, Scott!

    Your explanation of the difference of time-horizon makes perfect sense — and also the use of “owner” because a dislike of “manager”, even though I still think it’s a poor choice. To me, the title “owner” would be more appropriate for the person holding the budget and having a final say of what’s going to be built, and in what order, while “manager” could be the person making the per-release and per-sprint decisions, but oh well :-). In my projects using Scrum (obviously adapted) we don’t use the term PO.

    As for the “secretly enjoy” when there is no clarity around goals for a project, the reason I said “secretly” is that the head of IT in charge of implementing the solution is normally disappointed when he realizes the business asking for a solution can’t answer my questions about “why”. So, it would be rubbing salt in his wounds to openly celebrate the fact that I’ll be able to work on identifying the problem to be solved myself ;-).

    Undoubtedly, any BA / PO / PM should be questioning the relevance of those goals even when they are clearly defined. But in my experience, in the rare occasions the business has been through the exercise of defining goals (real goals, not “build a product” or “deliver project scope), they typically got it right…

  12. Scott, This is great food for thought. WIll be adding a link to your posting on our Operations Design & Delivery team Sharepoint site. We are already talking amongst ourselves about #1 requirements without a user and #6 focusing on the right market. We feel we are getting better at these things, learning lessons and moving forward in a better way with new projects. Cheers to you! -Glenn

  13. Its great list of questions Scott – the one thing I would change is that the blog post title sells well but doesn’t even hint at the contents.

    FYI This just got added to my 16 page reading list for new ScrumMasters

    Cheers
    Mark Levison

    • Thanks, Mark!

      I’m going to keep the title – it was intended to be provocative, and to have a correlation with the content (not be descriptive). Based on the positive feedback so far, it seems to be working. Also, thanks for adding it to your reading list!

  14. Scott, I couldn’t resist and added a mention to this article to my last post, which is based on two recent work experiences: http://bealprojects.com/no-matter-how-good-a-framework-or-technique-is-its-not-going-work-in-all-your-software-projects/

    For anyone who doesn’t want to click through, the intention of the post is to alert business analysts to the fact that most “good practices” are good under certain circumstances, so you can’t just assume winning techniques will still work when the circumstances change. Besides making sure the team is working on the right problem, as part of the due diligence in each project, I believe it’s important to pick the “right rituals” as well.

    I’m curious on whether you agree (including Elena and everybody else in this thread). Definitely because I work as an external consultant who’s accountable for getting the problem, solution and requirements right, I’m more used to seeing “wrong project rituals” (due to automatic adoption) rather than “cargo cult requirements” in my projects.

    • Hey Adriana, commented directly on your post, but I would say my executive summary is that there are two things that can go wrong. 1) going through the motions pointlessly or 2) thinking each technique has only one type of application. So for example, it appeared that you were saying that personas didn’t make sense in a particular project you were coaching, because you had real people to talk with. I think I might have used personas in that project, but gotten a different value from them than the one you might have looked for.

      While I agree completely that you shouldn’t just go through the motions (any motions) unless they are value-add for a specific project, I would also say that you should not dismiss specific techniques that are loved by their practitioners too soon, if you can find a new way to use those techniques. That’s how we got Christmas and Easter around dates where pagans already had holidays. Coopt rather than challenge, when you can, that’s my motto! :-)

      E

    • Thanks, Adriana! And for folks coming to this later – it is worth the click to read Adriana’s post :)

  15. @Elena: Thanks for sharing your views! I am 100% with you that techniques may have several applications (for example, I’ve successfully used user stories in projects following a “mini-waterfall’ model).

    The project in question wasn’t one I was coaching, but rather one in which I am the lead BA. Since we can go and talk to the stakeholders directly about what goals and behaviors they have, I still think personas wouldn’t add a lot of value there, but I’ve used this technique in the past and would not be against using it again in the future.

5 Trackbacks

  1. [...] Replacing “Requirements Gathering” with Something That Works Отличная статья Jared Spool о том, почему этап сбора требований из классических схем процесса проектирования интерфейсов — это миф и самообман. Он полагается на то, что заказчик точно и исчерпывающе знает требования к продукту и они не меняются по ходу проекта, что обычно не так. А Scott Sehlhorst подсказывает, как уйти от карго-культа в описании требований. [...]

  2. [...] Scott Sehlhorst recently published an excellent article that illustrates the dangers of focusing on “the rituals of good software engineering practices” without any assurance that the team is building the right product. [...]

  3. By Why Write Requirements | All About Agile on July 18, 2013 at 11:57 am

    [...] means to an end – creating the right product.  Don’t just follow some prescription and go through the motions of documenting requirements.  Take a step back and consider the following goals for your [...]

  4. [...] real problems – and without the why, we won’t find the real solutions. Or end up with Cargo Cult requirements – a very good post, go ahead and read [...]

  5. By Top 25 Project Management Blogs - oBlog on November 11, 2013 at 6:01 pm

    [...] In this blog, Scott Sehlhorst shares lessons he’s learned from his own project management experiences as well as from what he’s observed as a consultant for companies both large and small. Read First: Cargo Cult Requirements [...]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>