Agile Documentation

Agile values working software over comprehensive documentation – it is 1/4th of the original manifesto. That doesn’t mean don’t document! It means don’t document more than you need to document. Documentation does have value, but the practice of documenting got excessive – that’s why a reaction to the bad stuff earned a spot as one of the pillars of agile. How do you avoid over-reacting when changing a culture of over-documentation?

The Need for Documentation

We need documentation to help us communicate. You can define an agile team as “the people creating the product.” You can interpret “creating” as the people we normally think of as executing to accomplish the goals, or you can be more inclusive, and think of the people who have the goals as being part of the team too.

Philosophically, agile stresses collaboration. Usually it is couched in terms of (1) people who are executing collaborate to devise the best solutions and (2) people who are part of the team collaborate with the people for whom they are creating the product. Personally, my experience has been that projects with committed sponsors succeed, and projects without them fail – so I always think of the sponsors as part of the team.

Regardless of your definition – collaboration requires communication, and communication benefits from documentation.

Temporal Communication

Communication among groups of people happens over time.

Some collaboration is transient – communication happens right now, and is only important right now. Other communications are persistent – the collaboration happens right now, but we need to remember later what we agreed upon and why. There are variations of “right now” and varying degrees of “later”, but slightly oversimplifying,

• There is communication for now.
• There is communication for later.

Both are important.

Documentation, in the vision from your nightmares – giant requirements documents, architectural analyses, and detailed psychographic profiles – is very inefficient when communicating for now.

These massive documents, while getting in the way of a conversation, do provide the opportunity to remember (in the future) why we make the decisions we make (today).

Documentation, as emphasized in most agile discussions – user stories and acceptance criteria, on an index card, taped on the wall – is not a robust solution for communication that happens later.

This low-overhead approach to capturing today’s ideas actually facilitates and improves today’s conversation – but does not give us a record we can use in the future for why we made our decisions.

As an agile team, there are ways we can approach both the persistent and transient documentation tactics to make them more valuable. We can tweak our transient documents to make them more useful for persistence. We can take an agile approach to development of persistent documents, so that we only do just enough documentation – trading off the minimum amount of short term efficiency to avoid long-term blunders.

Transient Documents for the Future

The image above is from a workshop defining the product backlog for an initial sprint for a new project. The team identified the most important stories for the first sprint, their acceptance criteria, and the size, in story points, for each story. As part of scrum, the stories are written in user-story format, with acceptance criteria for each story identified and documented on additional index cards (taped to the index cards for the associated stories). Great transient way to capture the must do and makes it better stories that the product owner is thinking about.

As a team, we made a couple minor tweaks to make these story cards more useful. First, all of the stories were organized into themes that help establish the context (for the team), and combine the stories into “meaningful areas of focus” for communication with stakeholders.

Each story got a blob of color in the top left hand corner, identifying which of the five themes were being supported. You can see a few of them circled in the image above.

We also developed a set of 8 proto-personas (prototype, early draft, very rough personas – almost stereotypes) that the overall product / project was designed to support. Each persona got a different color star.

Each user story got a corresponding colored-star sticker to indicate for which persona the story has been written. Specifically, when multiple personas could use a story, we identified for which persona we were implementing the story in this sprint.

In the “right now” world, these tweaks helped us understand, question, and confirm the themes and personas being supported in the first story. Those conversations allowed us to reach pragmatic compromises about which stakeholders were going to benefit and when. They also allowed us to craft the outbound message that is step one of stakeholder expectation management.

The list of themes changed while we were doing this – moving from 4 to 5 themes, as we realized that there was value in splitting one of the themes. The group of proto-personas grew from an initial set of 3 to a set of 8, as we quickly realized that the team was trying to “build something for everyone” and we wanted to avoid the elastic user problem, and instead, roll out capabilities sequentially that satisfied valuable groups of users with similar goals and perspectives.

Culture Change for Stakeholders

As you would expect, having a bunch of cards stuck on the wall provides a great, visceral, now communication for the team – both in implementing and communicating. However, cards on the wall is very transient. It also doesn’t provide a good way to communicate with old-school stakeholders (imagine the execs who have their admins print out their emails for them to read and annotate – those folks still exist!).

This can make it hard for the “not part of the implementation” part of the team to collaborate and communicate.

In User Stories Applied, Mike Cohn stresses that the brevity of user stories is intentionally designed to facilitate (and I would add “dependent upon”) conversation. I find it to be both a strength and a weakness of user stories. It is strong because you can cover a lot of ground quickly (breadth) and capture a number of stories, much like the list above. It is weak because the requirements documentation does not stand on its own – it requires conversation to fill in the details. I’ve had some success using the “Verify that…” user acceptance tests as the method of documenting those details (depth), in conjunction with the brief, easily consumable stories.

Stakeholders in a Barrel

Those conversations happened as part of getting the stories defined, getting them on the board in the right order, and defining the acceptance criteria. Stakeholders often have a short memory, especially when they were the ones making a compromise. You need some way to retain the context of the conversation that led to the particular organization and content of the stories on the wall.

Persistent Documents in the Present

Agile proponents complain about the massive get-in-your-way documents that slow down the start of projects, prevent the teams from adapting to changing market conditions, and otherwise make it harder to deliver great products. I agree. Agile proponents also rail against the “build it all, then release it,” waterfall, process for creating software – and propose an alternative – iterative development of the software. I agree again.

Why not apply the same principle of iterative development to persistent documents?

Consider the example above of using colored-star stickers to identify the people for whom each story is being implemented in any given sprint. The majority of the time spent on this analysis is around understanding which people are important (to the success of the product), and for which people the ability to perform this user story is important (to the person). The combination of these two importances is an input to prioritization. A minimal amount of time is spent putting stickers on cards, and creating multiple cards for the same story (each with a different sticker for a different persona and potentially different acceptance criteria).

Add a tiny bit of overhead, and capture the information in a spreadsheet (or whatever).

The image above shows a mapping of use cases to actors. It could just as easily show a mapping of user stories to personas. The transition from thinking about actors to thinking about personas is a small one. This chart comes from an earlier article on communicating a delivery schedule with use cases – it helps stakeholders get a big picture view of who benefits when – a user-centric, yet still top-down perspective.

Note: Another nuance to incremental delivery is that you can introduce the “bare bones” version of a story now, and the “better” version of it later. You can also communicate that stories get better over time, for some users, with the same type of chart.

A very similar, and also simple to create table can show how each theme is getting “representation” within each sprint. That view facilitates great discussions about trying to emphasize one theme and then another sequentially, versus doing the most important items in each theme first – gradually making progress in each theme. Creating that view, and using it to communicate, has helped me address concerns that “external” stakeholders have shared about how a team appeared to be “chaotic and random” in their approach to prioritization and implementation.

Many Models

There are many types of documentation – and many types of models for emphasizing and discussing particular aspects of a complex system (like users and goals and solutions). Creating a simple diagram like the following whiteboard sketch is almost required for effective transient communication in some projects.

[from Simple Agile Model Example]

When most folks are complaining about documentation, they are not complaining about the sketch above. They would consider that sketch to be augmenting a conversation. Fine. That’s what I did, at the time. Then I took a picture of it. Suddenly, it was documentation. A stakeholder came by later, confirmed that the system needed to work this way, and signed the whiteboard – and the photo was updated.

Every heavyweight document can start out this way. And it can evolve. It should evolve. If you’re going to succeed, it must evolve as your team gets smarter, your competitors react, and your market changes.

The trick is that you don’t have to write the final version before you get started. Capture at a high level what you know right now – just like a roadmap. Capture in just enough detail what you need right now – just like a story backlog. And change it as you go.

Conclusion

Documentation is not bad. Documenting stuff you will never use is bad. Documenting stuff you don’t need for a long time is risky, because it will probably change before you refer back to your document.

Documenting why you made decisions right now, as part of transient collaboration and communication is important, so that you collectively remember. That documentation persists and your team can build upon knowledge, incrementally – just as your team builds upon the evolving code base, incrementally.