Dec 21, 2018

Agile Development Methodology: To Document or Not to Document?

agile-development-documentation (1)

If Agile were anti-documentation, why is there a manifesto?

The “Agile Manifesto” was put together by seventeen software developers in 2001 – among them Ward Cunningham, Jim Highsmith, Alistair Cockburn, and Bob Martin – and officially introduced the Agile development methodology as an alternative to the conventional documentation-driven, heavyweight software development process. It put forward the following fundamental principles:

  • Individuals and interactions over processes and tools

  • Working software over comprehensive documentation

  • Customer collaboration over contract negotiation

  • Responding to change over following a plan

As if anticipating that this simplicity might lead to misconceptions, the manifesto clarifies:

That is, while there is value in the items on the right, we value the items on the left more.

Even so, misinterpretations and confusion about Agile development methodology have persisted. “Over” got replaced by “instead of” and what was intended as “It is best to dedicate more attention to the software rather than invest time in overly detailed upfront documentation” seems to have turned into “Let's ditch documentation altogether and hope to remember everything we talk about.”

Dilbert_Training_Agile_Programming

Agile myth: “Documentation is not important”

Teams that follow the traditional approach to IT projects define and document requirements in the early stages of the project. They hand them in to developers later on, who then immediately start implementing it.

And while Agile development methodology was created as an alternative to this documentation-driven development process, it did not set out to eliminate documentation completely. It simply placed more value on working software than on comprehensive documentation because of the dynamic nature of software development.

So there is nothing in the Agile development methodology that inherently prevents us from creating as much documentation as the project requires. There are, in fact, situations in which documentation is absolutely required. Adding user stories to the backlog, creating flowcharts, drafting wireframes, documenting client meetings – Agile simply suggests being smart about it.

Documentation should be “just barely good enough” (JBGE). Too much or overly comprehensive documentation would be a waste of time, and developers rarely trust detailed documentation anyway because it's usually out of sync with the actual code. On the other hand, experience shows that too little documentation is always a source of problems with communication, learning, and knowledge sharing.

Reasons to invest in Agile documentation

The creation and maintenance of Agile documentation is a “necessary evil” to some and an enjoyable task for others. Either way, there are several valid reasons to invest time in it:

  1. Your project stakeholders require it. The creation of documentation is fundamentally a business decision, you are investing the resources of the project stakeholders in the development of the documentation, so they should have a say on whether their money is to be spent that way.

  2. To support communication with an external group. It isn't always possible to co-locate a development team and it isn't always possible to have project stakeholders available at all times. Shared documentation is often part of the solution in combination with occasional face-to-face discussions, teleconferencing, email, and collaborative tools.

  3. To support organizational memory. One of the principles of Agile Modeling is “Enabling the next effort is your secondary goal”, which is meant as a counter-balance to the principle “Working software is your primary goal”. That means that while you need to develop software, you also need to develop the supporting documentation required to use, operate, support, and maintain it over time.

  4. For audit purposes. Depending on the type of system you are developing, it might fall under certain audit guidelines. In that case, you would need to follow a defined process and capture proof that you did so, resulting in more documentation. However, if you really dig into what compliance requires, lightweight documentation usually works just fine.

  5. To think something through. The act of putting ideas down on paper can help you solidify them and discover problems with your thinking. What appears clear and straightforward in your mind can often prove to be very complicated once you attempt to describe it in detail, and you can often benefit from writing it down first.

While all of the above may be legitimate reasons to document, we always ask ourselves the question: What’s the least amount of deliverable that we need right now?

How to document things in Agile

4 rules of Agile documentation

Documentation in Agile is “living” and needs be collaboratively maintained by the whole team. To make the most of the time we invest in documentation, we follow a few simple rules:

  • Make sure it's always “just barely good enough”(JBGE).

    Keep in mind that any doc you create you will have to maintain later on. If the documentation is light, uncomplicated, and not too detailed, it will be easier to comprehend and update.

  • Write it “just in time” (JIT).

    Wait before documenting – it's the best way to avoid the false and outdated information. Produce documentation when it is needed, not before.

  • Keep it in one place and make it accessible.

    If documentation is only useful if it's accessible. Store your product documentation in a place where you and your team members can easily find it.

  • Collaborate with your team.

    Maintaining documentation in Agile teams is a collaborative process, and every team member should be encouraged to contribute.

To make all of the above possible, a flexible, transparent, and easily accessible documentation tool is needed. Unable to find a solution that would tick all the right boxes, a few years ago our team decided to scratch our own itch and build a better tool ourselves. As a result, today we manage all documentation using Nuclino, which serves as our internal company wiki, team knowledge base, and collaboration workspace. Our Help Center and this blog reside there as well.

requirements-document-nuclino

When to document

The iterative delivery of working software in Agile effectively replaces much (though not all) of the comprehensive upfront requirements documentation. The idea is to keep documentation as simple and lightweight as possible, especially during the early stages of the project. So at what point is it worth investing time in documentation?

A common agile practice is to defer the creation of all deliverable documentation as late as possible, creating the docs just before you need to actually deliver them. For example, system overviews and support documentation are best written towards the end of the software development life cycle (SDLC). Essentially, this approach is to wait until the information has been finalized before you capture it.

Agile-documentation-late

Another Agile documentation strategy is to document continuously. You write your deliverable documentation throughout the project, updating it as you update your code. This approach is in line with the Agile philosophy of always producing a solution that is potentially shippable.

However, because the requirements are likely to evolve over the course of the project, it takes a considerable amount of time to keep the deliverable documentation up-to-date. From the Agile point of view, this is “traveling heavy”.

In practice, Agile commonly put off writing documentation until they have time to do so, in effect moving towards the document-late practice even if they initially decided to document continuously.

Agile-documentation-continuous

What to document

Examples of possible documents used in Agile documentation include case studies, diagrams, tables, and sitemaps. Here are some of the documents you might want to consider creating over the course of your project:

Document

Audience

Description

Best Practice

Product vision

Senior Management, User Management, Project Management

A description of the core essence of a product and a summary of the current cost estimates, predicted benefits, risks, staffing estimates, and scheduled milestones.

  • Keep it short and to the point.

  • Write in bullet points.

Project overview

Developers, Managers, Maintenance Developers, Operations Staff

A summary of critical information relevant to the project, such as primary user contacts, technologies and tools used to build the system, etc.

  • Maintain it throughout development.

  • Keep it short.

  • Use it as a starting point for anyone new to the team.

Design decisions

Developers, Maintenance Developers, Project Managers

An overview of critical decisions related to design and architecture that the team made throughout the project.

  • Document them throughout other artifacts, such as system overviews and source code.

Operations documentation

Operations Staff

A description of the dependencies that your system is involved with, references to backup procedures, troubleshooting guidelines, etc.

  • Your operations department often has a standard format for this type of documentation.

Requirements document

Developers, Maintenance Developers,Users, User Managers

An overview of what the system does, including use cases, user stories, or essential user interface prototypes, etc.

  • Writing it early in the process is not advised.

  • Capture the requirements as executable specs.

Support documentation

Support Staff

Training materials specific to support staff, user documentation to use as reference, trouble-shooting guide, etc.

  • Like the operations department, the support team may have standard templates or examples that you can work from.

System documentation

Maintenance Developers, Developers

An overview of the system, including technical architecture, business architecture, and high-level requirements.

  • It helps ensure that if the development team leaves, critical information is left behind.

User documentation

Users, User Managers

Reference manuals and support guides for the users.

  • Keep it short and simple. The solution design is flawed if you need to train your users on how to use it.

TL;DR

“We embrace documentation, but not hundreds of pages of never-maintained and rarely-used tomes.”

— Jim Highsmith, History: The Agile Manifesto

The Agile development methodology is in no way anti-documentation. It simply reminds teams to document no more than necessary, and when necessary, keeping documentation as simple as possible. The idea is to choose a format and level of detail that allows change and delivers just enough value to keep the team moving forward in the right direction.

lossy-compressed