Some days back I was having a conversation about the importance of memos and documents in an agile/fast-moving startup company. The person I was talking with was surprised with my thoughts and asked me to send some links so they could follow up and share with others in the company. Unfortunately, after some searching around, I did not find anything online that was comprehensive enough to include all my thoughts. So, here I am trying to be good to the internet by filling in the void on a topic that is important but maybe not “cool” enough for tech leaders to spend much time sharing. To be clear, I don’t claim to be an expert on this, and, I would love to hear your feedback.

I have been leading technical teams for many years now, and something that I have learned to appreciate is the importance of… writing documents. Yes, you heard me right. I know what some of you might be thinking now: this guy has not read the Agile Manifesto and doesn’t understand that in-person meetings are much better than documentation or that code and tests are the best documentation for any piece of software. As a matter of fact, I have reached this conclusion after many years of trying many flavors of agile methodologies, both formal and informal. I now believe that having the right documentation is precisely one of the best tools to be agile and move fast. As Eric Ries describes in his Lean Startup book, the main goal of a past-paced organization is to maximize learning rate. And, in order to maximize learning rate, you need to use data and document your decisions.

But, you don’t need to trust my word on this though. The most agile and innovative companies that you can probably think of, from Netflix to Amazon, and also including Facebook, Google or the likes, they all use a flavor of what I will call “The Memo Culture”.

The Memo Culture

The most extreme and clear example is Bezos’ Amazon, where every meeting needs to be preceded by a short document written by the organizer of the meeting (see here, for example). The document is shared right at the meeting and the first few minutes of every meeting are used for attendees to read the document. There is no expectation of anyone reading the document before the meeting. There are different goals of such process at Amazon:

  • Make sure that any meeting that is called is actually needed. If you, as the organizer, are not willing to spend a few hours preparing a document it is very likely the meeting is a waste of time.
  • Make sure the meeting has clear goals and focus
  • Make sure everyone in the meeting has the right context before the discussion starts
  • Use the memo as documentation of the meeting and its final resolutions

I have seen several variations of this particular culture. Most companies won’t really require a document for every single meeting, but rather establish a subset of meetings that do benefit from them. I find this to be a less extreme and more productive process. That said, it does have the downside of not strictly leading to a reduction in meetings.

Another variation that I like is sharing the document before the actual meeting. If you use an online collaborative tool like Googledocs or Quip, starting the discussion on the document before the meeting can be extremely useful. As a matter of fact, I have seen many times situations where the discussions on the comment threads in the doc where so useful that a meeting was no longer needed. The other benefit is that discussions on the doc give a clear preview to the organizer of what elements need more clarification and discussion during the meeting and avoid the need to start with an open-ended discussion to gauge general opinions. The obvious downside is that following this process you are not guaranteed to reduce the overall time spent on making a decision.

It is also important to address the two elephants in the room: Powerpoint and meetings.

I have seen organizations using Powerpoint (or equivalent) as their main documentation format. I hope I can easily convince you that this is plain wrong. Powerpoint is designed as a one-directional tool for a presenter to backup their presentation with visuals. Period. Any extension of this is likely to fail miserably. A set of slides, for example, tends to maximize visually appealing presentations and leave out many details that will be filled in by the presenter. Trying to reconstruct those details from only the slides is most of the time next to impossible. Also, slides are not a great vehicle for collaboration and online discussion. So, please, do your organization a favor and do limit the use of Powerpoint to external presentations where all the issues above do not matter that much. If you don’t trust me on the potential evils of Powerpoint, you can read Wired, New York Times, and Harvard Business Review writing about it.

Regarding meetings, I know that some agile methodologies promote the value of “in-person communication and I agree there is some truth to this. However, it is also true that many organizations that attempt to become “agile” end up throwing their employees into a non-sustainable cadence of meetings. If your technical people are spending over 20% of their time on meetings, you probably have a problem (see this article for some good thoughts on the problem with meetings heavy cultures). Collaborative documentation is a way to avoid this. Remember the Bezos rules for memos is precisely designed to avoid useless or costly meetings and to make them more useful. Yes, excessive useless documentation should be avoided, but the same is true for excessive useless meetings. And, the good news is that both these things tend to balance each other if managed right.

Documents used in the memo culture should be treated as “living documents”. This means that they should go through as many short iterations as possible during the initial phases. Those iterations should include comments and feedback from all stakeholders (see Jeff Bezos’ recommendation on how to write good memos). It is likely that the discussions on the document might lead to some meetings where those can be addressed and discussed in person with the relevant participants. Even once the initial iterations on the document are finalized, it should not be considered as “closed” since new revisions could come at any point in time.

This gets me to the important question of how to keep documents updated. I unfortunately don’t have a silver bullet for you here. I totally understand why developers prefer to keep documentation close and coupled to the code to avoid them drifting apart and updated. That is hard to accomplish with other kinds of documents. My suggestion here is to treat documents as an evolving entity and use versioning and change logs as much as possible. Furthermore, as you will see in the next section, some documents (e.g. postmortem or AB Test design) never become outdated. You should push your organization to write mostly evergreen documents and avoid as much as possible those that are likely to become easily outdated.

Some example documents

Software/system design doc

A software/system design doc will usually include the following sections:

  1. Use cases
  2. Requirements
  3. Architecture
  4. Tools/libraries/components
  5. Risks/unknowns
  6. Expected timeline

The software/system design document is used by many organizations. You will easily find many example of templates online designed by anyone from NASA to the EU Commission. Keep in mind that you want to keep your document light, and alive. You do not need a 100 page document unless you are literally designing a rocket from scratch.

Build vs. buy doc

In any case, the build vs. buy documentation should include the following:

  1. Supported use cases and high level requirements
  2. High level description of in-house alternative (can be skipped if this is part of the system design doc)
  3. Third party alternatives
  4. Pros/cons of (2) and (3)
  5. Decision and rationale

Post-mortem

A post-mortem document document usually includes the following information:

  1. Date of the incident and timeline
  2. Summary of the incident and impact
  3. Root cause summary
  4. Detailed description
  5. What worked well/could have worked better
  6. Corrective actions

See this great post for more details on the postmortem process as a whole and pointers to post-mortem templates.

AB Test design doc

The purpose of the AB test design doc is to make sure all the stakeholders are on the same page about what is being tested and how it is being tested. It is not very different from a traditional science experiment design document really. As always, the document should be initially considered as a vehicle for feedback and discussion and eventually evolve into being documentation that should be useful years from now.

Some of the sections an AB Test Design doc should include:

  1. High level description of the goals
  2. Hypothesis that are being tested
  3. Primary and secondary metrics
  4. Cohort selection and duration of the AB test
  5. Engineering design or pointer to software/system design where this is described

See this example of AB Test design document from Optimizely.

AB Test results doc

That being said, the goal of this doc/section is to explain the details of the results of test, document the final decision and its rationale. As such, the document should include the following:

  1. Test result on primary/secondary metrics
  2. Interpretation/discussion of the results
  3. Final decision & next steps

Data Analysis doc

The Data Analysis doc is, in reality, a more general purpose document than the AB Test Results one. You can use it to document anything from exploratory data analysis using product data to market research. It is hard to standardize the format because its contents can be very diverse. That said, any data analysis doc should probably include:

  1. Goal of the analysis
  2. Description of the data analyzed including possible shortcomings, etc…
  3. Data analysis results (of course with lots of pretty graphs and plots)
  4. Recommendations

Conclusions

More where this came from

Follow our publication to see more product & design stories featured by the Journal team.

Cofounder/CTO at Curai (AI for healthcare). Former Quora VP, Netflix Director. Software, Machine Learning, Data, Recsys... From Barcelona, in the Valley