Why Process Documentation Matters

Working in a company with well-defined processes and workflows is a common preference. While some thrill-seekers enjoy the chaos of early-stage startups, most people simply want to get their work done efficiently and have time left for their personal lives. It's frustrating to miss out on a great evening with friends because you’re stuck working overtime due to poor company processes.

As a manager, your responsibility is to help your team work productively, efficiently, and with minimal stress. I want to focus on one important aspect: the significance of process documentation for your project and its positive impact on your team. This advice is primarily intended for team leads, scrum masters, product and project managers who want to improve their teams' workflow in software development. However, these tips can be applied across various industries.

So, what makes a leader? In essence, a leader is someone within a company's hierarchy who sets the direction and organises the team's efforts within their scope of authority. A powerful way to establish this authority is by documenting processes in a centralised space. The person who writes the rules is the one in charge.

This can be likened to how governments operate. We have rules to follow, and there are consequences for not adhering to them. Over time, these rules may change as the system evolves, but they continue to define what is expected from each individual. A similar approach can be applied at the company level, and that’s what I want to explore in this article.

The Value of Process Documentation

Let’s start with a basic software development process. There are several steps needed to get things done:

1. Preparation of business requirements.
2. Preparation of technical specifications and designs.
3. Requirements estimation.
4. Release planning.
5. Implementation.
6. Testing and stabilisation.
7. Release delivery.

As a manager, you need to find ways to move through this process more smoothly and efficiently.

• Were our estimations accurate?

• Are there any infrastructure issues?

• What does the release delivery day look like?

Agile retrospectives can help bring some of these problems to light and allow you to discuss them with your team. Perhaps some issues get fixed, and the next release improves over the previous one. However, you may still find yourself repeating the same mistakes, which is a great opportunity to review your process documentation.

A clear sign that something is missing in your processes can be seen in the following example:

• Dev: “I finished my task on time and committed everything. I don’t know why testing started so late! If we’d found this problem earlier, we would’ve had plenty of time to fix it! That’s why the release was postponed.”

• QA: “We would have started testing earlier if we’d known the task was complete. During the status update, the dev team mentioned having issues with this feature and said they needed extra time to sort it out. Late on Wednesday evening, they closed the subtask without updating the main task’s status or notifying the QA engineer. On our dashboard, the task was still marked as ‘in development,’ so we had no idea we were supposed to start working on it.”

Even from this dialogue, several questions may arise:

1. How should the dev team close their tasks?
2. Is it necessary to have both a task and a subtask?
3. How should the QA team be notified when a dev task is complete?
4. Should this notification be manual, or can we automate it?
5. How does the QA dashboard function, and are they working with the correct work items?
6. Do we need separate dashboards for each sub-team?
7. Why did the dev team finish their task late on Wednesday evening? Perhaps it’s worth discussing in the next one-to-one meeting.

This example may be exaggerated but is, unfortunately, all too real. In some companies, processes might sound fine, but during iterations, even experienced team members can forget to update something, miss a status change, or fail to mention someone important in a task. We’re all human, and in a high-pressure routine, it’s natural to focus on doing your work well while occasionally overlooking the connected ‘bureaucracy’ that helps the team function smoothly. This is especially true in software development, where many brilliant engineers are deeply engaged in creating something meaningful.

The team might know the rules established in previous iterations, but everyone can understand and prioritise them differently. Someone might be on vacation during a sprint when new policies are decided, and not everyone has the ability to intuit all changes. Especially those known only to the release manager.

In general, traditions and the team’s native understanding of the development process work well but have certain limits. They allow software to be released and can even be effective, especially if there’s a manager with enough time to facilitate the process. However, to reach a new level of speed, quality, and team capability, this isn’t enough. This is where proper process documentation presents a great opportunity for improvement.

Process documentation should help you and your team understand what needs to be done at each stage of the development process and who is responsible for each step.

Documentation coverage can be divided into two layers:

1. Schematic.
2. Process instructions.

In an ideal process, both are necessary, but these activities can be highly time-consuming. That’s why the first stage is schematic coverage.

Schematic coverage

The first step in any project should be schematic coverage - a graphic representation of your process. This coverage should provide the team with a general understanding of the established workflow and answer key questions:

• What steps does our process include?

• Where am I within this process?

• What is my responsibility in this process?

It’s unrealistic to map out every single detail that occurs in the development process, and it may not even be necessary to illustrate every subprocess. Fortunately, this can be broken down into different levels as needed.

Basic Schematic Level

It’s not necessary to use complex graph notation or specialised software for the basic level. The process should be clear to your team and convey the main idea of your iterations. In my opinion, one of the first tasks for the team manager is to publish this level.

For example, if we have a two-week development cycle, a basic structure like this can be created in any spreadsheet application to get started:

Here are some insights for both current and new team members:

• Our development process spans two weeks.
• We start our iteration on Wednesday, with one week for development and one week for stabilisation, aiming to finish by Tuesday morning.
• The product team must prepare design and requirements before implementation begins. We don’t work with requirements created during the sprint.
• The dev team should address tech debt in the first week.
• While the dev team works on feature implementation in the first week, the QA team should complete test design tasks and be ready for testing.

This structure should be displayed as the key illustration in the team’s public space within a wiki system like Confluence, Notion, or SharePoint. If the company doesn’t currently have a wiki system, this is an excellent opportunity to establish one, as it’s essential for future improvements.

• One of the biggest impacts of this image is that it prompts additional questions from the team:
• But when, and who, needs to maintain and switch branches?
• Who’s responsible for managing the tech debt backlog?
• You’ve missed DevOps tasks!
• The product department should be added to the implementation stage because we assist with tasks 1, 2, and 3…
• This table is overcomplicated. I don’t understand anything from it!

And that’s how process documentation begins to evolve. This structure can and should be updated to reflect the real process, but it’s important not to overcomplicate it with details. It represents the primary concept of the process. Everything beyond this should be handled at other levels and with additional tools.

This initial step took only thirty minutes, yet it established some foundational rules for the whole team. The basic level indicates that if we’re still in the implementation phase by Thursday or Friday of the second week, there’s clearly an issue, and we might need to push some features to the following sprint.

Detailed Schematic Level

According to the response at the basic level, there should be dozens of questions about how to handle specific cases or communicate with other departments. And here lies a controversial area. And here’s the tricky part: you and your teammates could dive into drawing detailed processes for every department, but it’s highly time-consuming, and the impact isn’t so obvious.

This approach is possible in large corporations with resources for methodologists, but in most cases, it’s a waste of time and electricity.

What should be covered at the detailed level:

1. Certain communication processes that are stable and can be as quickly outlined as the basic table above. These processes serve the same purpose: to show the core rules of the subprocess without an overload of details.
2. Task workflows.

The team should have a ticket and tracking system, such as Jira or Microsoft Azure DevOps, containing work items with specific workflows. Documenting schematic rules for statuses and responsibilities at each stage of a work item is essential. It’s better to have articles for each type of work item. Start with the most common, like Task or Product Bug, and go deeper from there. Even if standard processes are used, it’s crucial to describe what each stage represents.

Process Instructions

Schematic coverage helps keep the process moving but still misses many small tasks that can be overlooked during an iteration. This means that these smaller tasks should be documented in some way as instructions, with a designated responsible person. The biggest challenge at this stage is that creating proper instructions takes a lot of time, so it’s not necessary to cover every detail. Process documentation can, however, be broken down into levels:

1. Check-list level.
2. Instruction level.

Check-list Level

Check-lists help organise work during each stage of the process. In schematic coverage, we outlined the sequence of stages for the development process. At this level, we can specify tasks and checks that the responsible team member should complete to finish each stage.

The depth and complexity of the check-list level depend on your team’s experience. If the team is experienced and generally knows what to do, the check-list can be built based on past oversights:

• In the last release, we discovered that we forgot to publish the updated product documentation. Let’s add a checkbox for this on a public page or in the release task.
• In the following release, we found some changes that had been made without a task and overlooked by QA. To prevent this, let’s add a check for commit reviews.

After a few months, you will likely have a proper list of activities to check at each stage. If, on the other hand, you have a new team, members borrowed from other teams, or team members who are not yet experienced working together, it’s better to create a basic list of activities for each stage yourself and gradually add new points as issues arise, similar to an experienced team.

If you have a new team, borrowed members, or people who aren’t yet used to working together, it’s best to create a basic list of activities for each stage yourself. Gradually add new points as issues come up, just as you would with an experienced team. It’s also beneficial to discuss your ideas with each responsible team member to ensure the list is useful for everyone. For a new team, it’s essential to outline the expected work process from the start. For an experienced team, it serves as a way to fine-tune certain processes.

It’s also helpful to add some basic steps later, even for an experienced team. Team members may change, and these steps will be useful for new members if there’s available time. It’s also essential to keep this list up-to-date, balancing between completeness and avoiding unnecessary details.

A check-list can provide a few insights:

1. If everything is on track, team members will spend only a couple of minutes checking off completed tasks. This means the process is working well and serves as a reminder, prompting us to think about new ways to improve.

2. If a team member forgets or cannot complete a task, it presents a valuable opportunity for improvement:

   1. The task might be too complex.
   2. The task might be unclear.
   3. The task might have started in the wrong stage.
   4. The responsibility might be misplaced, and the task should be assigned to someone else.

This is a simple framework to guide future meetings and refine your processes. Sometimes unclear and overly complex tasks can be broken down into several check-list items. But sometimes, that’s just how the process works, and this is when it’s better to move to the next level - the Instruction level.

Instruction Level

This is the most detailed level, suitable for complex subprocesses. The instruction level describes everything that needs to be done, right down to the specific buttons that should be clicked in company systems.

The necessity of this level is best illustrated with an example:

Our product can be distributed as either SaaS or On-Prem software. The SaaS version can be deployed automatically, but the On-Prem version requires manual installation by the Delivery team. In the release check-list, there is a task stating, “The product release distributive should be uploaded to the Delivery team’s local storage”.

It’s manageable when this is done biweekly, and everything is clear. But what if the release manager goes on a planned vacation, or there’s a new release manager? Which distributive should be used? Where is the Delivery team’s local storage? Is there a standard for naming directories there? Where can I get permission to upload to this folder? What should I do if I accidentally upload the wrong version?

This is an ideal process to document with instructions. At this level, redundancy is acceptable because a thorough and detailed instruction promotes independence - a critical goal for an effective process. The absence of one manager should not disrupt the workflow, allowing everyone to complete the task successfully.

Here, we should create “shall” statements specifying which role is responsible, where and how to obtain the distributive, how to connect to the local storage, and how to upload it correctly. A troubleshooting section could follow, detailing how to obtain permissions and what to do if issues arise with the storage. Each instruction should have an assigned owner, an update date, and an author’s changelog.

Writing these instructions is time-consuming but provides more than just a sequence of actions. As you write, you may start asking yourself, “Why do we do it this way? Maybe we don’t need double checks, and it’s possible to automate this?” At this stage, many process issues can come to light, opening up a whole new backlog of improvements.

A key tip at this stage is not to be afraid to document the process as is if it somehow works. Even if you know something could be improved, it might not be easy to immediately define a better process for your team. When you draft a new version, test it in the next release cycle before establishing it as a permanent rule. Bearing in mind that engineers are bound to be a bit grumpy about added bureaucracy. As a manager, your job is simply to gauge if their response is more or less “fine”.

For approved instructions, it’s useful to create links between schematic coverage, check-lists, and detailed descriptions to ensure clear and easily accessible documentation.

Process Methodologist

All the aforementioned levels of process coverage require a dedicated person responsible for process improvements - the Process Methodologist. This person should be actively involved in the workflow and understand it from within. For this reason, I believe it should be a part of someone’s current job, rather than an additional position.

It’s neither necessary nor advisable to write all process documentation yourself or assign it to a single team member, but it’s essential to have one person overseeing and ensuring the quality of the documentation.

Process Methodologist Responsibilities:

1. Creating and maintaining templates and structure for each level of process documentation.
2. Identifying areas for project improvement and creating a backlog for process enhancements.
3. Defining tasks for new process documentation artefacts.
4. Ensuring quality, completeness, and consistency of process descriptions, both within each level and in relation to the overall process.
5. Keeping process documentation up-to-date as improvements are made.

These tasks suit roles like Release Manager or Scrum Master, and it can be useful to include them in the job descriptions and responsibilities for these roles.

If there’s no extra time available for someone to act as a process methodologist, a well-organised team can often create process documentation on their own. This happens quite often among software developers, who document approved patterns and style guides. While this approach works well, and having this documentation is better than not having it, a person who can create a cohesive structure for this knowledge can significantly enhance process flow and the onboarding experience.

Using the Processes in Team Life

Writing a process is one thing. Implementing it in the team’s daily life is another challenge. If you’re starting a process with a new team on a fresh project, this can go smoothly. However, integrating a new process culture into an existing team is often more difficult. There’s no universal solution. It largely depends on the maturity of each team member and the team as a whole.

The first step is to ensure that processes are accessible to everyone. In task management systems like Jira or Azure DevOps, there is typically a project description page. This is where you can place a link to the process documentation. The same applies to corporate wikis, such as Confluence or Notion, which store requirements and other company documentation. Your documentation space should be published on the main pages of these systems. It’s also helpful to notify the team via email or channels in corporate messengers and to mention it in the daily meeting. The team should know that documented processes are available for them to review.

The next step is to integrate processes into tasks. The Definition of Done should include a link to the relevant process, detailing how, when, and according to which standards the task should be completed. When closing a task, the team member should understand that the outcome will be evaluated based on process documentation. Over time, this creates a system where employees look to process documents for guidance on completing tasks in line with company standards.

If part of a process is working well and the documentation is clear and approved by the team, the final step is to contact the HR department. Discuss formalising these work terms and processes in an official company document or incorporating relevant parts into job descriptions.

All these steps are necessary to achieve a few main goals:

1. Increase process transparency. Everyone is treated equally, with clear rules describing how to perform tasks.
2. Improve team manageability. It’s easier to plan your responsibilities and the work of your team members when the scope is clearly defined.
3. Reduce dependency on key employees and managers. Team members, or even you, can leave the company without risking the business’s continuity. If processes work seamlessly without you, that’s a sign of well-designed processes, and it means you’ve done your job right.

An additional benefit of process documentation is its usefulness in resolving work conflicts. It’s easier to provide feedback based on written rules rather than personal impressions. The same applies to conflicts between divisions. If someone implements a process incorrectly, or if a process misses a specific situation, it’s an opportunity for improvement. That’s great! It means there’s a new idea on how to make things better for everyone.

Minuses of Proper Process Documentation

As with any theory, documenting processes has its own limitations that need to be mitigated.

1. It’s time-consuming.

   This is a long journey, and while I highly recommend undertaking it, at the beginning, you have a myriad of topics to cover. It’s hard to estimate the total investment in this activity, and it can quickly overwhelm. The tip is to use levels of coverage - start with a simple graphical process representation and go deeper, iteration by iteration.

2. Overly detailed processes can stifle creativity and demoralise talented team members.

   Depending on their level of experience, people might receive tasks as strict instructions at the beginner level, or more open-ended directions at a higher level of proficiency. Overly developed processes and rigid instructions can lead to burnout, as employees may feel there’s no room for growth or innovation. Even when they have bright ideas, they may feel confined by overly complicated procedures.

   However, in general, good processes help the whole team work efficiently and avoid long hours. If you have well-documented processes but still work on weekends, then your processes are flawed - you’re doing it wrong. This perspective can help illustrate the value of good processes, even to skeptics. Those team members with big, bright ideas should become your contributors and allies in building an ideal way of working.

   Don’t be afraid to try new methods, but remember not to change the process drastically more than once every six months (and even that is a minimal period). Otherwise, you’ll create chaos, and the team may start looking for a company with better processes. Small changes, however, are fine.

3. Less motivated team members may use the absence of certain process descriptions as an excuse for failures.

   It’s impossible to cover every detail from the start, and there will always be some gaps. There are simply too many scenarios. In a company culture that relies on written rules, there may be employees who claim they missed a deadline or failed a task solely because something wasn’t documented, and they were too hesitant to make a decision. Even if you address this gap in the next iteration, they may find another one.

   This can actually help identify problematic team members, as the overall team performance often falls to the level of the weakest member. Perhaps they are burned out or underqualified for their tasks. Either way, this insight can support decisions that keep your team in good health. The absence of certain process descriptions rarely demoralises healthy, motivated team members.

• It requires strong motivation from the process methodologist.

  If the methodologist doesn’t believe in or isn’t committed to improving the process, this activity can end up being a waste of time. You need someone who envisions ideal processes and is motivated to work towards that vision. If this person is one of your team members, it might be worth reconsidering their role or even restructuring your team. If it’s you, it may be time to explore new career opportunities or assess for burnout. Sometimes, a good vacation can help reignite the desire to improve processes.

Summing Up: Key Steps for Effective Process Documentation

Process documentation can serve as a framework to improve team performance and build reliable workflows. To summarise, let’s review the main steps in this process:

1. Identify the Process Methodologist. This should be someone who is ardent about process improvement, actively engaged in identifying gaps and passionate about refining workflows.
2. Decide where to store your process documentation. It should be easy to access, ideally using existing company tools to avoid additional costs.
3. Publish a basic scheme of your process to establish the documentation workflow and gather initial insights.
4. Publish workflows for the most frequently used work items.
5. Create check-lists for tasks required to release a new software version, assigning responsibilities for each task.
6. Publish detailed instructions for complex tasks.
7. Continuously observe your process over iterations. Add new schematic diagrams, update check-lists, and expand the instructions library as needed.

Proper documentation should reduce stress levels and support improved management methods, ultimately leading to a successful and prosperous business.