Whether you're answering support tickets or troubleshooting software, infrastructure technology (IT) operators need to follow actionable procedures in order to achieve results.
These step-by-step instructions help new and experienced IT employees quickly address issues — and always get the right outcome.
Here's why you need runbooks on your team.
What is a runbook?
A runbook is a documented, sometimes automated, procedure outlining how to complete a frequent process in the IT function.
Runbooks act as references to automate or enable employees to follow processes, resolve issues and properly use the tools available.
A thorough runbook usually includes the following:
- Overview. Give a brief description of the method of procedure or service.
- Authorization. Determine who has access to the runbook and in what capacities.
- Steps in the procedure. Enter information about the installation and deployment of all necessary protocols.
- Monitoring system data. Step-by-step instructions for activating all available monitoring system alarms are provided.
- Prepare for disaster recovery with these strategies. Assemble all the service-level agreement documentation as well as the escalation methods and incident response documentation that are needed.
- Technical manuals. Any configuration, analytics, or other crucial system information can be referenced or included.
Runbooks tend to focus on one specific task, offering a step-by-step breakdown of actions and outcomes.
What is the difference between runbooks & playbooks?
While their names are often used interchangeably, runbooks and playbooks are very different.
- Playbooks are comprehensive solutions that focus on big-picture objectives.
- Runbooks tend to concentrate on workflows for a specific task.
For example, you wouldn't include detailed descriptions of how a system works in your runbook's list of tasks.
💡 Learn more: Runbooks vs. Playbooks
When should you use a runbook?
In certain cases, runbooks are used for routine processes — like removing sensitive data from a production database and putting it in an isolated development environment. Another runbook could support debugging a service outage or problem.
Runbooks might also automate system and application upkeep. For example, database backups and modifying access rights of access rights are two routine actions you might document in a runbook.
They're also a lifesaver for emergency response protocols. We recommend making a list of your common processes and then listing them in order from highest to lowest priority. Runbooks will give your subject matter experts (SMEs) the chance to pool their collective knowledge and experience.
This means less need for escalation and higher operational productivity for fewer on-call IT staff.
Types of runbooks (+ Examples)
The amount and type of runbooks you have available to employees depend on your company's size and maturity.
Here are the three primary runbook types you'll likely see.
General runbooks for routine tasks, like everyday system processes and activity logs.
These runbooks help the IT staff handle any low-level tickets coming from the team. You might have a general runbook for:
- Performing backups.
- Evaluating audit logs.
- Maintaining system performance.
... or any other standard practice. An agent can refer to the runbook when responding to a ticket or troubleshooting a problem.
Here’s a general runbook example on installing Apache, helping agents automate the process of installing the web server.
2. Specialized runbooks
Specialized runbooks are for more complicated processes and emergencies. These runbooks help the IT staff with more advanced and intricate issues, like disaster recovery.
You can create specialized runbooks for cases like:
- A server outage.
- Hardware failure.
- Damage control.
- Network failure.
- Security breach.
Think of specialized runbooks as emergency escape routes to quickly mitigate risk and minimize damage.
You can also specialized runbooks for specific use cases. Here’s a specialized runbooks example of hardening an Ubuntu server. This runbook automates an Ubuntu server’s security configuration to protect it against viruses and threats.
3. Automated and non-automated runbooks
You can also create runbooks with different levels of automation.
Here are the three main types of runbooks depending on the degree of automation:
- Manual: involves zero automation — Operators follow step-by-step directions in an operations manual or runbook doc.
- Semi-automatic: involves some level of automation but still requires the operator’s role.
- Automatic: fully automated runbooks programmed to run without the operator.
When possible, automate redundant, easy tasks. It's best not to automate for:
- One-off edge cases.
- Complex processes.
- A high rate of change.
Compliance or other issues outside the system also may need to rely on human judgment or approval. If there aren't enough developers to properly automate the process, then it might have to be done manually.
Elements of an effective runbook
Here are the five characteristics that make a great runbook.
- Actionable. It serves as a record of what has to be done in the event of a crisis.
- Accessible. Members of the group are aware of its whereabouts.
- Accurate. It contains accurate and up-to-date data.
- Authoritative. Each MSP or IT documentation or procedure has its own runbook.
- Adaptable. Modifying it to eliminate future duplication is a snap.
Let's examine each of these elements.
Actionable: a runbook detailing specific tasks
Any operator should be able to follow along and execute the processes outlined in your runbook. Here's how to make your runbook actionable:
Ensure that the runbook's actions build toward one objective
Each runbook should have a clear list of step-by-step instructions that build toward a single outcome.
Remove things that don't help that goal. Runbooks should be written in a way the end-user can follow along with.
Keep in mind that a runbook for a beginner to your system will differ from the runbook for a senior SRE who knows your system well.
Clearly define and separate each step
Whether you're using bullet points, spacing or hierarchical text, make sure it's easy to follow from one step to the next.
Only include necessary information
Runbooks don't require bulk text or extra detail. Keep your document concise and focused on the task at hand.
Add screenshots or other helpful visuals when possible.
Use automation when possible
Using automation where it makes sense. This might include improving system visibility, making changes or condensing whole runbooks into a few commands.
Accessible: locating the proper runbook
Make your runbooks easy to access. This is particularly valuable when the runbook helps manage or investigate an outage.
Support each alert
In practice, a runbook should support each alert. Even when the runbook only contains generic instructions, it can be built upon as a project progresses.
Add searchability to your runbooks
This increases runbook usability and accessibility.
Regardless of where they're stored, both the content and metadata should be accessible.
When deciding how to make your data searchable, consider whether the end-user will be using the command line or a chat system (such as Slack or Microsoft Teams).
Secure your confidential information
Be careful not to save credentials in a runbook, and be mindful of access levels.
You can do this by using secure tools to build your runbooks.
For example, Scribe, a process documentation tool, uses OpenAI LLM APIs, which has a strict set of limitations on how data can be used. Scribe doesn't allow OpenAI to use your data to train our or their models. All data is discarded after 30 days, so it is processed but not stored by OpenAI and is only used for purposes of service provision.
OpenAI even has a signed data protection agreement with Scribe that binds the vendor to the same or more rigorous security controls that Scribe implements (e.g., data encryption, backups and retention measures, etc.).
Accurate: Runbooks containing the correct information
You can easily steer users in the wrong direction with improper instructions, leading to confusion and costly mistakes.
Loop in subject matter experts
Make sure that your go-to people are involved. Have them use tools like Scribe so they can quickly build the runbooks themselves without losing any time. Or, have them participate as a stakeholder that ensures the runbooks are verified and accurate.
Add a runbook approval process
This is a great way to loop in your go-to people without having them create the runbooks themselves. Several tools offer feedback functionality, so you can easily submit drafts for review before they're published.
Make runbook updates as straightforward as possible
The goal is to reduce the amount of time it takes to make changes to a runbook while maintaining its accuracy.
This will vary from doc to doc: changes to a runbook that explains how to set up a development environment should be evaluated more thoroughly than changes made to a runbook that reports a difference in your production database.
Document all runbook updates
Keep in mind a runbook's latest update date, as well as the date it was the last run. Track this via links to event notifications or Slack chats. A runbook can additionally include a final instruction to update the "last run" column.
Editing a runbook can cause it to overflow. Don't lose sight of the big picture. Adding references after splitting runbooks is not anything to be concerned about.
Authoritative: A runbook containing reliable information
Focus on one process at a time
Remember, each runbook should focus only on one specific process. Outline the actions in a clear and concise manner.
If the process follows an innumerable number of steps, consider breaking it down into even smaller runbook documents.
Combine connected runbooks
If you have several processes for a larger task, you can combine them via link or using a tool like Scribe Pages.
This ensures that your runbooks can function on their own, but are easy to sort according to the process or system.
Categorize your runbooks
Use a standard nomenclature for all related runbooks. Avoid synonyms to reduce potential confusion.
If possible, store relevant runbooks in a single file or Page so they are easy to pull.
Archive redundant documentation
If more than one runbook exists for a particular scenario, you should consolidate them into a single document and archive the others.
Imagine following a set of instructions only to discover that they're out of date or inaccurate and that the proper runbook was actually the second search result. It's not only irritating, it's terribly nonproductive.
Depending on the size of your business, you may want to create a feedback mechanism that allows anybody who comes across an outdated or redundant runbook to notify the owners of the problem.
Adaptable: Runbooks should be able to adapt to changing circumstances.
Almost all software systems stay in a state of flux and evolution. If your runbooks don't follow suit, they won't be reliable enough to make an impact.
There are several ways to foster adaptation, such as:
Clearly assigning roles
Make sure you have a clear runbook owner, system owner and other stakeholders. Socialize these responsibilities so that colleagues know who to connect with if they have questions or feedback.
Create a feedback loop
Include runbook updates as part of your software release and incident management post-mortem.
Make sure that you have mechanisms in place to update your runbooks as needed. Encourage team members to either recommend or make changes as processes or systems evolve.
Note: It's vital that your security and regulatory compliance are not compromised. Set permissions as needed.
Set regular audits
Review your runbooks quarterly for accuracy and functionality. Deal with broken runbooks in the same manner as a broken test, and make sure that pruning your runbooks and deleting stale data becomes a normal part of your software documentation and development.
How to write a runbook
Here are X easy steps on how to create a runbook.
Step 1: Planning a new runbook
There are two things to keep in mind while creating a new runbook.
- What are the most prevalent issues or tasks that your team encounters on a regular basis?
- In the past, what have been the most successful methods of dealing with those problems?
In order to decide what you need a runbook for and the type of runbook to use, review previous incident reports and post-mortems. We also recommend requesting input from your team.
Start with the most common issues or jobs, and then prioritize according to importance and urgency level.
🎓 Related Resource: How to Write an Operations Manual
Step 2: Write and templatize your runbook
Once you've decided on the technique for your runbook, you can start writing. Most runbooks stick to these core details.
Each step in the runbook should include these core data fields. Build a template so that you can easily duplicate and repeat.
- Task ID.
- Task name.
- Task details.
- Task description.
- Task owner.
- Team executing the task.
- Duration of completing the task.
To get started, here are some things to keep in mind.
- Stick to the point - don't go into too much detail.
- Use simple language in your documentation so it's easy to comprehend and follow.
- Customize your document to reflect your specific business procedures.
- Make your runbook adaptable to your systems and applications
- Ensure uniformity across all platforms. Provide the operator with all he or she needs to know about each one.
The last step before submitting your runbook for review and approval is to put it to the test in the real world.
Step 3: Test, update & improve your runbooks
Runbooks need to be checked and updated on a regular basis to guarantee that they continue to work at optimal levels, regardless of whether your systems or applications have changed.
The most effective characteristics of a runbook are flexibility and ease of adaptation to the dynamic nature of the IT operations environment.
Free runbook template
Kick start your runbook documentation with our free runbook template. Use it to reduce errors and ensure your systems and applications are running smoothly.
Conclusion: Start building runbooks today
Runbooks, even when automated, provide a permanent record of a process or operation. Make sure your runbooks are:
... so that they can successfully support your evolving IT team.
Want to build and share visual runbooks in seconds? Generate runbooks for free!