If I had more time, I would have written a shorter letter
The process of implementing DevOps principles into a business that has existed for decades is a difficult one on several levels.
These difficulties reveal themselves in multiple ways. The first of which is working around existing legacy systems that were developed at a time when the lessons of automation were still to be learned. Another is the culture change needed to invest in said principles.
Recently, I read an excellent article (which I cannot currently find) about introducing automation pragmatically and in stages.
Say a person encounters a recurring task. Given they've done this before, they start documenting the process so others can do it if they are not able. This is just simple documentation.
The next step may be to have some one-liners or something that can by copied and pasted in a terminal as a person would walk through the steps.
Next would be converting those one-off copy and paste jobs into a script that can be run. Once you've gotten there, the next step is just implementing this into a CI/CD pipeline or a Jenkins/Rundeck job.
I like this approach because it is at heart very pragmatic. These are all steps anybody can take to document and automate. It doesn't set the bar so high that would discourage even getting started.
Ruminating on all of these things, I realized that there is a tool that could bring somebody closer to this ideal.
I was a Python programmer for years and yet I never found a use-case for Jupyter Notebooks in my day to day work.
I know it is a very popular solution for data scientists to share collective information and I would imagine it would work great for training.
It wasn't until I had read the pragmatic article on automation that I thought Jupyter would be a great tool for developing documentation, troubleshooting, and automation notebooks.
Since Jupyter allows either cells that can contain Markdown or Python (or other languages), a person could simply create their documentation in a notebook in Markdown.
The next time that person goes to do the task, they review the notebook and perhaps add a Python cell that can be run to test to see if something is working like it ought.
Maintaining this cycle of small step implementations, you could end up with something that could be fully automated or at least a notebook that could carry a person a long way through troubleshooting a particular problem.
If in the end, you find you have an entire notebook of just Python script, pull it out into an Ansible playbook and you're good to go.
Another great benefit to this method is that the entire suite of notebooks can be stored in a git repository and made available to the entire team. Progress and be tracked and reverted if needed.
All in all, it's a great step toward the goal of automation and a huge benefit to documenting troubleshooting procedure.