The Path to Creative Deployments
Tags: productivity, devops, management, career
Published: 2022-09-25 12:04:30

Photo by <a href="https://unsplash.com/@goian?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Ian Schneider</a> on <a href="https://unsplash.com/s/photos/creative-team?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText">Unsplash</a>

> **Note**:
>
> The diagrams shown are in code (mermaid), like the document itself (Markdown). The code is available [here](https://raw.githubusercontent.com/imjoseangel/sandbox/devel/docs/automation.md).

## Introduction - Fail Fast and Teams

Many Teams try to implement automated deployments for their applications or Cloud components. Most of them fall in the same hole: **Fail Fast** and **Confidence**.

Different reasons prevent a **Growth Mindset**:

* Non-sense strict deadlines
* Lack of confidence
* Complain of Failure
* Judging performance
* Lack of innovation

We can improve the manual creation of repetitive content by using Human creativity and Input.

### Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better. - Samuel Beckett

Automation requires time to start, creativity, investigation, and innovation. Failing fast is part of the learning process.

Check if the implemented process is *optimal and scalable*. Investigate alternative solutions and improvements. With the right mindset to *refactor* the process and the code, every Team will reach its full potential.

## From Human Intervention to Full Automation

Let's define a possible timeline of how could be the process from a manual run to a fully automated procedure.

### The first stage - Fear of losing a job

When a new technology knocks on the door, multiple reasons could stop a Team from progressing on innovation and adopting new techniques.

* The *damned* comfort zone
* Lost one's job
* False sense of control

Having the right coaching and time, Teams will gain confidence and improve the above procedure.

With these feelings over the table and other external factors, the best approach could be:

![First Stage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9jus2c26055rnn3rzklm.png)

It works and is automated. But there is still some field for improvement.

### The second stage - Starting the Dunning-Kruger Effect

At this stage, the Team starts to gain confidence and is pretty damn sure they know a lot. Automation starts to grow, and the Team initiates to learn new techniques. They use *GIT* and *pipelines*.

![Second Stage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fvp0rrgkwttyb14u97se.png)

It seems similar to the *first stage*, but not in the least. The process is:

* Centralized in a GIT repository.
* Runnable by anyone on the Team.
* It has a history log.
* It can be rolled back and traced.

#### Using a CI/CD Tool with inline code

> **Warning!!!**
>
> Having all the logic in the pipeline or using graphical options is not recommended.
>
>It is not scalable, has a lot of dependency on the CI/CD tool, and doesn't offer the advantages like rollbacks, traces, and logging given the *"as code"* options.
>
>It could be something like:

![Not Scalable](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k81wcxvqk8g35hty8su2.png)

### The third stage - Questioning our existing implementation

Hosting the code in GIT is a further important step towards automation.

Curiosity opens new ways of working with GIT, like *GIT Hooks*. The Team discovers how to apply Continuous Integration in their code. The original scripts could be custom *Bash*, *Python*, or *Powershell* and use *ShellCheck*, *PyLint*, or *PSScriptAnalyzer*. The **Pre-Commit**, **Static Analysis**, and **Pull Requests** have arrived to stay.

![Third Stage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/awxj2db0qtp6254sjjkc.png)

Check if the automation in place fits with the following rules:

* Can Create new components
* Can Modify existing components
* Can Remove existing components

In this stage, thoughts about recovery speed and rolling back in case of disaster emerge.

### The fourth stage - Realizing there is still a long way to go

With Continuous Integration, the process changes dramatically. The Teams start to look for new automation techniques and think about changing their scripts for standard tools like *Terraform*, *Pulumi*, *Ansible*, or *Chef*, among others. Using a tool like *Terraform* doesn't mean the Team will improve its process. Compared with Bash, the *Terraform* code is just another script launched from the local machine.

**Speed** is becoming an important asset, and the referred external tools help to achieve it. Most of these tools are prepared to run different jobs in parallel.

**Security** is key to automation, and with standard tools, it is possible to do **Static Security Analysis** before running the deployments.

**Refactoring** is a new word and becomes part of the process. The Team realizes it is time to review the existing implementation and learn from previous mistakes.

*GIT* is now the standard, and the Pull Request Process is improving. The Team starts to implement other testing and quality gates before the approvals.

![Fourth Stage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q8pgwgs77jfzbhy217pp.png)

### The fifth stage - The Grown-Up Mountain

**Continuous Delivery** and other techniques are now knocking on the door. The Team is out of its comfort zone and starts to investigate and learn new ways of improving. Probably, it is too early for Continuous Deployment.

**Tags** and **releases** are part of the process, just as creating **artifacts** and pushing to the respective servers.

**Documentation as code**, **Containers**, **Dynamic Security**, and **Integration Testing** is part of the improvement plan.

**Makefiles** facilitate self-documentation and serve as a startup point for all the implementations.

The diagram is now larger with lots of components. The process is becoming better and better:

![Fifth Stage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7rz4ral8tw7625fujg24.png)

Everything should be [modular](https://en.wikipedia.org/wiki/Orthogonality_(programming)) and repeatable now. As examples:

* Jinja2 Templates
* Code generation Templates
* Golang Templates
* CookieCutter
* Modules, Roles, Providers, or Packages in Terraform, Ansible, and Pulumi

### The sixth stage - Plateau of Productivity

In this stage, **GITOps**, **APIs**, **WebHooks**, **Trunk Based**, or **Containers** are part of the game. But also **Observability**, **Chaos Testing**, **Continuous Deployment**, or **Canary and Blue-Green** deployments, among others.

The whole process can be summarized like this:

![Sixth Stage](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c5imzddsk3w4rod9h7mh.png)

Don't stop here. **Be more creative**:

* Deliver Automation as a Service
* Deliver APIs for Self-Service
* Add central logging
* Add orchestration monitoring
* Packetize solutions with Containers

## Summary

Hitting excellence is the result given by the confidence gained by the Team during the whole process, but also from the business.

**Automation doesn’t mean taking away the creative process.** Investing time in new ways of working and techniques is essential to continue improving. Getting out of our comfort zone, discussing and applying our learning to our job are symbols of a well-balanced Team.