Boosting Your Development Feedback Loop with Preview Environments
Tags: devops, docker, testing, cloud
Published: 2022-06-07 09:20:28

# Boosting Your Development Feedback Loop with Preview Environments

The process of developing a new feature for a project usually involves the same steps:

- Create a **new branch** in the repository for the new feature (feature branch)
- Commit the code and create a **new Pull Request**
- **Review** the code changes
- **Merge** the Pull Request into the base branch
- **Deploy** the base branch to the Staging environment
- **Test** the feature (QA) in Staging
- **Approve and deploy** to Production or **request changes** opening the PR again.

If you follow *git-flow* (or an alternative flow based on feature branches) this will sound familiar to you. The main point here is that until you don't deploy your new feature to the Staging environment it can't be tested in a realistic environment (local environments don't count 🙃 ).

Also, this workflow leads to the following questions:

- What happens when the team is developing multiple features at the same time?
- Is the code in the base branch always deployable (in terms of business logic)?
- What if two new features are merged and tested but the team only wants to release one of them and delay the other one?
- Which feature is responsible for a bug found in Staging if two or more feature branches were merged into the base branch?

As you imagine **this way of working with branches and environments produces situations which would be better to avoid**. The solution is to implement **Preview Environments** (sometimes also known as Preview Deployments).

A Preview Environment allows us to have a new complete and isolated environment for each new feature. The advantages of having Preview Environments are:

- Test features **independently**: preview features
- Improves **feedback loop**
- **Avoid merging into the base branch** until the feature is approved
- **Avoid mixing new features** in the Staging environment
- **Preview changes** as they would look in Production

The following diagram reflects different environments: Production, Staging, and one Preview Environment for every active Pull Request

![Preview environments in a Project](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vht4bnptj30cr6ynvui1.png)

## Lifecycle

Preview Environments are **disposable environments**, which means that they are created, updated, and destroyed. They have the purpose of testing a feature, and **once the task is done it doesn't make sense to keep them alive**. This process can take from minutes to months, it depends on the size of the feature, the time it takes to test it, if it makes sense to merge the Pull Request at the moment due to business requirements, etc.

When a new Pull Request is created **a new Preview Environment is created automatically** with the name of the Pull Request.

If the Pull Request is updated with new commits then it is **deployed again** to show the latest changes.

When a Pull Request is merged or closed the environment **is destroyed automatically**.


![Preview environments lifecycle](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k0stuohelcslvmlf73zf.png)

To be able to test a Preview Environment, every environment must have a unique preview URL. This URL can be automatically generated from the branch name. e.g.: for a branch named `feature/user-register` in a project named `Manhattan` the preview URL could be `https://feature-user-register.manhattan.com`.

## Environment Data

Another advantage of Preview Environments is that **data can be seeded and updated without affecting other environments**. It means that every environment must create a new exclusive database when the environment is created, and delete the database once the environment is destroyed.

The initial data of this database can be cloned from the Staging database or seeded with default values. **This makes the data in this environment independent from other environments**, so QAs can create, update and delete anything they need without worrying about affecting other QAs testing other features. Also, the data in the environment can be reset to the initial values as needed.

![One database for every Preview Environment](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/15g47vim1oik06d4vx06.png)

## Service Dependency

When we think of a new feature in a project we tend to think about a new branch in a repository, this can be usual in projects with a single repository and a single service, but this is not the common case in modern projects where multiple services are involved.

**A new feature could concern a backend service and a frontend service** at the same time. For the new feature, the frontend could depend on a new endpoint developed in the backend. Therefore we need a way to coordinate both repositories if we want to test the new feature. We can't just create a new Preview Environment for the frontend and then use the API available in the Staging environment because the new endpoints in the backend won't exist.

**A good practice is to use the same branch name in both repositories**, e.g.: `feature/new-feature`. Then we could deploy both backend and frontend to the same Preview Environment. By doing this we could be able to test the new feature completely and the frontend will use the backend endpoints of the new feature.

![Services in Preview Environments](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h1kmzhu1u4h6onpulxwp.png)

## Setting up Environments

Creating a Preview Environment can be done in many different ways. Creating the environment depends on the design of the project infrastructure and the way of deploying the application to Production. In this article, I am going to cover two options: deploying the app code directly to a server and using Docker containers.

### Deploy code to servers

This process deploys the code to a server using methods like uploading the code using SFTP via SSH. As the environment doesn't exist it requires some steps before being able to deploy the code. Supposing we already have an existing server dedicated to Preview Environments, the step to create a new Environment would be:

- **Install and set up** everything needed to run the code to be deployed: Web server, Database, Language Runtime, Virtual hosts, etc.
- **Create DNS records** for the preview URL(s)
- **Deploy the code** using SFTP

This way of creating Preview Environments has the following **problems**:

- **Slow**: Installing and setting up services takes a lot of time. Also, if the code of the project has hundreds of files the deployment process will be slow too. We want to have these Preview Environments ready as soon as possible.
- **Complex**: There are a lot of steps to maintain. We need an orchestration tool (like Puppet, Chef, or Ansible) to execute all the steps required and maintain that code in the future.
- **Error-prone**: Sometimes setting up a virtual host can fail due to a misconfiguration of the webserver or a repository with an external dependency can be down. If the process fails the environment won't be created.

### Containers

A **much better alternative** is to use Docker containers. Supposing we already have an existing server dedicated to Preview Environments with Docker installed, the steps to create a new Environment would be:

- **Create DNS records** for the preview URL(s)
- **Build the application container** in the CI/CD service
- **Upload the container** to a container image registry
- Connect to the server and **run the Docker container**

This process seems to be more agile than before, but there are still some questions to solve:

- How are the Docker services exposed as external services? We would need something like a container to act as a reverse proxy with forwarding rules.
- How do we request the issue and manage the renovation of SSL certificates using [Let's Encrypt](https://letsencrypt.org/)?
- How many servers do we need for Preview Environments in our organization?
- How do projects map to Preview Environment servers in the organization?

It would be great if we could have a cluster of servers and we could forget about the specific server where the code must be deployed. If we need more resources we can just add more servers to the cluster. **The best alternative to achieve this is to use a container scheduler like Kubernetes**. We just need:

- **A Kubernetes cluster** in a provider (like DigitalOcean Kubernetes, Civo, Google GKE, Amazon EKS, etc)
- **Write Dockerfiles** for every repository
- **Write the Kubernetes manifests** to deploy every service

Writing Dockerfiles and Kubernetes manifest is out of the scope of this article.

Instead of using a provider to launch a maintain your own Kubernetes cluster **you can use existing services to achieve the same result**. One of my favorite options is [Okteto](https://www.okteto.com/preview-environments/) **Kubernetes service** and their feature to create Preview Environments from our code. This allows us to **forget about creating and maintaining the Kubernetes cluster**, dealing with SSL certificates, and creating DNS records so that we can **focus on what provides value for the project**.

## Conclusion

- Using only environments like Staging and Production is not a good idea
- Having an independent environment for every new feature improves the feedback loop of the team
- Getting feedback fast means you decrease the time needed to deploy a new feature to Production, and also the costs associated
- Using containers and schedulers like Kubernetes simplifies the creation of Preview Environments
- Working with external services like [Okteto](https://www.okteto.com/preview-environments/) makes the whole process easier to create and maintain