How to quickly replace environment variables in a file
Tags: kubernetes, environmentvariables, envsubst, k8s
Published: 2021-05-22 21:14:02

---
title: How to quickly replace environment variables in a file
published: true
date: 2020-08-23 03:53:02 UTC
tags: kubernetes,environmentvariables,envsubst,k8s
canonical_url: 
---

![](https://cdn-images-1.medium.com/max/1024/0*D3l-Hk66J03OOCyD.jpg)

We know storing credentials or other sensitive values in a configuration file (e.g. Kubernetes yaml file) is bad, but how can we get values easily replaced without having to do a complicated string substitution or writing a custom Python script?

I often have personal environment variable files for projects that I use to store credentials and configurations in. Before working on a project I would the corresponding configuration file into the shell session. However, these files cannot be stored in git repositories or shared with coworkers or bots. Even worse, sometimes the repositories have files in them that need to be changed, which is dangerous, because it’s easy to accidentally commit these files.

Well as it turns out, there already is a good solution and it is called envsubst. We can use envsubst to substitute environment variable placeholders inside configuration files and we can even pipe it into other commands like Kubernetes' kubectl.

```
envsubst < config.txt
```

#### EnvSubst

The [envsubst](https://www.gnu.org/software/gettext/manual/gettext.html#envsubst-Invocation) is part of the [gettext](https://www.gnu.org/software/gettext/) internationalization (i18n) and localization (l10n) project for unix. It's usage is quite easy and I hope this will explain it.

> Some systems have gettext with envsubst preinstalled. However, if it is missing, you can install it using a package manager. For macOS you can use _homebrew:_

> brew install gettext

> Learn more about homebrew in [my article about setting about your development machine in one script](https://skofgar.ch/computer-science/2020/04/setting-up-your-dev-machine-in-one/)

#### Example

Let’s say, we have an existing configuration file, that want to give to someone or use with a bot. Ideally we don’t want to include credentials in that file.

```
# my configuration file
server: https://gitlab.com/skofgar
username: foo_user
password: mymonkey
```

#### 1. Create sample configuration file

We don’t want to check that information into a git repository nor do we want it laying around or send it to someone like this , but how can we improve this? Lets replace the information we don’t want in the file with environment variables:

```
server: $SERVER_URL
username: $USER_NAME
password: $USER_PASSWORD
```

#### 2. Configure environment variables

Then define these environment variables either by defining them in the shell session with:

```
export SERVER_URL=https://gitlab.com/skofgar
export USER_NAME=foo_user
export USER_PASSWORD=mymonkey
```

or save them to a file (e.g. .env) and then loading them into your current shell session by using source .env

> Make sure to use export, otherwise your variables are considered _shell_ variables and might not be accessible to envsubst

> Read more here: [https://ostechnix.com/difference-between-defining-bash-variables-with-and-without-export/](https://ostechnix.com/difference-between-defining-bash-variables-with-and-without-export/)

#### 3. Substitution

To run an actual substitution, perform the following commands.

```
> envsubst < config.txt  
server: https://gitlab.com/skofgar
username: foo_user
password: mymonkey
```

It is also possible to write your substitution to a new file:

```
> envsubst < config.txt > confidential_config.txt
```

#### Piping substitution into Kubernetes and other tools

It is possible to pipe the output into other commands like less or kubectl for Kubernetes (k8s).

```
# pipe into less
> envsubst < config.txt | less

# pipe a deployment "deploy.yml" into kubectl apply
> envsubst < deploy.yml | kubectl apply -f -
```

#### Conclusion

This is a great way to improve your _Continuous Integration and Continuous Deployment (CI/CD)_ pipelines or just simplify your own workflow. I use this for a project where we have to replace credentials that are temporarily needed, that we didn’t want to check into the git repository and that a CI pipeline also needs to access.

Let me know in the comments what you think of this solution and if you have come across an alternative approach.

Here’s a summary of all steps combined:

```
# print content of configuration file
> cat config.txt
server: $SERVER_URL
username: $USER_NAME
password: $USER_PASSWORD

# load environment variables
> source .env

# replace environment variables in file content
> envsubst < config.txt
server: https://gitlab.com/skofgar
username: foo_user
password: mymonkey

# replace environment variables and write to new file
> envsubst < config.txt > confidential_config.txt

# pipe into less
> envsubst < config.txt | less

# pipe a deployment "deploy.yml" into kubectl apply
> envsubst < deploy.yml | kubectl apply -f -
```

#### Sources

Some of the sources I used for this article are:

- envsubst for Kubernetes: [https://serverfault.com/a/843883](https://serverfault.com/a/843883)
- gettext documentation: [https://www.gnu.org/software/gettext/](https://www.gnu.org/software/gettext/)

_Originally published at_ [_Skofgar’s Blog_](https://skofgar.ch/dev/2020/08/how-to-quickly-replace-environment-variables-in-a-file/)_._