How to Manage Your Secrets with git-crypt
Tags: security, webdev, devops, tutorial
Published: 2020-10-07 13:16:49

Many software projects use secrets - usually, keys to external APIs or credentials to access an external resource such as a database. Your application needs these keys at runtime, so you need to be able to provide them when you deploy your application, or as a step in preparing your deployment environment.

In this article, I'm going to show you how to use [git-crypt](https://github.com/AGWA/git-crypt) so that you can safely keep your application secrets in your source code repositories, even if they're public.


### The Problem With Application Secrets

Most projects have some sort of secret keys or credentials. For example, if your application is hosted on [Heroku](https://www.heroku.com/), you might provide an API key to your Heroku application using a command like this:


```
$ heroku config:set API_KEY=my-sooper-sekrit-api-key
```


By running this command before you (re)deploy your application, you give it an environment variable _at runtime_ called `API_KEY` with the value `my-sooper-sekrit-api-key`. However, keeping track of these secret values _outside_ of Heroku (or wherever you deploy your application) is still a challenge. 

I always try to set up my projects so that I can run a single command to deploy them from scratch without any separate, manual steps. For our example, this means I need to store the value `my-sooper-sekrit-api-key` _somewhere_ so that my deployment code can use it (in this case, to run the `heroku config:set...` the command above).

My project source code is always stored in git and usually hosted on github.com or bitbucket.com or some other source code hosting service. I _could_ store my API_KEY value in my source code repository, however, there are some downsides to this:



*   I can't share my repository with anyone else unless I'm comfortable with them accessing my secrets. This means all my application repositories with secrets in them need to be private.
*   Presumably many staff members at Github/bitbucket/wherever would also have access to my secrets, which I might not be okay with (depending on the secret).
*   It's easy to forget about the secrets in a private repository if I later choose to make it public. So I could accidentally disclose important secrets.

I could store my secrets somewhere separate from my application source code, but this has its own problems:



*   I need a way to get my secrets from wherever they're stored, at or before deployment time, and give my deployment code access to them.
*   My secrets may not be stored as robustly as my source code. For example, I could keep secrets in a `.env` file on my laptop, and make sure I never check that into the git repository. However, if I lose that file (such as if my laptop gets damaged/stolen), then I also lose that secret.


## git-crypt

Git-crypt aims to solve this problem by encrypting your secrets whenever you push them to your git repository, and decrypting them whenever you pull them. This happens transparently, from your point of view. So the secrets are in cleartext as far as you and your deployment code are concerned, but nobody else can read them, even if your source code is in a public Github repository.

Let's look at an example.


### 1. Install git-crypt.

There are instructions for Linux, Mac, and Windows on the [git-crypt install page](https://github.com/AGWA/git-crypt/blob/master/INSTALL.md)

If like me, you're using a Mac with [Homebrew](https://brew.sh/) installed, you can run:


```
$ brew install git-crypt
```



### 2. Create a new git repository.


```
$ mkdir myproject 
$ cd myproject 
$ git init 
$ echo "This is some text" > file.txt 
$ git add file.txt 
$ git commit -m "Initial commit"
```


Now we have a git repository containing a single text file.


### 3. Set up the repository to use git-crypt.


```
$ git-crypt init
```


You should see the output:


```
Generating key...
```


Before we do anything else, please run the following command:


```
$ git-crypt export-key ../git-crypt-key
```


This command creates a copy of the git-crypt symmetric key that was generated for this repository. We're putting it in the directory above this repository so that we can re-use the same key across multiple git repositories.

>By default, git-crypt stores the generated key in the file `.git/git-crypt/keys/default` so you can achieve the same result by running `cp .git/git-crypt/keys/default ../git-crypt-key`

This `git-crypt-key` the file is important. It's the key that can unlock all the encrypted files in our repository. We'll see how to use this key later on.


### 4. Tell git-crypt which files to encrypt.

Imagine our application needs an API key, and we want to store it in a file called `api.key.`

**Before** we add that file to our repository, we will tell git-crypt that we want the `api.key` file to be encrypted whenever we commit it.

We do that using the `.gitattributes` file. This is a file we can use to add extra metadata to our git repository. It's not specific to git-crypt, so you might already have a .gitattributes file in your repository. If so, just add the relevant lines—don't replace the whole file.

In our case, we don't have a .gitattributes file, so we need to create one. The .gitattributes file contains lines of the form:


```
[file pattern] attr1=value1 attr2=value2
```


For git-crypt, the file pattern needs to match all the files we want git-crypt to encrypt, and the attributes are always the same: `filter` and `diff`, both of which we set to `git-crypt.`

So, our .gitattributes file should contain this:


```
api.key filter=git-crypt diff=git-crypt
```


Create that file, and add and commit it to your git repository:


```
$ echo "api.key filter=git-crypt diff=git-crypt" > .gitattributes 
$ git add .gitattributes 
$ git commit -m "Tell git-crypt to encrypt api.key"
```


I've used the literal filename `api.key` in my .gitattributes file, but it can be any file pattern that includes the file(s) you want to encrypt, so I could have used `*.key`, for instance. Alternatively, you can just add a line for each file you want to encrypt.

It can be easy to make a mistake in your .gitattributes file if you're trying to protect several files with a single pattern entry. So, I strongly recommend reading this section of the [git-crypt README](https://github.com/AGWA/git-crypt#gitattributes-file), which highlights some of the common gotchas.


### 5. Add a secret.

Now that we have told git-crypt we want to encrypt the `api.key` file, let's add that to our repository.

>It's always a good idea to test your setup by adding a dummy value first, and confirming that it's successfully encrypted, before committing your real secret.


```
$ echo "dummy value" > api.key
```


We haven't added `api.key` to git yet, but we can check what git-crypt is _going _to do by running:


```
$ git-crypt status
```


You should see the following output:


```
    encrypted: api.key 
not encrypted: .gitattributes 
not encrypted: file.txt
```


So, even though the api.key file has not yet been committed to our git repository, this tells you that git-crypt is going to encrypt it for you.

Let's add and commit the file:


```
$ git add api.key 
$ git commit -m "Added the API key file"
```



### 6. Confirm our secret is encrypted.

We've told git-crypt to encrypt, and we've added `api.key` to our repository. However, if we look at, nothing seems different:


```
$ cat api.key 
dummy value
```


The reason for this is that git-crypt transparently encrypts and decrypts files as you push and pull them to your repository. So, the `api.key` file looks like a normal, cleartext file.


```
$ file api.key 
api.key: ASCII text
```


One way to confirm that your files really are being encrypted is to push your repository to GitHub. When you view the `api.key` file using the GitHub web interface, you'll see that it's an encrypted binary file rather than text.

An easier way to see how the repository would look to someone without the decryption key is to run:


```
$ git-crypt lock
```


Now if we look at our `api.key` file, things are different:


```
$ file api.key 
api.key: data 

$ cat api.key 
GITCRYPTROܮ7y\R*^
```


You will see some different garbage output to what I get, but it's clear the file is encrypted. This is what would be stored on GitHub.

To go back to having a cleartext `api.key` file, run:


```
$ git-crypt unlock ../git-crypt-key
```


The `../git-crypt-key` the file is the one we saved earlier using `git-crypt export-key...`


## Checkpoint

Let's do a quick review of where we are now.



*   Initialize git-crypt on a git repository using `git-crypt init`
*   Use file patterns in `.gitattributes` to tell git-crypt which files to encrypt
*   `git-crypt lock` will encrypt all the specified files in our repository
*   `git-crypt unlock [path to keyfile]` will decrypt the encrypted files

The `git-crypt-key` the file is very important. Without it, you won't be able to decrypt any of the encrypted files in your repository. Anyone who has a copy of that file has access to all of the encrypted secrets in your repository. So you need to keep that file safe and secure.


## Re-using Your git-crypt Key File

We used `git-crypt init` and `git-crypt export-key` to create our `git-crypt-key` file. But, if we have to have a separate key file for each of our repositories, then we haven't improved our secret management very much.

Fortunately, it's very easy to use the same git-crypt key file for multiple git repositories.

To use an existing key file, just use `git-crypt unlock` instead of `git-crypt init` when you set up your git repository to use git-crypt, like this:


```
$ mkdir my-other-project   # At the same directory level as `myproject` 
$ cd my-other-project 
$ git init 
$ echo "Something" > file.txt 
$ git add file.txt 
$ git commit -m "initial commit" 
$ git-crypt unlock ../git-crypt-key
```


>If you run the `git-crypt unlock` command before adding any files to your git repository, you will see a message like this:


```
fatal: You are on a branch yet to be born 
Error: 'git checkout' failed 
git-crypt has been set up but existing encrypted files have not been decrypted
```


>This still works just fine, but it's a bit confusing, so I made sure to add and commit at least one file before running `git-crypt unlock...`

Re-using your git-crypt key file is convenient, but it does mean that if anyone else gets a copy of your key file, _all_ of your encrypted secrets are exposed.

This is the same kind of security trade-off as using a password manager like [LastPass](https://www.lastpass.com/) or [1password](https://1password.com/). Rather than managing multiple secrets (passwords), each with its own risk of exposure, you keep them all in a secure store and use a single master password to unlock _that_.

The idea here is that it's easier to manage one important secret than many lesser secrets.


## When NOT to Use git-crypt

Git-crypt is a great way to keep the secrets your applications need right in the git repository, alongside the application source code. However, like every other security measure, it's not **always** going to be appropriate or advisable.

Here are some things to consider to decide whether it's the right solution for your particular project: 



*   git-crypt is designed for situations where the majority of the files in your git repository can remain in cleartext, and you just need to encrypt a few files that contain secrets. If you need to encrypt most or all of the files in your repository, then other solutions may be a better fit.
*   There is no easy way to revoke access to the secrets in a repository once someone has the key file, and no easy way to rotate (i.e. replace) a key file (although changing the git-crypt key file doesn't help much unless you also rotate all of the actual secrets in the repository).
*   git-crypt only encrypts the _contents_ of files. Therefore, it's not suitable if the _metadata_ of your repository is also sensitive (i.e. filenames, modification dates, commit messages, and so on).
*   Some GUI git applications may not work reliably with git-crypt. (Although the specific case of Atlassian SourceTree, mentioned in the README, has been [fixed](https://jira.atlassian.com/browse/SRCTREE-2511).)

>There is more information in this section of the [git-crypt README](https://github.com/AGWA/git-crypt#limitations).


## A Better Way to Use git-crypt

Rather than managing your git-crypt key file directly, there is a better way to manage encrypted repositories by integrating git-crypt with [gpg](https://gnupg.org/), so that you can use your gpg private key to decrypt the git repository. This also allows you to add multiple collaborators to a git repository without transmitting any secrets between the parties. However, this requires a more complicated setup, so we'll save that for another article.