Securing API Keys with Environment Variables Using Anaconda | by Raymond Willey | Towards Data Science

If you are reading this, I assume you are already familiar with APIs, tokens, private keys, and public keys. Your private and public keys serve as a form of authentication to identify you and track your activity with a developer’s API. What this means is that, if someone got ahold of your keys, they could access data in the API while impersonating you, which can have some dire consequences.
The challenge faced by most up-and-coming data scientists is that they want to promote their project work in a public forum. If any of those projects use APIs, the individual will have to load their keys into their notebooks. In other words, making the notebook public means jeopardizing the security of the API key.
There are a few common solutions to this problem: the first is to replace the key with something like ****** or YOUR_PRIVATE_KEY_HERE. This works to a certain extent, but the methodology is fallible to human error. With multiple commits on a single project, the token must be removed and re-added each time. The second you push a commit and forget to clear the key, it is compromised. It’s not a matter of if this will happen, but when. Another option is to save API keys to a separate JSON file, read the private key, and exclude the file when pushing to GitHub. While this is a little better, the problem here is that others wanting to explore your work must make edits to the forked repository to run the code. Moreover, this does not fully eliminate the potential for human error.
Wouldn’t it be great if there was a way to save keys and tokens to our system files to be called when desired? And wouldn’t it be even better if others can use the same code to call their own keys and tokens from their own systems? As it turns out, there is a way: environment variables!
Environment variables are precisely what their name implies: variables saved in the local environment that can be called into any project. Because they are not part of the repository itself, there is no need to restrict the information that is pushed to GitHub. In the example below, a private API key is already saved as an environment variable called PRIVATE_API_KEY. We will look at how to create the variable itself a bit later.
import osapi_key = os.environ.get("PRIVATE_API_KEY")
As we can see, this looks pretty clean and simple. Whenever we need to use the key, all we have to do is call the api_key variable. More seasoned data scientists reviewing this code should immediately recognize the need to use their own private keys, saving them in their own environments. Once completed, the program should run as-is without any hiccups.
The one caveat here is that the reviewer will need to save the API key using the same variable name. If the person already has the relevant key saved to their environment using a different variable name, they will likely opt to modify the code rather than their environment. What would make this process even more elegant is if API developers provided recommended naming conventions for these keys in their documentation. But that’s a topic for another day.
In this post, we are not going to get into the best practices for managing environments across projects. Instead, I am assuming you are already working in the proper environment and know how to switch between them using Anaconda. If you are not familiar with this, I would recommend checking out this video as a primer for the rest of this post.
The overall process for creating these variables is as follows:
Navigate to the environment path for storing packages and details.Create files and directories for activating variables when called and deactivating them when done using them.Add the variables into the relevant files.
Navigate to the environment path for storing packages and details.
Create files and directories for activating variables when called and deactivating them when done using them.
Add the variables into the relevant files.
I will show you how to handle all three of these steps in macOS, Linux, and Windows. All instructions are adapted directly from the Anaconda documentation.
The following commands can be used to handle steps 1 and 2 using the terminal:
cd $CONDA_PREFIXmkdir -p ./etc/conda/activate.dmkdir -p ./etc/conda/deactivate.dtouch ./etc/conda/activate.d/env_vars.shtouch ./etc/conda/deactivate.d/env_vars.sh
So, we now have two files called env_vars.sh that are each stored in separate directories. We need to open these files in a text editor of your choice to create the variables. For the file in the activate.d directory, we will add the following:
#!/bin/shexport PRIVATE_API_KEY='YOUR_PRIVATE_KEY_HERE'
For the file in the deactivate.d directory, we add this:
#!/bin/shunset MY_KEY
The first line #!/bin/sh lets the operating system know that the files are shell scripts to ensure the export and unset commands are properly executed.
Once these files are saved and closed, enter conda activate analytics into the terminal to save YOUR_PRIATE_KEY_HERE to the environment variable PRIVATE_API_KEY. When done using these variables, they can be erased from the environment using the conda deactivate.
The following commands can be used to handle steps 1 and 2 using the command shell:
cd %CONDA_PREFIX%mkdir .\etc\conda\activate.dmkdir .\etc\conda\deactivate.dtype NUL > .\etc\conda\activate.d\env_vars.battype NUL > .\etc\conda\deactivate.d\env_vars.bat
So, we now have two files called env_vars.bat that are each stored in separate directories. We need to open these files in a text editor of your choice to create the variables. For the file in the activate.d directory, we will add the following:
set PRIVATE_API_KEY='YOUR_PRIVATE_KEY_HERE'
For the file in the deactivate.d directory, we add this:
unset PRIVATE_API_KEY=
Once these files are saved and closed, enter conda activate analytics into the terminal to save YOUR_PRIATE_KEY_HERE to the environment variable PRIVATE_API_KEY. When done using these variables, they can be erased from the environment using the conda deactivate.
As you can see, though there are a few steps to saving your API keys as environment variables, there is nothing inherently difficult or complicated about it. Once in place, the variables are quick and easy to access, and they can be deactivated with a single command when you are done with them. It is much more efficient than adding and removing keys every time you want to make an update or push a new commit to GitHub. And it gives you more safety and security when showcasing your projects to the outside world.