Creating Virtual Environments for Python Data Science Projects | by Christine Egan | Towards Data Science

If you’ve used Python before, you may have encountered a situation known as “dependency hell”. Some packages depend on other packages to work. Sometimes, the package that another package depends on must be a certain version. Later on, you might need to use a different version of that package because there is yet another package that depends on that package being another version. An efficient way to avoid dependency hell and create professional projects is to use virtual environments.
A virtual environment is like a little capsule for your project that contains all of the proper packages, in the proper version, in one spot. This makes it so you can switch from one project to another with ease. It also makes it so others can reproduce it later on. There are different virtual environment packages, but I like to use Pyenv.
In this tutorial I will explain how you can use Pyenv, Virtualenv, and Pip in conjunction to manage your virtual environments on mac OS Big Sur. These instructions will likely also be similar for Linux. If you have Windows you can learn more about how to use Pyenv with Windows here.
Note: There is an archived version of this tutorial for Catalina users. The primary difference is that for Catalina users, I advised using Bash, based on my personal preference. However, Zsh is now the default shell for Big Sur and the typical procedure for accomplishing this setup using Bash is not effective on Big Sur.
According to their slogan, Homebrew is “the missing package manager for mac OS and Linux.” To check to see if you already have Homebrew, enter:
% brew update
Note for beginners: When following these tutorials, you do not need to enter the percent sign (%). It should already be there for you. When you see a percent sign (%) or a dollar sign ($) in code snippets from tutorials, that generally indicates that this code snippet should be entered on the command line (in Terminal). A percent sign indicates that the language used in the terminal is Zsh, and a dollar sign indicates that the language is Bash. Check out this article to learn more about the differences.
If it is indicated that you do not have Homebrew, I recommend reading this tutorial to ensure that you are properly set up to complete these instructions.
Now, we can install Pyenv with Homebrew.
% brew install pyenv
I know this seemed a little too easy, so here is the catch. There is a special file that you need to access to make this actually work. As a beginner, I always found this part to be tricky, so I’ve broken it down into little pieces. This explanation might be a little granular for a more advanced user, but I tried to be a specific as possible for the sake of the beginner.
The file you need to access is known as .zshrc. Notice the dot (.) at the front. When you see a dot in front of a file name, that means that the file is a hidden file.
When we start the terminal, we are starting in our home directory. Your home directory is generally named your username. To see the files in your home directory, you can type the command ls. However, if you want to see hidden files you need to add the the argument -a.
Enter the following:
% ls -a
A list of all the files, including the hidden files, will appear.
Look carefully at the files. Do you see a file called .zshrc? If not, we will make it together. Choose one of the two ways below to create this file.
To create .zshrc with any text editor app, simply open up your home directory in Finder.
Next, enter Command + Shift + . to display the secret files.
If you do not see .zshrc, go ahead and open up the text editor and paste the following lines into a new document.
eval “$(pyenv init -)”eval “$(pyenv virtualenv-init -)”
Save the file in your home directory as .zshrc.
Macs come loaded with a light text editor that you can access in Terminal called Nano. To open Nano, simply enter:
% nano .zshrc
This command indicates that you wish to run the program Nano, and .zshrc is the name of the file that we will create. If this or any file has already been created, instead of creating a new file, Nano will open the existing file with that name.
Now, simply copy and paste the following into Nano:
eval “$(pyenv init -)”eval “$(pyenv virtualenv-init -)”
Exit Nano by entering CTRL + x. When prompted, enter Y to save the file.
Finally, we can add Pyenv to our path by running the following in Terminal in the home directory
echo ‘export PYENV_ROOT=”$HOME/.pyenv”’ >> ~/.zshrcecho ‘export PATH=”$PYENV_ROOT/bin:$PATH”’ >> ~/.zshrc
This will add Pyenv to your path so that you can use Pyenv commands in the command line.
The next step is installing Virtualenv to use with Pyenv to manage our virtual environments and subsequent dependencies.
Use the following commands to call on Pip to install Virtualenv.
% pip3 install virtualenv% pip3 install virtualenvwrapper% brew install pyenv-virtualenv
1. Updated Homebrew and installed Pyenv2. Displayed secret files in our home folder.3. Located and/or created a .zshrc file using nano.4. Added Pyenv to our path so we can use it in the command line.5. Installed virtualenv, virtualenvwrapper, and pyenv-virtualenv.6. Properly configured pyenv and virtualenv to our shell.
In the next tutorial, Getting Started with Jupyter Notebooks, I’ll show you how to create a unique virtual environment for your project and install some of the most popular data science packages, like Jupyter Notebook and Pandas.