Creating Python Virtual Environments | by Wei-Meng Lee | Towards Data Science

In Python, a virtual environment is an isolated environment for running your Python programs. Using a virtual environment allows your program to have its own dependencies (different versions of packages). For example, Program A uses a specific version of packageX, while Program B uses an older version of packageX. Here is the problem visually:
If you put the two program into the same default environment, you will run into problem with packageX — if you try to install a specific version of packageX, it will override the other version.
A better solution is to create two isolated environments— aka virtual environments. Each virtual environment will then host the specific versions of packageX that each program needs, like this:
In this article, I will show you how you can create virtual environments using the conda package manager. In addition, I will also show you how you can obtain a list of packages that you have installed in your environment (through the pip command) and perform the same installation on another virtual environment using the requirements.txt file.
Conda is an open source package and environment management system that runs on Windows, Mac OS and Linux.
This article assumes that you have Anaconda (https://www.anaconda.com/products/individual-d) installed on your computer.
To create a new virtual environment, use the conda command with the following options:
conda create --name Project1 python=3.9
The above command creates a virtual environment named Project1, with the Python version set at 3.9:
Once the virtual environment is created, you can view the list of virtual environments you have on your computer using the following command:
conda info --envs
You should see something like the following:
(base) weimenglee@Wei-Mengs-Mini ~ % conda info --envs# conda environments:#base                *  /Users/weimenglee/miniforge3Project1               /Users/weimenglee/miniforge3/envs/Project1
With the new virtual environment created, it is time to activate it:
(base) weimenglee@Wei-Mengs-Mini ~ % conda activate Project1(Project1) weimenglee@Wei-Mengs-Mini ~ %
As you can see above, once the Project1 virtual environment is activated, you will see the name of the virtual environment (Project1) in front of the command prompt:
(Project1) weimenglee@Wei-Mengs-Mini ~ %
To know what are the packages currently installed in your virtual environment, use the pip freeze command:
(Project1) weimenglee@Wei-Mengs-Mini ~ % pip freeze
This command will return you a list of packages installed in the current environment, together with their version numbers. Here is an example of the output of my Project1 virtual environment:
certifi==2021.5.30
Occasionally the pip freeze command will include direct references for distributions installed from direct URL references, such as the following:
certifi==2021.5.30chardet @ file:///Users/runner/miniforge3/conda-bld/chardet_1610093454858/work
To prevent this from happening, use the following alternative command for pip freeze:
pip list --format=freeze
This will ensure that the result will always be in this format:
<package_name>==<version_number>
Let’s now try to install some additional packages in our Project1 virtual environment using the pip install command:
(Project1) weimenglee@Wei-Mengs-Mini ~ % pip install pandas numpy matplotlib
The above command installs the following packages:
Pandas
NumPy
Matplotlib
You can now use the pip freeze command (or pip list —-format=freeze) to view the list of packages you have, including the ones that you have just installed.
(Project1) weimenglee@Wei-Mengs-Mini ~ % pip freezecertifi==2021.5.30cycler==0.10.0kiwisolver==1.3.2matplotlib==3.4.3numpy==1.21.2pandas==1.3.2Pillow==8.3.2pyparsing==2.4.7python-dateutil==2.8.2pytz==2021.1six==1.16.0
In addition, you will also save the list to a file named requirements.txt:
(Project1) weimenglee@Wei-Mengs-Mini ~ % pip freeze > requirements.txt
The requirements.txt file contains the list of packages that are installed in the current environment. You will make use of it later to reinstall all the packages in another virtual environment.
Key takeaway — whatever packages you install in this virtual environment stays in this virtual environment. It will not conflict with other similar packages you install in another virtual environment.
You can now test to see if the environment works correctly. Type the following command to start the Python interpreter:
(Project1) weimenglee@Wei-Mengs-Mini ~ % python
Then, import the Pandas package and print out its version:
>>> import pandas>>> print(pandas.__version__)
If you are able to see the output above, the packages are installed correctly.
Type exit() to exit from the Python interpreter.
When you are done with the environment, you can use the conda deactivate command to bring you back to the base environment:
(Project1) weimenglee@Wei-Mengs-Mini ~ % conda deactivate
After you are done with your project and have no more need for the environment, you can remove the environment using the following command:
(base) weimenglee@Wei-Mengs-Mini ~ % conda env remove --name Project1Remove all packages in environment /Users/weimenglee/miniforge3/envs/Project1:
Let’s now create another virtual environment and call it, say Project2:
(base) weimenglee@Wei-Mengs-Mini ~ % conda create --name Project2 python=3.9
Once created, activate the new virtual environment:
(base) weimenglee@Wei-Mengs-Mini ~ % conda activate Project2(Project2) weimenglee@Wei-Mengs-Mini ~ %
Let’s try the Python interpreter and see if the Pandas module is available:
(Project2) weimenglee@Wei-Mengs-Mini ~ % pythonPython 3.9.6 (default, Aug 16 2021, 12:43:27)[Clang 12.0.0 ] :: Anaconda, Inc. on darwinType "help", "copyright", "credits" or "license" for more information.>>> import pandasTraceback (most recent call last):File "<stdin>", line 1, in <module>ModuleNotFoundError: No module named 'pandas'>>> exit()
As predicted, Pandas is not installed by default. If you want to install the set of packages that you have installed in the Project1 virtual environment, you can now make use of the requirements.txt file you created earlier:
(Project2) weimenglee@Wei-Mengs-Mini ~ % pip install -r requirements.txtRequirement already satisfied: certifi==2021.5.30 in ./miniforge3/envs/Project2/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (2021.5.30)Collecting cycler==0.10.0...Collecting matplotlib==3.4.3Using cached matplotlib-3.4.3-cp39-cp39-macosx_11_0_arm64.whlCollecting numpy==1.21.2Using cached numpy-1.21.2-cp39-cp39-macosx_11_0_arm64.whl (12.4 MB)Collecting pandas==1.3.2Using cached pandas-1.3.2-cp39-cp39-macosx_11_0_arm64.whl...Installing collected packages: six, pytz, python-dateutil, pyparsing, Pillow, numpy, kiwisolver, cycler, pandas, matplotlibSuccessfully installed Pillow-8.3.2 cycler-0.10.0 kiwisolver-1.3.2 matplotlib-3.4.3 numpy-1.21.2 pandas-1.3.2 pyparsing-2.4.7 python-dateutil-2.8.2 pytz-2021.1 six-1.16.0
All the packages specified in the requirements.txt file will now be installed in the Project2 virtual environment.
Using the requirements.txt file is a good way to specify the dependencies of your Python program. In this file, you put all the packages names (as well as their specific version numbers) that your application needs so that whoever runs your program simply needs to perform a pip install using the requirements.txt file.