Conda + Google Colab. A guide to installing Conda when using... | by David R. Pugh | Towards Data Science

Conda is the recommended environment and package management solution for a number of popular data science tools including Pandas, Scikit-Learn, PyTorch, NVIDIA Rapids and many others. Conda also dramatically simplifies the process of installing popular deep learning tools like TensorFlow.
Google Colab is a free service providing interactive computing resources via a user interface that is very similar to Jupyter notebooks, runs on Google Cloud Platform (GCP), and provides free access to GPUs and TPUs. Google Colab is a great teaching platform and is also perhaps the only free solution available for sharing GPU or TPU accelerated code with your peers. Unfortunately, Conda is not available by default on Google Colab and getting Conda installed and working properly within Google Colab’s default Python environment is a bit of a chore.
In this article, I will walk you through the process that I go through when I need to use Conda to install packages when working in Google Colab.
First, you need to confirm which Python is being used by default in Google Colab. Running the following command returns the absolute path to the default Python executable.
!which python # should return /usr/local/bin/python
Now check the version number of this default Python.
!python --version
At time of writing the above command returns Python 3.6.9. This means that, in order to use all of the preinstalled Google Colab packages, you will need to install a version of Miniconda that is compatible with Python 3.6 by default. More recent versions (i.e., 4.5.12+) of Miniconda target either Python 3.7 or Python 3.8 by default. The most recent version of Miniconda that targets Python 3.6 is Miniconda 4.5.4 so this is the version that you should install.
Finally, check to see if the PYTHONPATH variable has been set.
!echo $PYTHONPATH
At the time of writing this command returns only /env/python (oddly this is a directory that doesn’t seem to exist within Google Colab filesystem).
Typically, it is a good idea to unset the PYTHONPATH variable before installing Miniconda as it can cause problems if there are packages installed and accessible via directories included in the PYTHONPATH that are not compatible with the version of Python included with Miniconda.
You can unset the PYTHONPATH variable with the following command. This step is optional but if you don’t unset this variable then you will see a warning message after installing Miniconda.
%env PYTHONPATH=
When executed in a Google Colab cell, the code below will download the installer script for the appropriate version of Miniconda and install it into /usr/local. Installing directly into /usr/local, rather than into the default location ~/miniconda3, insures that Conda and all its required dependencies will be automatically available for use within Google Colab.
%%bashMINICONDA_INSTALLER_SCRIPT=Miniconda3-4.5.4-Linux-x86_64.shMINICONDA_PREFIX=/usr/localwget https://repo.continuum.io/miniconda/$MINICONDA_INSTALLER_SCRIPTchmod +x $MINICONDA_INSTALLER_SCRIPT./$MINICONDA_INSTALLER_SCRIPT -b -f -p $MINICONDA_PREFIX
Once you have installed Miniconda you should be able to see that the Conda executable is available...
!which conda # should return /usr/local/bin/conda
...and that the version number is correct.
!conda --version # should return 4.5.4
Note that while installing Miniconda does not appear to impact the Python executable...
!which python # still returns /usr/local/bin/python
...however, Miniconda has actually installed a slightly different version of Python.
!python --version # now returns Python 3.6.5 :: Anaconda, Inc.
Now that you have installed Conda you need to update Conda and all its dependencies to their most recent versions without updating Python to 3.7 (or 3.8). The conda install command below actually updates Conda to the most recent version whilst holding the Python version fixed at 3.6. The conda update command then updates all of Conda’s dependencies to their most recent versions.
%%bashconda install --channel defaults conda python=3.6 --yesconda update --channel defaults --all --yes
Now you can confirm the update by checking the version number for Conda.
!conda --version # now returns 4.8.3
Also, note that the Python version has changed yet again.
!python --version # now returns Python 3.6.10 :: Anaconda, Inc.
Now that you have installed Miniconda you need to add the directory where Conda will install packages to the list of directories that Python will search when looking for modules to import. You can see the current list of directories that Python will search when looking for modules to import by inspecting the sys.path.
import syssys.path
As of this writing the sys.path on Google Colab looks as follows.
['',   '/env/python', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-packages', # pre-installed packages '/usr/lib/python3/dist-packages', '/usr/local/lib/python3.6/dist-packages/IPython/extensions', '/root/.ipython']
Note that the preinstalled packages included with Google Colab are installed into the /usr/local/lib/python3.6/dist-packages directory. You can get an idea of what packages are available by simply listing the contents of this directory.
!ls /usr/local/lib/python3.6/dist-packages
Any package that you install with Conda will be installed into the directory /usr/local/lib/python3.6/site-packages so you will need to add this directory to sys.path in order for these packages to be available for import.
import sys_ = (sys.path        .append("/usr/local/lib/python3.6/site-packages"))
Note that because the /usr/local/lib/python3.6/dist-packages directory containing the pre-installed Google Colab packages appears ahead of the /usr/local/lib/python3.6/site-packages directory where Conda installs packages, the version of a package available via Google Colab will take precedence over any version of the same package installed via Conda.
Now all you need to do is install you favorite packages. Just remember to include the --yes flag when installing your packages to avoid getting prompted to confirm the package plan.
!conda install --channel conda-forge featuretools --yes
In this article, I covered the process that I use to install and configure Miniconda when I need to use Conda to manage packages on Google Colab. Hopefully, this will help you the next time you need to share a Conda-managed data science project on Google Colab.