14 Must-Know pip Commands For Data Scientists and Engineers | Towards Data Science

Even though Python standard library offers a wide range of tools and functionality, developers still need to use external (to the standard library) packages that are widely available. Most programming languages have their standard package managers that allow users to manage the dependencies of their projects.
The most commonly used package manager in Python is definitely pip. In today’s article we are going to explore a few useful pip commands that you can use to manage the dependencies of your Python projects properly.
pip is a Python package manager that enables the installation of packages that are not included in the standard library of the language. It’s equivalent to npm for JavaScript or NuGet for .NET.
Python is among the most popular programming languages and its community is quite active when it comes down to the development of packages that serve specific purposes. These packages are usually uploaded on Python Package Index (PyPI). By default, pip uses PyPI as its source for resolving all dependencies required for a particular application.
If you want to know about how you can distribute your own Python projects and make them widely available, make sure to read the guide below.
towardsdatascience.com
If you are using Python 2 >=2.7.9 or Python 3 >=3.4 then pip should already be installed on your machine. The same applies if you are working in a virtual environment that has been created either using venv or virtualenv.
In the following sections we discuss 14 commonly used pip commands that every Python Developer must be aware of. Note that it is highly recommended that for each Python project you also create a separate virtual environment in which you’ll be able to install and manage all the required dependencies.
In order to install a package in your current environment you need to run pip install that apart from the package itself, it is also going to install all of its dependencies.
$ python3 -m pip install package_name
If for any reason you want to uninstall a package you have to use pip uninstall command.
$ python3 -m pip uninstall package_name
Note however that this command won’t uninstall the dependencies of this package. If for any reason you wish to also uninstall the unused dependencies as well, take a look at pip-autoremove that you can use to do so.
Now let’s suppose that you have a requirements.txt that lists all the required dependencies for a particular package.
# requirements.txtpandas==1.0.0pyspark
Instead of going through the packages one by one, you can install all the dependencies directly from that file using the command shown below.
$ python3 -m pip install -r requirements.txt
If you want to list all the Python packages installed in an environment, pip list command is what you are looking for.
$ python3 -m pip list
The command will return all the packages installed, along with their specific version and location.
Package                Version           Location                                            ---------------------- ----------------- -------------pandas                 1.0.0pyspark                2.1.0
If a package is installed from a remote host (for example PyPI or Nexus) the location will be empty. When a package is installed from a location on your machine then the specific location will be listed to the corresponding column of the output.
You can even use pip list command in order to fetch the version of a particular package should you need to. You can do so using
$ python3 -m pip list | grep package_name
While developing your Python application you may need to install other dependencies that will help you implement whatever you need to. Instead of having to write the packages one by one in a requirements.txt file so that other people can also install the required dependencies when using your package, you can use pip freeze to get all the packages installed in your environment.
$ python3 -m pip freeze > requirements.txt
Note that this command will also hardcode the specific version of the packages that you have already installed in your environment (e.g. pandas==1.0.0), which in certain cases may not be what you really want. In this case you may have to manually remove or amend the versions.
Now if you are developing a package locally you may have to install it in editable mode. This means that the installation will link the package to the specified local location so that any changes made to the package will be reflected directly in your environment.
$ cd /path/to/your/local/python/package$ python3 -m pip install -e .
The above command will install the package in the current directory in editable (or develop) mode. Now if you run pip list | grep package_name the location of the package should be mapping to the location on your local machine (e.g. path/to/your/local/python/package).
If one of your installed packages is outdated and you wish to upgrade it (always recommended but this might break the compatibility with other packages), you can do so by running
$ python3 -m pip install package_name --upgrade
Now if for any reason you want to remove all packages installed by pip, then you can run the command shown below that creates an intermediate requirements.txt file with the existing installed packages and then uses pip uninstall to uninstall all of them.
$ python3 -m pip freeze > requirements.txt && python3 -m pip uninstall -r requirements.txt -y
If you want to skip the creation of any intermediate files (i.e. requirements.txt) then you can run the command below.
$ python3 -m pip uninstall -y -r <(pip freeze)
$ python3 -m pip check
If no compatibility issues are identified the output of the command will be
No broken requirements found.
If you want to retrieve information about a specific package, pip show will return details such as name, version, summary, license, dependencies and other.
$ python3 -m pip show package_name
An example output for pandas is shown below:
$ pip show pandasName: pandasVersion: 1.2.3Summary: Powerful data structures for data analysis, time series, and statisticsHome-page: https://pandas.pydata.orgAuthor: NoneAuthor-email: NoneLicense: BSDLocation: /path/to/lib/python3.6/site-packagesRequires: pytz, python-dateutil, numpyRequired-by: shap, seaborn
As we mentioned already, pip uses PyPI as its default source from which it retrieves packages. pip search command is used to search the index and identify packages that match the search terms.
$ python3 -m pip search <search_term>
For example, python3 -m pip search pandas will return all the packages that satisfy the search term pandas .
Wheels are part of the Python ecosystem which are built-package formats that offers the advantage of not recompiling your software during every install. pip wheel command is used to build Wheel archives for your requirements and dependencies.
$ python3 -m pip wheel
Note that pip uses the bdist_wheel setuptools extension from the wheel package to build individual wheels.
pip comes with its own configuration that can obviously be modified based on your use-case’s requirements. pip comes with the config option that allows use to edit, get, list, set or unset configurations.
$ python3 -m pip config <action name>
For example, to list the configuration options enabled, you can run
$ python3 -m pip config list
In order to set a specific configuration option, then you can use the set method. For example, if you need to change the global.index configuration parameter you have to run
$ python3 -m pip config set global.index-url https://your.global.index
Now if you want to retrieve debugging information specific to pip you can use pip debug. The command will return information relevant to pip installation and local system environment.
$ python3 -m pip debug
Note that this command is only meant for debugging and thus you must not use it to parse or infer details as part of your production workflow as the output of the command might change without notice.
In today’s article we’ve had a quick take on pip which is the most commonly used package manager in Python. Additionally, we introduced Python Package Index (or PyPI) that hosts all third-party packages available to Python developers. Finally, we explored 14 different pip commands that you should definitely know in order to properly manage the dependencies properly.
For the full list of pip options, simply run pip --help in your terminal and the command will return the usage information.
Usage:pip <command> [options]Commands:install      Install packages.download     Download packages.uninstall    Uninstall packages.freeze       Output installed packages in requirements format.list         List installed packages.show         Show information about installed packages.check        Verify installed packages have compatible dependencies.config       Manage local and global configuration.search       Search PyPI for packages.cache        Inspect and manage pip's wheel cache.wheel        Build wheels from your requirements.hash         Compute hashes of package archives.completion   A helper command used for command completion.debug        Show information useful for debugging.help         Show help for commands.