pypi_multidl 1.0.0
A tool to download Python packages for later offline installation on many platforms

# pypi-multidl

This tool was made to make it easier to download Python packages from one
computer and install them on other computers that are not connected to the
internet (and may run different OSes). This project is not officially endorsed
by anyone (for example, the maintainers of PyPI or `pip`).

Its input is a list of dependency specifications (package name and versions),
and the output is a destination directory populated with all the sdists and
wheels from PyPI that matches the specifications. This directory can then be
used offline with `pip` using the `-f/--find-links` option as a lightweight PyPI
mirror.

This tool does no dependency resolution. You are expected to use `pip-compile`
from the `pip-tools` package (or an equivalent tool) to find all the
dependencies of the dependencies, and pinning down all the exact version.
`pip-compile` can be used to process a `setup.py`/`setup.cfg`/`pyproject.toml`
file into a `requirements.txt` to be used by `pypi-multidl`.

## Example Usage: Download Dependencies for a Project

Assume you have a Python project you want to be able to install offline, and
that its dependencies aren't pinned to specific version. Then begin by compiling
the list of transitive pinned dependencies:

    cd some-project
	pip-compile setup.py # ...or setup.cf, or pyproject.toml

Now you have a `requriements.txt` file. Continue to download all the
dependencies to the `deps/` directory:

    pypi-multidl -r requirements.txt -d deps/

The project can then be installed offline like this:

    pip install --no-index --find-links deps/ .

## Example Usage: List all Distribution Packages Avaiable for a Project

The `pypi-multidl` tool can also be given dependency specifications directly on
the command line:

    pypi-multidl --dry-run 'appdirs' 'sphinx >=4.0.0, <5.0.0'

## Limitations

The approach used by pypi-multidl does not work in the general case. The
dependency resolution step `pip-compile` is actually platform-specific. A
package can have different dependencies on different platforms. For example,
Sphinx depends on `colorama`, but only on Windows. What you get from
`pip-compile` is the set of dependencies specific for the plaform that
`pip-compile` was run on. If no packages have platform-specific dependencies,
then everything should work. Currently, `pypi-multidl` does not make any attempt
to validate that the `requirements.txt` is platform-independent.

This tool currently only supports the PEP 691 JSON API via the simple endpoint.

Yanked files are never downloaded, not even when pinned using `==` (see PEP
592).

Requires-python and other metadata is not downloaded.

This tool only produces a flat directory of files, not a "simple" style
directory tree to be served by a web server.

## Comparison to other tools

The `pip download` command only supports downloading packages for one system at
a time. It also only downloads an sdist or a wheel, but never both.

The `pip-downloader` tool solves the same problem as `pypi-multi`, but is
implemented in a different way. It performs dependency resolution using `pip`
and uses its internal API. It needs to continually be updated whenever `pip`
changes in order not to break.

## Usage with a Local Cache

When experimenting with downloading from PyPI it is recommended to setup a local
PyPI cache. The `devpi` tool is easy to get started with:

    pip install devpi
	devpi-init
	devpi-server  # Runs an index on http://localhost:3141/root/pypi/+simple/

Leave that running in a terminal (or add it to your init system) and add this to
your pip config:

    [global]
    index-url = http://localhost:3141/root/pypi/+simple/

`pip`, `pip-compile` and `pypi-multidl` will then automatically use the cache.
Note that you also need to pass `--no-emit-options` to `pip-compile` so that it
doen't add an `--index-url` line to the `requirements.txt` (which `pypi-multidl`
doesn't understand).