Repository: pypa/pip

Issue:
Command to know what file would be downloaded for a requirement
**What's the problem this feature will solve?**

I want to know what version (and potentially ahich artifact) would be downloaded/installed when I `pip install` a package. Currently I do something like `pip download --no-deps --dest %TMP%` and read the output, but that can take a long time if the package is large (e.g. numpy).

**Describe the solution you'd like**

Something like `pip find numpy` to only execute the finding part, but instead of downloading (and inspecting) the package, print the link that would be downloaded instead:

```
$ py -3.7 -m pip find numpy
https://files.pythonhosted.org/packages/4e/9d/c129d78e6b942303b762ccfdf1f8339de80c5e6021b14ef0c99ec5bdc6aa/numpy-1.16.3-cp37-cp37m-win_amd64.whl
```

**Alternative Solutions**

My current approach is described above. Alternative design includes `pip install --dry-run`, but that does not feel right because it implies dependency resolution (which requires the package to be downloaded).

An alternative output is to list all applicable links, instead of only the best match.

**Additional context**

A command that automatically download packages (to a temporary location) and produce a requirements.txt could be useful as well, but I can write a script using existing pip commands easily enough, so pip don’t really need to provide that functionality.



Discussion:
I don't think it's entirely possible (and I think the initial snippet is wrong) once we get a real version solver in place. It's entirely possible that ``pip install foo`` will traverse the dependency graph and find that the latest version of foo is being excluded for some reason by one of it's dependencies (or we cannot install the latest version because of a conflict) and an older version will instead be installed.

However, the hypothetical ``pip find foo`` that does not do dependency resolution will just grab the latest version that matches the specifier, which would be different than what ``pip install foo`` would do in the above scenario.
Good point, I did not think of that. How about defining this as “outputting all possible links for this specification” instead? This don’t need to guarentee what would be actually installed (or anything at all), only list the links found.

My use case is exactly to feed the output to a resolver, but I don’t want to re-implement all the configuration and edge cases in pip.
In #6740 I made a similar request:

**What's the problem this feature will solve?**
Currently there is no way for third-party projects to use `PackageFinder`'s logic via the command line. Instead, projects like `pip-compile` and `pipenv` need to import from the `_internal` API.

**Describe the solution you'd like**
Ideally this would mimic `PackageFinder.find_all_candidates()` as much as possible, and would have a human-friendly and machine-friendly format. Something like:

```
$ pip find bar
bar    0.1.0    https://files.pythonhosted.org/packages/2c/85/d6397edeceb1c81b5b99b1046033c8854abac3e50971c8d2915e63ce54c0/bar-0.1.0.tar.gz#sha256=6a711b5b06a662fe3f8e897310456b02127fb15d89310161fc8af7f415e270a1 (from https://pypi.org/simple/bar/)
bar    0.2.0    https://files.pythonhosted.org/packages/ba/bf/81ef8a2c80f4f86e9e6570605b7b5a277950cbd0ff9be901633159f4110e/bar-0.2.0.tar.gz#sha256=4a97e8664a22eec73f65d522cc36b4c693b61f0557760a856bb18a8fb67a8881 (from https://pypi.org/simple/bar/)
bar    0.2.1    https://files.pythonhosted.org/packages/77/d6/760c7a0ee7385ba4b0abb1d38af97e8c2f34cb47fe301315e48204088047/bar-0.2.1.tar.gz#sha256=b0a71c865729f9e8ac0f74e666b46c5ed1f5addac3425be7e06ca4828cec90ef (from https://pypi.org/simple/bar/)

$ pip find bar --format json
[
  {
    "name": "bar",
    "version": "0.1.0",
    "link": "https://files.pythonhosted.org/packages/2c/85/d6397edeceb1c81b5b99b1046033c8854abac3e50971c8d2915e63ce54c0/bar-0.1.0.tar.gz#sha256=6a711b5b06a662fe3f8e897310456b02127fb15d89310161fc8af7f415e270a1",
    "index": "https://pypi.org/simple/bar/"
  }, {
    "name": "bar",
    "version": "0.2.0",
    "link": "https://files.pythonhosted.org/packages/ba/bf/81ef8a2c80f4f86e9e6570605b7b5a277950cbd0ff9be901633159f4110e/bar-0.2.0.tar.gz#sha256=4a97e8664a22eec73f65d522cc36b4c693b61f0557760a856bb18a8fb67a8881",
    "index": "https://pypi.org/simple/bar/"
  }, {
    "name": "bar",
    "version": "0.2.1",
    "link": "https://files.pythonhosted.org/packages/77/d6/760c7a0ee7385ba4b0abb1d38af97e8c2f34cb47fe301315e48204088047/bar-0.2.1.tar.gz#sha256=b0a71c865729f9e8ac0f74e666b46c5ed1f5addac3425be7e06ca4828cec90ef",
    "index": "https://pypi.org/simple/bar/"
  }
]
```

**Alternative Solutions**
* Splitting out the `PackageFinder` logic into a separate package (#5800) and having third-parties use this instead
* Refactor `pip search` to use `PackageFinder` (#395) and add extra options there to overload the current behavior/API of `pip search`

**Additional context**
Recent internal API changes in 19.2 will break `pip-compile`: https://github.com/jazzband/pip-tools/issues/853
FYI, I posted PR #6910, which makes progress towards this.
Also, FWIW, re: the original issue, I like the idea of a `pip find` command.
FYI PR #7016, which was just posted, relates to this.
Similarly to #7975, I think this issue was solved by #8978 and further improvements are currently tracked #10052.

Please excuse my nit-pickiness, just checking out the issues that CodeTriage sends me. 😅 
Neither `pip index version` now and #10052 covers what I originally needed. I was looking for the information “what file will be downloaded when I install this requirement”. Both features you linked only operate at the *version* level, but I need the information at the *file* level. Numpy 1.16.3, for example, [contains 22 wheels and 1 sdist](https://pypi.org/project/numpy/1.16.3/#files), I want to know which one will be selected.
Is this a subset of the broader "do a dry-run, and tell me what you would've done" ALA #53?
Yes if we provide good enough dry-run logging (I want to parse the filename in a script).

Resolving patch:
diff --git a/src/pip/_internal/commands/install.py b/src/pip/_internal/commands/install.py
--- a/src/pip/_internal/commands/install.py
+++ b/src/pip/_internal/commands/install.py
@@ -1,4 +1,5 @@
 import errno
+import json
 import operator
 import os
 import shutil
@@ -7,6 +8,7 @@
 from typing import Iterable, List, Optional
 
 from pip._vendor.packaging.utils import canonicalize_name
+from pip._vendor.rich import print_json
 
 from pip._internal.cache import WheelCache
 from pip._internal.cli import cmdoptions
@@ -21,6 +23,7 @@
 from pip._internal.locations import get_scheme
 from pip._internal.metadata import get_environment
 from pip._internal.models.format_control import FormatControl
+from pip._internal.models.installation_report import InstallationReport
 from pip._internal.operations.build.build_tracker import get_build_tracker
 from pip._internal.operations.check import ConflictDetails, check_install_conflicts
 from pip._internal.req import install_given_reqs
@@ -250,6 +253,20 @@ def add_options(self) -> None:
         self.parser.insert_option_group(0, index_opts)
         self.parser.insert_option_group(0, self.cmd_opts)
 
+        self.cmd_opts.add_option(
+            "--report",
+            dest="json_report_file",
+            metavar="file",
+            default=None,
+            help=(
+                "Generate a JSON file describing what pip did to install "
+                "the provided requirements. "
+                "Can be used in combination with --dry-run and --ignore-installed "
+                "to 'resolve' the requirements. "
+                "When - is used as file name it writes to stdout."
+            ),
+        )
+
     @with_cleanup
     def run(self, options: Values, args: List[str]) -> int:
         if options.use_user_site and options.target_dir is not None:
@@ -353,6 +370,20 @@ def run(self, options: Values, args: List[str]) -> int:
                 reqs, check_supported_wheels=not options.target_dir
             )
 
+            if options.json_report_file:
+                logger.warning(
+                    "--report is currently an experimental option. "
+                    "The output format may change in a future release "
+                    "without prior warning."
+                )
+
+                report = InstallationReport(requirement_set.requirements_to_install)
+                if options.json_report_file == "-":
+                    print_json(data=report.to_dict())
+                else:
+                    with open(options.json_report_file, "w", encoding="utf-8") as f:
+                        json.dump(report.to_dict(), f, indent=2, ensure_ascii=False)
+
             if options.dry_run:
                 would_install_items = sorted(
                     (r.metadata["name"], r.metadata["version"])
diff --git a/src/pip/_internal/models/installation_report.py b/src/pip/_internal/models/installation_report.py
new file mode 100644
--- /dev/null
+++ b/src/pip/_internal/models/installation_report.py
@@ -0,0 +1,53 @@
+from typing import Any, Dict, Sequence
+
+from pip._vendor.packaging.markers import default_environment
+
+from pip import __version__
+from pip._internal.req.req_install import InstallRequirement
+
+
+class InstallationReport:
+    def __init__(self, install_requirements: Sequence[InstallRequirement]):
+        self._install_requirements = install_requirements
+
+    @classmethod
+    def _install_req_to_dict(cls, ireq: InstallRequirement) -> Dict[str, Any]:
+        assert ireq.download_info, f"No download_info for {ireq}"
+        res = {
+            # PEP 610 json for the download URL. download_info.archive_info.hash may
+            # be absent when the requirement was installed from the wheel cache
+            # and the cache entry was populated by an older pip version that did not
+            # record origin.json.
+            "download_info": ireq.download_info.to_dict(),
+            # is_direct is true if the requirement was a direct URL reference (which
+            # includes editable requirements), and false if the requirement was
+            # downloaded from a PEP 503 index or --find-links.
+            "is_direct": bool(ireq.original_link),
+            # requested is true if the requirement was specified by the user (aka
+            # top level requirement), and false if it was installed as a dependency of a
+            # requirement. https://peps.python.org/pep-0376/#requested
+            "requested": ireq.user_supplied,
+            # PEP 566 json encoding for metadata
+            # https://www.python.org/dev/peps/pep-0566/#json-compatible-metadata
+            "metadata": ireq.get_dist().metadata_dict,
+        }
+        if ireq.user_supplied and ireq.extras:
+            # For top level requirements, the list of requested extras, if any.
+            res["requested_extras"] = list(sorted(ireq.extras))
+        return res
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {
+            "version": "0",
+            "pip_version": __version__,
+            "install": [
+                self._install_req_to_dict(ireq) for ireq in self._install_requirements
+            ],
+            # https://peps.python.org/pep-0508/#environment-markers
+            # TODO: currently, the resolver uses the default environment to evaluate
+            # environment markers, so that is what we report here. In the future, it
+            # should also take into account options such as --python-version or
+            # --platform, perhaps under the form of an environment_override field?
+            # https://github.com/pypa/pip/issues/11198
+            "environment": default_environment(),
+        }


Test patch:
diff --git a/tests/functional/test_install_report.py b/tests/functional/test_install_report.py
new file mode 100644
--- /dev/null
+++ b/tests/functional/test_install_report.py
@@ -0,0 +1,214 @@
+import json
+from pathlib import Path
+from typing import Any, Dict
+
+import pytest
+from packaging.utils import canonicalize_name
+
+from ..lib import PipTestEnvironment, TestData
+
+
+def _install_dict(report: Dict[str, Any]) -> Dict[str, Any]:
+    return {canonicalize_name(i["metadata"]["name"]): i for i in report["install"]}
+
+
+@pytest.mark.usefixtures("with_wheel")
+def test_install_report_basic(
+    script: PipTestEnvironment, shared_data: TestData, tmp_path: Path
+) -> None:
+    report_path = tmp_path / "report.json"
+    script.pip(
+        "install",
+        "simplewheel",
+        "--dry-run",
+        "--no-index",
+        "--find-links",
+        str(shared_data.root / "packages/"),
+        "--report",
+        str(report_path),
+        allow_stderr_warning=True,
+    )
+    report = json.loads(report_path.read_text())
+    assert "install" in report
+    assert len(report["install"]) == 1
+    simplewheel_report = _install_dict(report)["simplewheel"]
+    assert simplewheel_report["metadata"]["name"] == "simplewheel"
+    assert simplewheel_report["requested"] is True
+    assert simplewheel_report["is_direct"] is False
+    url = simplewheel_report["download_info"]["url"]
+    assert url.startswith("file://")
+    assert url.endswith("/packages/simplewheel-2.0-1-py2.py3-none-any.whl")
+    assert (
+        simplewheel_report["download_info"]["archive_info"]["hash"]
+        == "sha256=191d6520d0570b13580bf7642c97ddfbb46dd04da5dd2cf7bef9f32391dfe716"
+    )
+
+
+@pytest.mark.usefixtures("with_wheel")
+def test_install_report_dep(
+    script: PipTestEnvironment, shared_data: TestData, tmp_path: Path
+) -> None:
+    """Test dependencies are present in the install report with requested=False."""
+    report_path = tmp_path / "report.json"
+    script.pip(
+        "install",
+        "require_simple",
+        "--dry-run",
+        "--no-index",
+        "--find-links",
+        str(shared_data.root / "packages/"),
+        "--report",
+        str(report_path),
+        allow_stderr_warning=True,
+    )
+    report = json.loads(report_path.read_text())
+    assert len(report["install"]) == 2
+    assert _install_dict(report)["require-simple"]["requested"] is True
+    assert _install_dict(report)["simple"]["requested"] is False
+
+
+@pytest.mark.network
+@pytest.mark.usefixtures("with_wheel")
+def test_install_report_index(script: PipTestEnvironment, tmp_path: Path) -> None:
+    """Test report for sdist obtained from index."""
+    report_path = tmp_path / "report.json"
+    script.pip(
+        "install",
+        "--dry-run",
+        "Paste[openid]==1.7.5.1",
+        "--report",
+        str(report_path),
+        allow_stderr_warning=True,
+    )
+    report = json.loads(report_path.read_text())
+    assert len(report["install"]) == 2
+    install_dict = _install_dict(report)
+    assert install_dict["paste"]["requested"] is True
+    assert install_dict["python-openid"]["requested"] is False
+    paste_report = install_dict["paste"]
+    assert paste_report["download_info"]["url"].startswith(
+        "https://files.pythonhosted.org/"
+    )
+    assert paste_report["download_info"]["url"].endswith("/Paste-1.7.5.1.tar.gz")
+    assert (
+        paste_report["download_info"]["archive_info"]["hash"]
+        == "sha256=11645842ba8ec986ae8cfbe4c6cacff5c35f0f4527abf4f5581ae8b4ad49c0b6"
+    )
+    assert paste_report["requested_extras"] == ["openid"]
+    assert "requires_dist" in paste_report["metadata"]
+
+
+@pytest.mark.network
+@pytest.mark.usefixtures("with_wheel")
+def test_install_report_vcs_and_wheel_cache(
+    script: PipTestEnvironment, tmp_path: Path
+) -> None:
+    """Test report for VCS reference, and interactions with the wheel cache."""
+    cache_dir = tmp_path / "cache"
+    report_path = tmp_path / "report.json"
+    script.pip(
+        "install",
+        "git+https://github.com/pypa/pip-test-package"
+        "@5547fa909e83df8bd743d3978d6667497983a4b7",
+        "--cache-dir",
+        str(cache_dir),
+        "--report",
+        str(report_path),
+        allow_stderr_warning=True,
+    )
+    report = json.loads(report_path.read_text())
+    assert len(report["install"]) == 1
+    pip_test_package_report = report["install"][0]
+    assert pip_test_package_report["is_direct"] is True
+    assert pip_test_package_report["requested"] is True
+    assert (
+        pip_test_package_report["download_info"]["url"]
+        == "https://github.com/pypa/pip-test-package"
+    )
+    assert pip_test_package_report["download_info"]["vcs_info"]["vcs"] == "git"
+    assert (
+        pip_test_package_report["download_info"]["vcs_info"]["commit_id"]
+        == "5547fa909e83df8bd743d3978d6667497983a4b7"
+    )
+    # Now do it again to make sure the cache is used and that the report still contains
+    # the original VCS url.
+    report_path.unlink()
+    result = script.pip(
+        "install",
+        "pip-test-package @ git+https://github.com/pypa/pip-test-package"
+        "@5547fa909e83df8bd743d3978d6667497983a4b7",
+        "--ignore-installed",
+        "--cache-dir",
+        str(cache_dir),
+        "--report",
+        str(report_path),
+        allow_stderr_warning=True,
+    )
+    assert "Using cached pip_test_package" in result.stdout
+    report = json.loads(report_path.read_text())
+    assert len(report["install"]) == 1
+    pip_test_package_report = report["install"][0]
+    assert pip_test_package_report["is_direct"] is True
+    assert pip_test_package_report["requested"] is True
+    assert (
+        pip_test_package_report["download_info"]["url"]
+        == "https://github.com/pypa/pip-test-package"
+    )
+    assert pip_test_package_report["download_info"]["vcs_info"]["vcs"] == "git"
+    assert (
+        pip_test_package_report["download_info"]["vcs_info"]["commit_id"]
+        == "5547fa909e83df8bd743d3978d6667497983a4b7"
+    )
+
+
+@pytest.mark.network
+@pytest.mark.usefixtures("with_wheel")
+def test_install_report_vcs_editable(
+    script: PipTestEnvironment, tmp_path: Path
+) -> None:
+    """Test report remote editable."""
+    report_path = tmp_path / "report.json"
+    script.pip(
+        "install",
+        "--editable",
+        "git+https://github.com/pypa/pip-test-package"
+        "@5547fa909e83df8bd743d3978d6667497983a4b7"
+        "#egg=pip-test-package",
+        "--report",
+        str(report_path),
+        allow_stderr_warning=True,
+    )
+    report = json.loads(report_path.read_text())
+    assert len(report["install"]) == 1
+    pip_test_package_report = report["install"][0]
+    assert pip_test_package_report["is_direct"] is True
+    assert pip_test_package_report["download_info"]["url"].startswith("file://")
+    assert pip_test_package_report["download_info"]["url"].endswith(
+        "/src/pip-test-package"
+    )
+    assert pip_test_package_report["download_info"]["dir_info"]["editable"] is True
+
+
+@pytest.mark.usefixtures("with_wheel")
+def test_install_report_to_stdout(
+    script: PipTestEnvironment, shared_data: TestData
+) -> None:
+    result = script.pip(
+        "install",
+        "simplewheel",
+        "--quiet",
+        "--dry-run",
+        "--no-index",
+        "--find-links",
+        str(shared_data.root / "packages/"),
+        "--report",
+        "-",
+        allow_stderr_warning=True,
+    )
+    assert result.stderr == (
+        "WARNING: --report is currently an experimental option. "
+        "The output format may change in a future release without prior warning.\n"
+    )
+    report = json.loads(result.stdout)
+    assert "install" in report
+    assert len(report["install"]) == 1