Repository: trailofbits/pip-audit (python)

Issue:
pip-audit not auditing the same package versions as pip installs.
## Bug description

My understanding of pip-audit is that, when given a ```requirements.txt``` file, it would audit the same versions of the same packages that ```pip``` would install. This doesn't seem to be the case.

I have a ```requirements.txt``` file containing just the cryptography module with no version constraint specified.

If I create a venv and install the package then it installs:
* cryptography 36.0.2
* cffi 1.15.0 (as cryptography has a requirement for cffi>=1.1.2)
* pcparser 2.21

If I run `pip-audit -r requirement.txt` then it tries to install cffi version 1.0.2-2 rather than 1.15.0 and fails with:

`pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/tmp/tmpffo5omkp/bin/python3', '-m', 'pip', 'install', '/tmp/tmp4dbeewpp/cffi-1.0.2-2.tar.gz']`

If I change the `requirements.txt` file to contain `cryptography==36.0.2`, the same thing happens.

Likewise, if I append `cffi==1.15.0` to the `requirements.txt` file pip-audit still tries, and fails, to use cffi 1.0.2-2

## Reproduction steps

Running in Ubuntu 22.04 Docker container with just python3, venv, git installed.
Running as a non-root user in the container I install pip-audit, either via pip, or directly from the git repo
Create requirements.txt containing just `cryptography`
Create a venv, install cryptography package from requirements.txt using pip
Run pip list to confirm package versions installed as listed above (specifically cffi 1.15.0)
Run pip-audit -r requirements.txt
pip-audit tries to install cffi 1.0.2.2 and fails, as above

## Expected behavior

pip-audit should audit the same packages and the same versions of the packages as pip install installs
pip-audit should not fail 

## Screenshots and logs

Dockerfile used:
```
# Start with up-to-date Ubuntu

FROM ubuntu:22.04

# User to run with

ARG BUILD_USER=build
ARG BUILD_UID=1000
ARG BUILD_GID=1000
ARG HOME_DIR=/home/build

# Update & upgrade, install minimal Python setup

RUN apt update
RUN apt upgrade --yes
RUN apt install --yes apt-utils
RUN apt install --yes python3 python3-pip python3-venv git

# Add the user

RUN adduser --disabled-password --gecos '' ${BUILD_USER}
USER ${BUILD_UID}

WORKDIR ${HOME_DIR}

# Install pip-audit

#RUN pip install --upgrade --no-warn-script-location pip-audit
RUN python3 -m pip install git+https://github.com/trailofbits/pip-audit
```

Contents of requirements.txt file:
```
cryptography
```

Output from `pip list` in venv after installing cryptography package:

```
Package      Version
------------ -------
cffi         1.15.0
cryptography 36.0.2
pip          22.0.2
pycparser    2.21
setuptools   59.6.0
```

Output from `pip-audit -v -r requirements.txt`:
```
DEBUG:pip_audit._cli:parsed arguments: Namespace(local=False, requirements=[<_io.TextIOWrapper name='requirements.txt' mode='r' encoding='UTF-8'>], project_path=None, format=<OutputFormatChoice.Columns: 'columns'>, vulnerability_service=<VulnerabilityServiceChoice.Pypi: 'pypi'>, dry_run=False, strict=False, desc=<VulnerabilityDescriptionChoice.Auto: 'auto'>, cache_dir=None, progress_spinner=<ProgressSpinnerChoice.On: 'on'>, timeout=15, paths=[], verbose=True, fix=False, require_hashes=False, index_url='https://pypi.org/simple', extra_index_urls=[], skip_editable=False)
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cryptography" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): pypi.org:443
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cryptography HTTP/1.1" 301 118
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/cryptography"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cryptography/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 153
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 153
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/5d/a9/b73a5d6f50a7b2f6ef65a2d2a14e848b62dfc79d10d29277586a94cf1f23/cryptography-36.0.2-cp36-abi3-macosx_10_10_universal2.whl" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): files.pythonhosted.org:443
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/5d/a9/b73a5d6f50a7b2f6ef65a2d2a14e848b62dfc79d10d29277586a94cf1f23/cryptography-36.0.2-cp36-abi3-macosx_10_10_universal2.whl HTTP/1.1" 200 4716191
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/5d/a9/b73a5d6f50a7b2f6ef65a2d2a14e848b62dfc79d10d29277586a94cf1f23/cryptography-36.0.2-cp36-abi3-macosx_10_10_universal2.whl"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cffi" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://pypi.org:443 "GET /simple/cffi HTTP/1.1" 301 110
DEBUG:cachecontrol.controller:Updating cache with response from "https://pypi.org/simple/cffi"
DEBUG:cachecontrol.controller:Caching permanent redirect
DEBUG:cachecontrol.controller:Looking up "https://pypi.org/simple/cffi/" in the cache
DEBUG:cachecontrol.controller:Current age based on date: 153
DEBUG:cachecontrol.controller:Freshness lifetime from max-age: 600
DEBUG:cachecontrol.controller:The response is "fresh", returning cached response
DEBUG:cachecontrol.controller:600 > 153
DEBUG:cachecontrol.controller:Looking up "https://files.pythonhosted.org/packages/ef/23/c6f7003ebb7b4b3fe4872f112b18ee181a3ec2b137e964093a8b35d4a5bd/cffi-1.0.2-2.tar.gz" in the cache
DEBUG:cachecontrol.controller:No cache entry available
DEBUG:urllib3.connectionpool:https://files.pythonhosted.org:443 "GET /packages/ef/23/c6f7003ebb7b4b3fe4872f112b18ee181a3ec2b137e964093a8b35d4a5bd/cffi-1.0.2-2.tar.gz HTTP/1.1" 200 317417
DEBUG:cachecontrol.controller:Ignoring unknown cache-control directive: immutable
DEBUG:cachecontrol.controller:Updating cache with response from "https://files.pythonhosted.org/packages/ef/23/c6f7003ebb7b4b3fe4872f112b18ee181a3ec2b137e964093a8b35d4a5bd/cffi-1.0.2-2.tar.gz"
DEBUG:cachecontrol.controller:etag object cached for 1209600 seconds
DEBUG:cachecontrol.controller:Caching due to etag
Traceback (most recent call last):
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_virtual_env.py", line 103, in post_setup
    run(package_install_cmd, state=self._state)
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_subprocess.py", line 51, in run
    raise CalledProcessError(f"{pretty_args} exited with {process.returncode}")
pip_audit._subprocess.CalledProcessError: python3 -m pip install /tmp/tmps9lu0wkz/cffi-1.0.2-2.tar.gz exited with 1

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/build/.local/bin/pip-audit", line 8, in <module>
    sys.exit(audit())
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_cli.py", line 357, in audit
    for (spec, vulns) in auditor.audit(source):
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_audit.py", line 66, in audit
    for dep, vulns in self._service.query_all(specs):
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_service/interface.py", line 142, in query_all
    for spec in specs:
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/requirement.py", line 98, in collect
    for _, deps in self._resolver.resolve_all(iter(req_values)):
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/interface.py", line 87, in resolve_all
    yield (req, self.resolve(req))
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/resolvelib.py", line 73, in resolve
    result = self.resolver.resolve([req])
  File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 373, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
  File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 213, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
  File "/home/build/.local/lib/python3.10/site-packages/resolvelib/resolvers.py", line 203, in _get_updated_criteria
    for requirement in self._p.get_dependencies(candidate=candidate):
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 341, in get_dependencies
    return candidate.dependencies
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 119, in dependencies
    self._dependencies = list(self._get_dependencies())
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 101, in _get_dependencies
    deps: List[str] = self.metadata.get_all("Requires-Dist", [])
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 94, in metadata
    self._metadata = self._get_metadata_for_sdist()
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_dependency_source/resolvelib/pypi_provider.py", line 162, in _get_metadata_for_sdist
    ve.create(ve_dir)
  File "/usr/lib/python3.10/venv/__init__.py", line 78, in create
    self.post_setup(context)
  File "/home/build/.local/lib/python3.10/site-packages/pip_audit/_virtual_env.py", line 105, in post_setup
    raise VirtualEnvError(f"Failed to install packages: {package_install_cmd}") from cpe
pip_audit._virtual_env.VirtualEnvError: Failed to install packages: ['/tmp/tmpjk63y53b/bin/python3', '-m', 'pip', 'install', '/tmp/tmps9lu0wkz/cffi-1.0.2-2.tar.gz']
```

## Platform information

* OS name and version: Ubuntu 22.04 (in Docker container)
* `pip-audit` version (`pip-audit -V`): pip-audit 2.1.0
* Python version (`python -V` or `python3 -V`): Python 3.10.3
* `pip` version (`pip -V` or `pip3 -V`): pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)

Pull request description:
pip_audit/dependency_source: match candidate names against project
See: https://github.com/pypa/packaging/issues/527.

Fixes #248.

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e28cd2c..399b074 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,11 @@ All versions prior to 0.0.9 are untracked.
 
 ## [Unreleased] - ReleaseDate
 
+### Fixed
+
+* Dependency sources: A bug caused by ambiguous parses of source distribution
+  files was fixed ([#249](https://github.com/trailofbits/pip-audit/pull/249))
+
 ## [2.1.0] - 2022-03-11
 
 ### Added
diff --git a/pip_audit/_dependency_source/resolvelib/pypi_provider.py b/pip_audit/_dependency_source/resolvelib/pypi_provider.py
index e10ecfe..e5e7a62 100644
--- a/pip_audit/_dependency_source/resolvelib/pypi_provider.py
+++ b/pip_audit/_dependency_source/resolvelib/pypi_provider.py
@@ -310,6 +310,13 @@ class PyPIProvider(AbstractProvider):
                 )
                 if candidate.version not in bad_versions
                 and all(candidate.version in r.specifier for r in requirements)
+                # HACK(ww): Additionally check that each candidate's name matches the
+                # expected project name (identifier).
+                # This technically shouldn't be required, but parsing distribution names
+                # from package indices is imprecise/unreliable when distribution filenames
+                # are PEP 440 compliant but not normalized.
+                # See: https://github.com/pypa/packaging/issues/527
+                and candidate.name == identifier
             ],
             key=attrgetter("version", "is_wheel"),
             reverse=True,
@@ -330,8 +337,6 @@ class PyPIProvider(AbstractProvider):
         """
         See `resolvelib.providers.AbstractProvider.is_satisfied_by`.
         """
-        if canonicalize_name(requirement.name) != candidate.name:
-            return False
         return candidate.version in requirement.specifier
 
     def get_dependencies(self, candidate):
diff --git a/setup.py b/setup.py
index 7f6626c..dc10878 100644
--- a/setup.py
+++ b/setup.py
@@ -41,6 +41,8 @@ setup(
             "bump >= 1.3.1",
             "flake8",
             "black",
+            # See: https://github.com/psf/black/issues/2964
+            "click >= 8.0.0, < 8.1.0",
             "isort",
             "pytest",
             "pytest-cov",


Test patch:
diff --git a/test/dependency_source/test_resolvelib.py b/test/dependency_source/test_resolvelib.py
index 5ac27b7..189f68e 100644
--- a/test/dependency_source/test_resolvelib.py
+++ b/test/dependency_source/test_resolvelib.py
@@ -7,7 +7,7 @@ from packaging.requirements import Requirement
 from packaging.version import Version
 from pip_api import Requirement as ParsedRequirement
 from requests.exceptions import HTTPError
-from resolvelib.resolvers import InconsistentCandidate, ResolutionImpossible
+from resolvelib.resolvers import ResolutionImpossible
 
 from pip_audit._dependency_source import resolvelib
 from pip_audit._dependency_source.resolvelib import pypi_provider
@@ -172,6 +172,34 @@ def test_resolvelib_sdist_patched(monkeypatch, suffix):
     assert resolved_deps[req] == [ResolvedDependency("flask", Version("2.0.1"))]
 
 
+def test_resolvelib_sdist_vexing_parse(monkeypatch):
+    # Some sdist filenames have ambiguous parses: `cffi-1.0.2-2.tar.gz`
+    # could be parsed as `(cffi, 1.0.2.post2)` or `(cffi-1-0-2, 2)`.
+    # `packaging.utils.parse_sdist_filename` parses it as the latter, which results
+    # in a wrong version for `cffi`.
+    # When this happens, we filter by distribution to ensure we don't select
+    # an incorrect version number.
+    data = (
+        '<a href="https://files.pythonhosted.org/packages/54/4f/'
+        "1b294c1a4ab7b2ad5ca5fc4a9a65a22ef1ac48be126289d97668852d4ab3/cffi-1.0.2-2.tar.gz#"
+        'sha256=a6209ca15eb63fc9385f38e452704113d679511d9574d09b2cf9183ae7d20dc9">'
+        "cffi-1.0.2-2.tar.gz</a><br/>"
+    )
+
+    monkeypatch.setattr(
+        pypi_provider.Candidate, "_get_metadata_for_wheel", lambda _: get_metadata_mock()
+    )
+
+    resolver = resolvelib.ResolveLibResolver()
+    monkeypatch.setattr(
+        resolver.provider.session, "get", lambda _url, **kwargs: get_package_mock(data)
+    )
+
+    req = Requirement("cffi")
+    with pytest.raises(ResolutionImpossible):
+        dict(resolver.resolve_all(iter([req])))
+
+
 def test_resolvelib_wheel_python_version(monkeypatch):
     # Some versions stipulate a particular Python version and should be skipped by the provider.
     # Since `pip-audit` doesn't support Python 2.7, the Flask version below should always be skipped
@@ -194,7 +222,7 @@ def test_resolvelib_wheel_python_version(monkeypatch):
 
 
 def test_resolvelib_wheel_canonical_name_mismatch(monkeypatch):
-    # Call the underlying wheel, Mask instead of Flask. This should throw an `InconsistentCandidate`
+    # Call the underlying wheel, Mask instead of Flask. This should throw an `ResolutionImpossible`
     # error.
     data = (
         '<a href="https://files.pythonhosted.org/packages/54/4f/'
@@ -213,7 +241,7 @@ def test_resolvelib_wheel_canonical_name_mismatch(monkeypatch):
     )
 
     req = Requirement("flask==2.0.1")
-    with pytest.raises(InconsistentCandidate):
+    with pytest.raises(ResolutionImpossible):
         dict(resolver.resolve_all(iter([req])))