Repository: pex-tool/pex

Issue:
Pex requirement parsing is tripped up by files in the CWD with the same name as requirements' project names.
For example:
```
$ touch requests && pex requests==2.0.0 -- -c 'import requests; print(requests.__file__)'
Traceback (most recent call last):
  File "/home/jsirois/.venv/pex/bin/pex", line 8, in <module>
    sys.exit(main())
  File "/home/jsirois/.venv/pex/lib/python3.9/site-packages/pex/bin/pex.py", line 1068, in main
    pex_builder = build_pex(options.requirements, options, cache=ENV.PEX_ROOT)
  File "/home/jsirois/.venv/pex/lib/python3.9/site-packages/pex/bin/pex.py", line 946, in build_pex
    resolveds = resolve_multi(
  File "/home/jsirois/.venv/pex/lib/python3.9/site-packages/pex/resolver.py", line 1068, in resolve_multi
    direct_requirements = _parse_reqs(requirements, requirement_files, network_configuration)
  File "/home/jsirois/.venv/pex/lib/python3.9/site-packages/pex/resolver.py", line 945, in _parse_reqs
    parsed_requirements.extend(parse_requirement_strings(requirements))
  File "/home/jsirois/.venv/pex/lib/python3.9/site-packages/pex/requirements.py", line 774, in parse_requirement_strings
    yield _parse_requirement_line(
  File "/home/jsirois/.venv/pex/lib/python3.9/site-packages/pex/requirements.py", line 615, in _parse_requirement_line
    requirement = parse_requirement_from_dist(
  File "/home/jsirois/.venv/pex/lib/python3.9/site-packages/pex/requirements.py", line 300, in parse_requirement_from_dist
    project_name_and_version = dist_metadata.project_name_and_version(dist)
  File "/home/jsirois/.venv/pex/lib/python3.9/site-packages/pex/dist_metadata.py", line 200, in project_name_and_version
    return ProjectNameAndVersion.from_filename(dist)
  File "/home/jsirois/.venv/pex/lib/python3.9/site-packages/pex/dist_metadata.py", line 180, in from_filename
    raise MetadataError(
pex.dist_metadata.MetadataError: The distribution at path '/home/jsirois/dev/pantsbuild/jsirois-pex/requests' does not have a file name matching known sdist or wheel file name formats.
```

This only happens for files. Directories are handled just fine as is often exercised in the Pex repo where there is a top-level `pex` dir; yet `pex pex -cpex -opex.pex` works just fine.



Discussion:
This was spotted over in Pants trying to upgrade to Pex 2.1.25. Pants is notable here in that it uses the Pex PEX (a file named `pex`) to do requirement resolves. When it tries to resolve the requirement string `pex==2.1.25` it hits:
```
$ ./pants test src/python/pants/init/load_backends_integration_test.py 
16:13:03.88 [INFO] initializing pantsd...
16:13:04.71 [INFO] pantsd initialized.
16:13:10.53 [INFO] Completed: Resolving 3rdparty/python/constraints.txt
16:13:10.53 [INFO] Canceled: Building pytest.pex with 6 requirements: ipdb, pygments, pytest-cov>=2.10.1,<2.11, pytest-icdiff, pytest>=6.0.1,<6.3, zipp==2.1.0
16:13:10.53 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/bin/local_pants_runner.py", line 265, in run
    engine_result = self._perform_run(goals)
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/bin/local_pants_runner.py", line 188, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/bin/local_pants_runner.py", line 210, in _perform_run_body
    poll_delay=(0.1 if poll else None),
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/init/engine_initializer.py", line 132, in run_goal_rules
    goal_product, params, poll=poll, poll_delay=poll_delay
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/engine/internals/scheduler.py", line 568, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/engine/internals/scheduler.py", line 532, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),

Exception message: 1 Exception encountered:

Engine traceback:
  in select
  in pants.core.goals.test.run_tests
  in pants.core.goals.test.enrich_test_result (src/python/pants/init/load_backends_integration_test.py:integration)
  in pants.backend.python.goals.pytest_runner.run_python_test (src/python/pants/init/load_backends_integration_test.py:integration)
  in pants.backend.python.goals.pytest_runner.setup_pytest_for_target
  in pants.backend.python.util_rules.pex.create_pex
  in pants.engine.process.fallible_to_exec_result_or_raise
Traceback (most recent call last):
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/engine/process.py", line 259, in fallible_to_exec_result_or_raise
    description.value,
pants.engine.process.ProcessExecutionFailure: Process 'Resolving 3rdparty/python/constraints.txt' failed with exit code 1.
stdout:

stderr:
Traceback (most recent call last):
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 477, in execute
    exit_code = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 394, in _wrap_coverage
    return runner(*args)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 425, in _wrap_profiling
    return runner(*args)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 533, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 635, in execute_entry
    return runner(entry_point)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 651, in execute_pkg_resources
    return runner()
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/bin/pex.py", line 1068, in main
    pex_builder = build_pex(options.requirements, options, cache=ENV.PEX_ROOT)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/bin/pex.py", line 964, in build_pex
    ignore_errors=options.ignore_errors,
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/resolver.py", line 1068, in resolve_multi
    direct_requirements = _parse_reqs(requirements, requirement_files, network_configuration)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/resolver.py", line 945, in _parse_reqs
    parsed_requirements.extend(parse_requirement_strings(requirements))
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/requirements.py", line 780, in parse_requirement_strings
    end_line=1,
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/requirements.py", line 616, in _parse_requirement_line
    archive_or_project_path, extras=extras, marker=marker
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/requirements.py", line 300, in parse_requirement_from_dist
    project_name_and_version = dist_metadata.project_name_and_version(dist)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/dist_metadata.py", line 200, in project_name_and_version
    return ProjectNameAndVersion.from_filename(dist)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/dist_metadata.py", line 182, in from_filename
    "file name formats.".format(path)
pex.dist_metadata.MetadataError: The distribution at path '/tmp/process-executionqRWad1/pex' does not have a file name matching known sdist or wheel file name formats.


Traceback (most recent call last):
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/bin/local_pants_runner.py", line 265, in run
    engine_result = self._perform_run(goals)
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/bin/local_pants_runner.py", line 188, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/bin/local_pants_runner.py", line 210, in _perform_run_body
    poll_delay=(0.1 if poll else None),
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/init/engine_initializer.py", line 132, in run_goal_rules
    goal_product, params, poll=poll, poll_delay=poll_delay
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/engine/internals/scheduler.py", line 568, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/engine/internals/scheduler.py", line 532, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),
pants.engine.internals.scheduler.ExecutionError: 1 Exception encountered:

Engine traceback:
  in select
  in pants.core.goals.test.run_tests
  in pants.core.goals.test.enrich_test_result (src/python/pants/init/load_backends_integration_test.py:integration)
  in pants.backend.python.goals.pytest_runner.run_python_test (src/python/pants/init/load_backends_integration_test.py:integration)
  in pants.backend.python.goals.pytest_runner.setup_pytest_for_target
  in pants.backend.python.util_rules.pex.create_pex
  in pants.engine.process.fallible_to_exec_result_or_raise
Traceback (most recent call last):
  File "/home/jsirois/dev/pantsbuild/jsirois-pants/src/python/pants/engine/process.py", line 259, in fallible_to_exec_result_or_raise
    description.value,
pants.engine.process.ProcessExecutionFailure: Process 'Resolving 3rdparty/python/constraints.txt' failed with exit code 1.
stdout:

stderr:
Traceback (most recent call last):
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 477, in execute
    exit_code = self._wrap_coverage(self._wrap_profiling, self._execute)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 394, in _wrap_coverage
    return runner(*args)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 425, in _wrap_profiling
    return runner(*args)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 533, in _execute
    return self.execute_entry(self._pex_info.entry_point)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 635, in execute_entry
    return runner(entry_point)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.bootstrap/pex/pex.py", line 651, in execute_pkg_resources
    return runner()
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/bin/pex.py", line 1068, in main
    pex_builder = build_pex(options.requirements, options, cache=ENV.PEX_ROOT)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/bin/pex.py", line 964, in build_pex
    ignore_errors=options.ignore_errors,
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/resolver.py", line 1068, in resolve_multi
    direct_requirements = _parse_reqs(requirements, requirement_files, network_configuration)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/resolver.py", line 945, in _parse_reqs
    parsed_requirements.extend(parse_requirement_strings(requirements))
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/requirements.py", line 780, in parse_requirement_strings
    end_line=1,
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/requirements.py", line 616, in _parse_requirement_line
    archive_or_project_path, extras=extras, marker=marker
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/requirements.py", line 300, in parse_requirement_from_dist
    project_name_and_version = dist_metadata.project_name_and_version(dist)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/dist_metadata.py", line 200, in project_name_and_version
    return ProjectNameAndVersion.from_filename(dist)
  File "/home/jsirois/.cache/pants/named_caches/pex_root/unzipped_pexes/139c4021d7418da8f4255e16de6a69aa46c7d82c/.deps/pex-2.1.25-py2.py3-none-any.whl/pex/dist_metadata.py", line 182, in from_filename
    "file name formats.".format(path)
pex.dist_metadata.MetadataError: The distribution at path '/tmp/process-executionqRWad1/pex' does not have a file name matching known sdist or wheel file name formats.

```

Resolving patch:
diff --git a/pex/dist_metadata.py b/pex/dist_metadata.py
--- a/pex/dist_metadata.py
+++ b/pex/dist_metadata.py
@@ -28,6 +28,10 @@ class MetadataError(Exception):
     """Indicates an error reading distribution metadata."""
 
 
+class UnrecognizedDistributionFormat(MetadataError):
+    """Indicates a distribution file is not of any recognized format."""
+
+
 _PKG_INFO_BY_DIST = {}  # type: Dict[Distribution, Optional[Message]]
 
 
@@ -177,7 +181,7 @@ def from_filename(cls, path):
             project_name, version = fname.rsplit("-", 1)
             return cls(project_name=project_name, version=version)
 
-        raise MetadataError(
+        raise UnrecognizedDistributionFormat(
             "The distribution at path {!r} does not have a file name matching known sdist or wheel "
             "file name formats.".format(path)
         )
diff --git a/pex/requirements.py b/pex/requirements.py
--- a/pex/requirements.py
+++ b/pex/requirements.py
@@ -612,10 +612,17 @@ def _parse_requirement_line(
                 marker=marker,
                 editable=editable,
             )
-        requirement = parse_requirement_from_dist(
-            archive_or_project_path, extras=extras, marker=marker
-        )
-        return URLRequirement.create(line, archive_or_project_path, requirement, editable=editable)
+        try:
+            requirement = parse_requirement_from_dist(
+                archive_or_project_path, extras=extras, marker=marker
+            )
+            return URLRequirement.create(
+                line, archive_or_project_path, requirement, editable=editable
+            )
+        except dist_metadata.UnrecognizedDistributionFormat:
+            # This is not a recognized local archive distribution. Fall through and try parsing as a
+            # PEP-440 requirement.
+            pass
 
     # Handle PEP-440. See: https://www.python.org/dev/peps/pep-0440.
     #


Test patch:
diff --git a/tests/test_requirements.py b/tests/test_requirements.py
--- a/tests/test_requirements.py
+++ b/tests/test_requirements.py
@@ -309,6 +309,11 @@ def test_parse_requirements_stress(chroot):
             ),
         )
     )
+
+    # Ensure local non-distribution files matching distribution names are not erroneously probed
+    # as distributions to find name and version metadata.
+    touch("nose")
+
     touch("downloads/numpy-1.9.2-cp34-none-win32.whl")
     with environment_as(PROJECT_NAME="Project"):
         results = normalize_results(req_iter)