Repository: pypa/pip

Issue:
pip 21.1 fails with ResolutionTooDeep while 21.0.1 exits with clear error
### Description

pip 21.0.1 has no error and highlights the dependency error right away while pip 2.1 runs for minutes then throws a _ResolutionTooDeep_ exception.


### Expected behavior

pip version 21.0.1 produces the expected output which includes this diagnostic:

```
The conflict is caused by:
The user requested hyperlink==19.0.0
    autobahn 20.12.3 depends on hyperlink>=20.0.1
```

### pip version

21.1

### Python version

3.6.13

### OS

Ubuntu 16.04.7 LTS

### How to Reproduce

1. Create a python3.6 virtualenv
2. activate
3. Ensure pip v21.1 is installed in the virtualenv
4. run `pip -r r.txt` where r.txt has this content:

```
attrs==19.3.0
autobahn==20.6.2
hyperlink==19.0.0
cffi==1.14.0
cryptography>=3.2
idna==2.10
pycparser==2.20
txaio==20.4.1
```

5. Replace `autobahn==20.6.2` with `autobahn==20.12.3` in r.txt
6. run `pip -r r.txt`

### Output

```sh-session
Lots of spew, then:

  Requirement already satisfied: txaio==20.4.1 in ./venv/lib/python3.6/site-packages (from -rr test.txt (line 8)) (20.4.1)
  INFO: pip is looking at multiple versions of attrs to determine which version is compatible with other requirements. This could take a while.

Then pip seems to hang.  If you wait long enough: 5 minutes? it prints:

ERROR: Exception:
Traceback (most recent call last):
  File "/home/sigma/dev/contour/daqAdaptor/venv/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 180, in _main
    status = self.run(options, args)
  File "/home/sigma/dev/contour/daqAdaptor/venv/lib/python3.6/site-packages/pip/_internal/cli/req_command.py", line 204, in wrapper
    return func(self, options, args)
  File "/home/sigma/dev/contour/daqAdaptor/venv/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 319, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/home/sigma/dev/contour/daqAdaptor/venv/lib/python3.6/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 128, in resolve
    requirements, max_rounds=try_to_avoid_resolution_too_deep
  File "/home/sigma/dev/contour/daqAdaptor/venv/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 473, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/home/sigma/dev/contour/daqAdaptor/venv/lib/python3.6/site-packages/pip/_vendor/resolvelib/resolvers.py", line 384, in resolve
    raise ResolutionTooDeep(max_rounds)
pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 2000000
```


### Code of Conduct

- [X] I agree to follow the [PSF Code of Conduct](https://www.python.org/psf/conduct/).


Discussion:
FYI the versions are 21.0.1 and 21.1, not 2.0 and 2.1. You’re one digit off.
Ahh.  What's  a digit here or there :)   I updated the original comment.
I'm also facing this error. 
@junpuf Please describe what you were doing when you encountered the issue.
Hi @uranusjr 
Sorry for the delayed response.

I have seen this error from one of my conda environment build process using `environment.yml`, and pip dependencies were used and listed below:
```
"boto3",
"s3fs",
"multi-model-server==1.1.2",
"keras-mxnet==2.2.4.2",
"opencv-python==4.5.1.48"
```

Below are the error message.

```
Collecting aiobotocore>=1.0.1
Downloading aiobotocore-1.3.0.tar.gz (48 kB)
Downloading aiobotocore-1.2.2.tar.gz (48 kB)
Downloading aiobotocore-1.2.1.tar.gz (48 kB)
Downloading aiobotocore-1.2.0.tar.gz (47 kB)
Downloading aiobotocore-1.1.2-py3-none-any.whl (45 kB)
Downloading aiobotocore-1.1.1-py3-none-any.whl (45 kB)
Downloading aiobotocore-1.1.0-py3-none-any.whl (43 kB)
Downloading aiobotocore-1.0.7-py3-none-any.whl (42 kB)
Downloading aiobotocore-1.0.6-py3-none-any.whl (42 kB)
Downloading aiobotocore-1.0.5-py3-none-any.whl (42 kB)
Downloading aiobotocore-1.0.4-py3-none-any.whl (41 kB)
Downloading aiobotocore-1.0.3-py3-none-any.whl (40 kB)
Downloading aiobotocore-1.0.2-py3-none-any.whl (40 kB)
Downloading aiobotocore-1.0.1-py3-none-any.whl (40 kB)
INFO: pip is looking at multiple versions of fsspec to determine which version is compatible with other requirements. This could take a while.
Pip subprocess error:
ERROR: Exception:
...
...
pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 2000000
··failed
CondaEnvException: Pip failed

```
Thanks for the test cases! I have identified the issue and will post a patch shortly.

Resolving patch:
diff --git a/src/pip/_internal/resolution/resolvelib/factory.py b/src/pip/_internal/resolution/resolvelib/factory.py
--- a/src/pip/_internal/resolution/resolvelib/factory.py
+++ b/src/pip/_internal/resolution/resolvelib/factory.py
@@ -240,18 +240,29 @@ def _iter_found_candidates(
             hashes &= ireq.hashes(trust_internet=False)
             extras |= frozenset(ireq.extras)
 
-        # Get the installed version, if it matches, unless the user
-        # specified `--force-reinstall`, when we want the version from
-        # the index instead.
-        installed_candidate = None
-        if not self._force_reinstall and name in self._installed_dists:
-            installed_dist = self._installed_dists[name]
-            if specifier.contains(installed_dist.version, prereleases=True):
-                installed_candidate = self._make_candidate_from_dist(
-                    dist=installed_dist,
-                    extras=extras,
-                    template=template,
-                )
+        def _get_installed_candidate() -> Optional[Candidate]:
+            """Get the candidate for the currently-installed version."""
+            # If --force-reinstall is set, we want the version from the index
+            # instead, so we "pretend" there is nothing installed.
+            if self._force_reinstall:
+                return None
+            try:
+                installed_dist = self._installed_dists[name]
+            except KeyError:
+                return None
+            # Don't use the installed distribution if its version does not fit
+            # the current dependency graph.
+            if not specifier.contains(installed_dist.version, prereleases=True):
+                return None
+            candidate = self._make_candidate_from_dist(
+                dist=installed_dist,
+                extras=extras,
+                template=template,
+            )
+            # The candidate is a known incompatiblity. Don't use it.
+            if id(candidate) in incompatible_ids:
+                return None
+            return candidate
 
         def iter_index_candidate_infos():
             # type: () -> Iterator[IndexCandidateInfo]
@@ -283,7 +294,7 @@ def iter_index_candidate_infos():
 
         return FoundCandidates(
             iter_index_candidate_infos,
-            installed_candidate,
+            _get_installed_candidate(),
             prefers_installed,
             incompatible_ids,
         )


Test patch:
diff --git a/tests/functional/test_new_resolver.py b/tests/functional/test_new_resolver.py
--- a/tests/functional/test_new_resolver.py
+++ b/tests/functional/test_new_resolver.py
@@ -1827,3 +1827,33 @@ def test_new_resolver_direct_url_with_extras(tmp_path, script):
     assert not get_created_direct_url(result, "pkg1")
     assert get_created_direct_url(result, "pkg2")
     assert not get_created_direct_url(result, "pkg3")
+
+
+def test_new_resolver_modifies_installed_incompatible(script):
+    create_basic_wheel_for_package(script, name="a", version="1")
+    create_basic_wheel_for_package(script, name="a", version="2")
+    create_basic_wheel_for_package(script, name="a", version="3")
+    create_basic_wheel_for_package(script, name="b", version="1", depends=["a==1"])
+    create_basic_wheel_for_package(script, name="b", version="2", depends=["a==2"])
+    create_basic_wheel_for_package(script, name="c", version="1", depends=["a!=1"])
+    create_basic_wheel_for_package(script, name="c", version="2", depends=["a!=1"])
+    create_basic_wheel_for_package(script, name="d", version="1", depends=["b", "c"])
+
+    script.pip(
+        "install",
+        "--no-cache-dir", "--no-index",
+        "--find-links", script.scratch_path,
+        "b==1",
+    )
+
+    # d-1 depends on b and c. b-1 is already installed and therefore first
+    # pinned, but later found to be incompatible since the "a==1" dependency
+    # makes all c versions impossible to satisfy. The resolver should be able to
+    # discard b-1 and backtrack, so b-2 is selected instead.
+    script.pip(
+        "install",
+        "--no-cache-dir", "--no-index",
+        "--find-links", script.scratch_path,
+        "d==1",
+    )
+    assert_installed(script, d="1", c="2", b="2", a="2")