Repository: jazzband/pip-tools

Issue:
pip-compile breaks with Pip 22.1.1 - AttributeError: 'RequirementSet' object has no attribute 'add_requirement'.
Running pip-compile will break when using the latest Pip release, 22.1.1.

[This commit](https://github.com/pypa/pip/commit/d673aa14284788ea12a789b34846353b7cb3d46f) appears to be the cause of the break.

#### Environment Versions

1. OS Type: Windows 11, fully up to date
1. Python version: `Python 3.10.4`
1. pip version: `pip 22.1.1 from %MyProject%\venv\lib\site-packages\pip (python 3.10)`
1. pip-tools version: `pip-compile, version 6.6.1`

#### Steps to replicate

0. These directions are using PowerShell. They should work on other shells as well with some adjustments
1. Create a fresh virtualenv and activate it
2. python -m pip install pip==22.1.1
3. python -m pip install pip-tools==6.6.1
4. Set-Content -Value 'django' -Path .\example.in
5. pip-compile .\example.in

#### Expected result

pip-compile produces a requirements.txt file without error

#### Actual result

No requirements.txt file is produced. This callstack is shown
```
Traceback (most recent call last):
  File "C:\Program Files\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "%MyProject%\venv\Scripts\pip-compile.exe\__main__.py", line 7, in <module>
  File "%MyProject%\venv\lib\site-packages\click\core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "%MyProject%\venv\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "%MyProject%\venv\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "%MyProject%\venv\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "%MyProject%\venv\lib\site-packages\click\decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "%MyProject%\venv\lib\site-packages\piptools\scripts\compile.py", line 466, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "%MyProject%\venv\lib\site-packages\piptools\resolver.py", line 175, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "%MyProject%\venv\lib\site-packages\piptools\resolver.py", line 319, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "%MyProject%\venv\lib\site-packages\piptools\resolver.py", line 428, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "%MyProject%\venv\lib\site-packages\piptools\repositories\pypi.py", line 242, in get_dependencies
    self._dependencies_cache[ireq] = self.resolve_reqs(
  File "%MyProject%\venv\lib\site-packages\piptools\repositories\pypi.py", line 192, in resolve_reqs
    reqset.add_requirement(ireq)
AttributeError: 'RequirementSet' object has no attribute 'add_requirement'. Did you mean: 'all_requirements'?
```



Discussion:
More of a general question, but this is the second Pip update in 9 days that has rendered pip-tools unusable. Is it a bad idea to be always updating Pip to the latest release?
> More of a general question, but this is the second Pip update in 9 days that has rendered pip-tools unusable. Is it a bad idea to be always updating Pip to the latest release?

IMO it's not pip-tool's fault. pip didn't follow semantic versioning convention when a) it removed a method (=backwards incompatible change) and b) it didn't upgrade the major version.

Under these circumstances the only reliable way to go is to pin to an exact pip version (which is not a good idea anyway).


Sorry if I came across as blaming pip-tools for this, I recognize that the pip project is the one that made an incompatible change in both these instances. However, I haven't found great guidance on how to avoid things like this. Even if pip upgraded the major version for this change, the pip-tools README states that pip-tools 6.5+ is compatible with all versions of pip greater than or equal to 21.2. So based on this information, it would be reasonable to expect pip-tools to work with this new pip version, even it has a bigger version number than it actually did.
I feel like between this and #1617, there needs to be a fundamental change in how `pip` and `pip-tools` are tested/released to make this sustainable.

Some ideas:

- Put `pip-tools` testing into `pip`'s CI so that `pip` cannot be released if it breaks the latest `pip-tools`
- Vendor the important parts of `pip` into `pip-tools`
- Move `pip-tools` into the `pip` repo and version/release them together

For _users_, there are also alternative locking solutions that don't depend on `pip` internals, such as [Poetry](https://github.com/python-poetry/poetry) to consider?
> IMO it's not pip-tool's fault. pip didn't follow semantic versioning convention when a) it removed a method (=backwards incompatible change) and b) it didn't upgrade the major version.

`pip-tools` uses internals (`from pip._internal.req import InstallRequirement, RequirementSet`), so any versioning convention will not work here. `pip` is written as command line tool, not library. 

> Under these circumstances the only reliable way to go is to pin to an exact pip version (which is not a good idea anyway).

Maybe vendoring the important parts of `pip` is solution. 

Resolving patch:
diff --git a/piptools/repositories/pypi.py b/piptools/repositories/pypi.py
--- a/piptools/repositories/pypi.py
+++ b/piptools/repositories/pypi.py
@@ -189,7 +189,12 @@ def resolve_reqs(
 
             reqset = RequirementSet()
             ireq.user_supplied = True
-            reqset.add_requirement(ireq)
+            if PIP_VERSION[:3] < (22, 1, 1):
+                reqset.add_requirement(ireq)
+            elif getattr(ireq, "name", None):
+                reqset.add_named_requirement(ireq)
+            else:
+                reqset.add_unnamed_requirement(ireq)
 
             resolver = self.command.make_resolver(
                 preparer=preparer,


Test patch: