Repository: pypa/pip

Issue:
extra is not honored with the dependency resolver
Steps to reproduce:

```bash
env VIRTUALENV_PIP=20.2b1 virtualenv venv --clear; env PIP_UNSTABLE_FEATURE="resolver" venv/bin/pip install 'setuptools_scm >= 3.5[toml]'                                                                           created virtual environment CPython3.8.3.final.0-64 in 1969ms
  creator CPython3Posix(dest=/Users/bgabor8/git/dqc/ecobre/venv, clear=True, global=False)
  seeder FromAppData(download=False, pip=20.2b1, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/bgabor8/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Looking in indexes: http://localhost:3141/bernat/bb
Collecting setuptools_scm>=3.5[toml]
  Downloading http://localhost:3141/bernat/bb/%2Bf/0d2/3db3d43e0a43e/setuptools_scm-3.5.0-py2.py3-none-any.whl (26 kB)
Installing collected packages: setuptools-scm
Successfully installed setuptools-scm-3.5.0
```

without the resolver:

```bash
env VIRTUALENV_PIP=20.2b1 virtualenv venv --clear; env venv/bin/pip install 'setuptools_scm >= 3.5[toml]'                                                                                                            created virtual environment CPython3.8.3.final.0-64 in 2233ms
  creator CPython3Posix(dest=/Users/bgabor8/git/dqc/ecobre/venv, clear=True, global=False)
  seeder FromAppData(download=False, pip=20.2b1, setuptools=latest, wheel=latest, via=copy, app_data_dir=/Users/bgabor8/Library/Application Support/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Looking in indexes: http://localhost:3141/bernat/bb
Collecting setuptools_scm>=3.5[toml]
  Downloading http://localhost:3141/bernat/bb/%2Bf/0d2/3db3d43e0a43e/setuptools_scm-3.5.0-py2.py3-none-any.whl (26 kB)
Collecting toml; extra == "toml"
  Downloading http://localhost:3141/bernat/bb/%2Bf/bda/89d5935c2eac5/toml-0.10.1-py2.py3-none-any.whl (19 kB)
Installing collected packages: toml, setuptools-scm
```

This sadly is blocking me from doing more testing, as does not allow me to build a package I'll install afterwards.


Discussion:
Hmm. We read extras here:

https://github.com/pypa/pip/blob/3ec6b824ddbdef2128238858473bc80c7bf502e8/src/pip/_internal/resolution/resolvelib/requirements.py#L53-L58

But weirdly `ireq.req.extras` does not contain the extras, only `ireq.extra` does. I don’t really understand why, but will file a PR to change this anyway. We are already using `ireq.extras` in many other places in the resolver anyway, and I guess it makes sense to always do the same?

(The weird thing about this is that extras listed in a package’s `Requires-Dist` are correctly listed in `ireq.req.extras`, only the root requirements’ aren’t.)
Ah, I think I know what’s going on. `setuptools_scm >= 3.5[toml]` is actually not valid PEP 508 syntax, but pip accepts it anyway. The parser augments `packaging`’s `Requirement` to parse this additional part, and attaches that onto `InstallRequirement` to go along with the extras in `Requirement`.

So:

* `setuptools_scm>=3.5[toml]` will have `ireq.extras == {'toml'}` but `ireq.req.extras == set()`.
* `setuptools_scm[toml]>=3.5` will have `ireq.extras == ireq.req.extras == {'toml'}`.

We currently only tests the valid PEP 508 syntax variant, which is why this slipped. I’ll file the PR to switch to `ireq.extras`, but @gaborbernat you should be able to use the PEP 508 syntax to work around the issue.

Resolving patch:
diff --git a/src/pip/_internal/resolution/resolvelib/requirements.py b/src/pip/_internal/resolution/resolvelib/requirements.py
--- a/src/pip/_internal/resolution/resolvelib/requirements.py
+++ b/src/pip/_internal/resolution/resolvelib/requirements.py
@@ -55,7 +55,7 @@ def __init__(self, ireq, factory):
         assert ireq.link is None, "This is a link, not a specifier"
         self._ireq = ireq
         self._factory = factory
-        self.extras = ireq.req.extras
+        self.extras = set(ireq.extras)
 
     def __str__(self):
         # type: () -> str


Test patch:
diff --git a/tests/functional/test_new_resolver.py b/tests/functional/test_new_resolver.py
--- a/tests/functional/test_new_resolver.py
+++ b/tests/functional/test_new_resolver.py
@@ -189,7 +189,40 @@ def test_new_resolver_ignore_dependencies(script):
     assert_not_installed(script, "dep")
 
 
-def test_new_resolver_installs_extras(script):
+@pytest.mark.parametrize(
+    "root_dep",
+    [
+        "base[add]",
+        "base[add] >= 0.1.0",
+        # Non-standard syntax. To deprecate, see pypa/pip#8288.
+        "base >= 0.1.0[add]",
+    ],
+)
+def test_new_resolver_installs_extras(tmpdir, script, root_dep):
+    req_file = tmpdir.joinpath("requirements.txt")
+    req_file.write_text(root_dep)
+
+    create_basic_wheel_for_package(
+        script,
+        "base",
+        "0.1.0",
+        extras={"add": ["dep"]},
+    )
+    create_basic_wheel_for_package(
+        script,
+        "dep",
+        "0.1.0",
+    )
+    script.pip(
+        "install", "--unstable-feature=resolver",
+        "--no-cache-dir", "--no-index",
+        "--find-links", script.scratch_path,
+        "-r", req_file,
+    )
+    assert_installed(script, base="0.1.0", dep="0.1.0")
+
+
+def test_new_resolver_installs_extras_warn_missing(script):
     create_basic_wheel_for_package(
         script,
         "base",