Repository: pypa/pip

Issue:
Specifying --use-pep517 to a non-pyproject.toml project should populate build requirements
<!--
If you're reporting an issue for `--use-feature=2020-resolver`, use the "Dependency resolver failures / errors" template instead.
-->

**Environment**

* pip version: 21
* Python version: 3.8
* OS: macOS 11

<!-- Feel free to add more information about your environment here -->

**Description**

Attempting to download a project that depends on `termcolor~=1.1.0` fails to build the metadata for termcolor if setuptools isn't implicitly installed, even with pep517 enabled.

In order to wean myself from implicit setuptools, I've left setuptools uninstalled from my environment, and instead rely on passing `--use-pep517` to invoke the "fallback to setuptools" behavior for packages not supplying pyproject.toml. Downloading just `termcolor` by itself works without error, but attempting to download a project that depends on termcolor (such as tensorflow) will fail to build the metadata for termcolor. It seems that the pep517 fallback to setuptools behavior doesn't take effect in the download command.

**Expected behavior**

Passing `--use-pep517` to pip should enable the fallback to setuptools behavior (and make it available for a project if the build dependencies aren't specified in pyproject.toml).

**How to Reproduce**

```
$ python3.8 -m pip download --use-pep517 tensorflow
Collecting tensorflow
  Using cached tensorflow-2.4.1-cp38-cp38-macosx_10_11_x86_64.whl (173.9 MB)
Collecting termcolor~=1.1.0
  Using cached termcolor-1.1.0.tar.gz (3.9 kB)
    ERROR: Command errored out with exit status 1:
     command: /Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-78dnfhfc/termcolor_3986fcda2fa34e69b903950f54c7bcc5/setup.py'"'"'; __file__='"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-78dnfhfc/termcolor_3986fcda2fa34e69b903950f54c7bcc5/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-pip-egg-info-38q33ums
         cwd: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-78dnfhfc/termcolor_3986fcda2fa34e69b903950f54c7bcc5/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'
Collecting tensorflow
...
<proceeds to download every version of tensorflow>
```



Discussion:
I can replicate the issue in isolation using this local package:

```
draft $ tree pkg
pkg
├── pyproject.toml
└── setup.cfg

0 directories, 2 files
draft $ cat pkg/*
[build-system]
requires=['setuptools', 'wheel']
[metadata]
name=pkg
version=1.0

[options]
install_requires=termcolor~=1.1
draft $ python3.8 -m pip download --use-pep517 ./pkg
Processing ./pkg
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting termcolor~=1.1
  Using cached termcolor-1.1.0.tar.gz (3.9 kB)
    ERROR: Command errored out with exit status 1:
     command: /Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-29mgi_lq/termcolor_7d20508cc80849e78090a2c413349ce7/setup.py'"'"'; __file__='"'"'/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-29mgi_lq/termcolor_7d20508cc80849e78090a2c413349ce7/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-pip-egg-info-oeb1bqb9
         cwd: /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-download-29mgi_lq/termcolor_7d20508cc80849e78090a2c413349ce7/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz#sha256=1d6d69ce66211143803fbc56652b41d73b4a400a2891d7bf7a1cdf4c02de613b (from https://pypi.org/simple/termcolor/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement termcolor~=1.1 (from foo)
ERROR: No matching distribution found for termcolor~=1.1
```
Yes, and `wheel` too, or it will fails when trying to build the wheel...
I edited the title to reflect my understanding to the root cause.
FWIW, this is where the `--use-pep517` flag gets lost: <https://github.com/pypa/pip/blob/21.2.4/src/pip/_internal/resolution/resolvelib/factory.py#L493>.

I tried adding `use_pep517=comes_from.use_pep517` to the call and that fixes the issue, although I'm not well enough versed in the codebase to know whether that's the right solution.
@SpecLad I think that’s on the right track. Feel free to submit a PR and a test case and we can go from there.

Resolving patch:
diff --git a/src/pip/_internal/commands/download.py b/src/pip/_internal/commands/download.py
--- a/src/pip/_internal/commands/download.py
+++ b/src/pip/_internal/commands/download.py
@@ -122,6 +122,7 @@ def run(self, options: Values, args: List[str]) -> int:
             finder=finder,
             options=options,
             ignore_requires_python=options.ignore_requires_python,
+            use_pep517=options.use_pep517,
             py_version_info=options.python_version,
         )
 


Test patch:
diff --git a/tests/functional/test_download.py b/tests/functional/test_download.py
--- a/tests/functional/test_download.py
+++ b/tests/functional/test_download.py
@@ -9,7 +9,12 @@
 
 from pip._internal.cli.status_codes import ERROR
 from tests.conftest import MockServer, ScriptFactory
-from tests.lib import PipTestEnvironment, TestData, create_really_basic_wheel
+from tests.lib import (
+    PipTestEnvironment,
+    TestData,
+    create_basic_sdist_for_package,
+    create_really_basic_wheel,
+)
 from tests.lib.server import file_response
 
 
@@ -1166,3 +1171,46 @@ def test_download_editable(
     downloads = os.listdir(download_dir)
     assert len(downloads) == 1
     assert downloads[0].endswith(".zip")
+
+
+def test_download_use_pep517_propagation(
+    script: PipTestEnvironment, tmpdir: Path, common_wheels: Path
+) -> None:
+    """
+    Check that --use-pep517 applies not just to the requirements specified
+    on the command line, but to their dependencies too.
+    """
+
+    create_basic_sdist_for_package(script, "fake_proj", "1.0", depends=["fake_dep"])
+
+    # If --use-pep517 is in effect, then setup.py should be running in an isolated
+    # environment that doesn't have pip in it.
+    create_basic_sdist_for_package(
+        script,
+        "fake_dep",
+        "1.0",
+        setup_py_prelude=textwrap.dedent(
+            """\
+            try:
+                import pip
+            except ImportError:
+                pass
+            else:
+                raise Exception(f"not running in isolation")
+            """
+        ),
+    )
+
+    download_dir = tmpdir / "download_dir"
+    script.pip(
+        "download",
+        f"--dest={download_dir}",
+        "--no-index",
+        f"--find-links={common_wheels}",
+        f"--find-links={script.scratch_path}",
+        "--use-pep517",
+        "fake_proj",
+    )
+
+    downloads = os.listdir(download_dir)
+    assert len(downloads) == 2
diff --git a/tests/lib/__init__.py b/tests/lib/__init__.py
--- a/tests/lib/__init__.py
+++ b/tests/lib/__init__.py
@@ -1211,12 +1211,17 @@ def create_basic_sdist_for_package(
     *,
     fails_egg_info: bool = False,
     fails_bdist_wheel: bool = False,
+    depends: Optional[List[str]] = None,
+    setup_py_prelude: str = "",
 ) -> pathlib.Path:
     files = {
-        "setup.py": f"""\
+        "setup.py": textwrap.dedent(
+            """\
             import sys
             from setuptools import find_packages, setup
 
+            {setup_py_prelude}
+
             fails_bdist_wheel = {fails_bdist_wheel!r}
             fails_egg_info = {fails_egg_info!r}
 
@@ -1226,19 +1231,22 @@ def create_basic_sdist_for_package(
             if fails_bdist_wheel and "bdist_wheel" in sys.argv:
                 raise Exception("Simulated failure for building a wheel.")
 
-            setup(name={name!r}, version={version!r})
-        """,
+            setup(name={name!r}, version={version!r},
+                install_requires={depends!r})
+        """
+        ).format(
+            name=name,
+            version=version,
+            depends=depends or [],
+            setup_py_prelude=setup_py_prelude,
+            fails_bdist_wheel=fails_bdist_wheel,
+            fails_egg_info=fails_egg_info,
+        ),
     }
 
     # Some useful shorthands
     archive_name = f"{name}-{version}.tar.gz"
 
-    # Replace key-values with formatted values
-    for key, value in list(files.items()):
-        del files[key]
-        key = key.format(name=name)
-        files[key] = textwrap.dedent(value)
-
     # Add new files after formatting
     if extra_files:
         files.update(extra_files)