Repository: pypa/pip

Issue:
Passing --no-cache-dir should not affect whether to use a wheel
At the moment the help output for `--no-cache-dir` just says: `Disable the cache.`

Till a week ago, I was under the impression that it meant that pip wouldn't use a cached wheel, and wouldn't cache a wheel fetched from PyPI.

What I didn't know was that when this option is enabled and a package has to be installed from an sdist, pip doesn't actually create a wheel and ends up creating egg-info directories, instead of dist-info ones.

Quick example:

```
$ pip install https://files.pythonhosted.org/packages/ca/a9/62f96decb1e309d6300ebe7eee9acfd7bccaeedd693794437005b9067b44/pytz-2018.5.tar.gz
Collecting https://files.pythonhosted.org/packages/ca/a9/62f96decb1e309d6300ebe7eee9acfd7bccaeedd693794437005b9067b44/pytz-2018.5.tar.gz
  Downloading https://files.pythonhosted.org/packages/ca/a9/62f96decb1e309d6300ebe7eee9acfd7bccaeedd693794437005b9067b44/pytz-2018.5.tar.gz (318kB)
    100% |████████████████████████████████| 327kB 17.3MB/s 
Building wheels for collected packages: pytz <------------------------- Builds a 'wheel'
  Running setup.py bdist_wheel for pytz ... done
  Stored in directory: /home/nwani/.cache/pip/wheels/0d/29/29/5c4384ddcad4ea97cb91f7ff4ef13ab3cff23ddaf35ce02d21
Successfully built pytz
Installing collected packages: pytz
Successfully installed pytz-2018.5
$ find $PREFIX -type d | grep pytz-2018.5
/tmp/wani.1535901395/dev/lib/python2.7/site-packages/pytz-2018.5.dist-info  <------ 'dist-info'
```

```
$ pip install https://files.pythonhosted.org/packages/ca/a9/62f96decb1e309d6300ebe7eee9acfd7bccaeedd693794437005b9067b44/pytz-2018.5.tar.gz --no-cache-dir
Collecting https://files.pythonhosted.org/packages/ca/a9/62f96decb1e309d6300ebe7eee9acfd7bccaeedd693794437005b9067b44/pytz-2018.5.tar.gz
  Downloading https://files.pythonhosted.org/packages/ca/a9/62f96decb1e309d6300ebe7eee9acfd7bccaeedd693794437005b9067b44/pytz-2018.5.tar.gz (318kB)
    100% |████████████████████████████████| 327kB 52.1MB/s 
Installing collected packages: pytz
  Running setup.py install for pytz ... done  <------------------------ No 'wheel' ?
Successfully installed pytz-2018.5
$ find $PREFIX -type d | grep pytz-2018.5
/tmp/wani.1535901395/dev/lib/python2.7/site-packages/pytz-2018.5-py2.7.egg-info <----- 'egg-info'
```

In the case of '--no-cache-dir', would it not be possible for pip to still build the wheel but throw it away after installation?



Discussion:
In my case, this breaks installation of a library via `pip install`, because the wheel step includes building of a necessary shared library (which otherwise works fine using `setup.py bdist_wheel` and then pointing pip at the result).
> In the case of '--no-cache-dir', would it not be possible for pip to still build the wheel but throw it away after installation?

I thought it was already supposed to do this? Can you retry with the latest version of pip (19.0.3)? If it's not building a wheel, it's possible there's a separate reason for that (pip has separate logic to decide whether to build a wheel).


I was using 19.0.3 (also tried other versions). See https://github.com/pypa/pip/blob/88bcb261f7c11da7f9f7113124079e9e30b2db72/src/pip/_internal/commands/install.py#L330-L336
19.0.3 also doesn't create the wheel. 
See also this code which does what you suggest, but under a different scenario:
https://github.com/pypa/pip/blob/88bcb261f7c11da7f9f7113124079e9e30b2db72/src/pip/_internal/wheel.py#L783-L786
AFAICT that is only applicable to the "pep 517" path. In my case -- and I suspect the repro in the original issue -- `pep517_requirements` is empty, and the needed package is in `legacy_requirements`.
Yes. Let’s keep this issue open to discuss whether the behavior should be changed.
I see your confused emoji. What I'm saying is that instead of documenting the current behavior which is what you did in the PR #6284 that you submitted, let's instead discuss whether pip can be changed in the way that the person opening the issue first suggested:
> In the case of '--no-cache-dir', would it not be possible for pip to still build the wheel but throw it away after installation?

Also, to clarify the above, the code I referenced is applicable in both the pep 517 and non-pep 517 cases in some scenarios (e.g. without `--no-cache-dir`). You can see that the return value of `should_use_ephemeral_cache ()` is used in `WheelBuilder.build()`, prior to determining whether `_build_one_pep517()` or `_build_one_legacy()` should be used. This is also consistent with a wheel being built for the package when `--no-cache-dir` _wasn't_ provided.

Changing the behavior would be great. The current behavior seems contrary to the [pip install documentation](https://pip.pypa.io/en/stable/reference/pip_install/#build-system-interface) which states that pip runs `bdist_wheel` when installing from source. That guarantee is important to me, and probably others, because `bdist_wheel` is familiar and at least somewhat debuggable (whereas I don't really know how to debug pip build failures because the intermediates are removed even with `--no-clean`).
For historical purposes, I found where this logic was first added: https://github.com/pypa/pip/issues/2780

I think the “ephemeral cache” was added a lot more recently, so they didn’t have as many options to address their issue back then.
I was hoping that specifying a specific (empty) `--cache-dir` would help. That does force a rebuild, but appears to skip `bdist_wheel` in the same way. Whereas deleting the global cache directory and re-installing works.
FYI, I just filed PR #6303 to address this.

Also, I just marked this issue as an enhancement. It's possible one could argue it's a bug, but since it's debatable IMO I chose the more forgiving option.

Resolving patch:
diff --git a/src/pip/_internal/commands/install.py b/src/pip/_internal/commands/install.py
--- a/src/pip/_internal/commands/install.py
+++ b/src/pip/_internal/commands/install.py
@@ -33,6 +33,43 @@
 logger = logging.getLogger(__name__)
 
 
+def is_wheel_installed():
+    """
+    Return whether the wheel package is installed.
+    """
+    try:
+        import wheel  # noqa: F401
+    except ImportError:
+        return False
+
+    return True
+
+
+def build_wheels(builder, pep517_requirements, legacy_requirements, session):
+    """
+    Build wheels for requirements, depending on whether wheel is installed.
+    """
+    # We don't build wheels for legacy requirements if wheel is not installed.
+    should_build_legacy = is_wheel_installed()
+
+    # Always build PEP 517 requirements
+    build_failures = builder.build(
+        pep517_requirements,
+        session=session, autobuilding=True
+    )
+
+    if should_build_legacy:
+        # We don't care about failures building legacy
+        # requirements, as we'll fall through to a direct
+        # install for those.
+        builder.build(
+            legacy_requirements,
+            session=session, autobuilding=True
+        )
+
+    return build_failures
+
+
 class InstallCommand(RequirementCommand):
     """
     Install packages from:
@@ -327,34 +364,18 @@ def run(self, options, args):
                         else:
                             legacy_requirements.append(req)
 
-                    # We don't build wheels for legacy requirements if we
-                    # don't have wheel installed or we don't have a cache dir
-                    try:
-                        import wheel  # noqa: F401
-                        build_legacy = bool(options.cache_dir)
-                    except ImportError:
-                        build_legacy = False
-
-                    wb = WheelBuilder(
+                    wheel_builder = WheelBuilder(
                         finder, preparer, wheel_cache,
                         build_options=[], global_options=[],
                     )
 
-                    # Always build PEP 517 requirements
-                    build_failures = wb.build(
-                        pep517_requirements,
-                        session=session, autobuilding=True
+                    build_failures = build_wheels(
+                        builder=wheel_builder,
+                        pep517_requirements=pep517_requirements,
+                        legacy_requirements=legacy_requirements,
+                        session=session,
                     )
 
-                    if build_legacy:
-                        # We don't care about failures building legacy
-                        # requirements, as we'll fall through to a direct
-                        # install for those.
-                        wb.build(
-                            legacy_requirements,
-                            session=session, autobuilding=True
-                        )
-
                     # If we're using PEP 517, we cannot do a direct install
                     # so we fail here.
                     if build_failures:


Test patch:
diff --git a/tests/unit/test_command_install.py b/tests/unit/test_command_install.py
new file mode 100644
--- /dev/null
+++ b/tests/unit/test_command_install.py
@@ -0,0 +1,68 @@
+from mock import Mock, call, patch
+
+from pip._internal.commands.install import build_wheels
+
+
+class TestWheelCache:
+
+    def check_build_wheels(
+        self,
+        pep517_requirements,
+        legacy_requirements,
+        session,
+    ):
+        """
+        Return: (mock_calls, return_value).
+        """
+        def build(reqs, **kwargs):
+            # Fail the first requirement.
+            return [reqs[0]]
+
+        builder = Mock()
+        builder.build.side_effect = build
+
+        build_failures = build_wheels(
+            builder=builder,
+            pep517_requirements=pep517_requirements,
+            legacy_requirements=legacy_requirements,
+            # A session value isn't needed.
+            session='<session>',
+        )
+
+        return (builder.build.mock_calls, build_failures)
+
+    @patch('pip._internal.commands.install.is_wheel_installed')
+    def test_build_wheels__wheel_installed(self, is_wheel_installed):
+        is_wheel_installed.return_value = True
+
+        mock_calls, build_failures = self.check_build_wheels(
+            pep517_requirements=['a', 'b'],
+            legacy_requirements=['c', 'd'],
+            session='<session>',
+        )
+
+        # Legacy requirements were built.
+        assert mock_calls == [
+            call(['a', 'b'], autobuilding=True, session='<session>'),
+            call(['c', 'd'], autobuilding=True, session='<session>'),
+        ]
+
+        # Legacy build failures are not included in the return value.
+        assert build_failures == ['a']
+
+    @patch('pip._internal.commands.install.is_wheel_installed')
+    def test_build_wheels__wheel_not_installed(self, is_wheel_installed):
+        is_wheel_installed.return_value = False
+
+        mock_calls, build_failures = self.check_build_wheels(
+            pep517_requirements=['a', 'b'],
+            legacy_requirements=['c', 'd'],
+            session='<session>',
+        )
+
+        # Legacy requirements were not built.
+        assert mock_calls == [
+            call(['a', 'b'], autobuilding=True, session='<session>'),
+        ]
+
+        assert build_failures == ['a']