Repository: pypa/pip

Issue:
pip wheel does not cache wheels it built locally
**Environment**

* pip version: 19.2
* Python version: 3.7 (probably irrelevant)
* OS: Linux (probably irrelevant, but necesary for the reproducer below)

**Description**

pip wheel does not cache wheels it built locally, whereas pip install does.

My use case for `pip wheel` is mostly `pip wheel -r frozen-requirements.txt` to build
and collect everything needed for testing on CI and deployment. For wheels that have a long build time it would help to have them cached too in such a scenario.

**Expected behavior**

pip wheel should cache all wheels it built locally (i.e. built from sdists on pypi),
like pip install does.

**How to Reproduce**

Here is an example with pyyaml 5.1.2 wich, as of today, has no linux wheel on PyPI.

First make sure the pip cache is empty.

Use pip wheel, notice it uses the (possibly cached) sdist, and builds the wheel.
Repeat and notice the behaviour remains the same.

```console
$ venv/bin/pip wheel pyyaml==5.1.2
Collecting pyyaml==5.1.2
  Using cached https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz
Building wheels for collected packages: pyyaml
  Building wheel for pyyaml (setup.py) ... done
  Created wheel for pyyaml: filename=PyYAML-5.1.2-cp37-cp37m-linux_x86_64.whl size=44104 sha256=0c0a590c4188291ddce3d984cab5d4868115181183a3972fdf95560cc9530f93
  Stored in directory: /tmp/brol1
Successfully built pyyaml
```

Next, pip install the same, and notice pip builds and caches the wheel.

```console
$ venv/bin/pip install pyyaml==5.1.2
...
  Building wheel for pyyaml (setup.py) ... done
  Created wheel for pyyaml: filename=PyYAML-5.1.2-cp37-cp37m-linux_x86_64.whl size=44104 sha256=a8625826f5bdcb1434d9d8339542db1d47d3f84bf27c2180c39345f34d98821c
  Stored in directory: ~/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030
...
```

Finally pip wheel again and notice it uses the cached wheel.

```console
$ rm PyYAML-5.1.2*
$ venv/bin/pip wheel pyyaml==5.1.2
Collecting pyyaml==5.1.2
  Saved ./PyYAML-5.1.2-cp37-cp37m-linux_x86_64.whl
Skipping pyyaml, due to already being wheel.
```

**Analysis**

I noticed this issue while working on #6640.

I think the problem arises in this [area](https://github.com/pypa/pip/blob/63639bfc0f6daa5cf18580db71d44e37d2a3b682/src/pip/_internal/wheel.py#L1083-L1097).
The output directory provided to `_build_one` is either the cache (when `autobuilding`), or the target directory provided by the user in case of `pip wheel`, in which case the cache is bypassed.

A solution could be to always build to the cache directory, and in case we are not autobuilding copy `wheel_file` to `self._wheel_dir` after building.



Discussion:

Resolving patch:
diff --git a/src/pip/_internal/wheel.py b/src/pip/_internal/wheel.py
--- a/src/pip/_internal/wheel.py
+++ b/src/pip/_internal/wheel.py
@@ -1102,7 +1102,7 @@ def build(
         :param should_unpack: If True, after building the wheel, unpack it
             and replace the sdist with the unpacked version in preparation
             for installation.
-        :return: True if all the wheels built correctly.
+        :return: The list of InstallRequirement that failed to build.
         """
         # pip install uses should_unpack=True.
         # pip install never provides a _wheel_dir.
@@ -1124,19 +1124,15 @@ def build(
             ):
                 continue
 
-            # Determine where the wheel should go.
-            if should_unpack:
-                if (
-                    cache_available and
-                    should_cache(req, self.check_binary_allowed)
-                ):
-                    output_dir = self.wheel_cache.get_path_for_link(req.link)
-                else:
-                    output_dir = self.wheel_cache.get_ephem_path_for_link(
-                        req.link
-                    )
+            if (
+                cache_available and
+                should_cache(req, self.check_binary_allowed)
+            ):
+                output_dir = self.wheel_cache.get_path_for_link(req.link)
             else:
-                output_dir = self._wheel_dir
+                output_dir = self.wheel_cache.get_ephem_path_for_link(
+                    req.link
+                )
 
             buildset.append((req, output_dir))
 
@@ -1174,10 +1170,6 @@ def build(
                     python_tag=python_tag,
                 )
                 if wheel_file:
-                    build_success.append(req)
-                    self.wheel_filenames.append(
-                        os.path.relpath(wheel_file, output_dir)
-                    )
                     if should_unpack:
                         # XXX: This is mildly duplicative with prepare_files,
                         # but not close enough to pull out to a single common
@@ -1202,6 +1194,25 @@ def build(
                         assert req.link.is_wheel
                         # extract the wheel into the dir
                         unpack_file(req.link.file_path, req.source_dir)
+                    else:
+                        # copy from cache to target directory
+                        try:
+                            ensure_dir(self._wheel_dir)
+                            shutil.copy(
+                                os.path.join(output_dir, wheel_file),
+                                self._wheel_dir,
+                            )
+                        except OSError as e:
+                            logger.warning(
+                                "Building wheel for %s failed: %s",
+                                req.name, e,
+                            )
+                            build_failure.append(req)
+                            continue
+                    self.wheel_filenames.append(
+                        os.path.relpath(wheel_file, output_dir)
+                    )
+                    build_success.append(req)
                 else:
                     build_failure.append(req)
 


Test patch:
diff --git a/tests/functional/test_wheel.py b/tests/functional/test_wheel.py
--- a/tests/functional/test_wheel.py
+++ b/tests/functional/test_wheel.py
@@ -62,6 +62,30 @@ def test_pip_wheel_success(script, data):
     assert "Successfully built simple" in result.stdout, result.stdout
 
 
+def test_pip_wheel_build_cache(script, data):
+    """
+    Test 'pip wheel' builds and caches.
+    """
+    result = script.pip(
+        'wheel', '--no-index', '-f', data.find_links,
+        'simple==3.0',
+    )
+    wheel_file_name = 'simple-3.0-py%s-none-any.whl' % pyversion[0]
+    wheel_file_path = script.scratch / wheel_file_name
+    assert wheel_file_path in result.files_created, result.stdout
+    assert "Successfully built simple" in result.stdout, result.stdout
+    # remove target file
+    (script.scratch_path / wheel_file_name).unlink()
+    # pip wheel again and test that no build occurs since
+    # we get the wheel from cache
+    result = script.pip(
+        'wheel', '--no-index', '-f', data.find_links,
+        'simple==3.0',
+    )
+    assert wheel_file_path in result.files_created, result.stdout
+    assert "Successfully built simple" not in result.stdout, result.stdout
+
+
 def test_basic_pip_wheel_downloads_wheels(script, data):
     """
     Test 'pip wheel' downloads wheels