Repository: pypa/build
Pull request #230 (2021-02-12 02:45:08)

env: fix issue installing wheels on Big Sur
Followup to #229. This fixes an issue I've seen trying to install `cmake` on Big Sur (Intel), and preemptively fixes an issue for Big Sur (Apple Silicon).

Edit: I'm now questioning if I got the version right for pip breaking on Big Sur Intel...
EditEdit: Ahh, I have a custom wheel cached as workaround. 🤦 Yes, 20.2 does break, as promised. 

To see the problem, `pyenv install 3.9.1`, activate it, install build, then build the following `pyproject.toml`:

```toml
[build-system]
requires = ["flit_core >=2,<4", "cmake"]
build-backend = "flit_core.buildapi"

[tool.flit.metadata]
module = "foobar"
author = "Sir Robin"
author-email = "robin@camelot.uk"
home-page = "https://github.com/sirrobin/foobar"
```

You will see `Building wheel for cmake (PEP 517)` because it's using the venv's pip 20.2.

henryiii: I can't test this locally anymore because brew's Python 3.9 now using pip 21.0.1. I can see what pyenv 3.9.1 comes with, I suppose. I know for a long time I was getting 20.2.x and nothing depending on `cmake` worked, because it couldn't get the wheel.
henryiii: I'm getting a compilation from mypy 2.7, call to untyped function from a typed context for https://docs.python.org/2.7/library/platform.html#platform.mac_ver . (Which is very much there in 2.7's docs and at runtime). It's happy on 3.8 mode, though.
layday: It's just not been annotated in the typeshed, see https://github.com/python/typeshed/blob/27f38b8c23da0a2d0ff9ce486f84664c8b9a2d58/stdlib/%40python2/platform.pyi#L23.
henryiii: Works locally:

```bash
pip install git+https://github.com/henryiii/build@env/macOS
python -m build
```


```
Collecting pip
  Using cached pip-21.0.1-py3-none-any.whl (1.5 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.2.3
    Uninstalling pip-20.2.3:
      Successfully uninstalled pip-20.2.3
Successfully installed pip-21.0.1
Found existing installation: setuptools 49.2.1
Uninstalling setuptools-49.2.1:
  Successfully uninstalled setuptools-49.2.1
Collecting flit_core<4,>=2
  Using cached flit_core-3.0.0-py3-none-any.whl (36 kB)
Collecting cmake
  Using cached cmake-3.18.4.post1-py3-none-macosx_10_6_x86_64.whl (42.8 MB)
Collecting pytoml
```

As noted in the description above, you need official CPython 3.9.1 or earlier, the homebrew one is patched to include the latest pip in venv (probably to solve the Apple Silicon issue, I'd guess).
henryiii: Thoughts for a (nice) way to work around it for now? I can't add `type: ignore`, because that causes an error for 3.8 where the annotation is given.
hauntsaninja: You could add a redundant sys.version check, one with type ignore, one without. Or maybe `cast(Any, platform.mac_ver)()` will work.
henryiii: I already tried:

```python
        if sys.version_info < (3,):
            platform_mac_ver = platform.mac_ver()  # type: ignore
        else:
            platform_mac_ver = platform.mac_ver()
```

I thought that was the "not-nice" workaround, but turns out it doesn't even work. In 3.8 mode, it still complains. Before that, I tried a cast, which also failed (though I didn't try Any, I put the whole tuple thing in, but assume the result should be the same).


```bash
mypy --python-version 3.8  src/
src/build/env.py:198: error: unused 'type: ignore' comment
```
hauntsaninja: No, you want to try `cast(Any, platform.mac_ver)()`, not `cast(Any / Tuple, platform.mac_ver())`
henryiii: Yes, just found that out. :) Works now.
henryiii: I didn't see the parenthesis in your example, you had it right. :)
henryiii: By the way, this only implements this for venv, not virtualenv, matching the old behavior. Is there a reason there wasn't done for virtualenv before? If it was because virtualenv almost always had 10+, that might not be good enough for macOS. If it's because a user can upgrade virtualenv, and it's not build's fault if virtualenv is too old, then that's okay to leave off?
gaborbernat: > By the way, this only implements this for venv, not virtualenv, matching the old behavior. Is there a reason there wasn't done for virtualenv before? If it was because virtualenv almost always had 10+, that might not be good enough for macOS. If it's because a user can upgrade virtualenv, and it's not build's fault if virtualenv is too old, then that's okay to leave off?

virtualenv has smart auto-upgrade functionality, https://virtualenv.pypa.io/en/latest/user_guide.html#wheels, so it is very hard for it to pull old pip versions. Note virtualenv is also a lot faster-creating build environments, so if you want some speed-up install `build[virtualenv]` and avoid a lot of these problems.
FFY00: @gaborbernat please no merge commits, rebase instead :blush:

I will fix it.
gaborbernat: Just fixed it.
FFY00: Thanks! While you're at it, could you prefix the `More compact implementation` commit? Should be fairly simple :grin: 
gaborbernat: Done.

Patch:
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
index 662819c..ac4cc57 100644
--- a/.github/CODEOWNERS
+++ b/.github/CODEOWNERS
@@ -1,1 +1,1 @@
-*       @FFY00 @gaborbernat
+*       @FFY00 @gaborbernat @layday
diff --git a/CHANGELOG.rst b/CHANGELOG.rst
index 7ec4a05..a1d3a33 100644
--- a/CHANGELOG.rst
+++ b/CHANGELOG.rst
@@ -9,6 +9,7 @@ Unreleased
 - Upgrade pip based on venv pip version, avoids error from unrecognised pip flag on Debian Python 3.6.5-3.8 (`PR #229`_, Fixes `#228`_)
 - Build dependencies in isolation, instead of in the build environment (`PR #232`_, Fixes `#231`_)
 
+.. _PR #230: https://github.com/pypa/build/pull/230
 .. _PR #229: https://github.com/pypa/build/pull/229
 .. _#228: https://github.com/pypa/build/issues/228
 .. _PR #232: https://github.com/pypa/build/pull/232
diff --git a/README.md b/README.md
index 010045e..f38e41f 100644
--- a/README.md
+++ b/README.md
@@ -4,6 +4,7 @@
 [![codecov](https://codecov.io/gh/pypa/build/branch/master/graph/badge.svg)](https://codecov.io/gh/pypa/build)
 [![Documentation Status](https://readthedocs.org/projects/pypa-build/badge/?version=latest)](https://pypa-build.readthedocs.io/en/latest/?badge=latest)
 [![PyPI version](https://badge.fury.io/py/build.svg)](https://pypi.org/project/build/)
+[![Discord](https://img.shields.io/discord/803025117553754132?label=Discord%20chat%20%23build&style=flat-square)](https://discord.gg/pypa)
 
 A simple, correct PEP517 package builder.
 
diff --git a/src/build/__init__.py b/src/build/__init__.py
index 284df22..f6baa55 100644
--- a/src/build/__init__.py
+++ b/src/build/__init__.py
@@ -143,7 +143,7 @@ class ProjectBuilder(object):
             raise BuildException('Failed to parse {}: {} '.format(spec_file, e))
 
         build_system = spec.get('build-system')
-        # if pyproject.toml is missing (per PEP 517) or [build-system] is missing (pep PEP 518),
+        # if pyproject.toml is missing (per PEP 517) or [build-system] is missing (per PEP 518),
         # use default values.
         if build_system is None:
             _find_typo(spec, 'build-system')
diff --git a/src/build/env.py b/src/build/env.py
index d2f031a..4afcc04 100644
--- a/src/build/env.py
+++ b/src/build/env.py
@@ -167,55 +167,69 @@ def _create_isolated_env_virtualenv(path):  # type: (str) -> Tuple[str, str]
     return executable, script_dir
 
 
-def _create_isolated_env_venv(path):  # type: (str) -> Tuple[str, str]
-    """
-    On Python 3 we use the venv package from the standard library.
-
-    :param path: The path where to create the isolated build environment
-    :return: The Python executable and script folder
-    """
-    import venv
-
-    venv.EnvBuilder(with_pip=True).create(path)
-    executable, script_dir, purelib = _find_executable_and_scripts(path)
-
-    # Get the version of pip in the environment
-    pip_distribution = next(iter(metadata.distributions(name='pip', path=[purelib])))
-    pip_version = packaging.version.Version(pip_distribution.version)
-
-    # Currently upgrade if Pip 19.1+ not available, since Pip 19 is the first
-    # one to officially support PEP 517, and 19.1 supports manylinux1.
-    if pip_version < packaging.version.Version('19.1'):
-        subprocess.check_call([executable, '-m', 'pip', 'install', '-U', 'pip'])
-
-    # Avoid the setuptools from ensurepip to break the isolation
-    subprocess.check_call([executable, '-m', 'pip', 'uninstall', 'setuptools', '-y'])
-    return executable, script_dir
+# venv only exists on Python 3+
+if sys.version_info >= (3,):  # noqa: C901
 
+    def _create_isolated_env_venv(path):  # type: (str) -> Tuple[str, str]
+        """
+        On Python 3 we use the venv package from the standard library.
 
-def _find_executable_and_scripts(path):  # type: (str) -> Tuple[str, str, str]
-    """
-    Detect the Python executable and script folder of a virtual environment.
+        :param path: The path where to create the isolated build environment
+        :return: The Python executable and script folder
+        """
+        import venv
+
+        venv.EnvBuilder(with_pip=True).create(path)
+        executable, script_dir, purelib = _find_executable_and_scripts(path)
+
+        # Get the version of pip in the environment
+        pip_distribution = next(iter(metadata.distributions(name='pip', path=[purelib])))
+        pip_version = packaging.version.Version(pip_distribution.version)
+
+        needs_pip_upgrade = False
+        if pip_version < packaging.version.Version('19.1'):
+            # PEP-517 and manylinux1 was first implemented in 19.1
+            needs_pip_upgrade = True
+        elif platform.system() == 'Darwin':
+            # macOS 11+ name scheme change requires 20.3. Intel macOS 11.0 can be told to report 10.16 for backwards
+            # compatibility; but that also fixes earlier versions of pip so this is only needed for 11+.
+
+            if int(platform.mac_ver()[0].split('.')[0]) >= 11:
+                needs_pip_upgrade = pip_version < packaging.version.Version('20.3') or (
+                    pip_version < packaging.version.Version('21.0.1')  # Apple Silicon macOS requires 21.0.1+
+                    and sys.version_info >= (3, 6)
+                    and platform.machine() != 'x86_64'
+                )
+        if needs_pip_upgrade:
+            subprocess.check_call([executable, '-m', 'pip', 'install', '-U', 'pip'])
+
+        # Avoid the setuptools from ensurepip to break the isolation
+        subprocess.check_call([executable, '-m', 'pip', 'uninstall', 'setuptools', '-y'])
+        return executable, script_dir
+
+    def _find_executable_and_scripts(path):  # type: (str) -> Tuple[str, str, str]
+        """
+        Detect the Python executable and script folder of a virtual environment.
 
-    :param path: The location of the virtual environment
-    :return: The Python executable, script folder, and purelib folder
-    """
-    config_vars = sysconfig.get_config_vars().copy()  # globally cached, copy before altering it
-    config_vars['base'] = path
-    env_scripts = sysconfig.get_path('scripts', vars=config_vars)
-    if not env_scripts:
-        raise RuntimeError("Couldn't get environment scripts path")
-    exe = 'pypy3' if platform.python_implementation() == 'PyPy' else 'python'
-    if os.name == 'nt':
-        exe = '{}.exe'.format(exe)
-    executable = os.path.join(env_scripts, exe)
-    if not os.path.exists(executable):
-        raise RuntimeError('Virtual environment creation failed, executable {} missing'.format(executable))
-
-    purelib = sysconfig.get_path('purelib', vars=config_vars)
-    if not purelib:
-        raise RuntimeError("Couldn't get environment purelib folder")
-    return executable, env_scripts, purelib
+        :param path: The location of the virtual environment
+        :return: The Python executable, script folder, and purelib folder
+        """
+        config_vars = sysconfig.get_config_vars().copy()  # globally cached, copy before altering it
+        config_vars['base'] = path
+        env_scripts = sysconfig.get_path('scripts', vars=config_vars)
+        if not env_scripts:
+            raise RuntimeError("Couldn't get environment scripts path")
+        exe = 'pypy3' if platform.python_implementation() == 'PyPy' else 'python'
+        if os.name == 'nt':
+            exe = '{}.exe'.format(exe)
+        executable = os.path.join(env_scripts, exe)
+        if not os.path.exists(executable):
+            raise RuntimeError('Virtual environment creation failed, executable {} missing'.format(executable))
+
+        purelib = sysconfig.get_path('purelib', vars=config_vars)
+        if not purelib:
+            raise RuntimeError("Couldn't get environment purelib folder")
+        return executable, env_scripts, purelib
 
 
 __all__ = (


Test patch:
diff --git a/tests/test_env.py b/tests/test_env.py
index b9f6742..792b525 100644
--- a/tests/test_env.py
+++ b/tests/test_env.py
@@ -1,4 +1,5 @@
 # SPDX-License-Identifier: MIT
+import collections
 import os
 import platform
 import shutil
@@ -14,6 +15,8 @@ import build.env
 
 
 IS_PYPY3 = sys.version_info[0] == 3 and platform.python_implementation() == 'PyPy'
+IS_PY35 = sys.version_info[:2] == (3, 5)
+IS_PY2 = sys.version_info[0] == 2
 
 
 @pytest.mark.isolated
@@ -34,12 +37,12 @@ def test_isolated_environment_install(mocker):
         subprocess.check_call.assert_not_called()
 
         env.install(['some', 'requirements'])
-        if sys.version_info[:2] != (3, 5):
+        if not IS_PY35:
             subprocess.check_call.assert_called()
         args = subprocess.check_call.call_args[0][0][:-1]
         assert args == [
             env.executable,
-            '-{}m'.format('E' if sys.version_info[0] == 2 else 'I'),
+            '-{}m'.format('E' if IS_PY2 else 'I'),
             'pip',
             'install',
             '--use-pep517',
@@ -48,7 +51,7 @@ def test_isolated_environment_install(mocker):
         ]
 
 
-@pytest.mark.skipif(sys.version_info[0] == 2, reason='venv module used on Python 3 only')
+@pytest.mark.skipif(IS_PY2, reason='venv module used on Python 3 only')
 @pytest.mark.skipif(IS_PYPY3, reason='PyPy3 uses get path to create and provision venv')
 def test_fail_to_get_script_path(mocker):
     mocker.patch.object(build.env, 'virtualenv', None)
@@ -61,7 +64,7 @@ def test_fail_to_get_script_path(mocker):
     assert get_path.call_count == 1
 
 
-@pytest.mark.skipif(sys.version_info[0] == 2, reason='venv module used on Python 3 only')
+@pytest.mark.skipif(IS_PY2, reason='venv module used on Python 3 only')
 @pytest.mark.skipif(IS_PYPY3, reason='PyPy3 uses get path to create and provision venv')
 def test_fail_to_get_purepath(mocker):
     sysconfig_get_path = sysconfig.get_path
@@ -80,7 +83,7 @@ def test_fail_to_get_purepath(mocker):
             pass
 
 
-@pytest.mark.skipif(sys.version_info[0] == 2, reason='venv module used on Python 3 only')
+@pytest.mark.skipif(IS_PY2, reason='venv module used on Python 3 only')
 @pytest.mark.skipif(IS_PYPY3, reason='PyPy3 uses get path to create and provision venv')
 def test_executable_missing_post_creation(mocker):
     mocker.patch.object(build.env, 'virtualenv', None)
@@ -128,3 +131,29 @@ def test_default_pip_is_never_too_old():
             [env.executable, '-c', 'import pip; print(pip.__version__)'], universal_newlines=True
         ).strip()
         assert Version(version) >= Version('19.1')
+
+
+@pytest.mark.isolated
+@pytest.mark.parametrize('pip_version', ['20.2.0', '20.3.0', '21.0.0', '21.0.1'])
+@pytest.mark.parametrize('arch', ['x86_64', 'arm64'])
+@pytest.mark.skipif(IS_PY35, reason="Python 3.5 does not run on macOS 11, and pip can't upgrade to 21 there")
+@pytest.mark.skipif(IS_PY2, reason='venv module used on Python 3 only')
+def test_pip_needs_upgrade_mac_os_11(mocker, pip_version, arch):
+    SimpleNamespace = collections.namedtuple('SimpleNamespace', 'version')
+
+    check_call = mocker.patch('subprocess.check_call')
+    mocker.patch('platform.system', return_value='Darwin')
+    mocker.patch('platform.machine', return_value=arch)
+    mocker.patch('platform.mac_ver', return_value=('11.0', ('', '', ''), ''))
+    mocker.patch('build.env.metadata.distributions', return_value=(SimpleNamespace(version=pip_version),))
+    mocker.patch.object(build.env, 'virtualenv', None)  # hide virtualenv
+
+    min_version = Version('20.3' if arch == 'x86_64' else '21.0.1')
+    with build.env.IsolatedEnvBuilder():
+        if Version(pip_version) < min_version:
+            upgrade_call, uninstall_call = check_call.call_args_list
+            assert upgrade_call[0][0][1:] == ['-m', 'pip', 'install', '-U', 'pip']
+            assert uninstall_call[0][0][1:] == ['-m', 'pip', 'uninstall', 'setuptools', '-y']
+        else:
+            (uninstall_call,) = check_call.call_args_list
+            assert uninstall_call[0][0][1:] == ['-m', 'pip', 'uninstall', 'setuptools', '-y']
diff --git a/tests/test_projectbuilder.py b/tests/test_projectbuilder.py
index 50dc08b..88f930c 100644
--- a/tests/test_projectbuilder.py
+++ b/tests/test_projectbuilder.py
@@ -349,7 +349,7 @@ def test_build_with_dep_on_console_script(tmp_path, demo_pkg_inline, capfd, mock
     """
     All command-line scripts provided by the build-required packages must be present in the build environment's PATH.
     """
-    # we first install install demo pkg inline as build dependency (as this provides a console script we can check)
+    # we first install demo pkg inline as build dependency (as this provides a console script we can check)
     # to validate backend invocations contain the correct path we use an inline backend that will fail, but first
     # provides the PATH information (and validates shutil.which is able to discover the executable - as PEP states)
     toml = '[build-system]\nrequires = ["demo_pkg_inline"]\nbuild-backend = "build"\nbackend-path = ["."]\n'