Repository: pypa/cibuildwheel

Issue:
Fix cp38-macosx universal2 and arm64 Python pkg URL
Previous URLs pointed to x86_64 version


Discussion:
Unfortunately we don't want to upgrade the bundled Python because it would increase the minimum MacOS version for the x86_64 arch from 10.9 to 11.0, which is a pretty massive jump. 
Sorry, could you please explain how arm64 Python URL affects x86_64 minimum macOS version?
Ah, I understand what's going on. We still have no arm64 CI, so we've never actually had a chance to test running all this on arm64. The version of 3.8 that we install only runs on x86_64. So when you run it on arm64, it's emulating x86_64. The wheel that you built is fine, but you're not able to test it.

This PR seems to only change the universal2/arm64 build identifiers. I don't think that will change anything, as on macOS you can only have one minor version of Python installed at once, and the first one that cibuildwheel finds, it will install. So if we were going to make this change we'd have to change all three 3.8 URLs.

The Python 3.8 arm64 support was added in https://github.com/pypa/cibuildwheel/pull/704. I don't have time right now but perhaps there are some clues in there why we're not using the `macos11` versions.
We tried very hard to use this when it came out, but it doesn’t work. CPython only has partial support for 3.8 on ARM - it was squeezed in at the last minute in the last binary release. IIRC you can’t produce 10.9-10.15 compatible wheels for Intel from the ARM version since Python didn’t include a 10.9 compatible Intel version in the binary, like they normally do (and do in 3.9+). You can look at the PR where 3.8 was added, I think.

I think to make this “work” we’d need to special case when running on ARM and have the (big) caveat that you can’t build for macOS 10 from AS (but only for 3.8). Is 3.8 still the version Apple ships? I’m traveling and won’t be my AS machine till the 27th. 
Read here: 

* https://github.com/pypa/cibuildwheel/pull/663#discussion_r628731708
* https://github.com/scipy/oldest-supported-numpy/pull/20#discussion_r641740734
* https://github.com/pypa/cibuildwheel/pull/704

I think we should see if we can build from AS too, but we haven’t had CI runners for it and it will take some changes and should error out of macOS version is set <11.

I’m pretty sure we use the AS version from this table when building pure AS wheels from Intel.

> IIRC you can’t produce 10.9-10.15 compatible wheels for Intel from the ARM version since Python didn’t include a 10.9 compatible Intel version in the binary, like they normally do (and do in 3.9+).

Now that `MACOSX_DEPLOYMENT_TARGET` is always set, I don't think there's a big issue on what you can target (i.e. will still be 10.9 for x86_64) unless I'm missing something. However, using a macOS 10.x runner to do the build won't be possible anymore with this installer which requires macOS 11+.
From what I understand, you can't set `MACOSX_DEPLOYMENT_TARGET` lower than whatever CPython was prepared with, otherwise we'd not need to download the official binaries at all. I could be wrong, but I'm pretty sure `MACOSX_DEPLOYMENT_TARGET` can only be increased.
yes. here is a `wheel` code fragment with a warning about this:

https://github.com/pypa/wheel/blob/9d4264d2f23ea44dd85639a27fa6b284261921cd/src/wheel/macosx_libfile.py#L411-L419
Thanks all for the informative discussion. I guess the upshot is that we keep the macos10.9 installer, and we can't test 3.8 arm64 wheels, even when running on arm64. 

However, we should change the behaviour when running on Apple Silicon, to not try to test the arm64 3.8 wheel, and include an informative message explaining why.

Resolving patch:
diff --git a/cibuildwheel/macos.py b/cibuildwheel/macos.py
--- a/cibuildwheel/macos.py
+++ b/cibuildwheel/macos.py
@@ -465,6 +465,23 @@ def build(options: Options, tmp_path: Path) -> None:
                         # skip this test
                         continue
 
+                    if testing_arch == "arm64" and config.identifier.startswith("cp38-"):
+                        log.warning(
+                            unwrap(
+                                """
+                                While cibuildwheel can build CPython 3.8 universal2/arm64 wheels, we
+                                cannot test the arm64 part of them, even when running on an Apple
+                                Silicon machine. This is because we use the x86_64 installer of
+                                CPython 3.8. See the discussion in
+                                https://github.com/pypa/cibuildwheel/pull/1169 for the details. To
+                                silence this warning, set `CIBW_TEST_SKIP: cp38-macosx_*:arm64`.
+                                """
+                            )
+                        )
+
+                        # skip this test
+                        continue
+
                     log.step(
                         "Testing wheel..."
                         if testing_arch == machine_arch


Test patch:
diff --git a/test/test_macos_archs.py b/test/test_macos_archs.py
--- a/test/test_macos_archs.py
+++ b/test/test_macos_archs.py
@@ -133,3 +133,36 @@ def test_universal2_testing(tmp_path, capfd, skip_arm64_test):
     expected_wheels = [w for w in ALL_MACOS_WHEELS if "cp39" in w and "universal2" in w]
 
     assert set(actual_wheels) == set(expected_wheels)
+
+
+def test_cp38_arm64_testing(tmp_path, capfd):
+    if utils.platform != "macos":
+        pytest.skip("this test is only relevant to macos")
+    if get_xcode_version() < (12, 2):
+        pytest.skip("this test only works with Xcode 12.2 or greater")
+    if platform.machine() != "arm64":
+        pytest.skip("this test only works on arm64")
+
+    project_dir = tmp_path / "project"
+    basic_project.generate(project_dir)
+
+    actual_wheels = utils.cibuildwheel_run(
+        project_dir,
+        add_env={
+            "CIBW_BUILD": "cp38-*",
+            "CIBW_TEST_COMMAND": '''python -c "import platform; print('running tests on ' + platform.machine())"''',
+            "CIBW_ARCHS": "x86_64,universal2,arm64",
+        },
+    )
+
+    captured = capfd.readouterr()
+
+    assert "running tests on x86_64" in captured.out
+    assert "running tests on arm64" not in captured.out
+
+    warning_message = "While cibuildwheel can build CPython 3.8 universal2/arm64 wheels, we cannot test the arm64 part of them"
+    assert warning_message in captured.err
+
+    expected_wheels = [w for w in ALL_MACOS_WHEELS if "cp38" in w]
+
+    assert set(actual_wheels) == set(expected_wheels)
diff --git a/test/utils.py b/test/utils.py
--- a/test/utils.py
+++ b/test/utils.py
@@ -154,8 +154,8 @@ def expected_wheels(
             python_abi_tags += ["pp37-pypy37_pp73", "pp38-pypy38_pp73", "pp39-pypy39_pp73"]
 
         if platform == "macos" and machine_arch == "arm64":
-            # currently, arm64 macs are only supported by cp39, cp310 & cp311
-            python_abi_tags = ["cp39-cp39", "cp310-cp310", "cp311-cp311"]
+            # arm64 macs are only supported by cp38+
+            python_abi_tags = ["cp38-cp38", "cp39-cp39", "cp310-cp310", "cp311-cp311"]
 
     wheels = []
 
@@ -193,7 +193,7 @@ def expected_wheels(
                 platform_tags = ["win32", "win_amd64"]
 
         elif platform == "macos":
-            if python_abi_tag == "cp39-cp39" and machine_arch == "arm64":
+            if machine_arch == "arm64":
                 arm64_macosx_deployment_target = _get_arm64_macosx_deployment_target(
                     macosx_deployment_target
                 )