Repository: pypa/cibuildwheel

Issue:
PyPy has released bug fixes and a python3.9
### Description

PyPy released version v7.3.8. It would be nice to be able to use it in cibuildwheel, including the newly released python3.9

### Build log

_No response_

### CI config

_No response_

Discussion:
mattip: Thanks!
henryiii: This looks like it is trying to test pypy on manylinux 2010, is that still supported? And I see:

> cibuildwheel: python available on PATH doesn't match our installed instance. If you have modified PATH, ensure that you don't overwrite cibuildwheel's entry or insert python above it.
mayeut: > This looks like it is trying to test pypy on manylinux 2010, is that still supported?

It's not supported anymore, the old versions are still installed on manylinux2010 but there's no PyPy 3.9.
Updated the test to skip PyPy 3.9 on manylinux2010. 
mattip: I am sorry to report that PyPy3.7-v7.3.8 broke ABI compatibility with previous versions. Please pin PyPy3.7 to v7.3.7. I will be releasing a bug fix 😞. For more details [here](https://foss.heptapod.net/pypy/pypy/-/issues/3688) is the issue about broken ABI compatiblitiy, and [here](https://doc.pypy.org/en/latest/release-v7.3.8.html#python-3-7-bugfixes) are the PyPy3.7-v7.3.8 changes. 
mattip: The changes to update to pypy3.7-v7.3.8 were made in #1027. Should I make a separate PR to back those out or can it be done as part of this PR?
henryiii: Reverting in #1035. By the way, I'm just starting trying to make build's test suite pass on Windows, and I noticed that the bug in `Path.resolve` was fixed in CPython 3.8, but not PyPy 3.8: https://github.com/ofek/hatch/pull/136/files#diff-442ff120586375226c1759bc7fcfc162e7ca986db5a58f47ae45b45fb492c6f8. Also, `winget` is now installed by default on Windows (AFAICT), but there's no official PyPy package there (just a really weird winpython-pypy (3.7) / winpython-dot-pypy (3.8) that's 500 MB and not clearly versioned). Given that's Windows official (vs. third party tools like chocolaty) CLI app installer, having PyPy there would be great!
mattip: > the bug in Path.resolve was fixed in CPython 3.8, but not PyPy 3.8

I am not sure I follow. Which bug and how was it resolved? Is this part of the (lack of) good support for symlinks on [PyPy windows](https://foss.heptapod.net/pypy/pypy/-/issues?scope=all&state=opened&search=symlink)? 
mattip: About winget, I commented on an [open issue there](https://github.com/microsoft/winget-pkgs/issues/23460). We have done 90% of the work to port the CPython installer to PyPy, which means there is only 80% left :smile:. The process is layered on top of PyPy, so does not involve much actual PyPy-specific knowledge, rather the work needs someone who can deal with the minutia of windows to get it over the finish line.
henryiii: I think it was this one: https://bugs.python.org/issue38671 and it was backported to 3.8 and 3.9 from 3.10, so it might based on the stdlib patch version.
mattip: if it was part of the stdlib, it should be part of PyPy. Can we move the discussion of `path.resolve()` somewhere where we can discuss this more deeply, preferably with a test that CPython3.8 passes but PyPy3.8 fails? I don't understand what is failing and how CPython fixed it.
henryiii: I think it's this test: https://github.com/python/cpython/pull/26184.
henryiii: (Back on topic) Unless manylinux releases a mixed pypy version image, I think we can't blend manylinux images based on PyPy versions. (A user can via overrides, actually, but not our defaults.)
mayeut: The plan in https://github.com/pypa/manylinux/pull/1289 is to revert PyPy 3.7 to 7.3.7 and go on with 7.3.8 for PyPy 3.8 & 3.9 

Patch:
diff --git a/README.md b/README.md
index 41cb6203..702b9baa 100644
--- a/README.md
+++ b/README.md
@@ -31,6 +31,7 @@
 | CPython 3.10  | ✅ | ✅  | ✅  | ✅  | ✅² | ✅ | ✅  | ✅ | ✅  | ✅  |
 | PyPy 3.7 v7.3 | ✅ | N/A | ✅  | N/A | N/A | ✅¹ | ✅¹  | ✅¹ | N/A | N/A |
 | PyPy 3.8 v7.3 | ✅ | N/A | ✅  | N/A | N/A | ✅¹ | ✅¹  | ✅¹ | N/A | N/A |
+| PyPy 3.9 v7.3 | ✅ | N/A | ✅  | N/A | N/A | ✅¹ | ✅¹  | ✅¹ | N/A | N/A |
 
 <sup>¹ PyPy is only supported for manylinux wheels.</sup><br>
 <sup>² Windows arm64 support is experimental.</sup><br>
diff --git a/bin/update_pythons.py b/bin/update_pythons.py
index 30859259..e4939f79 100755
--- a/bin/update_pythons.py
+++ b/bin/update_pythons.py
@@ -109,7 +109,12 @@ def __init__(self, arch_str: ArchStr):
         response = requests.get("https://downloads.python.org/pypy/versions.json")
         response.raise_for_status()
 
-        releases = [r for r in response.json() if r["pypy_version"] != "nightly"]
+        releases = [
+            r
+            for r in response.json()
+            if r["pypy_version"] != "nightly"
+            and f'{r["python_version"]}-{r["pypy_version"]}' != "3.7.12-7.3.8"
+        ]
         for release in releases:
             release["pypy_version"] = Version(release["pypy_version"])
             release["python_version"] = Version(release["python_version"])
diff --git a/cibuildwheel/resources/build-platforms.toml b/cibuildwheel/resources/build-platforms.toml
index 13625461..5bb0350e 100644
--- a/cibuildwheel/resources/build-platforms.toml
+++ b/cibuildwheel/resources/build-platforms.toml
@@ -12,6 +12,7 @@ python_configurations = [
   { identifier = "cp310-manylinux_i686", version = "3.10", path_str = "/opt/python/cp310-cp310" },
   { identifier = "pp37-manylinux_x86_64", version = "3.7", path_str = "/opt/python/pp37-pypy37_pp73" },
   { identifier = "pp38-manylinux_x86_64", version = "3.8", path_str = "/opt/python/pp38-pypy38_pp73" },
+  { identifier = "pp39-manylinux_x86_64", version = "3.9", path_str = "/opt/python/pp39-pypy39_pp73" },
   { identifier = "cp36-manylinux_aarch64", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
   { identifier = "cp37-manylinux_aarch64", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
   { identifier = "cp38-manylinux_aarch64", version = "3.8", path_str = "/opt/python/cp38-cp38" },
@@ -29,8 +30,10 @@ python_configurations = [
   { identifier = "cp310-manylinux_s390x", version = "3.10", path_str = "/opt/python/cp310-cp310" },
   { identifier = "pp37-manylinux_aarch64", version = "3.7", path_str = "/opt/python/pp37-pypy37_pp73" },
   { identifier = "pp38-manylinux_aarch64", version = "3.8", path_str = "/opt/python/pp38-pypy38_pp73" },
+  { identifier = "pp39-manylinux_aarch64", version = "3.9", path_str = "/opt/python/pp39-pypy39_pp73" },
   { identifier = "pp37-manylinux_i686", version = "3.7", path_str = "/opt/python/pp37-pypy37_pp73" },
   { identifier = "pp38-manylinux_i686", version = "3.8", path_str = "/opt/python/pp38-pypy38_pp73" },
+  { identifier = "pp39-manylinux_i686", version = "3.9", path_str = "/opt/python/pp39-pypy39_pp73" },
   { identifier = "cp36-musllinux_x86_64", version = "3.6", path_str = "/opt/python/cp36-cp36m" },
   { identifier = "cp37-musllinux_x86_64", version = "3.7", path_str = "/opt/python/cp37-cp37m" },
   { identifier = "cp38-musllinux_x86_64", version = "3.8", path_str = "/opt/python/cp38-cp38" },
@@ -71,8 +74,9 @@ python_configurations = [
   { identifier = "cp310-macosx_x86_64", version = "3.10", url = "https://www.python.org/ftp/python/3.10.2/python-3.10.2-macos11.pkg" },
   { identifier = "cp310-macosx_arm64", version = "3.10", url = "https://www.python.org/ftp/python/3.10.2/python-3.10.2-macos11.pkg" },
   { identifier = "cp310-macosx_universal2", version = "3.10", url = "https://www.python.org/ftp/python/3.10.2/python-3.10.2-macos11.pkg" },
-  { identifier = "pp37-macosx_x86_64", version = "3.7", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.8-osx64.tar.bz2" },
+  { identifier = "pp37-macosx_x86_64", version = "3.7", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.7-osx64.tar.bz2" },
   { identifier = "pp38-macosx_x86_64", version = "3.8", url = "https://downloads.python.org/pypy/pypy3.8-v7.3.8-osx64.tar.bz2" },
+  { identifier = "pp39-macosx_x86_64", version = "3.9", url = "https://downloads.python.org/pypy/pypy3.9-v7.3.8-osx64.tar.bz2" },
 ]
 
 [windows]
@@ -89,6 +93,7 @@ python_configurations = [
   { identifier = "cp310-win_amd64", version = "3.10.2", arch = "64" },
   { identifier = "cp39-win_arm64", version = "3.9.10", arch = "ARM64" },
   { identifier = "cp310-win_arm64", version = "3.10.2", arch = "ARM64" },
-  { identifier = "pp37-win_amd64", version = "3.7", arch = "64", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.8-win64.zip" },
+  { identifier = "pp37-win_amd64", version = "3.7", arch = "64", url = "https://downloads.python.org/pypy/pypy3.7-v7.3.7-win64.zip" },
   { identifier = "pp38-win_amd64", version = "3.8", arch = "64", url = "https://downloads.python.org/pypy/pypy3.8-v7.3.8-win64.zip" },
+  { identifier = "pp39-win_amd64", version = "3.9", arch = "64", url = "https://downloads.python.org/pypy/pypy3.9-v7.3.8-win64.zip" },
 ]
diff --git a/cibuildwheel/resources/pinned_docker_images.cfg b/cibuildwheel/resources/pinned_docker_images.cfg
index 3b946830..ffcb0f03 100644
--- a/cibuildwheel/resources/pinned_docker_images.cfg
+++ b/cibuildwheel/resources/pinned_docker_images.cfg
@@ -1,43 +1,43 @@
 [x86_64]
 manylinux1 = quay.io/pypa/manylinux1_x86_64:2022-02-20-044a1ea
-manylinux2010 = quay.io/pypa/manylinux2010_x86_64:2022-02-20-e7cad68
-manylinux2014 = quay.io/pypa/manylinux2014_x86_64:2022-02-20-e7cad68
-manylinux_2_24 = quay.io/pypa/manylinux_2_24_x86_64:2022-02-20-e7cad68
-musllinux_1_1 = quay.io/pypa/musllinux_1_1_x86_64:2022-02-20-e7cad68
+manylinux2010 = quay.io/pypa/manylinux2010_x86_64:2022-02-24-3876535
+manylinux2014 = quay.io/pypa/manylinux2014_x86_64:2022-02-24-3876535
+manylinux_2_24 = quay.io/pypa/manylinux_2_24_x86_64:2022-02-24-3876535
+musllinux_1_1 = quay.io/pypa/musllinux_1_1_x86_64:2022-02-24-3876535
 
 [i686]
 manylinux1 = quay.io/pypa/manylinux1_i686:2022-02-20-044a1ea
-manylinux2010 = quay.io/pypa/manylinux2010_i686:2022-02-20-e7cad68
-manylinux2014 = quay.io/pypa/manylinux2014_i686:2022-02-20-e7cad68
-manylinux_2_24 = quay.io/pypa/manylinux_2_24_i686:2022-02-20-e7cad68
-musllinux_1_1 = quay.io/pypa/musllinux_1_1_i686:2022-02-20-e7cad68
+manylinux2010 = quay.io/pypa/manylinux2010_i686:2022-02-24-3876535
+manylinux2014 = quay.io/pypa/manylinux2014_i686:2022-02-24-3876535
+manylinux_2_24 = quay.io/pypa/manylinux_2_24_i686:2022-02-24-3876535
+musllinux_1_1 = quay.io/pypa/musllinux_1_1_i686:2022-02-24-3876535
 
 [pypy_x86_64]
-manylinux2010 = quay.io/pypa/manylinux2010_x86_64:2022-02-20-e7cad68
-manylinux2014 = quay.io/pypa/manylinux2014_x86_64:2022-02-20-e7cad68
-manylinux_2_24 = quay.io/pypa/manylinux_2_24_x86_64:2022-02-20-e7cad68
+manylinux2010 = quay.io/pypa/manylinux2010_x86_64:2022-02-24-3876535
+manylinux2014 = quay.io/pypa/manylinux2014_x86_64:2022-02-24-3876535
+manylinux_2_24 = quay.io/pypa/manylinux_2_24_x86_64:2022-02-24-3876535
 
 [pypy_i686]
-manylinux2010 = quay.io/pypa/manylinux2010_i686:2022-02-20-e7cad68
-manylinux2014 = quay.io/pypa/manylinux2014_i686:2022-02-20-e7cad68
-manylinux_2_24 = quay.io/pypa/manylinux_2_24_i686:2022-02-20-e7cad68
+manylinux2010 = quay.io/pypa/manylinux2010_i686:2022-02-24-3876535
+manylinux2014 = quay.io/pypa/manylinux2014_i686:2022-02-24-3876535
+manylinux_2_24 = quay.io/pypa/manylinux_2_24_i686:2022-02-24-3876535
 
 [aarch64]
-manylinux2014 = quay.io/pypa/manylinux2014_aarch64:2022-02-20-e7cad68
-manylinux_2_24 = quay.io/pypa/manylinux_2_24_aarch64:2022-02-20-e7cad68
-musllinux_1_1 = quay.io/pypa/musllinux_1_1_aarch64:2022-02-20-e7cad68
+manylinux2014 = quay.io/pypa/manylinux2014_aarch64:2022-02-24-3876535
+manylinux_2_24 = quay.io/pypa/manylinux_2_24_aarch64:2022-02-24-3876535
+musllinux_1_1 = quay.io/pypa/musllinux_1_1_aarch64:2022-02-24-3876535
 
 [ppc64le]
-manylinux2014 = quay.io/pypa/manylinux2014_ppc64le:2022-02-20-e7cad68
-manylinux_2_24 = quay.io/pypa/manylinux_2_24_ppc64le:2022-02-20-e7cad68
-musllinux_1_1 = quay.io/pypa/musllinux_1_1_ppc64le:2022-02-20-e7cad68
+manylinux2014 = quay.io/pypa/manylinux2014_ppc64le:2022-02-24-3876535
+manylinux_2_24 = quay.io/pypa/manylinux_2_24_ppc64le:2022-02-24-3876535
+musllinux_1_1 = quay.io/pypa/musllinux_1_1_ppc64le:2022-02-24-3876535
 
 [s390x]
-manylinux2014 = quay.io/pypa/manylinux2014_s390x:2022-02-20-e7cad68
-manylinux_2_24 = quay.io/pypa/manylinux_2_24_s390x:2022-02-20-e7cad68
-musllinux_1_1 = quay.io/pypa/musllinux_1_1_s390x:2022-02-20-e7cad68
+manylinux2014 = quay.io/pypa/manylinux2014_s390x:2022-02-24-3876535
+manylinux_2_24 = quay.io/pypa/manylinux_2_24_s390x:2022-02-24-3876535
+musllinux_1_1 = quay.io/pypa/musllinux_1_1_s390x:2022-02-24-3876535
 
 [pypy_aarch64]
-manylinux2014 = quay.io/pypa/manylinux2014_aarch64:2022-02-20-e7cad68
-manylinux_2_24 = quay.io/pypa/manylinux_2_24_aarch64:2022-02-20-e7cad68
+manylinux2014 = quay.io/pypa/manylinux2014_aarch64:2022-02-24-3876535
+manylinux_2_24 = quay.io/pypa/manylinux_2_24_aarch64:2022-02-24-3876535
 
diff --git a/docs/options.md b/docs/options.md
index 46f2d24e..45f67873 100644
--- a/docs/options.md
+++ b/docs/options.md
@@ -211,6 +211,7 @@ ### `CIBW_BUILD`, `CIBW_SKIP` {: #build-skip}
 | Python 3.10  | cp310-macosx_x86_64<br/>cp310-macosx_universal2<br/>cp310-macosx_arm64 | cp310-win_amd64<br/>cp310-win32<br/>cp310-win_arm64   | cp310-manylinux_x86_64<br/>cp310-manylinux_i686<br/>cp310-musllinux_x86_64<br/>cp310-musllinux_i686 | cp310-manylinux_aarch64<br/>cp310-manylinux_ppc64le<br/>cp310-manylinux_s390x<br/>cp310-musllinux_aarch64<br/>cp310-musllinux_ppc64le<br/>cp310-musllinux_s390x |
 | PyPy3.7 v7.3 | pp37-macosx_x86_64                                                     | pp37-win_amd64                   | pp37-manylinux_x86_64<br/>pp37-manylinux_i686                                                       | pp37-manylinux_aarch64                                                                                                                                          |
 | PyPy3.8 v7.3 | pp38-macosx_x86_64                                                     | pp38-win_amd64                   | pp38-manylinux_x86_64<br/>pp38-manylinux_i686                                                       | pp38-manylinux_aarch64                                                                                                                                          |
+| PyPy3.9 v7.3 | pp39-macosx_x86_64                                                     | pp39-win_amd64                   | pp39-manylinux_x86_64<br/>pp39-manylinux_i686                                                       | pp39-manylinux_aarch64                                                                                                                                          |
 
 The list of supported and currently selected build identifiers can also be retrieved by passing the `--print-build-identifiers` flag to cibuildwheel.
 The format is `python_tag-platform_tag`, with tags similar to those in [PEP 425](https://www.python.org/dev/peps/pep-0425/#details).


Test patch:
diff --git a/test/test_manylinuxXXXX_only.py b/test/test_manylinuxXXXX_only.py
index f7c533d4..7ae5ebb1 100644
--- a/test/test_manylinuxXXXX_only.py
+++ b/test/test_manylinuxXXXX_only.py
@@ -71,6 +71,9 @@ def test(manylinux_image, tmp_path):
     if manylinux_image in {"manylinux1"}:
         # We don't have a manylinux1 image for PyPy & CPython 3.10 and above
         add_env["CIBW_SKIP"] = "pp* cp31*"
+    if manylinux_image in {"manylinux2010"}:
+        # We don't have a manylinux2010 image for PyPy 3.9
+        add_env["CIBW_SKIP"] = "pp39*"
 
     actual_wheels = utils.cibuildwheel_run(project_dir, add_env=add_env)
 
@@ -88,4 +91,7 @@ def test(manylinux_image, tmp_path):
     if manylinux_image in {"manylinux1"}:
         # remove PyPy & CPython 3.10 and above
         expected_wheels = [w for w in expected_wheels if "-pp" not in w and "-cp31" not in w]
+    if manylinux_image in {"manylinux2010"}:
+        # remove PyPy 3.9
+        expected_wheels = [w for w in expected_wheels if "-pp39" not in w]
     assert set(actual_wheels) == set(expected_wheels)
diff --git a/test/utils.py b/test/utils.py
index 98715954..0248a7c1 100644
--- a/test/utils.py
+++ b/test/utils.py
@@ -135,7 +135,7 @@ def expected_wheels(
     python_abi_tags = ["cp36-cp36m", "cp37-cp37m", "cp38-cp38", "cp39-cp39", "cp310-cp310"]
 
     if machine_arch in ["x86_64", "AMD64", "x86", "aarch64"]:
-        python_abi_tags += ["pp37-pypy37_pp73", "pp38-pypy38_pp73"]
+        python_abi_tags += ["pp37-pypy37_pp73", "pp38-pypy38_pp73", "pp39-pypy39_pp73"]
 
     if platform == "macos" and machine_arch == "arm64":
         # currently, arm64 macs are only supported by cp39 & cp310
diff --git a/unit_test/option_prepare_test.py b/unit_test/option_prepare_test.py
index d21ba65d..354cda4a 100644
--- a/unit_test/option_prepare_test.py
+++ b/unit_test/option_prepare_test.py
@@ -11,7 +11,7 @@
 from cibuildwheel import linux, util
 from cibuildwheel.__main__ import main
 
-ALL_IDS = {"cp36", "cp37", "cp38", "cp39", "cp310", "pp37", "pp38"}
+ALL_IDS = {"cp36", "cp37", "cp38", "cp39", "cp310", "pp37", "pp38", "pp39"}
 
 
 @pytest.fixture
@@ -133,7 +133,7 @@ def test_build_with_override_launches(mock_build_docker, monkeypatch, tmp_path):
 
     identifiers = {x.identifier for x in kwargs["platform_configs"]}
     assert identifiers == {
-        f"{x}-manylinux_x86_64" for x in ALL_IDS - {"cp36", "cp310", "pp37", "pp38"}
+        f"{x}-manylinux_x86_64" for x in ALL_IDS - {"cp36", "cp310", "pp37", "pp38", "pp39"}
     }
     assert kwargs["options"].build_options("cp37-manylinux_x86_64").before_all == ""
 
@@ -146,6 +146,7 @@ def test_build_with_override_launches(mock_build_docker, monkeypatch, tmp_path):
         "cp310-manylinux_x86_64",
         "pp37-manylinux_x86_64",
         "pp38-manylinux_x86_64",
+        "pp39-manylinux_x86_64",
     }
 
     kwargs = build_on_docker.call_args_list[3][1]