Repository: spack/spack
Pull request #49680: py-setuptools: add v76.1.0 -> v78.1.0
Author: login: RobertMaaskant
Labels: name: new-version
name: python
name: dependencies
name: update-package
Created: 2025-03-25T15:57:00Z | Merged: 2025-04-16T13:35:11Z

Add versions:
- 78.0.2
- 78.0.1
- 77.0.3
- 77.0.1
- 76.1.0
- 75.3.2
- 75.3.1

Changelog: https://setuptools.pypa.io/en/latest/history.html

Tested deprecating versions are not breaking dependents by commenting them out (fake delete) and running `spack audit packages`.

Testing new versions don't break dependents using CI runs by adding them one by one and fixing found issues. This is a WIP.


Diff:
diff --git a/var/spack/repos/builtin/packages/py-flatbuffers/package.py b/var/spack/repos/builtin/packages/py-flatbuffers/package.py
index 4f667ed7dd031e..b17ece10bf5d3d 100644
--- a/var/spack/repos/builtin/packages/py-flatbuffers/package.py
+++ b/var/spack/repos/builtin/packages/py-flatbuffers/package.py
@@ -25,4 +25,6 @@ class PyFlatbuffers(PythonPackage):
     version("2.0", sha256="12158ab0272375eab8db2d663ae97370c33f152b27801fa6024e1d6105fd4dd2")
     version("1.12", sha256="63bb9a722d5e373701913e226135b28a6f6ac200d5cc7b4d919fa38d73b44610")
 
+    # https://setuptools.pypa.io/en/latest/history.html#v77-0-0
+    depends_on("py-setuptools@:76", type="build", when="@:24.3.25")
     depends_on("py-setuptools", type="build")
diff --git a/var/spack/repos/builtin/packages/py-setuptools/package.py b/var/spack/repos/builtin/packages/py-setuptools/package.py
index 8d4510adc50963..3a7355d10132a1 100644
--- a/var/spack/repos/builtin/packages/py-setuptools/package.py
+++ b/var/spack/repos/builtin/packages/py-setuptools/package.py
@@ -19,11 +19,19 @@ class PySetuptools(Package, PythonExtension):
     # Requires railroad
     skip_modules = ["setuptools._vendor", "pkg_resources._vendor"]
 
+    version("78.1.0", sha256="3e386e96793c8702ae83d17b853fb93d3e09ef82ec62722e61da5cd22376dcd8")
+    version("78.0.2", sha256="4a612c80e1f1d71b80e4906ce730152e8dec23df439f82731d9d0b608d7b700d")
+    version("78.0.1", sha256="1cc9b32ee94f93224d6c80193cbb768004667aa2f2732a473d6949b0236c1d4e")
+    version("77.0.3", sha256="67122e78221da5cf550ddd04cf8742c8fe12094483749a792d56cd669d6cf58c")
+    version("77.0.1", sha256="81a234dff81a82bb52e522c8aef145d0dd4de1fd6de4d3b196d0f77dc2fded26")
+    version("76.1.0", sha256="34750dcb17d046929f545dec9b8349fe42bf4ba13ddffee78428aec422dbfb73")
     version("76.0.0", sha256="199466a166ff664970d0ee145839f5582cb9bca7a0a3a2e795b6a9cb2308e9c6")
     version("75.9.1", sha256="0a6f876d62f4d978ca1a11ab4daf728d1357731f978543ff18ecdbf9fd071f73")
     version("75.8.2", sha256="558e47c15f1811c1fa7adbd0096669bf76c1d3f433f58324df69f3f5ecac4e8f")
     version("75.8.1", sha256="3bc32c0b84c643299ca94e77f834730f126efd621de0cc1de64119e0e17dab1f")
     version("75.8.0", sha256="e3982f444617239225d675215d51f6ba05f845d4eec313da4418fdbb56fb27e3")
+    version("75.3.2", sha256="90ab613b6583fc02d5369cbca13ea26ea0e182d1df2d943ee9cbe81d4c61add9")
+    version("75.3.1", sha256="ccd77cda9d3bc3d3e99036d221b91d15f86e53195139d643b5b5299d42463cd3")
     version("75.3.0", sha256="f2504966861356aa38616760c0f66568e535562374995367b4e69c7143cf6bcd")
     version("69.2.0", sha256="c21c49fb1042386df081cb5d86759792ab89efca84cf114889191cd09aacc80c")
     version("69.1.1", sha256="02fa291a0471b3a18b2b2481ed902af520c69e8ae0919c13da936542754b4c56")
diff --git a/var/spack/repos/builtin/packages/py-torch-cluster/package.py b/var/spack/repos/builtin/packages/py-torch-cluster/package.py
index d0d04a2d951a69..4d6fcfff0c82e4 100644
--- a/var/spack/repos/builtin/packages/py-torch-cluster/package.py
+++ b/var/spack/repos/builtin/packages/py-torch-cluster/package.py
@@ -21,6 +21,8 @@ class PyTorchCluster(PythonPackage):
     depends_on("cxx", type="build")
 
     depends_on("python", type=("build", "link", "run"))
+    # https://setuptools.pypa.io/en/latest/history.html#v77-0-0
+    depends_on("py-setuptools@:76", type="build", when="@:1.6.3")
     depends_on("py-setuptools", type="build")
     depends_on("py-scipy", type=("build", "run"))
 
diff --git a/var/spack/repos/builtin/packages/py-torch-scatter/package.py b/var/spack/repos/builtin/packages/py-torch-scatter/package.py
index 7d9a5d4e0e8407..8741fa6c222fa6 100644
--- a/var/spack/repos/builtin/packages/py-torch-scatter/package.py
+++ b/var/spack/repos/builtin/packages/py-torch-scatter/package.py
@@ -21,6 +21,8 @@ class PyTorchScatter(PythonPackage):
     depends_on("cxx", type="build")
 
     depends_on("python", type=("build", "link", "run"))
+    # https://setuptools.pypa.io/en/latest/history.html#v77-0-0
+    depends_on("py-setuptools@:76", type="build", when="@:2.1.2")
     depends_on("py-setuptools", type="build")
 
     # Undocumented dependencies
diff --git a/var/spack/repos/builtin/packages/py-torch-spline-conv/package.py b/var/spack/repos/builtin/packages/py-torch-spline-conv/package.py
index 39bd4e3c383812..47f5ef8c3b041b 100644
--- a/var/spack/repos/builtin/packages/py-torch-spline-conv/package.py
+++ b/var/spack/repos/builtin/packages/py-torch-spline-conv/package.py
@@ -21,6 +21,8 @@ class PyTorchSplineConv(PythonPackage):
     depends_on("cxx", type="build")
 
     depends_on("python", type=("build", "link", "run"))
+    # https://setuptools.pypa.io/en/latest/history.html#v77-0-0
+    depends_on("py-setuptools@:76", type="build", when="@:1.2.2")
     depends_on("py-setuptools", type="build")
 
     # Undocumented dependencies
diff --git a/var/spack/repos/builtin/packages/py-torchvision/package.py b/var/spack/repos/builtin/packages/py-torchvision/package.py
index 0d96754d6fb5d4..191fae7b4bb50a 100644
--- a/var/spack/repos/builtin/packages/py-torchvision/package.py
+++ b/var/spack/repos/builtin/packages/py-torchvision/package.py
@@ -48,36 +48,13 @@ class PyTorchvision(PythonPackage):
     version("0.9.1", sha256="79964773729880e0eee0e6af13f336041121d4cc8491a3e2c0e5f184cac8a718")
     version("0.9.0", sha256="9351ed92aded632f8c7f59dfadac13c191a834babe682f5785ea47e6fcf6b472")
     version("0.8.2", sha256="9a866c3c8feb23b3221ce261e6153fc65a98ce9ceaa71ccad017016945c178bf")
-    version(
-        "0.8.1",
-        sha256="c46734c679c99f93e5c06654f4295a05a6afe6c00a35ebd26a2cce507ae1ccbd",
-        deprecated=True,
-    )
-    version(
-        "0.8.0",
-        sha256="b5f040faffbfc7bac8d4687d8665bd1196937334589b3fb5fcf15bb69ca25391",
-        deprecated=True,
-    )
-    version(
-        "0.7.0",
-        sha256="fa0a6f44a50451115d1499b3f2aa597e0092a07afce1068750260fa7dd2c85cb",
-        deprecated=True,
-    )
-    version(
-        "0.6.1",
-        sha256="8173680a976c833640ecbd0d7e6f0a11047bf8833433e2147180efc905e48656",
-        deprecated=True,
-    )
-    version(
-        "0.6.0",
-        sha256="02de11b3abe6882de4032ce86dab9c7794cbc84369b44d04e667486580f0f1f7",
-        deprecated=True,
-    )
-    version(
-        "0.5.0",
-        sha256="eb9afc93df3d174d975ee0914057a9522f5272310b4d56c150b955c287a4d74d",
-        deprecated=True,
-    )
+    with default_args(deprecated=True):
+        version("0.8.1", sha256="c46734c679c99f93e5c06654f4295a05a6afe6c00a35ebd26a2cce507ae1ccbd")
+        version("0.8.0", sha256="b5f040faffbfc7bac8d4687d8665bd1196937334589b3fb5fcf15bb69ca25391")
+        version("0.7.0", sha256="fa0a6f44a50451115d1499b3f2aa597e0092a07afce1068750260fa7dd2c85cb")
+        version("0.6.1", sha256="8173680a976c833640ecbd0d7e6f0a11047bf8833433e2147180efc905e48656")
+        version("0.6.0", sha256="02de11b3abe6882de4032ce86dab9c7794cbc84369b44d04e667486580f0f1f7")
+        version("0.5.0", sha256="eb9afc93df3d174d975ee0914057a9522f5272310b4d56c150b955c287a4d74d")
 
     desc = "Enable support for native encoding/decoding of {} formats in torchvision.io"
     variant("png", default=True, description=desc.format("PNG"))
@@ -145,6 +122,8 @@ class PyTorchvision(PythonPackage):
     depends_on("ninja", type="build")
 
     # setup.py
+    # https://setuptools.pypa.io/en/latest/history.html#v77-0-0
+    depends_on("py-setuptools@:76", type="build", when="@:0.21.0")
     depends_on("py-setuptools", type="build")
     depends_on("py-numpy", type=("build", "run"))
     # https://github.com/pytorch/vision/issues/8460


Reviews:
user: login: adamjstewart
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T12:02:47Z
user: login: RobertMaaskant
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-15T12:52:17Z
user: login: adamjstewart
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T13:18:13Z
user: login: adamjstewart
body: Let's merge to get this out and try to fix these individual packages later.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-16T13:34:52Z

Review comments:
diff_hunk: @@ -145,6 +122,8 @@ class PyTorchvision(PythonPackage):
     depends_on("ninja", type="build")
 
     # setup.py
+    # https://setuptools.pypa.io/en/latest/history.html#v77-0-0
+    depends_on("py-setuptools@:76", type="build", when="@:0.21.0")
path: var/spack/repos/builtin/packages/py-torchvision/package.py
user: login: adamjstewart
body: Can you tell me more about these additions? Why were they added? Have they actually been fixed in main, or are you just assuming this? Have these issues been reported upstream? The release notes don't really explain why these are here.
created_at: 2025-04-15T12:02:47Z
author_association: MEMBER
diff_hunk: @@ -145,6 +122,8 @@ class PyTorchvision(PythonPackage):
     depends_on("ninja", type="build")
 
     # setup.py
+    # https://setuptools.pypa.io/en/latest/history.html#v77-0-0
+    depends_on("py-setuptools@:76", type="build", when="@:0.21.0")
path: var/spack/repos/builtin/packages/py-torchvision/package.py
user: login: RobertMaaskant
body: `py-setuptools@77` contains breaking changes around how to define licenses. It is to be expected some packages are affected, since they were released before the breaking change even existed. So I don't see these as issues, but just incompatible with the new setuptools versions.

In this case the error was `error: metadata-generation-failed`. For exact details see the CI logs: https://gitlab.spack.io/spack/spack/-/jobs/16211924. The issues have not been reported upstream and I have not found existing issues. And to be honest, that feels more like a nice to have than a must have.

I have consciously only added the constraint to the current versions, so that newer versions don't automatically stay on old `py-setuptools` versions.
created_at: 2025-04-15T12:52:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -145,6 +122,8 @@ class PyTorchvision(PythonPackage):
     depends_on("ninja", type="build")
 
     # setup.py
+    # https://setuptools.pypa.io/en/latest/history.html#v77-0-0
+    depends_on("py-setuptools@:76", type="build", when="@:0.21.0")
path: var/spack/repos/builtin/packages/py-torchvision/package.py
user: login: adamjstewart
body: I would personally leave off the upper bound so that newer versions don't immediately break when added, but I guess CI will catch that. I'll try to fix torchvision if I can reproduce the issue, that's the only package I personally care about.
created_at: 2025-04-15T13:18:13Z
author_association: MEMBER

PR comments:
user: login: spackbot-app[bot]
created_at: 2025-03-25T15:57:09Z
body: Hi @RobertMaaskant! I noticed that the following package(s) don't yet have maintainers:

* py-setuptools

Are you interested in adopting any of these package(s)? If so, simply add the following to the package class:
```python
    maintainers("RobertMaaskant")
```
If not, could you contact the developers of this package and see if they are interested? You can quickly see who has worked on a package with `spack blame`:

```bash
$ spack blame py-setuptools
```
Thank you for your help! Please don't add maintainers without their consent.

_You don't have to be a Spack expert or package developer in order to be a "maintainer," it just gives us a list of users willing to review PRs or debug issues relating to this package. A package can have multiple maintainers; just add a list of GitHub handles of anyone who wants to volunteer._

author_association: NONE
user: login: adamjstewart
created_at: 2025-04-06T18:57:07Z
body: Why are 77 and 78 commented out?

Why are older versions deprecated?
author_association: MEMBER
user: login: RobertMaaskant
created_at: 2025-04-06T23:41:47Z
body: > Why are 77 and 78 commented out?

Because if I add them directly, CI will only run with 78. I want to be sure all three non-deprecated versions don't cause trouble. I do not have the compute locally to do this without CI.
 
> Why are older versions deprecated?

Because they usually contain bugs that are fixed in newer versions.
author_association: CONTRIBUTOR
user: login: adamjstewart
created_at: 2025-04-07T06:36:52Z
body: You don't have to add both 77 and 78, you could just add 78. 

We don't usually remove old versions unless there are known issues. This hurts reproducibility. 
author_association: MEMBER
user: login: RobertMaaskant
created_at: 2025-04-08T11:33:30Z
body: > You don't have to add both 77 and 78, you could just add 78.

Well, [setuptools follows semver](https://setuptools.pypa.io/en/latest/development/developer-guide.html#semantic-versioning). So it might be the case that a downstream package is broken by v78, but not by v77. So if I only add v78, dependents are artificially stuck on v76. And I find it weird that the timing of the pull requests would matter for what versions are available. So I would prefer to at least add and test all the latest major versions.

I am working on setting up some compute locally, so if spending the CI cycles is the problem, that should be solved pretty soon.

> We don't usually remove old versions unless there are known issues. This hurts reproducibility.

Ok, I have added them back to the normal versions.
author_association: CONTRIBUTOR
user: login: RobertMaaskant
created_at: 2025-04-09T13:23:33Z
body: @spackbot re-run pipeline -- unrelated runner failures
author_association: CONTRIBUTOR
user: login: spackbot-app[bot]
created_at: 2025-04-09T13:23:41Z
body: I've started that [pipeline](https://gitlab.spack.io/spack/spack/-/pipelines/1046028) for you!
author_association: NONE
user: login: RobertMaaskant
created_at: 2025-04-11T06:37:18Z
body: @spackbot re-run pipeline -- unrelated runner failures
author_association: CONTRIBUTOR
user: login: spackbot-app[bot]
created_at: 2025-04-11T06:37:26Z
body: I've started that [pipeline](https://gitlab.spack.io/spack/spack/-/pipelines/1048652) for you!
author_association: NONE
user: login: RobertMaaskant
created_at: 2025-04-11T14:44:27Z
body: @adamjstewart pr is finally ready for review :)
author_association: CONTRIBUTOR