Repository: spack/spack

Issue:
py-urllib3: new versions 2.*
This adds a new bugfix release in the 1.26 series (https://github.com/urllib3/urllib3/compare/1.26.12...1.26.16, no changes to dependency versions), and the first version in the 2.* series, which brings a switch to hatchling. New variant for `zstd` support. Default remains the most recent 1.26.* release.

TODO:
- [x] The `secure` variant is deprecated (https://github.com/urllib3/urllib3/issues/2680) and will be removed in 2.1. Do we have a way of marking variants as deprecated?
- [x] Figure out what to do with `py-brotlipy` which doesn't have a recent enough version, and there's some duplication with `brotli` maybe.


Discussion:
@spackbot fix style
Let me see if I can fix that for you!

I was able to run `spack style --fix` for you!
<details>
<summary><b>spack style --fix</b></summary>

```bash
==> Running style checks on spack
  selected: isort, black, flake8, mypy
==> Modified files
  var/spack/repos/builtin/packages/py-urllib3/package.py
==> Running isort checks
  isort checks were clean
==> Running black checks
reformatted var/spack/repos/builtin/packages/py-urllib3/package.py
All done! ✨ 🍰 ✨
1 file reformatted.
  black checks were clean
==> Running flake8 checks
  flake8 checks were clean
==> Running mypy checks
Success: no issues found in 577 source files
  mypy checks were clean
==> spack style checks were clean

```
</details>
Keep in mind that I cannot fix your flake8 or mypy errors, so if you have any you'll need to fix them and update the pull request.
If I was able to push to your branch, if you make further changes you will need to pull from your updated branch before pushing again.


I've updated the branch with style fixes.
> Default remains the most recent 1.26.* release.

Why?

> Do we have a way of marking variants as deprecated?

We can't deprecate but we can restrict:
```python
variant("secure", default=False, description="Add SSL/TLS support", when="@:2.0")
```
@alalazo I'll open an issue to mark variants as deprecated so we don't forget this request. I know it's low priority.

> Figure out what to do with py-brotlipy

Let me know if you need any guidance from me and I can take a look
> Why?

There is some potential migration to be done by users of this package: https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html. Admittedly, 2.0.* deprecates those features and starts warning about them, and 2.1.* will introduce the breaking changes, so maybe that's a good argument to make 2.0.* the default.
This is now ready for review.

```conso==> Installed packages
-- linux-ubuntu23.04-skylake / gcc@13.1.0 -----------------------
bb5j35y py-urllib3@2.0.3+brotli+secure+socks+zstd build_system=python_pip
==> 1 installed package
```

Unrelated gitlab/ci pipeline failure at https://gitlab.spack.io/spack/spack/-/jobs/7651219#L181
@spackbot run pipeline 
I've started that [pipeline](https://gitlab.spack.io//spack/spack/-/pipelines/444183) for you!
(comment) I'm in favor of not using preferred=True to avoid breakage, but instead let things break and set compat bounds accordingly -- otherwise it will go undetected and those bounds never get added.
One potential reason to keep urllib3 preferred to <2 is that botocore only recently supports urllib3=2, see https://github.com/urllib3/urllib3/blob/main/docs/v2-migration-guide.rst#importerror-cannot-import-name-default_ciphers-from-urllib3utilssl_. For spack installs which use buildcaches on S3, botocore could be provided by system packages which are commonly older (and botocore isn't bootstrapped). Once urllib3=2 is installed, this seems to leave spack in a degraded state because the system botocore tries to use the spack urllib3 (at least in environments)...

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/py-brotli/package.py b/var/spack/repos/builtin/packages/py-brotli/package.py
new file mode 100644
--- /dev/null
+++ b/var/spack/repos/builtin/packages/py-brotli/package.py
@@ -0,0 +1,17 @@
+# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
+# Spack Project Developers. See the top-level COPYRIGHT file for details.
+#
+# SPDX-License-Identifier: (Apache-2.0 OR MIT)
+
+from spack.package import *
+
+
+class PyBrotli(PythonPackage):
+    """Python bindings for the Brotli compression library."""
+
+    homepage = "https://github.com/google/brotli"
+    pypi = "Brotli/Brotli-1.1.0.tar.gz"
+
+    version("1.1.0", sha256="81de08ac11bcb85841e440c13611c00b67d3bf82698314928d0b676362546724")
+
+    depends_on("py-setuptools", type="build")
diff --git a/var/spack/repos/builtin/packages/py-urllib3/package.py b/var/spack/repos/builtin/packages/py-urllib3/package.py
--- a/var/spack/repos/builtin/packages/py-urllib3/package.py
+++ b/var/spack/repos/builtin/packages/py-urllib3/package.py
@@ -12,7 +12,10 @@ class PyUrllib3(PythonPackage):
 
     homepage = "https://urllib3.readthedocs.io/"
     pypi = "urllib3/urllib3-1.25.6.tar.gz"
+    git = "https://github.com/urllib3/urllib3.git"
 
+    version("2.0.6", sha256="b19e1a85d206b56d7df1d5e683df4a7725252a964e3993648dd0fb5a1c157564")
+    version("2.0.5", sha256="13abf37382ea2ce6fb744d4dad67838eec857c9f4f57009891805e0b5e123594")
     version("1.26.12", sha256="3fa96cf423e6987997fc326ae8df396db2a8b7c667747d47ddd8ecba91f4a74e")
     version("1.26.6", sha256="f57b4c16c62fa2760b7e3d97c35b255512fb6b59a259730f36ba32ce9f8e342f")
     version("1.25.9", sha256="3018294ebefce6572a474f0604c2021e33b3fd8006ecd11d62107a5d2a963527")
@@ -23,22 +26,29 @@ class PyUrllib3(PythonPackage):
     version("1.20", sha256="97ef2b6e2878d84c0126b9f4e608e37a951ca7848e4855a7f7f4437d5c34a72f")
     version("1.14", sha256="dd4fb13a4ce50b18338c7e4d665b21fd38632c5d4b1d9f1a1379276bd3c08d37")
 
-    variant("socks", default=False, description="SOCKS and HTTP proxy support")
+    variant("brotli", default=False, when="@1.25:", description="Add Brotli support")
     variant("secure", default=False, description="Add SSL/TLS support")
-    variant("brotli", default=False, description="Add Brotli support")
+    variant("socks", default=False, when="@1.15:", description="SOCKS and HTTP proxy support")
 
-    depends_on("python@2.7:2.8,3.4:", when="@:1.25", type=("build", "run"))
-    depends_on("python@2.7:2.8,3.5:", when="@1.26.6", type=("build", "run"))
-    depends_on("python@2.7:2.8,3.6:3", when="@1.26.12:", type=("build", "run"))
+    depends_on("py-hatchling@1.6:1", when="@2:", type="build")
 
-    depends_on("py-setuptools", type="build")
+    with when("+brotli"):
+        depends_on("py-brotli@1.0.9:", when="@1.26.9:", type=("build", "run"))
 
-    depends_on("py-pyopenssl@0.14:", when="+secure")
-    depends_on("py-cryptography@1.3.4:", when="+secure")
-    depends_on("py-idna@2:", when="+secure")
-    depends_on("py-certifi", when="+secure")
-    depends_on("py-urllib3-secure-extra", when="+secure @1.26.12:")
+        # Historical dependencies
+        depends_on("py-brotlipy@0.6:", when="@:1.26.8", type=("build", "run"))
 
-    depends_on("py-pysocks@1.5.6,1.5.8:1", when="+socks")
+    with when("+secure"):
+        depends_on("py-pyopenssl@17.1:", when="@2:", type=("build", "run"))
+        depends_on("py-pyopenssl@0.14:", when="@1", type=("build", "run"))
+        depends_on("py-cryptography@1.9:", when="@2:", type=("build", "run"))
+        depends_on("py-cryptography@1.3.4:", when="@1", type=("build", "run"))
+        depends_on("py-idna@2:", type=("build", "run"))
+        depends_on("py-certifi", type=("build", "run"))
+        depends_on("py-urllib3-secure-extra", when="@1.26.12:", type=("build", "run"))
 
-    depends_on("py-brotlipy@0.6:", when="+brotli")
+    depends_on("py-pysocks@1.5.6,1.5.8:1", when="+socks", type=("build", "run"))
+
+    # Historical dependencies
+    depends_on("py-setuptools", when="@1", type="build")
+    depends_on("python@3.6:3", when="@1.26.12:1", type=("build", "run"))


Test patch: