Repository: unionai-oss/pandera (language: python)

Issue:
BackendNotFoundError on databricks/pyspark cluster
Hi,

I am trying to get pandera up and running on databricks.
However, I receive the following `BackendNotFoundError` error and do not know what the cause is:

```python
BackendNotFoundError: Backend not found for backend, class: (<class 'pandera.api.pyspark.container.DataFrameSchema'>, <class 'pyspark.sql.connect.dataframe.DataFrame'>). Looked up the following base classes: (<class 'pyspark.sql.connect.dataframe.DataFrame'>, <class 'object'>)
```

Code example
```python
import pandera.pyspark as pa
import pyspark.sql.types as T

class TestSchema(pa.DataFrameModel):
    id: T.IntegerType() = pa.Field(gt=5)
    name: T.StringType() = pa.Field(str_startswith="B")

df = spark.createDataFrame([(5, "Bread"), (15, "Butter")], ["id", "name"])

TestSchema.validate(check_obj=df)
```

Complete error

```python
BackendNotFoundError: Backend not found for backend, class: (<class 'pandera.api.pyspark.container.DataFrameSchema'>, <class 'pyspark.sql.connect.dataframe.DataFrame'>). Looked up the following base classes: (<class 'pyspark.sql.connect.dataframe.DataFrame'>, <class 'object'>)
File <command-2461794647677534>, line 10
      6     name: T.StringType() = pa.Field(str_startswith="B")
      8 df = spark.createDataFrame([(5, "Bread"), (15, "Butter")], ["id", "name"])
---> 10 TestSchema.validate(check_obj=df)
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pandera/api/pyspark/model.py:289, in DataFrameModel.validate(cls, check_obj, head, tail, sample, random_state, lazy, inplace)
    274 @classmethod
    275 @docstring_substitution(validate_doc=DataFrameSchema.validate.__doc__)
    276 def validate(
   (...)
    284     inplace: bool = False,
    285 ) -> Optional[DataFrameBase[TDataFrameModel]]:
    286     """%(validate_doc)s"""
    287     return cast(
    288         DataFrameBase[TDataFrameModel],
--> 289         cls.to_schema().validate(
    290             check_obj, head, tail, sample, random_state, lazy, inplace
    291         ),
    292     )
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pandera/api/pyspark/container.py:333, in DataFrameSchema.validate(self, check_obj, head, tail, sample, random_state, lazy, inplace)
    330     return check_obj
    331 error_handler = ErrorHandler(lazy)
--> 333 return self._validate(
    334     check_obj=check_obj,
    335     head=head,
    336     tail=tail,
    337     sample=sample,
    338     random_state=random_state,
    339     lazy=lazy,
    340     inplace=inplace,
    341     error_handler=error_handler,
    342 )
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pandera/api/pyspark/container.py:364, in DataFrameSchema._validate(self, check_obj, head, tail, sample, random_state, lazy, inplace, error_handler)
    355 if self._is_inferred:
    356     warnings.warn(
    357         f"This {type(self)} is an inferred schema that hasn't been "
    358         "modified. It's recommended that you refine the schema "
   (...)
    361         UserWarning,
    362     )
--> 364 return self.get_backend(check_obj).validate(
    365     check_obj=check_obj,
    366     schema=self,
    367     head=head,
    368     tail=tail,
    369     sample=sample,
    370     random_state=random_state,
    371     lazy=lazy,
    372     inplace=inplace,
    373     error_handler=error_handler,
    374 )
File /local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/pandera/api/base/schema.py:96, in BaseSchema.get_backend(cls, check_obj, check_type)
     94     except KeyError:
     95         pass
---> 96 raise BackendNotFoundError(
     97     f"Backend not found for backend, class: {(cls, check_obj_cls)}. "
     98     f"Looked up the following base classes: {classes}"
     99 )
```

Runtime environment short:

- databricks-runtime 14.3 LTS
- python 3.10.12
- pyspark 3.5.0
- pandera 0.19.3


Discussion / hints:
Hi Tobi!

I want to add the following observation:
The same error also occurs when running it in a Databricks Notebook in the web UI with a shared all-purpose cluster with version 14.3 (also <class 'pyspark.sql.connect.dataframe.DataFrame'> even though it's not connect).

However, the code example works on a 14.3 as well as a 15.3 personal compute in the web UI. Yet, it doesn't work with databricks connect from VS code - same error. 

Trying to run the file from vs code with "upload and run file on databricks" with the databricks extension again works only with the personal computes, not with the shared. 


So the place where the backends are registered is defined here:
https://github.com/unionai-oss/pandera/blob/main/pandera/backends/pyspark/register.py

Is there a difference between `pyspark.sql.DataFrame` and `pyspark.sql.connect.dataframe.DataFrame`? This may be the issue.
@TobiRoby @gidiLuke I'm trying to reproduce this error. How can I do so with a local setup? (i.e. with no databricks or pyspark cluster)?
Probably these specific modules `*.connect.*` are related to the new [Spark Connect decoupled architecture](https://spark.apache.org/docs/latest/spark-connect-overview.html). This [link](https://spark.apache.org/docs/latest/api/python/getting_started/quickstart_connect.html) has some info on how to setup it.
Spark connect is most probably the centerpiece of the issue in my view as well.
Local reproduction would therefore be possible with the info provided by @filipeo2-mck - thanks! 

@cosmicBboy 
The apis of pyspark.sql.DataFrame and pyspark.sql.connect.dataframe.DataFrame should be the same.
=> how can pyspark.sql.connect.dataframe.DataFrame be added as backend?
Hello!
As I'm also interested in this fix, I'll take this issue and check if I'm able to raise a fix PR for it.

Fix patch:
diff --git a/ci/requirements-py3.10-pandas1.5.3-pydantic1.10.11.txt b/ci/requirements-py3.10-pandas1.5.3-pydantic1.10.11.txt
index 49676b30b..7dc9a5cf8 100644
--- a/ci/requirements-py3.10-pandas1.5.3-pydantic1.10.11.txt
+++ b/ci/requirements-py3.10-pandas1.5.3-pydantic1.10.11.txt
@@ -1,606 +1,194 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.2.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   distributed
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.2.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   ipython
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   asv-runner
-    #   dask
-    #   jupyter-cache
-    #   keyring
-    #   myst-nb
-    #   twine
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.26.0
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.22.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 myst-parser==4.0.0
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 numpy==1.26.4
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==1.5.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   dask
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 pydantic==1.10.11
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   fastapi
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
-pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
+pyspark==3.5.2
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.14.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
-    #   sphinx
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   sqlalchemy
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp6ejs7w6z
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.10-pandas1.5.3-pydantic2.3.0.txt b/ci/requirements-py3.10-pandas1.5.3-pydantic2.3.0.txt
index 8f9c28f75..3889e7e98 100644
--- a/ci/requirements-py3.10-pandas1.5.3-pydantic2.3.0.txt
+++ b/ci/requirements-py3.10-pandas1.5.3-pydantic2.3.0.txt
@@ -1,611 +1,196 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.2.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   distributed
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.2.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   ipython
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   asv-runner
-    #   dask
-    #   jupyter-cache
-    #   keyring
-    #   myst-nb
-    #   twine
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.26.0
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.22.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 myst-parser==4.0.0
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 numpy==1.26.4
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==1.5.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   dask
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 pydantic==2.3.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   fastapi
 pydantic-core==2.6.3
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
-pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
+pyspark==3.5.2
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.14.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
-    #   sphinx
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pydantic-core
-    #   sqlalchemy
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptvr6b9t2
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.10-pandas2.2.2-pydantic1.10.11.txt b/ci/requirements-py3.10-pandas2.2.2-pydantic1.10.11.txt
index b492e8f60..5084b71c4 100644
--- a/ci/requirements-py3.10-pandas2.2.2-pydantic1.10.11.txt
+++ b/ci/requirements-py3.10-pandas2.2.2-pydantic1.10.11.txt
@@ -1,614 +1,196 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.8.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   dask-expr
-    #   distributed
 dask-expr==1.1.10
-    # via dask
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   ipython
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   asv-runner
-    #   dask
-    #   jupyter-cache
-    #   keyring
-    #   myst-nb
-    #   twine
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.26.0
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.31.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 myst-parser==4.0.0
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 numpy==2.0.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==2.2.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   dask
-    #   dask-expr
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   dask-expr
 pydantic==1.10.11
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   fastapi
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
 pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.14.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
-    #   sphinx
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   sqlalchemy
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 tzdata==2024.1
-    # via pandas
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeteiyyih
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.10-pandas2.2.2-pydantic2.3.0.txt b/ci/requirements-py3.10-pandas2.2.2-pydantic2.3.0.txt
index 74109bac1..b98e8a73b 100644
--- a/ci/requirements-py3.10-pandas2.2.2-pydantic2.3.0.txt
+++ b/ci/requirements-py3.10-pandas2.2.2-pydantic2.3.0.txt
@@ -1,619 +1,198 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.8.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   dask-expr
-    #   distributed
 dask-expr==1.1.10
-    # via dask
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   ipython
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   asv-runner
-    #   dask
-    #   jupyter-cache
-    #   keyring
-    #   myst-nb
-    #   twine
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.26.0
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.31.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 myst-parser==4.0.0
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 numpy==2.0.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==2.2.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   dask
-    #   dask-expr
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   dask-expr
 pydantic==2.3.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   fastapi
 pydantic-core==2.6.3
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
 pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.14.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
-    #   sphinx
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pydantic-core
-    #   sqlalchemy
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 tzdata==2024.1
-    # via pandas
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpzu9tkl04
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.11-pandas1.5.3-pydantic1.10.11.txt b/ci/requirements-py3.11-pandas1.5.3-pydantic1.10.11.txt
index 0e7917302..eea6e24cd 100644
--- a/ci/requirements-py3.11-pandas1.5.3-pydantic1.10.11.txt
+++ b/ci/requirements-py3.11-pandas1.5.3-pydantic1.10.11.txt
@@ -1,587 +1,193 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.2.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   distributed
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.2.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   asv-runner
-    #   dask
-    #   jupyter-cache
-    #   keyring
-    #   myst-nb
-    #   twine
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.26.0
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.22.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 myst-parser==4.0.0
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 numpy==1.26.4
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==1.5.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   dask
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 pydantic==1.10.11
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   fastapi
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
-pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
+pyspark==3.5.2
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.14.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via asv
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   sqlalchemy
-    #   typeguard
-    #   typer
-    #   typing-inspect
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpcgdgfpte
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.11-pandas1.5.3-pydantic2.3.0.txt b/ci/requirements-py3.11-pandas1.5.3-pydantic2.3.0.txt
index 525c347b5..7acd63154 100644
--- a/ci/requirements-py3.11-pandas1.5.3-pydantic2.3.0.txt
+++ b/ci/requirements-py3.11-pandas1.5.3-pydantic2.3.0.txt
@@ -1,592 +1,195 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.2.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   distributed
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.2.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   asv-runner
-    #   dask
-    #   jupyter-cache
-    #   keyring
-    #   myst-nb
-    #   twine
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.26.0
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.22.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 myst-parser==4.0.0
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 numpy==1.26.4
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==1.5.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   dask
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 pydantic==2.3.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   fastapi
 pydantic-core==2.6.3
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
-pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
+pyspark==3.5.2
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.14.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via asv
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pydantic-core
-    #   sqlalchemy
-    #   typeguard
-    #   typer
-    #   typing-inspect
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpa1pv_jna
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.11-pandas2.2.2-pydantic1.10.11.txt b/ci/requirements-py3.11-pandas2.2.2-pydantic1.10.11.txt
index 6d07529e2..6e75fbaaf 100644
--- a/ci/requirements-py3.11-pandas2.2.2-pydantic1.10.11.txt
+++ b/ci/requirements-py3.11-pandas2.2.2-pydantic1.10.11.txt
@@ -1,595 +1,195 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.8.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   dask-expr
-    #   distributed
 dask-expr==1.1.10
-    # via dask
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   asv-runner
-    #   dask
-    #   jupyter-cache
-    #   keyring
-    #   myst-nb
-    #   twine
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.26.0
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.31.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 myst-parser==4.0.0
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 numpy==2.0.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==2.2.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   dask
-    #   dask-expr
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   dask-expr
 pydantic==1.10.11
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   fastapi
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
 pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.14.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via asv
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   sqlalchemy
-    #   typeguard
-    #   typer
-    #   typing-inspect
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 tzdata==2024.1
-    # via pandas
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpu85ag4st
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.11-pandas2.2.2-pydantic2.3.0.txt b/ci/requirements-py3.11-pandas2.2.2-pydantic2.3.0.txt
index 2337005ff..2ec41e6d6 100644
--- a/ci/requirements-py3.11-pandas2.2.2-pydantic2.3.0.txt
+++ b/ci/requirements-py3.11-pandas2.2.2-pydantic2.3.0.txt
@@ -1,600 +1,197 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.8.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   dask-expr
-    #   distributed
 dask-expr==1.1.10
-    # via dask
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   asv-runner
-    #   dask
-    #   jupyter-cache
-    #   keyring
-    #   myst-nb
-    #   twine
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.26.0
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.31.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 myst-parser==4.0.0
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 numpy==2.0.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==2.2.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   dask
-    #   dask-expr
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   dask-expr
 pydantic==2.3.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   fastapi
 pydantic-core==2.6.3
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
 pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.14.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via asv
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pydantic-core
-    #   sqlalchemy
-    #   typeguard
-    #   typer
-    #   typing-inspect
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 tzdata==2024.1
-    # via pandas
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpwdokpijq
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.8-pandas1.5.3-pydantic1.10.11.txt b/ci/requirements-py3.8-pandas1.5.3-pydantic1.10.11.txt
index 2c4c71cab..d63b1e63a 100644
--- a/ci/requirements-py3.8-pandas1.5.3-pydantic1.10.11.txt
+++ b/ci/requirements-py3.8-pandas1.5.3-pydantic1.10.11.txt
@@ -1,634 +1,200 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.13
-    # via sphinx
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via
-    #   ipykernel
-    #   ipython
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   fiona
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backcall==0.2.0
-    # via ipython
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   fiona
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   click-plugins
-    #   cligj
-    #   dask
-    #   distributed
-    #   fiona
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 click-plugins==1.1.1
-    # via fiona
 cligj==0.7.2
-    # via fiona
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2023.5.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   distributed
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2023.5.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 docutils==0.19
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 fiona==1.9.6
-    # via geopandas
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2023.3.27
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 geopandas==0.13.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   asv-runner
-    #   build
-    #   dask
-    #   fiona
-    #   jupyter-cache
-    #   jupyter-client
-    #   keyring
-    #   myst-nb
-    #   sphinx
-    #   twine
-    #   typeguard
 importlib-resources==6.4.0
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
-    #   keyring
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.12.3
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==0.6.1
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==2.2.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.3.5
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.22.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==0.17.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 myst-parser==0.18.1
-    # via myst-nb
 nbclient==0.7.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 numpy==1.24.4
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   dask
-    #   modin
-    #   pandas
-    #   pyarrow
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==1.5.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   dask
-    #   geopandas
-    #   modin
 pandas-stubs==2.0.3.230814
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 parso==0.8.4
-    # via jedi
 partd==1.4.1
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pickleshare==0.7.5
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 pkginfo==1.10.0
-    # via twine
 pkgutil-resolve-name==1.3.10
-    # via jsonschema
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 pre-commit==3.5.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 pydantic==1.10.11
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   fastapi
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 pympler==1.1
-    # via asv
 pyproj==3.5.0
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
-pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
+pyspark==3.5.2
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   babel
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 readme-renderer==43.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.10.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   fiona
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==5.3.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 sphinx-design==0.5.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 sphinxcontrib-applehelp==1.0.4
-    # via sphinx
 sphinxcontrib-devhelp==1.0.2
-    # via sphinx
 sphinxcontrib-htmlhelp==2.0.1
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==1.0.3
-    # via sphinx
 sphinxcontrib-serializinghtml==1.1.5
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   myst-parser
-    #   pydantic
-    #   pylint
-    #   rich
-    #   sqlalchemy
-    #   starlette
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmp21wht3lv
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via
-    #   importlib-metadata
-    #   importlib-resources
+zipp==3.20.0
diff --git a/ci/requirements-py3.8-pandas1.5.3-pydantic2.3.0.txt b/ci/requirements-py3.8-pandas1.5.3-pydantic2.3.0.txt
index 185c25930..d3160ffd5 100644
--- a/ci/requirements-py3.8-pandas1.5.3-pydantic2.3.0.txt
+++ b/ci/requirements-py3.8-pandas1.5.3-pydantic2.3.0.txt
@@ -1,640 +1,202 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.13
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via
-    #   ipykernel
-    #   ipython
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   fiona
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backcall==0.2.0
-    # via ipython
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   fiona
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   click-plugins
-    #   cligj
-    #   dask
-    #   distributed
-    #   fiona
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 click-plugins==1.1.1
-    # via fiona
 cligj==0.7.2
-    # via fiona
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2023.5.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   distributed
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2023.5.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 docutils==0.19
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 fiona==1.9.6
-    # via geopandas
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2023.3.27
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 geopandas==0.13.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   asv-runner
-    #   build
-    #   dask
-    #   fiona
-    #   jupyter-cache
-    #   jupyter-client
-    #   keyring
-    #   myst-nb
-    #   sphinx
-    #   twine
-    #   typeguard
 importlib-resources==6.4.0
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
-    #   keyring
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.12.3
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==0.6.1
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==2.2.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.3.5
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.22.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==0.17.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 myst-parser==0.18.1
-    # via myst-nb
 nbclient==0.7.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 numpy==1.24.4
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   dask
-    #   modin
-    #   pandas
-    #   pyarrow
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==1.5.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   dask
-    #   geopandas
-    #   modin
 pandas-stubs==2.0.3.230814
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 parso==0.8.4
-    # via jedi
 partd==1.4.1
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pickleshare==0.7.5
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 pkginfo==1.10.0
-    # via twine
 pkgutil-resolve-name==1.3.10
-    # via jsonschema
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 pre-commit==3.5.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 pydantic==2.3.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   fastapi
 pydantic-core==2.6.3
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 pympler==1.1
-    # via asv
 pyproj==3.5.0
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
-pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
+pyspark==3.5.2
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   babel
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 readme-renderer==43.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.10.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   fiona
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==5.3.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 sphinx-design==0.5.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 sphinxcontrib-applehelp==1.0.4
-    # via sphinx
 sphinxcontrib-devhelp==1.0.2
-    # via sphinx
 sphinxcontrib-htmlhelp==2.0.1
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==1.0.3
-    # via sphinx
 sphinxcontrib-serializinghtml==1.1.5
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   annotated-types
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   myst-parser
-    #   pydantic
-    #   pydantic-core
-    #   pylint
-    #   rich
-    #   sqlalchemy
-    #   starlette
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpki59_lmj
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via
-    #   importlib-metadata
-    #   importlib-resources
+zipp==3.20.0
diff --git a/ci/requirements-py3.9-pandas1.5.3-pydantic1.10.11.txt b/ci/requirements-py3.9-pandas1.5.3-pydantic1.10.11.txt
index ea71b730c..14605ecde 100644
--- a/ci/requirements-py3.9-pandas1.5.3-pydantic1.10.11.txt
+++ b/ci/requirements-py3.9-pandas1.5.3-pydantic1.10.11.txt
@@ -1,612 +1,194 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.2.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   distributed
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.2.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   ipython
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   asv-runner
-    #   build
-    #   dask
-    #   jupyter-cache
-    #   jupyter-client
-    #   keyring
-    #   myst-nb
-    #   sphinx
-    #   twine
-    #   typeguard
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.18.1
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.22.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 myst-parser==3.0.1
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 numpy==1.26.4
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==1.5.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   dask
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 pydantic==1.10.11
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   fastapi
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
-pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
+pyspark==3.5.2
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.13.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
-    #   sphinx
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pylint
-    #   sqlalchemy
-    #   starlette
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpeifj3nvu
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.9-pandas1.5.3-pydantic2.3.0.txt b/ci/requirements-py3.9-pandas1.5.3-pydantic2.3.0.txt
index b9bacf5f1..94a84daad 100644
--- a/ci/requirements-py3.9-pandas1.5.3-pydantic2.3.0.txt
+++ b/ci/requirements-py3.9-pandas1.5.3-pydantic2.3.0.txt
@@ -1,617 +1,196 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.2.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   distributed
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.2.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   ipython
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   asv-runner
-    #   build
-    #   dask
-    #   jupyter-cache
-    #   jupyter-client
-    #   keyring
-    #   myst-nb
-    #   sphinx
-    #   twine
-    #   typeguard
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.18.1
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.22.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 myst-parser==3.0.1
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 numpy==1.26.4
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==1.5.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   dask
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 pydantic==2.3.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   fastapi
 pydantic-core==2.6.3
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
-pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
+pyspark==3.5.2
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.13.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
-    #   sphinx
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pydantic-core
-    #   pylint
-    #   sqlalchemy
-    #   starlette
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmpqccnkfd8
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.9-pandas2.2.2-pydantic1.10.11.txt b/ci/requirements-py3.9-pandas2.2.2-pydantic1.10.11.txt
index 5db04fb5e..ab8ce642c 100644
--- a/ci/requirements-py3.9-pandas2.2.2-pydantic1.10.11.txt
+++ b/ci/requirements-py3.9-pandas2.2.2-pydantic1.10.11.txt
@@ -1,620 +1,196 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.8.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   dask-expr
-    #   distributed
 dask-expr==1.1.10
-    # via dask
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   ipython
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   asv-runner
-    #   build
-    #   dask
-    #   jupyter-cache
-    #   jupyter-client
-    #   keyring
-    #   myst-nb
-    #   sphinx
-    #   twine
-    #   typeguard
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.18.1
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.31.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 myst-parser==3.0.1
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 numpy==2.0.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==2.2.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   dask
-    #   dask-expr
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   dask-expr
 pydantic==1.10.11
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   fastapi
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
 pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.13.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
-    #   sphinx
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pylint
-    #   sqlalchemy
-    #   starlette
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 tzdata==2024.1
-    # via pandas
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmprghjfjjf
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/ci/requirements-py3.9-pandas2.2.2-pydantic2.3.0.txt b/ci/requirements-py3.9-pandas2.2.2-pydantic2.3.0.txt
index b4906944a..bd84392a7 100644
--- a/ci/requirements-py3.9-pandas2.2.2-pydantic2.3.0.txt
+++ b/ci/requirements-py3.9-pandas2.2.2-pydantic2.3.0.txt
@@ -1,625 +1,198 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.8.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   dask-expr
-    #   distributed
 dask-expr==1.1.10
-    # via dask
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   ipython
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 geopandas==1.0.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   asv-runner
-    #   build
-    #   dask
-    #   jupyter-cache
-    #   jupyter-client
-    #   keyring
-    #   myst-nb
-    #   sphinx
-    #   twine
-    #   typeguard
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.18.1
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.31.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 mypy==1.10.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 myst-parser==3.0.1
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 numpy==2.0.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==2.2.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   dask
-    #   dask-expr
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 pre-commit==3.8.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   dask-expr
 pydantic==2.3.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   fastapi
 pydantic-core==2.6.3
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
 pyspark==3.5.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 pytest==8.3.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 pytest-cov==5.0.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 pytest-xdist==3.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.13.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 shapely==2.0.5
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 sphinx-design==0.6.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 sphinx-docsearch==0.0.7
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
-    #   sphinx
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 typeguard==4.3.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 types-requests==2.32.0.20240712
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 types-setuptools==71.1.0.20240806
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 typing-extensions==4.12.2
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pydantic-core
-    #   pylint
-    #   sqlalchemy
-    #   starlette
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 tzdata==2024.1
-    # via pandas
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
-    #   astroid
 xdoctest==1.1.6
-    # via -r /var/folders/wd/sx8dvgys011_mrcsd1_vrz1m0000gn/T/tmptmrbt8dt
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/dev/requirements-3.10.txt b/dev/requirements-3.10.txt
index f32af0e9a..817543d9d 100644
--- a/dev/requirements-3.10.txt
+++ b/dev/requirements-3.10.txt
@@ -1,619 +1,198 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r requirements.in
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r requirements.in
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.8.0
-    # via
-    #   -r requirements.in
-    #   dask-expr
-    #   distributed
 dask-expr==1.1.10
-    # via dask
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.8.0
-    # via -r requirements.in
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   ipython
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r requirements.in
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r requirements.in
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r requirements.in
 geopandas==1.0.1
-    # via -r requirements.in
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r requirements.in
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r requirements.in
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r requirements.in
-    #   asv-runner
-    #   dask
-    #   jupyter-cache
-    #   keyring
-    #   myst-nb
-    #   twine
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.26.0
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r requirements.in
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r requirements.in
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.31.0
-    # via -r requirements.in
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r requirements.in
 mypy==1.10.0
-    # via -r requirements.in
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r requirements.in
 myst-parser==4.0.0
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r requirements.in
 numpy==2.0.1
-    # via
-    #   -r requirements.in
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r requirements.in
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==2.2.2
-    # via
-    #   -r requirements.in
-    #   dask
-    #   dask-expr
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r requirements.in
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r requirements.in
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r requirements.in
 pre-commit==3.8.0
-    # via -r requirements.in
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r requirements.in
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via
-    #   -r requirements.in
-    #   dask-expr
 pydantic==2.8.2
-    # via
-    #   -r requirements.in
-    #   fastapi
 pydantic-core==2.20.1
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r requirements.in
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
 pyspark==3.5.1
-    # via -r requirements.in
 pytest==8.3.2
-    # via
-    #   -r requirements.in
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r requirements.in
 pytest-cov==5.0.0
-    # via -r requirements.in
 pytest-xdist==3.6.1
-    # via -r requirements.in
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r requirements.in
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r requirements.in
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r requirements.in
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r requirements.in
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r requirements.in
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.14.0
-    # via -r requirements.in
 shapely==2.0.5
-    # via
-    #   -r requirements.in
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r requirements.in
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r requirements.in
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r requirements.in
 sphinx-design==0.6.1
-    # via -r requirements.in
 sphinx-docsearch==0.0.7
-    # via -r requirements.in
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
-    #   sphinx
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r requirements.in
 typeguard==4.3.0
-    # via -r requirements.in
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r requirements.in
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r requirements.in
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r requirements.in
 types-requests==2.32.0.20240712
-    # via -r requirements.in
 types-setuptools==71.1.0.20240806
-    # via -r requirements.in
 typing-extensions==4.12.2
-    # via
-    #   -r requirements.in
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pydantic-core
-    #   sqlalchemy
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r requirements.in
 tzdata==2024.1
-    # via pandas
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r requirements.in
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r requirements.in
-    #   astroid
 xdoctest==1.1.6
-    # via -r requirements.in
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/dev/requirements-3.11.txt b/dev/requirements-3.11.txt
index 8370ca410..1c9bbb43f 100644
--- a/dev/requirements-3.11.txt
+++ b/dev/requirements-3.11.txt
@@ -1,600 +1,197 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r requirements.in
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r requirements.in
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.8.0
-    # via
-    #   -r requirements.in
-    #   dask-expr
-    #   distributed
 dask-expr==1.1.10
-    # via dask
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.8.0
-    # via -r requirements.in
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r requirements.in
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r requirements.in
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r requirements.in
 geopandas==1.0.1
-    # via -r requirements.in
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r requirements.in
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r requirements.in
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r requirements.in
-    #   asv-runner
-    #   dask
-    #   jupyter-cache
-    #   keyring
-    #   myst-nb
-    #   twine
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.26.0
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r requirements.in
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r requirements.in
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.31.0
-    # via -r requirements.in
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r requirements.in
 mypy==1.10.0
-    # via -r requirements.in
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r requirements.in
 myst-parser==4.0.0
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r requirements.in
 numpy==2.0.1
-    # via
-    #   -r requirements.in
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r requirements.in
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==2.2.2
-    # via
-    #   -r requirements.in
-    #   dask
-    #   dask-expr
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r requirements.in
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r requirements.in
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r requirements.in
 pre-commit==3.8.0
-    # via -r requirements.in
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r requirements.in
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via
-    #   -r requirements.in
-    #   dask-expr
 pydantic==2.8.2
-    # via
-    #   -r requirements.in
-    #   fastapi
 pydantic-core==2.20.1
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r requirements.in
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
 pyspark==3.5.1
-    # via -r requirements.in
 pytest==8.3.2
-    # via
-    #   -r requirements.in
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r requirements.in
 pytest-cov==5.0.0
-    # via -r requirements.in
 pytest-xdist==3.6.1
-    # via -r requirements.in
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r requirements.in
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r requirements.in
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r requirements.in
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r requirements.in
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r requirements.in
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.14.0
-    # via -r requirements.in
 shapely==2.0.5
-    # via
-    #   -r requirements.in
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r requirements.in
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r requirements.in
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r requirements.in
 sphinx-design==0.6.1
-    # via -r requirements.in
 sphinx-docsearch==0.0.7
-    # via -r requirements.in
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via asv
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r requirements.in
 typeguard==4.3.0
-    # via -r requirements.in
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r requirements.in
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r requirements.in
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r requirements.in
 types-requests==2.32.0.20240712
-    # via -r requirements.in
 types-setuptools==71.1.0.20240806
-    # via -r requirements.in
 typing-extensions==4.12.2
-    # via
-    #   -r requirements.in
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pydantic-core
-    #   sqlalchemy
-    #   typeguard
-    #   typer
-    #   typing-inspect
 typing-inspect==0.9.0
-    # via -r requirements.in
 tzdata==2024.1
-    # via pandas
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r requirements.in
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r requirements.in
-    #   astroid
 xdoctest==1.1.6
-    # via -r requirements.in
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/dev/requirements-3.8.txt b/dev/requirements-3.8.txt
index e28e14429..2fd407a14 100644
--- a/dev/requirements-3.8.txt
+++ b/dev/requirements-3.8.txt
@@ -1,642 +1,203 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.13
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via
-    #   ipykernel
-    #   ipython
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r requirements.in
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   fiona
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backcall==0.2.0
-    # via ipython
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r requirements.in
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   fiona
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   click-plugins
-    #   cligj
-    #   dask
-    #   distributed
-    #   fiona
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 click-plugins==1.1.1
-    # via fiona
 cligj==0.7.2
-    # via fiona
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2023.5.0
-    # via
-    #   -r requirements.in
-    #   distributed
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2023.5.0
-    # via -r requirements.in
 docutils==0.19
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r requirements.in
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 fiona==1.9.6
-    # via geopandas
 frictionless==4.40.8
-    # via -r requirements.in
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2023.3.27
-    # via -r requirements.in
 geopandas==0.13.2
-    # via -r requirements.in
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r requirements.in
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r requirements.in
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r requirements.in
-    #   asv-runner
-    #   build
-    #   dask
-    #   fiona
-    #   jupyter-cache
-    #   jupyter-client
-    #   keyring
-    #   myst-nb
-    #   sphinx
-    #   twine
-    #   typeguard
 importlib-resources==6.4.0
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
-    #   keyring
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.12.3
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r requirements.in
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r requirements.in
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==0.6.1
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==2.2.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.3.5
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.23.1.post0
-    # via -r requirements.in
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r requirements.in
 mypy==1.10.0
-    # via -r requirements.in
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==0.17.2
-    # via -r requirements.in
 myst-parser==0.18.1
-    # via myst-nb
 nbclient==0.7.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r requirements.in
 numpy==1.24.4
-    # via
-    #   -r requirements.in
-    #   dask
-    #   modin
-    #   pandas
-    #   pyarrow
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r requirements.in
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==2.0.3
-    # via
-    #   -r requirements.in
-    #   dask
-    #   geopandas
-    #   modin
 pandas-stubs==2.0.3.230814
-    # via -r requirements.in
 parso==0.8.4
-    # via jedi
 partd==1.4.1
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pickleshare==0.7.5
-    # via ipython
 pip==24.2
-    # via -r requirements.in
 pkginfo==1.10.0
-    # via twine
 pkgutil-resolve-name==1.3.10
-    # via jsonschema
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r requirements.in
 pre-commit==3.5.0
-    # via -r requirements.in
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r requirements.in
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via -r requirements.in
 pydantic==2.8.2
-    # via
-    #   -r requirements.in
-    #   fastapi
 pydantic-core==2.20.1
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r requirements.in
 pympler==1.1
-    # via asv
 pyproj==3.5.0
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
-pyspark==3.5.1
-    # via -r requirements.in
+pyspark==3.5.2
 pytest==8.3.2
-    # via
-    #   -r requirements.in
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r requirements.in
 pytest-cov==5.0.0
-    # via -r requirements.in
 pytest-xdist==3.6.1
-    # via -r requirements.in
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r requirements.in
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r requirements.in
-    #   babel
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r requirements.in
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.10.0
-    # via -r requirements.in
 readme-renderer==43.0
-    # via twine
 recommonmark==0.7.1
-    # via -r requirements.in
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.10.1
-    # via -r requirements.in
 shapely==2.0.5
-    # via
-    #   -r requirements.in
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   fiona
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==5.3.0
-    # via
-    #   -r requirements.in
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r requirements.in
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r requirements.in
 sphinx-design==0.5.0
-    # via -r requirements.in
 sphinx-docsearch==0.0.7
-    # via -r requirements.in
 sphinxcontrib-applehelp==1.0.4
-    # via sphinx
 sphinxcontrib-devhelp==1.0.2
-    # via sphinx
 sphinxcontrib-htmlhelp==2.0.1
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==1.0.3
-    # via sphinx
 sphinxcontrib-serializinghtml==1.1.5
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r requirements.in
 typeguard==4.3.0
-    # via -r requirements.in
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r requirements.in
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r requirements.in
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r requirements.in
 types-requests==2.32.0.20240712
-    # via -r requirements.in
 types-setuptools==71.1.0.20240806
-    # via -r requirements.in
 typing-extensions==4.12.2
-    # via
-    #   -r requirements.in
-    #   annotated-types
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   myst-parser
-    #   pydantic
-    #   pydantic-core
-    #   pylint
-    #   rich
-    #   sqlalchemy
-    #   starlette
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r requirements.in
 tzdata==2024.1
-    # via pandas
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r requirements.in
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r requirements.in
-    #   astroid
 xdoctest==1.1.6
-    # via -r requirements.in
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via
-    #   importlib-metadata
-    #   importlib-resources
+zipp==3.20.0
diff --git a/dev/requirements-3.9.txt b/dev/requirements-3.9.txt
index 2fa0d6dc6..08ace7e87 100644
--- a/dev/requirements-3.9.txt
+++ b/dev/requirements-3.9.txt
@@ -1,625 +1,198 @@
 aiosignal==1.3.1
-    # via ray
 alabaster==0.7.16
-    # via sphinx
 annotated-types==0.7.0
-    # via pydantic
 anyio==4.4.0
-    # via starlette
 appnope==0.1.4
-    # via ipykernel
 argcomplete==3.5.0
-    # via nox
 astroid==2.15.8
-    # via pylint
 asttokens==2.4.1
-    # via stack-data
 asv==0.6.3
-    # via -r requirements.in
 asv-runner==0.2.1
-    # via asv
 attrs==24.2.0
-    # via
-    #   hypothesis
-    #   jsonschema
-    #   jupyter-cache
-    #   referencing
 babel==2.16.0
-    # via sphinx
 backports-tarfile==1.2.0
-    # via jaraco-context
 beautifulsoup4==4.12.3
-    # via furo
 black==24.8.0
-    # via -r requirements.in
 build==1.2.1
-    # via asv
 certifi==2024.7.4
-    # via
-    #   pyogrio
-    #   pyproj
-    #   requests
 cfgv==3.4.0
-    # via pre-commit
 chardet==5.2.0
-    # via frictionless
 charset-normalizer==3.3.2
-    # via requests
 click==8.1.7
-    # via
-    #   black
-    #   dask
-    #   distributed
-    #   jupyter-cache
-    #   ray
-    #   typer
-    #   uvicorn
 cloudpickle==3.0.0
-    # via
-    #   dask
-    #   distributed
 colorlog==6.8.2
-    # via nox
 comm==0.2.2
-    # via ipykernel
 commonmark==0.9.1
-    # via recommonmark
 coverage==7.6.1
-    # via pytest-cov
 dask==2024.8.0
-    # via
-    #   -r requirements.in
-    #   dask-expr
-    #   distributed
 dask-expr==1.1.10
-    # via dask
 debugpy==1.8.5
-    # via ipykernel
 decorator==5.1.1
-    # via ipython
 dill==0.3.8
-    # via pylint
 distlib==0.3.8
-    # via virtualenv
 distributed==2024.8.0
-    # via -r requirements.in
 docutils==0.21.2
-    # via
-    #   myst-parser
-    #   readme-renderer
-    #   recommonmark
-    #   sphinx
 exceptiongroup==1.2.2
-    # via
-    #   anyio
-    #   hypothesis
-    #   ipython
-    #   pytest
 execnet==2.1.1
-    # via pytest-xdist
 executing==2.0.1
-    # via stack-data
 fastapi==0.112.0
-    # via -r requirements.in
 fastjsonschema==2.20.0
-    # via nbformat
 filelock==3.15.4
-    # via
-    #   ray
-    #   virtualenv
 frictionless==4.40.8
-    # via -r requirements.in
 frozenlist==1.4.1
-    # via
-    #   aiosignal
-    #   ray
 fsspec==2024.6.1
-    # via
-    #   dask
-    #   modin
 furo==2024.8.6
-    # via -r requirements.in
 geopandas==1.0.1
-    # via -r requirements.in
+googleapis-common-protos==1.63.2
 grpcio==1.65.4
-    # via -r requirements.in
+grpcio-status==1.65.4
 h11==0.14.0
-    # via uvicorn
-hypothesis==6.110.1
-    # via -r requirements.in
+hypothesis==6.111.0
 identify==2.6.0
-    # via pre-commit
 idna==3.7
-    # via
-    #   anyio
-    #   requests
 imagesize==1.4.1
-    # via sphinx
 importlib-metadata==8.2.0
-    # via
-    #   -r requirements.in
-    #   asv-runner
-    #   build
-    #   dask
-    #   jupyter-cache
-    #   jupyter-client
-    #   keyring
-    #   myst-nb
-    #   sphinx
-    #   twine
-    #   typeguard
 iniconfig==2.0.0
-    # via pytest
 ipykernel==6.29.5
-    # via myst-nb
 ipython==8.18.1
-    # via
-    #   ipykernel
-    #   myst-nb
 isodate==0.6.1
-    # via frictionless
 isort==5.13.2
-    # via
-    #   -r requirements.in
-    #   pylint
 jaraco-classes==3.4.0
-    # via keyring
 jaraco-context==5.3.0
-    # via keyring
 jaraco-functools==4.0.2
-    # via keyring
 jedi==0.19.1
-    # via ipython
 jinja2==3.1.4
-    # via
-    #   distributed
-    #   frictionless
-    #   myst-parser
-    #   sphinx
 joblib==1.4.2
-    # via -r requirements.in
 json5==0.9.25
-    # via asv
 jsonschema==4.23.0
-    # via
-    #   frictionless
-    #   nbformat
-    #   ray
 jsonschema-specifications==2023.12.1
-    # via jsonschema
 jupyter-cache==1.0.0
-    # via myst-nb
 jupyter-client==8.6.2
-    # via
-    #   ipykernel
-    #   nbclient
 jupyter-core==5.7.2
-    # via
-    #   ipykernel
-    #   jupyter-client
-    #   nbclient
-    #   nbformat
 keyring==25.3.0
-    # via twine
 lazy-object-proxy==1.10.0
-    # via astroid
 locket==1.0.0
-    # via
-    #   distributed
-    #   partd
 markdown-it-py==3.0.0
-    # via
-    #   mdit-py-plugins
-    #   myst-parser
-    #   rich
 marko==2.1.2
-    # via frictionless
 markupsafe==2.1.5
-    # via jinja2
 matplotlib-inline==0.1.7
-    # via
-    #   ipykernel
-    #   ipython
 mccabe==0.7.0
-    # via pylint
 mdit-py-plugins==0.4.1
-    # via myst-parser
 mdurl==0.1.2
-    # via markdown-it-py
 modin==0.31.0
-    # via -r requirements.in
 more-itertools==10.4.0
-    # via
-    #   jaraco-classes
-    #   jaraco-functools
 msgpack==1.0.8
-    # via
-    #   distributed
-    #   ray
 multimethod==1.10
-    # via -r requirements.in
 mypy==1.10.0
-    # via -r requirements.in
 mypy-extensions==1.0.0
-    # via
-    #   black
-    #   mypy
-    #   typing-inspect
 myst-nb==1.1.1
-    # via -r requirements.in
 myst-parser==3.0.1
-    # via myst-nb
 nbclient==0.10.0
-    # via
-    #   jupyter-cache
-    #   myst-nb
 nbformat==5.10.4
-    # via
-    #   jupyter-cache
-    #   myst-nb
-    #   nbclient
 nest-asyncio==1.6.0
-    # via ipykernel
 nh3==0.2.18
-    # via readme-renderer
 nodeenv==1.9.1
-    # via pre-commit
 nox==2024.4.15
-    # via -r requirements.in
 numpy==2.0.1
-    # via
-    #   -r requirements.in
-    #   dask
-    #   geopandas
-    #   modin
-    #   pandas
-    #   pandas-stubs
-    #   pyarrow
-    #   pyogrio
-    #   scipy
-    #   shapely
 packaging==24.1
-    # via
-    #   -r requirements.in
-    #   black
-    #   build
-    #   dask
-    #   distributed
-    #   geopandas
-    #   ipykernel
-    #   modin
-    #   nox
-    #   pyogrio
-    #   pytest
-    #   ray
-    #   sphinx
 pandas==2.2.2
-    # via
-    #   -r requirements.in
-    #   dask
-    #   dask-expr
-    #   geopandas
-    #   modin
 pandas-stubs==2.2.2.240807
-    # via -r requirements.in
 parso==0.8.4
-    # via jedi
 partd==1.4.2
-    # via dask
 pathspec==0.12.1
-    # via black
 petl==1.7.15
-    # via frictionless
 pexpect==4.9.0
-    # via ipython
 pip==24.2
-    # via -r requirements.in
 pkginfo==1.10.0
-    # via twine
 platformdirs==4.2.2
-    # via
-    #   black
-    #   jupyter-core
-    #   pylint
-    #   virtualenv
 pluggy==1.5.0
-    # via pytest
 polars==1.4.1
-    # via -r requirements.in
 pre-commit==3.8.0
-    # via -r requirements.in
 prompt-toolkit==3.0.47
-    # via ipython
 protobuf==5.27.3
-    # via
-    #   -r requirements.in
-    #   ray
 psutil==6.0.0
-    # via
-    #   distributed
-    #   ipykernel
-    #   modin
 ptyprocess==0.7.0
-    # via pexpect
 pure-eval==0.2.3
-    # via stack-data
 py4j==0.10.9.7
-    # via pyspark
 pyarrow==17.0.0
-    # via
-    #   -r requirements.in
-    #   dask-expr
 pydantic==2.8.2
-    # via
-    #   -r requirements.in
-    #   fastapi
 pydantic-core==2.20.1
-    # via pydantic
 pygments==2.18.0
-    # via
-    #   furo
-    #   ipython
-    #   readme-renderer
-    #   rich
-    #   sphinx
 pylint==2.17.3
-    # via -r requirements.in
 pympler==1.1
-    # via asv
 pyogrio==0.9.0
-    # via geopandas
 pyproj==3.6.1
-    # via geopandas
 pyproject-hooks==1.1.0
-    # via build
 pyspark==3.5.1
-    # via -r requirements.in
 pytest==8.3.2
-    # via
-    #   -r requirements.in
-    #   pytest-asyncio
-    #   pytest-cov
-    #   pytest-xdist
 pytest-asyncio==0.23.8
-    # via -r requirements.in
 pytest-cov==5.0.0
-    # via -r requirements.in
 pytest-xdist==3.6.1
-    # via -r requirements.in
 python-dateutil==2.9.0.post0
-    # via
-    #   frictionless
-    #   jupyter-client
-    #   pandas
 python-multipart==0.0.9
-    # via -r requirements.in
 python-slugify==8.0.4
-    # via frictionless
 pytz==2024.1
-    # via
-    #   -r requirements.in
-    #   pandas
 pyyaml==6.0.2
-    # via
-    #   -r requirements.in
-    #   asv
-    #   dask
-    #   distributed
-    #   frictionless
-    #   jupyter-cache
-    #   myst-nb
-    #   myst-parser
-    #   pre-commit
-    #   ray
 pyzmq==26.1.0
-    # via
-    #   ipykernel
-    #   jupyter-client
 ray==2.34.0
-    # via -r requirements.in
 readme-renderer==44.0
-    # via twine
 recommonmark==0.7.1
-    # via -r requirements.in
 referencing==0.35.1
-    # via
-    #   jsonschema
-    #   jsonschema-specifications
 requests==2.32.3
-    # via
-    #   frictionless
-    #   ray
-    #   requests-toolbelt
-    #   sphinx
-    #   twine
 requests-toolbelt==1.0.0
-    # via twine
 rfc3986==2.0.0
-    # via
-    #   frictionless
-    #   twine
 rich==13.7.1
-    # via
-    #   twine
-    #   typer
 rpds-py==0.20.0
-    # via
-    #   jsonschema
-    #   referencing
 scipy==1.13.1
-    # via -r requirements.in
 shapely==2.0.5
-    # via
-    #   -r requirements.in
-    #   geopandas
 shellingham==1.5.4
-    # via typer
 simpleeval==0.9.13
-    # via frictionless
 six==1.16.0
-    # via
-    #   asttokens
-    #   isodate
-    #   python-dateutil
 sniffio==1.3.1
-    # via anyio
 snowballstemmer==2.2.0
-    # via sphinx
 sortedcontainers==2.4.0
-    # via
-    #   distributed
-    #   hypothesis
 soupsieve==2.5
-    # via beautifulsoup4
 sphinx==7.3.7
-    # via
-    #   -r requirements.in
-    #   furo
-    #   myst-nb
-    #   myst-parser
-    #   recommonmark
-    #   sphinx-autodoc-typehints
-    #   sphinx-basic-ng
-    #   sphinx-copybutton
-    #   sphinx-design
-    #   sphinx-docsearch
 sphinx-autodoc-typehints==1.14.1
-    # via -r requirements.in
 sphinx-basic-ng==1.0.0b2
-    # via furo
 sphinx-copybutton==0.5.2
-    # via -r requirements.in
 sphinx-design==0.6.1
-    # via -r requirements.in
 sphinx-docsearch==0.0.7
-    # via -r requirements.in
 sphinxcontrib-applehelp==2.0.0
-    # via sphinx
 sphinxcontrib-devhelp==2.0.0
-    # via sphinx
 sphinxcontrib-htmlhelp==2.1.0
-    # via sphinx
 sphinxcontrib-jsmath==1.0.1
-    # via sphinx
 sphinxcontrib-qthelp==2.0.0
-    # via sphinx
 sphinxcontrib-serializinghtml==2.0.0
-    # via sphinx
 sqlalchemy==2.0.32
-    # via jupyter-cache
 stack-data==0.6.3
-    # via ipython
 starlette==0.37.2
-    # via fastapi
 stringcase==1.2.0
-    # via frictionless
 tabulate==0.9.0
-    # via
-    #   asv
-    #   frictionless
-    #   jupyter-cache
 tblib==3.0.0
-    # via distributed
 text-unidecode==1.3
-    # via python-slugify
 tomli==2.0.1
-    # via
-    #   asv
-    #   black
-    #   build
-    #   coverage
-    #   mypy
-    #   nox
-    #   pylint
-    #   pytest
-    #   sphinx
 tomlkit==0.13.0
-    # via pylint
 toolz==0.12.1
-    # via
-    #   dask
-    #   distributed
-    #   partd
 tornado==6.4.1
-    # via
-    #   distributed
-    #   ipykernel
-    #   jupyter-client
 traitlets==5.14.3
-    # via
-    #   comm
-    #   ipykernel
-    #   ipython
-    #   jupyter-client
-    #   jupyter-core
-    #   matplotlib-inline
-    #   nbclient
-    #   nbformat
 twine==5.1.1
-    # via -r requirements.in
 typeguard==4.3.0
-    # via -r requirements.in
 typer==0.12.3
-    # via frictionless
 types-click==7.1.8
-    # via -r requirements.in
 types-pytz==2024.1.0.20240417
-    # via
-    #   -r requirements.in
-    #   pandas-stubs
 types-pyyaml==6.0.12.20240808
-    # via -r requirements.in
 types-requests==2.32.0.20240712
-    # via -r requirements.in
 types-setuptools==71.1.0.20240806
-    # via -r requirements.in
 typing-extensions==4.12.2
-    # via
-    #   -r requirements.in
-    #   anyio
-    #   astroid
-    #   black
-    #   fastapi
-    #   ipython
-    #   mypy
-    #   myst-nb
-    #   pydantic
-    #   pydantic-core
-    #   pylint
-    #   sqlalchemy
-    #   starlette
-    #   typeguard
-    #   typer
-    #   typing-inspect
-    #   uvicorn
 typing-inspect==0.9.0
-    # via -r requirements.in
 tzdata==2024.1
-    # via pandas
 urllib3==2.2.2
-    # via
-    #   distributed
-    #   requests
-    #   twine
-    #   types-requests
 uvicorn==0.30.5
-    # via -r requirements.in
 validators==0.33.0
-    # via frictionless
 virtualenv==20.26.3
-    # via
-    #   asv
-    #   nox
-    #   pre-commit
 wcwidth==0.2.13
-    # via prompt-toolkit
 wrapt==1.16.0
-    # via
-    #   -r requirements.in
-    #   astroid
 xdoctest==1.1.6
-    # via -r requirements.in
 zict==3.0.0
-    # via distributed
-zipp==3.19.2
-    # via importlib-metadata
+zipp==3.20.0
diff --git a/environment.yml b/environment.yml
index b6ce19a8c..611dd9d00 100644
--- a/environment.yml
+++ b/environment.yml
@@ -25,7 +25,7 @@ dependencies:
   - pandas-stubs
 
   # pyspark extra
-  - pyspark >= 3.2.0
+  - pyspark[connect] >= 3.2.0
 
   # polars extra
   - polars >= 0.20.0
diff --git a/noxfile.py b/noxfile.py
index e79a1a677..139a41c53 100644
--- a/noxfile.py
+++ b/noxfile.py
@@ -362,6 +362,7 @@ def ci_requirements(session: Session, pandas: str, pydantic: str) -> None:
             _ci_requirement_file_name(session, pandas, pydantic),
             "--no-header",
             "--upgrade",
+            "--no-annotate",
         )
 
 
@@ -379,6 +380,7 @@ def dev_requirements(session: Session) -> None:
         output_file,
         "--no-header",
         "--upgrade",
+        "--no-annotate",
     )
 
 
diff --git a/pandera/accessors/pyspark_sql_accessor.py b/pandera/accessors/pyspark_sql_accessor.py
index d59dbc996..807d25ee7 100644
--- a/pandera/accessors/pyspark_sql_accessor.py
+++ b/pandera/accessors/pyspark_sql_accessor.py
@@ -3,7 +3,9 @@
 
 import warnings
 from typing import Optional
+from packaging import version
 
+import pyspark
 from pandera.api.base.error_handler import ErrorHandler
 from pandera.api.pyspark.container import DataFrameSchema
 
@@ -104,7 +106,7 @@ def decorator(accessor):
 
 def register_dataframe_accessor(name):
     """
-    Register a custom accessor with a DataFrame
+    Register a custom accessor with a classical Spark DataFrame
 
     :param name: name used when calling the accessor after its registered
     :returns: a class decorator callable.
@@ -115,6 +117,19 @@ def register_dataframe_accessor(name):
     return _register_accessor(name, DataFrame)
 
 
+def register_connect_dataframe_accessor(name):
+    """
+    Register a custom accessor with a Spark Connect DataFrame
+
+    :param name: name used when calling the accessor after its registered
+    :returns: a class decorator callable.
+    """
+    # pylint: disable=import-outside-toplevel
+    from pyspark.sql.connect.dataframe import DataFrame as psc_DataFrame
+
+    return _register_accessor(name, psc_DataFrame)
+
+
 class PanderaDataFrameAccessor(PanderaAccessor):
     """Pandera accessor for pyspark DataFrame."""
 
@@ -127,3 +142,6 @@ def check_schema_type(schema):
 
 
 register_dataframe_accessor("pandera")(PanderaDataFrameAccessor)
+# Handle optional Spark Connect imports for pyspark>=3.4 (if available)
+if version.parse(pyspark.__version__) >= version.parse("3.4"):
+    register_connect_dataframe_accessor("pandera")(PanderaDataFrameAccessor)
diff --git a/pandera/api/pyspark/types.py b/pandera/api/pyspark/types.py
index 6ae1afb99..e4196a778 100644
--- a/pandera/api/pyspark/types.py
+++ b/pandera/api/pyspark/types.py
@@ -2,13 +2,27 @@
 
 from functools import lru_cache
 from typing import List, NamedTuple, Tuple, Type, Union
+from numpy import bool_ as np_bool
+from packaging import version
 
 import pyspark.sql.types as pst
 from pyspark.sql import DataFrame
 
+import pyspark
 from pandera.api.checks import Check
 from pandera.dtypes import DataType
 
+# pylint: disable=reimported
+# Handles optional Spark Connect imports for pyspark>=3.4 (if available)
+if version.parse(pyspark.__version__) >= version.parse("3.4"):
+    from pyspark.sql.connect.dataframe import DataFrame as psc_DataFrame
+else:
+    from pyspark.sql import (
+        DataFrame as psc_DataFrame,
+    )
+
+DataFrameTypes = Union[DataFrame, psc_DataFrame]
+
 CheckList = Union[Check, List[Check]]
 
 PysparkDefaultTypes = Union[
@@ -57,7 +71,7 @@
 class PysparkDataframeColumnObject(NamedTuple):
     """Pyspark Object which holds dataframe and column value in a named tuble"""
 
-    dataframe: DataFrame
+    dataframe: DataFrameTypes
     column_name: str
 
 
@@ -69,6 +83,7 @@ def supported_types() -> SupportedTypes:
 
     try:
         table_types.append(DataFrame)
+        table_types.append(psc_DataFrame)
 
     except ImportError:  # pragma: no cover
         pass
@@ -89,4 +104,4 @@ def is_table(obj):
 
 def is_bool(x):
     """Verifies whether an object is a boolean type."""
-    return isinstance(x, (bool, type(pst.BooleanType())))
+    return isinstance(x, (bool, type(pst.BooleanType()), np_bool))
diff --git a/pandera/backends/pyspark/base.py b/pandera/backends/pyspark/base.py
index 1a31e792d..80f4099d9 100644
--- a/pandera/backends/pyspark/base.py
+++ b/pandera/backends/pyspark/base.py
@@ -22,6 +22,7 @@
     scalar_failure_case,
 )
 from pandera.errors import FailureCaseMetadata, SchemaError, SchemaWarning
+from pandera.api.pyspark.types import DataFrameTypes
 
 
 class ColumnInfo(NamedTuple):
@@ -34,7 +35,7 @@ class ColumnInfo(NamedTuple):
     lazy_exclude_column_names: List
 
 
-FieldCheckObj = Union[col, DataFrame]
+FieldCheckObj = Union[col, DataFrameTypes]
 
 T = TypeVar(
     "T",
@@ -50,7 +51,7 @@ class PysparkSchemaBackend(BaseSchemaBackend):
 
     def subsample(
         self,
-        check_obj: DataFrame,
+        check_obj: DataFrameTypes,
         head: Optional[int] = None,
         tail: Optional[int] = None,
         sample: Optional[float] = None,
diff --git a/pandera/backends/pyspark/checks.py b/pandera/backends/pyspark/checks.py
index 5746e50ed..9b0c5fc79 100644
--- a/pandera/backends/pyspark/checks.py
+++ b/pandera/backends/pyspark/checks.py
@@ -1,10 +1,7 @@
 """Check backend for pyspark."""
 
 from functools import partial
-from typing import Dict, List, Optional
-
-from multimethod import DispatchError, overload
-from pyspark.sql import DataFrame
+from typing import Dict, List, Optional, Union
 
 from pandera.api.base.checks import CheckResult, GroupbyObject
 from pandera.api.checks import Check
@@ -14,6 +11,7 @@
     is_table,
 )
 from pandera.backends.base import BaseCheckBackend
+from pandera.api.pyspark.types import DataFrameTypes
 
 
 class PySparkCheckBackend(BaseCheckBackend):
@@ -26,7 +24,7 @@ def __init__(self, check: Check):
         self.check = check
         self.check_fn = partial(check._check_fn, **check._check_kwargs)
 
-    def groupby(self, check_obj: DataFrame):  # pragma: no cover
+    def groupby(self, check_obj: DataFrameTypes):  # pragma: no cover
         """Implements groupby behavior for check object."""
         assert self.check.groupby is not None, "Check.groupby must be set."
         if isinstance(self.check.groupby, (str, list)):
@@ -45,61 +43,34 @@ def aggregate(self, check_obj):
     def _format_groupby_input(
         groupby_obj: GroupbyObject,
         groups: Optional[List[str]],
-    ) -> Dict[str, DataFrame]:  # pragma: no cover
+    ) -> Dict[str, DataFrameTypes]:  # pragma: no cover
         raise NotImplementedError
 
-    @overload  # type: ignore [no-redef]
     def preprocess(
         self,
-        check_obj: DataFrame,
+        check_obj: DataFrameTypes,
         key: str,  # type: ignore [valid-type]
-    ) -> DataFrame:
+    ) -> DataFrameTypes:
         return check_obj
 
-    # Workaround for multimethod not supporting Optional arguments
-    # such as `key: Optional[str]` (fails in multimethod)
-    # https://github.com/coady/multimethod/issues/90
-    # FIXME when the multimethod supports Optional args # pylint: disable=fixme
-    @overload  # type: ignore [no-redef]
-    def preprocess(
+    def apply(
         self,
-        check_obj: DataFrame,  # type: ignore [valid-type]
-    ) -> DataFrame:
-        return check_obj
-
-    @overload
-    def apply(self, check_obj):
-        """Apply the check function to a check object."""
-        raise NotImplementedError
-
-    @overload  # type: ignore [no-redef]
-    def apply(self, check_obj: DataFrame):
-        return self.check_fn(check_obj)  # pragma: no cover
-
-    @overload  # type: ignore [no-redef]
-    def apply(self, check_obj: is_table):  # type: ignore [valid-type]
-        return self.check_fn(check_obj)  # pragma: no cover
-
-    @overload  # type: ignore [no-redef]
-    def apply(self, check_obj: DataFrame, column_name: str, kwargs: dict):  # type: ignore [valid-type]
-        # kwargs['column_name'] = column_name
-        # return self.check._check_fn(check_obj, *list(kwargs.values()))
-        check_obj_and_col_name = PysparkDataframeColumnObject(
-            check_obj, column_name
-        )
-        return self.check._check_fn(check_obj_and_col_name, **kwargs)
+        check_obj: Union[DataFrameTypes, is_table],
+        column_name: str = None,
+        kwargs: dict = None,
+    ):
+        if column_name and kwargs:
+            check_obj_and_col_name = PysparkDataframeColumnObject(
+                check_obj, column_name
+            )
+            return self.check._check_fn(check_obj_and_col_name, **kwargs)
 
-    @overload
-    def postprocess(self, check_obj, check_output):
-        """Postprocesses the result of applying the check function."""
-        raise TypeError(  # pragma: no cover
-            f"output type of check_fn not recognized: {type(check_output)}"
-        )
+        else:
+            return self.check_fn(check_obj)  # pragma: no cover
 
-    @overload  # type: ignore [no-redef]
     def postprocess(
         self,
-        check_obj,
+        check_obj: DataFrameTypes,
         check_output: is_bool,  # type: ignore [valid-type]
     ) -> CheckResult:
         """Postprocesses the result of applying the check function."""
@@ -112,29 +83,13 @@ def postprocess(
 
     def __call__(
         self,
-        check_obj: DataFrame,
+        check_obj: DataFrameTypes,
         key: Optional[str] = None,
     ) -> CheckResult:
-        if key is None:
-            # pylint:disable=no-value-for-parameter
-            check_obj = self.preprocess(check_obj)
-        else:
-            check_obj = self.preprocess(check_obj, key)
-
-        try:
-            if key is None:
-                check_output = self.apply(check_obj)
-            else:
-                check_output = (
-                    self.apply(  # pylint:disable=too-many-function-args
-                        check_obj, key, self.check._check_kwargs
-                    )
-                )
-
-        except DispatchError as exc:  # pragma: no cover
-            if exc.__cause__ is not None:
-                raise exc.__cause__
-            raise exc
-        except TypeError as err:
-            raise err
+        check_obj = self.preprocess(check_obj, key)
+
+        check_output = self.apply(  # pylint:disable=too-many-function-args
+            check_obj, key, self.check._check_kwargs
+        )
+
         return self.postprocess(check_obj, check_output)
diff --git a/pandera/backends/pyspark/container.py b/pandera/backends/pyspark/container.py
index 8250f02a6..768fb3f09 100644
--- a/pandera/backends/pyspark/container.py
+++ b/pandera/backends/pyspark/container.py
@@ -553,18 +553,6 @@ def unique(
 
         return check_obj
 
-    def _check_uniqueness(
-        self,
-        obj: DataFrame,
-        schema,
-    ) -> DataFrame:
-        """Ensure uniqueness in dataframe columns.
-
-        :param obj: dataframe to check.
-        :param schema: schema object.
-        :returns: dataframe checked.
-        """
-
     ##########
     # Checks #
     ##########
diff --git a/pandera/backends/pyspark/register.py b/pandera/backends/pyspark/register.py
index 4f9d8429d..1a6c04f48 100644
--- a/pandera/backends/pyspark/register.py
+++ b/pandera/backends/pyspark/register.py
@@ -1,8 +1,15 @@
 """Register pyspark backends."""
 
 from functools import lru_cache
+from packaging import version
 
-import pyspark.sql as pst
+import pyspark
+import pyspark.sql as ps
+
+# Handles optional Spark Connect imports for pyspark>=3.4 (if available)
+CURRENT_PYSPARK_VERSION = version.parse(pyspark.__version__)
+if CURRENT_PYSPARK_VERSION >= version.parse("3.4"):
+    from pyspark.sql.connect import dataframe as psc
 
 
 @lru_cache
@@ -28,7 +35,14 @@ def register_pyspark_backends():
     from pandera.backends.pyspark.components import ColumnBackend
     from pandera.backends.pyspark.container import DataFrameSchemaBackend
 
-    Check.register_backend(pst.DataFrame, PySparkCheckBackend)
-    ColumnSchema.register_backend(pst.DataFrame, ColumnSchemaBackend)
-    Column.register_backend(pst.DataFrame, ColumnBackend)
-    DataFrameSchema.register_backend(pst.DataFrame, DataFrameSchemaBackend)
+    # Register classical DataFrame
+    Check.register_backend(ps.DataFrame, PySparkCheckBackend)
+    ColumnSchema.register_backend(ps.DataFrame, ColumnSchemaBackend)
+    Column.register_backend(ps.DataFrame, ColumnBackend)
+    DataFrameSchema.register_backend(ps.DataFrame, DataFrameSchemaBackend)
+    # Register Spark Connect DataFrame, if available
+    if CURRENT_PYSPARK_VERSION >= version.parse("3.4"):
+        Check.register_backend(psc.DataFrame, PySparkCheckBackend)
+        ColumnSchema.register_backend(psc.DataFrame, ColumnSchemaBackend)
+        Column.register_backend(psc.DataFrame, ColumnBackend)
+        DataFrameSchema.register_backend(psc.DataFrame, DataFrameSchemaBackend)
diff --git a/requirements.in b/requirements.in
index 450ba01de..df53991bf 100644
--- a/requirements.in
+++ b/requirements.in
@@ -16,7 +16,7 @@ pyarrow
 pydantic
 multimethod <= 1.10.0
 pandas-stubs
-pyspark >= 3.2.0
+pyspark[connect] >= 3.2.0
 polars >= 0.20.0
 modin
 protobuf
diff --git a/setup.py b/setup.py
index 663f226fa..749915fe6 100644
--- a/setup.py
+++ b/setup.py
@@ -11,7 +11,7 @@
     "strategies": ["hypothesis >= 6.92.7"],
     "hypotheses": ["scipy"],
     "io": ["pyyaml >= 5.1", "black", "frictionless <= 4.40.8"],
-    "pyspark": ["pyspark >= 3.2.0"],
+    "pyspark": ["pyspark[connect] >= 3.2.0"],
     "modin": ["modin", "ray", "dask[dataframe]"],
     "modin-ray": ["modin", "ray"],
     "modin-dask": ["modin", "dask[dataframe]"],


Test patch:
diff --git a/tests/pyspark/conftest.py b/tests/pyspark/conftest.py
index 28f0a1e5a..bda6fe647 100644
--- a/tests/pyspark/conftest.py
+++ b/tests/pyspark/conftest.py
@@ -2,6 +2,7 @@
 
 # pylint:disable=redefined-outer-name
 import datetime
+import os
 
 import pyspark.sql.types as T
 import pytest
@@ -15,7 +16,21 @@ def spark() -> SparkSession:
     """
     creates spark session
     """
-    return SparkSession.builder.getOrCreate()
+    spark: SparkSession = SparkSession.builder.getOrCreate()
+    yield spark
+    spark.stop()
+
+
+@pytest.fixture(scope="session")
+def spark_connect() -> SparkSession:
+    """
+    creates spark connection session
+    """
+    # Set location of localhost Spark Connect server
+    os.environ["SPARK_LOCAL_REMOTE"] = "sc://localhost"
+    spark: SparkSession = SparkSession.builder.getOrCreate()
+    yield spark
+    spark.stop()
 
 
 @pytest.fixture(scope="session")
diff --git a/tests/pyspark/test_pyspark_check.py b/tests/pyspark/test_pyspark_check.py
index c20d231ec..3be0be6ab 100644
--- a/tests/pyspark/test_pyspark_check.py
+++ b/tests/pyspark/test_pyspark_check.py
@@ -32,6 +32,10 @@
 from pandera.pyspark import Column, DataFrameModel, DataFrameSchema, Field
 from pandera.validation_depth import ValidationScope
 
+pytestmark = pytest.mark.parametrize(
+    "spark_session", ["spark", "spark_connect"]
+)
+
 
 @pytest.fixture(scope="function")
 def extra_registered_checks():
@@ -60,10 +64,11 @@ class TestDecorator:
     """This class is used to test the decorator to check datatype mismatches and unacceptable datatype"""
 
     @validate_scope(scope=ValidationScope.DATA)
-    def test_datatype_check_decorator(self, spark):
+    def test_datatype_check_decorator(self, spark_session, request):
         """
         Test to validate the decorator to check datatype mismatches and unacceptable datatype
         """
+        spark = request.getfixturevalue(spark_session)
         schema = DataFrameSchema(
             {
                 "product": Column(StringType()),
@@ -386,8 +391,11 @@ def get_data_param(self):
 
     @validate_scope(scope=ValidationScope.DATA)
     @pytest.mark.parametrize("check_fn", [pa.Check.equal_to, pa.Check.eq])
-    def test_equal_to_check(self, spark, check_fn, datatype, data) -> None:
+    def test_equal_to_check(
+        self, spark_session, check_fn, datatype, data, request
+    ) -> None:
         """Test the Check to see if all the values are equal to defined value"""
+        spark = request.getfixturevalue(spark_session)
         self.check_function(
             spark,
             check_fn,
@@ -400,9 +408,10 @@ def test_equal_to_check(self, spark, check_fn, datatype, data) -> None:
     @validate_scope(scope=ValidationScope.DATA)
     @pytest.mark.parametrize("check_fn", [pa.Check.equal_to, pa.Check.eq])
     def test_failed_unaccepted_datatypes(
-        self, spark, check_fn, datatype, data
+        self, spark_session, check_fn, datatype, data, request
     ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(TypeError):
             self.check_function(
                 spark,
@@ -514,8 +523,11 @@ def get_data_param(self):
 
     @validate_scope(scope=ValidationScope.DATA)
     @pytest.mark.parametrize("check_fn", [pa.Check.not_equal_to, pa.Check.ne])
-    def test_not_equal_to_check(self, spark, check_fn, datatype, data) -> None:
+    def test_not_equal_to_check(
+        self, spark_session, check_fn, datatype, data, request
+    ) -> None:
         """Test the Check to see if all the values are equal to defined value"""
+        spark = request.getfixturevalue(spark_session)
         self.check_function(
             spark,
             check_fn,
@@ -528,9 +540,10 @@ def test_not_equal_to_check(self, spark, check_fn, datatype, data) -> None:
     @validate_scope(scope=ValidationScope.DATA)
     @pytest.mark.parametrize("check_fn", [pa.Check.not_equal_to, pa.Check.ne])
     def test_failed_unaccepted_datatypes(
-        self, spark, check_fn, datatype, data
+        self, spark_session, check_fn, datatype, data, request
     ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(TypeError):
             self.check_function(
                 spark,
@@ -636,8 +649,11 @@ def get_data_param(self):
 
     @validate_scope(scope=ValidationScope.DATA)
     @pytest.mark.parametrize("check_fn", [pa.Check.greater_than, pa.Check.gt])
-    def test_greater_than_check(self, spark, check_fn, datatype, data) -> None:
+    def test_greater_than_check(
+        self, spark_session, check_fn, datatype, data, request
+    ) -> None:
         """Test the Check to see if all the values are equal to defined value"""
+        spark = request.getfixturevalue(spark_session)
         self.check_function(
             spark,
             check_fn,
@@ -650,9 +666,10 @@ def test_greater_than_check(self, spark, check_fn, datatype, data) -> None:
     @validate_scope(scope=ValidationScope.DATA)
     @pytest.mark.parametrize("check_fn", [pa.Check.greater_than, pa.Check.gt])
     def test_failed_unaccepted_datatypes(
-        self, spark, check_fn, datatype, data
+        self, spark_session, check_fn, datatype, data, request
     ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(TypeError):
             self.check_function(
                 spark,
@@ -761,9 +778,10 @@ def get_data_param(self):
         "check_fn", [pa.Check.greater_than_or_equal_to, pa.Check.ge]
     )
     def test_greater_than_or_equal_to_check(
-        self, spark, check_fn, datatype, data
+        self, spark_session, check_fn, datatype, data, request
     ) -> None:
         """Test the Check to see if all the values are equal to defined value"""
+        spark = request.getfixturevalue(spark_session)
         self.check_function(
             spark,
             check_fn,
@@ -778,9 +796,10 @@ def test_greater_than_or_equal_to_check(
         "check_fn", [pa.Check.greater_than_or_equal_to, pa.Check.ge]
     )
     def test_failed_unaccepted_datatypes(
-        self, spark, check_fn, datatype, data
+        self, spark_session, check_fn, datatype, data, request
     ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(TypeError):
             self.check_function(
                 spark,
@@ -898,8 +917,11 @@ def get_data_param(self):
 
     @validate_scope(scope=ValidationScope.DATA)
     @pytest.mark.parametrize("check_fn", [pa.Check.less_than, pa.Check.lt])
-    def test_less_than_check(self, spark, check_fn, datatype, data) -> None:
+    def test_less_than_check(
+        self, spark_session, check_fn, datatype, data, request
+    ) -> None:
         """Test the Check to see if all the values are equal to defined value"""
+        spark = request.getfixturevalue(spark_session)
         self.check_function(
             spark,
             check_fn,
@@ -912,9 +934,10 @@ def test_less_than_check(self, spark, check_fn, datatype, data) -> None:
     @validate_scope(scope=ValidationScope.DATA)
     @pytest.mark.parametrize("check_fn", [pa.Check.less_than, pa.Check.lt])
     def test_failed_unaccepted_datatypes(
-        self, spark, check_fn, datatype, data
+        self, spark_session, check_fn, datatype, data, request
     ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(TypeError):
             self.check_function(
                 spark,
@@ -928,9 +951,10 @@ def test_failed_unaccepted_datatypes(
     @validate_scope(scope=ValidationScope.DATA)
     @pytest.mark.parametrize("check_fn", [pa.Check.less_than, pa.Check.lt])
     def test_failed_none_expression(
-        self, spark, check_fn, datatype, data
+        self, spark_session, check_fn, datatype, data, request
     ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(ValueError):
             self.check_function(
                 spark,
@@ -1051,9 +1075,10 @@ def get_data_param(self):
         "check_fn", [pa.Check.less_than_or_equal_to, pa.Check.le]
     )
     def test_less_than_or_equal_to_check(
-        self, spark, check_fn, datatype, data
+        self, spark_session, check_fn, datatype, data, request
     ) -> None:
         """Test the Check to see if all the values are equal to defined value"""
+        spark = request.getfixturevalue(spark_session)
         self.check_function(
             spark,
             check_fn,
@@ -1068,9 +1093,10 @@ def test_less_than_or_equal_to_check(
         "check_fn", [pa.Check.less_than_or_equal_to, pa.Check.le]
     )
     def test_failed_unaccepted_datatypes(
-        self, spark, check_fn, datatype, data
+        self, spark_session, check_fn, datatype, data, request
     ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(TypeError):
             self.check_function(
                 spark,
@@ -1086,9 +1112,10 @@ def test_failed_unaccepted_datatypes(
         "check_fn", [pa.Check.less_than_or_equal_to, pa.Check.le]
     )
     def test_failed_none_expression(
-        self, spark, check_fn, datatype, data
+        self, spark_session, check_fn, datatype, data, request
     ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(ValueError):
             self.check_function(
                 spark,
@@ -1209,8 +1236,9 @@ def get_data_param(self):
         }
 
     @validate_scope(scope=ValidationScope.DATA)
-    def test_isin_check(self, spark, datatype, data) -> None:
+    def test_isin_check(self, spark_session, datatype, data, request) -> None:
         """Test the Check to see if all the values are is in the defined value"""
+        spark = request.getfixturevalue(spark_session)
         self.check_function(
             spark,
             pa.Check.isin,
@@ -1221,8 +1249,11 @@ def test_isin_check(self, spark, datatype, data) -> None:
         )
 
     @validate_scope(scope=ValidationScope.DATA)
-    def test_failed_unaccepted_datatypes(self, spark, datatype, data) -> None:
+    def test_failed_unaccepted_datatypes(
+        self, spark_session, datatype, data, request
+    ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(TypeError):
             self.check_function(
                 spark,
@@ -1335,9 +1366,9 @@ def get_data_param(self):
         }
 
     @validate_scope(scope=ValidationScope.DATA)
-    def test_notin_check(self, spark, datatype, data) -> None:
+    def test_notin_check(self, spark_session, datatype, data, request) -> None:
         """Test the Check to see if all the values are equal to defined value"""
-
+        spark = request.getfixturevalue(spark_session)
         self.check_function(
             spark,
             pa.Check.notin,
@@ -1348,8 +1379,11 @@ def test_notin_check(self, spark, datatype, data) -> None:
         )
 
     @validate_scope(scope=ValidationScope.DATA)
-    def test_failed_unaccepted_datatypes(self, spark, datatype, data) -> None:
+    def test_failed_unaccepted_datatypes(
+        self, spark_session, datatype, data, request
+    ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(TypeError):
             self.check_function(
                 spark,
@@ -1365,8 +1399,9 @@ class TestStringType(BaseClass):
     """This class is used to test the string types checks"""
 
     @validate_scope(scope=ValidationScope.DATA)
-    def test_str_startswith_check(self, spark) -> None:
+    def test_str_startswith_check(self, spark_session, request) -> None:
         """Test the Check to see if any value is not in the specified value"""
+        spark = request.getfixturevalue(spark_session)
         check_func = pa.Check.str_startswith
         check_value = "B"
 
@@ -1377,8 +1412,9 @@ def test_str_startswith_check(self, spark) -> None:
         )
 
     @validate_scope(scope=ValidationScope.DATA)
-    def test_str_endswith_check(self, spark) -> None:
+    def test_str_endswith_check(self, spark_session, request) -> None:
         """Test the Check to see if any value is not in the specified value"""
+        spark = request.getfixturevalue(spark_session)
         check_func = pa.Check.str_endswith
         check_value = "d"
 
@@ -1389,8 +1425,9 @@ def test_str_endswith_check(self, spark) -> None:
         )
 
     @validate_scope(scope=ValidationScope.DATA)
-    def test_str_contains_check(self, spark) -> None:
+    def test_str_contains_check(self, spark_session, request) -> None:
         """Test the Check to see if any value is not in the specified value"""
+        spark = request.getfixturevalue(spark_session)
         check_func = pa.Check.str_contains
         check_value = "Ba"
 
@@ -1507,9 +1544,10 @@ def get_data_param(self):
 
     @validate_scope(scope=ValidationScope.DATA)
     def test_inrange_exclude_min_max_check(
-        self, spark, datatype, data
+        self, spark_session, datatype, data, request
     ) -> None:
         """Test the Check to see if any value is not in the specified value"""
+        spark = request.getfixturevalue(spark_session)
         min_val, max_val, add_value = self.create_min_max(data)
         self.check_function(
             spark,
@@ -1522,9 +1560,10 @@ def test_inrange_exclude_min_max_check(
 
     @validate_scope(scope=ValidationScope.DATA)
     def test_inrange_exclude_min_only_check(
-        self, spark, datatype, data
+        self, spark_session, datatype, data, request
     ) -> None:
         """Test the Check to see if any value is not in the specified value"""
+        spark = request.getfixturevalue(spark_session)
         min_val, max_val, add_value = self.create_min_max(data)
         self.check_function(
             spark,
@@ -1537,9 +1576,10 @@ def test_inrange_exclude_min_only_check(
 
     @validate_scope(scope=ValidationScope.DATA)
     def test_inrange_exclude_max_only_check(
-        self, spark, datatype, data
+        self, spark_session, datatype, data, request
     ) -> None:
         """Test the Check to see if any value is not in the specified value"""
+        spark = request.getfixturevalue(spark_session)
         min_val, max_val, add_value = self.create_min_max(data)
         self.check_function(
             spark,
@@ -1552,9 +1592,10 @@ def test_inrange_exclude_max_only_check(
 
     @validate_scope(scope=ValidationScope.DATA)
     def test_inrange_include_min_max_check(
-        self, spark, datatype, data
+        self, spark_session, datatype, data, request
     ) -> None:
         """Test the Check to see if any value is not in the specified value"""
+        spark = request.getfixturevalue(spark_session)
         (
             min_val,
             max_val,
@@ -1570,8 +1611,11 @@ def test_inrange_include_min_max_check(
         )
 
     @validate_scope(scope=ValidationScope.DATA)
-    def test_failed_unaccepted_datatypes(self, spark, datatype, data) -> None:
+    def test_failed_unaccepted_datatypes(
+        self, spark_session, datatype, data, request
+    ) -> None:
         """Test the Check to see if error is raised for datatypes which are not accepted for this function"""
+        spark = request.getfixturevalue(spark_session)
         with pytest.raises(TypeError):
             self.check_function(
                 spark,
@@ -1599,7 +1643,6 @@ def _check_extension(
         """
         This function does performs the actual validation
         """
-
         spark_schema = StructType(
             [
                 StructField("product", StringType(), False),
@@ -1621,9 +1664,10 @@ def _check_extension(
                 raise PysparkSchemaError
 
     def test_extension(
-        self, spark, extra_registered_checks
+        self, spark_session, extra_registered_checks, request
     ):  # pylint: disable=unused-argument
         """Test custom extension with DataFrameSchema way of defining schema"""
+        spark = request.getfixturevalue(spark_session)
         schema = DataFrameSchema(
             {
                 "product": Column(StringType()),
@@ -1644,9 +1688,10 @@ def test_extension(
         )
 
     def test_extension_pydantic(
-        self, spark, extra_registered_checks
+        self, spark_session, extra_registered_checks, request
     ):  # pylint: disable=unused-argument
         """Test custom extension with DataFrameModel way of defining schema"""
+        spark = request.getfixturevalue(spark_session)
 
         class Schema(DataFrameModel):
             """Test Schema"""
diff --git a/tests/pyspark/test_pyspark_config.py b/tests/pyspark/test_pyspark_config.py
index d4a4de023..ffb7a2119 100644
--- a/tests/pyspark/test_pyspark_config.py
+++ b/tests/pyspark/test_pyspark_config.py
@@ -15,14 +15,21 @@
 )
 from tests.pyspark.conftest import spark_df
 
+pytestmark = pytest.mark.parametrize(
+    "spark_session", ["spark", "spark_connect"]
+)
+
 
 class TestPanderaConfig:
     """Class to test all the different configs types"""
 
     sample_data = [("Bread", 9), ("Cutter", 15)]
 
-    def test_disable_validation(self, spark, sample_spark_schema):
+    def test_disable_validation(
+        self, spark_session, sample_spark_schema, request
+    ):
         """This function validates that a none object is loaded if validation is disabled"""
+        spark = request.getfixturevalue(spark_session)
         pandera_schema = DataFrameSchema(
             {
                 "product": Column(T.StringType(), Check.str_startswith("B")),
@@ -50,9 +57,9 @@ class TestSchema(DataFrameModel):
             assert TestSchema.validate(input_df) == input_df
 
     # pylint:disable=too-many-locals
-    def test_schema_only(self, spark, sample_spark_schema):
+    def test_schema_only(self, spark_session, sample_spark_schema, request):
         """This function validates that only schema related checks are run not data level"""
-
+        spark = request.getfixturevalue(spark_session)
         pandera_schema = DataFrameSchema(
             {
                 "product": Column(T.StringType(), Check.str_startswith("B")),
@@ -138,9 +145,9 @@ class TestSchema(DataFrameModel):
         )
 
     # pylint:disable=too-many-locals
-    def test_data_only(self, spark, sample_spark_schema):
+    def test_data_only(self, spark_session, sample_spark_schema, request):
         """This function validates that only data related checks are run not schema level"""
-
+        spark = request.getfixturevalue(spark_session)
         pandera_schema = DataFrameSchema(
             {
                 "product": Column(T.StringType(), Check.str_startswith("B")),
@@ -231,8 +238,11 @@ class TestSchema(DataFrameModel):
         )
 
     # pylint:disable=too-many-locals
-    def test_schema_and_data(self, spark, sample_spark_schema):
+    def test_schema_and_data(
+        self, spark_session, sample_spark_schema, request
+    ):
         """This function validates that both data and schema level checks are validated"""
+        spark = request.getfixturevalue(spark_session)
         # self.remove_python_module_cache()
         pandera_schema = DataFrameSchema(
             {
@@ -363,6 +373,7 @@ def test_cache_dataframe_settings(
         self,
         cache_dataframe,
         keep_cached_dataframe,
+        spark_session,  # pylint:disable=unused-argument
     ):
         """This function validates setters and getters for cache/keep_cache options."""
         # Set expected properties in Config object
diff --git a/tests/pyspark/test_pyspark_container.py b/tests/pyspark/test_pyspark_container.py
index 84ff179c0..ad448349e 100644
--- a/tests/pyspark/test_pyspark_container.py
+++ b/tests/pyspark/test_pyspark_container.py
@@ -7,21 +7,23 @@
 
 import pyspark.sql.types as T
 import pytest
-from pyspark.sql import DataFrame, Row, SparkSession
+from pyspark.sql import DataFrame, Row
 
 import pandera.errors
 import pandera.pyspark as pa
 from pandera.config import PanderaConfig, ValidationDepth
 from pandera.pyspark import Column, DataFrameModel, DataFrameSchema
 
-spark = SparkSession.builder.getOrCreate()
+pytestmark = pytest.mark.parametrize(
+    "spark_session", ["spark", "spark_connect"]
+)
 
 
-def test_pyspark_dataframeschema():
+def test_pyspark_dataframeschema(spark_session, request):
     """
     Test creating a pyspark DataFrameSchema object
     """
-
+    spark = request.getfixturevalue(spark_session)
     schema = DataFrameSchema(
         {
             "name": Column(T.StringType()),
@@ -47,11 +49,13 @@ def test_pyspark_dataframeschema():
 
 def test_pyspark_dataframeschema_with_alias_types(
     config_params: PanderaConfig,
+    spark_session,
+    request,
 ):
     """
     Test creating a pyspark DataFrameSchema object
     """
-
+    spark = request.getfixturevalue(spark_session)
     schema = DataFrameSchema(
         columns={
             "product": Column("str", checks=pa.Check.str_startswith("B")),
@@ -92,7 +96,9 @@ def test_pyspark_dataframeschema_with_alias_types(
                 raise pandera.errors.PysparkSchemaError
 
 
-def test_pyspark_column_metadata():
+def test_pyspark_column_metadata(
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Test creating a pyspark Column object with metadata
     """
@@ -128,11 +134,11 @@ def test_pyspark_column_metadata():
     assert schema.get_metadata() == expected
 
 
-def test_pyspark_sample():
+def test_pyspark_sample(spark_session, request):
     """
     Test the sample functionality of pyspark
     """
-
+    spark = request.getfixturevalue(spark_session)
     schema = DataFrameSchema(
         columns={
             "product": Column("str", checks=pa.Check.str_startswith("B")),
@@ -165,11 +171,11 @@ def test_pyspark_sample():
     assert isinstance(df_out, DataFrame)
 
 
-def test_pyspark_regex_column():
+def test_pyspark_regex_column(spark_session, request):
     """
     Test creating a pyspark DataFrameSchema object with regex columns
     """
-
+    spark = request.getfixturevalue(spark_session)
     schema = DataFrameSchema(
         {
             # Columns with all caps names must have string values
@@ -193,11 +199,11 @@ def test_pyspark_regex_column():
     assert not df_out.pandera.errors
 
 
-def test_pyspark_nullable():
+def test_pyspark_nullable(spark_session, request):
     """
     Test the nullable functionality of pyspark
     """
-
+    spark = request.getfixturevalue(spark_session)
     data = [
         ("Bread", 9),
         ("Butter", 15),
@@ -238,10 +244,11 @@ def test_pyspark_nullable():
     assert df_out.pandera.errors == {}
 
 
-def test_pyspark_unique_field():
+def test_pyspark_unique_field(spark_session, request):
     """
     Test that field unique True raise an error.
     """
+    spark = request.getfixturevalue(spark_session)
     with pytest.raises(pandera.errors.SchemaInitError):
         # pylint: disable=W0223
         class PanderaSchema(DataFrameModel):
@@ -260,10 +267,11 @@ class PanderaSchema(DataFrameModel):
         assert len(df_out.pandera.errors) == 0
 
 
-def test_pyspark_unique_config():
+def test_pyspark_unique_config(spark_session, request):
     """
     Test the sample functionality of pyspark
     """
+    spark = request.getfixturevalue(spark_session)
 
     # pylint: disable=W0223
     class PanderaSchema(DataFrameModel):
@@ -330,7 +338,10 @@ def schema_with_complex_datatypes():
     return schema
 
 
-def test_schema_to_structtype(schema_with_complex_datatypes):
+def test_schema_to_structtype(
+    schema_with_complex_datatypes,
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Test the conversion from a schema to a StructType object through `to_structtype()`.
     """
@@ -441,7 +452,10 @@ def test_schema_to_structtype(schema_with_complex_datatypes):
     )
 
 
-def test_schema_to_ddl(schema_with_complex_datatypes):
+def test_schema_to_ddl(
+    schema_with_complex_datatypes,
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Test the conversion from a schema to a DDL string through `to_ddl()`.
     """
@@ -488,8 +502,10 @@ def test_schema_to_ddl(schema_with_complex_datatypes):
     )
 
 
-@pytest.fixture(scope="module")
-def schema_with_simple_datatypes():
+@pytest.fixture(scope="function")
+def schema_with_simple_datatypes(
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Model containing all common datatypes for PySpark namespace, supported by CSV.
     """
@@ -517,11 +533,13 @@ def schema_with_simple_datatypes():
     platform.system() == "Windows",
     reason="skipping due to issues with opening file names for temp files.",
 )
-def test_pyspark_read(schema_with_simple_datatypes, tmp_path, spark):
+def test_pyspark_read(
+    schema_with_simple_datatypes, tmp_path, spark_session, request
+):
     """
     Test reading a file using an automatically generated schema object.
     """
-
+    spark = request.getfixturevalue(spark_session)
     original_pyspark_schema = T.StructType(
         [
             T.StructField(
diff --git a/tests/pyspark/test_pyspark_decorators.py b/tests/pyspark/test_pyspark_decorators.py
index 48661ca05..98c2afc46 100644
--- a/tests/pyspark/test_pyspark_decorators.py
+++ b/tests/pyspark/test_pyspark_decorators.py
@@ -14,15 +14,22 @@
 from pandera.pyspark import Check, Column, DataFrameSchema
 from tests.pyspark.conftest import spark_df
 
+pytestmark = pytest.mark.parametrize(
+    "spark_session", ["spark", "spark_connect"]
+)
+
 
 class TestPanderaDecorators:
     """Class to test all the different configs types"""
 
     sample_data = [("Bread", 9), ("Cutter", 15)]
 
-    def test_cache_dataframe_requirements(self, spark, sample_spark_schema):
+    def test_cache_dataframe_requirements(
+        self, spark_session, sample_spark_schema, request
+    ):
         """Validates if decorator can only be applied in a proper function."""
         # Set expected properties in Config object
+        spark = request.getfixturevalue(spark_session)
         input_df = spark_df(spark, self.sample_data, sample_spark_schema)
 
         class FakeDataFrameSchemaBackend:
@@ -74,17 +81,19 @@ def func_wo_check_obj(self, message: str):
     # pylint:disable=too-many-locals
     def test_cache_dataframe_settings(
         self,
-        spark,
+        spark_session,
         sample_spark_schema,
         cache_enabled,
         keep_cache_enabled,
         expected_caching_message,
         expected_unpersisting_message,
         caplog,
+        request,
     ):
         """This function validates that caching/unpersisting works as expected."""
         # Set expected properties in Config object
         # Prepare test data
+        spark = request.getfixturevalue(spark_session)
         input_df = spark_df(spark, self.sample_data, sample_spark_schema)
         pandera_schema = DataFrameSchema(
             {
diff --git a/tests/pyspark/test_pyspark_dtypes.py b/tests/pyspark/test_pyspark_dtypes.py
index 1b17f6024..9c2ba5504 100644
--- a/tests/pyspark/test_pyspark_dtypes.py
+++ b/tests/pyspark/test_pyspark_dtypes.py
@@ -2,6 +2,7 @@
 
 from typing import Any
 
+import pytest
 import pyspark
 import pyspark.sql.types as T
 from pyspark.sql import DataFrame
@@ -12,6 +13,10 @@
 from pandera.validation_depth import ValidationScope
 from tests.pyspark.conftest import spark_df
 
+pytestmark = pytest.mark.parametrize(
+    "spark_session", ["spark", "spark_connect"]
+)
+
 
 class BaseClass:
     """Base class for all the dtypes"""
@@ -134,33 +139,36 @@ def create_schema(self, column_name, datatype):
         return spark_schema
 
     def test_pyspark_all_float_types(
-        self, spark, sample_data, pandera_equivalent
+        self, spark_session, sample_data, pandera_equivalent, request
     ):
         """
         Test float dtype column
         """
+        spark = request.getfixturevalue(spark_session)
         column_name = "price"
         spark_schema = self.create_schema(column_name, T.FloatType())
         df = spark_df(spark, sample_data, spark_schema)
         self.validate_data(df, pandera_equivalent, column_name)
 
     def test_pyspark_all_double_types(
-        self, spark, sample_data, pandera_equivalent
+        self, spark_session, sample_data, pandera_equivalent, request
     ):
         """
         Test double dtype column
         """
+        spark = request.getfixturevalue(spark_session)
         column_name = "price"
         spark_schema = self.create_schema(column_name, T.DoubleType())
         df = spark_df(spark, sample_data, spark_schema)
         self.validate_data(df, pandera_equivalent, column_name)
 
     def test_pyspark_decimal_default_types(
-        self, spark, sample_data, pandera_equivalent
+        self, spark_session, sample_data, pandera_equivalent, request
     ):
         """
         Test decimal dtype column with default values
         """
+        spark = request.getfixturevalue(spark_session)
         column_name = "price"
         spark_schema = self.create_schema(column_name, T.DecimalType())
         df = spark_df(spark, sample_data, spark_schema)
@@ -168,11 +176,12 @@ def test_pyspark_decimal_default_types(
 
     @validate_scope(scope=ValidationScope.SCHEMA)
     def test_pyspark_decimal_parameterized_types(
-        self, spark, sample_data, pandera_equivalent
+        self, spark_session, sample_data, pandera_equivalent, request
     ):
         """
         Test decimal dtype column with parameterized inputs
         """
+        spark = request.getfixturevalue(spark_session)
         column_name = "price"
         spark_schema = self.create_schema(column_name, T.DecimalType(20, 5))
         df = spark_df(spark, sample_data, spark_schema)
@@ -195,44 +204,48 @@ def test_pyspark_decimal_parameterized_types(
         }
 
     def test_pyspark_all_int_types(
-        self, spark, sample_data, pandera_equivalent
+        self, spark_session, sample_data, pandera_equivalent, request
     ):
         """
         Test int dtype column
         """
+        spark = request.getfixturevalue(spark_session)
         column_name = "price"
         spark_schema = self.create_schema(column_name, T.IntegerType())
         df = spark_df(spark, sample_data, spark_schema)
         self.validate_data(df, pandera_equivalent, column_name)
 
     def test_pyspark_all_longint_types(
-        self, spark, sample_data, pandera_equivalent
+        self, spark_session, sample_data, pandera_equivalent, request
     ):
         """
         Test long dtype column
         """
+        spark = request.getfixturevalue(spark_session)
         column_name = "price"
         spark_schema = self.create_schema(column_name, T.LongType())
         df = spark_df(spark, sample_data, spark_schema)
         self.validate_data(df, pandera_equivalent, column_name)
 
     def test_pyspark_all_shortint_types(
-        self, spark, sample_data, pandera_equivalent
+        self, spark_session, sample_data, pandera_equivalent, request
     ):
         """
         Test short int dtype column
         """
+        spark = request.getfixturevalue(spark_session)
         column_name = "price"
         spark_schema = self.create_schema(column_name, T.ShortType())
         df = spark_df(spark, sample_data, spark_schema)
         self.validate_data(df, pandera_equivalent, column_name)
 
     def test_pyspark_all_bytetint_types(
-        self, spark, sample_data, pandera_equivalent
+        self, spark_session, sample_data, pandera_equivalent, request
     ):
         """
         Test byte int dtype column
         """
+        spark = request.getfixturevalue(spark_session)
         column_name = "price"
         spark_schema = self.create_schema(column_name, T.ByteType())
         df = spark_df(spark, sample_data, spark_schema)
@@ -273,7 +286,10 @@ class TestAllDatetimeTestClass(BaseClass):
     }
 
     def test_pyspark_all_date_types(
-        self, pandera_equivalent, sample_date_object
+        self,
+        pandera_equivalent,
+        sample_date_object,
+        spark_session,  # pylint:disable=unused-argument
     ):
         """
         Test date dtype column
@@ -283,7 +299,10 @@ def test_pyspark_all_date_types(
         self.validate_data(df, pandera_equivalent, column_name)
 
     def test_pyspark_all_datetime_types(
-        self, pandera_equivalent, sample_date_object
+        self,
+        pandera_equivalent,
+        sample_date_object,
+        spark_session,  # pylint:disable=unused-argument
     ):
         """
         Test datetime dtype column
@@ -314,7 +333,10 @@ class TestBinaryStringTypes(BaseClass):
     }
 
     def test_pyspark_all_binary_types(
-        self, pandera_equivalent, sample_string_binary_object
+        self,
+        pandera_equivalent,
+        sample_string_binary_object,
+        spark_session,  # pylint:disable=unused-argument
     ):
         """
         Test binary dytpe column
@@ -324,7 +346,10 @@ def test_pyspark_all_binary_types(
         self.validate_data(df, pandera_equivalent, column_name)
 
     def test_pyspark_all_string_types(
-        self, pandera_equivalent, sample_string_binary_object
+        self,
+        pandera_equivalent,
+        sample_string_binary_object,
+        spark_session,  # pylint:disable=unused-argument
     ):
         """
         Test string dtype column
@@ -361,7 +386,12 @@ class TestComplexType(BaseClass):
     }
 
     @validate_scope(scope=ValidationScope.SCHEMA)
-    def test_pyspark_array_type(self, sample_complex_data, pandera_equivalent):
+    def test_pyspark_array_type(
+        self,
+        sample_complex_data,
+        pandera_equivalent,
+        spark_session,  # pylint:disable=unused-argument
+    ):
         """
         Test array dtype column
         """
@@ -383,7 +413,12 @@ def test_pyspark_array_type(self, sample_complex_data, pandera_equivalent):
         }
 
     @validate_scope(scope=ValidationScope.SCHEMA)
-    def test_pyspark_map_type(self, sample_complex_data, pandera_equivalent):
+    def test_pyspark_map_type(
+        self,
+        sample_complex_data,
+        pandera_equivalent,
+        spark_session,  # pylint:disable=unused-argument
+    ):
         """
         Test map dtype column
         """
diff --git a/tests/pyspark/test_pyspark_error.py b/tests/pyspark/test_pyspark_error.py
index b8fb5d176..b937e67c0 100644
--- a/tests/pyspark/test_pyspark_error.py
+++ b/tests/pyspark/test_pyspark_error.py
@@ -2,12 +2,8 @@
 
 # pylint:disable=redefined-outer-name,abstract-method
 
-from typing import Union
-
 import pyspark.sql.types as T
 import pytest
-from pyspark.sql import DataFrame, SparkSession
-from pyspark.sql.functions import col
 from pyspark.sql.types import StringType
 
 import pandera.pyspark as pa
@@ -16,38 +12,34 @@
 from pandera.pyspark import Column, DataFrameModel, DataFrameSchema, Field
 from tests.pyspark.conftest import spark_df
 
-spark = SparkSession.builder.getOrCreate()
+pytestmark = pytest.mark.parametrize(
+    "spark_session", ["spark", "spark_connect"]
+)
 
 
-@pytest.mark.parametrize(
-    "schema, invalid_data",
-    [
-        [
-            pa.DataFrameSchema(
-                {
-                    "product": pa.Column(StringType()),
-                    "code": pa.Column(StringType(), pa.Check.not_equal_to(30)),
-                }
-            ),
-            spark.createDataFrame(
-                data=[("23", 31), ("34", 35)], schema=["product", "code"]
-            ),
-        ],
-    ],
-)
 def test_dataframe_add_schema(
-    schema: pa.DataFrameSchema,
-    invalid_data: Union[DataFrame, col],
+    spark_session,
+    request,
 ) -> None:
     """
     Test that pyspark object contains schema metadata after pandera validation.
     """
+    spark = request.getfixturevalue(spark_session)
+    schema = pa.DataFrameSchema(
+        {
+            "product": pa.Column(StringType()),
+            "code": pa.Column(StringType(), pa.Check.not_equal_to(30)),
+        }
+    )
+    invalid_data = spark.createDataFrame(
+        data=[("23", 31), ("34", 35)], schema=["product", "code"]
+    )
     schema(invalid_data, lazy=True)  # type: ignore[arg-type]
 
 
-def test_pyspark_check_eq(spark, sample_spark_schema):
+def test_pyspark_check_eq(spark_session, sample_spark_schema, request):
     """Test creating a pyspark DataFrameSchema object"""
-
+    spark = request.getfixturevalue(spark_session)
     pandera_schema = DataFrameSchema(
         columns={
             "product": Column("str", checks=pa.Check.str_startswith("B")),
@@ -92,11 +84,11 @@ def test_pyspark_check_eq(spark, sample_spark_schema):
     assert dict(df_out.pandera.errors["DATA"]) == expected
 
 
-def test_pyspark_check_nullable(spark, sample_spark_schema):
+def test_pyspark_check_nullable(spark_session, sample_spark_schema, request):
     """
     Test creating a pyspark DataFrameSchema object to validate the nullability functionality
     """
-
+    spark = request.getfixturevalue(spark_session)
     pandera_schema = DataFrameSchema(
         columns={
             "product": Column("str", checks=pa.Check.str_startswith("B")),
@@ -130,11 +122,11 @@ def test_pyspark_check_nullable(spark, sample_spark_schema):
     assert dict(dataframe_output.pandera.errors["SCHEMA"]) == expected
 
 
-def test_pyspark_schema_data_checks(spark):
+def test_pyspark_schema_data_checks(spark_session, request):
     """
     Test schema and data level checks to check the Complex type data match
     """
-
+    spark = request.getfixturevalue(spark_session)
     pandera_schema = DataFrameSchema(
         columns={
             "product": Column("str", checks=pa.Check.str_startswith("B")),
@@ -201,10 +193,11 @@ def test_pyspark_schema_data_checks(spark):
     assert dict(output_data.pandera.errors["SCHEMA"]) == expected["SCHEMA"]
 
 
-def test_pyspark_fields(spark):
+def test_pyspark_fields(spark_session, request):
     """
     Test schema and data level checks for pydantic validation
     """
+    spark = request.getfixturevalue(spark_session)
 
     class PanderaSchema(DataFrameModel):
         """Test case schema class"""
@@ -287,7 +280,9 @@ class PanderaSchema(DataFrameModel):
     assert schema_errors == expected["SCHEMA"]
 
 
-def test_pyspark__error_handler_lazy_validation():
+def test_pyspark__error_handler_lazy_validation(
+    spark_session,  # pylint:disable=unused-argument
+):
     """This function tests the lazy validation for the error handler class of pyspark"""
 
     errors_not_lazy = error_handler.ErrorHandler(lazy=False)
diff --git a/tests/pyspark/test_pyspark_model.py b/tests/pyspark/test_pyspark_model.py
index 69f029cf6..82747e5bd 100644
--- a/tests/pyspark/test_pyspark_model.py
+++ b/tests/pyspark/test_pyspark_model.py
@@ -18,8 +18,14 @@
 from pandera.pyspark import DataFrameModel, DataFrameSchema, Field
 from tests.pyspark.conftest import spark_df
 
+pytestmark = pytest.mark.parametrize(
+    "spark_session", ["spark", "spark_connect"]
+)
+
 
-def test_schema_with_bare_types():
+def test_schema_with_bare_types(
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Test that DataFrameModel can be defined without generics.
     """
@@ -45,7 +51,9 @@ class Model(DataFrameModel):
     assert expected == Model.to_schema()
 
 
-def test_schema_with_bare_types_and_field():
+def test_schema_with_bare_types_and_field(
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Test that DataFrameModel can be defined without generics.
     """
@@ -70,10 +78,11 @@ class Model(DataFrameModel):
     assert expected == Model.to_schema()
 
 
-def test_schema_with_bare_types_field_and_checks(spark):
+def test_schema_with_bare_types_field_and_checks(spark_session, request):
     """
     Test that DataFrameModel can be defined without generics.
     """
+    spark = request.getfixturevalue(spark_session)
 
     class Model(DataFrameModel):
         """Model Schema"""
@@ -109,10 +118,11 @@ class Model(DataFrameModel):
     assert df_out.pandera.errors is not None
 
 
-def test_schema_with_bare_types_field_type(spark):
+def test_schema_with_bare_types_field_type(spark_session, request):
     """
     Test that DataFrameModel can be defined without generics.
     """
+    spark = request.getfixturevalue(spark_session)
 
     class Model(DataFrameModel):
         """Model Schema"""
@@ -136,10 +146,11 @@ class Model(DataFrameModel):
     assert df_out.pandera.errors is not None
 
 
-def test_pyspark_bare_fields(spark):
+def test_pyspark_bare_fields(spark_session, request):
     """
     Test schema and data level checks
     """
+    spark = request.getfixturevalue(spark_session)
 
     class PanderaSchema(DataFrameModel):
         """Test schema"""
@@ -185,7 +196,9 @@ class PanderaSchema(DataFrameModel):
     assert df_out.pandera.errors is not None
 
 
-def test_pyspark_fields_metadata():
+def test_pyspark_fields_metadata(
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Test schema and metadata on field
     """
@@ -245,9 +258,9 @@ class Config:
     ],
     ids=["no_data", "unique_data", "duplicated_data"],
 )
-def test_dataframe_schema_unique(spark, data, expectation):
+def test_dataframe_schema_unique(spark_session, data, expectation, request):
     """Test uniqueness checks on pyspark dataframes."""
-
+    spark = request.getfixturevalue(spark_session)
     df = spark.createDataFrame(data, "a: int, b: int")
 
     # Test `unique` configuration with a single column
@@ -304,9 +317,11 @@ class Config:
         "multiple_wrong_columns_w_empty",
     ],
 )
-def test_dataframe_schema_unique_wrong_column(spark, unique_column_name):
+def test_dataframe_schema_unique_wrong_column(
+    spark_session, unique_column_name, request
+):
     """Test uniqueness checks on pyspark dataframes."""
-
+    spark = request.getfixturevalue(spark_session)
     df = spark.createDataFrame(([1, 2],), "a: int, b: int")
 
     # Test `unique` configuration with a single, wrongly named column
@@ -325,11 +340,14 @@ class Config:
         _ = UniqueMultipleColumns.validate(check_obj=df)
 
 
-def test_dataframe_schema_strict(spark, config_params: PanderaConfig) -> None:
+def test_dataframe_schema_strict(
+    spark_session, config_params: PanderaConfig, request
+) -> None:
     """
     Checks if strict=True whether a schema error is raised because either extra columns are present in the dataframe
     or missing columns in dataframe
     """
+    spark = request.getfixturevalue(spark_session)
     if config_params.validation_depth != ValidationDepth.DATA_ONLY:
         schema = DataFrameSchema(
             {
@@ -376,7 +394,9 @@ def test_dataframe_schema_strict(spark, config_params: PanderaConfig) -> None:
                 raise pa.PysparkSchemaError
 
 
-def test_docstring_substitution() -> None:
+def test_docstring_substitution(
+    spark_session,  # pylint:disable=unused-argument
+) -> None:
     """Test the docstring substitution decorator"""
 
     @docstring_substitution(
@@ -417,7 +437,10 @@ class Schema(pa.DataFrameModel):
     return Schema
 
 
-def test_optional_column(test_schema_optional_columns) -> None:
+def test_optional_column(
+    test_schema_optional_columns,
+    spark_session,  # pylint:disable=unused-argument
+) -> None:
     """Test that optional columns are not required."""
 
     schema = test_schema_optional_columns.to_schema()
@@ -433,10 +456,10 @@ def test_optional_column(test_schema_optional_columns) -> None:
 
 
 def test_validation_succeeds_with_missing_optional_column(
-    spark, test_schema_optional_columns
+    spark_session, test_schema_optional_columns, request
 ) -> None:
     """Test that validation succeeds even when an optional column is missing."""
-
+    spark = request.getfixturevalue(spark_session)
     data = [("5", "b"), ("15", "b")]
     spark_schema = T.StructType(
         [
@@ -454,7 +477,9 @@ def test_validation_succeeds_with_missing_optional_column(
     ), "No error should be raised in case of a missing optional column."
 
 
-def test_invalid_field() -> None:
+def test_invalid_field(
+    spark_session,  # pylint:disable=unused-argument
+) -> None:
     """Test that invalid feilds raises a schemaInitError."""
 
     class Schema(DataFrameModel):  # pylint:disable=missing-class-docstring
@@ -467,8 +492,12 @@ class Schema(DataFrameModel):  # pylint:disable=missing-class-docstring
         Schema.to_schema()
 
 
-def test_registered_dataframemodel_checks(spark) -> None:
+# For the second parameterized `spark_session` run, `@pax.register_check_method` will
+# raise a ValueError due to a duplicated registration tentative
+@pytest.mark.xfail(raises=ValueError)
+def test_registered_dataframemodel_checks(spark_session, request) -> None:
     """Check that custom registered checks work"""
+    spark = request.getfixturevalue(spark_session)
 
     @pax.register_check_method(
         supported_types=DataFrame,
@@ -497,8 +526,10 @@ class Config:
     assert not out.pandera.errors
 
 
-@pytest.fixture(scope="module")
-def model_with_datatypes():
+@pytest.fixture(scope="function")
+def model_with_datatypes(
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Model containing all common datatypes for PySpark namespace.
     """
@@ -527,8 +558,10 @@ class SchemaWithDatatypes(DataFrameModel):
     return SchemaWithDatatypes
 
 
-@pytest.fixture(scope="module")
-def model_with_multiple_parent_classes():
+@pytest.fixture(scope="function")
+def model_with_multiple_parent_classes(
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Model inherited from multiple parent classes.
     """
@@ -561,7 +594,10 @@ class BaseClassFinal(BaseClassB, BaseClassC):
     return BaseClassFinal
 
 
-def test_schema_to_structtype(model_with_datatypes):
+def test_schema_to_structtype(
+    model_with_datatypes,
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Test the conversion from a model to a StructType object through `to_structtype()`.
     """
@@ -619,7 +655,10 @@ def test_schema_to_structtype(model_with_datatypes):
     )
 
 
-def test_schema_to_ddl(model_with_datatypes):
+def test_schema_to_ddl(
+    model_with_datatypes,
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Test the conversion from a model to a DDL string through `to_ddl()`.
     """
@@ -646,7 +685,10 @@ def test_schema_to_ddl(model_with_datatypes):
     )
 
 
-def test_inherited_schema_to_structtype(model_with_multiple_parent_classes):
+def test_inherited_schema_to_structtype(
+    model_with_multiple_parent_classes,
+    spark_session,  # pylint:disable=unused-argument
+):
     """
     Test the final inheritance for a model with a longer parent class structure.
     """
diff --git a/tests/pyspark/test_schemas_on_pyspark_pandas.py b/tests/pyspark/test_schemas_on_pyspark_pandas.py
index c19ffa1de..b1c9542ee 100644
--- a/tests/pyspark/test_schemas_on_pyspark_pandas.py
+++ b/tests/pyspark/test_schemas_on_pyspark_pandas.py
@@ -6,10 +6,10 @@
 
 import numpy as np
 import pandas as pd
+import pyspark
 import pyspark.pandas as ps
 import pytest
 from packaging import version
-from pyspark import SparkContext
 
 import pandera as pa
 from pandera import dtypes, extensions, system
@@ -63,7 +63,7 @@
     pandas_engine.Date,
 }
 
-SPARK_VERSION = version.parse(SparkContext.getOrCreate().version)
+SPARK_VERSION = version.parse(pyspark.__version__)
 
 if SPARK_VERSION < version.parse("3.3.0"):
     PYSPARK_PANDAS_UNSUPPORTED.add(numpy_engine.Timedelta64)