DEPS: update setuptools min version
State: closed

According to https://setuptools.readthedocs.io/en/latest/userguide/declarative_config.html, the min version of setuptools that support `entry_points` in `setup.cfg` is  `51.0.0`.

Labels:
name: Dependencies
description: Required and optional dependencies

Comments:
cc @simonjayhawkins 
are we not testing with the minimum setuptools?
> are we not testing with the minimum setuptools?

No, we are not. 
in doc/source/development/contributing_environment.rst

```
Creating a Python environment (pip)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you aren't using conda for your development environment, follow these instructions.
You'll need to have at least the :ref:`minimum Python version <install.version>` that pandas supports. If your Python version
is 3.8.0 (or later), you might need to update your ``setuptools`` to version 42.0.0 (or later)
in your development environment before installing the build dependencies::
```
can be updated.
can you update the min pin in environment.yaml (and other places if needed)?
also this should be added to the wheel recipe @simonjayhawkins  (practically this already works, but should add)
test failure unrelated

```
E           AssertionError: Caused unexpected warning(s): [('ResourceWarning', ResourceWarning("unclosed <ssl.SSLSocket fd=8, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.1.1.54', 48258), raddr=('52.217.136.144', 443)>"), '/home/runner/work/pandas/pandas/pandas/io/sas/sas7bdat.py', 381)]
```
> can you update the min pin in environment.yaml (and other places if needed)?

@fangchenli can you address this
> also this should be added to the wheel recipe @simonjayhawkins (practically this already works, but should add)

pandas-feedstock or MacPython or both?
> can you update the min pin in environment.yaml (and other places if needed)?

We don't list setuptools in `environment.yaml`.
labelled as blocker as we won't be changing min versions of dependencies in patch releases? 
yeah i think we should just add this to `environment.yaml`. it is technically only needed if we use matplotlib so its not a required dep (so only add it where we have matplotlib, which is most places).

conda-incubator/setup-miniconda@v2 does not seem to like the setuptools pin, although I _think_ there have been conda-incubator/setup-miniconda@v2 timeouts on other PRs
The version in pandas-dev on CI / Checks for master (https://github.com/pandas-dev/pandas/runs/2950514145) is

```
2021-06-30T09:08:01.2373051Z setuptools                49.6.0           py38h578d9bd_3    conda-forge
```

and creating dev env locally

my backport env (1.3.x)
```
(pandas-backport) simon@ProDesk:~/pandas (backport-of-pr-42318-on-1.3.x)$ conda list setuptools
# packages in environment at /home/simon/miniconda3/envs/pandas-backport:
#
# Name                    Version                   Build  Channel
setuptools                49.6.0           py38h578d9bd_3    conda-forge
```

my master env
```
(pandas-dev) simon@P340:~/pandas (master)$ conda list setuptools
# packages in environment at /home/simon/miniconda3/envs/pandas-dev:
#
# Name                    Version                   Build  Channel
setuptools                49.6.0           py38h578d9bd_3    conda-forge
```

> According to https://setuptools.readthedocs.io/en/latest/userguide/declarative_config.html, the min version of setuptools that support `entry_points` in `setup.cfg` is `51.0.0`.

hmm? our tests pass locally?
@jreback ok to release without this PR, or delay release to allow time for others to look at this? @pandas-dev/pandas-core 
should
include this
> should
> include this

it appears can't create a dev env with conda with this pin
Could leave it out of the yml file and then do a pip install -r requirements-dev.txt (assuming it is in requirements-dev.txt). This will no op everything that is installed, but upgrade setuptools if below.

> > should
> > include this
> 
> it appears can't create a dev env with conda with this pin

can create a dev env locally from scratch using this pin. so maybe something related to  conda-incubator/setup-miniconda@v2

creating a dev env from scratch without this pin, the only diff is

```
< setuptools=49.6.0=py38h578d9bd_3
---
> setuptools=52.0.0=py38h06a4308_0
```

so it's a mystery to me how conda chooses the version.
hmm yeah this should just work. 
@fangchenli can you summarize the issue this PR is trying to solve
> > are we not testing with the minimum setuptools?
> 
> No, we are not.

@fangchenli should we test with the min version?

> yeah i think we should just add this to `environment.yaml`. it is technically only needed if we use matplotlib so its not a required dep (so only add it where we have matplotlib, which is most places).

and have a min pin in the envs with matplotlib?

---

> my master env
> 
> ```
> (pandas-dev) simon@P340:~/pandas (master)$ conda list setuptools
> # packages in environment at /home/simon/miniconda3/envs/pandas-dev:
> #
> # Name                    Version                   Build  Channel
> setuptools                49.6.0           py38h578d9bd_3    conda-forge
> ```
> 
> > According to https://setuptools.readthedocs.io/en/latest/userguide/declarative_config.html, the min version of setuptools that support `entry_points` in `setup.cfg` is `51.0.0`.
> 
> hmm? our tests pass locally?

Why I am not getting any failures? are we missing tests?
> According to https://setuptools.readthedocs.io/en/latest/userguide/declarative_config.html, the min version of setuptools that support entry_points in setup.cfg is 51.0.0.

I think we need to distinguish the install-time and run-time dependency here. For installing pandas with setting up the entrypoint, we need a recent setuptools (since we are using `setup.cfg` now), but for *running* pandas and using setuptools to find the entry point for a given plotting backend, we use the `setuptools.pkg_resources` submodule. I think this module is already existing much longer in setuptools, and I don't think we need a minimum version for setuptools as a run-time dependency.

For setuptools as a build/install-time dependency, we only need to update the pyproject.toml, I think.


>  I think this module is already existing much longer in setuptools, and I don't think we need a minimum version for setuptools as a run-time dependency.

If setuptools isn't listed as a run-time dependency at all, it is possible that an install might not have it.  It looks like pkg_resources is imported in an unprotected manner here

https://github.com/pandas-dev/pandas/blob/059c8bac51e47d6eaaa3e36d6a293a22312925e6/pandas/tests/plotting/test_backend.py#L4

which I think means test discovery will fail.  FWIW the test that uses it is skipped if MPL is not installed (and setuptools is not in the MPL feature bundle for pandas), but the test discovery is not protected.

Should this import be moved inside the function that uses it?  Perhaps `pkg_resources` should always be lazily imported (it is only used in 3 places, one code, one test and one benchmark).
> Why I am not getting any failures? are we missing tests?

AFAICT the only way to get a failure (which will be an error in test discovery) is to run the tests in an environment without setuptools.  This is possible, but in most cases would need a deliberate attempt to create (i.e., install pandas, remove setuptools, run tests).
> For setuptools as a build/install-time dependency, we only need to update the pyproject.toml, I think.

I agree. Also, we don't know what's the min version of setuptools that supports the function we used in matplotlib backend. The previous min version `38.6.0` was also determined by an attribute we used in setup.cfg. 

> I agree. Also, we don't know what's the min version of setuptools that supports the function we used in matplotlib backend. The previous min version `38.6.0` was also determined by an attribute we used in setup.cfg.

Is there some reason not to lazily import it in the test file where it appears? If it is lazily imported when only will cause a problem is someone (a) has matplotlib and (b) does not have setuptools.
> labelled as blocker as we won't be changing min versions of dependencies in patch releases?

if we are no longer changing the min version for users, this is no longer a blocker? will release tomorrow even if this PR is not ready?
yep let's just release
this is effectively docs at this point
not yet triaged the pypy failures on conda-forge https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=345935&view=results, but did notice setuptools was pre 0.51