RFC Write an explicit rule about bumping our minimum dependencies

Roughly a year ago, [SPEC0](https://scientific-python.org/specs/spec-0000/) was rejected following a vote and we said we would write our own rule, but we did not 😅. 

Until now 💪.

This was spurred by a [Discord discussion](https://discord.com/channels/731163543038197871/1046822941586898974/1338208487657701446) with @lucascolley, @betatim, @jeremiedbb and @ogrisel.

cc @glemaitre whom I had a chat with about this.

### Proposed rule

- **Python**: in each scikit-learn December release, we bump our minimum supported Python to the Python version that was released a bit more than 3 years ago (Python releases happened yearly beginning of October). 
- **non pure-Python dependencies** (numpy, scipy, pandas, etc ...): in each December release they are bumped to the minimum minor release that has wheels for the minimum Python version. 
- **pure Python dependencies**: in each release (December and June) bump to the most recent minor release older than 2 years old
- we expect that exceptions may arise, although hopefully not too often, for example security or critical bug fixes

### Rationale

- we want a simple rule
- we don't want to be even more conservative that what we have been doing historically
- in an ideal world, we would want to try to avoid requiring newer versions if there is not a "good reason" too, although there is some tension between having a "simple rule" and this bullet point

### Proposed plan

we didn't bump our dependency in 1.6 so we would bump them in 1.7 (June 2025) and start the regular December version bump in December 2025.

This is what it would look like for the next 4 scikit-learn releases (`python-date-diff` column is the age of the min Python at the time of the scikit-learn release, and similarly for other dependencies).

Python:
```
  scikit-learn scikit-learn-date python  python-date-diff
0          1.7        2025-06-01   3.10          3.660274
1          1.8        2025-12-01   3.11          3.106849
2          1.9        2026-06-01   3.11          3.605479
3         1.10        2026-12-01   3.12          3.167123
```

non pure-Python dependencies:
```
  scikit-learn scikit-learn-date numpy  numpy-date-diff scipy  scipy-date-diff pandas  pandas-date-diff
0          1.7        2025-06-01  1.22         3.416438   1.8         3.317808    1.4          3.356164
1          1.8        2025-12-01  1.24         2.953425  1.11         2.435616    2.0          2.663014
2          1.9        2026-06-01  1.24         3.452055  1.11         2.934247    2.0          3.161644
3         1.10        2026-12-01  1.26         3.208219  1.12         2.863014    2.2          2.863014
```

pure Python dependencies:
```
  scikit-learn scikit-learn-date joblib  joblib-date-diff threadpoolctl  threadpoolctl-date-diff
0          1.7        2025-06-01    1.2          2.706849           3.1                 3.331507
1          1.8        2025-12-01    1.3          2.427397           3.2                 2.386301
2          1.9        2026-06-01    1.4          2.145205           3.5                 2.087671
3         1.10        2026-12-01    1.4          2.646575           3.5                 2.589041
```


### Comparison to SPEC0

- Python: the proposed rule corresponds to dropping Python support after 4 years (instead of 3 years for SPEC0)
- non pure-Python dependencies: This corresponds to dropping support for a version that is a bit more than 3 years old (rather than 2 years for SPEC0).
- pure-Python dependencies: same as SPEC0

Note: SPEC0 talks about the age of the dropped dependency release, which I find slightly harder to think about than the age of the minimum depency release. The age of the dropped dependency release also depends on the dependency release cadence. This does not matter than much for dependencies with regular releases like numpy, but makes more of a difference for dependencies with less regular releases, like joblib.

### "What is our implicit rule?" investigation

tl; dr: very ad-hoc and sometimes more adventurous than the proposed rule

I used a quick and dirty [script](https://gist.github.com/lesteve/6099307249f3235e16063ad847ff0348) to figure out what we have been doing historically.

Python:
```
  scikit-learn python  python-date-diff
0         0.21    3.5          3.657534
1         0.22    3.5          4.224658
2         0.23    3.6          3.386301
3          1.0    3.7          3.246575
4          1.1    3.8          2.578082
5          1.2    3.8          3.153425
6          1.3    3.8          3.712329
7          1.4    3.9          3.287671
8          1.5    3.9          3.627397
9          1.6    3.9          4.180822
```

the proposed rule means our minimum Python is a bit more than 3 years old, which fits what we have done historically. Note that scikit-learn 1.1 dropped 3.7 quite aggressively (https://github.com/scikit-learn/scikit-learn/pull/22617 does not mention any particular reason), and that we did not bump our dependency for 1.6.

numpy:
```
  scikit-learn python   numpy  numpy-date-diff
0         0.21    3.5  1.11.0         3.117808
1         0.22    3.5  1.11.0         3.684932
2         0.23    3.6  1.13.3         2.616438
3          1.0    3.7  1.14.6         3.002740
4          1.1    3.8  1.17.3         2.567123
5          1.2    3.8  1.17.3         3.142466
6          1.3    3.8  1.17.3         3.701370
7          1.4    3.9  1.19.5         3.032877
8          1.5    3.9  1.19.5         3.372603
9          1.6    3.9  1.19.5         3.926027
```
roughly 3 years in terms of time-based rule, which roughly matches what the proposed rule would do. Note the aggressive bump in 1.1.

scipy:
```
  scikit-learn python   scipy  scipy-date-diff
0         0.21    3.5  0.17.0         3.293151
1         0.22    3.5  0.17.0         3.860274
2         0.23    3.6  0.19.1         2.887671
3          1.0    3.7   1.1.0         3.389041
4          1.1    3.8   1.3.2         2.504110
5          1.2    3.8   1.3.2         3.079452
6          1.3    3.8   1.5.0         3.021918
7          1.4    3.9   1.6.0         3.046575
8          1.5    3.9   1.6.0         3.389041
9          1.6    3.9   1.6.0         3.942466
```
roughly 3 years in terms of time-based rule, which roughly matches what the proposed rule would do. Note the aggressive bump in 1.1.

pandas:
```
  scikit-learn python  pandas  scipy-date-diff
0         0.21    3.5     NaN         3.293151
1         0.22    3.5     NaN         3.860274
2         0.23    3.6     NaN         2.887671
3          1.0    3.7  0.25.0         3.389041
4          1.1    3.8   1.0.5         2.504110
5          1.2    3.8   1.0.5         3.079452
6          1.3    3.8   1.0.5         3.021918
7          1.4    3.9   1.1.5         3.046575
8          1.5    3.9   1.1.5         3.389041
9          1.6    3.9   1.1.5         3.942466
```
roughly 3 years in terms of time-based rule, which roughly matches what the proposed rule would do. Note the aggressive bump in 1.1.

joblib:
```
  scikit-learn python joblib  joblib-date-diff
0         0.21    3.5   0.11          2.186301
1         0.22    3.5   0.11          2.756164
2         0.23    3.6   0.11          3.197260
3          1.0    3.7   0.11          4.567123
4          1.1    3.8  1.0.0          1.408219
5          1.2    3.8  1.1.1          0.161644
6          1.3    3.8  1.1.1          0.717808
7          1.4    3.9  1.2.0          1.336986
8          1.5    3.9  1.2.0          1.679452
9          1.6    3.9  1.2.0          2.232877
```
the proposed rule (2 years) roughly corresponds to what we have been doing when deciding to bump. Note:
- aggressive bumping in 1.1: joblib min version was less than 1.5 year old
- bumping in 1.2 was due to a security bug-fix

```
  scikit-learn python threadpoolctl  threadpoolctl-date-diff
0         0.21    3.5           NaN                      NaN
1         0.22    3.5           NaN                      NaN
2         0.23    3.6           NaN                      NaN
3          1.0    3.7         2.0.0                 1.802740
4          1.1    3.8         2.0.0                 2.432877
5          1.2    3.8         2.0.0                 3.008219
6          1.3    3.8         2.0.0                 3.567123
7          1.4    3.9         2.0.0                 4.120548
8          1.5    3.9         3.1.0                 2.304110
9          1.6    3.9         3.1.0                 2.854795
```
The proposed rule (2 years) roughly corresponds to what we have been doing historically when deciding to bump.
- bumping in 1.5 was done with the "more than 2 years old is OK" argument https://github.com/scikit-learn/scikit-learn/pull/28838#issue-2243861595.

COMMENT:
Here is a draft of what the consensus seems to be for our "dependencies minimum version bumping rule":

### Guideline

- **minimum Python version**: at the time of a minor scikit-learn release (X.Y.0), we drop Python whose initial release was more than 4 years old. In other words, our minimum Python version is between 3 and 4 years old.
- **compiled dependencies** (numpy, scipy, as well as compiled optional dependencies like pandas, matplotlib, pyamg, pillow): we take the oldest minor release (X.Y.0) that has wheels for our minimum Python version. In practice this means that our minimum supported version is around 3 years old, maybe a bit less.
- **pure Python dependencies** (joblib, threadpoolctl): at the time of the scikit-learn release our minimum supported version is the most recent minor release (X.Y.0) that is at least 2 years old.
- we may decide to be less conservative than this guideline in some edge cases. These edge cases include: a security bugfix in one of our dependencies or a critical bugfix in one of our dependencies makes it too costly to support it in terms of maintenance.

### Example for the upcoming scikit-learn 1.8 release

The upcoming 1.8 release is scheduled around November/December, let's pretend it will happen on 2025-12-01, Python 3.10 (released in October 2021) will be more than 4 years old and will be dropped so our minimum supported Python will be Python 3.11 (a bit more than 3 years old).

The dependencies minimum versions will be bumped as follow:
- python: 3.10 -> 3.11
- numpy: 1.22.0 -> 1.24.0 (oldest version with Python 3.11 wheels)
- scipy: 1.8.0 -> 1.10.0 (oldest scipy version with Python 3.11 wheels)
- pandas: 1.4.0 -> 1.5.0 (oldest pandas version with Python 3.11 wheels)
- matplotlib: 3.5.0 -> 3.6.0 (oldest matplotlib version with Python 3.11 wheels)
- joblib: 1.2.0 -> 1.3.0 (1.3.0 will be ~2 years 5 month old, 1.4.0 will be less than 2 years old, see [release history](https://pypi.org/project/joblib/#history))
- threadpoolctl: 3.1.0 -> 3.2.0 (3.2.0 will be ~2 years 5 months old, 3.3.0 will be less than 2 years old, see [release history](https://pypi.org/project/threadpoolctl/#history))

The new minimum version were generated from this hacky [script](https://gist.github.com/lesteve/4a7d42bfaab461ab90fcdcc613d01081), with definitely some room for improvement.

### Rough Comparison with SPEC0

- Python has one more year of support: 4 years instead of 3 years.
- compiled dependencies have roughly one more year of support: roughly 3 years instead of 2 years. In practice the exact support window depends how quickly our dependencies released a new minor version (X.Y.0) that supported our minimum Python version.
- pure Python dependencies: a bit more conservative than SPEC0. Our minimum supported version is at least 2 years old, whereas SPEC0 says you can drop support for a version if it's more than 2 years old. This also depends on our dependencies release cadence. joblib and threadpoolctl tends to not have very regular release cadences, compared to scikit-learn or numpy (roughly 2 minor releases a year). This makes SPEC0 too aggressive compared to what we have been doing in scikit-learn. For example for scikit-learn 1.8 release (scheduled 2025-12-01), SPEC0 would recommend to drop joblib 1.3 (1.3.0 is a bit more than 2 years old at the time of release), but the next minor joblib version 1.4.0 happened in April 2024 so it would be less than one year old in December 2025, which seems way too recent compared to what we have been doing historically (minimum version is roughly 2 years old).

Labels: RFC