Talip7/scikit-learn-issues-multilabel huggingface dataset:
DOC: Example of `train_test_split` with `pandas` DataFrames

### Describe the issue linked to the documentation

Currently, the example [here](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html) only illustrates the use case of `train_test_split` for `numpy` arrays. I think an additional example featuring a `pandas` DataFrame would  make this page more beginner-friendly. Would you guys be interested? 

### Suggest a potential alternative/fix

The modification in [`model_selection/_split`](https://github.com/scikit-learn/scikit-learn/blob/d666202a9349893c1bd106cc9ee0ff0a807c7cf3/sklearn/model_selection/_split.py) would be the following:
```
"""
Example: Data are a `numpy` array
--------
>>> Current example

Example: Data are a `pandas` DataFrame
--------
>>> from sklearn import datasets
>>> from sklearn.model_selection import train_test_split
>>> iris = datasets.load_iris(as_frame=True)
>>> X, y = iris['data'], iris['target']
>>> X.head()
   sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)
0                5.1               3.5                1.4               0.2
1                4.9               3.0                1.4               0.2
2                4.7               3.2                1.3               0.2
3                4.6               3.1                1.5               0.2
4                5.0               3.6                1.4               0.2
>>> y.head()
0    0
1    0
2    0
3    0
4    0
>>> X_train, X_test, y_train, y_test = train_test_split(
... X, y, test_size=0.33, random_state=42) # rows will be shuffled
>>> X_train.head()
     sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)
96                 5.7               2.9                4.2               1.3
105                7.6               3.0                6.6               2.1
66                 5.6               3.0                4.5               1.5
0                  5.1               3.5                1.4               0.2
122                7.7               2.8                6.7               2.0
>>> y_train
96     1
105    2
66     1
0      0
122    2
>>> X_test.head()
     sepal length (cm)  sepal width (cm)  petal length (cm)  petal width (cm)
73                 6.1               2.8                4.7               1.2
18                 5.7               3.8                1.7               0.3
118                7.7               2.6                6.9               2.3
78                 6.0               2.9                4.5               1.5
76                 6.8               2.8                4.8               1.4
>>> y_test.head()
73     1
18     0
118    2
78     1
76     1
"""
```

COMMENT:
Hi, I would like to work on this issue. Can you please assign it to me?

Labels: Documentation
BUG: n_outputs problem for RandomForestClassifier when the design matrix is skewed

While investigating a downstream problem in `shap` (https://github.com/shap/shap/issues/3948), I noticed that my reproducer (see below) is producing a `clf[0].tree_.value[0].shape` that is not consistent with a single-output classifier, probably because of some kind of numerical instability issue? I spent most of my time debugging `shap` source code, but now I'm not so sure they are wrong to assume that `clf[0].tree_.value[0].shape` should have a sub-array length that corresponds to `n_outputs_`.

If you run this as-is, the assertion will fail, but if you drop the number of columns by a factor of 10 to `9_000` the assertion will pass. The broader context is that we're doing some high feature count (high dimensionality) ML and this fell out from a much larger real case.

```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

seed = 0
n_rows = 3
n_cols = 90_000
rng = np.random.default_rng(seed)
X = rng.integers(low=0, high=2, size=(n_rows, n_cols)).astype(np.float64)
y = rng.integers(low=0, high=2, size=n_rows)
clf = RandomForestClassifier(n_estimators=1, random_state=seed)
clf.fit(X, y)
assert clf.n_outputs_ == clf[0].n_outputs_ == y.ndim == clf[0].tree_.value[0].shape[1] == clf[0].tree_.n_outputs
```

COMMENT:
I'm not actually sure where the "fault" lies actually, looking at the source for `sklearn/tree/_tree.pyx` I see `self.node_count`, `self.n_outputs`, `self.max_n_classes` as defining the shape of `value` in `_get_value_ndarray`, and `(3, 1, 2)` would be acceptable by that definition for the single tree, but this does seem to be somewhat surprising based on the `shap` source.

Anyway, clarifying whether this is "formally" valid will help determine if a `shap` bug or not.

Labels: Needs Triage
BUG: n_outputs problem for RandomForestClassifier when the design matrix is skewed

While investigating a downstream problem in `shap` (https://github.com/shap/shap/issues/3948), I noticed that my reproducer (see below) is producing a `clf[0].tree_.value[0].shape` that is not consistent with a single-output classifier, probably because of some kind of numerical instability issue? I spent most of my time debugging `shap` source code, but now I'm not so sure they are wrong to assume that `clf[0].tree_.value[0].shape` should have a sub-array length that corresponds to `n_outputs_`.

If you run this as-is, the assertion will fail, but if you drop the number of columns by a factor of 10 to `9_000` the assertion will pass. The broader context is that we're doing some high feature count (high dimensionality) ML and this fell out from a much larger real case.

```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

seed = 0
n_rows = 3
n_cols = 90_000
rng = np.random.default_rng(seed)
X = rng.integers(low=0, high=2, size=(n_rows, n_cols)).astype(np.float64)
y = rng.integers(low=0, high=2, size=n_rows)
clf = RandomForestClassifier(n_estimators=1, random_state=seed)
clf.fit(X, y)
assert clf.n_outputs_ == clf[0].n_outputs_ == y.ndim == clf[0].tree_.value[0].shape[1] == clf[0].tree_.n_outputs
```

COMMENT:
Perhaps `shap` needs shims for `self.max_n_classes`, not sure though.

Labels: Needs Triage
BUG: n_outputs problem for RandomForestClassifier when the design matrix is skewed

While investigating a downstream problem in `shap` (https://github.com/shap/shap/issues/3948), I noticed that my reproducer (see below) is producing a `clf[0].tree_.value[0].shape` that is not consistent with a single-output classifier, probably because of some kind of numerical instability issue? I spent most of my time debugging `shap` source code, but now I'm not so sure they are wrong to assume that `clf[0].tree_.value[0].shape` should have a sub-array length that corresponds to `n_outputs_`.

If you run this as-is, the assertion will fail, but if you drop the number of columns by a factor of 10 to `9_000` the assertion will pass. The broader context is that we're doing some high feature count (high dimensionality) ML and this fell out from a much larger real case.

```python
import numpy as np
from sklearn.ensemble import RandomForestClassifier

seed = 0
n_rows = 3
n_cols = 90_000
rng = np.random.default_rng(seed)
X = rng.integers(low=0, high=2, size=(n_rows, n_cols)).astype(np.float64)
y = rng.integers(low=0, high=2, size=n_rows)
clf = RandomForestClassifier(n_estimators=1, random_state=seed)
clf.fit(X, y)
assert clf.n_outputs_ == clf[0].n_outputs_ == y.ndim == clf[0].tree_.value[0].shape[1] == clf[0].tree_.n_outputs
```

COMMENT:
I think `sklearn` is "ok" here.

Labels: Needs Triage
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
We could potentially leverage the experimental GHA Windows ARM64 with cibuildwheel. For `conda-forge`, we should check the progress in https://github.com/conda-forge/conda-forge.github.io/issues/1940.

So I can assess that we could be able to deliver some wheel or binaries. It might take some time because we need the infrastructure that we depend on to catch up.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Uhm actually `cibuildwheel` uses cross-compilation and not the native GHA workers (only available for Team and Enterprise Cloud plans) so it might be super slow.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I would wait until numpy and scipy provide a Windows ARM64 wheel, see https://github.com/numpy/numpy/issues/22530 for the ongoing discussion in numpy

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@CanePlayz out of curiosity what's the current status for you, do you build from source, are you unable to install scikit-learn on your Window ARM64 machine?

From the numpy discussion linked above, there seems to be https://github.com/cgohlke/win_arm64-wheels that has unofficial wheels for quite a number of packages including scikit-learn 1.5.2 but that mentions that the scipy build is unstable (without more details ...). There is also https://github.com/matthew-brett/scipy-windows-arm that may have more relevant info for scipy.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@lesteve I used the regular `pip install` and it started building from source but it failed because I have no Fortran compiler installed. It should be no problem to install one (I haven't tried because I don't need scikit immediately on this device), but that's where my wish for pre-built ARM64 wheels came from.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
The fortran code is in SciPy and a scikit-learn dependence. So you will need to that the entire ecosystem is providing wheels.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
> The fortran code is in SciPy and a scikit-learn dependence. So you will need to that the entire ecosystem is providing wheels.

I have fortran running on arm64 windows and all other dependencies as well, similar build to what @cgohlke is running to build his wheels. I built some whls using msys2 clangarm64. 
But still encountered a bug that i think is caused by something that scikit-learn does in its build process, will disclose more in seperate issue since i think it warrants it.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@glemaitre  @lesteve 

SciPy got merged this weekend: https://github.com/scipy/scipy/pull/23122
Nightly builds are available here: https://anaconda.org/scientific-python-nightly-wheels/scipy/files

I manage to build/install  Scikit-learn in my Window ARM64 laptop from latest of master and also all  pytest sklearn, tests appears to pass.


python -m venv sklearn-dev-env
sklearn-dev-env\Scripts\activate
pip install -U pip
pip install wheel numpy cython meson-python ninja
pip install  --index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple scipy
pip install --editable . --no-build-isolation --verbose --config-settings editable-verbose=true


And all tests appears to pass as follows. No failure

====32589 passed, 7551 skipped, 151 xfailed, 63 xpassed, 3568 warnings in 1011.81s (0:16:51) ===========


Can Scikit-learn formally release binaries and wheels for Windows ARM64?

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@glemaitre @lesteve 
I am creating WoA GHA runners as we have runners available for windows ARM64. Once I am done, I will update the PRs.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@lesteve any idea when we'll see the actual wheel on pypi? 😉

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@khmyznikov surely you are joking right? The schedule hasn't changed, please check my answer from one week ago https://github.com/scikit-learn/scikit-learn/issues/32491#issuecomment-3574723763!

Let me try to be clear: as I told you already you (and the people who were working with you on the Windows arm64 wheels effort and I interacted with on this repo) are pushing way too aggressively and pinging way too often. To set clear expectations, pinging after 2 weeks why not, pinging after 1 week way too annoying for my taste.

I get it, you are on a **funded** mission to push Windows arm64 wheels across the whole Scientific Python ecosystem and you would want things to go forward as fast as possible.

Please try to show a little bit of understanding for the maintainer side as well :pray:! Pinging often will not make things go faster, it only makes maintainer more grumpy and likely to burn out. Hopefully that's not what you want, right?

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Have we even decided if we want the wheels? Last we talked, the wheel builder was failing too frequently for us to want to include it in the official release. Has that changed?

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Here is the current status about intermittent Windows arm wheels CI failures:
- recently it seems like we are seeing more and more `ValueError: concurrent send_bytes() calls are not supported` see [summary](https://github.com/scikit-learn/scikit-learn/issues/30810#issuecomment-3567508679). 4 failures out of 7 runs in the last week. IMO this is on the verge on being annoying enough to not being included in the release if this keep happening that often. The root cause is likely a Python issue https://github.com/python/cpython/issues/130733.
- the segmentation fault which was happening quite often in the past has disappeared https://github.com/scikit-learn/scikit-learn/issues/32491#issuecomment-3569781862.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Ok, so it seems to be for the release schedule we have, it's better to make a call and exclude the wheel from the official release, since we definitely don't want to release something where a simple call to `HistGradientBoosting` would cause such an error so frequently, and let Windows ARM64 folks figure out the issue, and when resolved, reconsider the issue.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@lesteve Oh man, really sorry about that. I’ve been buried in a lot of work these past weeks and completely forgot I’d already asked you recently. [In that issue](https://github.com/scikit-learn/scikit-learn/issues/32491#issuecomment-3574723763), I didn’t notice you mentioning the `concurrent send_bytes()` calls is this something new? I’ve seen it happen mostly on Arm, but occasionally on AMD too. I’m happy to help investigate this. Btw, feel free to ping me anytime if any new problems come up with Arm builds or tests.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@khmyznikov scikit-learn 1.8.0 was released on December 10 and has Windows arm64 wheels, see https://pypi.org/project/scikit-learn/1.8.0/#files.

Full disclosure: I was hesitant to include Windows arm64 wheels, but since the intermittent error never showed up during the release procedure I decided to leave them in. Note that depending on how often these intermittent issues show up, we may decide to remove Windows arm64 wheels from future releases.

FYI the latest weird Windows arm64 issue is https://github.com/scikit-learn/scikit-learn/issues/32393#issuecomment-3653252124. Probably worth waiting a bit more to see if it happens often enough to start investigating.

Labels: Build / CI
Provide wheel for Windows ARM64

### Describe the workflow you want to enable

Pretty simple, I want to be able to more easily use scikit-learn on my Windows ARM64 machine.

### Describe your proposed solution

Build a wheel for Windows ARM64.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
@lesteve Thanks for helping make this a success! What an awesome holiday gift for all of us 😊 We’re ready to dive into any oddities if they pop up. 🫡

Labels: Build / CI
`GridSearchCV` optimization by early elimination of bad performing configurations

### Describe the workflow you want to enable

`GridSearchCV` currently tries every configuration k-times (of `KFold`). But the bad performing configurations could be tried less than k-times. This could decrease the training time/resources when using `GridSearchCV` by about 30% (depending on the variance of the scores).

So for example a configuration with a score of `0.73` in 1 out of 5 folds can't perform better than a configuration with `[0.99, 0.98, 1.0, 0.96, 0.99]` as scores and therefore the other 4 folds of the former mentioned configuration can be ignored.

### Describe your proposed solution

Therefore I propose the following scheme:
1. Do 1 out of `k` rounds for every configuration.
2. Do a round for the configuration which scored best in the previous round(s)
3. If all `k` rounds were applied to a configuration, eliminate all configurations which scores combined with the best possible score for the missing rounds are lower than the best finished score.

Step `3.` can also be applied when a new step is done and the highest possible overall score for this configuration is lower than the overall score of the best finished configuration.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I can contribute to this if it's not assigned to anyone. LMK @TheAlgorythm

Labels: New Feature, Needs Triage
`GridSearchCV` optimization by early elimination of bad performing configurations

### Describe the workflow you want to enable

`GridSearchCV` currently tries every configuration k-times (of `KFold`). But the bad performing configurations could be tried less than k-times. This could decrease the training time/resources when using `GridSearchCV` by about 30% (depending on the variance of the scores).

So for example a configuration with a score of `0.73` in 1 out of 5 folds can't perform better than a configuration with `[0.99, 0.98, 1.0, 0.96, 0.99]` as scores and therefore the other 4 folds of the former mentioned configuration can be ignored.

### Describe your proposed solution

Therefore I propose the following scheme:
1. Do 1 out of `k` rounds for every configuration.
2. Do a round for the configuration which scored best in the previous round(s)
3. If all `k` rounds were applied to a configuration, eliminate all configurations which scores combined with the best possible score for the missing rounds are lower than the best finished score.

Step `3.` can also be applied when a new step is done and the highest possible overall score for this configuration is lower than the overall score of the best finished configuration.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
We already have successive halving strategy that goes in this direction: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.HalvingGridSearchCV.html#sklearn.model_selection.HalvingGridSearchCV

Labels: New Feature, Needs Triage