Talip7/scikit-learn-issues-multilabel huggingface dataset:
Transformer for nominal categories, with the goal of improving category support in decision trees

I'd like to find out how keen people would be for adding a transformer like this.

### Describe the workflow you want to enable

Improved support for nominal categories in tree based models. Nominal categories are ones with no order to them, for example colour or country (c.f. ordinal categories that are ordered).

### Describe your proposed solution

In #12866 @amueller linked to [Splitting on categorical predictors in random forests](https://peerj.com/articles/6339). This proposes to replace categorical values (red, green, blue, ..) by cleverly computed numerical values (blue -> 0.643, green ->0.123, ...) that allow you to achieve similar/equal performance by using `<` as splitting decision in a tree node compared to performing an exhaustive search of all possible categorical splits.

I would implement this as a transformer that you use together with a random forest or other tree based model in a pipeline.

### Describe alternatives you've considered, if relevant

There have been several PRs attempting to add native category support to decision trees. #12866 is the latest one. My impression is that it would be cool to have this in trees but that it is not easy to do, several people have tried but no PR has landed yet. The "Breimann trick" that is used is also limited in the number of categorical values it supports, where this new idea seems to support unlimited categorical values and multi class classification.

### Additional context

An earlier paper that is cited in "Splitting categorical predictors...": https://link.springer.com/article/10.1023/A:1009869804967

An open question for me is how to measure how good this kind of transformation is compared to the exhaustive split (which is "the best possible" but too expensive to compute to be used in practice). The method proposed by the paper seems simple (famous last words), and they make claims about the performance but it would be nice to know how close this gets us. Trust, but verify ;)

COMMENT:
I think one problem that the approach from [Splitting on categorical predictors in random forests](https://peerj.com/articles/6339) has is that it has "target leakage". The values of `y` are used during the construction of the features. This means the features contain information about what the value of `y` is. Same as all the other target encoders. So we need a way to solve that, some form of splitting the data like the cuml `TargetEncoder` does?

Labels: New Feature
Transformer for nominal categories, with the goal of improving category support in decision trees

I'd like to find out how keen people would be for adding a transformer like this.

### Describe the workflow you want to enable

Improved support for nominal categories in tree based models. Nominal categories are ones with no order to them, for example colour or country (c.f. ordinal categories that are ordered).

### Describe your proposed solution

In #12866 @amueller linked to [Splitting on categorical predictors in random forests](https://peerj.com/articles/6339). This proposes to replace categorical values (red, green, blue, ..) by cleverly computed numerical values (blue -> 0.643, green ->0.123, ...) that allow you to achieve similar/equal performance by using `<` as splitting decision in a tree node compared to performing an exhaustive search of all possible categorical splits.

I would implement this as a transformer that you use together with a random forest or other tree based model in a pipeline.

### Describe alternatives you've considered, if relevant

There have been several PRs attempting to add native category support to decision trees. #12866 is the latest one. My impression is that it would be cool to have this in trees but that it is not easy to do, several people have tried but no PR has landed yet. The "Breimann trick" that is used is also limited in the number of categorical values it supports, where this new idea seems to support unlimited categorical values and multi class classification.

### Additional context

An earlier paper that is cited in "Splitting categorical predictors...": https://link.springer.com/article/10.1023/A:1009869804967

An open question for me is how to measure how good this kind of transformation is compared to the exhaustive split (which is "the best possible" but too expensive to compute to be used in practice). The method proposed by the paper seems simple (famous last words), and they make claims about the performance but it would be nice to know how close this gets us. Trust, but verify ;)

COMMENT:
Another big library that contains many different categorical encoders https://contrib.scikit-learn.org/category_encoders/

Labels: New Feature
Regression in 1.2.dev: GenericUnivariateSelect with _parameter_constraints

### Describe the bug

When `mode="k_best"`, previously `param="all"` would be accepted, but that option is not provided in `GenericUnivariateSelect._parameter_constraints`, and so an error is raised.

It's perhaps not a common usecase, and my code that's breaking with this probably should just use `SelectKBest` directly, but I thought I should bring it up.

### Steps/Code to Reproduce

```python
from sklearn.feature_selection import GenericUnivariateSelect
from sklearn.datasets import load_wine
X, y = load_wine(return_X_y=True)

select = GenericUnivariateSelect(mode="k_best", param="all")
select.fit(X, y)
```

### Expected Results

No error is thrown.

### Actual Results

```pytb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\breiniger\Miniconda3\envs\pnl_dev\lib\site-packages\sklearn\feature_selection\_univariate_selection.py", line 465, in fit
    self._validate_params()
  File "C:\Users\breiniger\Miniconda3\envs\pnl_dev\lib\site-packages\sklearn\base.py", line 570, in _validate_params
    validate_parameter_constraints(
  File "C:\Users\breiniger\Miniconda3\envs\pnl_dev\lib\site-packages\sklearn\utils\_param_validation.py", line 95, in validate_parameter_constraints      
    raise ValueError(
ValueError: The 'param' parameter of GenericUnivariateSelect must be a float in the range [0, inf). Got 'all' instead.
```

### Versions

```shell
System:
    python: 3.8.10 (default, May 19 2021, 13:12:57) [MSC v.1916 64 bit (AMD64)]
executable: C:\Users\breiniger\Miniconda3\envs\pnl_dev\python.exe
   machine: Windows-10-10.0.19044-SP0

Python dependencies:
      sklearn: 1.2.dev0
          pip: 21.1.3
   setuptools: 52.0.0.post20210125
        numpy: 1.20.3
        scipy: 1.6.2
       Cython: 0.29.21
       pandas: 1.4.1
   matplotlib: 3.3.4
       joblib: 1.2.0
threadpoolctl: 2.2.0

Built with OpenMP: True

threadpoolctl info:
       filepath: C:\Users\breiniger\Miniconda3\envs\pnl_dev\Lib\site-packages\sklearn\.libs\vcomp140.dll
         prefix: vcomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 8

       filepath: C:\Users\breiniger\Miniconda3\envs\pnl_dev\Library\bin\mkl_rt.1.dll
         prefix: mkl_rt
       user_api: blas
   internal_api: mkl
        version: 2021.3-Product
    num_threads: 4
threading_layer: intel
```

COMMENT:
Yes indeed. `"all"` is a valid input in this case. It means that we should also add a non-regression test.

Labels: Bug, Blocker
Regression in 1.2.dev: GenericUnivariateSelect with _parameter_constraints

### Describe the bug

When `mode="k_best"`, previously `param="all"` would be accepted, but that option is not provided in `GenericUnivariateSelect._parameter_constraints`, and so an error is raised.

It's perhaps not a common usecase, and my code that's breaking with this probably should just use `SelectKBest` directly, but I thought I should bring it up.

### Steps/Code to Reproduce

```python
from sklearn.feature_selection import GenericUnivariateSelect
from sklearn.datasets import load_wine
X, y = load_wine(return_X_y=True)

select = GenericUnivariateSelect(mode="k_best", param="all")
select.fit(X, y)
```

### Expected Results

No error is thrown.

### Actual Results

```pytb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\breiniger\Miniconda3\envs\pnl_dev\lib\site-packages\sklearn\feature_selection\_univariate_selection.py", line 465, in fit
    self._validate_params()
  File "C:\Users\breiniger\Miniconda3\envs\pnl_dev\lib\site-packages\sklearn\base.py", line 570, in _validate_params
    validate_parameter_constraints(
  File "C:\Users\breiniger\Miniconda3\envs\pnl_dev\lib\site-packages\sklearn\utils\_param_validation.py", line 95, in validate_parameter_constraints      
    raise ValueError(
ValueError: The 'param' parameter of GenericUnivariateSelect must be a float in the range [0, inf). Got 'all' instead.
```

### Versions

```shell
System:
    python: 3.8.10 (default, May 19 2021, 13:12:57) [MSC v.1916 64 bit (AMD64)]
executable: C:\Users\breiniger\Miniconda3\envs\pnl_dev\python.exe
   machine: Windows-10-10.0.19044-SP0

Python dependencies:
      sklearn: 1.2.dev0
          pip: 21.1.3
   setuptools: 52.0.0.post20210125
        numpy: 1.20.3
        scipy: 1.6.2
       Cython: 0.29.21
       pandas: 1.4.1
   matplotlib: 3.3.4
       joblib: 1.2.0
threadpoolctl: 2.2.0

Built with OpenMP: True

threadpoolctl info:
       filepath: C:\Users\breiniger\Miniconda3\envs\pnl_dev\Lib\site-packages\sklearn\.libs\vcomp140.dll
         prefix: vcomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 8

       filepath: C:\Users\breiniger\Miniconda3\envs\pnl_dev\Library\bin\mkl_rt.1.dll
         prefix: mkl_rt
       user_api: blas
   internal_api: mkl
        version: 2021.3-Product
    num_threads: 4
threading_layer: intel
```

COMMENT:
Thanks @bmreiniger for reporting before the release :)

Labels: Bug, Blocker
KFold returning folds with different lengths of Train and test splits

### Describe the bug

The sizes of the train and test split change over different splits. 

For example, the size of the IRIS dataset is 150 rows. In 4-fold cross-validation, we would expect to see either 112-38 splits or 113-37 splits for all 4 folds. But actually, we are getting a mix of both. I tested this for another dataset with for 2-fold and 3-fold in which the len(data) is not divisible by 4. The same issue was there.

Would be much appreciated if someone could fix it. 

Why I need this fix is because, in my application, I have to assume that all folds are of equal sizes for a specific reason. 

Thank you!

### Steps/Code to Reproduce

```
from sklearn.model_selection import KFold,StratifiedKFold
from sklearn import datasets

def next_observation():
    iris = datasets.load_iris()
    X = iris.data[:, :2]
    y = iris.target
    n_k_splits = 4

    kf = KFold(n_splits=n_k_splits)


    for train_index, test_index in kf.split(X):
            X_train, X_test = X[train_index], X[test_index]
            y_train, y_test = y[train_index], y[test_index]
            len_tr = X_train.shape[0]
            len_ts = X_test.shape[0]
            print(("-----"))
            print(len_tr)
            print(len_ts)

next_observation()
```

### Expected Results

```
-----
112
38
-----
112
38
-----
112
38
-----
112
38
```

### Actual Results

```
-----
112
38
-----
112
38
-----
113
37
-----
113
37
```

### Versions

```shell
System:
    python: 3.10.6 (main, Oct  7 2022, 15:17:36) [Clang 12.0.0 ]
executable: /Users/nrweerad/opt/anaconda3/envs/mlp/bin/python3
   machine: macOS-12.4-arm64-arm-64bit

Python dependencies:
      sklearn: 1.1.2
          pip: 22.2.2
   setuptools: 63.4.1
        numpy: 1.23.3
        scipy: 1.9.2
       Cython: None
       pandas: 1.5.0
   matplotlib: 3.6.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/nrweerad/opt/anaconda3/envs/mlp/lib/python3.10/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/nrweerad/opt/anaconda3/envs/mlp/lib/python3.10/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.20
threading_layer: pthreads
   architecture: armv8
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/nrweerad/opt/anaconda3/envs/mlp/lib/python3.10/site-packages/scipy/.dylibs/libopenblasp-r0.3.20.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: armv8
    num_threads: 8
```

COMMENT:
150 isn't divisible by 4.

Labels: Bug, Needs Triage
KFold returning folds with different lengths of Train and test splits

### Describe the bug

The sizes of the train and test split change over different splits. 

For example, the size of the IRIS dataset is 150 rows. In 4-fold cross-validation, we would expect to see either 112-38 splits or 113-37 splits for all 4 folds. But actually, we are getting a mix of both. I tested this for another dataset with for 2-fold and 3-fold in which the len(data) is not divisible by 4. The same issue was there.

Would be much appreciated if someone could fix it. 

Why I need this fix is because, in my application, I have to assume that all folds are of equal sizes for a specific reason. 

Thank you!

### Steps/Code to Reproduce

```
from sklearn.model_selection import KFold,StratifiedKFold
from sklearn import datasets

def next_observation():
    iris = datasets.load_iris()
    X = iris.data[:, :2]
    y = iris.target
    n_k_splits = 4

    kf = KFold(n_splits=n_k_splits)


    for train_index, test_index in kf.split(X):
            X_train, X_test = X[train_index], X[test_index]
            y_train, y_test = y[train_index], y[test_index]
            len_tr = X_train.shape[0]
            len_ts = X_test.shape[0]
            print(("-----"))
            print(len_tr)
            print(len_ts)

next_observation()
```

### Expected Results

```
-----
112
38
-----
112
38
-----
112
38
-----
112
38
```

### Actual Results

```
-----
112
38
-----
112
38
-----
113
37
-----
113
37
```

### Versions

```shell
System:
    python: 3.10.6 (main, Oct  7 2022, 15:17:36) [Clang 12.0.0 ]
executable: /Users/nrweerad/opt/anaconda3/envs/mlp/bin/python3
   machine: macOS-12.4-arm64-arm-64bit

Python dependencies:
      sklearn: 1.1.2
          pip: 22.2.2
   setuptools: 63.4.1
        numpy: 1.23.3
        scipy: 1.9.2
       Cython: None
       pandas: 1.5.0
   matplotlib: 3.6.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/nrweerad/opt/anaconda3/envs/mlp/lib/python3.10/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/nrweerad/opt/anaconda3/envs/mlp/lib/python3.10/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.20
threading_layer: pthreads
   architecture: armv8
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/nrweerad/opt/anaconda3/envs/mlp/lib/python3.10/site-packages/scipy/.dylibs/libopenblasp-r0.3.20.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: armv8
    num_threads: 8
```

COMMENT:
@s-banach I agree with your idea. Thanks for the comment. Feel free to close the issue if you do not find it a bug anymore.

Labels: Bug, Needs Triage
KFold returning folds with different lengths of Train and test splits

### Describe the bug

The sizes of the train and test split change over different splits. 

For example, the size of the IRIS dataset is 150 rows. In 4-fold cross-validation, we would expect to see either 112-38 splits or 113-37 splits for all 4 folds. But actually, we are getting a mix of both. I tested this for another dataset with for 2-fold and 3-fold in which the len(data) is not divisible by 4. The same issue was there.

Would be much appreciated if someone could fix it. 

Why I need this fix is because, in my application, I have to assume that all folds are of equal sizes for a specific reason. 

Thank you!

### Steps/Code to Reproduce

```
from sklearn.model_selection import KFold,StratifiedKFold
from sklearn import datasets

def next_observation():
    iris = datasets.load_iris()
    X = iris.data[:, :2]
    y = iris.target
    n_k_splits = 4

    kf = KFold(n_splits=n_k_splits)


    for train_index, test_index in kf.split(X):
            X_train, X_test = X[train_index], X[test_index]
            y_train, y_test = y[train_index], y[test_index]
            len_tr = X_train.shape[0]
            len_ts = X_test.shape[0]
            print(("-----"))
            print(len_tr)
            print(len_ts)

next_observation()
```

### Expected Results

```
-----
112
38
-----
112
38
-----
112
38
-----
112
38
```

### Actual Results

```
-----
112
38
-----
112
38
-----
113
37
-----
113
37
```

### Versions

```shell
System:
    python: 3.10.6 (main, Oct  7 2022, 15:17:36) [Clang 12.0.0 ]
executable: /Users/nrweerad/opt/anaconda3/envs/mlp/bin/python3
   machine: macOS-12.4-arm64-arm-64bit

Python dependencies:
      sklearn: 1.1.2
          pip: 22.2.2
   setuptools: 63.4.1
        numpy: 1.23.3
        scipy: 1.9.2
       Cython: None
       pandas: 1.5.0
   matplotlib: 3.6.1
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: libomp
       filepath: /Users/nrweerad/opt/anaconda3/envs/mlp/lib/python3.10/site-packages/sklearn/.dylibs/libomp.dylib
        version: None
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/nrweerad/opt/anaconda3/envs/mlp/lib/python3.10/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.20
threading_layer: pthreads
   architecture: armv8
    num_threads: 8

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: /Users/nrweerad/opt/anaconda3/envs/mlp/lib/python3.10/site-packages/scipy/.dylibs/libopenblasp-r0.3.20.dylib
        version: 0.3.18
threading_layer: pthreads
   architecture: armv8
    num_threads: 8
```

COMMENT:
This is an expected behaviour. Thanks @s-banach for the detailed explanations.
Closing.

Labels: Bug, Needs Triage
Bug in fetch_lfw_people() function

### Describe the bug

There is a bug on line 162 of _lfw.py. That line currently reads:

`pil_img.crop((w_slice.start, h_slice.start, w_slice.stop, h_slice.stop))`

It should read:

`pil_img = pil_img.crop((w_slice.start, h_slice.start, w_slice.stop, h_slice.stop))`

Because the cropped image isn't assigned back to `pil_img`, the `slice_` parameter passed to `fetch_lfw_people` is ignored. Consequently, `fetch_lfw_people` always returns uncropped images.


### Steps/Code to Reproduce

```python
# The following code should display 24 neatly cropped facial images if run in a Jupyter notebook.
# Instead, it shows the full, uncropped facial images.

%matplotlib inline
import matplotlib.pyplot as plt
from sklearn.datasets import fetch_lfw_people

faces = fetch_lfw_people(min_faces_per_person=100)

fig, ax = plt.subplots(3, 8, figsize=(18, 10))
for i, axi in enumerate(ax.flat):
    axi.imshow(faces.images[i], cmap='gist_gray')
   axi.set(xticks=[], yticks=[], xlabel=faces.target_names[faces.target[i]])
```

### Expected Results

![image](https://user-images.githubusercontent.com/13542501/201976958-18e76326-300f-4701-a98d-87c9c8554735.png)

### Actual Results


![image](https://user-images.githubusercontent.com/13542501/201977665-3cb3ee79-a4f0-4ad5-8aeb-097be55d962d.png)


### Versions

```shell
System:
    python: 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)]
executable: c:\users\jeffp\appdata\local\programs\python\python38\python.exe
   machine: Windows-10-10.0.22000-SP0

Python dependencies:
      sklearn: 1.1.1
          pip: 22.1.1
   setuptools: 62.3.2
        numpy: 1.22.4
        scipy: 1.4.1
       Cython: 0.29.28
       pandas: 1.2.5
   matplotlib: 3.3.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: C:\Users\jeffp\AppData\Local\Programs\Python\Python38\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 20

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\Users\jeffp\AppData\Local\Programs\Python\Python38\Lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell
    num_threads: 20

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\Users\jeffp\AppData\Local\Programs\Python\Python38\Lib\site-packages\scipy\.libs\libopenblas.PYQHXLVVQ7VESDPUVUADXEVJOBGHJPAY.gfortran-win_amd64.dll
        version: 0.3.7
threading_layer: pthreads
   architecture: Prescott
    num_threads: 20
```

COMMENT:
Indeed, this happen when we moved from array slicing to cropping using PIL.
So we release this bug in 1.1 so we need a non-regression test and an entry in the changelog.
Tagging as blocker to incorporate in 1.2.

Labels: Bug, Blocker