Talip7/scikit-learn-issues-multilabel huggingface dataset:
Bug in fetch_lfw_people() function

### Describe the bug

There is a bug on line 162 of _lfw.py. That line currently reads:

`pil_img.crop((w_slice.start, h_slice.start, w_slice.stop, h_slice.stop))`

It should read:

`pil_img = pil_img.crop((w_slice.start, h_slice.start, w_slice.stop, h_slice.stop))`

Because the cropped image isn't assigned back to `pil_img`, the `slice_` parameter passed to `fetch_lfw_people` is ignored. Consequently, `fetch_lfw_people` always returns uncropped images.


### Steps/Code to Reproduce

```python
# The following code should display 24 neatly cropped facial images if run in a Jupyter notebook.
# Instead, it shows the full, uncropped facial images.

%matplotlib inline
import matplotlib.pyplot as plt
from sklearn.datasets import fetch_lfw_people

faces = fetch_lfw_people(min_faces_per_person=100)

fig, ax = plt.subplots(3, 8, figsize=(18, 10))
for i, axi in enumerate(ax.flat):
    axi.imshow(faces.images[i], cmap='gist_gray')
   axi.set(xticks=[], yticks=[], xlabel=faces.target_names[faces.target[i]])
```

### Expected Results

![image](https://user-images.githubusercontent.com/13542501/201976958-18e76326-300f-4701-a98d-87c9c8554735.png)

### Actual Results


![image](https://user-images.githubusercontent.com/13542501/201977665-3cb3ee79-a4f0-4ad5-8aeb-097be55d962d.png)


### Versions

```shell
System:
    python: 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)]
executable: c:\users\jeffp\appdata\local\programs\python\python38\python.exe
   machine: Windows-10-10.0.22000-SP0

Python dependencies:
      sklearn: 1.1.1
          pip: 22.1.1
   setuptools: 62.3.2
        numpy: 1.22.4
        scipy: 1.4.1
       Cython: 0.29.28
       pandas: 1.2.5
   matplotlib: 3.3.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: C:\Users\jeffp\AppData\Local\Programs\Python\Python38\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 20

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\Users\jeffp\AppData\Local\Programs\Python\Python38\Lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell
    num_threads: 20

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\Users\jeffp\AppData\Local\Programs\Python\Python38\Lib\site-packages\scipy\.libs\libopenblas.PYQHXLVVQ7VESDPUVUADXEVJOBGHJPAY.gfortran-win_amd64.dll
        version: 0.3.7
threading_layer: pthreads
   architecture: Prescott
    num_threads: 20
```

COMMENT:
Awesome! Thanks for the fast response!

Labels: Bug, Blocker
Add __get_item__() to FeatureUnion

### Describe the bug

When debugging larger pipelines for a consistent syntax we’d like to reference individual sub components of `FeatureUnion` by index, slice or tuple name in the same way as the `Pipeline`. This way user does not need to know the types of the aggregate components.


### Steps/Code to Reproduce

for example:
```python
model = Pipeline(steps=[
   (“step1”, Component()),
   (“step2” FeatureUnion(transformer_list=[
       (“step2a”, Component()),
       (“step2b”, Component()),
   ])),
])
```

### Expected Results

would be accessed as:
```python
comp = model[“step2”][“step2a”]
```

### Actual Results

rather than accessing as:
```python
comp = model[“step2”].transformer_list[0][1]  # transformer_list is a implementation detail to FeatureUnion only.
```

### Versions

```shell
Latest
```

COMMENT:
Looks like a nice UX improvement to me.

Labels: New Feature, Needs Decision - Include Feature
Add __get_item__() to FeatureUnion

### Describe the bug

When debugging larger pipelines for a consistent syntax we’d like to reference individual sub components of `FeatureUnion` by index, slice or tuple name in the same way as the `Pipeline`. This way user does not need to know the types of the aggregate components.


### Steps/Code to Reproduce

for example:
```python
model = Pipeline(steps=[
   (“step1”, Component()),
   (“step2” FeatureUnion(transformer_list=[
       (“step2a”, Component()),
       (“step2b”, Component()),
   ])),
])
```

### Expected Results

would be accessed as:
```python
comp = model[“step2”][“step2a”]
```

### Actual Results

rather than accessing as:
```python
comp = model[“step2”].transformer_list[0][1]  # transformer_list is a implementation detail to FeatureUnion only.
```

### Versions

```shell
Latest
```

COMMENT:
Would be similar to `Pipeline`, would be nice to have.

Labels: New Feature, Needs Decision - Include Feature
Inconsistency between GridSearchCV 'roc_auc' scoring and roc_auc_score metric

### Describe the bug

I am using GridSearchCV and obtaining different ROC_AUC scores. From my understanding of the documentation and source code, passing 'roc_auc' as a scorer uses the metric.roc_auc_score function. However, passing in the metric scorer and the string result in two different scores (0.833 and 1.0).

I have replicated the metric score by manually doing the gridCV steps including cross-validation and classifying.

### Steps/Code to Reproduce

```
from sklearn.datasets import make_classification
from sklearn.metrics import roc_auc_score, make_scorer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV,StratifiedKFold
from sklearn.pipeline import Pipeline

X,y = make_classification()

lScorers = {'AUC_func':make_scorer(roc_auc_score),'AUC_str':'roc_auc','accuracy':'accuracy'}

pipe = Pipeline(steps=[('clf', None)])
params = [{'clf': [RandomForestClassifier()],'clf__n_estimators': [10],'clf__max_depth': [3],'clf__random_state': [3]}]
clf = GridSearchCV(pipe,param_grid=params,scoring=lScorers, refit='accuracy', n_jobs=-1, cv= StratifiedKFold(3))
clf.fit(X,y)

print(f"AUC From makescorer(roc_auc_score): {clf.cv_results_['mean_test_AUC_func'][0]: .4f}")
print(f"AUC From string roc_auc: {clf.cv_results_['mean_test_AUC_str'][0]:.4f}")
```

### Expected Results

Both AUCs should be the same as they are both using the same scoring function.

### Actual Results

![Screen Shot 2022-11-12 at 5 49 57 PM](https://user-images.githubusercontent.com/10987493/201497580-d7656dbf-e9c0-4dfe-aa8c-07ae6ecfbafa.jpg)


### Versions

```shell
System:
    python: 3.10.0 (default, Mar  3 2022, 09:58:08) [GCC 7.5.0]
executable: /root/anaconda3/envs/py10/bin/python
   machine: Linux-5.15.35-1-pve-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.1.2
          pip: 22.2.2
   setuptools: 65.5.0
        numpy: 1.22.3
        scipy: 1.7.3
       Cython: None
       pandas: 1.4.4
   matplotlib: 3.5.3
       joblib: 1.1.1
threadpoolctl: 2.2.0

Built with OpenMP: True

threadpoolctl info:
       filepath: /root/anaconda3/envs/py10/lib/libmkl_rt.so.1
         prefix: libmkl_rt
       user_api: blas
   internal_api: mkl
        version: 2021.4-Product
    num_threads: 24
threading_layer: intel

       filepath: /root/anaconda3/envs/py10/lib/libiomp5.so
         prefix: libiomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 28

       filepath: /root/anaconda3/envs/py10/lib/libgomp.so.1.0.0
         prefix: libgomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 28
```

COMMENT:
You should pass `needs_thresholds=True` in `make_scorer` and you will get the equivalence. The metrics is based on non-thresholded scores, this is the reason why you need this parameter.

Labels: Bug, Needs Triage
Inconsistency between GridSearchCV 'roc_auc' scoring and roc_auc_score metric

### Describe the bug

I am using GridSearchCV and obtaining different ROC_AUC scores. From my understanding of the documentation and source code, passing 'roc_auc' as a scorer uses the metric.roc_auc_score function. However, passing in the metric scorer and the string result in two different scores (0.833 and 1.0).

I have replicated the metric score by manually doing the gridCV steps including cross-validation and classifying.

### Steps/Code to Reproduce

```
from sklearn.datasets import make_classification
from sklearn.metrics import roc_auc_score, make_scorer
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import GridSearchCV,StratifiedKFold
from sklearn.pipeline import Pipeline

X,y = make_classification()

lScorers = {'AUC_func':make_scorer(roc_auc_score),'AUC_str':'roc_auc','accuracy':'accuracy'}

pipe = Pipeline(steps=[('clf', None)])
params = [{'clf': [RandomForestClassifier()],'clf__n_estimators': [10],'clf__max_depth': [3],'clf__random_state': [3]}]
clf = GridSearchCV(pipe,param_grid=params,scoring=lScorers, refit='accuracy', n_jobs=-1, cv= StratifiedKFold(3))
clf.fit(X,y)

print(f"AUC From makescorer(roc_auc_score): {clf.cv_results_['mean_test_AUC_func'][0]: .4f}")
print(f"AUC From string roc_auc: {clf.cv_results_['mean_test_AUC_str'][0]:.4f}")
```

### Expected Results

Both AUCs should be the same as they are both using the same scoring function.

### Actual Results

![Screen Shot 2022-11-12 at 5 49 57 PM](https://user-images.githubusercontent.com/10987493/201497580-d7656dbf-e9c0-4dfe-aa8c-07ae6ecfbafa.jpg)


### Versions

```shell
System:
    python: 3.10.0 (default, Mar  3 2022, 09:58:08) [GCC 7.5.0]
executable: /root/anaconda3/envs/py10/bin/python
   machine: Linux-5.15.35-1-pve-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.1.2
          pip: 22.2.2
   setuptools: 65.5.0
        numpy: 1.22.3
        scipy: 1.7.3
       Cython: None
       pandas: 1.4.4
   matplotlib: 3.5.3
       joblib: 1.1.1
threadpoolctl: 2.2.0

Built with OpenMP: True

threadpoolctl info:
       filepath: /root/anaconda3/envs/py10/lib/libmkl_rt.so.1
         prefix: libmkl_rt
       user_api: blas
   internal_api: mkl
        version: 2021.4-Product
    num_threads: 24
threading_layer: intel

       filepath: /root/anaconda3/envs/py10/lib/libiomp5.so
         prefix: libiomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 28

       filepath: /root/anaconda3/envs/py10/lib/libgomp.so.1.0.0
         prefix: libgomp
       user_api: openmp
   internal_api: openmp
        version: None
    num_threads: 28
```

COMMENT:
Interesting, thank you for the clarification! Cheers.

Labels: Bug, Needs Triage
Inconsistent results with HalvingGridSearchCV

### Describe the bug

`best_params_` does not match top `rank_test_scores` in`cv_results_`.

### Steps/Code to Reproduce

Please see https://stackoverflow.com/q/74408418/4996152

### Expected Results

Expect best params to be top ranked result.

### Actual Results

The output of `best_params_` is not within the top 4 `rank_test_scores` in`cv_results_`.

### Versions

```shell
System:
    python: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.7.3
       Cython: 0.29.32
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
Could you provide a minimum example with simple data such that we can reproduce the issue.

Labels: Documentation, help wanted, module:model_selection
Inconsistent results with HalvingGridSearchCV

### Describe the bug

`best_params_` does not match top `rank_test_scores` in`cv_results_`.

### Steps/Code to Reproduce

Please see https://stackoverflow.com/q/74408418/4996152

### Expected Results

Expect best params to be top ranked result.

### Actual Results

The output of `best_params_` is not within the top 4 `rank_test_scores` in`cv_results_`.

### Versions

```shell
System:
    python: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.7.3
       Cython: 0.29.32
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
That is going to be hard. I am working on the Titanic dataset but I run a lengthy pipeline to get to training and test data. Can I attach a notebook that you can run?

Labels: Documentation, help wanted, module:model_selection
Inconsistent results with HalvingGridSearchCV

### Describe the bug

`best_params_` does not match top `rank_test_scores` in`cv_results_`.

### Steps/Code to Reproduce

Please see https://stackoverflow.com/q/74408418/4996152

### Expected Results

Expect best params to be top ranked result.

### Actual Results

The output of `best_params_` is not within the top 4 `rank_test_scores` in`cv_results_`.

### Versions

```shell
System:
    python: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.7.3
       Cython: 0.29.32
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
I could put my training set out on github and just provide code from there if that helps.

Labels: Documentation, help wanted, module:model_selection
Inconsistent results with HalvingGridSearchCV

### Describe the bug

`best_params_` does not match top `rank_test_scores` in`cv_results_`.

### Steps/Code to Reproduce

Please see https://stackoverflow.com/q/74408418/4996152

### Expected Results

Expect best params to be top ranked result.

### Actual Results

The output of `best_params_` is not within the top 4 `rank_test_scores` in`cv_results_`.

### Versions

```shell
System:
    python: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.7.3
       Cython: 0.29.32
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
yep until this is a file and a possibly short snippet of code that reproduce the issue, then it will be helpful for debugging.

Labels: Documentation, help wanted, module:model_selection
Inconsistent results with HalvingGridSearchCV

### Describe the bug

`best_params_` does not match top `rank_test_scores` in`cv_results_`.

### Steps/Code to Reproduce

Please see https://stackoverflow.com/q/74408418/4996152

### Expected Results

Expect best params to be top ranked result.

### Actual Results

The output of `best_params_` is not within the top 4 `rank_test_scores` in`cv_results_`.

### Versions

```shell
System:
    python: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.7.3
       Cython: 0.29.32
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
I am leaving you a link to a colab notebook that demonstrates the problem.

https://colab.research.google.com/drive/1b7mSvj-4k_Jo90HBwf-qP2M0Yog0NZCi?usp=sharing

Labels: Documentation, help wanted, module:model_selection
Inconsistent results with HalvingGridSearchCV

### Describe the bug

`best_params_` does not match top `rank_test_scores` in`cv_results_`.

### Steps/Code to Reproduce

Please see https://stackoverflow.com/q/74408418/4996152

### Expected Results

Expect best params to be top ranked result.

### Actual Results

The output of `best_params_` is not within the top 4 `rank_test_scores` in`cv_results_`.

### Versions

```shell
System:
    python: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.7.3
       Cython: 0.29.32
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
Thanks for making the notebook. It looks like the test scores for all parameter combinations is NaN. This means there is no good way to rank the combinations. All you can really say is that combinations with a test score of NaN are all equally bad, and should be ranked last. However, the last release of scikit-learn didn't implement this behaviour.

Can you try the latest scikit-learn from `main`? In https://github.com/scikit-learn/scikit-learn/pull/24539 we tried to fix how NaN test scores are handled to implement the above ("all combos with NaN share the lowest rank")

Labels: Documentation, help wanted, module:model_selection
Inconsistent results with HalvingGridSearchCV

### Describe the bug

`best_params_` does not match top `rank_test_scores` in`cv_results_`.

### Steps/Code to Reproduce

Please see https://stackoverflow.com/q/74408418/4996152

### Expected Results

Expect best params to be top ranked result.

### Actual Results

The output of `best_params_` is not within the top 4 `rank_test_scores` in`cv_results_`.

### Versions

```shell
System:
    python: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.7.3
       Cython: 0.29.32
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
> Can you try the latest scikit-learn from main?

In case you don't want to build from source, you can install the nightly build: https://scikit-learn.org/stable/developers/advanced_installation.html#installing-nightly-builds

Labels: Documentation, help wanted, module:model_selection
Inconsistent results with HalvingGridSearchCV

### Describe the bug

`best_params_` does not match top `rank_test_scores` in`cv_results_`.

### Steps/Code to Reproduce

Please see https://stackoverflow.com/q/74408418/4996152

### Expected Results

Expect best params to be top ranked result.

### Actual Results

The output of `best_params_` is not within the top 4 `rank_test_scores` in`cv_results_`.

### Versions

```shell
System:
    python: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.7.3
       Cython: 0.29.32
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
Thanks for those thoughtful comments. If I understand correctly, I easily could see higher MTS on prior iterations given they come from reduced sample size. Hence, I should ignore the table and just concentrate on the last iteration. But you already do that with `best_params_`.

What I don't understand, and is probably a separate thread, is why I am getting nan values with KNN. KNN does have a `predict_proba` method so it seems I can calculate auc. So not sure what I am missing there. I also commented out the socring line (`#scoring="roc_auc"`), which I believe then defaults to the `score` method on KNN. Still getting warnings and nans. If someone asked about all these warnings and you answered in another post, please give me the link, thanks.

Labels: Documentation, help wanted, module:model_selection
Inconsistent results with HalvingGridSearchCV

### Describe the bug

`best_params_` does not match top `rank_test_scores` in`cv_results_`.

### Steps/Code to Reproduce

Please see https://stackoverflow.com/q/74408418/4996152

### Expected Results

Expect best params to be top ranked result.

### Actual Results

The output of `best_params_` is not within the top 4 `rank_test_scores` in`cv_results_`.

### Versions

```shell
System:
    python: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.7.3
       Cython: 0.29.32
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
I have no good answer/knowledge of why the warnings occur :-/ I think the best way to figure it out would be to run the example in a debugger with a break point somewhere "far enough before but not too far before" the place the warning is raised. Then look around and try to understand why it happens. From making a change to the code a while back I remember that you have to have an agile mind to be able to follow what is happening where.

I think an actionable thing to do for this issue is to think about how to improve the documentation for `HalvingGridSearchCV` and friends. Or maybe add some functions/methods to help with interpreting the results. If you count the issue from a while back that pointed out that the ranking wasn't quite right when NaNs were present and this issue, then you have two in a short span of time that are at least semi related to the fact that it is tricky to understand what is what in the results from `HalvingGridSearchCV`. Three instances if you count the fact that it  took me a moment of poking around to realise what was happening, even though I worked on a the code a few weeks ago.

Labels: Documentation, help wanted, module:model_selection
Inconsistent results with HalvingGridSearchCV

### Describe the bug

`best_params_` does not match top `rank_test_scores` in`cv_results_`.

### Steps/Code to Reproduce

Please see https://stackoverflow.com/q/74408418/4996152

### Expected Results

Expect best params to be top ranked result.

### Actual Results

The output of `best_params_` is not within the top 4 `rank_test_scores` in`cv_results_`.

### Versions

```shell
System:
    python: 3.7.15 (default, Oct 12 2022, 19:14:55)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.10.133+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.6
        scipy: 1.7.3
       Cython: 0.29.32
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
Hi!

Just came across this issue investigating why the best score was not the rank 1 in the cv_results dictionary. I think it would make sense to specify this in the documentation (https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.HalvingGridSearchCV.html). In the `best_estimator_` attribute, something in the lines of "_Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data **in the last iteration of the gridsearch**._"

What do you think? I can propose the PR

Labels: Documentation, help wanted, module:model_selection