Talip7/scikit-learn-issues-multilabel huggingface dataset:
load_iris documentation target_names name wrong type

### Describe the issue linked to the documentation

In the documentation of load_iris (https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_iris.html) the type of target_names is list but in code it's a numpyarray.

**Version Checked**
Version: 1.6.1

### Suggest a potential alternative/fix

Hello i'm new to the opensource world so this would be my first issue raised.

There would be two way to fix it : either change the documentation to reflect the type of the data or change the data to type to be in line with the feature_names and the documentation (a list)

COMMENT:
Happy to see the documentation fixed. PR welcome.

Labels: Documentation
Attributes decleared in document and does not exist in ConfusionMatrixDisplay class

### Describe the issue linked to the documentation

In the class `ConfusionMatrixDisplay` in the file `sklearn/metrics/_plot/confusion_matrix.py`
There are extra attributes that does not exist in the class

Attributes:
```
im_: matplotlib AxesImage Image representing the confusion matrix.

text_: ndarray of shape (n_classes, n_classes), dtype=matplotlib Text, or None Array of matplotlib axes.
  None if include_values is false.

ax_:matplotlib Axes Axes with confusion matrix.

figure_:matplotlib Figure Figure containing the confusion matrix.
```

### Suggest a potential alternative/fix

_No response_

COMMENT:
The attributes im_, text_, ax_, and figure_ do exist in the ConfusionMatrixDisplay class, but only after the plot method is called. 

Ambiguity in Documentation: The attributes are listed without specifying that they are populated post-plot, leading users to expect them immediately after instantiation.
Possible Typo: The text_ description contains an error ("axes" instead of "Text"), adding to potential confusion.
Suggested Fix
To resolve this issue, the documentation should be updated for clarity. Here’s a revised version of the "Attributes" section:

Steps to reproduce
```
from sklearn.metrics import ConfusionMatrixDisplay
display = ConfusionMatrixDisplay([[1, 2], [3, 4]])
print(display.im_)  # This fails because plot() hasn't been called yet

```

```
Attributes (available after calling `plot`):
    im_ : matplotlib AxesImage
        Image representing the confusion matrix. Set after `plot` is called.
    text_ : ndarray of shape (n_classes, n_classes), dtype=matplotlib Text, or None
        Array of matplotlib Text objects displaying matrix values. Set after `plot` is called; None if `include_values=False`.
    ax_ : matplotlib Axes
        Axes containing the confusion matrix. Set after `plot` is called.
    figure_ : matplotlib Figure
        Figure containing the confusion matrix. Set after `plot` is called.
```

Labels: Documentation, Needs Triage
Attributes decleared in document and does not exist in ConfusionMatrixDisplay class

### Describe the issue linked to the documentation

In the class `ConfusionMatrixDisplay` in the file `sklearn/metrics/_plot/confusion_matrix.py`
There are extra attributes that does not exist in the class

Attributes:
```
im_: matplotlib AxesImage Image representing the confusion matrix.

text_: ndarray of shape (n_classes, n_classes), dtype=matplotlib Text, or None Array of matplotlib axes.
  None if include_values is false.

ax_:matplotlib Axes Axes with confusion matrix.

figure_:matplotlib Figure Figure containing the confusion matrix.
```

### Suggest a potential alternative/fix

_No response_

COMMENT:
That's a very normal patter in scikit-learn. Attributes ending with `_` are the ones which are created after the object is "fitted" or a plotting function is called. They don't exist on the object after `__init__`.

Labels: Documentation, Needs Triage
Make the halving searches scoring parameter also accept single value containers

Currently, the HalvingRandomSearchCV and the HalvingGridSearchCV support only a single scoring metric. Because of that, only a single string or callable is accepted as scoring parameter.
However, this causes it to not accept a single metric if it's wrapped in a container (meaning a list or tuple with only one element or a dict with only one key-value pair).

While in the long run, the halving search variants should also accept and use multiple scoring metrics, it would still be a good improvement for consistency and for frameworks that use the different searches interchangeably if the halving search variants could also accept those containers as long as they contain just one element.

COMMENT:
Could you add some code to your post to explain a bit what you are trying to do. Right now I am not sure I am following.

Labels: New Feature, Needs Decision - Close, Needs Investigation
Make the halving searches scoring parameter also accept single value containers

Currently, the HalvingRandomSearchCV and the HalvingGridSearchCV support only a single scoring metric. Because of that, only a single string or callable is accepted as scoring parameter.
However, this causes it to not accept a single metric if it's wrapped in a container (meaning a list or tuple with only one element or a dict with only one key-value pair).

While in the long run, the halving search variants should also accept and use multiple scoring metrics, it would still be a good improvement for consistency and for frameworks that use the different searches interchangeably if the halving search variants could also accept those containers as long as they contain just one element.

COMMENT:
Yeah of course:

```python
from sklearn.ensemble import RandomForestRegressor
from sklearn.experimental import enable_halving_search_cv
from sklearn.model_selection import HalvingRandomSearchCV, RandomizedSearchCV

import numpy as np

# Define the data points and search parameters
input_values = np.random.rand(100, 10)
output_values = np.random.rand(100, 4)
model = RandomForestRegressor()
hyperparameter_grid = {"n_estimators": [10, 100]}

# This works (Normal search, single metric as string)
random_search = RandomizedSearchCV(model, hyperparameter_grid, scoring="r2")
random_search.fit(input_values, output_values)

# This works (Halving search, single metric as string)
random_search = HalvingRandomSearchCV(model, hyperparameter_grid, scoring="r2")
random_search.fit(input_values, output_values)

# This works (Normal search, single metric as dict)
random_search = RandomizedSearchCV(model, hyperparameter_grid, scoring={"r2_score": "r2"}, refit=False)
random_search.fit(input_values, output_values)

# This doesn't work (Halving search, single metric as dict)
random_search = HalvingRandomSearchCV(model, hyperparameter_grid, scoring={"r2_score": "r2"}, refit=False)
random_search.fit(input_values, output_values)
```

(Obviously, a dict with multiple entries also doesn't work with a halving search, but while it would be great to have that working in the long run, this issue is about it not accepting a single parameter in a container like a dict, tuple or list.)

Labels: New Feature, Needs Decision - Close, Needs Investigation
Make the halving searches scoring parameter also accept single value containers

Currently, the HalvingRandomSearchCV and the HalvingGridSearchCV support only a single scoring metric. Because of that, only a single string or callable is accepted as scoring parameter.
However, this causes it to not accept a single metric if it's wrapped in a container (meaning a list or tuple with only one element or a dict with only one key-value pair).

While in the long run, the halving search variants should also accept and use multiple scoring metrics, it would still be a good improvement for consistency and for frameworks that use the different searches interchangeably if the halving search variants could also accept those containers as long as they contain just one element.

COMMENT:
@Korne127 @lesteve  Hello, I would like to work on the issue. I am also facing the same issue. Thank you.

Labels: New Feature, Needs Decision - Close, Needs Investigation
Make the halving searches scoring parameter also accept single value containers

Currently, the HalvingRandomSearchCV and the HalvingGridSearchCV support only a single scoring metric. Because of that, only a single string or callable is accepted as scoring parameter.
However, this causes it to not accept a single metric if it's wrapped in a container (meaning a list or tuple with only one element or a dict with only one key-value pair).

While in the long run, the halving search variants should also accept and use multiple scoring metrics, it would still be a good improvement for consistency and for frameworks that use the different searches interchangeably if the halving search variants could also accept those containers as long as they contain just one element.

COMMENT:
Thanks for the notice and ping :)

I don't think anyone else is working on this (but I'm no project maintainer, so I can't say this for sure). Nevertheless, I'd definitely appreciate it, so if you want to do this, feel free!

Labels: New Feature, Needs Decision - Close, Needs Investigation
Make the halving searches scoring parameter also accept single value containers

Currently, the HalvingRandomSearchCV and the HalvingGridSearchCV support only a single scoring metric. Because of that, only a single string or callable is accepted as scoring parameter.
However, this causes it to not accept a single metric if it's wrapped in a container (meaning a list or tuple with only one element or a dict with only one key-value pair).

While in the long run, the halving search variants should also accept and use multiple scoring metrics, it would still be a good improvement for consistency and for frameworks that use the different searches interchangeably if the halving search variants could also accept those containers as long as they contain just one element.

COMMENT:
Looking over "Needs Info" issues, I am only getting back to this now 😓.

I tweaked your snippet to make more sense (in particular you need `refit` parameters when using multiple metrics).

About having being able to provide a `dict` with a single entry and having it "just work", I am not really convinced, it would complicate the parameter checking which is already quite hairy IIRC. Putting the burden on the user for the "dict with a single entry" case is not great but seems acceptable.

About the longer-term support of multiple metrics for HalvingSearchCV, reading the [user guide](https://scikit-learn.org/stable/modules/grid_search.html#searching-for-optimal-parameters-with-successive-halving) it's not clear to me how you would select the candidates for HalvingSearchCV with multiple metrics. https://github.com/scikit-learn/scikit-learn/pull/13900 may have more info about the choice of a single metrics, but it is a very long PR so finding this kind of information may be tricky 😅.

Labels: New Feature, Needs Decision - Close, Needs Investigation
enh: support aggregation/bagging functions other than mean

### Describe the workflow you want to enable

Currently KNNRegressor, BaggingRegressor and ForestRegressor only support mean

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/neighbors/_regression.py#L254-L262

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/ensemble/_bagging.py#L1295

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/ensemble/_forest.py#L1079-L1084



### Describe your proposed solution

BaggingRegressor and ForestRegressor could also support median and custom aggregation function that user specifies, but will be mean by default to ensure no breaking change.

For eg:

```python
RandomForestRegressor(..., agg="median", ...)
RandomForestRegressor(..., agg=foo, ...) # where foo is user-defined function
RandomForestRegressor(..., ...) # defaults to mean
```

Proposed implementation

```python
if type(agg) == str:
     agg = getattr(np, agg) # where np is numpy
# else agg is a function, so no transformation required

y_hat = agg(all_y_hat)
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

**Why?**
Robustness to inaccuracies of individual estimators.

Consider individual predictions are 101, 102, 103, 104, 150.
- With mean aggregation, the output will be 112 (current implementation)
- With median aggregation, the output will be 103

COMMENT:
Thanks for the issue, could you give more context and in particular why you think this is important and/or give some links to scientific articles that indicate this is useful in practice?

Labels: New Feature
enh: support aggregation/bagging functions other than mean

### Describe the workflow you want to enable

Currently KNNRegressor, BaggingRegressor and ForestRegressor only support mean

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/neighbors/_regression.py#L254-L262

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/ensemble/_bagging.py#L1295

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/ensemble/_forest.py#L1079-L1084



### Describe your proposed solution

BaggingRegressor and ForestRegressor could also support median and custom aggregation function that user specifies, but will be mean by default to ensure no breaking change.

For eg:

```python
RandomForestRegressor(..., agg="median", ...)
RandomForestRegressor(..., agg=foo, ...) # where foo is user-defined function
RandomForestRegressor(..., ...) # defaults to mean
```

Proposed implementation

```python
if type(agg) == str:
     agg = getattr(np, agg) # where np is numpy
# else agg is a function, so no transformation required

y_hat = agg(all_y_hat)
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

**Why?**
Robustness to inaccuracies of individual estimators.

Consider individual predictions are 101, 102, 103, 104, 150.
- With mean aggregation, the output will be 112 (current implementation)
- With median aggregation, the output will be 103

COMMENT:
**Usecase**

> Why?
> Robustness to inaccuracies of individual estimators.
>
> Consider individual predictions are 101, 102, 103, 104, 150.
>
> With mean aggregation, the output will be 112 (current implementation)
> With median aggregation, the output will be 103

This is similar to #9553, but it was only for RandomForest. I'm requesting for all the above mentioned estimators.

**Scientific Papers**

I did not make this request referring to any paper. If I find any, I'll cite here ASAP.

Labels: New Feature
enh: support aggregation/bagging functions other than mean

### Describe the workflow you want to enable

Currently KNNRegressor, BaggingRegressor and ForestRegressor only support mean

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/neighbors/_regression.py#L254-L262

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/ensemble/_bagging.py#L1295

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/ensemble/_forest.py#L1079-L1084



### Describe your proposed solution

BaggingRegressor and ForestRegressor could also support median and custom aggregation function that user specifies, but will be mean by default to ensure no breaking change.

For eg:

```python
RandomForestRegressor(..., agg="median", ...)
RandomForestRegressor(..., agg=foo, ...) # where foo is user-defined function
RandomForestRegressor(..., ...) # defaults to mean
```

Proposed implementation

```python
if type(agg) == str:
     agg = getattr(np, agg) # where np is numpy
# else agg is a function, so no transformation required

y_hat = agg(all_y_hat)
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

**Why?**
Robustness to inaccuracies of individual estimators.

Consider individual predictions are 101, 102, 103, 104, 150.
- With mean aggregation, the output will be 112 (current implementation)
- With median aggregation, the output will be 103

COMMENT:
Hi @lesteve, hope you are doing well.

I was not able to find any papers for this.

If it is not a good idea to implement this in sciki-learn, we can close this issue to keep the issues list clean.

Those interested can create a custom estimator on their own using my proposed logic.

Labels: New Feature
enh: support aggregation/bagging functions other than mean

### Describe the workflow you want to enable

Currently KNNRegressor, BaggingRegressor and ForestRegressor only support mean

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/neighbors/_regression.py#L254-L262

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/ensemble/_bagging.py#L1295

https://github.com/scikit-learn/scikit-learn/blob/98ed9dc73a86f5f11781a0e21f24c8f47979ec67/sklearn/ensemble/_forest.py#L1079-L1084



### Describe your proposed solution

BaggingRegressor and ForestRegressor could also support median and custom aggregation function that user specifies, but will be mean by default to ensure no breaking change.

For eg:

```python
RandomForestRegressor(..., agg="median", ...)
RandomForestRegressor(..., agg=foo, ...) # where foo is user-defined function
RandomForestRegressor(..., ...) # defaults to mean
```

Proposed implementation

```python
if type(agg) == str:
     agg = getattr(np, agg) # where np is numpy
# else agg is a function, so no transformation required

y_hat = agg(all_y_hat)
```

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

**Why?**
Robustness to inaccuracies of individual estimators.

Consider individual predictions are 101, 102, 103, 104, 150.
- With mean aggregation, the output will be 112 (current implementation)
- With median aggregation, the output will be 103

COMMENT:
Thanks, let's close this then.

Labels: New Feature
The estimators_ attribute can no longer be accessed for the AdaBoostClassifier class

### Describe the bug

The [documentation of the AdaBoostClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html) reads that there is a `estimators_` attribute. However, if I try to access this attribute, I get the error

> AttributeError: 'AdaBoostClassifier' object has no attribute 'estimators_'

### Steps/Code to Reproduce

```python
from sklearn.datasets import load_digits
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import AdaBoostClassifier

# Load dataset
digits = load_digits()
X = digits.data
y = digits.target

# Initialize AdaBoost classifier
ada_clf = AdaBoostClassifier(n_estimators=n_estimators)
scores = cross_val_score(ada_clf, X, y, cv=5)

ada_clf.estimators_
```

### Expected Results

No error is shown

### Actual Results

```bash
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[29], line 14
     11 ada_clf = AdaBoostClassifier(n_estimators=n_estimators)
     12 scores = cross_val_score(ada_clf, X, y, cv=5)
---> 14 ada_clf.estimators_

AttributeError: 'AdaBoostClassifier' object has no attribute 'estimators_'
```

### Versions

```shell
System:
    python: 3.11.11 | packaged by conda-forge | (main, Mar  3 2025, 20:43:55) [GCC 13.3.0]
executable: /home/kkladny/psi4conda/envs/adaboost/bin/python
   machine: Linux-6.8.0-52-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.6.1
          pip: 25.0.1
   setuptools: 75.8.2
        numpy: 2.2.3
        scipy: 1.15.2
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.1
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/kkladny/psi4conda/envs/adaboost/lib/python3.11/site-packages/numpy.libs/libscipy_openblas64_-6bb31eeb.so
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/kkladny/psi4conda/envs/adaboost/lib/python3.11/site-packages/scipy.libs/libscipy_openblas-68440149.so
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 20
         prefix: libgomp
       filepath: /home/kkladny/psi4conda/envs/adaboost/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
You want to access the fitted estimators on each fold. Use `cross_validate(..., return_estimators=True)` see [doc](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_validate.html).

The original estimator is cloned and thus not fitted which is why it does not have `estimators_` attribute.

Labels: Bug, Needs Triage
The estimators_ attribute can no longer be accessed for the AdaBoostClassifier class

### Describe the bug

The [documentation of the AdaBoostClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html) reads that there is a `estimators_` attribute. However, if I try to access this attribute, I get the error

> AttributeError: 'AdaBoostClassifier' object has no attribute 'estimators_'

### Steps/Code to Reproduce

```python
from sklearn.datasets import load_digits
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import AdaBoostClassifier

# Load dataset
digits = load_digits()
X = digits.data
y = digits.target

# Initialize AdaBoost classifier
ada_clf = AdaBoostClassifier(n_estimators=n_estimators)
scores = cross_val_score(ada_clf, X, y, cv=5)

ada_clf.estimators_
```

### Expected Results

No error is shown

### Actual Results

```bash
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[29], line 14
     11 ada_clf = AdaBoostClassifier(n_estimators=n_estimators)
     12 scores = cross_val_score(ada_clf, X, y, cv=5)
---> 14 ada_clf.estimators_

AttributeError: 'AdaBoostClassifier' object has no attribute 'estimators_'
```

### Versions

```shell
System:
    python: 3.11.11 | packaged by conda-forge | (main, Mar  3 2025, 20:43:55) [GCC 13.3.0]
executable: /home/kkladny/psi4conda/envs/adaboost/bin/python
   machine: Linux-6.8.0-52-generic-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.6.1
          pip: 25.0.1
   setuptools: 75.8.2
        numpy: 2.2.3
        scipy: 1.15.2
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.1
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/kkladny/psi4conda/envs/adaboost/lib/python3.11/site-packages/numpy.libs/libscipy_openblas64_-6bb31eeb.so
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: blas
   internal_api: openblas
    num_threads: 20
         prefix: libscipy_openblas
       filepath: /home/kkladny/psi4conda/envs/adaboost/lib/python3.11/site-packages/scipy.libs/libscipy_openblas-68440149.so
        version: 0.3.28
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 20
         prefix: libgomp
       filepath: /home/kkladny/psi4conda/envs/adaboost/lib/python3.11/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None
```

COMMENT:
Thanks! It would be nice to return an error that says something like you wrote.

Labels: Bug, Needs Triage