Talip7/scikit-learn-issues-multilabel huggingface dataset:
Allow users to override `_fit_and_score` of the BaseSearchCV

### Describe the workflow you want to enable

Currently, the `BaseSearchCV` has some level of customization enabled with the `_run_search` method. I would like to enable more customization. 
In particular the `fit` method calls `_fit_and_score` function for all of the CV splits in parallel. This makes it difficult to modify the fitting and scoring behavior for particular models. Allowing users to create custom `_fit_and_score` methods would enable many use cases that are currently hard - such as working with other modeling and data ecosystems not fully supported by sklearn. 

### Describe your proposed solution

1. I would propose making `_fit_and_score` a method on the `BaseSearchCV` class. As far as I can tell. this function is only used in this class so it could be moved fully (with some deprecation period). Or, if you wanted to keep `_fit_and_score` as a function you could have a small method alias in `BaseSearchCV` that calls the function. 
2. Change the delayed call in `fit` https://github.com/scikit-learn/scikit-learn/blob/a63b021310ba13ea39ad3555f550d8aeec3002c5/sklearn/model_selection/_search.py#L914C1-L917C1
```
                out = parallel(
                    delayed(_fit_and_score)(
                        clone(base_estimator),
```
to using the method:
```
                out = parallel(
                    delayed(self._fit_and_score)(
                        clone(base_estimator),
```

For most users the function will continue as before. For users with more complex use-cases they are free to create a subclass with their own implementation of `_fit_and_score`.

### Describe alternatives you've considered, if relevant

1. Currently we would need to also implement the `fit` function to modify the lines above and achieve this. By allowing just the `_fit_and_score` to be modified we will make it easier for users. 

2. One can try to monkeypatch or otherwise modify the `_fit_and_score` function directly, but I have found this difficult due to the parallel call here. If there are suggestions to doing this consistently then that could be an avenue, but I think the subclassing route is more explicit. 

### Additional context

I am happy to provide a PR with the above changes and tests if there is interest in this.

COMMENT:
> Allowing users to create custom _fit_and_score methods would enable many use cases that are currently hard - such as working with other modeling and data ecosystems not fully supported by sklearn.

I would personally need more convincing to create some abstraction level here since this is to handle case where the underlying use cases are not scikit-learn compatible. So at this stage I would not be inclined with the change.

Regarding the change itself, we always used pure Python function with `joblib.Parallel` and never a method. I would not be surprised that using a class method would present some caveats.

Labels: New Feature, Needs Decision
When a Pipeline step is changed via set_params, the set_output state is cleared

### Describe the bug

When a Pipeline step is set via `set_params`, the subsequent output of `fit_transform` is a numpy ndarray even if previously the pipeline's output was set to be of type pandas.DataFrame via a call to `set_output(transform= 'pandas')`.

This only happens when the entire step is set: `set_params(step= some_value)`, not when only a step's parameters are set: `set_params(step__some_param= some_value)`.

The issue doesn't occur if, instead of calling `set_output(transform= 'pandas')` on the Pipeline object, the option is set globally with `sklearn.set_config(transform_output= 'pandas')`.

The same problem may or may not occur with ColumnTransformer, I haven't checked.

### Steps/Code to Reproduce

```
from pandas import DataFrame
from numpy import NaN
from sklearn.preprocessing import MinMaxScaler
from sklearn.pipeline import Pipeline

X = DataFrame({'N': [1, 2]})
pipe = Pipeline([('scale', MinMaxScaler())]).set_output(transform='pandas')
pipe.set_params(scale= MinMaxScaler())
pipe.fit_transform(X)
```

### Expected Results

A pandas DataFrame

### Actual Results

A numpy ndarray

### Versions

```shell
System:
    python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
executable: /usr/bin/python3
   machine: Linux-6.1.85+-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.2.2
          pip: 23.1.2
   setuptools: 67.7.2
        numpy: 1.25.2
        scipy: 1.11.4
       Cython: 3.0.10
       pandas: 2.0.3
   matplotlib: 3.7.1
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 2
         prefix: libopenblas
       filepath: /usr/local/lib/python3.10/dist-packages/numpy.libs/libopenblas64_p-r0-5007b62f.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 2
         prefix: libgomp
       filepath: /usr/local/lib/python3.10/dist-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 2
         prefix: libopenblas
       filepath: /usr/local/lib/python3.10/dist-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Haswell
```

COMMENT:
> At the same time, fixing this would probably involve going over all the pipeline estimators and set their outputs at fit time, so not sure this is worth the additional complexity?

Since we are not cloning estimator in `Pipeline`, making `set_params` call `set_output` would change the state of the external estimator. I think that I'm not a big fan of that.

I would not consider fixing it but instead we could document it in a section that we discuss `pipeline.set_output` where we could mention the behaviour of `set_params`

Labels: Documentation, Moderate, help wanted
When a Pipeline step is changed via set_params, the set_output state is cleared

### Describe the bug

When a Pipeline step is set via `set_params`, the subsequent output of `fit_transform` is a numpy ndarray even if previously the pipeline's output was set to be of type pandas.DataFrame via a call to `set_output(transform= 'pandas')`.

This only happens when the entire step is set: `set_params(step= some_value)`, not when only a step's parameters are set: `set_params(step__some_param= some_value)`.

The issue doesn't occur if, instead of calling `set_output(transform= 'pandas')` on the Pipeline object, the option is set globally with `sklearn.set_config(transform_output= 'pandas')`.

The same problem may or may not occur with ColumnTransformer, I haven't checked.

### Steps/Code to Reproduce

```
from pandas import DataFrame
from numpy import NaN
from sklearn.preprocessing import MinMaxScaler
from sklearn.pipeline import Pipeline

X = DataFrame({'N': [1, 2]})
pipe = Pipeline([('scale', MinMaxScaler())]).set_output(transform='pandas')
pipe.set_params(scale= MinMaxScaler())
pipe.fit_transform(X)
```

### Expected Results

A pandas DataFrame

### Actual Results

A numpy ndarray

### Versions

```shell
System:
    python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
executable: /usr/bin/python3
   machine: Linux-6.1.85+-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.2.2
          pip: 23.1.2
   setuptools: 67.7.2
        numpy: 1.25.2
        scipy: 1.11.4
       Cython: 3.0.10
       pandas: 2.0.3
   matplotlib: 3.7.1
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 2
         prefix: libopenblas
       filepath: /usr/local/lib/python3.10/dist-packages/numpy.libs/libopenblas64_p-r0-5007b62f.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 2
         prefix: libgomp
       filepath: /usr/local/lib/python3.10/dist-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 2
         prefix: libopenblas
       filepath: /usr/local/lib/python3.10/dist-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Haswell
```

COMMENT:
So I would probably suggest to amend the `examples/miscellaneous/plot_set_output.py` example in the pipeline section just to mention the subtlety.

Labels: Documentation, Moderate, help wanted
When a Pipeline step is changed via set_params, the set_output state is cleared

### Describe the bug

When a Pipeline step is set via `set_params`, the subsequent output of `fit_transform` is a numpy ndarray even if previously the pipeline's output was set to be of type pandas.DataFrame via a call to `set_output(transform= 'pandas')`.

This only happens when the entire step is set: `set_params(step= some_value)`, not when only a step's parameters are set: `set_params(step__some_param= some_value)`.

The issue doesn't occur if, instead of calling `set_output(transform= 'pandas')` on the Pipeline object, the option is set globally with `sklearn.set_config(transform_output= 'pandas')`.

The same problem may or may not occur with ColumnTransformer, I haven't checked.

### Steps/Code to Reproduce

```
from pandas import DataFrame
from numpy import NaN
from sklearn.preprocessing import MinMaxScaler
from sklearn.pipeline import Pipeline

X = DataFrame({'N': [1, 2]})
pipe = Pipeline([('scale', MinMaxScaler())]).set_output(transform='pandas')
pipe.set_params(scale= MinMaxScaler())
pipe.fit_transform(X)
```

### Expected Results

A pandas DataFrame

### Actual Results

A numpy ndarray

### Versions

```shell
System:
    python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
executable: /usr/bin/python3
   machine: Linux-6.1.85+-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.2.2
          pip: 23.1.2
   setuptools: 67.7.2
        numpy: 1.25.2
        scipy: 1.11.4
       Cython: 3.0.10
       pandas: 2.0.3
   matplotlib: 3.7.1
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 2
         prefix: libopenblas
       filepath: /usr/local/lib/python3.10/dist-packages/numpy.libs/libopenblas64_p-r0-5007b62f.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: Haswell

       user_api: openmp
   internal_api: openmp
    num_threads: 2
         prefix: libgomp
       filepath: /usr/local/lib/python3.10/dist-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 2
         prefix: libopenblas
       filepath: /usr/local/lib/python3.10/dist-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Haswell
```

COMMENT:
/take

Labels: Documentation, Moderate, help wanted
RFC module location in API table for the API reference page

I was looking at the API documentation on the new website. I was first not convinced to not have a different table for each module but at the end, if the search bar and the left navigation bart, I think this is just a matter to get use to it.

However, I thought that it might be less surprising to have the full module name appearing above the "Object" column instead of the current location (caption of the Description column). I find it more straightforward to see that the class/function is belonging to a certain module and I was thinking that these two information could be next to each other. Here, it is just a print screen of the current rendering to understand my remark.

<img width="911" alt="image" src="https://github.com/scikit-learn/scikit-learn/assets/7454015/76eaef67-5451-44c9-8109-430f54c56381">

@Charlie-XIAO Do you think this is feasible or this is just complex to achieve?

COMMENT:
![image](https://github.com/scikit-learn/scikit-learn/assets/108576690/81e126e7-d67a-44d3-9ef2-a87f2c705557)

Do you mean something like this? (This will not be complicated to implement.) I agree that it would be more straightforward (and I actually considered this when designing the API page UI), but since the Object column is narrower I think making it too "crowded" does not look so nice (and with too much blank space in the description column). WDYT?

Labels: Documentation, RFC
RFC module location in API table for the API reference page

I was looking at the API documentation on the new website. I was first not convinced to not have a different table for each module but at the end, if the search bar and the left navigation bart, I think this is just a matter to get use to it.

However, I thought that it might be less surprising to have the full module name appearing above the "Object" column instead of the current location (caption of the Description column). I find it more straightforward to see that the class/function is belonging to a certain module and I was thinking that these two information could be next to each other. Here, it is just a print screen of the current rendering to understand my remark.

<img width="911" alt="image" src="https://github.com/scikit-learn/scikit-learn/assets/7454015/76eaef67-5451-44c9-8109-430f54c56381">

@Charlie-XIAO Do you think this is feasible or this is just complex to achieve?

COMMENT:
Yep this is what I was thinking. I'm kind of fine with the spacing but I'm open to take the solution that people prefer.

Labels: Documentation, RFC
RFC module location in API table for the API reference page

I was looking at the API documentation on the new website. I was first not convinced to not have a different table for each module but at the end, if the search bar and the left navigation bart, I think this is just a matter to get use to it.

However, I thought that it might be less surprising to have the full module name appearing above the "Object" column instead of the current location (caption of the Description column). I find it more straightforward to see that the class/function is belonging to a certain module and I was thinking that these two information could be next to each other. Here, it is just a print screen of the current rendering to understand my remark.

<img width="911" alt="image" src="https://github.com/scikit-learn/scikit-learn/assets/7454015/76eaef67-5451-44c9-8109-430f54c56381">

@Charlie-XIAO Do you think this is feasible or this is just complex to achieve?

COMMENT:
> Do you mean something like this? (This will not be complicated to implement.) I agree that it would be more straightforward (and I actually considered this when designing the API page UI), but since the Object column is narrower I think making it too "crowded" does not look so nice (and with too much blank space in the description column). WDYT?

LGTM. I can try to work on this issue if it's not a problem. BTW, what means RFC?

Labels: Documentation, RFC
RFC module location in API table for the API reference page

I was looking at the API documentation on the new website. I was first not convinced to not have a different table for each module but at the end, if the search bar and the left navigation bart, I think this is just a matter to get use to it.

However, I thought that it might be less surprising to have the full module name appearing above the "Object" column instead of the current location (caption of the Description column). I find it more straightforward to see that the class/function is belonging to a certain module and I was thinking that these two information could be next to each other. Here, it is just a print screen of the current rendering to understand my remark.

<img width="911" alt="image" src="https://github.com/scikit-learn/scikit-learn/assets/7454015/76eaef67-5451-44c9-8109-430f54c56381">

@Charlie-XIAO Do you think this is feasible or this is just complex to achieve?

COMMENT:
> BTW, what means RFC?

RFC stands for "refactor" I believe.

> LGTM. I can try to work on this issue if it's not a problem.

I am however still hesitant about the change because of the reasons I mentioned in the earlier comment. I agree that UX is better but UI is not ideal from my perspective. I am thinking about putting the module name caption below the object name instead of above, so that we don't violate _the two information are next to each other_ (https://github.com/scikit-learn/scikit-learn/issues/29099#issue-2315035124), while each row of the two columns look more "aligned". Anyways, feel free to open a PR @ojoaomorais then we can discuss there :)

Since I did quite some customization for the API pages, you may want some pointers to get started. You may look at these:

https://github.com/scikit-learn/scikit-learn/blob/2b0e15a4ee0b0d89f80c6a0af1d7b6aaca5c845b/doc/api/index.rst.template#L41-L51

In particular, wrap ``` :obj:`~{{ module }}.{{ obj }}` ``` in a `div` with some class in the object column, similar to what is done in the description column. Then the CSS styling can be added around here:

https://github.com/scikit-learn/scikit-learn/blob/2b0e15a4ee0b0d89f80c6a0af1d7b6aaca5c845b/doc/scss/api-search.scss#L90

Labels: Documentation, RFC
RFC module location in API table for the API reference page

I was looking at the API documentation on the new website. I was first not convinced to not have a different table for each module but at the end, if the search bar and the left navigation bart, I think this is just a matter to get use to it.

However, I thought that it might be less surprising to have the full module name appearing above the "Object" column instead of the current location (caption of the Description column). I find it more straightforward to see that the class/function is belonging to a certain module and I was thinking that these two information could be next to each other. Here, it is just a print screen of the current rendering to understand my remark.

<img width="911" alt="image" src="https://github.com/scikit-learn/scikit-learn/assets/7454015/76eaef67-5451-44c9-8109-430f54c56381">

@Charlie-XIAO Do you think this is feasible or this is just complex to achieve?

COMMENT:
> RFC stands for "refactor" I believe.

Request for comments ;)

I would not start any work now until having more inputs. The matter is also not urgent so we can wait to gather more input.

Labels: Documentation, RFC
Deprecate copy in Birch

`Birch` doesn't perform inplace operations (at least not on the input array), so the `copy` parameter is useless and should be deprecated. It's even detrimental because by default it makes a copy.

The only place where an inplace operation happens is in the `update` method of `_CFSubcluster`: https://github.com/scikit-learn/scikit-learn/blob/11e8c216698370520a47d0639c69d959c0312a25/sklearn/cluster/_birch.py#L315-L320

However, `update` is call in 2 places. The first one is in the `_split_node` function, but here we first create 2 new `_CFSubcluster` objects and so the `update` performs inplace operations on newly created data, so the input data is not modified. The second one is in the `insert_cf_subcluster` method of `_CFNode` but is only triggered if the subcluster has a child, which can only come from splitted subclusters (i.e. after `_split_node`), so again we're not modifying the input data.

COMMENT:
/take

Labels: API
RFC Future of HalvingGridSearchCV

`HalvingGridSearchCV` has been in experimental mode since its conception in 2020-09, almost 4 years ago.

Things to note:
- we haven't seen many issues regarding these estimators, but that is probably because we are not advertising them enough, and they're in experimental mode.
- an issue such as https://github.com/scikit-learn/scikit-learn/issues/27422 gets very minimal traction and there are not many active maintainers who are actively maintaining that part of the codebase
- the API as is, is confusing and we could work on clearing things up and improving it, which shouldn't be painful since it's still experimental and we don't need to go through deprecation cycles

Now the question is, what do we think about it? Options we have:
- move them out of experimental w/o change: I don't think this is a good idea and we should probably improve things before doing so
- remove from scikit-learn: they are useful estimators, but are they used? Do we think making them more prominent in our documentation would help? Do we want to do that?
- improve the status quo of the estimator, improve documentation, move out of experimental: is this a priority for us? Are people here who are willing to dedicate time to work on it / review the work?

Also, would be nice to see if they're used, and if yes, how. Maybe @amueller or @NicolasHug would have an idea here?

cc @scikit-learn/core-devs

COMMENT:
> an issue such as https://github.com/scikit-learn/scikit-learn/issues/27422 gets very minimal traction

FWIW it's still not clear to me whether that issue is relevant or not. I'm unable to reproduce OP's problem.

I acknowledge the API surface of the halving estimators is quite large, and the exhaustive docs reflect that. The truth is, the halving algorithm is simple at first sight, but terribly complex in reality. That being said, it's not clear to me what needs to be improved here, both regarding the docs and the API. Do we have any concrete suggestions on that front?

Labels: RFC
RFC Future of HalvingGridSearchCV

`HalvingGridSearchCV` has been in experimental mode since its conception in 2020-09, almost 4 years ago.

Things to note:
- we haven't seen many issues regarding these estimators, but that is probably because we are not advertising them enough, and they're in experimental mode.
- an issue such as https://github.com/scikit-learn/scikit-learn/issues/27422 gets very minimal traction and there are not many active maintainers who are actively maintaining that part of the codebase
- the API as is, is confusing and we could work on clearing things up and improving it, which shouldn't be painful since it's still experimental and we don't need to go through deprecation cycles

Now the question is, what do we think about it? Options we have:
- move them out of experimental w/o change: I don't think this is a good idea and we should probably improve things before doing so
- remove from scikit-learn: they are useful estimators, but are they used? Do we think making them more prominent in our documentation would help? Do we want to do that?
- improve the status quo of the estimator, improve documentation, move out of experimental: is this a priority for us? Are people here who are willing to dedicate time to work on it / review the work?

Also, would be nice to see if they're used, and if yes, how. Maybe @amueller or @NicolasHug would have an idea here?

cc @scikit-learn/core-devs

COMMENT:
I think that this is a very useful estimator, even though it's API can possibly be improved. I would not be in favor of deprecating it, but rather improving it.

Labels: RFC