Talip7/scikit-learn-issues-multilabel huggingface dataset:
Cache final transformer in pipeline with memory setting

### Describe the bug

When setting the `memory` parameter of a transformer `Pipeline` (i.e., one whose last step is a transformer), the final transformer is not cached.

Discovered at https://stackoverflow.com/q/71812869/10495893.

### Steps/Code to Reproduce

```
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.pipeline import Pipeline
import time

class Test(BaseEstimator, TransformerMixin):
    def __init__(self, col):
        self.col = col

    def fit(self, X, y=None):
        print(self.col)
        return self

    def transform(self, X, y=None):
        for t in range(5):
            # just to slow it down / check caching.
            print(".")
            time.sleep(1)
        #print(self.col)
        return X

pipline = Pipeline(
    [
        ("test", Test(col="this_column")),
        ("test2", Test(col="that_column"))
    ],
    memory="tmp/cache",
)

pipline.fit(None)
pipline.fit(None)
pipline.fit(None)
```

### Expected Results

```
this_column
.
.
.
.
.
that_column
```

### Actual Results

```
this_column
.
.
.
.
.
that_column
that_column
that_column
```

### Versions

```shell
System:
    python: 3.7.13 (default, Mar 16 2022, 17:37:17)  [GCC 7.5.0]
executable: /usr/bin/python3
   machine: Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic

Python dependencies:
          pip: 21.1.3
   setuptools: 57.4.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.4.1
       Cython: 0.29.28
       pandas: 1.3.5
   matplotlib: 3.2.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
Any updates?

Labels: Bug, module:pipeline
Add "pre_dispatch" parameter to HalvingGridSearchCV

### Describe the workflow you want to enable

In base GridSearchCV it is possible to set a pre_dispatch parameter, so that RAM usage by new jobs is limited.
I would like to do the same thing for the newer and arguably better versions of gridsearch.

### Describe your proposed solution

Add the pre_dispatch parameter that already exists in scikit to the new grid searches.

### Describe alternatives you've considered, if relevant

If you one that would be great because currently I am running into heavy issues with catboost which only accepts n_jobs=1 when running on gpu :)

### Additional context

_No response_

COMMENT:
We have resisted adding `kwargs` from [joblib.Parallel](https://joblib.readthedocs.io/en/latest/generated/joblib.Parallel.html) into estimators and removed it recently: https://github.com/scikit-learn/scikit-learn/pull/18030#issuecomment-667915067

One option is to add a `parallel_kwargs` parameter that would pass the kwars down to `joblib.Parallel`.

Another options is to make the `parallel_backend` context manager work:

```python
from joblib import Parallel, delayed, parallel_backend
from math import sqrt

# This does not work
with parallel_backend("loky", pre_dispatch="2*n_jobs"):
    parallel = Parallel()
    out = parallel(delayed(sqrt)(i ** 2) for i in range(10))
print(out)
```

@lesteve @ogrisel What do you think?

Labels: New Feature, module:model_selection
Add "pre_dispatch" parameter to HalvingGridSearchCV

### Describe the workflow you want to enable

In base GridSearchCV it is possible to set a pre_dispatch parameter, so that RAM usage by new jobs is limited.
I would like to do the same thing for the newer and arguably better versions of gridsearch.

### Describe your proposed solution

Add the pre_dispatch parameter that already exists in scikit to the new grid searches.

### Describe alternatives you've considered, if relevant

If you one that would be great because currently I am running into heavy issues with catboost which only accepts n_jobs=1 when running on gpu :)

### Additional context

_No response_

COMMENT:
I don't understand why `pre_dispatch` would fundamentally change memory usage anyway: the memory mapping of the training data should prevent large data copies.

Labels: New Feature, module:model_selection
Add "pre_dispatch" parameter to HalvingGridSearchCV

### Describe the workflow you want to enable

In base GridSearchCV it is possible to set a pre_dispatch parameter, so that RAM usage by new jobs is limited.
I would like to do the same thing for the newer and arguably better versions of gridsearch.

### Describe your proposed solution

Add the pre_dispatch parameter that already exists in scikit to the new grid searches.

### Describe alternatives you've considered, if relevant

If you one that would be great because currently I am running into heavy issues with catboost which only accepts n_jobs=1 when running on gpu :)

### Additional context

_No response_

COMMENT:
@nilslacroix have you tried to hard-code a different value of `pre_dispatch` in your local copy of scikit-learn in the source code of `HalvingGridSearchCV` to see if it actually fix your problem?

If so, which values help, and which value make the execution crash and what it the error message you get?

Labels: New Feature, module:model_selection
Add "pre_dispatch" parameter to HalvingGridSearchCV

### Describe the workflow you want to enable

In base GridSearchCV it is possible to set a pre_dispatch parameter, so that RAM usage by new jobs is limited.
I would like to do the same thing for the newer and arguably better versions of gridsearch.

### Describe your proposed solution

Add the pre_dispatch parameter that already exists in scikit to the new grid searches.

### Describe alternatives you've considered, if relevant

If you one that would be great because currently I am running into heavy issues with catboost which only accepts n_jobs=1 when running on gpu :)

### Additional context

_No response_

COMMENT:
We could implement @thomasjpfan's suggestion but that will make joblib even more complex so I would like to make sure that it is actually useful before considering a pull request.

Labels: New Feature, module:model_selection
Add "pre_dispatch" parameter to HalvingGridSearchCV

### Describe the workflow you want to enable

In base GridSearchCV it is possible to set a pre_dispatch parameter, so that RAM usage by new jobs is limited.
I would like to do the same thing for the newer and arguably better versions of gridsearch.

### Describe your proposed solution

Add the pre_dispatch parameter that already exists in scikit to the new grid searches.

### Describe alternatives you've considered, if relevant

If you one that would be great because currently I am running into heavy issues with catboost which only accepts n_jobs=1 when running on gpu :)

### Additional context

_No response_

COMMENT:
I am closing this issue because there was no response to https://github.com/scikit-learn/scikit-learn/issues/23108#issuecomment-1171404045.

Labels: New Feature, module:model_selection
Add "pre_dispatch" parameter to HalvingGridSearchCV

### Describe the workflow you want to enable

In base GridSearchCV it is possible to set a pre_dispatch parameter, so that RAM usage by new jobs is limited.
I would like to do the same thing for the newer and arguably better versions of gridsearch.

### Describe your proposed solution

Add the pre_dispatch parameter that already exists in scikit to the new grid searches.

### Describe alternatives you've considered, if relevant

If you one that would be great because currently I am running into heavy issues with catboost which only accepts n_jobs=1 when running on gpu :)

### Additional context

_No response_

COMMENT:
I am also running into issues with this. `HalvingGridSearchCV` causes a memory explosion when working with large datasets on HPC clusters with many cores available and therefore using high `n_jobs`. Having the `pre_dispatch` parameter available here would help.

Labels: New Feature, module:model_selection
Add "pre_dispatch" parameter to HalvingGridSearchCV

### Describe the workflow you want to enable

In base GridSearchCV it is possible to set a pre_dispatch parameter, so that RAM usage by new jobs is limited.
I would like to do the same thing for the newer and arguably better versions of gridsearch.

### Describe your proposed solution

Add the pre_dispatch parameter that already exists in scikit to the new grid searches.

### Describe alternatives you've considered, if relevant

If you one that would be great because currently I am running into heavy issues with catboost which only accepts n_jobs=1 when running on gpu :)

### Additional context

_No response_

COMMENT:
> I don't understand why `pre_dispatch` would fundamentally change memory usage anyway: the memory mapping of the training data should prevent large data copies.

This discussion seems to suggest otherwise and seems to suggest that data replication occurs when using `n_jobs`.
https://datascience.stackexchange.com/questions/34416/scikit-learn-n-jobs-parameter-on-cpu-usage-memory

Labels: New Feature, module:model_selection
Add "pre_dispatch" parameter to HalvingGridSearchCV

### Describe the workflow you want to enable

In base GridSearchCV it is possible to set a pre_dispatch parameter, so that RAM usage by new jobs is limited.
I would like to do the same thing for the newer and arguably better versions of gridsearch.

### Describe your proposed solution

Add the pre_dispatch parameter that already exists in scikit to the new grid searches.

### Describe alternatives you've considered, if relevant

If you one that would be great because currently I am running into heavy issues with catboost which only accepts n_jobs=1 when running on gpu :)

### Additional context

_No response_

COMMENT:
Additionally, this makes the API inconsistent as the `GridSearchCV` makes this parameter available whilst the `HalvingGridSearchCV` doesn't.

Labels: New Feature, module:model_selection
`SequentialFeatureSelector` is not passing pandas df to estimator/pipeline

### Describe the bug

`SequentialFeatureSelector` cannot be used with a pipeline that expects a pandas dataframe (e.g. a one containing a `ColumnTransformer`) as an input.


At the same time the same pipeline can be used in `cross_val_score`.

### Steps/Code to Reproduce

```
from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler
from sklearn.compose import make_column_selector
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

import pandas as pd
import numpy as np

df = pd.DataFrame(data=np.random.rand(10,4), columns=["1","2", "3", "4"])
y = np.random.randint(0,2, size=(10,))

ct = ColumnTransformer(
    transformers=[
        ("numerical", StandardScaler(), make_column_selector(pattern="1")),
    ],
    remainder="passthrough",
)

pipeline = Pipeline([("ct", ct), ("lr", LogisticRegression())])

# this works
print("cross_val_score", cross_val_score(pipeline, df, y, cv=2))

sfs = SequentialFeatureSelector(pipeline, cv=2)
sfs.fit(df, y=y)
```

### Expected Results

No error is thrown and the features are selected.

### Actual Results

```
....sklearn/model_selection/_validation.py:372: FitFailedWarning: 
2 fits failed out of a total of 2.
...
ValueError: make_column_selector can only be applied to pandas dataframes
```

### Versions

```shell
System:
    python: 3.9.5 (default, Nov 23 2021, 15:27:38)  [GCC 9.3.0]
executable: /root/.cache/pypoetry/virtualenvs/science-yd9zGFjU-py3.9/bin/python
   machine: Linux-5.13.0-35-generic-x86_64-with-glibc2.31

Python dependencies:
          pip: 22.0.3
   setuptools: 60.9.3
      sklearn: 1.0.2
        numpy: 1.19.5
        scipy: 1.8.0
       Cython: None
       pandas: 1.4.1
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```
```

COMMENT:
What is the reason for not putting the `SequentialFeatureSelector` part of the `Pipeline`.
Right now, the score obtained with the `cross_val_score` function is not valid since the feature selection is not part of the pipeline itself.

Regarding the current workflow, this is a known limitation because the `SequentialFeatureSelector` will validate the input data and will transform it to a NumPy array that will be forwarded to the pipeline.

Future work regarding pandas in/out should probably solve this problem. Pinging @thomasjpfan since it could be one of the use cases to add to the SLEP on this subject.

Labels: Bug, module:feature_selection
`SequentialFeatureSelector` is not passing pandas df to estimator/pipeline

### Describe the bug

`SequentialFeatureSelector` cannot be used with a pipeline that expects a pandas dataframe (e.g. a one containing a `ColumnTransformer`) as an input.


At the same time the same pipeline can be used in `cross_val_score`.

### Steps/Code to Reproduce

```
from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler
from sklearn.compose import make_column_selector
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

import pandas as pd
import numpy as np

df = pd.DataFrame(data=np.random.rand(10,4), columns=["1","2", "3", "4"])
y = np.random.randint(0,2, size=(10,))

ct = ColumnTransformer(
    transformers=[
        ("numerical", StandardScaler(), make_column_selector(pattern="1")),
    ],
    remainder="passthrough",
)

pipeline = Pipeline([("ct", ct), ("lr", LogisticRegression())])

# this works
print("cross_val_score", cross_val_score(pipeline, df, y, cv=2))

sfs = SequentialFeatureSelector(pipeline, cv=2)
sfs.fit(df, y=y)
```

### Expected Results

No error is thrown and the features are selected.

### Actual Results

```
....sklearn/model_selection/_validation.py:372: FitFailedWarning: 
2 fits failed out of a total of 2.
...
ValueError: make_column_selector can only be applied to pandas dataframes
```

### Versions

```shell
System:
    python: 3.9.5 (default, Nov 23 2021, 15:27:38)  [GCC 9.3.0]
executable: /root/.cache/pypoetry/virtualenvs/science-yd9zGFjU-py3.9/bin/python
   machine: Linux-5.13.0-35-generic-x86_64-with-glibc2.31

Python dependencies:
          pip: 22.0.3
   setuptools: 60.9.3
      sklearn: 1.0.2
        numpy: 1.19.5
        scipy: 1.8.0
       Cython: None
       pandas: 1.4.1
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```
```

COMMENT:
> Future work regarding pandas in/out should probably solve this problem.

I think this issue is unrelated to the pandas in/out issue. In this case, `SequentialFeatureSelector` needs to slice data on `axis=1` without casting to a ndarray.

Aside: If we try to implement this, I think we would run into the same issue as https://github.com/scikit-learn/scikit-learn/pull/21811#issuecomment-1091869844.

Labels: Bug, module:feature_selection
When import sklearn, there is an AttributeError

### Describe the bug

When I wrote some code as follows and click run, it was broken.  
`import sklearn`
  
Python version: `3.8.2`
Scikit-learn version: `1.0.2`
Error Information:  
```
Traceback (most recent call last):
  File "/data/user/0/ru.iiec.pydroid3/files/accomp_files/iiec_run/iiec_run.py", line 31, in <module>
    start(fakepyfile,mainpyfile)
  File "/data/user/0/ru.iiec.pydroid3/files/accomp_files/iiec_run/iiec_run.py", line 30, in start
    exec(open(mainpyfile).read(),  __main__.__dict__)
  File "<string>", line 1, in <module>
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/sklearn/__init__.py", line 80, in <module>
    from .base import clone
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/sklearn/base.py", line 21, in <module>
    from .utils import _IS_32BIT
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/sklearn/utils/__init__.py", line 20, in <module>
    from scipy.sparse import issparse
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/__init__.py", line 156, in <module>
    from . import fft
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/fft/__init__.py", line 76, in <module>
    from ._basic import (
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/fft/_basic.py", line 1, in <module>
    from scipy._lib.uarray import generate_multimethod, Dispatchable
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/_lib/uarray.py", line 27, in <module>
    from ._uarray import *
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/_lib/_uarray/__init__.py", line 114, in <module>
    from ._backend import *
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/_lib/_uarray/_backend.py", line 78, in <module>
    atexit.register(_uarray.clear_all_globals)
AttributeError: module 'uarray._uarray' has no attribute 'clear_all_globals'
```

### Steps/Code to Reproduce

`import sklearn`

### Expected Results

Can import sklearn and no error is thrown.
Come on!

### Actual Results
No error and sklearn can be imported.

### Versions

```shell
It will throw a error named AttitubeError.

AttributeError: module 'uarray._uarray' has no attribute 'clear_all_globals'
```

COMMENT:
Looking at the traceback, the error is coming from `scipy`. Can you run the following and see it it works?

```python
from scipy.sparse import issparse
```

If that fails, I suggest reinstalling scipy and try to import `scipy` again.

Labels: Bug
When import sklearn, there is an AttributeError

### Describe the bug

When I wrote some code as follows and click run, it was broken.  
`import sklearn`
  
Python version: `3.8.2`
Scikit-learn version: `1.0.2`
Error Information:  
```
Traceback (most recent call last):
  File "/data/user/0/ru.iiec.pydroid3/files/accomp_files/iiec_run/iiec_run.py", line 31, in <module>
    start(fakepyfile,mainpyfile)
  File "/data/user/0/ru.iiec.pydroid3/files/accomp_files/iiec_run/iiec_run.py", line 30, in start
    exec(open(mainpyfile).read(),  __main__.__dict__)
  File "<string>", line 1, in <module>
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/sklearn/__init__.py", line 80, in <module>
    from .base import clone
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/sklearn/base.py", line 21, in <module>
    from .utils import _IS_32BIT
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/sklearn/utils/__init__.py", line 20, in <module>
    from scipy.sparse import issparse
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/__init__.py", line 156, in <module>
    from . import fft
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/fft/__init__.py", line 76, in <module>
    from ._basic import (
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/fft/_basic.py", line 1, in <module>
    from scipy._lib.uarray import generate_multimethod, Dispatchable
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/_lib/uarray.py", line 27, in <module>
    from ._uarray import *
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/_lib/_uarray/__init__.py", line 114, in <module>
    from ._backend import *
  File "/data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/lib/python3.8/site-packages/scipy/_lib/_uarray/_backend.py", line 78, in <module>
    atexit.register(_uarray.clear_all_globals)
AttributeError: module 'uarray._uarray' has no attribute 'clear_all_globals'
```

### Steps/Code to Reproduce

`import sklearn`

### Expected Results

Can import sklearn and no error is thrown.
Come on!

### Actual Results
No error and sklearn can be imported.

### Versions

```shell
It will throw a error named AttitubeError.

AttributeError: module 'uarray._uarray' has no attribute 'clear_all_globals'
```

COMMENT:
If reinstalling does not help, then I suggest opening an issue with SciPy. When opening the issue in SciPy, I recommend stating where you installed SciPy from. This will help them debug the issue.

Did you run `pip install scikit-learn`, which means `SciPy` was from pypi or did you install the library with another method?

**Edit**: Closing this issue because this is an import issue with `SciPy` and not actionable for scikit-learn itself.

Labels: Bug
Implement Min-Cut clustering

### Describe the workflow you want to enable

Use the exact min-cut culstering method with API similar to sklearn.cluster.SpectralClustering.

### Describe your proposed solution

Implement an algorithm to solve the min-cut problem as a means of dataset clustering. 

### Describe alternatives you've considered, if relevant

In practice SpectralClustering is usally a good alternative. In fact, SpectralClustering can be thought of as an approximate solution to the min-cut problem. 

### Additional context

Solving min-cut exactly is a computationally difficult problem and hence is not practical for large datasets. I think it would be valuable for educational purposes, though (e.g. to compare the performance with SpectralClustering on toy datasets). Furthermore, it may still be useful when working with smaller datasets.

COMMENT:
Every feature we include has a [maintenance cost](http://scikit-learn.org/dev/faq.html#why-are-you-so-selective-on-what-algorithms-you-include-in-scikit-learn). Our maintainers are mostly volunteers. For a new feature to be included, we need evidence that it is often useful and, ideally, [well-established](http://scikit-learn.org/dev/faq.html#what-are-the-inclusion-criteria-for-new-algorithms) in the literature or in practice.

Labels: New Feature, module:cluster, Needs Decision - Include Feature