Talip7/scikit-learn-issues-multilabel huggingface dataset:
Very old notes in docstrings

### Describe the issue linked to the documentation

In `sklearn/feature_extraction/image.py` very old notes are contained in the docstrings of two functions:

https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/feature_extraction/image.py#L178-L185

and

https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/feature_extraction/image.py#L232-L239

These notes refer to scikit-learn 0.14.1 which is approx. 10 years old:

```python
"""
    .
    .
    .
    Notes
    -----
    For scikit-learn versions 0.14.1 and prior, return_as=np.ndarray was
    handled by returning a dense np.matrix instance.  Going forward, np.ndarray
    returns an np.ndarray, as expected.
    .
    .
    .
"""
```

Maybe these notes are obsolet and can be removed?
What is the employed strategy in scikit-learn regarding such old notes?

### Suggest a potential alternative/fix

_No response_

COMMENT:
I'll close this for now. If someone wants to discuss/come up with a policy and concerted effort to cleanup all old notes we can re-open this or maybe even make a new issue listing all places that need fixing in the top comment

Labels: Documentation
ENH Plots for partial dependence and ICE of categoricals

### Describe the workflow you want to enable

Having support of categorical features in `inspection.partial_dependence()` and `inspection.PartialDependenceDisplay()` is great. However, I would like to challenge the current visualization as **bar plot**.

- Bars take a lot of space.
- They suggest that 0 has a special meaning.
- They look dominant when a mix of categoricals and non-categoricals are plotted in the same plot.

Visualizating them as dots and/or lines would be neater in most cases. This would also fix the issue that "individual" ICE curves of categoricals are not implemented. There, we would simply use the same visualization (lines) as with non-categoricals.


### Describe your proposed solution

Replace bar plots by dots and/or lines. If individual curves are drawn, simply plot lines as for non-categoricals.

Centered ICE curves of multiple models would look like this:

![image](https://github.com/scikit-learn/scikit-learn/assets/16206095/68527977-6e8c-4a94-96df-9ec769ff5d67)


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I'm not against exploring another way to represent the ICE and PDP for categorical features.

However, I'm not a big fan of the line proposal: the line create a relationship between one category and its two "1-neighbor" categories and it tends to overlook other categories. The bar plot at least does not bring such interpretation at a first sight.

The scatter plot might be better then. I am wondering if it could coupled together with a boxplot per category.

Labels: New Feature
ENH Plots for partial dependence and ICE of categoricals

### Describe the workflow you want to enable

Having support of categorical features in `inspection.partial_dependence()` and `inspection.PartialDependenceDisplay()` is great. However, I would like to challenge the current visualization as **bar plot**.

- Bars take a lot of space.
- They suggest that 0 has a special meaning.
- They look dominant when a mix of categoricals and non-categoricals are plotted in the same plot.

Visualizating them as dots and/or lines would be neater in most cases. This would also fix the issue that "individual" ICE curves of categoricals are not implemented. There, we would simply use the same visualization (lines) as with non-categoricals.


### Describe your proposed solution

Replace bar plots by dots and/or lines. If individual curves are drawn, simply plot lines as for non-categoricals.

Centered ICE curves of multiple models would look like this:

![image](https://github.com/scikit-learn/scikit-learn/assets/16206095/68527977-6e8c-4a94-96df-9ec769ff5d67)


### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Good point.

1. With kind "individual/both", we need the lines to glue together observations. 
2. With kind "average", we can use a scatterplot by default, and offer an option to switch to lines. The latter is often easier to read for ordered categoricals.

Labels: New Feature
Allow boosting of estimators in scikit-learn pipelines using residuals

### Describe the workflow you want to enable

I want to be able to use multiple estimators in one pipeline.  E.g.

```python
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.datasets import make_regression

X, y = make_regression(n_samples=100, n_features=10, noise=0.1)

pipeline = Pipeline([
    ('linear', LinearRegression()),
    ('gbm', GradientBoostingRegressor())
])

pipeline.fit(X, y)
```

This currently doesn't work: 
```python
TypeError: All intermediate steps should be transformers and implement fit and transform or be the string 'passthrough' 'LinearRegression()' (type <class 'sklearn.linear_model._base.LinearRegression'>) doesn't
```

This is for good reason: naively passing predict from the linear model to the fit for the gbm will lead to overfitting!

### Describe your proposed solution

Allow boosting estimators in scikit learn pipeline by subtracting the predict output from y at `fit` time and then adding the predict outputs at transform time. This enhancement would allow multiple regression estimators to be used consecutively, where each subsequent estimator fits on the residuals of the previous ones.

In my example above, at fit time the linear step proceeds as normal.  Then the gbm step detects 2 chained estimators, and subtracts the prediction from the linear step from y, and then fits to the residual.

At transform time, the predictions from the linear model would be added to the predictions from the gbm model.

To simplify things, I think you still wouldn't be allowed to connect an estimator to a transformer.

### Describe alternatives you've considered, if relevant

Vecstack is a good alternative, but its slow.  You have to cross-validate each estimator, which gets expensive if you chain more than 2 in a row.

```python
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression, Ridge
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.datasets import make_regression
from vecstack import StackingTransformer

X, y = make_regression(n_samples=100, n_features=10, noise=0.1)

pipeline = Pipeline([
    ('linear', StackingTransformer([('lr', LinearRegression())], regression=True, shuffle=True)),
    ('gbm', GradientBoostingRegressor())
])

pipeline.fit(X, y)
y_pred = pipeline.predict(X)
```

### Additional context

I love being able to use stacking inside sklearn pipelines.  It would be very cool to be able to use boosting too.  I haven't been able to figure out how to make a boosting class that adjusts y as it moves through the pipeline, which is why I think this feature would need to be implemented in the core library.

COMMENT:
Hmm— I wasn't aware of the init argument for the gbm class.  That's interesting.

Could I just do
```
pipe = Pipeline(
    steps=[("reg", GradientBoostingRegressor(init=LinearRegression(), random_state=42))]
)
```

?

Labels: New Feature, Needs Decision
Allow boosting of estimators in scikit-learn pipelines using residuals

### Describe the workflow you want to enable

I want to be able to use multiple estimators in one pipeline.  E.g.

```python
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.datasets import make_regression

X, y = make_regression(n_samples=100, n_features=10, noise=0.1)

pipeline = Pipeline([
    ('linear', LinearRegression()),
    ('gbm', GradientBoostingRegressor())
])

pipeline.fit(X, y)
```

This currently doesn't work: 
```python
TypeError: All intermediate steps should be transformers and implement fit and transform or be the string 'passthrough' 'LinearRegression()' (type <class 'sklearn.linear_model._base.LinearRegression'>) doesn't
```

This is for good reason: naively passing predict from the linear model to the fit for the gbm will lead to overfitting!

### Describe your proposed solution

Allow boosting estimators in scikit learn pipeline by subtracting the predict output from y at `fit` time and then adding the predict outputs at transform time. This enhancement would allow multiple regression estimators to be used consecutively, where each subsequent estimator fits on the residuals of the previous ones.

In my example above, at fit time the linear step proceeds as normal.  Then the gbm step detects 2 chained estimators, and subtracts the prediction from the linear step from y, and then fits to the residual.

At transform time, the predictions from the linear model would be added to the predictions from the gbm model.

To simplify things, I think you still wouldn't be allowed to connect an estimator to a transformer.

### Describe alternatives you've considered, if relevant

Vecstack is a good alternative, but its slow.  You have to cross-validate each estimator, which gets expensive if you chain more than 2 in a row.

```python
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression, Ridge
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.datasets import make_regression
from vecstack import StackingTransformer

X, y = make_regression(n_samples=100, n_features=10, noise=0.1)

pipeline = Pipeline([
    ('linear', StackingTransformer([('lr', LinearRegression())], regression=True, shuffle=True)),
    ('gbm', GradientBoostingRegressor())
])

pipeline.fit(X, y)
y_pred = pipeline.predict(X)
```

### Additional context

I love being able to use stacking inside sklearn pipelines.  It would be very cool to be able to use boosting too.  I haven't been able to figure out how to make a boosting class that adjusts y as it moves through the pipeline, which is why I think this feature would need to be implemented in the core library.

COMMENT:
The idea of fitting a new model on residuals may be useful for leveraging scikit-learn for causal effect estimation as well, so I think the general concept is potentially useful.

Labels: New Feature, Needs Decision
Allow boosting of estimators in scikit-learn pipelines using residuals

### Describe the workflow you want to enable

I want to be able to use multiple estimators in one pipeline.  E.g.

```python
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.datasets import make_regression

X, y = make_regression(n_samples=100, n_features=10, noise=0.1)

pipeline = Pipeline([
    ('linear', LinearRegression()),
    ('gbm', GradientBoostingRegressor())
])

pipeline.fit(X, y)
```

This currently doesn't work: 
```python
TypeError: All intermediate steps should be transformers and implement fit and transform or be the string 'passthrough' 'LinearRegression()' (type <class 'sklearn.linear_model._base.LinearRegression'>) doesn't
```

This is for good reason: naively passing predict from the linear model to the fit for the gbm will lead to overfitting!

### Describe your proposed solution

Allow boosting estimators in scikit learn pipeline by subtracting the predict output from y at `fit` time and then adding the predict outputs at transform time. This enhancement would allow multiple regression estimators to be used consecutively, where each subsequent estimator fits on the residuals of the previous ones.

In my example above, at fit time the linear step proceeds as normal.  Then the gbm step detects 2 chained estimators, and subtracts the prediction from the linear step from y, and then fits to the residual.

At transform time, the predictions from the linear model would be added to the predictions from the gbm model.

To simplify things, I think you still wouldn't be allowed to connect an estimator to a transformer.

### Describe alternatives you've considered, if relevant

Vecstack is a good alternative, but its slow.  You have to cross-validate each estimator, which gets expensive if you chain more than 2 in a row.

```python
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LinearRegression, Ridge
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.datasets import make_regression
from vecstack import StackingTransformer

X, y = make_regression(n_samples=100, n_features=10, noise=0.1)

pipeline = Pipeline([
    ('linear', StackingTransformer([('lr', LinearRegression())], regression=True, shuffle=True)),
    ('gbm', GradientBoostingRegressor())
])

pipeline.fit(X, y)
y_pred = pipeline.predict(X)
```

### Additional context

I love being able to use stacking inside sklearn pipelines.  It would be very cool to be able to use boosting too.  I haven't been able to figure out how to make a boosting class that adjusts y as it moves through the pipeline, which is why I think this feature would need to be implemented in the core library.

COMMENT:
I also propose that a list of feature names can be passed to each intermediate model.  For instance, I might want to regress housing price on lat/lon coordinates using kNeighborsRegressor and then regress the residuals on some other list of features using LinearRegression.

Labels: New Feature, Needs Decision
Name of a class impacts the value of its __metadata_request__* variables

### Describe the bug

The metadata request of an object are dependent on its name.
Basically identical class can have different behaviours when calling `class()._get_metadata_request())` if they have different name.

### Steps/Code to Reproduce

```
from sklearn.model_selection import GroupKFold

class class_1(GroupKFold):
    __metadata_request__split = {"groups": "sample_domain"}

class Class_1(GroupKFold):
    __metadata_request__split = {"groups": "sample_domain"}

print(class_1()._get_metadata_request())
print(Class_1()._get_metadata_request())
```

### Expected Results

```
{'split': {'groups': 'sample_domain'}}
{'split': {'groups': 'sample_domain'}}
```

### Actual Results

```
{'split': {'groups': 'sample_domain'}}
{'split': {'groups': True}}
```

### Versions

```shell
System:
    python: 3.9.18 (main, Nov  2 2023, 16:52:00)  [Clang 14.0.0 (clang-1400.0.29.202)]
executable: /Users/yanislalou/Documents/CMAP/scikit-learn/sklearn-env/bin/python
   machine: macOS-12.7.2-arm64-arm-64bit

Python dependencies:
      sklearn: 1.5.dev0
          pip: 23.3.1
   setuptools: 68.2.2
        numpy: 1.26.4
        scipy: 1.12.0
       Cython: 3.0.8
       pandas: None
   matplotlib: None
       joblib: 1.3.2
threadpoolctl: 3.3.0

Built with OpenMP: False

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Users/yanislalou/Documents/CMAP/scikit-learn/sklearn-env/lib/python3.9/site-packages/numpy/.dylibs/libopenblas64_.0.dylib
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: armv8

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Users/yanislalou/Documents/CMAP/scikit-learn/sklearn-env/lib/python3.9/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: armv8
```

COMMENT:
I see the reproducible here. Thanks. Let me investigate.

Labels: Bug
preprocesor gives a Value Error when using Simple Imputer + StandardScaler in Pipeline (reopened issue)

### Describe the bug

I get a ValueError when using classifier.predict with a variable that has been preprocesse diwht a ColumnTransofrmer that uses SimpleIMputer+StandardScaler with numerical var and OneHotEncoder with categorical var.

```
base.py:457: UserWarning: X has feature names, but LogisticRegression was fitted without feature names
  warnings.warn(
anaconda3\Lib\site-packages\pandas\core\generic.py in ?(self, dtype)
   1996     def __array__(self, dtype: npt.DTypeLike | None = None) -> np.ndarray:
   1997         values = self._values
-> 1998         arr = np.asarray(values, dtype=dtype)
   1999         if (
   2000             astype_is_view(values.dtype, arr.dtype)
   2001             and using_copy_on_write()

ValueError: could not convert string to float: 'High School'
```

### Steps/Code to Reproduce

```
X2 = loan dataframe
y = target variable (loan approved)
X2_train, X2_test, y2_train, y2_test = train_test_split(X2,y2, test_size = 0.20, random_state = 0)


numeric_transformer = Pipeline(steps = [("imputer", SimpleImputer(strategy="mean")),
                                       ("scaler", StandardScaler())
                                       ])
categorical_transformer = OneHotEncoder(drop = "first") 

preprocessor = ColumnTransformer(
            transformers = [("numerical", numeric_transformer, numeric_features), 
                            ("categorical", categorical_transformer, categorical_features)]
)
preprocessor
X2_train = preprocessor.fit_transform(X2_train)
X2_test = preprocessor.transform(X2_test)

```

### Expected Results

No error thrown, and X2 and y ready to use a LogisticRegression() as classifier

### Actual Results

```
base.py:457: UserWarning: X has feature names, but LogisticRegression was fitted without feature names
  warnings.warn(
anaconda3\Lib\site-packages\pandas\core\generic.py in ?(self, dtype)
   1996     def __array__(self, dtype: npt.DTypeLike | None = None) -> np.ndarray:
   1997         values = self._values
-> 1998         arr = np.asarray(values, dtype=dtype)
   1999         if (
   2000             astype_is_view(values.dtype, arr.dtype)
   2001             and using_copy_on_write()

ValueError: could not convert string to float: 'High School'
```

### Versions

```shell
Python 3.11.5
Windows 10-10.0.22.631
sklearn: 1.3.0
scipy 1.11.1
pandas 2.0.3
numpy 1.24.3
Built with OpenMP
```

COMMENT:
Please refer to the documentation: https://scikit-learn.org/stable/modules/compose.html#column-transformer. I think it provides concrete example how to specify the information regarding the columns.

Labels: Bug, Needs Triage
Provide stubs for TargetEncoder

### Describe the workflow you want to enable

Being able to get autocompletions and type checking for sklearn.preprocessing.TargetEncoder

### Describe your proposed solution

I noticed that TargetEncoder was failing to be recognized by pylance/pyright. I assume that being a relatively  new class some stub is missing.


### Describe alternatives you've considered, if relevant

Enabling kernel-based completions.

### Additional context

I reported it to https://github.com/microsoft/python-type-stubs/issues/304.

Maybe you're in a better position than me to quickly create the missing stub. If that's the case, could you lend a hand with https://github.com/microsoft/python-type-stubs/issues/304?

Thanks

COMMENT:
We are note maintaining stubs in this repository. I assume that you need to open a PR on the `python-type-stubs` and create a file in `sklearn/preprocessing/_target_encoder.pxi` as for the other files.

Labels: New Feature, Needs Triage
Provide stubs for TargetEncoder

### Describe the workflow you want to enable

Being able to get autocompletions and type checking for sklearn.preprocessing.TargetEncoder

### Describe your proposed solution

I noticed that TargetEncoder was failing to be recognized by pylance/pyright. I assume that being a relatively  new class some stub is missing.


### Describe alternatives you've considered, if relevant

Enabling kernel-based completions.

### Additional context

I reported it to https://github.com/microsoft/python-type-stubs/issues/304.

Maybe you're in a better position than me to quickly create the missing stub. If that's the case, could you lend a hand with https://github.com/microsoft/python-type-stubs/issues/304?

Thanks

COMMENT:
i am not being able to import TargetEncoder
![Uploading targetsencoder.png…]()

Labels: New Feature, Needs Triage
Provide stubs for TargetEncoder

### Describe the workflow you want to enable

Being able to get autocompletions and type checking for sklearn.preprocessing.TargetEncoder

### Describe your proposed solution

I noticed that TargetEncoder was failing to be recognized by pylance/pyright. I assume that being a relatively  new class some stub is missing.


### Describe alternatives you've considered, if relevant

Enabling kernel-based completions.

### Additional context

I reported it to https://github.com/microsoft/python-type-stubs/issues/304.

Maybe you're in a better position than me to quickly create the missing stub. If that's the case, could you lend a hand with https://github.com/microsoft/python-type-stubs/issues/304?

Thanks

COMMENT:
TargetEncoder is not imported when the feature-engine library is installed in Google Colab. Instead the following error message appears: "ImportError: cannot import name 'TargetEncoder' from 'sklearn.preprocessing' (/usr/local/lib/python3.10/dist-packages/sklearn/preprocessing/__init__.py)"

Labels: New Feature, Needs Triage
Provide stubs for TargetEncoder

### Describe the workflow you want to enable

Being able to get autocompletions and type checking for sklearn.preprocessing.TargetEncoder

### Describe your proposed solution

I noticed that TargetEncoder was failing to be recognized by pylance/pyright. I assume that being a relatively  new class some stub is missing.


### Describe alternatives you've considered, if relevant

Enabling kernel-based completions.

### Additional context

I reported it to https://github.com/microsoft/python-type-stubs/issues/304.

Maybe you're in a better position than me to quickly create the missing stub. If that's the case, could you lend a hand with https://github.com/microsoft/python-type-stubs/issues/304?

Thanks

COMMENT:
Same here. TargetEncoder is not imported when the feature-engine library is installed in local Jupyter Notebook. Is acusing "ImportError: cannot import name 'TargetEncoder' from 'sklearn.preprocessing' (/usr/local/lib/python3.10/dist-packages/sklearn/preprocessing/init.py)"

Labels: New Feature, Needs Triage