Talip7/scikit-learn-issues-multilabel huggingface dataset:
Add transformations to target column

### Describe the workflow you want to enable

I would like Pipelines to allow the transformation of the target columns of a dataset.
Additionally, to complement this, this transformation should be able to be reversed at the end of a pipeline, which is now impossible, as intermediate steps in a pipeline must implement transform, which most models don't implement, implying that there currently cannot be any step to transform the output of a model inside a pipeline.

### Describe your proposed solution

This would involve making the transform method on classes like FunctionTransformer accept a 'y' parameter and return the dataset as a tuple (X_new, y_new).

### Describe alternatives you've considered, if relevant

If this interferes too much with the existing implementation, a new method could be added that handles only target variables.

### Additional context

This comes from a proyect i'm working on where i have to develop a regression model. It turns out transforming the output by applying a logarithm before predicting the output and reverting the transformation after the model gives an answer works really well, but I'm forced to do these steps manually. It would be really useful for me to have this functionality as part of this package. 

Edit: An aditional use for this is applying normalization to the output of a regression model. If the target value has values that are too high, models like MLPs might have trouble giving satisfactory results. This also ocurred to me in another project i was working on.

COMMENT:
Thank you for your answer.
I had no idea this existed, it does exactly what i needed, my bad.

Labels: New Feature, Needs Triage
⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️

**CI is still failing on [Linux_Nightly_PyPy.pypy3](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=65359&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66)** (Mar 28, 2024)
Unable to find junit file. Please see link for details.

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=65394&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66) on Mar 29, 2024

Labels: Needs Investigation
⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️

**CI is still failing on [Linux_Nightly_PyPy.pypy3](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=65359&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66)** (Mar 28, 2024)
Unable to find junit file. Please see link for details.

COMMENT:
Looking at the past runs, this job fails every once in a while with this kind of error:
```
##[error]Exit code 137 returned from process: file name '/home/vsts/agents/3.234.0/externals/node20_1/bin/node', arguments '"/home/vsts/work/_tasks/CmdLine_d9bafed4-0b18-4f58-968d-86655b4d2ce9/2.231.1/cmdline.js"'.
```

looking around, code 137 seems to mean excessive memory usage.

Labels: Needs Investigation
⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️

**CI is still failing on [Linux_Nightly_PyPy.pypy3](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=65359&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66)** (Mar 28, 2024)
Unable to find junit file. Please see link for details.

COMMENT:
In an ideal world, someone would do something similar as https://github.com/scikit-learn/scikit-learn/issues/27662 to try to find a way to reduce memory usage of PyPy tests.

I don't plan to do this in the foreseeable future, and last time I looked, it was very time-consuming for a small benefit. I don't think that PyPy support is very high on the scikit-learn priority list. Some selected quotes from a Hacker News question [Ask HN: Is anyone using PyPy for real work?](https://news.ycombinator.com/item?id=36940871)

> It does seem like parsing logs and simulations is a sweet spot for PyPy

> I use PyPy quite often as a 'free' way to make some non-numpy CPU-bound Python script faster. This is also the context for when I bring up PyPy to others.

Labels: Needs Investigation
⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️

**CI is still failing on [Linux_Nightly_PyPy.pypy3](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=65359&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66)** (Mar 28, 2024)
Unable to find junit file. Please see link for details.

COMMENT:
When I say small benefit it seems like after my investigation and the "memory leak" fix https://github.com/scikit-learn/scikit-learn/pull/27670, the PyPy build was failing 3 times in a month https://github.com/scikit-learn/scikit-learn/issues/27750#issuecomment-1933539785 and now it is more like 12 times in a month (looking at the editions of the top comment). Moving to scipy 1.12 may be one of the reason not 100% sure.

Labels: Needs Investigation
⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️

**CI is still failing on [Linux_Nightly_PyPy.pypy3](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=65359&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66)** (Mar 28, 2024)
Unable to find junit file. Please see link for details.

COMMENT:
Cross-linking a proposal to downgrade our level of PyPy support:

https://github.com/scikit-learn/scikit-learn/pull/28567#issuecomment-1978427572

Labels: Needs Investigation
⚠️ CI failed on Linux_Nightly_PyPy.pypy3 ⚠️

**CI is still failing on [Linux_Nightly_PyPy.pypy3](https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=65359&view=logs&j=0b16f832-29d6-5b92-1c23-eb006f606a66)** (Mar 28, 2024)
Unable to find junit file. Please see link for details.

COMMENT:
I am +1 on downgrading our PyPy support.

Labels: Needs Investigation
Apparent mismatch between possible arguments for `average` in the base stochastic gradient class

### Describe the bug

Raised in https://github.com/scikit-learn/scikit-learn/pull/28373#discussion_r1482869890.

The  `average` parameter in [`BaseSGD`](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_stochastic_gradient.py#L82-L334) (which propagates to `SGDRegressor`, `SGDClassifier` and `SGDOneClassSVM`)  [is constrained](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_stochastic_gradient.py#L93) to non-negative integers or boolean values: `"average": [Interval(Integral, 0, None, closed="left"), bool, np.bool_]`.

This seems to be at odds with `average=0` seemingly meaning `average=True` which contradicts the typical truth-evaluation of `0`.


### Steps/Code to Reproduce

```
```

### Expected Results

```
```

### Actual Results

```
```

### Versions

This is in the main branch right now.

COMMENT:
Thanks for the report @bmgdc. Looking at the code, ``average=0`` seems to mean ``average=False`` since averaging is only triggered by ``if self.average > 0``. So to me the constraint is not confusing. I wouldn't mind that the interval starts at 1 but I guess we let 0 for convenience since the ``bool`` constraint doesn't treat 0 as a boolean (``isinstance(0, bool) -> False``).

Labels: Bug, Validation
Apparent mismatch between possible arguments for `average` in the base stochastic gradient class

### Describe the bug

Raised in https://github.com/scikit-learn/scikit-learn/pull/28373#discussion_r1482869890.

The  `average` parameter in [`BaseSGD`](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_stochastic_gradient.py#L82-L334) (which propagates to `SGDRegressor`, `SGDClassifier` and `SGDOneClassSVM`)  [is constrained](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_stochastic_gradient.py#L93) to non-negative integers or boolean values: `"average": [Interval(Integral, 0, None, closed="left"), bool, np.bool_]`.

This seems to be at odds with `average=0` seemingly meaning `average=True` which contradicts the typical truth-evaluation of `0`.


### Steps/Code to Reproduce

```
```

### Expected Results

```
```

### Actual Results

```
```

### Versions

This is in the main branch right now.

COMMENT:
ping @glemaitre @betatim who originaly discussed that in https://github.com/scikit-learn/scikit-learn/pull/28373#discussion_r1482869890, what's your opinion ?

Labels: Bug, Validation
Apparent mismatch between possible arguments for `average` in the base stochastic gradient class

### Describe the bug

Raised in https://github.com/scikit-learn/scikit-learn/pull/28373#discussion_r1482869890.

The  `average` parameter in [`BaseSGD`](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_stochastic_gradient.py#L82-L334) (which propagates to `SGDRegressor`, `SGDClassifier` and `SGDOneClassSVM`)  [is constrained](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_stochastic_gradient.py#L93) to non-negative integers or boolean values: `"average": [Interval(Integral, 0, None, closed="left"), bool, np.bool_]`.

This seems to be at odds with `average=0` seemingly meaning `average=True` which contradicts the typical truth-evaluation of `0`.


### Steps/Code to Reproduce

```
```

### Expected Results

```
```

### Actual Results

```
```

### Versions

This is in the main branch right now.

COMMENT:
Maybe this is better to keep as-is. Otherwise, we might start to break some code and request a boolean in the previous case of 0. This is maybe not worth it.

We might want to have a comment just to document this legacy behaviour.

Labels: Bug, Validation
Apparent mismatch between possible arguments for `average` in the base stochastic gradient class

### Describe the bug

Raised in https://github.com/scikit-learn/scikit-learn/pull/28373#discussion_r1482869890.

The  `average` parameter in [`BaseSGD`](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_stochastic_gradient.py#L82-L334) (which propagates to `SGDRegressor`, `SGDClassifier` and `SGDOneClassSVM`)  [is constrained](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_stochastic_gradient.py#L93) to non-negative integers or boolean values: `"average": [Interval(Integral, 0, None, closed="left"), bool, np.bool_]`.

This seems to be at odds with `average=0` seemingly meaning `average=True` which contradicts the typical truth-evaluation of `0`.


### Steps/Code to Reproduce

```
```

### Expected Results

```
```

### Actual Results

```
```

### Versions

This is in the main branch right now.

COMMENT:
or deprecate 0 ? Since we deprecated integers for the boolean constraint to make boolean parameters require an true boolean, deprecating 0 would make it consistent with the rest of the code base.

Labels: Bug, Validation
Apparent mismatch between possible arguments for `average` in the base stochastic gradient class

### Describe the bug

Raised in https://github.com/scikit-learn/scikit-learn/pull/28373#discussion_r1482869890.

The  `average` parameter in [`BaseSGD`](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_stochastic_gradient.py#L82-L334) (which propagates to `SGDRegressor`, `SGDClassifier` and `SGDOneClassSVM`)  [is constrained](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/linear_model/_stochastic_gradient.py#L93) to non-negative integers or boolean values: `"average": [Interval(Integral, 0, None, closed="left"), bool, np.bool_]`.

This seems to be at odds with `average=0` seemingly meaning `average=True` which contradicts the typical truth-evaluation of `0`.


### Steps/Code to Reproduce

```
```

### Expected Results

```
```

### Actual Results

```
```

### Versions

This is in the main branch right now.

COMMENT:
So let's make it consistent then and ask to pass False instead of 0

Labels: Bug, Validation
Proper sparse support in `IncrementalPCA`

### Describe the workflow you want to enable

As of version 1.4 the implementation for PCA supports sparse inputs. That's grand, but the PCA implementation doesn't offer a `.partial_fit()` method. The `IncrementalPCA` does, but the way it handles sparse input is by casting it to dense first. This "works", but it may be more elegant/preformant to apply the same sparse trick to the `IncrementalPCA` implementation. 

### Describe your proposed solution

As the [release notes](https://scikit-learn.org/stable/auto_examples/release_highlights/plot_release_highlights_1_4_0.html#improved-memory-and-runtime-efficiency-for-pca-on-sparse-data) suggest it may just be a matter of investigating if the `scipy.sparse.linalg.LinearOperator` can be used here as well. But I'd need to dive in and check the implementation to understand if this is possible. 

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I am trying to work on this.

Labels: New Feature
⚠️ CI failed on Wheel builder ⚠️

**CI failed on [Wheel builder](https://github.com/scikit-learn/scikit-learn/actions/runs/7794393529)** (Feb 06, 2024)

COMMENT:
## CI is no longer failing! ✅

[Successful run](https://github.com/scikit-learn/scikit-learn/actions/runs/7809629058) on Feb 07, 2024

Labels: Needs Triage
Allow AUC precision to be specified by user rather than hard-coded to 0.2f

Currently when calling the `RocCurveDisplay.from_predictions()` function, there's no way to specify the AUC precision.

2 decimal places is sufficient for most situations, but many times the user needs higher precision.  The API should be modified to enable the user to specify the precision used.

https://github.com/scikit-learn/scikit-learn/blob/7e1e6d09bcc2eaeba98f7e737aac2ac782f0e5f1/sklearn/metrics/_plot/roc_curve.py#L111

https://github.com/scikit-learn/scikit-learn/blob/7e1e6d09bcc2eaeba98f7e737aac2ac782f0e5f1/sklearn/metrics/_plot/roc_curve.py#L113

COMMENT:
> there's no way to specify the AUC precision.

Actually, you have the element to do it:

```python
import matplotlib.pyplot as plt
from sklearn.datasets import make_classification
from sklearn.metrics import RocCurveDisplay
from sklearn.model_selection import train_test_split
from sklearn.svm import SVC
X, y = make_classification(random_state=0)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, random_state=0)
clf = SVC(random_state=0).fit(X_train, y_train)
y_pred = clf.decision_function(X_test)
disp = RocCurveDisplay.from_predictions(y_test, y_pred)
# set some custom legend
disp.line_.set_label(f"Classifier (AUC = {disp.roc_auc:.3f})")
disp.ax_.legend()
```

The two last line are the one allowing to set a custom label.
I don't think that we should limit the number of keywords when it comes to tweaking the plot. If this is kind of straightforward to do it with matplotlib, this is better to let it as-is. We should focus on having sensible default though.

Labels: Needs Triage
Allow AUC precision to be specified by user rather than hard-coded to 0.2f

Currently when calling the `RocCurveDisplay.from_predictions()` function, there's no way to specify the AUC precision.

2 decimal places is sufficient for most situations, but many times the user needs higher precision.  The API should be modified to enable the user to specify the precision used.

https://github.com/scikit-learn/scikit-learn/blob/7e1e6d09bcc2eaeba98f7e737aac2ac782f0e5f1/sklearn/metrics/_plot/roc_curve.py#L111

https://github.com/scikit-learn/scikit-learn/blob/7e1e6d09bcc2eaeba98f7e737aac2ac782f0e5f1/sklearn/metrics/_plot/roc_curve.py#L113

COMMENT:
I think this is reasonable solution, closing.

Labels: Needs Triage
Allow AUC precision to be specified by user rather than hard-coded to 0.2f

Currently when calling the `RocCurveDisplay.from_predictions()` function, there's no way to specify the AUC precision.

2 decimal places is sufficient for most situations, but many times the user needs higher precision.  The API should be modified to enable the user to specify the precision used.

https://github.com/scikit-learn/scikit-learn/blob/7e1e6d09bcc2eaeba98f7e737aac2ac782f0e5f1/sklearn/metrics/_plot/roc_curve.py#L111

https://github.com/scikit-learn/scikit-learn/blob/7e1e6d09bcc2eaeba98f7e737aac2ac782f0e5f1/sklearn/metrics/_plot/roc_curve.py#L113

COMMENT:
Setting the precision with `line_.set_label()` worked, thank you!
```
disp = RocCurveDisplay.from_predictions(y_test, y_pred)
# set some custom legend
disp.line_.set_label(f"Classifier (AUC = {disp.roc_auc:.3f})")
disp.ax_.legend()
```

Labels: Needs Triage
GridSearchCV with PCA returns `object` masked array

### Describe the bug

I noticed this while looking into https://github.com/scikit-learn/scikit-learn/pull/28345

The dtype of the `components_col` is `object`, which means that the pandas object which is then created is of dtype `object`.

### Steps/Code to Reproduce
```python
import numpy as np

from sklearn import datasets
from sklearn.decomposition import PCA
from sklearn.model_selection import GridSearchCV

pca = PCA()
X_digits, y_digits = datasets.load_digits(return_X_y=True)
param_grid = {"n_components": [5, 15]}
search = GridSearchCV(pca, param_grid)
search.fit(X_digits, y_digits)

print(search.cv_results_['param_n_components'].data.dtype)
```
### Expected Results

int64

### Actual Results

object

### Versions

```shell
System:
    python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
executable: /home/marcogorelli/tmp/.venv/bin/python3.10
   machine: Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

Python dependencies:
      sklearn: 1.4.0
          pip: 22.0.2
   setuptools: 59.6.0
        numpy: 1.26.3
        scipy: 1.11.4
       Cython: None
       pandas: 2.2.0
   matplotlib: 3.8.2
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 16
         prefix: libopenblas
       filepath: /home/marcogorelli/tmp/.venv/lib/python3.10/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 16
         prefix: libgomp
       filepath: /home/marcogorelli/tmp/.venv/lib/python3.10/site-packages/scikit_learn.libs/libgomp-a34b3233.so.1.0.0
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 16
         prefix: libopenblas
       filepath: /home/marcogorelli/tmp/.venv/lib/python3.10/site-packages/scipy.libs/libopenblasp-r0-23e5df77.3.21.dev.so
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: SkylakeX
```

COMMENT:
Hmm, we were just talking the other day about improving `cv_results_` and the tools around it.

But this is true, that the dtype of `n_components` inside PCA is `int`, but here we get `object`.

Labels: Bug
RFC switch to Polars as the default dataframe lib in our examples

We now support polars as an output for our transformers and `ColumnTransformer`, but our examples use `pd.DataFrame`.

Our `datasets` module also returnes either a numpy array or a pandas DataFrame.

I'm suggesting that we enable datasets to return a polars dataframe, and to switch our examples from pandas to polars.

This has a few benefits:
- Polars on users' systems takes advantage of multi-core CPUs which is the case for pretty much all users these days. So it's quite faster in most cases.
- Pandas is dealing with issues related to Arrow, and even if they don't require Arrow as a required dependency, there will be behavior changes whether Arrow is installed or not at least on String Dtype.

Another thing is the (in)stability issues related to Pandas API where we need to deal with deprecation warnings very often. Although I'm not sure how stable the API is on the places where we touch the API on the polars side (maybe cc @MarcoGorelli )

WDYT @scikit-learn/core-devs @scikit-learn/documentation-team @scikit-learn/contributor-experience-team 

This is not really a core part of our library, but what we put in our examples and our default choices affect people since many learn from our examples.

COMMENT:
> I'm suggesting that we enable datasets to return a polars dataframe, and to switch our examples from pandas to polars.

While I love polars, I think that most users still use pandas. I thus would argue to have both in the examples. By both, I do not mean double the work, but switch a few, keep a few in pandas, and each time mention in a comment that scikit-learn works with both.

Labels: RFC
RFC switch to Polars as the default dataframe lib in our examples

We now support polars as an output for our transformers and `ColumnTransformer`, but our examples use `pd.DataFrame`.

Our `datasets` module also returnes either a numpy array or a pandas DataFrame.

I'm suggesting that we enable datasets to return a polars dataframe, and to switch our examples from pandas to polars.

This has a few benefits:
- Polars on users' systems takes advantage of multi-core CPUs which is the case for pretty much all users these days. So it's quite faster in most cases.
- Pandas is dealing with issues related to Arrow, and even if they don't require Arrow as a required dependency, there will be behavior changes whether Arrow is installed or not at least on String Dtype.

Another thing is the (in)stability issues related to Pandas API where we need to deal with deprecation warnings very often. Although I'm not sure how stable the API is on the places where we touch the API on the polars side (maybe cc @MarcoGorelli )

WDYT @scikit-learn/core-devs @scikit-learn/documentation-team @scikit-learn/contributor-experience-team 

This is not really a core part of our library, but what we put in our examples and our default choices affect people since many learn from our examples.

COMMENT:
I agree with @GaelVaroquaux. I would be considering having a mix of example, even more when we don't take profit from polars features.

Currently our support for polars is closer to rewrapping a NumPy arrays into a Polars DataFrame without leveraging the lazyness.

In an ideal world, if we come to have estimator to leverage those features (e.g. via the dataframe API) then we should certainly demonstrate it in the documentation.

Labels: RFC