Talip7/scikit-learn-issues-multilabel huggingface dataset:
PCA supports sparse now, docs suggest otherwise.

### Describe the issue linked to the documentation

The [latest release notes for 1.4](https://scikit-learn.org/stable/whats_new/v1.4.html) say the following about PCA. 

> Feature [decomposition.PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA) now supports [scipy.sparse.sparray](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.sparray.html#scipy.sparse.sparray) and [scipy.sparse.spmatrix](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.spmatrix.html#scipy.sparse.spmatrix) inputs when using the arpack solver. When used on sparse data like [datasets.fetch_20newsgroups_vectorized](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_20newsgroups_vectorized.html#sklearn.datasets.fetch_20newsgroups_vectorized) this can lead to speed-ups of 100x (single threaded) and 70x lower memory usage. Based on [Alexander Tarashansky](https://github.com/atarashansky)’s implementation in [scanpy](https://github.com/scverse/scanpy). [#18689](https://github.com/scikit-learn/scikit-learn/pull/18689) by [Isaac Virshup](https://github.com/ivirshup) and [Andrey Portnoy](https://github.com/andportnoy).

However, once you go to the [PCA docs](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA) it still says this. 

> Notice that this class does not support sparse input. See [TruncatedSVD](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.TruncatedSVD.html#sklearn.decomposition.TruncatedSVD) for an alternative with sparse data.

### Suggest a potential alternative/fix

I guess that one sentences can just be removed now? I can whip up a PR if folks agree.

COMMENT:
Indeed, we should correct the documentation.

Another PR to improve the situation is to change the behaviour of `"auto"` that leads to:

```pytb
TypeError: PCA only support sparse inputs with the "arpack" solver, while "auto" was passed. See TruncatedSVD for a possible alternative.
```

In a new PR, we should instead select the `"arpack"` solver and do not raise any error.

Labels: Documentation, Enhancement, help wanted
PCA supports sparse now, docs suggest otherwise.

### Describe the issue linked to the documentation

The [latest release notes for 1.4](https://scikit-learn.org/stable/whats_new/v1.4.html) say the following about PCA. 

> Feature [decomposition.PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA) now supports [scipy.sparse.sparray](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.sparray.html#scipy.sparse.sparray) and [scipy.sparse.spmatrix](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.spmatrix.html#scipy.sparse.spmatrix) inputs when using the arpack solver. When used on sparse data like [datasets.fetch_20newsgroups_vectorized](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_20newsgroups_vectorized.html#sklearn.datasets.fetch_20newsgroups_vectorized) this can lead to speed-ups of 100x (single threaded) and 70x lower memory usage. Based on [Alexander Tarashansky](https://github.com/atarashansky)’s implementation in [scanpy](https://github.com/scverse/scanpy). [#18689](https://github.com/scikit-learn/scikit-learn/pull/18689) by [Isaac Virshup](https://github.com/ivirshup) and [Andrey Portnoy](https://github.com/andportnoy).

However, once you go to the [PCA docs](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA) it still says this. 

> Notice that this class does not support sparse input. See [TruncatedSVD](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.TruncatedSVD.html#sklearn.decomposition.TruncatedSVD) for an alternative with sparse data.

### Suggest a potential alternative/fix

I guess that one sentences can just be removed now? I can whip up a PR if folks agree.

COMMENT:
> Indeed, we should correct the documentation.
> 
> Another PR to improve the situation is to change the behaviour of `"auto"` that leads to:
> 
> ```
> TypeError: PCA only support sparse inputs with the "arpack" solver, while "auto" was passed. See TruncatedSVD for a possible alternative.
> ```
> 
> In a new PR, we should instead select the `"arpack"` solver and do not raise any error.

I read the documentation and understand that `"auto"` will become `"randomized"` or `"full"` depending on `X.shape` and `n_components`. Should I replace  `"full"` with `"arpack"` to get rid of error raising?

Labels: Documentation, Enhancement, help wanted
PCA supports sparse now, docs suggest otherwise.

### Describe the issue linked to the documentation

The [latest release notes for 1.4](https://scikit-learn.org/stable/whats_new/v1.4.html) say the following about PCA. 

> Feature [decomposition.PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA) now supports [scipy.sparse.sparray](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.sparray.html#scipy.sparse.sparray) and [scipy.sparse.spmatrix](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.spmatrix.html#scipy.sparse.spmatrix) inputs when using the arpack solver. When used on sparse data like [datasets.fetch_20newsgroups_vectorized](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_20newsgroups_vectorized.html#sklearn.datasets.fetch_20newsgroups_vectorized) this can lead to speed-ups of 100x (single threaded) and 70x lower memory usage. Based on [Alexander Tarashansky](https://github.com/atarashansky)’s implementation in [scanpy](https://github.com/scverse/scanpy). [#18689](https://github.com/scikit-learn/scikit-learn/pull/18689) by [Isaac Virshup](https://github.com/ivirshup) and [Andrey Portnoy](https://github.com/andportnoy).

However, once you go to the [PCA docs](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA) it still says this. 

> Notice that this class does not support sparse input. See [TruncatedSVD](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.TruncatedSVD.html#sklearn.decomposition.TruncatedSVD) for an alternative with sparse data.

### Suggest a potential alternative/fix

I guess that one sentences can just be removed now? I can whip up a PR if folks agree.

COMMENT:
You may understand better if you look directly at the code.

https://github.com/scikit-learn/scikit-learn/blob/f323eb48a81bc7f2afd02587db83faf0d66b8bf3/sklearn/decomposition/_pca.py#L500-L508

My understanding is that if the input is sparse you directly select "arpack"; other things are left unchanged.

By the way you may need to refactor a bit to take care of the `n_components` as there is a special case for arpack.

Labels: Documentation, Enhancement, help wanted
PCA supports sparse now, docs suggest otherwise.

### Describe the issue linked to the documentation

The [latest release notes for 1.4](https://scikit-learn.org/stable/whats_new/v1.4.html) say the following about PCA. 

> Feature [decomposition.PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA) now supports [scipy.sparse.sparray](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.sparray.html#scipy.sparse.sparray) and [scipy.sparse.spmatrix](https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.spmatrix.html#scipy.sparse.spmatrix) inputs when using the arpack solver. When used on sparse data like [datasets.fetch_20newsgroups_vectorized](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_20newsgroups_vectorized.html#sklearn.datasets.fetch_20newsgroups_vectorized) this can lead to speed-ups of 100x (single threaded) and 70x lower memory usage. Based on [Alexander Tarashansky](https://github.com/atarashansky)’s implementation in [scanpy](https://github.com/scverse/scanpy). [#18689](https://github.com/scikit-learn/scikit-learn/pull/18689) by [Isaac Virshup](https://github.com/ivirshup) and [Andrey Portnoy](https://github.com/andportnoy).

However, once you go to the [PCA docs](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html#sklearn.decomposition.PCA) it still says this. 

> Notice that this class does not support sparse input. See [TruncatedSVD](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.TruncatedSVD.html#sklearn.decomposition.TruncatedSVD) for an alternative with sparse data.

### Suggest a potential alternative/fix

I guess that one sentences can just be removed now? I can whip up a PR if folks agree.

COMMENT:
> You may understand better if you look directly at the code.
> 
> https://github.com/scikit-learn/scikit-learn/blob/f323eb48a81bc7f2afd02587db83faf0d66b8bf3/sklearn/decomposition/_pca.py#L500-L508
> 
> My understanding is that if the input is sparse you directly select "arpack"; other things are left unchanged.
> 
> By the way you may need to refactor a bit to take care of the `n_components` as there is a special case for arpack.

Thanks for your clarification. I will do a PR on this

Labels: Documentation, Enhancement, help wanted
More intuitive check on GaussianMixture initialization

### Describe the workflow you want to enable

Hey!

I am using GaussianMixture and testing different initialization methods (kmeans, random, k-means++ and random_from_data). When I ran the code using `"init_params==kmeans++"` an error was produced in `sklearn/mixture/_base.py` `_initialize_parameters`, saying that responsibilities were referenced before assignment. Of course this is because I initialized the `GaussianMixture` incorrectly and typed `"kmeans++"` instead of `"k-means++"` (without the dash).

### Describe your proposed solution

It would be great if init_params was checked upon initialization of GaussianMixture: something like "if self.init_params not in ['random', 'random_from_data', 'kmeans', 'k-means++']" would produce an error.

Alternatively (and maybe even more robust), one could add the extra else statement in _base.py _initialize_parameters to make sure the initialization technique is spelled correctly.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
I assume that now that we have the parameter validation, the error message is catch up quickly and we get a much better error message.

I'm therefore closing.

Labels: New Feature
Data Frame columns get shuffled when applying "SimpleImputer".

### Describe the bug

When i apply the simple imputer to fill the missing values after imputation the columns get shuffled. It is difficult to find the and columns and name them.
![Screenshot 2024-02-10 141458](https://github.com/scikit-learn/scikit-learn/assets/131583096/e113ddd4-e975-40a4-ade3-2af35156ea82)
![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/df23acd8-fed9-4c7f-a5a8-51da90c77c74)
![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/84e068ad-7ee6-46d9-ab3a-aee457192f1f)


### Steps/Code to Reproduce

![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/ec119fe5-759d-4315-b2ee-7a303316b054)


### Expected Results

![Screenshot 2024-02-10 142448](https://github.com/scikit-learn/scikit-learn/assets/131583096/7c38f990-9d63-4df7-8c3e-7fa9deff0c29)

In the above column order but filled values

### Actual Results

![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/c96c3477-b0e2-4662-8de7-832a1ef2e49e)


### Versions

```shell
1.4.0
```

COMMENT:
I think the shuffling is from the `ColumnTransformer` and not the `SimpleImputer`. From the [Notes section in the docs](https://scikit-learn.org/stable/modules/generated/sklearn.compose.ColumnTransformer.html):

>The order of the columns in the transformed feature matrix follows the order of how the columns are specified in the transformers list. Columns of the original feature matrix that are not specified are dropped from the resulting transformed feature matrix, unless specified in the passthrough keyword. Those columns specified with passthrough are added at the right to the output of the transformers.

If you want to construct a `pandas.DataFrame` to match the output of the ` ColumnTransformer` you have at least two options:
1. Use the :method:`ColumnTransformer.set_output` and set `transform="pandas"` or set it globally with
```python
from sklearn import set_config

set_config(transform_output="pandas")
```
This will result in your `filled_X_train` being a `pandas.DataFrame` instead of a `numpy.ndarray`. [See here for more details](https://scikit-learn.org/stable/auto_examples/miscellaneous/plot_set_output.html).

2. Use the :method:`ColumnTransformer.get_feature_names_out` instead of the `X_train.columns` like so:
```python
pd.DataFrame(data=filled_X_train, columns=imputer.get_feature_names_out())
```
[Find more info here](https://scikit-learn.org/stable/modules/generated/sklearn.compose.ColumnTransformer.html#sklearn.compose.ColumnTransformer.get_feature_names_out).

Labels: Bug, Needs Triage
Data Frame columns get shuffled when applying "SimpleImputer".

### Describe the bug

When i apply the simple imputer to fill the missing values after imputation the columns get shuffled. It is difficult to find the and columns and name them.
![Screenshot 2024-02-10 141458](https://github.com/scikit-learn/scikit-learn/assets/131583096/e113ddd4-e975-40a4-ade3-2af35156ea82)
![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/df23acd8-fed9-4c7f-a5a8-51da90c77c74)
![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/84e068ad-7ee6-46d9-ab3a-aee457192f1f)


### Steps/Code to Reproduce

![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/ec119fe5-759d-4315-b2ee-7a303316b054)


### Expected Results

![Screenshot 2024-02-10 142448](https://github.com/scikit-learn/scikit-learn/assets/131583096/7c38f990-9d63-4df7-8c3e-7fa9deff0c29)

In the above column order but filled values

### Actual Results

![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/c96c3477-b0e2-4662-8de7-832a1ef2e49e)


### Versions

```shell
1.4.0
```

COMMENT:
This is not a bug this is indeed how `ColumnTransformer` works. Closing this issue.

Labels: Bug, Needs Triage
Data Frame columns get shuffled when applying "SimpleImputer".

### Describe the bug

When i apply the simple imputer to fill the missing values after imputation the columns get shuffled. It is difficult to find the and columns and name them.
![Screenshot 2024-02-10 141458](https://github.com/scikit-learn/scikit-learn/assets/131583096/e113ddd4-e975-40a4-ade3-2af35156ea82)
![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/df23acd8-fed9-4c7f-a5a8-51da90c77c74)
![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/84e068ad-7ee6-46d9-ab3a-aee457192f1f)


### Steps/Code to Reproduce

![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/ec119fe5-759d-4315-b2ee-7a303316b054)


### Expected Results

![Screenshot 2024-02-10 142448](https://github.com/scikit-learn/scikit-learn/assets/131583096/7c38f990-9d63-4df7-8c3e-7fa9deff0c29)

In the above column order but filled values

### Actual Results

![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/c96c3477-b0e2-4662-8de7-832a1ef2e49e)


### Versions

```shell
1.4.0
```

COMMENT:
Thanks @it176131 for putting to the right direction.

Labels: Bug, Needs Triage
Data Frame columns get shuffled when applying "SimpleImputer".

### Describe the bug

When i apply the simple imputer to fill the missing values after imputation the columns get shuffled. It is difficult to find the and columns and name them.
![Screenshot 2024-02-10 141458](https://github.com/scikit-learn/scikit-learn/assets/131583096/e113ddd4-e975-40a4-ade3-2af35156ea82)
![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/df23acd8-fed9-4c7f-a5a8-51da90c77c74)
![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/84e068ad-7ee6-46d9-ab3a-aee457192f1f)


### Steps/Code to Reproduce

![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/ec119fe5-759d-4315-b2ee-7a303316b054)


### Expected Results

![Screenshot 2024-02-10 142448](https://github.com/scikit-learn/scikit-learn/assets/131583096/7c38f990-9d63-4df7-8c3e-7fa9deff0c29)

In the above column order but filled values

### Actual Results

![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/c96c3477-b0e2-4662-8de7-832a1ef2e49e)


### Versions

```shell
1.4.0
```

COMMENT:
Thank you

Labels: Bug, Needs Triage
Data Frame columns get shuffled when applying "SimpleImputer".

### Describe the bug

When i apply the simple imputer to fill the missing values after imputation the columns get shuffled. It is difficult to find the and columns and name them.
![Screenshot 2024-02-10 141458](https://github.com/scikit-learn/scikit-learn/assets/131583096/e113ddd4-e975-40a4-ade3-2af35156ea82)
![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/df23acd8-fed9-4c7f-a5a8-51da90c77c74)
![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/84e068ad-7ee6-46d9-ab3a-aee457192f1f)


### Steps/Code to Reproduce

![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/ec119fe5-759d-4315-b2ee-7a303316b054)


### Expected Results

![Screenshot 2024-02-10 142448](https://github.com/scikit-learn/scikit-learn/assets/131583096/7c38f990-9d63-4df7-8c3e-7fa9deff0c29)

In the above column order but filled values

### Actual Results

![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/c96c3477-b0e2-4662-8de7-832a1ef2e49e)


### Versions

```shell
1.4.0
```

COMMENT:
As you say passthrough added the columns on which we don't apply simple imputer added to right of the transformed data frame .
But i want the want to order that is was before imputation

Labels: Bug, Needs Triage
Data Frame columns get shuffled when applying "SimpleImputer".

### Describe the bug

When i apply the simple imputer to fill the missing values after imputation the columns get shuffled. It is difficult to find the and columns and name them.
![Screenshot 2024-02-10 141458](https://github.com/scikit-learn/scikit-learn/assets/131583096/e113ddd4-e975-40a4-ade3-2af35156ea82)
![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/df23acd8-fed9-4c7f-a5a8-51da90c77c74)
![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/84e068ad-7ee6-46d9-ab3a-aee457192f1f)


### Steps/Code to Reproduce

![Screenshot 2024-02-10 141508](https://github.com/scikit-learn/scikit-learn/assets/131583096/ec119fe5-759d-4315-b2ee-7a303316b054)


### Expected Results

![Screenshot 2024-02-10 142448](https://github.com/scikit-learn/scikit-learn/assets/131583096/7c38f990-9d63-4df7-8c3e-7fa9deff0c29)

In the above column order but filled values

### Actual Results

![Screenshot 2024-02-10 141515](https://github.com/scikit-learn/scikit-learn/assets/131583096/c96c3477-b0e2-4662-8de7-832a1ef2e49e)


### Versions

```shell
1.4.0
```

COMMENT:
You can potentially use function together with a `FunctionTransformer` to do that.

Labels: Bug, Needs Triage
Add transformations to target column

### Describe the workflow you want to enable

I would like Pipelines to allow the transformation of the target columns of a dataset.
Additionally, to complement this, this transformation should be able to be reversed at the end of a pipeline, which is now impossible, as intermediate steps in a pipeline must implement transform, which most models don't implement, implying that there currently cannot be any step to transform the output of a model inside a pipeline.

### Describe your proposed solution

This would involve making the transform method on classes like FunctionTransformer accept a 'y' parameter and return the dataset as a tuple (X_new, y_new).

### Describe alternatives you've considered, if relevant

If this interferes too much with the existing implementation, a new method could be added that handles only target variables.

### Additional context

This comes from a proyect i'm working on where i have to develop a regression model. It turns out transforming the output by applying a logarithm before predicting the output and reverting the transformation after the model gives an answer works really well, but I'm forced to do these steps manually. It would be really useful for me to have this functionality as part of this package. 

Edit: An aditional use for this is applying normalization to the output of a regression model. If the target value has values that are too high, models like MLPs might have trouble giving satisfactory results. This also ocurred to me in another project i was working on.

COMMENT:
I would also like to add that I would be willing to program this functionality myself if you give this feature the green light.

Labels: New Feature, Needs Triage
Add transformations to target column

### Describe the workflow you want to enable

I would like Pipelines to allow the transformation of the target columns of a dataset.
Additionally, to complement this, this transformation should be able to be reversed at the end of a pipeline, which is now impossible, as intermediate steps in a pipeline must implement transform, which most models don't implement, implying that there currently cannot be any step to transform the output of a model inside a pipeline.

### Describe your proposed solution

This would involve making the transform method on classes like FunctionTransformer accept a 'y' parameter and return the dataset as a tuple (X_new, y_new).

### Describe alternatives you've considered, if relevant

If this interferes too much with the existing implementation, a new method could be added that handles only target variables.

### Additional context

This comes from a proyect i'm working on where i have to develop a regression model. It turns out transforming the output by applying a logarithm before predicting the output and reverting the transformation after the model gives an answer works really well, but I'm forced to do these steps manually. It would be really useful for me to have this functionality as part of this package. 

Edit: An aditional use for this is applying normalization to the output of a regression model. If the target value has values that are too high, models like MLPs might have trouble giving satisfactory results. This also ocurred to me in another project i was working on.

COMMENT:
It sounds like https://scikit-learn.org/stable/modules/generated/sklearn.compose.TransformedTargetRegressor.html would full fill your need. Have you had a look at it?

Labels: New Feature, Needs Triage