Talip7/scikit-learn-issues-multilabel huggingface dataset:
test_ridge_regression_vstacked_X is not stable

https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=40346&view=logs&jobId=97641769-79fb-5590-9088-a30ce9b850b9&j=97641769-79fb-5590-9088-a30ce9b850b9&t=4745baa1-36b5-56c8-9a8e-6480742db1a6

introduced in #22910. 
@lorentzenchr was it tested on "all" random seeds ? (`SKLEARN_TESTS_GLOBAL_RANDOM_SEED=all`)

COMMENT:
I got an error on `test_ridge_regression_vstacked_X[long-16-False-sag]`, second time on `test_ridge_regression_vstacked_X[long-63-True-sag]`, then no error running the tests 5 times, then error on `test_ridge_regression_vstacked_X[wide-75-True-sag]`, ...

> Is it possible that Ridge(solver="sag") is not completely deterministic even on fixed random_state ?

It should not be the case, so there seems to be a bug.

Labels: module:linear_model, module:test-suite
test_ridge_regression_vstacked_X is not stable

https://dev.azure.com/scikit-learn/scikit-learn/_build/results?buildId=40346&view=logs&jobId=97641769-79fb-5590-9088-a30ce9b850b9&j=97641769-79fb-5590-9088-a30ce9b850b9&t=4745baa1-36b5-56c8-9a8e-6480742db1a6

introduced in #22910. 
@lorentzenchr was it tested on "all" random seeds ? (`SKLEARN_TESTS_GLOBAL_RANDOM_SEED=all`)

COMMENT:
If `sag` uses BLAS, it's possible that rounding errors accumulate differently because of the multi-threading, (or maybe it's in the construction of the test).

Labels: module:linear_model, module:test-suite
Dataframe also as a sklearn transform output

Hi all, 
i work intensively with sklearn pipeline for building ML pipelines and pre-processing. Sklearn has been really useful in this case. 
The thing is post model training , i would like to know what my dataset looks like, which columns were dropped, or transformed. However, the sklearn Pipeline outputs numpy ndarray and this make its difficult. 
Lets take an example, i have Imputer(), but i make sure it outputs as a dataframe and the columns are available. Now i want to transform my test dataset, 
if my pipeline is suppose : 
```
preprocessing_pipeline = FeatureUnion(
            transformer_list=[
                ("cate_pipeline", categorical_pipeline),
                ("num_pipeline", numerical_pipeline),
            ]
        )

        sklearn_pipeline = Pipeline(
            steps=[
                ("ppreprocess", preprocessing_pipeline),
                ("Standard_scaling", StandardScaler()),
                ("model", XGBRegressor()),
            ]
        )
```
And now after training, i wish to transform my test data

```
preprocess_pipeline = model.best_estimator_.named_steps["ppreprocess"]
transform_testdata = preprocess_pipeline .transform(testdata)
```
This gives me output in numpy. 
I digged a little deeper into the sklearn code and found that in the pipeline.py in the transform function, the transformed data stacked with np.hstack() and is returned as numpy.ndarray  . 

https://github.com/scikit-learn/scikit-learn/blob/c61086bff7e086bccf3fcb40b80d1c5a9f5f2aa8/sklearn/pipeline.py#L1218

Is there any way i could receive my transforms along with column names?

Any leads on this would be helpful.
Thanks

COMMENT:
This issue has been raised several times. It's slowly going in that direction. See the latest thoughts on such API https://github.com/scikit-learn/scikit-learn/issues/23001

In the meantime you can access the features names through the `get_feature_names_out` of the transformers.

I'm closing this issue in favor of #23001

Labels: Needs Triage
Dataframe also as a sklearn transform output

Hi all, 
i work intensively with sklearn pipeline for building ML pipelines and pre-processing. Sklearn has been really useful in this case. 
The thing is post model training , i would like to know what my dataset looks like, which columns were dropped, or transformed. However, the sklearn Pipeline outputs numpy ndarray and this make its difficult. 
Lets take an example, i have Imputer(), but i make sure it outputs as a dataframe and the columns are available. Now i want to transform my test dataset, 
if my pipeline is suppose : 
```
preprocessing_pipeline = FeatureUnion(
            transformer_list=[
                ("cate_pipeline", categorical_pipeline),
                ("num_pipeline", numerical_pipeline),
            ]
        )

        sklearn_pipeline = Pipeline(
            steps=[
                ("ppreprocess", preprocessing_pipeline),
                ("Standard_scaling", StandardScaler()),
                ("model", XGBRegressor()),
            ]
        )
```
And now after training, i wish to transform my test data

```
preprocess_pipeline = model.best_estimator_.named_steps["ppreprocess"]
transform_testdata = preprocess_pipeline .transform(testdata)
```
This gives me output in numpy. 
I digged a little deeper into the sklearn code and found that in the pipeline.py in the transform function, the transformed data stacked with np.hstack() and is returned as numpy.ndarray  . 

https://github.com/scikit-learn/scikit-learn/blob/c61086bff7e086bccf3fcb40b80d1c5a9f5f2aa8/sklearn/pipeline.py#L1218

Is there any way i could receive my transforms along with column names?

Any leads on this would be helpful.
Thanks

COMMENT:
i do not use ColumnTransformer, im using Pipelines and FeatureUnion. Its in the FeatureUnion class where its converted into numpy ndarray.
AttributeError: 'FeatureUnion' object has no attribute 'get_feature_names_out'

Labels: Needs Triage
Dataframe also as a sklearn transform output

Hi all, 
i work intensively with sklearn pipeline for building ML pipelines and pre-processing. Sklearn has been really useful in this case. 
The thing is post model training , i would like to know what my dataset looks like, which columns were dropped, or transformed. However, the sklearn Pipeline outputs numpy ndarray and this make its difficult. 
Lets take an example, i have Imputer(), but i make sure it outputs as a dataframe and the columns are available. Now i want to transform my test dataset, 
if my pipeline is suppose : 
```
preprocessing_pipeline = FeatureUnion(
            transformer_list=[
                ("cate_pipeline", categorical_pipeline),
                ("num_pipeline", numerical_pipeline),
            ]
        )

        sklearn_pipeline = Pipeline(
            steps=[
                ("ppreprocess", preprocessing_pipeline),
                ("Standard_scaling", StandardScaler()),
                ("model", XGBRegressor()),
            ]
        )
```
And now after training, i wish to transform my test data

```
preprocess_pipeline = model.best_estimator_.named_steps["ppreprocess"]
transform_testdata = preprocess_pipeline .transform(testdata)
```
This gives me output in numpy. 
I digged a little deeper into the sklearn code and found that in the pipeline.py in the transform function, the transformed data stacked with np.hstack() and is returned as numpy.ndarray  . 

https://github.com/scikit-learn/scikit-learn/blob/c61086bff7e086bccf3fcb40b80d1c5a9f5f2aa8/sklearn/pipeline.py#L1218

Is there any way i could receive my transforms along with column names?

Any leads on this would be helpful.
Thanks

COMMENT:
What's your version of scikit-learn ? It's available in 1.0+

Labels: Needs Triage
Dataframe also as a sklearn transform output

Hi all, 
i work intensively with sklearn pipeline for building ML pipelines and pre-processing. Sklearn has been really useful in this case. 
The thing is post model training , i would like to know what my dataset looks like, which columns were dropped, or transformed. However, the sklearn Pipeline outputs numpy ndarray and this make its difficult. 
Lets take an example, i have Imputer(), but i make sure it outputs as a dataframe and the columns are available. Now i want to transform my test dataset, 
if my pipeline is suppose : 
```
preprocessing_pipeline = FeatureUnion(
            transformer_list=[
                ("cate_pipeline", categorical_pipeline),
                ("num_pipeline", numerical_pipeline),
            ]
        )

        sklearn_pipeline = Pipeline(
            steps=[
                ("ppreprocess", preprocessing_pipeline),
                ("Standard_scaling", StandardScaler()),
                ("model", XGBRegressor()),
            ]
        )
```
And now after training, i wish to transform my test data

```
preprocess_pipeline = model.best_estimator_.named_steps["ppreprocess"]
transform_testdata = preprocess_pipeline .transform(testdata)
```
This gives me output in numpy. 
I digged a little deeper into the sklearn code and found that in the pipeline.py in the transform function, the transformed data stacked with np.hstack() and is returned as numpy.ndarray  . 

https://github.com/scikit-learn/scikit-learn/blob/c61086bff7e086bccf3fcb40b80d1c5a9f5f2aa8/sklearn/pipeline.py#L1218

Is there any way i could receive my transforms along with column names?

Any leads on this would be helpful.
Thanks

COMMENT:
Ok that maybe that is the reason, i am using 0.24.2, i will update and check once, thanks

Labels: Needs Triage
Decouple CountVectorizer => TextTokenizer + ItemCountVectorizer

### Describe the workflow you want to enable

The `CountVectorizer` component has the responsibility of not just vectorizing term frequencies but also internally tokenizing & normalizing the input text into terms. While this functionality is convenient for NLP pipelines, it cannot be leveraged for similar type of non-NLP problems where one needs to perform tf-idf (or similar) on "document" bags of general items other than text.

### Describe your proposed solution

Decouple the `CountVectorizer` into the following 2 specific purpose & reusable components: `TextTokenizer` & `ItemCountVectorizer` such that when both are run sequentially in a pipeline they produce the same behavior.

### Describe alternatives you've considered, if relevant

In some but not all cases one could force this (vectorizing item frequency) behavior by carefully serializing the bag of items into a faked string document & configuring the `CountVectorizer` tokenizer parameters to split on specific delimiters. But this brittle serialization & string de-serialization becomes inefficient & a bit hacky.

### Additional context

_No response_

COMMENT:
If this functionality it deemed desired, attached below is a working draft of the proposed `ItemCountVectorizer` with limited functionality derived from the `CountVectorizer` that served our purposes: [ItemCountVectorizer.py.zip](https://github.com/scikit-learn/scikit-learn/files/8389759/ItemCountVectorizer.py.zip)

The `TextTokenizer` component would need refactoring out from `CountVectorizer`. Also for backwards compatibility with user pipelines depending on `CountVectorizer`, this component could extend from `Pipeline` implementing the combination of these two new components. 

Of course, all of this would need to be integrated, tested & documented to scikit-learn standards.

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
Decouple CountVectorizer => TextTokenizer + ItemCountVectorizer

### Describe the workflow you want to enable

The `CountVectorizer` component has the responsibility of not just vectorizing term frequencies but also internally tokenizing & normalizing the input text into terms. While this functionality is convenient for NLP pipelines, it cannot be leveraged for similar type of non-NLP problems where one needs to perform tf-idf (or similar) on "document" bags of general items other than text.

### Describe your proposed solution

Decouple the `CountVectorizer` into the following 2 specific purpose & reusable components: `TextTokenizer` & `ItemCountVectorizer` such that when both are run sequentially in a pipeline they produce the same behavior.

### Describe alternatives you've considered, if relevant

In some but not all cases one could force this (vectorizing item frequency) behavior by carefully serializing the bag of items into a faked string document & configuring the `CountVectorizer` tokenizer parameters to split on specific delimiters. But this brittle serialization & string de-serialization becomes inefficient & a bit hacky.

### Additional context

_No response_

COMMENT:
There has been discussion to have text preprocessing be more decoupled: https://github.com/scikit-learn/scikit-learn/issues/14951

From a pure unix point of view, I see the value of decoupling `CountVectorizer` into two objects. If I were to design this from the start, I would want the tokenizer to be it's own object. On the other hand, I am very concerned with the maintenance cost of adding two new public estimators. We are already limited in supporting our existing estimators.

Overall I am +0 with introducing `TextTokenizer` and `ItemCountVectorizer`.

Labels: New Feature, module:preprocessing, Needs Decision - Include Feature
Pandas Output Proposal Outline

With `get_feature_names_out` complete, I am currently reworking the SLEP for pandas output. I am thinking of only covering transformers in the SLEP to reduce the scope. This issue covers the complete idea for pandas output that covers all methods that return arrays: `transform`, `predict`, etc.

## API Prototype

I put together a functional prototype of this API that you can explore in [this colab notebook](https://colab.research.google.com/github/thomasjpfan/pandas-prototype-demo/blob/main/index.ipynb). Here is a [rendered version of the demo](https://nbviewer.org/github/thomasjpfan/pandas-prototype-demo/blob/main/index.ipynb). The demo includes the following use cases:

- DataFrame output from a Single transformer
- Column Transformer with DataFrame output
- Feature selection based on column names with cross validation
- Using HistGradientBoosting to select categories based on dtype
- Text preprocessing with sparse data

## Proposal

**TLDR**: The proposal is to add a `set_output` method to configure the output container. When `set_output(transform="pandas")` the output of the estimator is a pandas dataframe. In https://github.com/scikit-learn/scikit-learn/pull/16772, I have shown that sparse data will have a performance regression. To work around this, I propose `set_output(transform="frame_or_sparse")`, which returns a DataFrame for dense data and a custom `SKCSRMatrix` for sparse data. `SKCSRMatrix` is a subclass of `csr_matrix` so it will work with previous code. 

See [the rendered notebook](https://nbviewer.org/github/thomasjpfan/pandas-prototype-demo/blob/main/index.ipynb) to see the API in various use cases.

## Future Extensions

These are items that is **not** in the SLEP

### Future: Predictions

```python
log_reg = LogisticRegression()
log_reg.set_output(
    predict_proba="pandas",
    predict="pandas",
    decision_function="pandas",
)
log_reg.fit(X_df, y)

# classes are the column names
X_pred = log_reg.predict_proba(X)

# categorical where classes are the categories
X_pred = log_reg.predict(X)

# binary case, series with name=classes_[1]
# multiclass case, dataframe with columns=classes_
X_pred = log_reg.decision_function(X)
```

### Future: Pipeline with prediction

```python
log_reg = make_pipeline(
    StandardScalar(),  # only uses `transform="pandas"`
    LogisticRegression(), # only uses `predict="pandas"`
).set_output(
    predict="pandas",
    transform="pandas",
)

log_reg.fit(X_df, y)

# DataFrame
y_pred = log_reg.predict(X_df)
```

CC @amueller @glemaitre

COMMENT:
@thomasjpfan What about also adding a global value that could be set instead of having to ensure one sets this on each pipeline/transformer/estimator? E.g. `sklearn.set_config(set_output='pandas')` for everything to be set to pandas, or `sklearn.set_config(set_output={'transform':'pandas'})` for just transformations to be set to output as pandas?

Labels: API
Pandas Output Proposal Outline

With `get_feature_names_out` complete, I am currently reworking the SLEP for pandas output. I am thinking of only covering transformers in the SLEP to reduce the scope. This issue covers the complete idea for pandas output that covers all methods that return arrays: `transform`, `predict`, etc.

## API Prototype

I put together a functional prototype of this API that you can explore in [this colab notebook](https://colab.research.google.com/github/thomasjpfan/pandas-prototype-demo/blob/main/index.ipynb). Here is a [rendered version of the demo](https://nbviewer.org/github/thomasjpfan/pandas-prototype-demo/blob/main/index.ipynb). The demo includes the following use cases:

- DataFrame output from a Single transformer
- Column Transformer with DataFrame output
- Feature selection based on column names with cross validation
- Using HistGradientBoosting to select categories based on dtype
- Text preprocessing with sparse data

## Proposal

**TLDR**: The proposal is to add a `set_output` method to configure the output container. When `set_output(transform="pandas")` the output of the estimator is a pandas dataframe. In https://github.com/scikit-learn/scikit-learn/pull/16772, I have shown that sparse data will have a performance regression. To work around this, I propose `set_output(transform="frame_or_sparse")`, which returns a DataFrame for dense data and a custom `SKCSRMatrix` for sparse data. `SKCSRMatrix` is a subclass of `csr_matrix` so it will work with previous code. 

See [the rendered notebook](https://nbviewer.org/github/thomasjpfan/pandas-prototype-demo/blob/main/index.ipynb) to see the API in various use cases.

## Future Extensions

These are items that is **not** in the SLEP

### Future: Predictions

```python
log_reg = LogisticRegression()
log_reg.set_output(
    predict_proba="pandas",
    predict="pandas",
    decision_function="pandas",
)
log_reg.fit(X_df, y)

# classes are the column names
X_pred = log_reg.predict_proba(X)

# categorical where classes are the categories
X_pred = log_reg.predict(X)

# binary case, series with name=classes_[1]
# multiclass case, dataframe with columns=classes_
X_pred = log_reg.decision_function(X)
```

### Future: Pipeline with prediction

```python
log_reg = make_pipeline(
    StandardScalar(),  # only uses `transform="pandas"`
    LogisticRegression(), # only uses `predict="pandas"`
).set_output(
    predict="pandas",
    transform="pandas",
)

log_reg.fit(X_df, y)

# DataFrame
y_pred = log_reg.predict(X_df)
```

CC @amueller @glemaitre

COMMENT:
A global configuration option would enable a better user experience and can work within the proposal. The biggest issue with global configuration is how it adds another layer of complexity for writing a third party transformer. We can likely mitigate the issue by providing enough utilities so that pandas output becomes easy to implement (that follows sklearn's global configuration).

Labels: API
completeness score (v-measure) for trivial clustering is non-zero

### Describe the bug

If I calculate the completeness score for a trivial clustering (as many clusters as datapoints) I would expect the completness score to be 0.0, however, I get non-zero values if I have more than one class (true labels).

### Steps/Code to Reproduce

```
from sklearn.metrics.cluster import homogeneity_score, completeness_score
completeness_score([0, 0, 1, 1], [0, 1, 2, 3])
```

### Expected Results

0.0

### Actual Results

0.49999999999999994

### Versions

```shell
System:
    python: 3.9.6 (default, Nov  2 2021, 12:33:45)  [Clang 12.0.5 (clang-1205.0.22.11)]
executable: /Users/tkb/automatic/device9/ftrack-research/analysis/xdevice-testset/venv/bin/python
   machine: macOS-11.6-x86_64-i386-64bit

Python dependencies:
          pip: 22.0.3
   setuptools: 56.0.0
      sklearn: 1.0.2
        numpy: 1.21.3
        scipy: 1.7.2
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
1/2 is the expected result, strictly applying the formula ``C = 1 - H(ypred | ytrue) / H(ypred)``.

> I would expect the completness score to be 0.0, however, I get non-zero values if I have more than one class (true labels).

If you take ``y_true = [0,1,2,3]`` you have more than one class and you expect the completeness score to be 1, so this intuition does not hold.

Labels: module:metrics
completeness score (v-measure) for trivial clustering is non-zero

### Describe the bug

If I calculate the completeness score for a trivial clustering (as many clusters as datapoints) I would expect the completness score to be 0.0, however, I get non-zero values if I have more than one class (true labels).

### Steps/Code to Reproduce

```
from sklearn.metrics.cluster import homogeneity_score, completeness_score
completeness_score([0, 0, 1, 1], [0, 1, 2, 3])
```

### Expected Results

0.0

### Actual Results

0.49999999999999994

### Versions

```shell
System:
    python: 3.9.6 (default, Nov  2 2021, 12:33:45)  [Clang 12.0.5 (clang-1205.0.22.11)]
executable: /Users/tkb/automatic/device9/ftrack-research/analysis/xdevice-testset/venv/bin/python
   machine: macOS-11.6-x86_64-i386-64bit

Python dependencies:
          pip: 22.0.3
   setuptools: 56.0.0
      sklearn: 1.0.2
        numpy: 1.21.3
        scipy: 1.7.2
       Cython: None
       pandas: 1.3.4
   matplotlib: 3.4.3
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

COMMENT:
Yes, you are right about that, I'll close the issue.

Labels: module:metrics
Compute centroids using median for metric = 'cosine' in NearestCentroid

### Describe the workflow you want to enable

In the current NearestCentroid code, we get centroids computed using median only for metric = 'manhattan'.
If enabling centroids computation using median for metric = 'cosine' will help in online prediction tasks.

### Describe your proposed solution

Add 'metric' = 'cosine' for computing centroids using median of the cluster. If approved, I can work on the issue.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Do you have a reference where the median is used for computing the centroids with `metric='cosine'`?

Labels: New Feature, module:neighbors
Compute centroids using median for metric = 'cosine' in NearestCentroid

### Describe the workflow you want to enable

In the current NearestCentroid code, we get centroids computed using median only for metric = 'manhattan'.
If enabling centroids computation using median for metric = 'cosine' will help in online prediction tasks.

### Describe your proposed solution

Add 'metric' = 'cosine' for computing centroids using median of the cluster. If approved, I can work on the issue.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
Based on the following [post](https://skeptric.com/projective-centroid/), it seems that taking the average of the vectors is completely fine. @thomasjpfan do you want to have another look given the discussion in the post?

Labels: New Feature, module:neighbors
Compute centroids using median for metric = 'cosine' in NearestCentroid

### Describe the workflow you want to enable

In the current NearestCentroid code, we get centroids computed using median only for metric = 'manhattan'.
If enabling centroids computation using median for metric = 'cosine' will help in online prediction tasks.

### Describe your proposed solution

Add 'metric' = 'cosine' for computing centroids using median of the cluster. If approved, I can work on the issue.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

_No response_

COMMENT:
From the post in https://github.com/scikit-learn/scikit-learn/issues/22985#issuecomment-1382755720 and the reading of [this paper](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.31.7900&rep=rep1&type=pdf), I think the average makes sense cosine.

Labels: New Feature, module:neighbors