Talip7/scikit-learn-issues-multilabel huggingface dataset:
Support other dataframes like polars and pyarrow not just pandas

### Describe the workflow you want to enable

Currently, scikit-learn nowhere claims to support [pyarrow](https://arrow.apache.org/docs/python/) or [polars](https://www.pola.rs/). And indeed,
```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler, KBinsDiscretizer
from sklearn.compose import ColumnTransformer

X, y = load_iris(as_frame=True, return_X_y=True)
sepal_cols = ["sepal length (cm)", "sepal width (cm)"]
petal_cols = ["petal length (cm)", "petal width (cm)"]

preprocessor = ColumnTransformer(
    [
        ("scaler", StandardScaler(), sepal_cols),
        ("kbin", KBinsDiscretizer(encode="ordinal"), petal_cols),
    ],
    verbose_feature_names_out=False,
)

import polars as pl  # or import pyarrow as pa
X_pl = pl.from_pandas(X)  # or X_pa = pa.table(X)

preprocessor.fit_transform(X_pl)
# preprocessor.set_output(transform="pandas").fit_transform(X_pl)
```

errors with
```
AttributeError: 'numpy.ndarray' object has no attribute 'columns'

During handling of the above exception, another exception occurred:

ValueError: Specifying the columns using strings is only supported for pandas DataFrames
```

### Describe your proposed solution

scikit-learn should support those dataframes, maybe via the [python dataframe interchange protocol](https://data-apis.org/dataframe-protocol/latest/index.html).

In that regard, a new option like `set_output(transform="dataframe")` would be nice.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Some related discussion came up in #25813.

COMMENT:
I think most of the work is done for polars. But the `ColumnTransformer` (and maybe `OrdinalEncoder` and `LabelEncoder`) might still need work to support pyarrow properly.

We also would need `.set_output(transform="pyarrow")` in the transformer mixins.

Labels: New Feature, RFC
Support other dataframes like polars and pyarrow not just pandas

### Describe the workflow you want to enable

Currently, scikit-learn nowhere claims to support [pyarrow](https://arrow.apache.org/docs/python/) or [polars](https://www.pola.rs/). And indeed,
```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler, KBinsDiscretizer
from sklearn.compose import ColumnTransformer

X, y = load_iris(as_frame=True, return_X_y=True)
sepal_cols = ["sepal length (cm)", "sepal width (cm)"]
petal_cols = ["petal length (cm)", "petal width (cm)"]

preprocessor = ColumnTransformer(
    [
        ("scaler", StandardScaler(), sepal_cols),
        ("kbin", KBinsDiscretizer(encode="ordinal"), petal_cols),
    ],
    verbose_feature_names_out=False,
)

import polars as pl  # or import pyarrow as pa
X_pl = pl.from_pandas(X)  # or X_pa = pa.table(X)

preprocessor.fit_transform(X_pl)
# preprocessor.set_output(transform="pandas").fit_transform(X_pl)
```

errors with
```
AttributeError: 'numpy.ndarray' object has no attribute 'columns'

During handling of the above exception, another exception occurred:

ValueError: Specifying the columns using strings is only supported for pandas DataFrames
```

### Describe your proposed solution

scikit-learn should support those dataframes, maybe via the [python dataframe interchange protocol](https://data-apis.org/dataframe-protocol/latest/index.html).

In that regard, a new option like `set_output(transform="dataframe")` would be nice.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Some related discussion came up in #25813.

COMMENT:
Maybe we could have one such issue per-dataframe libraries we want to support, either for input only or input/output (e.g. at least pyarrow I think).

Labels: New Feature, RFC
Support other dataframes like polars and pyarrow not just pandas

### Describe the workflow you want to enable

Currently, scikit-learn nowhere claims to support [pyarrow](https://arrow.apache.org/docs/python/) or [polars](https://www.pola.rs/). And indeed,
```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler, KBinsDiscretizer
from sklearn.compose import ColumnTransformer

X, y = load_iris(as_frame=True, return_X_y=True)
sepal_cols = ["sepal length (cm)", "sepal width (cm)"]
petal_cols = ["petal length (cm)", "petal width (cm)"]

preprocessor = ColumnTransformer(
    [
        ("scaler", StandardScaler(), sepal_cols),
        ("kbin", KBinsDiscretizer(encode="ordinal"), petal_cols),
    ],
    verbose_feature_names_out=False,
)

import polars as pl  # or import pyarrow as pa
X_pl = pl.from_pandas(X)  # or X_pa = pa.table(X)

preprocessor.fit_transform(X_pl)
# preprocessor.set_output(transform="pandas").fit_transform(X_pl)
```

errors with
```
AttributeError: 'numpy.ndarray' object has no attribute 'columns'

During handling of the above exception, another exception occurred:

ValueError: Specifying the columns using strings is only supported for pandas DataFrames
```

### Describe your proposed solution

scikit-learn should support those dataframes, maybe via the [python dataframe interchange protocol](https://data-apis.org/dataframe-protocol/latest/index.html).

In that regard, a new option like `set_output(transform="dataframe")` would be nice.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Some related discussion came up in #25813.

COMMENT:
FYI, the above code snippet now works, I guess since #26464. So I'm inclined to close.

Labels: New Feature, RFC
Support other dataframes like polars and pyarrow not just pandas

### Describe the workflow you want to enable

Currently, scikit-learn nowhere claims to support [pyarrow](https://arrow.apache.org/docs/python/) or [polars](https://www.pola.rs/). And indeed,
```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler, KBinsDiscretizer
from sklearn.compose import ColumnTransformer

X, y = load_iris(as_frame=True, return_X_y=True)
sepal_cols = ["sepal length (cm)", "sepal width (cm)"]
petal_cols = ["petal length (cm)", "petal width (cm)"]

preprocessor = ColumnTransformer(
    [
        ("scaler", StandardScaler(), sepal_cols),
        ("kbin", KBinsDiscretizer(encode="ordinal"), petal_cols),
    ],
    verbose_feature_names_out=False,
)

import polars as pl  # or import pyarrow as pa
X_pl = pl.from_pandas(X)  # or X_pa = pa.table(X)

preprocessor.fit_transform(X_pl)
# preprocessor.set_output(transform="pandas").fit_transform(X_pl)
```

errors with
```
AttributeError: 'numpy.ndarray' object has no attribute 'columns'

During handling of the above exception, another exception occurred:

ValueError: Specifying the columns using strings is only supported for pandas DataFrames
```

### Describe your proposed solution

scikit-learn should support those dataframes, maybe via the [python dataframe interchange protocol](https://data-apis.org/dataframe-protocol/latest/index.html).

In that regard, a new option like `set_output(transform="dataframe")` would be nice.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Some related discussion came up in #25813.

COMMENT:
@scikit-learn/core-devs Should we make pyarrow tables work within scikit-learn (without requiring it as dependency, just like pandas and polars)?

Labels: New Feature, RFC
Support other dataframes like polars and pyarrow not just pandas

### Describe the workflow you want to enable

Currently, scikit-learn nowhere claims to support [pyarrow](https://arrow.apache.org/docs/python/) or [polars](https://www.pola.rs/). And indeed,
```python
import numpy as np
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler, KBinsDiscretizer
from sklearn.compose import ColumnTransformer

X, y = load_iris(as_frame=True, return_X_y=True)
sepal_cols = ["sepal length (cm)", "sepal width (cm)"]
petal_cols = ["petal length (cm)", "petal width (cm)"]

preprocessor = ColumnTransformer(
    [
        ("scaler", StandardScaler(), sepal_cols),
        ("kbin", KBinsDiscretizer(encode="ordinal"), petal_cols),
    ],
    verbose_feature_names_out=False,
)

import polars as pl  # or import pyarrow as pa
X_pl = pl.from_pandas(X)  # or X_pa = pa.table(X)

preprocessor.fit_transform(X_pl)
# preprocessor.set_output(transform="pandas").fit_transform(X_pl)
```

errors with
```
AttributeError: 'numpy.ndarray' object has no attribute 'columns'

During handling of the above exception, another exception occurred:

ValueError: Specifying the columns using strings is only supported for pandas DataFrames
```

### Describe your proposed solution

scikit-learn should support those dataframes, maybe via the [python dataframe interchange protocol](https://data-apis.org/dataframe-protocol/latest/index.html).

In that regard, a new option like `set_output(transform="dataframe")` would be nice.

### Describe alternatives you've considered, if relevant

_No response_

### Additional context

Some related discussion came up in #25813.

COMMENT:
Yes that would be great imo. I need to look into it more, but are there any major API incompatibilities?

Labels: New Feature, RFC
FeatureUnion: Add verbose_feature_names_out parameter

### Describe the workflow you want to enable

`ColumnTransformer` has the option to specify whether or not to "prefix all feature names with the name of the transformer that generated that feature" using the `verbose_feature_names_out` parameter.

`FeatureUnion` does not have this option. As a user I would like to control whether or not I want to have the prefix added. Right now, the prefix is added without any option to turn that functionality off.

### Describe your proposed solution

I propose to add the same parameter that exists in `ColumnTransformer` to `FeatureUnion` so that the user can decide whether or not they would like to have the prefix or not.

### Describe alternatives you've considered, if relevant

**The user can remove the prefox manually after FeatureUnion has transformed the data**
Requires to identify the columns in question and remove the prefix. Removing the prefix via a simple `str.split("__")[1]` could cause problems if the feature names already include `__`. If that is the case a more complicated way to remove the prefix is required by the user.

**The user can write their own implementation of FeatureUnion that changes the method in question**
```
from sklearn.pipeline import FeatureUnion

class NewFeatureUnion(FeatureUnion):
    def get_feature_names_out(self, input_features=None):
        """
        Overwrites the original method to not modify the features names
        """
        feature_names = []
        for name, trans, _ in self._iter():
            if not hasattr(trans, "get_feature_names_out"):
                raise AttributeError(
                    "Transformer %s (type %s) does not provide get_feature_names_out."
                    % (str(name), type(trans).__name__)
                )
            feature_names.extend(
                [f for f in trans.get_feature_names_out(input_features)]
            )
        return np.asarray(feature_names, dtype=object)
```
Adds more code to the user's codebase that needs to be maintained. Could also cause confusion because there is no obvious class name that explains that the new implementation does, so another developer would have to find the code snippet to figure out what the new implementation does. 

### Additional context

There might be a good reason why we don't want to have the same functionality in `FeatureUnion` that is available for `ColumnTransformer`. This is my first issue so I might be missing something.

A one-to-one implementation of the parameter might break things that are dependent on `FeatureUnion`. I am not that familiar with the rest of the codebase.

I think I would be able to create a merge request for it if this parameter request is approved.

COMMENT:
Is anyone else currently working on this issue? If not, would it be okay for me to take it on?

Labels: New Feature, module:pipeline
FeatureUnion: Add verbose_feature_names_out parameter

### Describe the workflow you want to enable

`ColumnTransformer` has the option to specify whether or not to "prefix all feature names with the name of the transformer that generated that feature" using the `verbose_feature_names_out` parameter.

`FeatureUnion` does not have this option. As a user I would like to control whether or not I want to have the prefix added. Right now, the prefix is added without any option to turn that functionality off.

### Describe your proposed solution

I propose to add the same parameter that exists in `ColumnTransformer` to `FeatureUnion` so that the user can decide whether or not they would like to have the prefix or not.

### Describe alternatives you've considered, if relevant

**The user can remove the prefox manually after FeatureUnion has transformed the data**
Requires to identify the columns in question and remove the prefix. Removing the prefix via a simple `str.split("__")[1]` could cause problems if the feature names already include `__`. If that is the case a more complicated way to remove the prefix is required by the user.

**The user can write their own implementation of FeatureUnion that changes the method in question**
```
from sklearn.pipeline import FeatureUnion

class NewFeatureUnion(FeatureUnion):
    def get_feature_names_out(self, input_features=None):
        """
        Overwrites the original method to not modify the features names
        """
        feature_names = []
        for name, trans, _ in self._iter():
            if not hasattr(trans, "get_feature_names_out"):
                raise AttributeError(
                    "Transformer %s (type %s) does not provide get_feature_names_out."
                    % (str(name), type(trans).__name__)
                )
            feature_names.extend(
                [f for f in trans.get_feature_names_out(input_features)]
            )
        return np.asarray(feature_names, dtype=object)
```
Adds more code to the user's codebase that needs to be maintained. Could also cause confusion because there is no obvious class name that explains that the new implementation does, so another developer would have to find the code snippet to figure out what the new implementation does. 

### Additional context

There might be a good reason why we don't want to have the same functionality in `FeatureUnion` that is available for `ColumnTransformer`. This is my first issue so I might be missing something.

A one-to-one implementation of the parameter might break things that are dependent on `FeatureUnion`. I am not that familiar with the rest of the codebase.

I think I would be able to create a merge request for it if this parameter request is approved.

COMMENT:
@jiawei-zhang-a Feel free to take it. I was gonna work on it probably next week but you can take it instead if you want.

Labels: New Feature, module:pipeline
FeatureUnion: Add verbose_feature_names_out parameter

### Describe the workflow you want to enable

`ColumnTransformer` has the option to specify whether or not to "prefix all feature names with the name of the transformer that generated that feature" using the `verbose_feature_names_out` parameter.

`FeatureUnion` does not have this option. As a user I would like to control whether or not I want to have the prefix added. Right now, the prefix is added without any option to turn that functionality off.

### Describe your proposed solution

I propose to add the same parameter that exists in `ColumnTransformer` to `FeatureUnion` so that the user can decide whether or not they would like to have the prefix or not.

### Describe alternatives you've considered, if relevant

**The user can remove the prefox manually after FeatureUnion has transformed the data**
Requires to identify the columns in question and remove the prefix. Removing the prefix via a simple `str.split("__")[1]` could cause problems if the feature names already include `__`. If that is the case a more complicated way to remove the prefix is required by the user.

**The user can write their own implementation of FeatureUnion that changes the method in question**
```
from sklearn.pipeline import FeatureUnion

class NewFeatureUnion(FeatureUnion):
    def get_feature_names_out(self, input_features=None):
        """
        Overwrites the original method to not modify the features names
        """
        feature_names = []
        for name, trans, _ in self._iter():
            if not hasattr(trans, "get_feature_names_out"):
                raise AttributeError(
                    "Transformer %s (type %s) does not provide get_feature_names_out."
                    % (str(name), type(trans).__name__)
                )
            feature_names.extend(
                [f for f in trans.get_feature_names_out(input_features)]
            )
        return np.asarray(feature_names, dtype=object)
```
Adds more code to the user's codebase that needs to be maintained. Could also cause confusion because there is no obvious class name that explains that the new implementation does, so another developer would have to find the code snippet to figure out what the new implementation does. 

### Additional context

There might be a good reason why we don't want to have the same functionality in `FeatureUnion` that is available for `ColumnTransformer`. This is my first issue so I might be missing something.

A one-to-one implementation of the parameter might break things that are dependent on `FeatureUnion`. I am not that familiar with the rest of the codebase.

I think I would be able to create a merge request for it if this parameter request is approved.

COMMENT:
@sebmoh Thanks! I will implement it within this week

Labels: New Feature, module:pipeline
FeatureUnion: Add verbose_feature_names_out parameter

### Describe the workflow you want to enable

`ColumnTransformer` has the option to specify whether or not to "prefix all feature names with the name of the transformer that generated that feature" using the `verbose_feature_names_out` parameter.

`FeatureUnion` does not have this option. As a user I would like to control whether or not I want to have the prefix added. Right now, the prefix is added without any option to turn that functionality off.

### Describe your proposed solution

I propose to add the same parameter that exists in `ColumnTransformer` to `FeatureUnion` so that the user can decide whether or not they would like to have the prefix or not.

### Describe alternatives you've considered, if relevant

**The user can remove the prefox manually after FeatureUnion has transformed the data**
Requires to identify the columns in question and remove the prefix. Removing the prefix via a simple `str.split("__")[1]` could cause problems if the feature names already include `__`. If that is the case a more complicated way to remove the prefix is required by the user.

**The user can write their own implementation of FeatureUnion that changes the method in question**
```
from sklearn.pipeline import FeatureUnion

class NewFeatureUnion(FeatureUnion):
    def get_feature_names_out(self, input_features=None):
        """
        Overwrites the original method to not modify the features names
        """
        feature_names = []
        for name, trans, _ in self._iter():
            if not hasattr(trans, "get_feature_names_out"):
                raise AttributeError(
                    "Transformer %s (type %s) does not provide get_feature_names_out."
                    % (str(name), type(trans).__name__)
                )
            feature_names.extend(
                [f for f in trans.get_feature_names_out(input_features)]
            )
        return np.asarray(feature_names, dtype=object)
```
Adds more code to the user's codebase that needs to be maintained. Could also cause confusion because there is no obvious class name that explains that the new implementation does, so another developer would have to find the code snippet to figure out what the new implementation does. 

### Additional context

There might be a good reason why we don't want to have the same functionality in `FeatureUnion` that is available for `ColumnTransformer`. This is my first issue so I might be missing something.

A one-to-one implementation of the parameter might break things that are dependent on `FeatureUnion`. I am not that familiar with the rest of the codebase.

I think I would be able to create a merge request for it if this parameter request is approved.

COMMENT:
/take

Labels: New Feature, module:pipeline
FeatureUnion: Add verbose_feature_names_out parameter

### Describe the workflow you want to enable

`ColumnTransformer` has the option to specify whether or not to "prefix all feature names with the name of the transformer that generated that feature" using the `verbose_feature_names_out` parameter.

`FeatureUnion` does not have this option. As a user I would like to control whether or not I want to have the prefix added. Right now, the prefix is added without any option to turn that functionality off.

### Describe your proposed solution

I propose to add the same parameter that exists in `ColumnTransformer` to `FeatureUnion` so that the user can decide whether or not they would like to have the prefix or not.

### Describe alternatives you've considered, if relevant

**The user can remove the prefox manually after FeatureUnion has transformed the data**
Requires to identify the columns in question and remove the prefix. Removing the prefix via a simple `str.split("__")[1]` could cause problems if the feature names already include `__`. If that is the case a more complicated way to remove the prefix is required by the user.

**The user can write their own implementation of FeatureUnion that changes the method in question**
```
from sklearn.pipeline import FeatureUnion

class NewFeatureUnion(FeatureUnion):
    def get_feature_names_out(self, input_features=None):
        """
        Overwrites the original method to not modify the features names
        """
        feature_names = []
        for name, trans, _ in self._iter():
            if not hasattr(trans, "get_feature_names_out"):
                raise AttributeError(
                    "Transformer %s (type %s) does not provide get_feature_names_out."
                    % (str(name), type(trans).__name__)
                )
            feature_names.extend(
                [f for f in trans.get_feature_names_out(input_features)]
            )
        return np.asarray(feature_names, dtype=object)
```
Adds more code to the user's codebase that needs to be maintained. Could also cause confusion because there is no obvious class name that explains that the new implementation does, so another developer would have to find the code snippet to figure out what the new implementation does. 

### Additional context

There might be a good reason why we don't want to have the same functionality in `FeatureUnion` that is available for `ColumnTransformer`. This is my first issue so I might be missing something.

A one-to-one implementation of the parameter might break things that are dependent on `FeatureUnion`. I am not that familiar with the rest of the codebase.

I think I would be able to create a merge request for it if this parameter request is approved.

COMMENT:
@thomasjpfan Hi Mr. Fan, I have made a PR for this issue, would mind taking a look at it.

Labels: New Feature, module:pipeline