`permutation_importance` errors with `polars` dataframe and `ColumnTransformer`

### Describe the bug

Having polars dataframe with `ColumnTransformer` lets `permutation_importance` crash.

Maybe related to the warnings reported in this issue https://github.com/scikit-learn/scikit-learn/issues/28488. But here, `permuation_importance` errors.

### Steps/Code to Reproduce

A MWE
```py
import polars as pl

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.inspection import permutation_importance

X, y = make_classification(n_samples=100, n_features=5, random_state=42)
feature_names = [f'feature_{i}' for i in range(X.shape[1])]

df = pl.DataFrame({name: X[:, i] for i, name in enumerate(feature_names)})
df = df.with_columns(pl.Series("target", y))

X_train, X_test, y_train, y_test = train_test_split(df.select(feature_names), df['target'], test_size=0.2, random_state=42)

preprocessor = ColumnTransformer(
    transformers=[
        ('scaler', StandardScaler(), feature_names)  # using feature names here
    ]
)
preprocessor.set_output(transform="polars")

pipeline = Pipeline(steps=[
    ('preprocessor', preprocessor),
    ('classifier', LogisticRegression())
])

model = pipeline.fit(X_train, y_train)

permutation_importance(model, X_test, y_test)
```

### Expected Results

No error is shown.

### Actual Results

```py
.../python3.12/site-packages/sklearn/utils/_indexing.py:259, in _safe_indexing(X, indices, axis)
    248     raise ValueError(
    249         "'X' should be a 2D NumPy array, 2D sparse matrix or "
    250         "dataframe when indexing the columns (i.e. 'axis=1'). "
    251         "Got {} instead with {} dimension(s).".format(type(X), len(X.shape))
    252     )
    254 if (
    255     axis == 1
    256     and indices_dtype == "str"
    257     and not (_is_pandas_df(X) or _use_interchange_protocol(X))
    258 ):
--> [259](https://vscode-remote+domino-002eapps-002elan-002dprd-002d001-002ek8s-002elan-002ehuk-002dcoburg-002ede.vscode-resource.vscode-cdn.net/repos/da-hf5-regress/.pixi/envs/default/lib/python3.12/site-packages/sklearn/utils/_indexing.py:259)     raise ValueError(
    260         "Specifying the columns using strings is only supported for dataframes."
    261     )
    263 if hasattr(X, "iloc"):
    264     # TODO: we should probably use _is_pandas_df_or_series(X) instead but this
    265     # would require updating some tests such as test_train_test_split_mock_pandas.
    266     return _pandas_indexing(X, indices, indices_dtype, axis=axis)

ValueError: Specifying the columns using strings is only supported for dataframes.
```

### Versions

```shell
System:
    python: 3.12.10 | packaged by conda-forge | (main, Apr 10 2025, 22:21:13) [GCC 13.3.0]
executable: .../default/bin/python
   machine: Linux-4.18.0-553.66.1.el8_10.x86_64-x86_64-with-glibc2.39

Python dependencies:
      sklearn: 1.6.1
          pip: 25.1.1
   setuptools: 80.1.0
        numpy: 2.2.5
        scipy: 1.15.2
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.1
       joblib: 1.5.0
threadpoolctl: 3.6.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 40
         prefix: libopenblas
       filepath: .../default/lib/libopenblasp-r0.3.29.so
        version: 0.3.29
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 40
         prefix: libgomp
       filepath: .../lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
Hi, I did some investigation on this issue. Here are my findings and the experiments I tried:

---

###  Reproducing the problem

* When using `permutation_importance` with a **Polars DataFrame**, I get

```
ValueError: Specifying the columns using strings is only supported for dataframes.
```

* The error comes from `_safe_indexing`, specifically this guard:

```python
if (
    axis == 1
    and indices_dtype == "str"
    and not (_is_pandas_df(X) or _use_interchange_protocol(X))
):
    raise ValueError(
        "Specifying the columns using strings is only supported for dataframes."
    )
```

---

###  Debugging steps

* To better understand what was happening, I added a `print(type(X))` just before the `raise ValueError(...)`.

  * In some runs I saw: `<class 'polars.dataframe.frame.DataFrame'>`
  * In other runs (through `permutation_importance`) I saw: `<class 'numpy.ndarray'>`

So sometimes the input is still a Polars DataFrame, but sometimes it has already been converted to numpy by the estimator (e.g. `LogisticRegression`).

* I also tested `_safe_indexing` directly:

```python
import polars as pl
from sklearn.utils._indexing import _safe_indexing

X = pl.DataFrame({"a": [1, 2, 3], "b": [4, 5, 6]})
out = _safe_indexing(X, ["a"], axis=1)
print(type(out))  # <class 'polars.dataframe.frame.DataFrame'>
```

This worked fine after adding a Polars check.

---

### Proposed fix

Update the guard in `_safe_indexing` to also accept Polars:

```python
if (
    axis == 1
    and indices_dtype == "str"
    and not (_is_pandas_df(X) or _is_polars_df_or_series(X) or _use_interchange_protocol(X))
):
    raise ValueError(
        "Specifying the columns using strings is only supported for dataframes."
    )
```

This prevents the unnecessary ValueError and allows `_polars_indexing` to be used.

---

###  Limitations

* This patch fixes the bug (unnecessary ValueError) when `X` is a Polars DataFrame.
* However, many estimators (e.g. `LogisticRegression`) automatically convert Polars → numpy internally, so in those cases `permutation_importance` still sees a numpy array.
* Therefore this is **not full Polars support**, but at least removes a blocker.
* The long-term solution should be to rely on the **DataFrame Interchange Protocol** for proper integration.

---

###  Next steps

I can open a small PR with this fix and a regression test, if this approach looks acceptable.

Labels: Bug, module:inspection