`permutation_importance` errors with `polars` dataframe and `ColumnTransformer`

### Describe the bug

Having polars dataframe with `ColumnTransformer` lets `permutation_importance` crash.

Maybe related to the warnings reported in this issue https://github.com/scikit-learn/scikit-learn/issues/28488. But here, `permuation_importance` errors.

### Steps/Code to Reproduce

A MWE
```py
import polars as pl

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.inspection import permutation_importance

X, y = make_classification(n_samples=100, n_features=5, random_state=42)
feature_names = [f'feature_{i}' for i in range(X.shape[1])]

df = pl.DataFrame({name: X[:, i] for i, name in enumerate(feature_names)})
df = df.with_columns(pl.Series("target", y))

X_train, X_test, y_train, y_test = train_test_split(df.select(feature_names), df['target'], test_size=0.2, random_state=42)

preprocessor = ColumnTransformer(
    transformers=[
        ('scaler', StandardScaler(), feature_names)  # using feature names here
    ]
)
preprocessor.set_output(transform="polars")

pipeline = Pipeline(steps=[
    ('preprocessor', preprocessor),
    ('classifier', LogisticRegression())
])

model = pipeline.fit(X_train, y_train)

permutation_importance(model, X_test, y_test)
```

### Expected Results

No error is shown.

### Actual Results

```py
.../python3.12/site-packages/sklearn/utils/_indexing.py:259, in _safe_indexing(X, indices, axis)
    248     raise ValueError(
    249         "'X' should be a 2D NumPy array, 2D sparse matrix or "
    250         "dataframe when indexing the columns (i.e. 'axis=1'). "
    251         "Got {} instead with {} dimension(s).".format(type(X), len(X.shape))
    252     )
    254 if (
    255     axis == 1
    256     and indices_dtype == "str"
    257     and not (_is_pandas_df(X) or _use_interchange_protocol(X))
    258 ):
--> [259](https://vscode-remote+domino-002eapps-002elan-002dprd-002d001-002ek8s-002elan-002ehuk-002dcoburg-002ede.vscode-resource.vscode-cdn.net/repos/da-hf5-regress/.pixi/envs/default/lib/python3.12/site-packages/sklearn/utils/_indexing.py:259)     raise ValueError(
    260         "Specifying the columns using strings is only supported for dataframes."
    261     )
    263 if hasattr(X, "iloc"):
    264     # TODO: we should probably use _is_pandas_df_or_series(X) instead but this
    265     # would require updating some tests such as test_train_test_split_mock_pandas.
    266     return _pandas_indexing(X, indices, indices_dtype, axis=axis)

ValueError: Specifying the columns using strings is only supported for dataframes.
```

### Versions

```shell
System:
    python: 3.12.10 | packaged by conda-forge | (main, Apr 10 2025, 22:21:13) [GCC 13.3.0]
executable: .../default/bin/python
   machine: Linux-4.18.0-553.66.1.el8_10.x86_64-x86_64-with-glibc2.39

Python dependencies:
      sklearn: 1.6.1
          pip: 25.1.1
   setuptools: 80.1.0
        numpy: 2.2.5
        scipy: 1.15.2
       Cython: None
       pandas: 2.2.3
   matplotlib: 3.10.1
       joblib: 1.5.0
threadpoolctl: 3.6.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 40
         prefix: libopenblas
       filepath: .../default/lib/libopenblasp-r0.3.29.so
        version: 0.3.29
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 40
         prefix: libgomp
       filepath: .../lib/libgomp.so.1.0.0
        version: None
```

COMMENT:
We don't really completely support `polars`, since there's a lot of `pandas` specific code in the codebase (assuming `X.copy()` exists is also `pandas` specific.

This particular issue comes from this code:

https://github.com/scikit-learn/scikit-learn/blob/f7e7ce02dc24708c36fb5dfc90a8ea3432dddb7c/sklearn/inspection/_permutation_importance.py#L269-L270

and fixing that line results in a different issue since we call `copy` on the input:

https://github.com/scikit-learn/scikit-learn/blob/f7e7ce02dc24708c36fb5dfc90a8ea3432dddb7c/sklearn/inspection/_permutation_importance.py#L63

While checking this, I found a bunch of `pandas` specific code in `check_array` as well as in `ColumnTransformer`.

I tried to see if there's a quick fix, but the fix becomes dirty real fast, and I rather use `narwhals` instead.

Do you think it's time for embracing the change @scikit-learn/core-devs ?

Labels: Bug, module:inspection