pipeline using FunctionTransformer with feature_names_out=... fails when applied to dataframe argument

### Describe the bug

(based on this stackoverflow question: https://stackoverflow.com/questions/77379286/sklearn-pipeline-get-feature-names-out-fails-unless-dataframe-has-matching-ren/77396145#77396145)

I have a simple sklearn (1.3.1) pipeline where the first step is renaming its input features, so I implemented feature_names_out as below.  If I fit the pipeline on a numpy array using `p.fit_transform(df.values)`, everything is fine and it reports output feature names as `x0__log`, `x1__log`.  However if I fit on the dataframe directly with `p.fit_transform(df)`, then `p.get_feature_names_out()` gives a stack trace ending with `ValueError: input_features is not equal to feature_names_in_`.

(from the answer) The problem is that FunctionTransformer by default applies func directly to the input without converting the input first; so `p[0].transform(df)` produces a dataframe with columns still `[a, b]`, and `p[1]` gets fitted on that frame, setting its `feature_names_in_` attribute also to `[a, b]`, which contradicts what comes out of `get_feature_names_out` (having been passed through your `with_suffix`).

The suggested workaround is to set `validate=True` in your FunctionTransformer: this will convert the input to a numpy array, so that the subsequent step won't be fitted on a dataframe, so won't have a `feature_names_in_` set.  (Or make sure a dataframe argument has its columns renamed to make `feature_names_out` as I ended up doing.)





### Steps/Code to Reproduce

```python
from typing import List
import numpy as np
import pandas as pd
from sklearn.preprocessing import FunctionTransformer, StandardScaler
from sklearn.pipeline import make_pipeline

def with_suffix(_, names: List[str]):
    return [name + '__log' for name in names]

p = make_pipeline(
    FunctionTransformer(np.log1p, feature_names_out=with_suffix),
    StandardScaler()
)

df = pd.DataFrame([[1,2], [3,4], [5,6]], columns=['a', 'b'])

p.fit_transform(df)              # <= works if we pass df.values instead
p.get_feature_names_out()     # <= fails when pipeline is applied to dataframe
```

### Expected Results

No error should be shown.

### Actual Results

```pytb
{
	"name": "ValueError",
	"message": "input_features is not equal to feature_names_in_",
	"stack": "---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/Users/psurry/Hopper/fintech-ml-core/feature-tables/scratch/binning.ipynb Cell 97 line 1
     <a href='vscode-notebook-cell:/Users/psurry/Hopper/fintech-ml-core/feature-tables/scratch/binning.ipynb#Y210sZmlsZQ%3D%3D?line=14'>15</a> df = pd.DataFrame([[1,2], [3,4], [5,6]], columns=['a', 'b'])
     <a href='vscode-notebook-cell:/Users/psurry/Hopper/fintech-ml-core/feature-tables/scratch/binning.ipynb#Y210sZmlsZQ%3D%3D?line=16'>17</a> p.fit_transform(df)              # <= works if we pass df.values instead
---> <a href='vscode-notebook-cell:/Users/psurry/Hopper/fintech-ml-core/feature-tables/scratch/binning.ipynb#Y210sZmlsZQ%3D%3D?line=17'>18</a> p.get_feature_names_out()     # <= fails when pipeline is applied to dataframe

File ~/miniconda3/envs/feature-tables/lib/python3.11/site-packages/sklearn/pipeline.py:820, in Pipeline.get_feature_names_out(self, input_features)
    814     if not hasattr(transform, \"get_feature_names_out\"):
    815         raise AttributeError(
    816             \"Estimator {} does not provide get_feature_names_out. \"
    817             \"Did you mean to call pipeline[:-1].get_feature_names_out\"
    818             \"()?\".format(name)
    819         )
--> 820     feature_names_out = transform.get_feature_names_out(feature_names_out)
    821 return feature_names_out

File ~/miniconda3/envs/feature-tables/lib/python3.11/site-packages/sklearn/base.py:949, in OneToOneFeatureMixin.get_feature_names_out(self, input_features)
    929 \"\"\"Get output feature names for transformation.
    930 
    931 Parameters
   (...)
    946     Same as input features.
    947 \"\"\"
    948 check_is_fitted(self, \"n_features_in_\")
--> 949 return _check_feature_names_in(self, input_features)

File ~/miniconda3/envs/feature-tables/lib/python3.11/site-packages/sklearn/utils/validation.py:2071, in _check_feature_names_in(estimator, input_features, generate_names)
   2067 input_features = np.asarray(input_features, dtype=object)
   2068 if feature_names_in_ is not None and not np.array_equal(
   2069     feature_names_in_, input_features
   2070 ):
-> 2071     raise ValueError(\"input_features is not equal to feature_names_in_\")
   2073 if n_features_in_ is not None and len(input_features) != n_features_in_:
   2074     raise ValueError(
   2075         \"input_features should have length equal to number of \"
   2076         f\"features ({n_features_in_}), got {len(input_features)}\"
   2077     )

ValueError: input_features is not equal to feature_names_in_"
}
```

### Versions

```shell
System:
    python: 3.11.6 | packaged by conda-forge | (main, Oct  3 2023, 10:40:37) [Clang 15.0.7 ]
executable: /Users/psurry/miniconda3/envs/feature-tables/bin/python
   machine: macOS-14.1-x86_64-i386-64bit

Python dependencies:
      sklearn: 1.3.1
          pip: 23.3
   setuptools: 68.2.2
        numpy: 1.26.0
        scipy: 1.11.3
       Cython: None
       pandas: 2.0.3
   matplotlib: 3.8.0
       joblib: 1.3.2
threadpoolctl: 3.2.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Users/psurry/miniconda3/envs/feature-tables/lib/libopenblasp-r0.3.24.dylib
        version: 0.3.24
threading_layer: openmp
   architecture: Nehalem

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/psurry/miniconda3/envs/feature-tables/lib/libomp.dylib
        version: None

       user_api: openmp
   internal_api: openmp
    num_threads: 8
         prefix: libomp
       filepath: /Users/psurry/miniconda3/envs/feature-tables/lib/python3.11/site-packages/sklearn/.dylibs/libomp.dylib
        version: None

       user_api: blas
   internal_api: openblas
    num_threads: 8
         prefix: libopenblas
       filepath: /Users/psurry/miniconda3/envs/feature-tables/lib/python3.11/site-packages/scipy/.dylibs/libopenblas.0.dylib
        version: 0.3.21.dev
threading_layer: pthreads
   architecture: Nehalem
```

COMMENT:
> ok - it would be great if that was documented somewhere with the feature_names_out discussion

Actually the UX is bad here.

Here, we benefit from the array function dispatcher (https://numpy.org/neps/nep-0018-array-function-protocol.html) from NumPy that returns a DataFrame since we input a DataFrame. So `get_feature_names_out` works as expected but the output that is configure to be default, return what NumPy is returning in this case and thus a DataFrame but with the original column name.

`validate=True` enforces a conversion from DataFrame to a NumPy array internally before calling the provided function. `set_output` will rewrap with the expected name.

So here we get a nasty bug but I am not sure what is the best strategy: we could make sure that when we have `default`, we force to return a NumPy array and hide the NumPy array dispatching mechanism.

Labels: Bug