Performance Degradation in FeatureUnion with String Columns when concatenate the outputs of the transformers

### Describe the bug

I am experiencing significant performance degradation when using FeatureUnion in a Pipeline with DataFrames that include string columns set to be concatenated in the passthrough, the execution time is notably slower.

### Steps/Code to Reproduce

```
import numpy as np
import pandas as pd
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.datasets import make_classification
from sklearn.pipeline import FeatureUnion, Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.compose import ColumnTransformer
from joblib import parallel_backend


class CustomStandardScaler(BaseEstimator, TransformerMixin):
    def __init__(self, columns=None):
        self.columns = columns
        self.scaler = None

    def fit(self, X, y=None):
        self.scaler = StandardScaler()
        self.scaler.fit(X[self.columns])
        return self

    def transform(self, X):
        X_scaled = X[self.columns].copy()
        X_scaled = self.scaler.transform(X_scaled)
        return X_scaled

    def fit_transform(self, X, y=None):
        self.fit(X[self.columns], y)
        X_scaled = self.scaler.transform(X[self.columns])
        return X_scaled

    def set_output(self, *, transform=None):
        pass


def benchmark_feature_union(n_samples, n_steps, n_features, n_string_features=0):
    X, y = make_classification(n_samples=n_samples, n_features=n_features, random_state=42)
    X = pd.DataFrame(X)
    X.columns = [f'feature_{i}' for i in range(n_features)]
    num_cols = X.columns.values

    if n_string_features > 0:
        string_data = np.array([f"text_{i}" for i in range(n_samples)])
        string_columns = np.tile(string_data[:, np.newaxis], (1, n_string_features))
        X_string = pd.DataFrame(string_columns, columns=[f'text_feature_{i}' for i in range(n_string_features)])
        X = pd.concat([X, X_string], axis=1)
    steps = []
    for i in range(n_steps):
        step_name = f'scaler_{i + 1}'
        steps.append((step_name, CustomStandardScaler(columns=num_cols)))
    steps.append(('pass', 'passthrough'))

    feature_union = FeatureUnion(steps, verbose=True)

    pipeline = Pipeline([
        ('union', feature_union),
    ], verbose=True)
    pipeline.set_output(transform='pandas')
    with parallel_backend("threading", n_jobs=5):
        transformed_data = pipeline.fit_transform(X, y)

    return transformed_data

# Benchmark with only numerical features
benchmark_feature_union(n_samples=2_000_000, n_steps=5, n_features=150)
# Benchmark with additional string features
benchmark_feature_union(n_samples=2_000_000, n_steps=5, n_features=150, n_string_features=10)
``` 

### Expected Results

The performance should not degrade substantially when string columns are included in the passthrough.

### Actual Results

The performance with string columns is ~5 times slower (~21s vs 1.5min)

[FeatureUnion] ...... (step 2 of 6) Processing scaler_2, total=  11.2s
[FeatureUnion] .......... (step 6 of 6) Processing pass, total=   0.0s
[FeatureUnion] ...... (step 3 of 6) Processing scaler_3, total=  11.3s
[FeatureUnion] ...... (step 4 of 6) Processing scaler_4, total=  11.3s
[FeatureUnion] ...... (step 5 of 6) Processing scaler_5, total=  11.9s
[FeatureUnion] ...... (step 1 of 6) Processing scaler_1, total=  11.9s
[Pipeline] ............. (step 1 of 1) Processing union, total=  21.7s


[FeatureUnion] ...... (step 3 of 6) Processing scaler_3, total=   9.7s[FeatureUnion] ...... (step 5 of 6) Processing scaler_5, total=   9.7s
[FeatureUnion] .......... (step 6 of 6) Processing pass, total=   0.0s

[FeatureUnion] ...... (step 4 of 6) Processing scaler_4, total=   9.7s
[FeatureUnion] ...... (step 2 of 6) Processing scaler_2, total=   9.7s
[FeatureUnion] ...... (step 1 of 6) Processing scaler_1, total=   9.9s
[Pipeline] ............. (step 1 of 1) Processing union, total= 1.5min

Process finished with exit code 0

### Versions

```shell
System:
    python: 3.10.13 (main, Mar 13 2024, 14:52:58) [GCC 11.4.1 20230605 (Red Hat 11.4.1-2)]
   machine: Linux-5.14.0-427.22.1.el9_4.x86_64-x86_64-with-glibc2.34

Python dependencies:
      sklearn: 1.5.0
          pip: 24.0
   setuptools: 71.0.2
        numpy: 1.26.4
        scipy: 1.14.0
       Cython: None
       pandas: 2.2.2
   matplotlib: 3.9.1
       joblib: 1.4.2
threadpoolctl: 3.5.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libopenblas
        version: 0.3.23.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: blas
   internal_api: openblas
    num_threads: 32
         prefix: libscipy_openblas
        version: 0.3.27.dev
threading_layer: pthreads
   architecture: SkylakeX

       user_api: openmp
   internal_api: openmp
    num_threads: 32
         prefix: libgomp
        version: None
```

COMMENT:
It looks like something in your custom transformer leads to the `dtype` of every column being `object` when there are string columns. This doesn't happen when there are only floats. This is probably the source of the slowdown.

From what I understand about what you are trying to do I think it would make sense to use a [`ColumnTransformer`](https://scikit-learn.org/stable/modules/generated/sklearn.compose.ColumnTransformer.html) instead of your custom transformer. It is designed to apply a transformer (like `StandardScaler`) to a subset of columns. If the problem occurs also in that case we might be able to help debug it here.

For questions why your custom code is slow I'd probably go to StackOverflow or the Discussions section. The issue tracker is used for discussions related to scikit-learn's code base.

Labels: Bug, Needs Triage