set_output API do not preserve original dtypes for pandas

### Describe the bug

Following issue #24182,

When using the set_output with expected output to be a pandas' data frame, while converting tougher columns with different dtypes the output does not preserve the original dtype but the "common type" by numpy.

Possible workaround is to create transformer for each column, but it doesn't feel like the right approach.

In collaboration with @BenEfrati

### Steps/Code to Reproduce

```python
import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer



X = pd.DataFrame({
                  "age": [1, 2, 4],
                  "float": [1.5, 2.5, None],
                  })
X["age"] = X["age"].astype(np.uint8)
X.dtypes

# age        uint8
# float    float64
# dtype: object

simple_imputer = SimpleImputer(strategy='constant', fill_value=1).set_output(transform="pandas")
X_trans_partial = simple_imputer.fit_transform(X)
X_trans_partial.dtypes

# age      float64
# float    float64
# dtype: object
```

### Expected Results

We'd expect the age column to keep being an unsigned 8-bit integer.

### Actual Results

the age column is now a 64-bit floating point.

### Versions

```shell
System:
    python: 3.9.12 (main, Apr  4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)]
executable: C:\New folder\venv\Scripts\python.exe
   machine: Windows-10-10.0.19044-SP0

Python dependencies:
      sklearn: 1.2.1
          pip: 23.0
   setuptools: 58.1.0
        numpy: 1.24.2
        scipy: 1.10.0
       Cython: None
       pandas: 1.5.3
   matplotlib: None
       joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

threadpoolctl info:
       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\New folder\venv\Lib\site-packages\numpy\.libs\libopenblas64__v0.3.21-gcc_10_3_0.dll
        version: 0.3.21
threading_layer: pthreads
   architecture: Haswell
    num_threads: 12

       user_api: openmp
   internal_api: openmp
         prefix: vcomp
       filepath: C:\New folder\venv\Lib\site-packages\sklearn\.libs\vcomp140.dll
        version: None
    num_threads: 12

       user_api: blas
   internal_api: openblas
         prefix: libopenblas
       filepath: C:\New folder\venv\Lib\site-packages\scipy.libs\libopenblas-802f9ed1179cb9c9b03d67ff79f48187.dll
        version: 0.3.18
threading_layer: pthreads
   architecture: Haswell
    num_threads: 12
```

COMMENT:
SimpleImputer has it's own edge cases to deal with. Let's say that `"age"` actually has a missing value using pandas `UInt8` dtype, the default `mean` strategy will require a float to hold the mean of `1.5` for age:

```python
import numpy as np
import pandas as pd
from sklearn.impute import SimpleImputer

X = pd.DataFrame(
    {
        "age": pd.Series([1, 2, None], dtype="UInt8"),
        "float": [1.5, 2.5, None],
    }
)
simple_imputer = SimpleImputer().set_output(transform="pandas")
X_trans_partial = simple_imputer.fit_transform(X)
X_trans_partial.dtypes
# age      float64
# float    float64
# dtype: object
```

So if we go with preserving the dtype for `SimpleImputer`, we have to do it with some rules:

1. If a feature has no missing values, then preserve the dtype.
2. If a feature has missing values, then `SimpleImputer` will try it's best to preserve dtypes. Even with `strategy="constant"`, when `fill_value=5.5`, then the output can not be `uint8` because it can not hold `5.5`.

I am unsure if the semantics listed above are too complex or surprising for users. The current "always output float64" is a fairly simple rule.

Labels: Bug, module:impute, Needs Decision - Include Feature