Repository: scikit-learn/scikit-learn
Pull request: FIX `TransformedTargetRegressor` warns when `set_output` expects dataframe
#### Reference Issues/PRs
closes #29361

#### What does this implement/fix? Explain your changes.
Prevents warnings on output type to be raised on `TransformedTargetRegressor().fit()` when global output is set to "pandas" or "polars".

#### Any other comments?
Did this together with @lesteve: we figured that `TransformedTargetRegressor` is in fact not a transformer and therefore there should not be a warning, neither would we expect that X is returned as a DataFrame, nor do we need to raise an exception. 
Setting `set_config(transform_output="pandas")` should have no effect within `TransformedTargetRegressor().fit()`.

I have also checked other places where `FunctionTransformer` is used, but these occurances all refer to transformers. So there should be no other places where we would need to apply the same fix.

In the second commit, I also suggest a documentation change to explain better the difference between `set_output(transform=None)` and `set_output(transform="default") ` (see [this comment](https://github.com/scikit-learn/scikit-learn/pull/24672#discussion_r997393483)).

Problem statement:
TransformedTargetRegressor warns about set_output set to pandas
### Describe the bug

If `set_output` is set to `"pandas"`, `TransformedTargetRegressor` warns unnecessarily.

### Steps/Code to Reproduce

```python
import numpy as np
import pandas as pd
from sklearn import set_config
from sklearn.compose import TransformedTargetRegressor
from sklearn.datasets import make_regression
from sklearn.linear_model import LinearRegression

set_config(transform_output="pandas")
X, y = make_regression()
y = np.abs(y) + 1
TransformedTargetRegressor(
    regressor=LinearRegression(),
    func=np.log,
    inverse_func=np.exp,
).fit(X, y)
```

### Expected Results

No warning.

### Actual Results

3 times the same warning:
```
python3.11/site-packages/sklearn/preprocessing/_function_transformer.py:303: UserWarning: When `set_output` is configured to be 'pandas', `func` should return a pandas DataFrame to follow the `set_output` API  or `feature_names_out` should be defined.
  warnings.warn(warn_msg.format("pandas"))
```

### Versions

```shell
System:
    python: 3.11.7
Python dependencies:
      sklearn: 1.5.0
      pandas: 2.2.2
```

Hints from issue discussion:
Thanks for the reproducer, this seems like an issue indeed, I'll try to take a closer look soonish.

Commit message: clearer documentation for args None and 'default'

Patch to review:
diff --git a/sklearn/_config.py b/sklearn/_config.py
index fc9392de68df6..e8da4bcaa3381 100644
--- a/sklearn/_config.py
+++ b/sklearn/_config.py
@@ -142,7 +142,7 @@ def set_config(
         - `"default"`: Default output format of a transformer
         - `"pandas"`: DataFrame output
         - `"polars"`: Polars output
-        - `None`: Transform configuration is unchanged
+        - `None`: Transform configuration is unchanged, global configuration is used
 
         .. versionadded:: 1.2
         .. versionadded:: 1.4
@@ -297,7 +297,7 @@ def config_context(
         - `"default"`: Default output format of a transformer
         - `"pandas"`: DataFrame output
         - `"polars"`: Polars output
-        - `None`: Transform configuration is unchanged
+        - `None`: Transform configuration is unchanged, global configuration is used
 
         .. versionadded:: 1.2
         .. versionadded:: 1.4
diff --git a/sklearn/compose/_column_transformer.py b/sklearn/compose/_column_transformer.py
index a5aa7db17d4ae..a8b6af47f0a98 100644
--- a/sklearn/compose/_column_transformer.py
+++ b/sklearn/compose/_column_transformer.py
@@ -350,7 +350,7 @@ def set_output(self, *, transform=None):
             - `"default"`: Default output format of a transformer
             - `"pandas"`: DataFrame output
             - `"polars"`: Polars output
-            - `None`: Transform configuration is unchanged
+            - `None`: Transform configuration is unchanged, global configuration is used
 
             .. versionadded:: 1.4
                 `"polars"` option was added.
diff --git a/sklearn/compose/_target.py b/sklearn/compose/_target.py
index ac33957b23ce2..800f7337129b5 100644
--- a/sklearn/compose/_target.py
+++ b/sklearn/compose/_target.py
@@ -198,6 +198,10 @@ def _fit_transformer(self, y):
                 validate=True,
                 check_inverse=self.check_inverse,
             )
+            # to force `_get_output_config()` to use the `set_output` setting of
+            # FunctionTransformer's instance and not the global configuration from
+            # `set_config`:
+            self.transformer_.set_output(transform="default")
         # XXX: sample_weight is not currently passed to the
         # transformer. However, if transformer starts using sample_weight, the
         # code should be modified accordingly. At the time to consider the
diff --git a/sklearn/compose/tests/test_target.py b/sklearn/compose/tests/test_target.py
index a971553b64739..8a96d3a297641 100644
--- a/sklearn/compose/tests/test_target.py
+++ b/sklearn/compose/tests/test_target.py
@@ -1,7 +1,7 @@
 import numpy as np
 import pytest
 
-from sklearn import datasets
+from sklearn import config_context, datasets
 from sklearn.base import BaseEstimator, TransformerMixin, clone
 from sklearn.compose import TransformedTargetRegressor
 from sklearn.dummy import DummyRegressor
@@ -393,3 +393,17 @@ def test_transform_target_regressor_pass_extra_predict_parameters():
     regr.fit(X, y)
     regr.predict(X, check_input=False)
     assert regr.regressor_.predict_called
+
+
+@pytest.mark.filterwarnings("error")
+@pytest.mark.parametrize("output_format", ["pandas", "polars"])
+def test_transform_target_regressor_not_warns_with_global_output_set(output_format):
+    """Test that TransformedTargetRegressor will not raise warnings if
+    set_config(transform_output="pandas"/"polars") is set globally; regression test for
+    issue #29361."""
+    X, y = datasets.make_regression()
+    y = np.abs(y) + 1
+    with config_context(transform_output=output_format):
+        TransformedTargetRegressor(
+            regressor=LinearRegression(), func=np.log, inverse_func=np.exp
+        ).fit(X, y)
diff --git a/sklearn/pipeline.py b/sklearn/pipeline.py
index 47faefcfd56ff..001c2d88f5c62 100644
--- a/sklearn/pipeline.py
+++ b/sklearn/pipeline.py
@@ -178,7 +178,7 @@ def set_output(self, *, transform=None):
             - `"default"`: Default output format of a transformer
             - `"pandas"`: DataFrame output
             - `"polars"`: Polars output
-            - `None`: Transform configuration is unchanged
+            - `None`: Transform configuration is unchanged, global configuration is used
 
             .. versionadded:: 1.4
                 `"polars"` option was added.
@@ -1456,7 +1456,7 @@ def set_output(self, *, transform=None):
             - `"default"`: Default output format of a transformer
             - `"pandas"`: DataFrame output
             - `"polars"`: Polars output
-            - `None`: Transform configuration is unchanged
+            - `None`: Transform configuration is unchanged, global configuration is used
 
         Returns
         -------
diff --git a/sklearn/utils/_set_output.py b/sklearn/utils/_set_output.py
index 42757dbb00fae..45e2893a7f34d 100644
--- a/sklearn/utils/_set_output.py
+++ b/sklearn/utils/_set_output.py
@@ -398,7 +398,7 @@ def set_output(self, *, transform=None):
             - `"default"`: Default output format of a transformer
             - `"pandas"`: DataFrame output
             - `"polars"`: Polars output
-            - `None`: Transform configuration is unchanged
+            - `None`: Transform configuration is unchanged, global configuration is used
 
             .. versionadded:: 1.4
                 `"polars"` option was added.


Code review comments:
diff_hunk: @@ -393,3 +393,17 @@ def test_transform_target_regressor_pass_extra_predict_parameters():
     regr.fit(X, y)
     regr.predict(X, check_input=False)
     assert regr.regressor_.predict_called
+
+
+@pytest.mark.filterwarnings("error")
path: sklearn/compose/tests/test_target.py
text: @user1:
Did you find this pattern used in other places in our test? Personally I would maybe find it a bit less clear than the [recommendation from Pytest](https://docs.pytest.org/en/7.1.x/how-to/capture-warnings.html#additional-use-cases-of-warnings-in-tests) for checking that no warnings is emitted:
```py
with warnings.catch_warnings():
    warnings.simplefilter("error")
    ...
```

If I had to give a small justification for it, I would say that the code checking that there is no warning is closer to code potentially creating the warning which maybe helps making the intent a bit clearer ...

@author:
Yes, I see what you mean. `@user2.mark.filterwarnings("error")` is actually not used anywhere else in scikit-learn. I was finding it by googling how to make the test not pass, when a warning is raised.

Let's better use `warnings.catch_warnings` instead.

diff_hunk: @@ -350,7 +350,7 @@ def set_output(self, *, transform=None):
             - `"default"`: Default output format of a transformer
             - `"pandas"`: DataFrame output
             - `"polars"`: Polars output
-            - `None`: Transform configuration is unchanged
+            - `None`: Transform configuration is unchanged, global configuration is used
path: sklearn/compose/_column_transformer.py
text: @user1:
Actually it is not necessarily using the global configuration, you could imagine something like this:

```py
estimator.set_output(transform='pandas')
estimator.set_output(transform=None)
```

The global configuration is `transform='default'` but the estimator is using `transform='pandas'`

More a meta comment: I get the fact that it may not be very clear from the docstring what `transform=None` does but I don't think there is an easy way to improve the situation in the docstring. Full disclosure: I am not sure about the use case for `transform=None` actually :thinking:.

@author:
Oh, I was thinking that we cannot actively set `transform=None`, because this is what the docstring says: `transform : {"default", "pandas", "polars"}, default=None`. 

Also, I have just discovered, there is
```
    supported_outputs = ADAPTERS_MANAGER.supported_outputs
    if dense_config not in supported_outputs:
        raise ValueError(
            f"output config must be in {sorted(supported_outputs)}, got {dense_config}"
        )
```
to prevent us from setting `transform=None`.

It can only stay `None` if it is untouched, and then the global configuration is used within the `_get_output_config()` function.

I'm happy to revert this if it's wrong, but I will keep my suggestions for now to facilitate discussion.

@user1:
> transform : {"default", "pandas", "polars"}, default=None

I guess the docstring is slightly misleading then. It seems like you can use `transform=None`:

```py
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler

X, y = load_iris(as_frame=True, return_X_y=True)

scaler = StandardScaler()
scaler.set_output(transform="pandas")
scaler.set_output(transform=None)

scaler.fit(X)
# this outputs a pandas DataFrame so transform=None did not change transform="pandas"
X_scaled = scaler.transform(X)
```


@user2:
`None` is used as a sentinel. In the future, f `set_output` supported `predict`, then the default `transform=None` will leave transform configuration alone:

```python
scaler.set_output(predict="pandas") 
```


@author:
Your example got me really confused @user1 , because I had tried a similar experiment with `FunctionTransformer`, that had led me to think that we cannot pass `transform=None`:

```python
from sklearn.datasets import load_iris
from sklearn.preprocessing import FunctionTransformer

#set_config(transform_output="pandas")  # same as using set_output before transform=None 
X, y = load_iris(as_frame=True, return_X_y=True)

functrans = FunctionTransformer()
functrans.set_output(transform="pandas")
functrans.set_output(transform=None)

functrans.fit(X)
# this raises a ValueError:
functrans.transform(X)
```

`ValueError: output config must be in ['default', 'pandas', 'polars'], got None`

It seems that `FunctionTransformer().set_output()` and `_SetOutputMixin.set_output() `(from `StandardScaler` ) are handeling this differently. The latter has an early [`if transform is None: return self`](https://github.com/scikit-learn/scikit-learn/blob/64ab789905077ba8990522688c11177442e5e91f/sklearn/utils/_set_output.py#L411), while the `FunctionTransformer` can pass [self._sklearn_output_config = {transform: None}](https://github.com/scikit-learn/scikit-learn/blob/64ab789905077ba8990522688c11177442e5e91f/sklearn/preprocessing/_function_transformer.py#L415C17-L415C61) to `_get_output_config()`, which would then raise the above error message.

I am not sure what to make of this, though...
Is this wanted/expected or should we allign this?

I think I should revert the documentation change, since `transform=None` is sometimes possible to pass and then the previous setting is used, not necessarily the global configuration.

@user1:
OK well, I think we should fix the bug first and leave the docstring modification for another PR when we understand a bit more the `.set_output` thing.



@author:
Yes, I agree. I will revert the documentation changes then.