Repository: sktime/sktime

Issue:
[BUG] `ignores-exogeneous-X` is kept for `TransformedTargetForecaster` even if transformer would use exog
**Describe the bug**
`ignores-exogeneous-X` is kept for `TransformedTargetForecaster` even if transformer would use exog

I want to have a forecasting pipeline where I define part of the logic in the transformer. For example, I want to impute holidays data. Holidays mask is exogeneous. I have implemented a transformer that does the job, but it is not working because my final transformer `ignores-exogeneous-X`.

**To Reproduce**
```python
from sktime.forecasting.naive import NaiveForecaster
from sktime.transformations.series.feature_selection import FeatureSelection
from sktime.forecasting.fbprophet import Prophet

print(f"{Prophet().get_tag('ignores-exogeneous-X') = }")
print(f"{(FeatureSelection() * Prophet()).get_tag('ignores-exogeneous-X') = }")

print(f"{NaiveForecaster().get_tag('ignores-exogeneous-X') = }")
print(f"{(FeatureSelection() * NaiveForecaster()).get_tag('ignores-exogeneous-X') = }")
```

```python
Prophet().get_tag('ignores-exogeneous-X') = False
(FeatureSelection() * Prophet()).get_tag('ignores-exogeneous-X') = False
NaiveForecaster().get_tag('ignores-exogeneous-X') = True
(FeatureSelection() * NaiveForecaster()).get_tag('ignores-exogeneous-X') = True
```

**Expected behavior**
```python
Prophet().get_tag('ignores-exogeneous-X') = False
(FeatureSelection() * Prophet()).get_tag('ignores-exogeneous-X') = False
NaiveForecaster().get_tag('ignores-exogeneous-X') = True
(FeatureSelection() * NaiveForecaster()).get_tag('ignores-exogeneous-X') = **False**
```

**Versions**
```
System:
    python: 3.10.12 | packaged by conda-forge | (main, Jun 23 2023, 22:39:40) [Clang 15.0.7 ]
executable: /opt/homebrew/Caskroom/miniconda/base/envs/hcem_test/bin/python
   machine: macOS-13.2.1-x86_64-i386-64bit

Python dependencies:
          pip: 23.2.1
       sktime: 0.24.0
      sklearn: 1.3.1
       skbase: 0.5.1
        numpy: 1.25.2
        scipy: 1.11.3
       pandas: 2.1.1
   matplotlib: 3.8.0
       joblib: 1.3.2
        numba: None
  statsmodels: 0.14.0
     pmdarima: 2.0.3
statsforecast: None
      tsfresh: None
      tslearn: None
        torch: None
   tensorflow: None
tensorflow_probability: None
```


Discussion:
`MultiplexForecaster` also has unexpected behavior:
```python
MultiplexForecaster(forecasters=[Prophet(), NaiveForecaster()]).get_tag('ignores-exogeneous-X') = False
MultiplexForecaster(forecasters=[NaiveForecaster(), Prophet()]).get_tag('ignores-exogeneous-X') = True
```
> `MultiplexForecaster` also has unexpected behavior:
> 
> ```python
> MultiplexForecaster(forecasters=[Prophet(), NaiveForecaster()]).get_tag('ignores-exogeneous-X') = False
> MultiplexForecaster(forecasters=[NaiveForecaster(), Prophet()]).get_tag('ignores-exogeneous-X') = True
> ```

This is not unexpected, and clearly documented. If `selected_forecaster` is not passed, it defaults to the first one.

---

Your original issue makes sense to me. Do you want to make a PR, similar to #5353 's tag cloning part?
@yarnabrina 
But if I use it to fit with ForecastingGridSearchCV, then if the first forecaster ignores exogeneous, and some others don't, then they will also ignore exogeneous because exog will be filtered out too early.
I will take a look at #5353 tomorrow and try to prepare a PR.
> But if I use it to fit with ForecastingGridSearchCV, then if the first forecaster ignores exogeneous, and some others don't, then they will also ignore exogeneous because exog will be filtered out too early.

!!!!

That's a good spot.
> But if I use it to fit with ForecastingGridSearchCV, then if the first forecaster ignores exogeneous, and some others don't, then they will also ignore exogeneous because exog will be filtered out too early.

Wait, is that actually true?

Internally, `ForecastingGridSearchCV` does `set_params`, which resets the forecaster internally, and also sets the tags correctly

Logically, internally we make clones of the multiplexer with `set_params`. After each `set_params` call, the tags are determined by the parameters.

So, this should not be a problem?
@SmirnGregHM, @yarnabrina, can we drill on the expected behaviour in the original post?

I think the expected behaviour is not "set it for False for any pipeline".

The behaviour imo should be (for `TransformedTargetForecaster`):

* if none of the transformers use `y`, i.e., all transformers have the tag `"y_inner_mtype": "None"`, then the resulting pipeline should have `"ignores-exogeneous-X": True`
* if at least one of the transformers uses `y`, i.e., at least one transformer has a tag `"y_inner_mtype"` with a value other than `"None"`, the resulting pipeline sohuld have `"ignores-exogeneous-X": False`

Can you confirm that this would be, or not, your precise expectation?

Also, @SmirnGregHM, as a workaround you can use `set_tag` to set the tag manually, unfortunately that won't work inside a composite or tuner etc.

Resolving patch:
diff --git a/sktime/forecasting/compose/_pipeline.py b/sktime/forecasting/compose/_pipeline.py
--- a/sktime/forecasting/compose/_pipeline.py
+++ b/sktime/forecasting/compose/_pipeline.py
@@ -843,6 +843,21 @@ def __init__(self, steps):
         self.clone_tags(self.forecaster_, tags_to_clone)
         self._anytagis_then_set("fit_is_empty", False, True, self.steps_)
 
+        # above, we cloned the ignores-exogeneous-X tag,
+        # but we also need to check whether X is used as y in some transformer
+        # in this case X is not ignored by the pipe, even if the forecaster ignores it
+        # logic below checks whether there is at least one such transformer
+        # if there is, we override the ignores-exogeneous-X tag to False
+        # also see discussion in bug issue #5518
+        pre_ts = self.transformers_pre_
+        post_ts = self.transformers_post_
+        pre_use_y = [est.get_tag("y_inner_mtype") != "None" for _, est in pre_ts]
+        post_use_y = [est.get_tag("y_inner_mtype") != "None" for _, est in post_ts]
+        any_t_use_y = any(pre_use_y) or any(post_use_y)
+
+        if any_t_use_y:
+            self.set_tags(**{"ignores-exogeneous-X": False})
+
     @property
     def forecaster_(self):
         """Return reference to the forecaster in the pipeline.


Test patch:
diff --git a/sktime/forecasting/compose/tests/test_pipeline.py b/sktime/forecasting/compose/tests/test_pipeline.py
--- a/sktime/forecasting/compose/tests/test_pipeline.py
+++ b/sktime/forecasting/compose/tests/test_pipeline.py
@@ -705,3 +705,47 @@ def test_featurizer_forecastingpipeline_logic():
 
     forecaster.fit(y_train, X=X_train, fh=[1])  # try to forecast next year
     forecaster.predict(X=X_test)  # dummy X to predict next year
+
+
+def test_exogenousx_ignore_tag_set():
+    """Tests that TransformedTargetForecaster sets X tag for feature selection.
+
+    If the forecaster ignores X, but the feature selector does not, then the
+    ignores-exogeneous-X tag should be correctly set to False, not True.
+
+    This is the failure case in bug report #5518.
+
+    More generally, the tag should be set to True iff all steps in the pipeline
+    ignore X.
+    """
+    from sktime.forecasting.compose import YfromX
+    from sktime.transformations.series.feature_selection import FeatureSelection
+
+    fcst_does_not_ignore_x = YfromX.create_test_instance()
+    fcst_ignores_x = NaiveForecaster()
+
+    trafo_ignores_x = ExponentTransformer()
+    trafo_does_not_ignore_x = FeatureSelection()
+
+    # check that ignores-exogeneous-X tag is set correctly
+    pipe1 = trafo_ignores_x * fcst_does_not_ignore_x
+    pipe2 = trafo_ignores_x * fcst_ignores_x
+    pipe3 = trafo_does_not_ignore_x * fcst_does_not_ignore_x
+    pipe4 = trafo_does_not_ignore_x * fcst_ignores_x
+    pipe5 = trafo_ignores_x * trafo_does_not_ignore_x * fcst_does_not_ignore_x
+    pipe6 = trafo_ignores_x * trafo_does_not_ignore_x * fcst_ignores_x
+    pipe7 = trafo_ignores_x * trafo_ignores_x * fcst_does_not_ignore_x
+    pipe8 = trafo_ignores_x * fcst_ignores_x * trafo_does_not_ignore_x
+    pipe9 = trafo_does_not_ignore_x * fcst_ignores_x * trafo_ignores_x
+    pipe10 = trafo_ignores_x * fcst_ignores_x * trafo_ignores_x
+
+    assert not pipe1.get_tag("ignores-exogeneous-X")
+    assert pipe2.get_tag("ignores-exogeneous-X")
+    assert not pipe3.get_tag("ignores-exogeneous-X")
+    assert not pipe4.get_tag("ignores-exogeneous-X")
+    assert not pipe5.get_tag("ignores-exogeneous-X")
+    assert not pipe6.get_tag("ignores-exogeneous-X")
+    assert not pipe7.get_tag("ignores-exogeneous-X")
+    assert not pipe8.get_tag("ignores-exogeneous-X")
+    assert not pipe9.get_tag("ignores-exogeneous-X")
+    assert pipe10.get_tag("ignores-exogeneous-X")