Repository: sktime/sktime

Issue:
[BUG] `DynamicFactor` cannot predict quantiles in presence of exogenous variables
**Describe the bug**
<!--
A clear and concise description of what the bug is.
-->
`DynamicFactor` has `"ignores-exogeneous-X": False` and `"capability:pred_int": True`, but fails to predict quantiles when `X` is present.

**To Reproduce**
<!--
Add a Minimal, Complete, and Verifiable example (for more details, see e.g. https://stackoverflow.com/help/mcve

If the code is too long, feel free to put it in a public gist and link it in the issue: https://gist.github.com
-->

```python
from sktime.datasets import load_longley
_, df = load_longley()

y = df[["GNPDEFL", "GNP"]]
X = df[["UNEMP", "POP"]]

from sktime.forecasting.dynamic_factor import DynamicFactor
forecaster = DynamicFactor().fit(y[:10], X=X[:10])

forecaster.predict_quantiles(fh=[1, 2, 3, 4, 5, 6], X=X[10:])
```

**Expected behavior**
<!--
A clear and concise description of what you expected to happen.
-->
No error. Currently it fails with this:

> ValueError: Out-of-sample operations in a model with a regression component require additional exogenous values via the `exog` argument.

**Additional context**
<!--
Add any other context about the problem here.
-->
This is likely failing due to some issue in check of `exog`. Since `predict` works by checking on `self._forecaster`, failure in `_predict_interval` can be because of using `self._fitted_forecaster`.

**Versions**
<details>

<!--
Please run the following code snippet and paste the output here:

from sktime import show_versions; show_versions()
-->

```pycon
>>> from sktime import show_versions; show_versions()

System:
    python: 3.9.16 (main, Mar  8 2023, 14:00:05)  [GCC 11.2.0]
executable: /path/to/venv/bin/python
   machine: Linux-5.15.90.1-microsoft-standard-WSL2-x86_64-with-glibc2.35

Python dependencies:
          pip: 23.1.2
       sktime: 0.19.2
      sklearn: 1.2.2
       skbase: 0.4.0
        numpy: 1.24.3
        scipy: 1.10.1
       pandas: 1.5.3
   matplotlib: 3.7.1
       joblib: 1.2.0
  statsmodels: 0.13.5
        numba: 0.57.1
     pmdarima: 2.0.3
      tsfresh: 0.20.0
   tensorflow: 2.12.0
tensorflow_probability: 0.19.0
>>> 
```

</details>

<!-- Thanks for contributing! -->



Discussion:
hm, two questions without answer that I have while reading this:

* is this covered by the new functionality in the `_StatsModelsAdapter`, potentially related?
* why do the tests not detect this? They should, and they should test the proba predictions with exogeneous data.
FYI @lbventura
confirmed on current `main`, windows, python 3.10

Resolving patch:
diff --git a/sktime/forecasting/dynamic_factor.py b/sktime/forecasting/dynamic_factor.py
--- a/sktime/forecasting/dynamic_factor.py
+++ b/sktime/forecasting/dynamic_factor.py
@@ -1,14 +1,13 @@
 # copyright: sktime developers, BSD-3-Clause License (see LICENSE file)
 """Implements DynamicFactor Model as interface to statsmodels."""
-
-import inspect
+from typing import List, Union
 
 import numpy as np
 import pandas as pd
 
 from sktime.forecasting.base.adapters import _StatsModelsAdapter
 
-_all_ = ["DynamicFactor"]
+__all__ = ["DynamicFactor"]
 __author__ = ["Ris-Bali", "lbventura"]
 
 
@@ -211,10 +210,8 @@ def _predict(self, fh, X=None):
         # beginning of the training series when passing integers
         start, end = fh.to_absolute_int(self._y.index[0], self.cutoff)[[0, -1]]
 
-        if "exog" in inspect.signature(self._forecaster.__init__).parameters.keys():
-            y_pred = self._fitted_forecaster.predict(start=start, end=end, exog=X)
-        else:
-            y_pred = self._fitted_forecaster.predict(start=start, end=end)
+        y_pred = self._fitted_forecaster.predict(start=start, end=end, exog=X)
+
         # statsmodels forecasts all periods from start to end of forecasting
         # horizon, but only return given time points in forecasting horizon
 
@@ -224,7 +221,7 @@ def _predict(self, fh, X=None):
             )
         return y_pred.loc[fh.to_absolute_index(self.cutoff)]
 
-    def _predict_interval(self, fh, X=None, coverage: [float] = None):
+    def _predict_interval(self, fh, X=None, coverage: Union[float, List[float]] = None):
         """Compute/return prediction quantiles for a forecast.
 
         private _predict_interval containing the core logic,
@@ -278,10 +275,7 @@ def _predict_interval(self, fh, X=None, coverage: [float] = None):
         for coverage in coverage_list:
             alpha = 1 - coverage
 
-            if "exog" in inspect.signature(model.__init__).parameters.keys():
-                y_pred = model.get_forecast(steps=steps, exog=X).conf_int(alpha=alpha)
-            else:
-                y_pred = model.get_forecast(steps=steps).conf_int(alpha=alpha)
+            y_pred = model.get_forecast(steps=steps, exog=X).conf_int(alpha=alpha)
 
             y_pred = y_pred.iloc[ix]
 


Test patch:
diff --git a/sktime/forecasting/tests/test_dynamic_factor.py b/sktime/forecasting/tests/test_dynamic_factor.py
new file mode 100644
--- /dev/null
+++ b/sktime/forecasting/tests/test_dynamic_factor.py
@@ -0,0 +1,131 @@
+"""Tests the DynamicFactor model."""
+import pytest
+from pandas.testing import assert_frame_equal
+
+from sktime.datasets import load_longley
+from sktime.forecasting.dynamic_factor import DynamicFactor
+from sktime.utils.validation._dependencies import _check_soft_dependencies
+
+__author__ = ["yarnabrina"]
+
+HISTORY_LENGTH = 10
+PREDICTION_LENGTH = 6
+
+K_FACTORS = 1
+FACTOR_ORDER = 1
+
+COVERAGES = [0.95, 0.99]
+
+_, MULTIVARIATE_DF = load_longley()
+
+TARGET_COLUMNS = ["GNPDEFL", "GNP"]
+FEATURE_COLUMNS = ["UNEMP", "POP"]
+
+ENDOGENOUS_DF = MULTIVARIATE_DF[TARGET_COLUMNS]
+EXOGENOUS_DF = MULTIVARIATE_DF[FEATURE_COLUMNS]
+
+TRAIN_Y = ENDOGENOUS_DF[:HISTORY_LENGTH]
+TRAIN_X = EXOGENOUS_DF[:HISTORY_LENGTH]
+
+PREDICT_X = EXOGENOUS_DF[HISTORY_LENGTH : (HISTORY_LENGTH + PREDICTION_LENGTH)]
+
+
+def compare_predictions_against_statsmodels(
+    sktime_point_predictions, sktime_interval_predictions, statsmodels_predictions
+) -> None:
+    """Compare predictions from ``sktime`` wrapper against ``statsmodels`` estimator.
+
+    Notes
+    -----
+    * compare point predictions - predictive mean
+    * compare confidence intervals for multiple coverage values, viz. ``COVERAGES``
+    """
+    statsmodels_point_predictions = statsmodels_predictions.predicted_mean
+    assert_frame_equal(sktime_point_predictions, statsmodels_point_predictions)
+
+    for coverage in COVERAGES:
+        statsmodels_interval_predictions = statsmodels_predictions.conf_int(
+            alpha=(1 - coverage)
+        )
+
+        for target in TARGET_COLUMNS:
+            sktime_results = sktime_interval_predictions.xs(
+                (target, coverage), axis="columns"
+            )
+
+            statsmodels_results = statsmodels_interval_predictions.filter(
+                regex=f"{target}$"
+            )
+            statsmodels_results.columns = ["lower", "upper"]
+
+            assert_frame_equal(sktime_results, statsmodels_results)
+
+
+@pytest.mark.skipif(
+    not _check_soft_dependencies("statsmodels", severity="none"),
+    reason="skip test if required soft dependency not available",
+)
+def test_DynamicFactor_without_exogenous_variables():
+    """Test ``DynamicFactor`` in absence of exogenous variables."""
+    from statsmodels.tsa.statespace.dynamic_factor import (
+        DynamicFactor as _DynamicFactor,
+    )
+
+    unfitted_sktime_model = DynamicFactor(
+        k_factors=K_FACTORS, factor_order=FACTOR_ORDER
+    )
+    fitted_sktime_model = unfitted_sktime_model.fit(TRAIN_Y)
+
+    sktime_point_predictions = fitted_sktime_model.predict(
+        fh=range(1, PREDICTION_LENGTH + 1)
+    )
+    sktime_interval_predictions = fitted_sktime_model.predict_interval(
+        fh=range(1, PREDICTION_LENGTH + 1), coverage=COVERAGES
+    )
+
+    unfitted_statsmodels_model = _DynamicFactor(TRAIN_Y, K_FACTORS, FACTOR_ORDER)
+    fitted_statsmodels_model = unfitted_statsmodels_model.fit()
+
+    statsmodels_predictions = fitted_statsmodels_model.get_prediction(
+        start=HISTORY_LENGTH, end=HISTORY_LENGTH + PREDICTION_LENGTH - 1
+    )
+
+    compare_predictions_against_statsmodels(
+        sktime_point_predictions, sktime_interval_predictions, statsmodels_predictions
+    )
+
+
+@pytest.mark.skipif(
+    not _check_soft_dependencies("statsmodels", severity="none"),
+    reason="skip test if required soft dependency not available",
+)
+def test_DynamicFactor_with_exogenous_variables():
+    """Test ``DynamicFactor`` in presence of exogenous variables."""
+    from statsmodels.tsa.statespace.dynamic_factor import (
+        DynamicFactor as _DynamicFactor,
+    )
+
+    unfitted_sktime_model = DynamicFactor(
+        k_factors=K_FACTORS, factor_order=FACTOR_ORDER
+    )
+    fitted_sktime_model = unfitted_sktime_model.fit(TRAIN_Y, X=TRAIN_X)
+
+    sktime_point_predictions = fitted_sktime_model.predict(
+        fh=range(1, PREDICTION_LENGTH + 1), X=PREDICT_X
+    )
+    sktime_interval_predictions = fitted_sktime_model.predict_interval(
+        fh=range(1, PREDICTION_LENGTH + 1), X=PREDICT_X, coverage=COVERAGES
+    )
+
+    unfitted_statsmodels_model = _DynamicFactor(
+        TRAIN_Y, K_FACTORS, FACTOR_ORDER, exog=TRAIN_X
+    )
+    fitted_statsmodels_model = unfitted_statsmodels_model.fit()
+
+    statsmodels_predictions = fitted_statsmodels_model.get_prediction(
+        start=HISTORY_LENGTH, end=HISTORY_LENGTH + PREDICTION_LENGTH - 1, exog=PREDICT_X
+    )
+
+    compare_predictions_against_statsmodels(
+        sktime_point_predictions, sktime_interval_predictions, statsmodels_predictions
+    )