Repository: sktime/sktime

Issue:
[BUG] `test_predict_proba` is expecting column name of `pred_dist` to be `0` always.
**Describe the bug**
<!--
A clear and concise description of what the bug is.
-->
All the tests in `check_estimator` are expecting column names to be preserved, while a single test `check_predict_proba` function, invoked inside `test_predict_proba`, in `sktime.forecasting.tests.test_all_forecasters.py` is expecting column name of `pred_dist` i.e., the output of `predict_proba`, to be `0` always.

**To Reproduce**
<!--
Add a Minimal, Complete, and Verifiable example (for more details, see e.g. https://stackoverflow.com/help/mcve

If the code is too long, feel free to put it in a public gist and link it in the issue: https://gist.github.com
-->
location: [here](https://github.com/sktime/sktime/blob/12d2770c213382f1ee99bb05a03f19e7ab4cb5e6/sktime/forecasting/tests/test_all_forecasters.py#L529).
```python
# existing code
if isinstance(y_train, pd.Series):
    assert (pred_cols == pd.Index([0])).all()
else:
    assert (pred_cols == y_train.columns).all()

# expected code
if isinstance(y_train, pd.Series):
    assert (pred_cols == y_train.name).all()
else:
    assert (pred_cols == y_train.columns).all()
```

**Expected behavior**
<!--
A clear and concise description of what you expected to happen.
-->
The function mentioned above has to be modified such that, it checks for column names of `y_train`, `pred_dist`, like rest of the tests.

**Additional context**
<!--
Add any other context about the problem here.
-->
Any changes in this test will affect existing univariate probabilistic forecasters such as `NaiveForecaster`. Therefore, fixing this issue also requires necessary changes in all the univariate probabilistic forecasters.

<!-- Thanks for contributing! -->



Discussion:
I think this is related to https://github.com/sktime/sktime/issues/4709 and https://github.com/sktime/sktime/pull/4780.

@Vasudeva-bit I think the issue is `y_train` may not always have a `name` attribute that is non-empty. May be something like this (not tested) will solve it?

```python3
if isinstance(y_train, pd.Series) and y_train.name:
    assert (pred_cols == y_train.name).all()
elif isinstance(y_train, pd.Series):  # automatically implies empty name
    assert (pred_cols == pd.Index([0])).all()
else:
    assert (pred_cols == y_train.columns).all()
```
> I think this is related to https://github.com/sktime/sktime/issues/4709 and https://github.com/sktime/sktime/pull/4780.

Hm, I think not as closely as one might suspect.

The two PRs and related design problems were restricted to interval and quantile forecasts.

I think it is more related to "should we use `name` as `col` or not if a conversion to a `DataFrame`-like type, from pd.Series`, happens", like the journey of suffering in these PR:
https://github.com/sktime/sktime/pull/3290
https://github.com/sktime/sktime/pull/3297
https://github.com/sktime/sktime/pull/3322
https://github.com/sktime/sktime/pull/3323
https://github.com/sktime/sktime/pull/4150
https://github.com/sktime/sktime/pull/4155
https://github.com/sktime/sktime/pull/4157

The root of evil is really that depending on how you convert or coerce `pd.Series` to `pd.DataFrame`, one of two things might happen: the `name` appears now as the single column name, or it gets purged and instead the integer `0` appears as single column name. It is one of the idiosyncrasies of `pandas`, and the "how" matters.

Unfortunately this also needs to be consistent in many places, conversions, back-conversions, etc. While the original problem - I hope - has been resolved, it seems to have reappeared with the `skpro` distribution output, which is strongly `pd.DataFrame` inspired, and has similar `index` and `columns` attributes.
> Maybe something like this (not tested) will solve it?

Yes, @yarnabrina, I think your test is the correct expectation. But it seems to me that it would be violated by many current forecasters, as @Vasudeva-bit has observed by accident.

Also, in case it is not the current behaviour, we need to run it through a deprecation period to move it to the one that seems more logical - people might have adapted to it downstream, and we do not want to break their code.
> @Vasudeva-bit I think the issue is `y_train` may not always have a `name` attribute that is non-empty. May be something like this (not tested) will solve it?

A few tests are expecting name to be preserved even if it is `None`, [here](https://github.com/sktime/sktime/blob/12d2770c213382f1ee99bb05a03f19e7ab4cb5e6/sktime/utils/_testing/forecasting.py#L138). Thats why I didn't handle the empty case separately. But as suggested, handling empty name `None` separately seems reasonable.

Could we return forecasts as `pd.DataFrame` always, irrespective of whether it is a univariate or multi-variate forecaster?

This way we can test for only one single type. And, while returning the predicted forecasts from `_base.py`, we could convert `pd.DataFrame` to `pd.Series` for univariate forecasters. When a `pd.DataFrame` is converted to `pd.Series`, the name of `pd.Series` would be same as `pd.DataFrame`'s name (attribute `name`, not `columns`).
> Could we return forecasts as `pd.DataFrame` always, irrespective of whether it is a univariate or multi-variate forecaster?

That is an interface decision we have already discussed a number of times, but then the question becomes, what do we do with other data containers, such as `dask` or `np.ndarray` - coerce to `pd.DataFrame` as well? If not, then only `pd.Series` gets special treatment here...

Not saying that it would be a bad choice, but it would be an exception from the "support multiple formats" that we currently have.
Oh, I get it. It's better not to treat them specially.
Having said that, the issue you highlighted originally seems to be an inconsistency we may like to fix - ahtough it requires deprecation.
https://www.sktime.net/en/stable/developer_guide/deprecation.html

Could you perhaps open a PR which has only your originally proposed fix (changes to test *and* arch), so we can use this as a starting point?

I would use that to look into the problem (domino effects on other estimators) and build a deprecation strategy around it.
I opened a PR #5384 with originally proposed fix.

Resolving patch:
diff --git a/sktime/forecasting/arch/_uarch.py b/sktime/forecasting/arch/_uarch.py
--- a/sktime/forecasting/arch/_uarch.py
+++ b/sktime/forecasting/arch/_uarch.py
@@ -251,6 +251,7 @@ def _fit(self, y, X=None, fh=None):
         if fh:
             self._horizon = fh
 
+        y_name = y.name
         self._forecaster = _ARCH(
             y=y,
             x=X,
@@ -277,6 +278,7 @@ def _fit(self, y, X=None, fh=None):
             options=self.options,
             backcast=self.backcast,
         )
+        y.name = y_name
         return self
 
     def _get_arch_result_object(self, fh=None, X=None):
@@ -344,7 +346,7 @@ def _predict(self, fh, X=None):
         y_pred = pd.Series(
             ArchResultObject.mean.values[-1],
             index=full_range,
-            name=str(self._y.name),
+            name=self._y.name,
         )
         y_pred = y_pred.loc[abs_idx.to_pandas()]
         y_pred.index = self._horizon.to_absolute_index(self.cutoff)
@@ -398,11 +400,15 @@ def _predict_interval(self, fh, X, coverage):
             upper_int = mean_forecast + (z_critical * std_err)
             lower_df = pd.DataFrame(
                 lower_int,
-                columns=[y_col_name + " " + str(alpha) + " " + "lower"],
+                columns=[
+                    y_col_name if y_col_name else "0" + " " + str(alpha) + " " + "lower"
+                ],
             )
             upper_df = pd.DataFrame(
                 upper_int,
-                columns=[y_col_name + " " + str(alpha) + " " + "upper"],
+                columns=[
+                    y_col_name if y_col_name else "0" + " " + str(alpha) + " " + "upper"
+                ],
             )
             df_list.append(pd.concat((lower_df, upper_df), axis=1))
         concat_df = pd.concat(df_list, axis=1)
@@ -410,7 +416,7 @@ def _predict_interval(self, fh, X, coverage):
             OrderedDict.fromkeys(
                 [
                     col_df
-                    for col in y_col_name
+                    for col in (y_col_name if y_col_name else "0")
                     for col_df in concat_df.columns
                     if col in col_df
                 ]
@@ -425,7 +431,7 @@ def _predict_interval(self, fh, X, coverage):
         final_columns = list(
             itertools.product(
                 *[
-                    [y_col_name],
+                    [y_col_name if y_col_name else 0],
                     coverage,
                     df.columns.get_level_values(2).unique(),
                 ]


Test patch:
diff --git a/sktime/forecasting/tests/test_all_forecasters.py b/sktime/forecasting/tests/test_all_forecasters.py
--- a/sktime/forecasting/tests/test_all_forecasters.py
+++ b/sktime/forecasting/tests/test_all_forecasters.py
@@ -379,7 +379,7 @@ def get_expected_columns():
 
         found = pred_ints.columns.to_flat_index()
         msg = (
-            "columns of returned prediction interval DataFrame do not"
+            "columns of returned prediction interval DataFrame do not "
             f"match up with expected columns. Expected: {expected},"
             f"found: {found}"
         )