Repository: sktime/sktime

Issue:
[BUG] UnobservedComponents throws error during predict() call, when wrapped under TransformedTargetForecaster and ForecastingPipeline
**Describe the bug**

> Initially identified issue:

_New version of sktime throws error for UnobservedComponents when wrapped under ForecastingGridSearchCV with TransformedTargetForecaster pipe._

> Updated description:

From sktime v.0.11.1 onwards untill the current version of main, UnobservedComponents throws error during Predict method call when it is piped under TransformedTargetForecaster(). 
_The same construct works with all the other models which I could test for example ARIMA, AutoETS etc. So the issue is how TransformedTargetForecaster calls predict and the cascading of the same method to UnobservedComponents class._

**To Reproduce**
The code example was taken from documentation of ForecastingGridSearchCV for advanced example. The only thing changed was calling UnobservedComponents instead of ExponentialSmoothing at the end of param_grid argument in gscv.
<!--
Add a Minimal, Complete, and Verifiable example (for more details, see e.g. https://stackoverflow.com/help/mcve

If the code is too long, feel free to put it in a public gist and link it in the issue: https://gist.github.com
-->

```python
from sktime.datasets import load_shampoo_sales
from sktime.forecasting.exp_smoothing import ExponentialSmoothing
from sktime.forecasting.naive import NaiveForecaster
from sktime.forecasting.model_selection import ExpandingWindowSplitter
from sktime.forecasting.model_selection import ForecastingGridSearchCV
from sktime.forecasting.compose import TransformedTargetForecaster
from sktime.forecasting.theta import ThetaForecaster
from sktime.transformations.series.impute import Imputer
from sktime.forecasting.structural import UnobservedComponents

y = load_shampoo_sales()
fh = [1,2,3]
pipe = TransformedTargetForecaster(steps=[
    ("imputer", Imputer()),
    ("forecaster", UnobservedComponents())])
cv = ExpandingWindowSplitter(
    initial_window=24,
    step_length=12,
    start_with_window=True,
    fh=[1,2,3])
gscv = ForecastingGridSearchCV(
    forecaster=pipe,
    param_grid=[{
        "forecaster": [NaiveForecaster(sp=12)],
        "forecaster__strategy": ["drift", "last", "mean"],
    },
    {
        "imputer__method": ["mean", "drift"],
        "forecaster": [ThetaForecaster(sp=12)],
    },
    {
        "imputer__method": ["mean", "last"],
        "forecaster": [UnobservedComponents()],
        "forecaster__seasonal": [12],
    },
    ],
    cv=cv,
    n_jobs=-1)
gscv.fit(y)

y_pred = gscv.predict(fh)
```

**Expected behavior**
<!--
A clear and concise description of what you expected to happen.
-->

**Additional context**
Error report:
```python
TypeError                                 Traceback (most recent call last)
~/work/chronos/chronos/pipeline/tests/test_pipeline.py in <module>
     37     cv=cv,
     38     n_jobs=-1)
---> 39 gscv.fit(y)
     40 
     41 y_pred = gscv.predict(fh)

~/work/sktime/sktime/forecasting/base/_base.py in fit(self, y, X, fh)
    262         # we call the ordinary _fit if no looping/vectorization needed
    263         if not vectorization_needed:
--> 264             self._fit(y=y_inner, X=X_inner, fh=fh)
    265         else:
    266             # otherwise we call the vectorized version of fit

~/work/sktime/sktime/forecasting/model_selection/_tune.py in _fit(self, y, X, fh)
    279 
    280         # Run grid-search cross-validation.
--> 281         results = self._run_search(evaluate_candidates)
    282 
    283         results = pd.DataFrame(results)

~/work/sktime/sktime/forecasting/model_selection/_tune.py in _run_search(self, evaluate_candidates)
    487         """Search all candidates in param_grid."""
    488         _check_param_grid(self.param_grid)
--> 489         return evaluate_candidates(ParameterGrid(self.param_grid))
    490 
    491     @classmethod

~/work/sktime/sktime/forecasting/model_selection/_tune.py in evaluate_candidates(candidate_params)
    266 
    267             out = parallel(
--> 268                 delayed(_fit_and_score)(params) for params in candidate_params
    269             )
    270 

~/opt/miniconda3/envs/chronos_dev2/lib/python3.7/site-packages/joblib/parallel.py in __call__(self, iterable)
   1054 
   1055             with self._backend.retrieval_context():
-> 1056                 self.retrieve()
   1057             # Make sure that we get a last message telling us we are done
   1058             elapsed_time = time.time() - self._start_time

~/opt/miniconda3/envs/chronos_dev2/lib/python3.7/site-packages/joblib/parallel.py in retrieve(self)
    933             try:
    934                 if getattr(self._backend, 'supports_timeout', False):
--> 935                     self._output.extend(job.get(timeout=self.timeout))
    936                 else:
    937                     self._output.extend(job.get())

~/opt/miniconda3/envs/chronos_dev2/lib/python3.7/site-packages/joblib/_parallel_backends.py in wrap_future_result(future, timeout)
    540         AsyncResults.get from multiprocessing."""
    541         try:
--> 542             return future.result(timeout=timeout)
    543         except CfTimeoutError as e:
    544             raise TimeoutError from e

~/opt/miniconda3/envs/chronos_dev2/lib/python3.7/concurrent/futures/_base.py in result(self, timeout)
    433                 raise CancelledError()
    434             elif self._state == FINISHED:
--> 435                 return self.__get_result()
    436             else:
    437                 raise TimeoutError()

~/opt/miniconda3/envs/chronos_dev2/lib/python3.7/concurrent/futures/_base.py in __get_result(self)
    382     def __get_result(self):
    383         if self._exception:
--> 384             raise self._exception
    385         else:
    386             return self._result

TypeError: No valid mtype could be identified
```
<!--
Add any other context about the problem here.
-->

**Versions**
System:
    python: 3.7.13 (default, Mar 28 2022, 07:24:34)  [Clang 12.0.0 ]
executable: .../miniconda3/envs/chronos_dev2/bin/python
   machine: Darwin-21.4.0-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.2
   setuptools: 58.0.4
      sklearn: 1.0.2
       sktime: 0.11.3
  statsmodels: 0.12.1
        numpy: 1.21.5
        scipy: 1.7.3
       pandas: 1.3.5
   matplotlib: 3.5.1
       joblib: 1.1.0
        numba: 0.55.1
     pmdarima: 1.8.5
      tsfresh: None

<!--
Please run the following code snippet and paste the output here:
 
from sktime import show_versions; show_versions()
-->


</details>

<!-- Thanks for contributing! -->



Discussion:
following minimal solution works on `main` WITHOUT `TransformedTargetForecaster`:

```python
from sktime.datasets import load_shampoo_sales
from sktime.forecasting.exp_smoothing import ExponentialSmoothing
from sktime.forecasting.naive import NaiveForecaster
from sktime.forecasting.model_selection import ExpandingWindowSplitter, SlidingWindowSplitter
from sktime.forecasting.model_selection import ForecastingGridSearchCV
from sktime.forecasting.compose import TransformedTargetForecaster
from sktime.forecasting.theta import ThetaForecaster
from sktime.transformations.series.impute import Imputer
from sktime.forecasting.structural import UnobservedComponents

y = load_shampoo_sales()
fh = [1,2,3]
forecaster = UnobservedComponents()
cv = ExpandingWindowSplitter(
    initial_window=24,
    step_length=12,
    start_with_window=True,
    fh=[1,2,3])
gscv = ForecastingGridSearchCV(
    forecaster=forecaster,
    param_grid=
    {
        "transformed": [True, False],
    },
    cv=cv,
    n_jobs=-1)
gscv.fit(y)

y_pred = gscv.predict(fh)

```

Following minimal solution works not on `main` WITH `TransformedTargetForecaster`:
```python

from sktime.datasets import load_shampoo_sales
from sktime.forecasting.exp_smoothing import ExponentialSmoothing
from sktime.forecasting.naive import NaiveForecaster
from sktime.forecasting.model_selection import ExpandingWindowSplitter, SlidingWindowSplitter
from sktime.forecasting.model_selection import ForecastingGridSearchCV
from sktime.forecasting.compose import TransformedTargetForecaster
from sktime.forecasting.theta import ThetaForecaster
from sktime.transformations.series.impute import Imputer
from sktime.forecasting.structural import UnobservedComponents

y = load_shampoo_sales()
fh = [1,2,3]
pipe = TransformedTargetForecaster(steps=[
    ("imputer", Imputer()),
    ("forecaster", UnobservedComponents())])
cv = ExpandingWindowSplitter(
    initial_window=24,
    step_length=12,
    start_with_window=True,
    fh=[1,2,3])
gscv = ForecastingGridSearchCV(
    forecaster=pipe,
    param_grid=
    {
        "forecaster__transformed": [True, False],
    },
    cv=cv,
    n_jobs=-1)
gscv.fit(y)

y_pred = gscv.predict(fh)
```
thanks @aiwalter for enhancement of bug description
@aiwalter 
This works completely
```python
y = load_shampoo_sales()
fh = [1,2,3]
pipe = TransformedTargetForecaster(steps=[
    ("imputer", Imputer()),
    ("forecaster", ThetaForecaster())])

pipe.fit(y)
pipe.predict(fh=fh)
```

Whereas this one only works until fit. So predict method is throwing error (which is pasted just below the code example)
```python
y = load_shampoo_sales()
fh = [1,2,3]
pipe = TransformedTargetForecaster(steps=[
    ("imputer", Imputer()),
    ("forecaster", UnobservedComponents())])

pipe.fit(y)
# predict throws error as pasted below
pipe.predict(fh=fh)
```
error report:
```python
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
      6 
      7 pipe.fit(y)
----> 8 pipe.predict(fh=fh)

~/work/sktime/sktime/forecasting/base/_base.py in predict(self, fh, X)
    327             self._y_mtype_last_seen,
    328             store=self._converter_store_y,
--> 329             store_behaviour="freeze",
    330         )
    331 

~/work/sktime/sktime/datatypes/_convert.py in convert_to(obj, to_type, as_scitype, store, store_behaviour)
    239 
    240     # now further narrow down as_scitype by inference from the obj
--> 241     from_type = infer_mtype(obj=obj, as_scitype=as_scitype)
    242     as_scitype = mtype_to_scitype(from_type)
    243 

~/work/sktime/sktime/datatypes/_check.py in mtype(obj, as_scitype, exclude_mtypes)
    311 
    312     if len(res) < 1:
--> 313         raise TypeError("No valid mtype could be identified")
    314 
    315     return res[0]

TypeError: No valid mtype could be identified
```
same error with `ForecastingPipeline`
I improved the error message on `mtype` in this PR (would appreciate a review and potential approval) https://github.com/alan-turing-institute/sktime/pull/2606
with the purpose to understand why the check is failing, inside the pipeline.
The result of that exercise gives this, more informative error:

``` No valid mtype could be identified for object of type <class 'pandas.core.frame.DataFrame'>. 
Errors returned are as follows, in format [mtype]: [error message] 
pd.DataFrame: <class 'pandas.core.indexes.base.Index'> is not supported for obj,
use one of (<class 'pandas.core.indexes.range.RangeIndex'>,
<class 'pandas.core.indexes.period.PeriodIndex'>,
<class 'pandas.core.indexes.datetimes.DatetimeIndex'>)
or integer index instead.
(etcetera)
```

This means, the prediction that is being produced has an index which is non-conformant with the `sktime` type specification for the mtype `pd.DataFrame` - because it is a `pd.Index`, and not one of the four allowed, more specialized index types.
so, two questions:

* where is the `pd.Index` created?
* why do the tests not catch this, they should catch output of non-conformant data types?

The place the error is raised is at the very end, i.e., just before we would return the output of the `TransformedTargetForecaster`, i.e., all predicts and transforms internally have been executed and have raised no similar error.
it's an empty `pd.DataFrame` when it reaches the `convert_to` the second time

it's coming out of `TransformedTargetForecaster._predict`
Could it be that the UnobservedComponents predict output creates this index type when it returns the output?? Meaning modifying the _predict method of UnobservedComponents may be required additionally in Sktime side instead of simply calling it from statsmodel adaptors? I also noticed that UC also renames the predict output unlike other forecasters. It renames the output to "preds".
Actually the renaming of pred output by this UC models makes it also incompatible with ColumnwiseTransformer (CwT), which applies transformations and their inverse_transformations by the names of columns in Y dataframe. So Y_train with name "Number of Air Passengers" is passed (after predict from UC) as pd.Series(name="mean_preds") to inverse_transform method of CwT, which then throws error as "mean_preds" does not exist in columns ; error is raised from _check_columns() function.
 
@aiwalter FYI - a new bug but I will create a detailed issue later for that, unless it is clear to you from above example already. 
> Could it be that the UnobservedComponents predict output creates this index type when it returns the output?

The problem is an *empty dataframe*, not a wrong index type. Empty dataframes have the `pd.Index` type, apparently, which is why the check says that.

Suspicion: the problem is probably caused by the name that `UnobservedComponents` gives the series before it returns it.
This makes it break with a `pd.DataFrame` input:

```python
y = pd.DataFrame(load_shampoo_sales())
fh = [1,2,3]

f = UnobservedComponents()

f.fit(y)
f.predict(fh=fh)
```
found the problem!

Combination of two issues:

* unintuitive behaviour of `pd.DataFrame` constructor. When called as `pd.DataFrame(my_series, columns=["col_name"])`, this will produce an *empty data frame* if `my_series` has a name, otherwise a `pd.DataFrame` with column name `"col_name"`
* `UnobserveComponents._predict` produces a non-conformant `pd.Series` as a return, in that it has a name (`predicted_mean`), as opposed to not having a name

I think we need to do two things:

* immediate bugfix: fix the converter so it can deal with `pd.Series` to `pd.DataFrame` even when the series has a name
* set up tests for the inner methods, whether they produce non-conformant returns. Currently, this is only tested for the outer `predict`, and only for certain scenarios.

Resolving patch:
diff --git a/sktime/datatypes/_series/_convert.py b/sktime/datatypes/_series/_convert.py
--- a/sktime/datatypes/_series/_convert.py
+++ b/sktime/datatypes/_series/_convert.py
@@ -56,14 +56,14 @@ def convert_UvS_to_MvS_as_Series(obj: pd.Series, store=None) -> pd.DataFrame:
     if not isinstance(obj, pd.Series):
         raise TypeError("input must be a pd.Series")
 
+    res = pd.DataFrame(obj)
+
     if (
         isinstance(store, dict)
         and "columns" in store.keys()
         and len(store["columns"]) == 1
     ):
-        res = pd.DataFrame(obj, columns=store["columns"])
-    else:
-        res = pd.DataFrame(obj)
+        res.columns = store["columns"]
 
     return res
 


Test patch: