Repository: sktime/sktime

Issue:
[BUG] Prophet does not work with integer indices
Update by @fkiraly:

after some discussion, I think this is the "actual" bug, i.e., that integer index does not work with `Prophet`:

Hm, I think it might be a bug, because this does not work:

```python
import numpy as np
import pandas as pd
from sktime.forecasting.fbprophet import Prophet

y = pd.DataFrame(np.arange(1, 100), columns=['0'])

f = Prophet()
f.fit(y)
predictions = f.predict(fh=[1])
```

I think it should, instead of erroring out.
The error message says that index needs to be sth or integer index, but it *is* integer index!

---

**Describe the bug**
<!--
A clear and concise description of what the bug is.
-->
Trying a flow similar (but not exactly the same) to the one in the Prophet getting started guide, I run into an issue when making the call to predict.

**To Reproduce**
<!--
Add a Minimal, Complete, and Verifiable example (for more details, see e.g. https://stackoverflow.com/help/mcve

If the code is too long, feel free to put it in a public gist and link it in the issue: https://gist.github.com
-->

```python
import numpy as np
import pandas as pd
from sktime.forecasting.fbprophet import Prophet

y = pd.DataFrame(np.arange(1, 100), columns=['0'])
y.index = y.index.astype('datetime64[ns]')
f = Prophet()
f.fit(y)
predictions = f.predict(fh=[1])
```

**Expected behavior**
<!--
A clear and concise description of what you expected to happen.
-->
Predictions should be made and stored in the `predictions` variable.

**Additional context**
<!--
Add any other context about the problem here.
-->

**Versions**
<details>
<!--
Please run the following code snippet and paste the output here:
from sktime import show_versions; show_versions()
-->

```
System:
    python: 3.8.6 (default, Nov 20 2020, 18:29:40)  [Clang 12.0.0 (clang-1200.0.32.27)]
executable: /Users/Pato/env/bin/python3.8
   machine: macOS-10.16-x86_64-i386-64bit
Python dependencies:
          pip: 20.2.4
   setuptools: 50.3.2
      sklearn: 1.0.2
       sktime: 0.11.2
  statsmodels: 0.12.1
        numpy: 1.21.5
        scipy: 1.4.1
       pandas: 1.1.5
   matplotlib: 3.3.3
       joblib: 0.17.0
        numba: None
     pmdarima: 1.8.0
      tsfresh: None
```

</details>



<!-- Thanks for contributing! -->



Discussion:
you need to set a frequency for the time index - what you are creating is a sequence where the time step is a minuscule fraction of a second, you probably do not want that...
So sth like this:

```python
import numpy as np
import pandas as pd
from sktime.forecasting.fbprophet import Prophet

y = pd.DataFrame(np.arange(1, 100), columns=['0'], index = pd.date_range(start='1/1/2018', end='4/9/2018'))

f = Prophet()
f.fit(y)
predictions = f.predict(fh=[1])
```
(not a bug, I think)
Thanks for the quick reply @fkiraly!

> you probably do not want that...

You're right I don't (Prophet may not play nice with such tiny scales), but am trying to call this under some constraints that mean I don't necessarily have access to the actual absolute timestamps, which is why I tried this approach with a relative index first.

I'll consider this solved for now, up to the maintainers to decide whether it's an actual bug or not.
Hm, I think it might be a bug, because this does not work:

```python
import numpy as np
import pandas as pd
from sktime.forecasting.fbprophet import Prophet

y = pd.DataFrame(np.arange(1, 100), columns=['0'])

f = Prophet()
f.fit(y)
predictions = f.predict(fh=[1])
```

I think it should.
`Prophet` works only with `pd.DatetimeIndex`. as descibed in docstring
> Prophet works only with pd.DatetimeIndex. as descibed in docstring

Yes, it is a `prophet` limitation not a `sktime` bug.
yes, but due to the `prophet` limitation, it violates the `sktime` interface contract that strongly implies it should work with any index (although strictly speaking, the `sktime` `Prophet` has a tag that tells you it doesn't, but it is more or less the only important forecaster that has that tag, so users will be unaware, hence the strong expectation that it should work).

A simple workaround would be: if you see an integer index, put in some dummy dates, make the forecast, remove the dummy dates. In such a case, of course, holiday dummies and the like will not make sense and the user should be forewarned by the docstring.

Resolving patch:
diff --git a/sktime/forecasting/base/adapters/_fbprophet.py b/sktime/forecasting/base/adapters/_fbprophet.py
--- a/sktime/forecasting/base/adapters/_fbprophet.py
+++ b/sktime/forecasting/base/adapters/_fbprophet.py
@@ -3,7 +3,7 @@
 # copyright: sktime developers, BSD-3-Clause License (see LICENSE file)
 """Implements adapter for Facebook prophet to be used in sktime framework."""
 
-__author__ = ["mloning", "aiwalter"]
+__author__ = ["mloning", "aiwalter", "fkiraly"]
 __all__ = ["_ProphetAdapter"]
 
 import os
@@ -11,7 +11,6 @@
 import pandas as pd
 
 from sktime.forecasting.base import BaseForecaster
-from sktime.utils.validation.forecasting import check_y_X
 
 
 class _ProphetAdapter(BaseForecaster):
@@ -24,6 +23,15 @@ class _ProphetAdapter(BaseForecaster):
         "handles-missing-data": False,
     }
 
+    def _convert_int_to_date(self, y):
+        """Convert int to date, for use by prophet."""
+        y = y.copy()
+        idx_max = y.index[-1] + 1
+        int_idx = pd.date_range(start="2000-01-01", periods=idx_max, freq="D")
+        int_idx = int_idx[y.index]
+        y.index = int_idx
+        return y
+
     def _fit(self, y, X=None, fh=None):
         """Fit to training data.
 
@@ -42,7 +50,22 @@ def _fit(self, y, X=None, fh=None):
         """
         self._instantiate_model()
         self._check_changepoints()
-        y, X = check_y_X(y, X, enforce_index_type=pd.DatetimeIndex)
+
+        if type(y.index) is pd.PeriodIndex:
+            raise NotImplementedError(
+                "pd.PeriodIndex is not supported for y, use "
+                "pd.DatetimeIndex or integer pd.Index instead."
+            )
+
+        # integer type indices are converted to datetime
+        # since facebook prophet can only deal with dates
+        if y.index.dtype == "int64":
+            self.y_index_was_int_ = True
+            y = self._convert_int_to_date(y)
+        else:
+            self.y_index_was_int_ = False
+        if X is not None and X.index.dtype == "int64":
+            X = self._convert_int_to_date(X)
 
         # We have to bring the data into the required format for fbprophet:
         df = pd.DataFrame({"y": y, "ds": y.index})
@@ -86,6 +109,15 @@ def _fit(self, y, X=None, fh=None):
 
         return self
 
+    def _get_prophet_fh(self):
+        """Get a prophet compatible fh, in datetime, even if fh was int."""
+        fh = self.fh.to_absolute(cutoff=self.cutoff).to_pandas()
+        if not isinstance(fh, pd.DatetimeIndex):
+            max_int = fh[-1] + 1
+            fh_date = pd.date_range(start="2000-01-01", periods=max_int, freq="D")
+            fh = fh_date[fh]
+        return fh
+
     def _predict(self, fh=None, X=None):
         """Forecast time series at future horizon.
 
@@ -108,13 +140,14 @@ def _predict(self, fh=None, X=None):
         Exception
             Error when merging data
         """
-        self._update_X(X, enforce_index_type=pd.DatetimeIndex)
-
-        fh = self.fh.to_absolute(cutoff=self.cutoff).to_pandas()
-        if not isinstance(fh, pd.DatetimeIndex):
-            raise ValueError("absolute `fh` must be represented as a pd.DatetimeIndex")
+        fh = self._get_prophet_fh()
         df = pd.DataFrame({"ds": fh}, index=fh)
 
+        if X is not None and X.index.dtype == "int64":
+            X = X.copy()
+            X = X.loc[self.fh.to_absolute(self.cutoff).to_numpy()]
+            X.index = fh
+
         # Merge X with df (of created future DatetimeIndex values)
         if X is not None:
             X = X.copy()
@@ -133,6 +166,10 @@ def _predict(self, fh=None, X=None):
         y_pred.reset_index(inplace=True)
         y_pred.index = y_pred["ds"].values
         y_pred.drop("ds", axis=1, inplace=True)
+
+        if self.y_index_was_int_:
+            y_pred.index = self.fh.to_absolute(cutoff=self.cutoff)
+
         return y_pred
 
     def _predict_interval(self, fh, X=None, coverage=0.90):
@@ -170,9 +207,12 @@ def _predict_interval(self, fh, X=None, coverage=0.90):
                 Upper/lower interval end forecasts are equivalent to
                 quantile forecasts at alpha = 0.5 - c/2, 0.5 + c/2 for c in coverage.
         """
-        fh = fh.to_absolute(cutoff=self.cutoff).to_pandas()
-        if not isinstance(fh, pd.DatetimeIndex):
-            raise ValueError("absolute `fh` must be represented as a pd.DatetimeIndex")
+        fh = self._get_prophet_fh()
+
+        if X is not None and X.index.dtype == "int64":
+            X = X.copy()
+            X = X.loc[self.fh.to_absolute(self.cutoff).to_numpy()]
+            X.index = fh
 
         # prepare the return DataFrame - empty with correct cols
         var_names = ["Coverage"]
@@ -205,6 +245,9 @@ def _predict_interval(self, fh, X=None, coverage=0.90):
             pred_int[("Coverage", c, "lower")] = out_prophet.min(axis=1)
             pred_int[("Coverage", c, "upper")] = out_prophet.max(axis=1)
 
+        if self.y_index_was_int_:
+            pred_int.index = self.fh.to_absolute(cutoff=self.cutoff)
+
         return pred_int
 
     def get_fitted_params(self):
diff --git a/sktime/forecasting/fbprophet.py b/sktime/forecasting/fbprophet.py
--- a/sktime/forecasting/fbprophet.py
+++ b/sktime/forecasting/fbprophet.py
@@ -17,6 +17,15 @@
 class Prophet(_ProphetAdapter):
     """Prophet forecaster by wrapping Facebook's prophet algorithm [1]_.
 
+    Direct interface to Facebook prophet, using the sktime interface.
+    All hyper-parameters are exposed via the constructor.
+
+    Data can be passed in one of the sktime compatible formats,
+    naming a column `ds` such as in the prophet package is not necessary.
+
+    Integer indices can also be passed, in which case internally a conversion
+    to days since Jan 1, 2000 is carried out before passing to prophet.
+
     Parameters
     ----------
     freq: str, default=None


Test patch: