Repository: statsmodels/statsmodels

Issue:
ENH: Add Prediction Intervals to Holt-Winters class
#### Is your feature request related to a problem? Please describe
I'd like for statsmodels holt-winters (HW) class to calculate prediction intervals (PI).

#### Describe the solution you'd like
Please include a parameter (or method, etc) in the [holt winters class](https://www.statsmodels.org/dev/generated/statsmodels.tsa.holtwinters.ExponentialSmoothing.html) that calculates prediction intervals for the user, including eg upper and lower x / y coordinates for various (and preferably customizable) confidence levels (eg 80%, 95%, etc).

#### Describe alternatives you have considered
It seems there are very few resources available regarding HW PI calculations.  Some only cover certain use cases - eg only additive, but not multiplicative, trend.  So performing the calculations myself in python seemed impractical and unreliable.  The only alternatives I know of are to use the R forecast library, which does perform HW PI calculations.  

#### Additional context
According to one of the more [commonly cited](https://otexts.com/fpp2/ets-forecasting.html) resources on the internet on this topic, HW PI calculations are more complex than other, more common PI calculations.  The same resource (and a couple others I found) mostly point to the [text book](http://www.exponentialsmoothing.net/) (specifically, chapter 6) written by the author of the R library that performs these calculations, to see further details about HW PI calculations.  

####  Additional References
- [The R library](https://www.rdocumentation.org/packages/stats/versions/3.6.1/topics/predict.HoltWinters) that implements holt-winters, including prediction interval calculations.
- Some academic papers that discuss HW PI calculations: [one](https://www.sciencedirect.com/science/article/abs/pii/016920709090103I) and [two](https://core.ac.uk/download/pdf/48532500.pdf)




Discussion:
This feature is the only reason my team hasn't fully migrated our HW forecasting app from R to Python . We've been successful with R for ~15 months, but have had to spend countless hours working around vague errors from R's forecast package. 

The PI feature is the only piece of code preventing us from fully migrating our enterprise forecasting tool from R to Python and benefiting from Python's much friendlier debugging experience.
Thanks for letting us know!

I'll just mention for the pure additive cases, v0.11 has a version of the exponential smoothing models that will allow for prediction intervals, via the model at `sm.tsa.statespace.ExponentialSmoothing`.

Updating the more general model to include them also is something that we'd like to do. In general, I think we can start by adding the versions of them computed via simulation, which is a general method that will work for all models. Then later we could also add the explicit formulas for specific models when they exist, if there is interest in doing so.
@ChadFulton good to know - our app allows for flexibility between additive and multiplicative seasonal patterns. Would both be supported with the changes you just mentioned?
The `sm.tsa.statespace.ExponentialSmoothing` model that is already implemented only supports fully additive models (error, trend, and seasonal).

The simulation approach to prediction intervals - that is not yet implemented - is general to any of the ETS models.
@ChadFulton: The simulation approach would be to use the state space formulation described [here](https://otexts.com/fpp2/ets.html) with random errors as forecast and estimating the interval from multiple runs, correct?

Are you already working on this or have this implemented somewhere? If not, I could try to implement it, and would appreciate some guidance on where and how.
> The simulation approach would be to use the state space formulation described here with random errors as forecast and estimating the interval from multiple runs, correct?

Yes, that's what I was thinking about.

> Are you already working on this or have this implemented somewhere? If not, I could try to implement it, and would appreciate some guidance on where and how.

We don't have an implementation of this right now, but I think it would probably be straightforward. My guess is you'd want to first add a `simulate` method to the `statsmodels.tsa.holtwinters.HoltWintersResults` class, which would simulate future paths of each of the possible models.

Right now, we have the filtering split into separate functions for each of the model cases (see e.g. https://github.com/statsmodels/statsmodels/blob/master/statsmodels/tsa/_exponential_smoothers.pyx#L72 and the other functions in that file), but I think it would be easier to just make one function, similar to what I suggested in #4183 (e.g. https://github.com/statsmodels/statsmodels/pull/4183/files#diff-be2317e3b78a68f56f1108b8fae17c38R34 - this was for the filtering procedure but it would be similar for simulation).

Then once you have `simulate`, prediction intervals just call that method repeatedly and then take quantiles to get the prediction interval.
Is there a reference implementation of the simulation method that I can use for testing? I didn't find it in the linked R library.
I think we can test against the `simulate.ets` function from the `forecast` package. Also, for the linear exponential smoothing models you can test against `sm.tsa.statespace.ExponentialSmoothing`, which allows simulation.
Now that we have the simulations, it should be relatively straightforward to construct the prediction intervals. Should that be a separate function, or an optional return value of `predict`?
I think the best way would be to keep it similar to the state space models, and so to create a `get_prediction` method that returns a results object.  For example:

```python
mod = sm.tsa.SARIMAX(endog)
res = mod.fit()

pred = res.get_prediction(start=mod.nobs, end=mod.nobs + 5)
```

See the `PredictionResults` object in `statespace/mlemodel.py`. For example, one of the methods is `summary_frame`, which allows creating a summary dataframe that looks like:

```
infl        mean   mean_se  mean_ci_lower  mean_ci_upper
2009Q4  1.780000  1.000000      -0.179964       3.739964
2010Q1  0.890000  1.118034      -1.301306       3.081306
2010Q2  0.445000  1.145644      -1.800421       2.690421
2010Q3  0.222500  1.152443      -2.036247       2.481247
2010Q4  0.111250  1.154137      -2.150816       2.373316
2011Q1  0.055625  1.154560      -2.207270       2.318520
```

Resolving patch:
diff --git a/statsmodels/tsa/exponential_smoothing/base.py b/statsmodels/tsa/exponential_smoothing/base.py
--- a/statsmodels/tsa/exponential_smoothing/base.py
+++ b/statsmodels/tsa/exponential_smoothing/base.py
@@ -198,6 +198,13 @@ def from_formula(
 
     def _wrap_data(self, data, start_idx, end_idx, names=None):
         # TODO: check if this is reasonable for statespace
+        # squeezing data: data may be:
+        # - m x n: m dates, n simulations -> squeeze does nothing
+        # - m x 1: m dates, 1 simulation -> squeeze removes last dimension
+        # - 1 x n: don't squeeze, already fine
+        # - 1 x 1: squeeze only second axis
+        if data.ndim > 1 and data.shape[1] == 1:
+            data = np.squeeze(data, axis=1)
         data = np.squeeze(data)
         if self.use_pandas:
             _, _, _, index = self._get_prediction_index(start_idx, end_idx)
@@ -438,8 +445,6 @@ def zvalues(self):
 
     def _get_prediction_start_index(self, anchor):
         """Returns a valid numeric start index for predictions/simulations"""
-        # TODO: once this is the base class for statespace models, use this
-        # method in simulate
         if anchor is None or anchor == "start":
             iloc = 0
         elif anchor == "end":
@@ -448,6 +453,7 @@ def _get_prediction_start_index(self, anchor):
             iloc, _, _ = self.model._get_index_loc(anchor)
             if isinstance(iloc, slice):
                 iloc = iloc.start
+            iloc += 1  # anchor is one before start of prediction/simulation
 
         if iloc < 0:
             iloc = self.nobs + iloc
@@ -588,11 +594,13 @@ def summary(
             ("HQIC", ["%#5.3f" % self.hqic]),
         ]
 
-        if (
-            hasattr(self, "filter_results")
-            and self.filter_results is not None
-            and self.filter_results.filter_concentrated
-        ):
+        if hasattr(self, "filter_results"):
+            if (
+                    self.filter_results is not None
+                    and self.filter_results.filter_concentrated
+            ):
+                top_right.append(("Scale", ["%#5.3f" % self.scale]))
+        else:
             top_right.append(("Scale", ["%#5.3f" % self.scale]))
 
         if hasattr(self, "cov_type"):
diff --git a/statsmodels/tsa/exponential_smoothing/ets.py b/statsmodels/tsa/exponential_smoothing/ets.py
--- a/statsmodels/tsa/exponential_smoothing/ets.py
+++ b/statsmodels/tsa/exponential_smoothing/ets.py
@@ -144,7 +144,7 @@
 import contextlib
 import numpy as np
 import pandas as pd
-from scipy.stats import _distn_infrastructure, rv_continuous, rv_discrete
+from scipy.stats import _distn_infrastructure, rv_continuous, rv_discrete, norm
 
 from statsmodels.base.covtype import descriptions
 import statsmodels.base.wrapper as wrap
@@ -1286,11 +1286,12 @@ def update(params, *args, **kwargs):
 
 class ETSResults(base.StateSpaceMLEResults):
     def __init__(self, model, params, results):
-        super().__init__(model, params)
         yhat, xhat = results
-        self._llf = self.model.loglike(params)
-        self._residuals = self.model._residuals(yhat)
+        self._llf = model.loglike(params)
+        self._residuals = model._residuals(yhat)
         self._fittedvalues = yhat
+        scale = np.mean(self._residuals**2)
+        super().__init__(model, params, scale=scale)
 
         # get model definition
         model_definition_attrs = [
@@ -1417,6 +1418,88 @@ def _get_prediction_params(self, start_idx):
             start_state[4:] = internal_states[start_idx - 1, :]
             return start_state
 
+    def _relative_forecast_variance(self, steps):
+        """
+        References
+        ----------
+        .. [1] Hyndman, R.J., & Athanasopoulos, G. (2019) *Forecasting:
+           principles and practice*, 3rd edition, OTexts: Melbourne,
+           Australia. OTexts.com/fpp3. Accessed on April 19th 2020.
+        """
+        h = steps
+        alpha = self.smoothing_level
+        if self.has_trend:
+            beta = self.smoothing_trend
+        if self.has_seasonal:
+            gamma = self.smoothing_seasonal
+            m = self.seasonal_periods
+            k = np.asarray((h - 1) / m, dtype=int)
+        if self.damped_trend:
+            phi = self.damping_trend
+        model = self.model.short_name
+        if model == "ANN":
+            return 1 + alpha ** 2 * (h - 1)
+        elif model == "AAN":
+            return 1 + (h - 1) * (
+                alpha ** 2 + alpha * beta * h
+                + beta ** 2 * h / 6 * (2 * h - 1)
+            )
+        elif model == "AAdN":
+            return (
+                1
+                + alpha ** 2 * (h - 1)
+                + (
+                    (beta * phi * h) / ((1 - phi) ** 2)
+                    * (2 * alpha * (1 - phi) + beta * phi)
+                )
+                - (
+                    (beta * phi * (1 - phi ** h))
+                    / ((1 - phi) ** 2 * (1 - phi ** 2))
+                    * (
+                        2 * alpha * (1 - phi ** 2)
+                        + beta * phi * (1 + 2 * phi - phi ** h)
+                    )
+                )
+            )
+        elif model == "ANA":
+            return (
+                1 + alpha ** 2 * (h - 1)
+                + gamma * k * (2 * alpha + gamma)
+            )
+        elif model == "AAA":
+            return (
+                1 + (h - 1) * (
+                    alpha ** 2 + alpha * beta * h
+                    + (beta ** 2) / 6 * h * (2 * h - 1)
+                )
+                + gamma * k * (
+                    2 * alpha + gamma + beta * m * (k + 1)
+                )
+            )
+        elif model == "AAdA":
+            return (
+                1
+                + alpha ** 2 * (h - 1)
+                + gamma * k * (2 * alpha + gamma)
+                + (beta * phi * h) / ((1 - phi) ** 2) * (
+                    2 * alpha * (1 - phi) + beta * phi
+                )
+                - (
+                    (beta * phi * (1 - phi ** h))
+                    / ((1 - phi) ** 2 * (1 - phi ** 2))
+                    * (
+                        2 * alpha * (1 - phi ** 2)
+                        + beta * phi * (1 + 2 * phi - phi ** h)
+                    )
+                )
+                + (
+                    (2 * beta * gamma * phi) / ((1 - phi) * (1 - phi ** m))
+                    * (k * (1 - phi ** m) - phi ** m * (1 - phi ** (m * k)))
+                )
+            )
+        else:
+            raise NotImplementedError
+
     def simulate(
         self,
         nsimulations,
@@ -1437,12 +1520,15 @@ def simulate(
             all existing datapoints prior to the `anchor`.  Type depends on the
             index of the given `endog` in the model. Two special cases are the
             strings 'start' and 'end'. `start` refers to beginning the
-            simulation at the first period of the sample, and `end` refers to
+            simulation at the first period of the sample (i.e. using the
+            initial values as simulation anchor), and `end` refers to
             beginning the simulation at the first period after the sample.
             Integer values can run from 0 to `nobs`, or can be negative to
             apply negative indexing. Finally, if a date/time index was provided
             to the model, then this argument can be a date string to parse or a
             datetime type. Default is 'start'.
+            Note: `anchor` corresponds to the observation right before the
+            `start` observation in the `predict` method.
         repetitions : int, optional
             Number of simulated paths to generate. Default is 1 simulated path.
         random_errors : optional
@@ -1621,7 +1707,8 @@ def simulate(
         y = np.empty((nsimulations, repetitions))
 
         # get random error eps
-        sigma = np.sqrt(np.sum(self.resid ** 2) / self.df_resid)
+        sigma = np.sqrt(self.sse / self.df_resid)
+        # sigma = np.sqrt(self.scale)
         if isinstance(random_errors, np.ndarray):
             if random_errors.shape != (nsimulations, repetitions):
                 raise ValueError(
@@ -1742,6 +1829,58 @@ def _forecast(self, steps, anchor):
             steps, anchor=anchor, random_errors=np.zeros((steps, 1))
         )
 
+    def _handle_prediction_index(self, start, dynamic, end, index):
+        if start is None:
+            start = 0
+
+        # Handle start, end, dynamic
+        start, end, out_of_sample, _ = self.model._get_prediction_index(
+            start, end, index
+        )
+        # if end was outside of the sample, it is now the last point in the
+        # sample
+        if start > end:
+            raise ValueError(
+                "Prediction start cannot lie outside of the sample."
+            )
+
+        # Handle `dynamic`
+        if isinstance(dynamic, (bytes, str)):
+            dynamic, _, _ = self.model._get_index_loc(dynamic)
+        elif isinstance(dynamic, bool):
+            if dynamic:
+                dynamic = 0
+            else:
+                dynamic = end + 1 - start
+
+        # start : index of first predicted value
+        # dynamic : offset to first dynamically predicted value
+        #     -> if dynamic == 0, only dynamic simulations
+        if dynamic == 0:
+            start_smooth = None
+            end_smooth = None
+            nsmooth = 0
+            start_dynamic = start
+        else:
+            # dynamic simulations from start + dynamic
+            start_smooth = start
+            end_smooth = min(start + dynamic - 1, end)
+            nsmooth = end_smooth - start_smooth + 1
+            start_dynamic = start + dynamic
+        # anchor for simulations is one before start_dynamic
+        if start_dynamic == 0:
+            anchor_dynamic = "start"
+        else:
+            anchor_dynamic = start_dynamic - 1
+        # end is last point in sample, out_of_sample gives number of
+        # simulations out of sample
+        end_dynamic = end + out_of_sample
+        ndynamic = end_dynamic - start_dynamic + 1
+        return (
+            start, end, start_smooth, end_smooth, anchor_dynamic,
+            start_dynamic, end_dynamic, nsmooth, ndynamic, index
+        )
+
     def predict(self, start=None, end=None, dynamic=False, index=None):
         """
         In-sample prediction and out-of-sample forecasting
@@ -1779,35 +1918,81 @@ def predict(self, start=None, end=None, dynamic=False, index=None):
             forecasts. An (npredict,) array. If original data was a pd.Series
             or DataFrame, a pd.Series is returned.
         """
-        if start is None:
-            start = 0
-
-        # Handle start, end, dynamic
-        start, end, out_of_sample, _ = self.model._get_prediction_index(
-            start, end, index
-        )
-
-        # Handle `dynamic`
-        if isinstance(dynamic, (bytes, str)):
-            dynamic, _, _ = self.model._get_index_loc(dynamic)
 
-        start_smooth = start
-        end_smooth = min(dynamic, end + 1)
-        nsmooth = end_smooth - start_smooth
-        start_forecast = end_smooth
-        end_forecast = end + out_of_sample + 1
-        nforecast = end_forecast - start_forecast
+        (
+            start,
+            end,
+            start_smooth,
+            end_smooth,
+            anchor_dynamic,
+            _,
+            end_dynamic,
+            nsmooth,
+            ndynamic,
+            index
+        ) = self._handle_prediction_index(start, dynamic, end, index)
 
-        y = np.empty(nsmooth + nforecast)
+        y = np.empty(nsmooth + ndynamic)
 
         # In sample nondynamic prediction: smoothing
-        y[0:nsmooth] = self.fittedvalues[start_smooth:end_smooth]
+        if nsmooth > 0:
+            y[0:nsmooth] = self.fittedvalues[start_smooth:end_smooth + 1]
 
         # Out of sample/dynamic prediction: forecast
-        y[nsmooth:] = self._forecast(nforecast, start_forecast)
+        if ndynamic > 0:
+            y[nsmooth:] = self._forecast(ndynamic, anchor_dynamic)
 
         # Wrap data / squeeze where appropriate
-        return self.model._wrap_data(y, start, end_forecast - 1)
+        return self.model._wrap_data(y, start, end_dynamic)
+
+    def get_prediction(self, start=None, end=None, dynamic=False,
+                       index=None, method=None, simulate_repetitions=1000,
+                       **simulate_kwargs):
+        """
+        Calculates mean prediction and prediction intervals.
+
+        Parameters
+        ----------
+        start : int, str, or datetime, optional
+            Zero-indexed observation number at which to start forecasting,
+            i.e., the first forecast is start. Can also be a date string to
+            parse or a datetime type. Default is the the zeroth observation.
+        end : int, str, or datetime, optional
+            Zero-indexed observation number at which to end forecasting, i.e.,
+            the last forecast is end. Can also be a date string to
+            parse or a datetime type. However, if the dates index does not
+            have a fixed frequency, end must be an integer index if you
+            want out of sample prediction. Default is the last observation in
+            the sample.
+        dynamic : bool, int, str, or datetime, optional
+            Integer offset relative to `start` at which to begin dynamic
+            prediction. Can also be an absolute date string to parse or a
+            datetime type (these are not interpreted as offsets).
+            Prior to this observation, true endogenous values will be used for
+            prediction; starting with this observation and continuing through
+            the end of prediction, forecasted endogenous values will be used
+            instead.
+        index : pd.Index, optional
+            Optionally an index to associate the predicted results to. If None,
+            an attempt is made to create an index for the predicted results
+            from the model's index or model's row labels.
+        method : str or None, optional
+            Method to use for calculating prediction intervals. 'exact'
+            (default, if available) or 'simulated'.
+        simulate_repetitions : int, optional
+            Number of simulation repetitions for calculating prediction
+            intervals when ``method='simulated'``. Default is 1000.
+        **simulate_kwargs :
+            Additional arguments passed to the ``simulate`` method.
+
+        Returns
+        -------
+        PredictionResults
+            Predicted mean values and prediction intervals
+        """
+        return PredictionResultsWrapper(PredictionResults(
+            self, start, end, dynamic, index, method, simulate_repetitions,
+            **simulate_kwargs))
 
     def summary(self, alpha=0.05, start=None):
         """
@@ -1878,3 +2063,187 @@ class ETSResultsWrapper(wrap.ResultsWrapper):
 
 
 wrap.populate_wrapper(ETSResultsWrapper, ETSResults)
+
+
+class PredictionResults:
+    """
+    ETS prediction results, containing mean prediction and prediction
+    intervals.
+
+    Parameters
+    ----------
+    results : ETSResults
+    start : int, str, or datetime, optional
+        Zero-indexed observation number at which to start forecasting,
+        i.e., the first forecast is start. Can also be a date string to
+        parse or a datetime type. Default is the the zeroth observation.
+    end : int, str, or datetime, optional
+        Zero-indexed observation number at which to end forecasting, i.e.,
+        the last forecast is end. Can also be a date string to
+        parse or a datetime type. However, if the dates index does not
+        have a fixed frequency, end must be an integer index if you
+        want out of sample prediction. Default is the last observation in
+        the sample.
+    dynamic : bool, int, str, or datetime, optional
+        Integer offset relative to `start` at which to begin dynamic
+        prediction. Can also be an absolute date string to parse or a
+        datetime type (these are not interpreted as offsets).
+        Prior to this observation, true endogenous values will be used for
+        prediction; starting with this observation and continuing through
+        the end of prediction, forecasted endogenous values will be used
+        instead.
+    index : pd.Index, optional
+        Optionally an index to associate the predicted results to. If None,
+        an attempt is made to create an index for the predicted results
+        from the model's index or model's row labels.
+    method : str or None, optional
+        Method to use for calculating prediction intervals. 'exact' (default,
+        if available) or 'simulated'.
+    simulate_repetitions : int, optional
+        Number of simulation repetitions for calculating prediction intervals.
+        Default is 1000.
+    **simulate_kwargs :
+        Additional arguments passed to the ``simulate`` method.
+    """
+    def __init__(self, results, start=None, end=None, dynamic=False,
+                 index=None, method=None, simulate_repetitions=1000,
+                 **simulate_kwargs):
+        self.use_pandas = results.model.use_pandas
+
+        if method is None:
+            exact_available = [
+                "ANN", "AAN", "AAdN", "ANA", "AAA", "AAdA"
+            ]
+            if results.model.short_name in exact_available:
+                method = 'exact'
+            else:
+                method = 'simulated'
+        self.method = method
+
+        (
+            start,
+            end,
+            start_smooth,
+            _,
+            anchor_dynamic,
+            start_dynamic,
+            end_dynamic,
+            nsmooth,
+            ndynamic,
+            index
+        ) = results._handle_prediction_index(start, dynamic, end, index)
+
+        self.predicted_mean = results.predict(
+            start=start, end=end_dynamic, dynamic=dynamic, index=index
+        )
+        self.row_labels = self.predicted_mean.index
+        self.endog = np.empty(nsmooth + ndynamic) * np.nan
+        self.endog[0:(end - start + 1)] = results.data.endog[start:(end + 1)]
+        self.model = Bunch(data=results.model.data.__class__(
+            endog=self.endog, predict_dates=self.row_labels
+        ))
+
+        if self.method == "simulated":
+
+            sim_results = []
+            # first, perform "non-dynamic" simulations, i.e. simulations of
+            # only one step, based on the previous step
+            if nsmooth > 1:
+                if start_smooth == 0:
+                    anchor = "start"
+                else:
+                    anchor = start_smooth - 1
+                for i in range(nsmooth):
+                    sim_results.append(results.simulate(
+                        1, anchor=anchor, repetitions=simulate_repetitions,
+                        **simulate_kwargs
+                    ))
+                    # anchor
+                    anchor = start_smooth + i
+            sim_results.append(results.simulate(
+                ndynamic, anchor=anchor_dynamic,
+                repetitions=simulate_repetitions, **simulate_kwargs
+            ))
+            self.simulation_results = np.concatenate(sim_results, axis=0)
+            # if self.use_pandas:
+            #     self.simulation_results = pd.DataFrame(
+            #         self.simulation_results, index=self.row_labels,
+            #         columns=sim_results[0].columns
+            #     )
+        else:  # method == 'exact'
+            steps = np.ones(ndynamic + nsmooth)
+            if ndynamic > 0:
+                steps[(start_dynamic - start):] = range(1, ndynamic + 1)
+            self.forecast_variance = (
+                results.mse * results._relative_forecast_variance(steps)
+            )
+
+    def pred_int(self, alpha=0.05):
+        """
+        Calculates prediction intervals by performing multiple simulations.
+
+        Parameters
+        ----------
+        alpha : float, optional
+            The significance level for the prediction interval. Default is
+            0.05, that is, a 95% prediction interval.
+        """
+
+        if self.method == "simulated":
+            simulated_upper_pi = np.quantile(
+                self.simulation_results, 1 - alpha / 2, axis=1
+            )
+            simulated_lower_pi = np.quantile(
+                self.simulation_results, alpha / 2, axis=1
+            )
+            pred_int = np.vstack(
+                (simulated_lower_pi, simulated_upper_pi)
+            ).T
+        else:
+            q = norm.ppf(1 - alpha / 2)
+            half_interval_size = q * np.sqrt(self.forecast_variance)
+            pred_int = np.vstack(
+                (
+                    self.predicted_mean - half_interval_size,
+                    self.predicted_mean + half_interval_size
+                )
+            ).T
+
+        if self.use_pandas:
+            pred_int = pd.DataFrame(
+                pred_int, index=self.row_labels
+            )
+            names = [
+                f"lower PI (alpha={alpha:f})",
+                f"upper PI (alpha={alpha:f})"
+            ]
+            pred_int.columns = names
+        return pred_int
+
+    def summary_frame(self, endog=0, alpha=0.05):
+        pred_int = np.asarray(self.pred_int(alpha=alpha))
+        to_include = {}
+        to_include["mean"] = self.predicted_mean
+        if self.method == "simulated":
+            to_include["mean_numerical"] = np.mean(
+                self.simulation_results, axis=1
+            )
+        to_include["pi_lower"] = pred_int[:, 0]
+        to_include['pi_upper'] = pred_int[:, 1]
+
+        res = pd.DataFrame(to_include, index=self.row_labels,
+                           columns=to_include.keys())
+        return res
+
+
+class PredictionResultsWrapper(wrap.ResultsWrapper):
+    _attrs = {
+        'predicted_mean': 'dates',
+        'simulation_results': 'dates',
+        'endog' : 'dates',
+    }
+    _wrap_attrs = wrap.union_dicts(_attrs)
+
+    _methods = {}
+    _wrap_methods = wrap.union_dicts(_methods)
+wrap.populate_wrapper(PredictionResultsWrapper, PredictionResults)  # noqa:E305


Test patch:
diff --git a/statsmodels/tsa/tests/test_exponential_smoothing.py b/statsmodels/tsa/tests/test_exponential_smoothing.py
--- a/statsmodels/tsa/tests/test_exponential_smoothing.py
+++ b/statsmodels/tsa/tests/test_exponential_smoothing.py
@@ -12,7 +12,8 @@
 import scipy.stats
 
 from statsmodels.tsa.exponential_smoothing.ets import ETSModel
-from statsmodels.tsa.holtwinters import ExponentialSmoothing
+import statsmodels.tsa.holtwinters as holtwinters
+import statsmodels.tsa.statespace.exponential_smoothing as statespace
 
 # This contains tests for the exponential smoothing implementation in
 # tsa/exponential_smoothing/ets.py.
@@ -119,6 +120,11 @@ def austourists_model(austourists):
     )
 
 
+@pytest.fixture
+def austourists_model_fit(austourists_model):
+    return austourists_model.fit(disp=False)
+
+
 @pytest.fixture
 def oildata_model(oildata):
     return ETSModel(oildata, error="add", trend="add", damped_trend=True,)
@@ -544,24 +550,20 @@ def test_seasonal_periods(austourists):
         pass
 
 
-def test_simulate_keywords(austourists_model):
+def test_simulate_keywords(austourists_model_fit):
     """
     check whether all keywords are accepted and work without throwing errors.
     """
-    fit = austourists_model.fit(disp=False)
+    fit = austourists_model_fit
 
     # test anchor
-    assert_almost_equal(
-        fit.simulate(4, anchor=0, random_state=0).values,
-        fit.simulate(4, anchor="start", random_state=0).values,
-    )
     assert_almost_equal(
         fit.simulate(4, anchor=-1, random_state=0).values,
         fit.simulate(4, anchor="2015-12-31", random_state=0).values,
     )
     assert_almost_equal(
         fit.simulate(4, anchor="end", random_state=0).values,
-        fit.simulate(4, anchor="2016-03-31", random_state=0).values,
+        fit.simulate(4, anchor="2015-12-31", random_state=0).values,
     )
 
     # test different random error options
@@ -579,6 +581,27 @@ def test_simulate_keywords(austourists_model):
     assert np.all(res == res2)
 
 
+def test_predict_ranges(austourists_model_fit):
+    # in total 68 observations
+    fit = austourists_model_fit
+
+    # first prediction is 0, last is 10 -> 11 predictions
+    pred = fit.predict(start=0, end=10)
+    assert len(pred) == 11
+
+    pred = fit.predict(start=10, end=20)
+    assert len(pred) == 11
+
+    pred = fit.predict(start=10, dynamic=10, end=30)
+    assert len(pred) == 21
+
+    # try boolean dynamic
+    pred = fit.predict(start=0, dynamic=True, end=70)
+    assert len(pred) == 71
+    pred = fit.predict(start=0, dynamic=True, end=70)
+    assert len(pred) == 71
+
+
 def test_summary(austourists_model):
     # just try to run summary to see if it works
     fit = austourists_model.fit(disp=False)
@@ -594,22 +617,260 @@ def test_summary(austourists_model):
     fit.summary()
 
 
-def test_score(austourists_model):
-    fit = austourists_model.fit(disp=False)
-    score_cs = austourists_model.score(fit.params)
-    score_fd = austourists_model.score(
-        fit.params, approx_complex_step=False, approx_centered=True,
+def test_score(austourists_model_fit):
+    score_cs = austourists_model_fit.model.score(austourists_model_fit.params)
+    score_fd = austourists_model_fit.model.score(
+        austourists_model_fit.params, approx_complex_step=False,
+        approx_centered=True,
     )
     assert_almost_equal(score_cs, score_fd, 4)
 
 
-def test_hessian(austourists_model):
+def test_hessian(austourists_model_fit):
     # The hessian approximations are not very consistent, but the test makes
     # sure they run
-    fit = austourists_model.fit(disp=False)
-    austourists_model.hessian(fit.params)
-    austourists_model.hessian(
-        fit.params, approx_complex_step=False, approx_centered=True,
+    austourists_model_fit.model.hessian(austourists_model_fit.params)
+    austourists_model_fit.model.hessian(
+        austourists_model_fit.params, approx_complex_step=False,
+        approx_centered=True,
+    )
+
+
+def test_prediction_results(austourists_model_fit):
+    # simple test case starting at 0
+    pred = austourists_model_fit.get_prediction(
+        start=0, dynamic=30, end=40,
+    )
+    summary = pred.summary_frame()
+    assert len(summary['mean'].values) == 41
+    assert np.all(~np.isnan(summary['mean']))
+
+    # simple test case starting at not 0
+    pred = austourists_model_fit.get_prediction(
+        start=10, dynamic=30, end=40
+    )
+    summary = pred.summary_frame()
+    assert len(summary['mean'].values) == 31
+    assert np.all(~np.isnan(summary['mean']))
+
+    # long out of sample prediction
+    pred = austourists_model_fit.get_prediction(
+        start=0, dynamic=30, end=80
+    )
+    summary = pred.summary_frame()
+    assert len(summary['mean'].values) == 81
+    assert np.all(~np.isnan(summary['mean']))
+
+    # only out of sample prediction
+    pred = austourists_model_fit.get_prediction(
+        start=67, end=80
+    )
+    summary = pred.summary_frame()
+    assert len(summary['mean'].values) == 14
+    assert np.all(~np.isnan(summary['mean']))
+
+
+@pytest.fixture
+def statespace_comparison(austourists):
+    ets_model = ETSModel(
+        austourists,
+        seasonal_periods=4,
+        error="add",
+        trend="add",
+        seasonal="add",
+        damped_trend=True,
+    )
+    ets_results = ets_model.fit(disp=False)
+
+    statespace_model = statespace.ExponentialSmoothing(
+        austourists,
+        trend=True,
+        damped_trend=True,
+        seasonal=4,
+        initialization_method="known",
+        initial_level=ets_results.initial_level,
+        initial_trend=ets_results.initial_trend,
+        initial_seasonal=ets_results.initial_seasonal,
+    )
+    with statespace_model.fix_params(
+        {
+            "smoothing_level": ets_results.smoothing_level,
+            "smoothing_trend": ets_results.smoothing_trend,
+            "smoothing_seasonal": ets_results.smoothing_seasonal,
+            "damping_trend": ets_results.damping_trend,
+        }
+    ):
+        statespace_results = statespace_model.fit()
+
+    return ets_results, statespace_results
+
+
+def test_results_vs_statespace(statespace_comparison):
+    ets_results, statespace_results = statespace_comparison
+
+    assert_almost_equal(
+        ets_results.llf, statespace_results.llf
+    )
+    assert_almost_equal(
+        ets_results.scale, statespace_results.scale
+    )
+    assert_almost_equal(
+        ets_results.fittedvalues.values,
+        statespace_results.fittedvalues.values
+    )
+
+
+def test_prediction_results_vs_statespace(statespace_comparison):
+    ets_results, statespace_results = statespace_comparison
+
+    # comparison of two predictions
+    ets_pred = ets_results.get_prediction(
+        start=10, dynamic=10, end=40
+    )
+    statespace_pred = statespace_results.get_prediction(
+        start=10, dynamic=10, end=40
+    )
+
+    statespace_summary = statespace_pred.summary_frame()
+    ets_summary = ets_pred.summary_frame()
+
+    # import matplotlib.pyplot as plt
+    # plt.switch_backend('TkAgg')
+    # plt.plot(ets_summary["mean"] - statespace_summary["mean"])
+    # plt.grid()
+    # plt.show()
+
+    assert_almost_equal(
+        ets_summary["mean"].values[:-10],
+        statespace_summary["mean"].values[:-10],
+    )
+
+    assert_almost_equal(
+        ets_summary["mean"].values[-10:],
+        statespace_summary["mean"].values[-10:],
+        4
+    )
+
+    # comparison of dynamic prediction at end of sample -> this works
+    ets_pred = ets_results.get_prediction(
+        start=60, end=80,
+    )
+    statespace_pred = statespace_results.get_prediction(
+        start=60, end=80
+    )
+    statespace_summary = statespace_pred.summary_frame()
+    ets_summary = ets_pred.summary_frame()
+
+    assert_almost_equal(
+        ets_summary["mean"].values,
+        statespace_summary["mean"].values,
+        4
+    )
+
+
+@pytest.mark.skip
+def test_prediction_results_slow_AAN(oildata):
+    # slow test with high number of simulation repetitions for comparison
+    # Note: runs succesfull with specified tolerance
+    fit = ETSModel(
+        oildata, error="add", trend="add"
+    ).fit(disp=False)
+
+    pred_exact = fit.get_prediction(
+        start=40, end=55
+    )
+    summary_exact = pred_exact.summary_frame()
+
+    pred_sim = fit.get_prediction(
+        start=40, end=55, simulate_repetitions=int(1e6),
+        random_state=11, method="simulated"
+    )
+    summary_sim = pred_sim.summary_frame()
+    # check if mean converges to expected mean
+    assert_allclose(
+        summary_sim["mean"].values,
+        summary_sim["mean_numerical"].values,
+        rtol=1e-3, atol=1e-3
+    )
+
+    import matplotlib.pyplot as plt
+    plt.switch_backend('TkAgg')
+    for i in range(1000):
+        plt.plot(pred_sim._results.simulation_results.iloc[:, i],
+                 color='grey', alpha=0.1)
+    plt.plot(oildata[40:], '-', label='data')
+    plt.plot(summary_exact["mean"], '--', label='mean')
+    plt.plot(summary_sim["pi_lower"], ':', label='sim lower')
+    plt.plot(summary_exact["pi_lower"], '.-', label='exact lower')
+    plt.plot(summary_sim["pi_upper"], ':', label='sim upper')
+    plt.plot(summary_exact["pi_upper"], '.-', label='exact upper')
+    # plt.legend()
+    plt.show()
+
+    # check if prediction intervals are equal
+    assert_allclose(
+        summary_sim["pi_lower"].values,
+        summary_exact["pi_lower"].values,
+        rtol=1e-4, atol=1e-4
+    )
+
+    assert_allclose(
+        summary_sim["pi_upper"].values,
+        summary_exact["pi_upper"].values,
+        rtol=1e-4, atol=1e-4
+    )
+
+
+@pytest.mark.skip
+def test_prediction_results_slow_AAdA(austourists):
+    # slow test with high number of simulation repetitions for comparison
+    # Note: succesfull with specified tolerance
+    fit = ETSModel(
+        austourists, error="add", trend="add", seasonal="add",
+        damped_trend=True,
+        seasonal_periods=4
+    ).fit(disp=False)
+    pred_exact = fit.get_prediction(
+        start=60, end=75
+    )
+    summary_exact = pred_exact.summary_frame()
+
+    pred_sim = fit.get_prediction(
+        start=60, end=75, simulate_repetitions=int(1e6),
+        random_state=11, method="simulated"
+    )
+    summary_sim = pred_sim.summary_frame()
+    # check if mean converges to expected mean
+    assert_allclose(
+        summary_sim["mean"].values,
+        summary_sim["mean_numerical"].values,
+        rtol=1e-3, atol=1e-3
+    )
+
+    import matplotlib.pyplot as plt
+    plt.switch_backend('TkAgg')
+    for i in range(1000):
+        plt.plot(pred_sim._results.simulation_results.iloc[:, i],
+                 color='grey', alpha=0.1)
+    plt.plot(fit.endog[60:], '-', label='data')
+    plt.plot(summary_exact["mean"], '--', label='mean')
+    plt.plot(summary_sim["pi_lower"], ':', label='sim lower')
+    plt.plot(summary_exact["pi_lower"], '.-', label='exact lower')
+    plt.plot(summary_sim["pi_upper"], ':', label='sim upper')
+    plt.plot(summary_exact["pi_upper"], '.-', label='exact upper')
+    plt.show()
+
+    # check if prediction intervals are equal
+    assert_allclose(
+        summary_sim["pi_lower"].values,
+        summary_exact["pi_lower"].values,
+        rtol=2e-2, atol=1e-4
+    )
+
+    assert_allclose(
+        summary_sim["pi_upper"].values,
+        summary_exact["pi_upper"].values,
+        rtol=2e-2, atol=1e-4
     )
 
 
@@ -621,21 +882,48 @@ def test_convergence_simple():
     for i in range(1, e.shape[0]):
         y[i] = y[i - 1] - 0.2 * e[i - 1] + e[i]
     y = y[200:]
-    res = ExponentialSmoothing(y, initialization_method="estimated").fit()
+    res = holtwinters.ExponentialSmoothing(y).fit()
     ets_res = ETSModel(y).fit()
 
     # the smoothing level should be very similar, the initial state might be
     # different as it doesn't influence the final result too much
-    assert_almost_equal(
+    assert_allclose(
         res.params['smoothing_level'],
         ets_res.smoothing_level,
-        3
+        rtol=1e-4, atol=1e-4
     )
 
     # the first few values are influenced by differences in initial state, so
     # we don't test them here
-    assert_almost_equal(
+    assert_allclose(
         res.fittedvalues[10:],
         ets_res.fittedvalues[10:],
-        3
+        rtol=1e-4, atol=1e-4
     )
+
+
+def test_exact_prediction_intervals(austourists_model_fit):
+
+    fit = austourists_model_fit._results
+
+    class DummyModel:
+        def __init__(self, short_name):
+            self.short_name = short_name
+
+    # compare AAdN with AAN
+    fit.damping_trend = 1 - 1e-3
+    fit.model = DummyModel("AAdN")
+    steps = 5
+    s_AAdN = fit._relative_forecast_variance(steps)
+    fit.model = DummyModel("AAN")
+    s_AAN = fit._relative_forecast_variance(steps)
+    assert_almost_equal(s_AAdN, s_AAN, 2)
+
+    # compare AAdA with AAA
+    fit.damping_trend = 1 - 1e-3
+    fit.model = DummyModel("AAdA")
+    steps = 5
+    s_AAdA = fit._relative_forecast_variance(steps)
+    fit.model = DummyModel("AAA")
+    s_AAA = fit._relative_forecast_variance(steps)
+    assert_almost_equal(s_AAdA, s_AAA, 3)