Repository: statsmodels/statsmodels

Issue:
AutoReg predict start point bug
#### Describe the bug
AugoReg predict does not work in some cases. Seems to be related to start point. Works if I move start point to end of endog sample (i.e. in the example below, `start=100`) but not if the start is further ahead. 

#### Code Sample, a copy-pastable example if possible
```python
from statsmodels.tsa.ar_model import AutoReg
import pandas as pd
import numpy as np

y_train = pd.Series(np.random.normal(size=100))
m = AutoReg(y_train, lags=10, trend="c")
mf = m.fit()
mf.predict(start=101, end=110)
```

## Error traceback
> Traceback (most recent call last):
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3331, in run_code
>     exec(code_obj, self.user_global_ns, self.user_ns)
>   File "<ipython-input-2-98ce42ecfe13>", line 1, in <module>
>     runfile('/Users/mloning/Documents/Research/toolboxes/sktime/sktime/notes/playground.py', wdir='/Users/mloning/Documents/Research/toolboxes/sktime/sktime/notes')
>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
>     pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
>     exec(compile(contents+"\n", file, 'exec'), glob, loc)
>   File "/Users/mloning/Documents/Research/toolboxes/sktime/sktime/notes/playground.py", line 8, in <module>
>     mf.predict(start=101, end=110)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels/base/wrapper.py", line 115, in wrapper
>     obj = data.wrap_output(func(results, *args, **kwargs), how)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels/tsa/ar_model.py", line 1981, in predict
>     exog_oos=exog_oos)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels/tsa/ar_model.py", line 662, in predict
>     exog, exog_oos)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels/tsa/ar_model.py", line 580, in _static_predict
>     return self._wrap_prediction(prediction, start, end + 1 + num_oos)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels/tsa/ar_model.py", line 481, in _wrap_prediction
>     return pd.Series(prediction, index=index)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/pandas/core/series.py", line 299, in __init__
>     "index implies {ind}".format(val=len(data), ind=len(index))
> ValueError: Length of passed values is 11, index implies 10

</details>


If the issue has not been resolved, please file it in the issue tracker.

#### Expected Output
Return predictions for interval [start, end].

#### Output of ``import statsmodels.api as sm; sm.show_versions()``

<details>

[paste the output of ``import statsmodels.api as sm; sm.show_versions()`` here below this line]
INSTALLED VERSIONS
------------------
Python: 3.7.3.final.0
OS: Darwin 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64
byteorder: little
LC_ALL: None
LANG: None
statsmodels
===========
Installed: 0.11.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels)
Required Dependencies
=====================
cython: 0.29.13 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/Cython)
numpy: 1.18.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/numpy)
scipy: 1.4.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/scipy)
pandas: 0.25.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/pandas)
    dateutil: 2.8.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/dateutil)
patsy: 0.5.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/patsy)
Optional Dependencies
=====================
matplotlib: 3.1.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/matplotlib)
    backend: module://backend_interagg 
cvxopt: Not installed
joblib: 0.14.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/joblib)
Developer Tools
================
IPython: 7.12.0 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/IPython)
    jinja2: 2.11.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/jinja2)
sphinx: 2.2.0 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/sphinx)
    pygments: 2.5.2 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/pygments)
pytest: 5.3.5 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/pytest)
virtualenv: Not installed

</details>

AutoReg predict start point bug
#### Describe the bug
AugoReg predict does not work in some cases. Seems to be related to start point. Works if I move start point to end of endog sample (i.e. in the example below, `start=100`) but not if the start is further ahead. 

#### Code Sample, a copy-pastable example if possible
```python
from statsmodels.tsa.ar_model import AutoReg
import pandas as pd
import numpy as np

y_train = pd.Series(np.random.normal(size=100))
m = AutoReg(y_train, lags=10, trend="c")
mf = m.fit()
mf.predict(start=101, end=110)
```

## Error traceback
> Traceback (most recent call last):
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3331, in run_code
>     exec(code_obj, self.user_global_ns, self.user_ns)
>   File "<ipython-input-2-98ce42ecfe13>", line 1, in <module>
>     runfile('/Users/mloning/Documents/Research/toolboxes/sktime/sktime/notes/playground.py', wdir='/Users/mloning/Documents/Research/toolboxes/sktime/sktime/notes')
>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
>     pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
>   File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
>     exec(compile(contents+"\n", file, 'exec'), glob, loc)
>   File "/Users/mloning/Documents/Research/toolboxes/sktime/sktime/notes/playground.py", line 8, in <module>
>     mf.predict(start=101, end=110)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels/base/wrapper.py", line 115, in wrapper
>     obj = data.wrap_output(func(results, *args, **kwargs), how)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels/tsa/ar_model.py", line 1981, in predict
>     exog_oos=exog_oos)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels/tsa/ar_model.py", line 662, in predict
>     exog, exog_oos)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels/tsa/ar_model.py", line 580, in _static_predict
>     return self._wrap_prediction(prediction, start, end + 1 + num_oos)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels/tsa/ar_model.py", line 481, in _wrap_prediction
>     return pd.Series(prediction, index=index)
>   File "/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/pandas/core/series.py", line 299, in __init__
>     "index implies {ind}".format(val=len(data), ind=len(index))
> ValueError: Length of passed values is 11, index implies 10

</details>


If the issue has not been resolved, please file it in the issue tracker.

#### Expected Output
Return predictions for interval [start, end].

#### Output of ``import statsmodels.api as sm; sm.show_versions()``

<details>

[paste the output of ``import statsmodels.api as sm; sm.show_versions()`` here below this line]
INSTALLED VERSIONS
------------------
Python: 3.7.3.final.0
OS: Darwin 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan  9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64
byteorder: little
LC_ALL: None
LANG: None
statsmodels
===========
Installed: 0.11.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/statsmodels)
Required Dependencies
=====================
cython: 0.29.13 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/Cython)
numpy: 1.18.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/numpy)
scipy: 1.4.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/scipy)
pandas: 0.25.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/pandas)
    dateutil: 2.8.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/dateutil)
patsy: 0.5.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/patsy)
Optional Dependencies
=====================
matplotlib: 3.1.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/matplotlib)
    backend: module://backend_interagg 
cvxopt: Not installed
joblib: 0.14.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/joblib)
Developer Tools
================
IPython: 7.12.0 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/IPython)
    jinja2: 2.11.1 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/jinja2)
sphinx: 2.2.0 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/sphinx)
    pygments: 2.5.2 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/pygments)
pytest: 5.3.5 (/Users/mloning/.conda/envs/sktime-dev/lib/python3.7/site-packages/pytest)
virtualenv: Not installed

</details>



Discussion:

Resolving patch:
diff --git a/statsmodels/tsa/ar_model.py b/statsmodels/tsa/ar_model.py
--- a/statsmodels/tsa/ar_model.py
+++ b/statsmodels/tsa/ar_model.py
@@ -468,8 +468,9 @@ def _setup_oos_forecast(self, add_forecasts, exog_oos):
         return x
 
     def _wrap_prediction(self, prediction, start, end):
+        n_values = end - start
         if not isinstance(self.data.orig_endog, (pd.Series, pd.DataFrame)):
-            return prediction
+            return prediction[-n_values:]
         index = self.data.orig_endog.index
         if end > self.endog.shape[0]:
             freq = getattr(index, 'freq', None)
@@ -478,6 +479,7 @@ def _wrap_prediction(self, prediction, start, end):
             else:
                 index = pd.RangeIndex(end)
         index = index[start:end]
+        prediction = prediction[-n_values:]
         return pd.Series(prediction, index=index)
 
     def _dynamic_predict(self, params, start, end, dynamic, num_oos, exog,


Test patch:
diff --git a/statsmodels/tsa/tests/test_ar.py b/statsmodels/tsa/tests/test_ar.py
--- a/statsmodels/tsa/tests/test_ar.py
+++ b/statsmodels/tsa/tests/test_ar.py
@@ -1203,3 +1203,13 @@ def test_forecast_start_end_equiv(dynamic):
     dates = pd.date_range("1-1-1950", periods=1021, freq="M")
     pred_dates = res.predict(dates[1000], dates[1020], dynamic=dynamic)
     assert_series_equal(pred_int, pred_dates)
+
+
+@pytest.mark.parametrize("start", [21, 25])
+def test_autoreg_start(start):
+    y_train = pd.Series(np.random.normal(size=20))
+    m = AutoReg(y_train, lags=2)
+    mf = m.fit()
+    end = start + 5
+    pred = mf.predict(start=start, end=end)
+    assert pred.shape[0] == end - start + 1