Repository: statsmodels/statsmodels

Issue:
VARMAXResults.extend() causes ValueError: array must not contain infs or NaNs
#### Describe the bug

Extending a VARMAX model with new observations throws a ValueError even though the analogous univariate extend with same syntax works for SARIMAX.

#### Code Sample, a copy-pastable example if possible


```python
from statsmodels.tsa.statespace.varmax import VARMAX
import numpy as np

np.random.seed(1)
y_hist = 100*np.random.rand(50,2)

model = VARMAX(endog= y_hist,order=(2,0)).fit()
print("VARMAX model summary")
print(model.summary())
next_y_hat = model.forecast()
print("\nPredicted next value")
print(next_y_hat)

# simulate next observed value
next_y = next_y_hat

# extend model
model = model.extend(endog = next_y) # ValueError: array must not contain infs or NaNs
```
<details>

The analogous use of SARIMAXResults.extend() works fine:

```python
from statsmodels.tsa.statespace.sarimax import SARIMAX

uni_model = SARIMAX(endog=y_hist[:,1],order=(2,0,0)).fit()
print("SARIMAX model summary")
print(uni_model.summary())
next_y_hat_uni = uni_model.forecast()
print("\nPredicted next value")
print(next_y_hat_uni)

# simulate next observed value
next_y_uni = next_y_hat_uni

# extend model
uni_model = uni_model.extend(endog = next_y_uni)
```

</details>

If the issue has not been resolved, please file it in the issue tracker.

#### Expected Output

`model.extend(endog = next_y)` should return a VARMAXResultsWrapper instead of ValueError

#### Output of ``import statsmodels.api as sm; sm.show_versions()``

<details>
INSTALLED VERSIONS
------------------
Python: 3.7.7.final.0
OS: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Mon Apr 27 20:09:39 PDT 2020; root:xnu-4903.278.35~1/RELEASE_X86_64 x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

statsmodels
===========

Installed: 0.11.1 (/usr/local/lib/python3.7/site-packages/statsmodels)

Required Dependencies
=====================

cython: 0.29.17 (/usr/local/lib/python3.7/site-packages/Cython)
numpy: 1.16.3 (/usr/local/lib/python3.7/site-packages/numpy)
scipy: 1.3.1 (/usr/local/lib/python3.7/site-packages/scipy)
pandas: 1.0.1 (/usr/local/lib/python3.7/site-packages/pandas)
    dateutil: 2.8.1 (/usr/local/lib/python3.7/site-packages/dateutil)
patsy: 0.5.1 (/usr/local/lib/python3.7/site-packages/patsy)

Optional Dependencies
=====================

matplotlib: 3.1.1 (/usr/local/lib/python3.7/site-packages/matplotlib)
    backend: module://ipykernel.pylab.backend_inline 
cvxopt: Not installed
joblib: 0.13.2 (/usr/local/lib/python3.7/site-packages/joblib)

Developer Tools
================

IPython: 7.13.0 (/usr/local/lib/python3.7/site-packages/IPython)
    jinja2: 2.11.1 (/usr/local/lib/python3.7/site-packages/jinja2)
sphinx: 2.2.0 (/usr/local/lib/python3.7/site-packages/sphinx)
    pygments: 2.6.1 (/usr/local/lib/python3.7/site-packages/pygments)
pytest: Not installed
virtualenv: Not installedINSTALLED VERSIONS
------------------
Python: 3.7.7.final.0
OS: Darwin 18.7.0 Darwin Kernel Version 18.7.0: Mon Apr 27 20:09:39 PDT 2020; root:xnu-4903.278.35~1/RELEASE_X86_64 x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

statsmodels
===========

Installed: 0.11.1 (/usr/local/lib/python3.7/site-packages/statsmodels)

Required Dependencies
=====================

cython: 0.29.17 (/usr/local/lib/python3.7/site-packages/Cython)
numpy: 1.16.3 (/usr/local/lib/python3.7/site-packages/numpy)
scipy: 1.3.1 (/usr/local/lib/python3.7/site-packages/scipy)
pandas: 1.0.1 (/usr/local/lib/python3.7/site-packages/pandas)
    dateutil: 2.8.1 (/usr/local/lib/python3.7/site-packages/dateutil)
patsy: 0.5.1 (/usr/local/lib/python3.7/site-packages/patsy)

Optional Dependencies
=====================

matplotlib: 3.1.1 (/usr/local/lib/python3.7/site-packages/matplotlib)
    backend: module://ipykernel.pylab.backend_inline 
cvxopt: Not installed
joblib: 0.13.2 (/usr/local/lib/python3.7/site-packages/joblib)

Developer Tools
================

IPython: 7.13.0 (/usr/local/lib/python3.7/site-packages/IPython)
    jinja2: 2.11.1 (/usr/local/lib/python3.7/site-packages/jinja2)
sphinx: 2.2.0 (/usr/local/lib/python3.7/site-packages/sphinx)
    pygments: 2.6.1 (/usr/local/lib/python3.7/site-packages/pygments)
pytest: Not installed
virtualenv: Not installed

</details>



Discussion:
I also posted this as a stackoverflow question:

https://stackoverflow.com/questions/63021311/varmax-results-extend-causes-valueerror-array-must-not-contain-infs-or-nans
Thanks for posting this issue! It is a bug in handling `extend` with a trend. It's an easy issue to fix, but until we get a new release out there is a workaround (which admittedly is a little annoying). Instead of including a trend in the model with the `trend` argument, you can manually include it as part of `exog`. These are identical results, but the `exog` approach is a little tedious.

```python
from statsmodels.tsa.statespace.varmax import VARMAX
import numpy as np

np.random.seed(1)
y_hist = 100*np.random.rand(50,2)

model = VARMAX(endog= y_hist,order=(2,0), trend='n', exog=np.ones(y_hist.shape[0])).fit()
print("VARMAX model summary")
print(model.summary())
next_y_hat = model.forecast(exog=[1])
print("\nPredicted next value")
print(next_y_hat)

# simulate next observed value
next_y = next_y_hat

# extend model
model = model.extend(endog = next_y, exog=[1]) # ValueError: array must not contain infs or NaNs
```
Thanks for the fast and helpful response!

I didn't even realize I was asking for a trend component.  IIUC, With [SARIMAX](https://www.statsmodels.org/devel/generated/statsmodels.tsa.statespace.sarimax.SARIMAX.html#statsmodels.tsa.statespace.sarimax.SARIMAX), the default trend is None.  So I probably wanted to do VARMAX with trend=None in the first place, but I didn't realize I had to specify that.

On one hand, I get that an ARIMA model might make more sense to have a default trend of None than a non-differenced model.  On the other hand, there might be something to be said for consistent defaults across statespace classes.

Would you prefer to take credit for posting the answer on StackOverflow, or should I just do it with attribution?

Resolving patch:
diff --git a/statsmodels/tsa/statespace/varmax.py b/statsmodels/tsa/statespace/varmax.py
--- a/statsmodels/tsa/statespace/varmax.py
+++ b/statsmodels/tsa/statespace/varmax.py
@@ -936,14 +936,11 @@ def _set_final_predicted_state(self, exog, out_of_sample):
 
         Notes
         -----
-        We need special handling for forecasting with `exog` or trend, because
+        We need special handling for forecasting with `exog`, because
         if we had these then the last predicted_state has been set to NaN since
-        we did not have the appropriate `exog` to create it. Since we handle
-        trend in the same way as `exog`, we still have this issue when only
-        trend is used without `exog`.
+        we did not have the appropriate `exog` to create it.
         """
-        flag = out_of_sample and (
-            self.model.k_exog > 0 or self.model.k_trend > 0)
+        flag = out_of_sample and self.model.k_exog > 0
 
         if flag:
             tmp_endog = concat([


Test patch:
diff --git a/statsmodels/tsa/statespace/tests/test_varmax.py b/statsmodels/tsa/statespace/tests/test_varmax.py
--- a/statsmodels/tsa/statespace/tests/test_varmax.py
+++ b/statsmodels/tsa/statespace/tests/test_varmax.py
@@ -1051,7 +1051,57 @@ def test_append_results():
                     res1.forecast(10, exog=np.ones(10)))
 
 
-def test_extend_results():
+@pytest.mark.parametrize('trend', ['n', 'c', 'ct'])
+@pytest.mark.parametrize('forecast', [True, False])
+def test_extend_results(trend, forecast):
+    endog = np.arange(200).reshape(100, 2)
+    trend_params = []
+    if trend == 'c':
+        trend_params = [0.1, 0.2]
+    if trend == 'ct':
+        trend_params = [0.1, 0.2, 1., 2.]
+    params = np.r_[trend_params,
+                   0.5, -0.1, 0.0, 0.2,
+                   1., 0., 1.]
+
+    mod1 = varmax.VARMAX(endog, order=(1, 0), trend=trend)
+    res1 = mod1.smooth(params)
+    if forecast:
+        # Call `forecast` to trigger the _set_final_exog and
+        # _set_final_predicted_state context managers
+        res1.forecast()
+
+    mod2 = mod1.clone(endog[:50])
+    res2 = mod2.smooth(params)
+    if forecast:
+        # Call `forecast` to trigger the _set_final_exog and
+        # _set_final_predicted_state context managers
+        res2.forecast()
+    res3 = res2.extend(endog[50:])
+
+    assert_allclose(res3.llf_obs, res1.llf_obs[50:])
+
+    for attr in [
+            'filtered_state', 'filtered_state_cov', 'predicted_state',
+            'predicted_state_cov', 'forecasts', 'forecasts_error',
+            'forecasts_error_cov', 'standardized_forecasts_error',
+            'scaled_smoothed_estimator',
+            'scaled_smoothed_estimator_cov', 'smoothing_error',
+            'smoothed_state',
+            'smoothed_state_cov', 'smoothed_state_autocov',
+            'smoothed_measurement_disturbance',
+            'smoothed_state_disturbance',
+            'smoothed_measurement_disturbance_cov',
+            'smoothed_state_disturbance_cov']:
+        desired = getattr(res1, attr)
+        if desired is not None:
+            desired = desired[..., 50:]
+        assert_allclose(getattr(res3, attr), desired, atol=1e-12)
+
+    assert_allclose(res3.forecast(10), res1.forecast(10))
+
+
+def test_extend_results_exog():
     endog = np.arange(200).reshape(100, 2)
     exog = np.ones(100)
     params = [0.1, 0.2,