Repository: statsmodels/statsmodels

Issue:
statespace ljung-box in summary, lag too large ?
 statsmodels.tsa.statespace.mlemodel MLEResults.summary uses ljungbox with default maxlag which is most likely too large.

`lb = self.test_serial_correlation(method='ljungbox')`

see #286  
related PR #3360 changes default for standalone function if 40 is too large

Hyndman's rule of thumb should be better http://robjhyndman.com/hyndsight/ljung-box-test/

Note, not backwards compatible, but it's not even released yet.



Discussion:
@ChadFulton  any opinion on this?
It does Hyndman's rule of thumb looks good to me; I think there's no problem changing the summary default (and fixing the incorrect docstring!). I can make a PR unless @zveryansky wants to.

Separate from this issue is that the summary doesn't allow setting the `lag` option (or other options for other reported tests), and does not even report what was used. I don't know if that's something to change, or if further / more specific diagnostic tests are something that users can do if they want to.
Based on the summary format, it looks like there is still space to add "xx lags" or a variation of it.

I'm not sure about options, generically we would need `**kwargs` in that case. But improvements to summary options are still needed. Durbin-Watson in OLS doesn't make much sense if it's cross-section data and ljung-box would be better than DW even in OLS.
@josef-pkt @ChadFulton 
I will fix it.
Just to make it clear:
1) in #286 and #3360 we decided to change default behavior to Hyndman's rule of thumb and some docstrings updates
2) add "lags" to summary

Am I correct?
@zveryansky 
to summarize:
We leave the stata like default as you have it in #3360 for the ljung-box function itself, the corresponding statespace method should use the same min.

When we actually use it and need just one number as in statespace summary, then we use the Hyndman default. We also add Hyndman default as recommendation to the docstring.
extra issue: @ChadFulton  can we detect seasonality to switch default? e.g. in SARIMAX, or do we stick with the non-seasonal Hyndman default?

to 2. we use the proper (Hyndman) `lags` in the ljung-box call, but I would leave out for now the additional issue about how to add keyword options to the summary signature itself.
The number of lags used can be added to the description/name of statistic in summary

We can detect seasonality in specific models, but not in the `MLEModel` class. If I'm reading Hyndman correctly, if there is seasonality he wants `lags = min(2m, T / 5)` where `m` is the seasonal period.

In `SARIMAX`, we could do the following to modify for seasonality:

(I haven't actually run this code, but it should be pretty close to this):

```python
def test_serial_correlation(self, method, lags=None):
    if lags is None and self.k_seasons > 0:
        lags = min(2 * self.k_seasons, self.nobs / 5)
    return super(SARIMAXResults, self).test_serial_correlation(self, method, lags=lags):
```

and for `UnobservedComponents`:

```python
def test_serial_correlation(self, method, lags=None):
    if lags is None and self.seasonal_period > 0:
        lags = min(2 * self.seasonal_period, self.nobs / 5)
    return super(UnobservedComponentsResults, self).test_serial_correlation(self, method, lags=lags):
```
That sounds like we could just to an hasattr(self, 'k_season')  hasattr(self, 'seasonal_period') check in MLEModel, and we don't have to change subclasses, unless they override the summary.
> That sounds like we could just to an hasattr(self, 'k_season') hasattr(self, 'seasonal_period') check in MLEModel, and we don't have to change subclasses, unless they override the summary.

That sounds like a better idea.

We can also standardize on one of the attribute names and migrate the other class to use that terminology (we can still maintain the old attribute for a while for backwards compatibility, depending on if we care about that for state space yet).
I'm a big fan of consistent namings :)
Is `seasonal_period` the better name? I know that R uses `frequency` in this context (e.g. http://robjhyndman.com/hyndsight/seasonal-periods/), but I think that is somewhat incorrect terminology.

i.e. we would say that a weekly seasonal effect has a period of 7 days per cycle (`period=7`), and a frequency of 1 cycle per 7 days (`frequency=1/7`).
Definitely not frequency because it also collides with `freq` for dates (and I never remember if this is the length or 1 / length seasons per period.)

I like something that explicitly refers to number of periods, The best I can come up with would be `season_length`  that would avoid conflicting jargon across fields. But I think `seasonal_periods` (maybe better plural) would also be good.
@ChadFulton Was this ever resolved?  Is a resolution still desired?

Resolving patch:
diff --git a/statsmodels/tsa/ar_model.py b/statsmodels/tsa/ar_model.py
--- a/statsmodels/tsa/ar_model.py
+++ b/statsmodels/tsa/ar_model.py
@@ -5,6 +5,7 @@
 import copy
 import datetime as dt
 from types import SimpleNamespace
+import warnings
 
 import numpy as np
 from numpy.linalg import inv, slogdet
@@ -226,13 +227,11 @@ def __init__(self, endog, lags, trend='c', seasonal=False, exog=None,
         if deterministic is not None:
             self._old_names = False
             if self._trend != "n" or self._seasonal:
-                import warnings
                 warnings.warn(
                     'When using deterministic, trend must be "n" and '
                     'seasonal must be False.', RuntimeWarning
                 )
         if self._old_names is None:
-            import warnings
             warnings.warn(
                 "The parameter names will change after 0.12 is "
                 "released. Set old_names to False to use the new names "
@@ -741,7 +740,6 @@ class AR(tsa_model.TimeSeriesModel):
                                     "extra_sections": ""}
 
     def __init__(self, endog, dates=None, freq=None, missing='none'):
-        import warnings
         warnings.warn(AR_DEPRECATION_WARN, FutureWarning)
         super(AR, self).__init__(endog, None, dates, freq, missing=missing)
         endog = self.endog  # original might not have been an ndarray
@@ -1863,6 +1861,7 @@ def test_serial_correlation(self, lags=None, model_df=None):
             The maximum number of lags to use in the test. Jointly tests that
             all autocorrelations up to and including lag j are zero for
             j = 1, 2, ..., lags. If None, uses lag=12*(nobs/100)^{1/4}.
+            After 0.12 the number of lags will change to min(10, nobs // 5).
         model_df : int
             The model degree of freedom to use when adjusting computing the
             test statistic to account for parameter estimation. If None, uses
@@ -1896,6 +1895,12 @@ def test_serial_correlation(self, lags=None, model_df=None):
         # Default lags for acorr_ljungbox is 40, but may not always have
         # that many observations
         if lags is None:
+            # TODO: Switch to min(10, nobs//5) after 0.12
+            warnings.warn("The default value of lags is changing.  After 0.12,"
+                          " this value will become min(10, nobs//5). Directly "
+                          "set lags to silence this warning.", FutureWarning)
+            # Future
+            # lags = min(nobs // 5, 10)
             lags = int(min(12 * (nobs_effective / 100) ** (1 / 4),
                            nobs_effective - 1))
         test_stats = acorr_ljungbox(self.resid, lags=lags, boxpierce=False,
diff --git a/statsmodels/tsa/statespace/mlemodel.py b/statsmodels/tsa/statespace/mlemodel.py
--- a/statsmodels/tsa/statespace/mlemodel.py
+++ b/statsmodels/tsa/statespace/mlemodel.py
@@ -3156,7 +3156,10 @@ def test_serial_correlation(self, method, lags=None):
             If lags is a list or array, then all lags are included up to the
             largest lag in the list, however only the tests for the lags in the
             list are reported.
-            If lags is None, then the default maxlag is 12*(nobs/100)^{1/4}
+            If lags is None, then the default maxlag is 12*(nobs/100)^{1/4}.
+            After 0.12 the default maxlag will change to min(10, nobs // 5) for
+            non-seasonal models and min(2*m, nobs // 5) for seasonal time
+            series where m is the seasonal period.
 
         Returns
         -------
@@ -3200,7 +3203,19 @@ def test_serial_correlation(self, method, lags=None):
             # Default lags for acorr_ljungbox is 40, but may not always have
             # that many observations
             if lags is None:
-                lags = min(40, nobs_effective - 1)
+                seasonal_periods = getattr(self.model, "seasonal_periods", 0)
+                if seasonal_periods:
+                    lags = min(2 * seasonal_periods, nobs_effective // 5)
+                else:
+                    lags = min(10, nobs_effective // 5)
+
+                warnings.warn(
+                    "The default value of lags is changing.  After 0.12, "
+                    "this value will become min(10, nobs//5) for non-seasonal "
+                    "time series and min (2*m, nobs//5) for seasonal time "
+                    "series. Directly set lags to silence this warning.",
+                    FutureWarning
+                )
 
             for i in range(self.model.k_endog):
                 results = acorr_ljungbox(


Test patch:
diff --git a/statsmodels/tsa/statespace/tests/test_conserve_memory.py b/statsmodels/tsa/statespace/tests/test_conserve_memory.py
--- a/statsmodels/tsa/statespace/tests/test_conserve_memory.py
+++ b/statsmodels/tsa/statespace/tests/test_conserve_memory.py
@@ -288,8 +288,10 @@ def test_fittedvalues_resid_predict(conserve_memory):
                     res2.test_normality('jarquebera'))
     assert_allclose(res1.test_heteroskedasticity('breakvar'),
                     res2.test_heteroskedasticity('breakvar'))
-    assert_allclose(res1.test_serial_correlation('ljungbox'),
-                    res2.test_serial_correlation('ljungbox'))
+    with pytest.warns(FutureWarning):
+        actual = res1.test_serial_correlation('ljungbox')
+        desired = res2.test_serial_correlation('ljungbox')
+    assert_allclose(actual, desired)
 
 
 def test_get_prediction_memory_conserve():
diff --git a/statsmodels/tsa/statespace/tests/test_fixed_params.py b/statsmodels/tsa/statespace/tests/test_fixed_params.py
--- a/statsmodels/tsa/statespace/tests/test_fixed_params.py
+++ b/statsmodels/tsa/statespace/tests/test_fixed_params.py
@@ -462,8 +462,10 @@ def check_results(res1, res2, check_lutkepohl=False, check_params=True):
     assert_allclose(res2.test_heteroskedasticity('breakvar'),
                     res1.test_heteroskedasticity('breakvar'))
 
-    assert_allclose(res2.test_serial_correlation('ljungbox'),
-                    res1.test_serial_correlation('ljungbox'))
+    with pytest.warns(FutureWarning):
+        actual = res2.test_serial_correlation('ljungbox')
+        desired = res1.test_serial_correlation('ljungbox')
+    assert_allclose(actual, desired)
 
 
 def test_sarimax_nonconsecutive():
diff --git a/statsmodels/tsa/statespace/tests/test_mlemodel.py b/statsmodels/tsa/statespace/tests/test_mlemodel.py
--- a/statsmodels/tsa/statespace/tests/test_mlemodel.py
+++ b/statsmodels/tsa/statespace/tests/test_mlemodel.py
@@ -836,17 +836,19 @@ def test_diagnostics():
     with pytest.raises(NotImplementedError):
         res.test_heteroskedasticity(method='invalid')
 
-    actual = res.test_serial_correlation(method=None)
-    desired = res.test_serial_correlation(method='ljungbox')
+    with pytest.warns(FutureWarning):
+        actual = res.test_serial_correlation(method=None)
+    with pytest.warns(FutureWarning):
+        desired = res.test_serial_correlation(method='ljungbox')
     assert_allclose(actual, desired)
 
     with pytest.raises(NotImplementedError):
         res.test_serial_correlation(method='invalid')
 
     # Smoke tests for other options
-    actual = res.test_heteroskedasticity(method=None, alternative='d',
-                                         use_f=False)
-    desired = res.test_serial_correlation(method='boxpierce')
+    res.test_heteroskedasticity(method=None, alternative='d', use_f=False)
+    with pytest.warns(FutureWarning):
+        res.test_serial_correlation(method='boxpierce')
 
 
 def test_diagnostics_nile_eviews():
diff --git a/statsmodels/tsa/statespace/tests/test_sarimax.py b/statsmodels/tsa/statespace/tests/test_sarimax.py
--- a/statsmodels/tsa/statespace/tests/test_sarimax.py
+++ b/statsmodels/tsa/statespace/tests/test_sarimax.py
@@ -2364,8 +2364,10 @@ def check_concentrated_scale(filter_univariate=False):
         desired = res_orig.test_heteroskedasticity(method='breakvar')
         assert_allclose(actual, desired, rtol=1e-5, atol=atol)
 
-        actual = res_conc.test_serial_correlation(method='ljungbox')
-        desired = res_orig.test_serial_correlation(method='ljungbox')
+        with pytest.warns(FutureWarning):
+            actual = res_conc.test_serial_correlation(method='ljungbox')
+        with pytest.warns(FutureWarning):
+            desired = res_orig.test_serial_correlation(method='ljungbox')
         assert_allclose(actual, desired, rtol=1e-5, atol=atol)
 
         # Test predict
diff --git a/statsmodels/tsa/tests/test_ar.py b/statsmodels/tsa/tests/test_ar.py
--- a/statsmodels/tsa/tests/test_ar.py
+++ b/statsmodels/tsa/tests/test_ar.py
@@ -784,9 +784,11 @@ def test_parameterless_autoreg():
                     'remove_data', 'save', 't_test', 't_test_pairwise',
                     'wald_test', 'wald_test_terms'):
             continue
+        warning = None if attr != "test_serial_correlation" else FutureWarning
         attr = getattr(res, attr)
         if callable(attr):
-            attr()
+            with pytest.warns(warning):
+                attr()
         else:
             assert isinstance(attr, object)