Repository: statsmodels/statsmodels

Issue:
acf / pacf do not work on pandas objects
Something I noticed writing some examples.



Discussion:
Update. They work fine on pandas objects. It's only when you have missing data that they break. E.g.,

```
from statsmodels.datasets.macrodata import load_pandas
cpi = load_pandas().data["cpi"]
sm.tsa.pacf(cpi.diff())
```

Not really sure what to do here yet. Thoughts?

Spoke too soon ACF doesn't work

``` python
dta = sm.datasets.sunspots.load_pandas().data
dta.index = pandas.Index(sm.tsa.datetools.dates_from_range('1700', '2008'))
del dta["YEAR"]
sm.tsa.acf(dta['SUNACTIVITY'])
#AssertionError: Index length did not match values
sm.tsa.acf(dta) # needs a .values.squeeze()
#ValueError: object too deep for desired array
```

There's actually another issue here. The object is too deep for desired array happens because DataFrames can't be 1d. So we need not only an asarray but a squeeze and probably a dims check for a better error message.

the asarray was only added to acovf. acf calls acovf if `fft=False`, but acf has it's own calculation if `fft=True`

suggested fix, add the same asarray and dim change to acf directly.

Alight, I'll try to look into the issue this weekend.

On Thursday, May 30, 2013, Josef Perktold wrote:

> the asarray was only added to acovf. acf calls acovf if fft=False, but
> acf has it's own calculation if fft=True
> 
> suggested fix, add the same asarray and dim change to acf directly.
> 
> —
> Reply to this email directly or view it on GitHubhttps://github.com/statsmodels/statsmodels/issues/322#issuecomment-18713845
> .

## 

Trent Hauck

this issue never mentions and links to the pull request #486 

@josef-pkt This appears to work in 0.5.0.  Maybe ready to be closed, or it just missing a test?

It's still broken

using the sunspot data from Skipper's example above

```
>>> dta
<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 309 entries, 1700-12-31 00:00:00 to 2008-12-31 00:00:00
Data columns (total 1 columns):
SUNACTIVITY    309  non-null values
dtypes: float64(1)
>>> sm.tsa.acf(dta, fft=False)
array([ 1.        ,  0.82020129,  0.45126849,  0.03957655, -0.27579196,
       -0.42523943, -0.37659509, -0.15737391,  0.15820254,  0.47309753,
        0.65898002,  0.65029082,  0.45666254,  0.16179329, -0.12205105,
       -0.3161808 , -0.37471125, -0.30605753, -0.1348069 ,  0.09158727,
        0.2975632 ,  0.4207074 ,  0.41183954,  0.27020758,  0.04496208,
       -0.17428715, -0.33045026, -0.37287834, -0.28555061, -0.11794414,
        0.08293231,  0.24897507,  0.32752101,  0.28335919,  0.1375272 ,
       -0.05526386, -0.22973205, -0.31338879, -0.29355684, -0.17897285,
       -0.01769038])
>>> sm.tsa.acf(dta, fft=True)
array([[  1.00000000e+00,              nan,   1.00000000e+00, ...,
          1.00000000e+00,   1.00000000e+00,              nan],
       [  7.49831457e-01,              nan,  -1.00000000e+00, ...,
         -1.00000000e+00,  -1.00000000e+00,              nan],
       [  5.68819900e-01,              nan,   5.00000000e-01, ...,
          5.00000000e-01,   5.00000000e-01,              nan],
       ..., 
       [  1.21401450e+01,              nan,   0.00000000e+00, ...,
          0.00000000e+00,   0.00000000e+00,              nan],
       [  8.49950450e+00,              nan,   7.09642837e-17, ...,
         -4.37578782e-17,   7.09642837e-17,              nan],
       [  1.74907666e+00,              nan,  -8.87053546e-18, ...,
          5.46973478e-18,  -8.87053546e-18,              nan]])
>>> sm.tsa.acf(dta, fft=True).shape
(41, 625)

>>> sm.tsa.acf(dta.values.squeeze(), fft=True)
array([ 1.        ,  0.82020129,  0.45126849,  0.03957655, -0.27579196,
       -0.42523943, -0.37659509, -0.15737391,  0.15820254,  0.47309753,
        0.65898002,  0.65029082,  0.45666254,  0.16179329, -0.12205105,
       -0.3161808 , -0.37471125, -0.30605753, -0.1348069 ,  0.09158727,
        0.2975632 ,  0.4207074 ,  0.41183954,  0.27020758,  0.04496208,
       -0.17428715, -0.33045026, -0.37287834, -0.28555061, -0.11794414,
        0.08293231,  0.24897507,  0.32752101,  0.28335919,  0.1375272 ,
       -0.05526386, -0.22973205, -0.31338879, -0.29355684, -0.17897285,
       -0.01769038])
```


Resolving patch:
diff --git a/statsmodels/tsa/stattools.py b/statsmodels/tsa/stattools.py
--- a/statsmodels/tsa/stattools.py
+++ b/statsmodels/tsa/stattools.py
@@ -418,6 +418,7 @@ def acf(x, unbiased=False, nlags=40, confint=None, qstat=False, fft=False,
         #acf = np.take(avf/avf[0], range(1,nlags+1))
         acf = avf[:nlags + 1] / avf[0]
     else:
+        x = np.squeeze(np.asarray(x))
         #JP: move to acovf
         x0 = x - x.mean()
         # ensure that we always use a power of 2 or 3 for zero-padding,


Test patch:
diff --git a/statsmodels/tsa/tests/test_stattools.py b/statsmodels/tsa/tests/test_stattools.py
--- a/statsmodels/tsa/tests/test_stattools.py
+++ b/statsmodels/tsa/tests/test_stattools.py
@@ -326,6 +326,12 @@ def test_arma_order_select_ic_failure():
     res = arma_order_select_ic(y)
 
 
+def test_acf_fft_dataframe():
+    # regression test #322
+
+    result = acf(sunspots.load_pandas().data[['SUNACTIVITY']], fft=True)
+    assert_equal(result.ndim, 1)
+
 if __name__=="__main__":
     import nose
 #    nose.runmodule(argv=[__file__, '-vvs','-x','-pdb'], exit=False)