Repository: statsmodels/statsmodels

Issue:
BUG : arima_model._get_predict_out_of_sample, ignores exogenous of there is no trend ?
Hello,

I have a feeling there is a problem with `arima_model._get_predict_out_of_sample()` if there is an exogenous variable, but no trend.  Looking at the code, it just doesn't use exog if there is no trend.

I have produced a sample script to reproduce the problem.

In the sample, arma_mod2.fit() produces a forecast of zero.  It produces a forecast of zero, whatever I set exog to be.  I believe the forecast should be the 'my_forecast' variable.

Apologies in advance if I have misread / misunderstood the documentation or how it is supposed to work.

Example code:

```
import numpy as np
import pandas as pd

import statsmodels.api as sm
from statsmodels.tsa.arima_process import arma_generate_sample

np.random.seed(12345)

arparams = np.array([.75, -.25])
maparams = np.array([.65, .35])

arparam = np.r_[1, -arparams]
maparam = np.r_[1, maparams]

nobs = 20

dates = sm.tsa.datetools.dates_from_range('1980',length=nobs)

y  = arma_generate_sample(arparams, maparams, nobs)

df       = pd.DataFrame({'y':y},index=dates)
df['y1'] = df['y'].shift(1)

arma_mod2     = sm.tsa.ARMA( df['y'][1:] , order=(0,0), exog=df['y1'][1:] )
arma_res2     = arma_mod2.fit(trend='nc', disp=False)    
arma_forecast = arma_res2.forecast(exog = df['y'][-1])

my_forecast   = df['y'][-1] * arma_res2.params[0]

print(' y[-1]            : {0:0.4f}'.format( df['y'][-1] ) )
print( 'Exog coef        : {0:0.4f}'.format( arma_res2.params[0] ) )
print( 'Model Forecast   : {0:0.4f}'.format( arma_forecast[0] ) )
print( 'My Forecast      : {0:0.4f}'.format( my_forecast ) )
```



Discussion:
What version are you using?

[**Edit**: nm. This problem exists in current master.]

I think there is a problem here but this is an odd case. I'm not sure ARMA should support an order (0,0) fit. Many changes would have to be made.

#1124 should fix this problem (though it likely still doesn't work for a (0,0) order model).

Thanks for fixing this!  I am on version 0.6.0.dev-3b7082c which I downloaded from http://statsmodels.sourceforge.net/binaries/, though when I downloaded it a few days ago there was a version for python 3.3, which does not appear to be available now.

How do I install the latest version of the toolkit with this patch? 

It's not merged into master yet, so the fix won't be there, though you can apply the patch to the source yourself. I have tentative plans for a merge sprint this weekend.

Thanks for the update!

Please try it out and let us know if you run into any difficulties.

Hello,

I hope you are all having a great weekend.   I have tried the update, but unfortunately it does not work for me.

To test the ARMA(0,0) model, I used an equivalent ARMA(1,0) model and a basic linear regression (which both agree).   See the code at the bottom of this post.

---

The problem:
In arima_model.py,
Line 227 states:
#NOTE: you shouldn't have to give in-sample exog!
However, from stepping through the code, at this point exog contains the in-sample exog as well as the single exog that I have passed to it.
Also, my example code calls _get_predict_out_of_sample twice.  Each time it requests a 1 step forecast.  The first time, the returned mu is of length 1.  The second time, the returned mu is of length 20.  So I believe something is wrong here.

---

To explain my strange use case...
I would like to do time series analysis on a series of closing prices.  e.g. Prices taken at end of every month.
However, I will only be able to act on the data on the 1st day of the next month.
So I want to predict price returns from 1st-last day of every month.
However, my historical monthly returns will be calculated from [last day of month] till the next [last day of month].
I am therefore using exogenous variables for [last day of month] to next [last day of month].  And using endogenous variables for [first to last day of month]
To see the code I have been using:

import numpy as np
import pandas as pd

from sklearn import linear_model

import statsmodels.api as sm
from statsmodels.tsa.arima_process import arma_generate_sample

np.random.seed(12345)

arparams = np.array([.75, -.25])
maparams = np.array([.65, .35])

arparam = np.r_[1, -arparams]
maparam = np.r_[1, maparams]

nobs = 20

dates = sm.tsa.datetools.dates_from_range('1980',length=nobs)

y  = arma_generate_sample(arparams, maparams, nobs)

df       = pd.DataFrame({'y':y},index=dates)
df['y1'] = df['y'].shift(1)

print('data is : {0}'.format( y.**str**() ))
print('y[-1]={0}'.format(y[-1]))

arma_mod1  = sm.tsa.ARMA( df['y'], order=(1, 0) )
arma_res1  = arma_mod1.fit(trend='nc', disp=False, method='css' )
arma_fc1   = arma_res1.forecast( steps=1 )

arma_mod2  = sm.tsa.ARMA( df['y'][1:] , order=(0,0), exog=df['y1'][1:] )
arma_res2  = arma_mod2.fit(trend='nc', disp=False)
arma_fc2   = arma_res2.forecast( steps=1, exog=[ y[-1] ] )   

clf1 = linear_model.LinearRegression( fit_intercept=False)
fX   = [[v] for v in df['y1'][1:].values]
fy   = df['y'][1:].values
clf1.fit(fX,fy)
clf_pred = clf1.predict(y[-1])

print( 'ARMA(1,0) coef    : {0:0.4f}'.format( arma_res1.params[0] ) )
print( 'ARMA(0,0) coef    : {0:0.4f}'.format( arma_res2.params[0] ) )
print( 'LinReg coef1      : {0:0.4f}'.format( clf1.coef_[0]) )

print( 'ARMA(1,0) FC   : {0:0.4f}'.format( arma_fc1[0] ) )
print( 'ARMA(0,0) FC   : {0:0.4f}'.format( arma_fc2[0][0] ) )
print( 'LinReg Pred    : {0:0.4f}'.format( clf_pred[0] ) )

On Sat, Oct 26, 2013 at 11:37 AM, Barnyard notifications@github.com wrote:

> Hello,
> I hope you are all having a great weekend. I have tried the update, but
> unfortunately it does not work for me.
> To test the ARMA(0,0) model, I used an equivalent ARMA(1,0) model and a
> basic linear regression (which both agree). See the code at the bottom of
> this post.
> 
> The problem:
> In arima_model.py,
> Line 227 states:
> #NOTE: you shouldn't have to give in-sample exog!
> However, from stepping through the code, at this point exog contains the
> in-sample exog as well as the single exog that I have passed to it.

Yes, this is intentional. I mean that the users shouldn't have to include
in-sample lagged exogenous variables when they're doing prediction, which
they did before.

On Sat, Oct 26, 2013 at 11:37 AM, Barnyard notifications@github.com wrote:

> Also, my example code calls getpredict_out_of_sample twice. Each time it requests a 1 step forecast. The first time, the returned mu is of length 1. The second time, the returned mu is of length 20. So I believe something is wrong here.

Can you post code that replicates this?

On Sat, Oct 26, 2013 at 11:37 AM, Barnyard notifications@github.com wrote:

> import numpy as np
> import pandas as pd
> 
> from sklearn import linear_model
> 
> import statsmodels.api as sm
> from statsmodels.tsa.arima_process import arma_generate_sample
> 
> np.random.seed(12345)
> 
> arparams = np.array([.75, -.25])
> maparams = np.array([.65, .35])
> 
> arparam = np.r_[1, -arparams]
> maparam = np.r_[1, maparams]
> 
> nobs = 20
> 
> dates = sm.tsa.datetools.dates_from_range('1980',length=nobs)
> 
> y = arma_generate_sample(arparams, maparams, nobs)
> 
> df = pd.DataFrame({'y':y},index=dates)
> df['y1'] = df['y'].shift(1)
> 
> print('data is : {0}'.format( y._str_() ))
> print('y[-1]={0}'.format(y[-1]))
> 
> arma_mod1 = sm.tsa.ARMA( df['y'], order=(1, 0) )
> arma_res1 = arma_mod1.fit(trend='nc', disp=False, method='css' )
> arma_fc1 = arma_res1.forecast( steps=1 )
> 
> arma_mod2 = sm.tsa.ARMA( df['y'][1:] , order=(0,0), exog=df['y1'][1:] )
> arma_res2 = arma_mod2.fit(trend='nc', disp=False)
> arma_fc2 = arma_res2.forecast( steps=1, exog=[ y[-1] ] )
> 
> clf1 = linear_model.LinearRegression( fit_intercept=False)
> fX = [[v] for v in df['y1'][1:].values]
> fy = df['y'][1:].values
> clf1.fit(fX,fy)
> clf_pred = clf1.predict(y[-1])
> 
> print( 'ARMA(1,0) coef : {0:0.4f}'.format( arma_res1.params[0] ) )
> print( 'ARMA(0,0) coef : {0:0.4f}'.format( arma_res2.params[0] ) )
> print( 'LinReg coef1 : {0:0.4f}'.format( clf1.coef_[0]) )
> 
> print( 'ARMA(1,0) FC : {0:0.4f}'.format( arma_fc1[0] ) )
> print( 'ARMA(0,0) FC : {0:0.4f}'.format( arma_fc2[0][0] ) )
> print( 'LinReg Pred : {0:0.4f}'.format( clf_pred[0] ) )

What am I supposed to be looking at here?

Also note that we do not support an ARMA(0,0) model, as I mentioned before.
I'm going to change the code to raise an error on this.

The first 3 lines of output from the above are:

ARMA(1,0) coef    : 0.5417
ARMA(0,0) coef    : 0.5417
LinReg coef1      : 0.5417

That shows that the 3 models have fitted the same parameter.  Therefore they should give the same forecasts.

The next 3 lines are:

ARMA(1,0) FC   : 0.3561
ARMA(0,0) FC   : -0.0961
LinReg Pred    : 0.3561

That shows that the ARMA(1,0) and Linear Regression models agree with each other.  But the ARMA(0,0) doesn't agree, even though it has the same parameter.

To see where the code goes wrong, insert a break point just after line 247 of arima_model.py which is

```
endog, resid, mu = _get_predict_out_of_sample(endog, p, q, k_trend, k_exog,
                                               start, errors, trendparam,
                                               exparams, arparams,
                                               maparams, steps, method,
                                               exog)
```

The example code will call the above line twice.  The first time from the arma(1,0) model:

 arma_fc1 = arma_res1.forecast( steps=1 )

The 2nd time from 

arma_fc2 = arma_res2.forecast( steps=1, exog=[ y[-1] ] )

In the first time, it correctly returns mu, of length 1.  As it is the mu of the first out of sample forecast.

The second time, it returns mu of length 20, even though it only needs 1 mu for the first out of sample step.  mu[0] is the mean for the first in-sample prediction. I believe that is where the problem is, as it should not be using the mu for an insample period.

Yes, don't fit an ARMA(0,0) model. An ARMA(0,0) model isn't an ARMA model, and it's not supported, so I wouldn't expect it to do the right thing.


Resolving patch:
diff --git a/statsmodels/tsa/arima_model.py b/statsmodels/tsa/arima_model.py
--- a/statsmodels/tsa/arima_model.py
+++ b/statsmodels/tsa/arima_model.py
@@ -369,6 +369,8 @@ def __init__(self, endog, order=None, exog=None, dates=None, freq=None,
                     "in the model constructor.", FutureWarning)
         else:
             _check_estimable(len(self.endog), sum(order))
+            if order[0] == 0 and order[1] == 0:
+                raise ValueError("Cannot fit order (0,0)")
             self.k_ar = k_ar = order[0]
             self.k_ma = k_ma = order[1]
             self.k_lags = k_lags = max(k_ar,k_ma+1)
@@ -805,6 +807,8 @@ def fit(self, order=None, start_params=None, trend='c', method = "css-mle",
                     "constructor.", FutureWarning)
 
             _check_estimable(len(self.endog), sum(order))
+            if order[0] == 0 and order[1] == 0:
+                raise ValueError("Cannot fit order (0,0)")
             # get model order and constants
             self.k_ar = k_ar = int(order[0])
             self.k_ma = k_ma = int(order[1])
@@ -909,6 +913,8 @@ def __new__(cls, endog, order, exog=None, dates=None, freq=None,
     def __init__(self, endog, order, exog=None, dates=None, freq=None,
                        missing='none'):
         p,d,q = order
+        if p == 0 and q == 0:
+            raise ValueError("Order (0, d, 0) not supported.")
         super(ARIMA, self).__init__(endog, (p,q), exog, dates, freq, missing)
         self.k_diff = d
         self.endog = np.diff(self.endog, n=d)


Test patch:
diff --git a/statsmodels/tsa/tests/test_arima.py b/statsmodels/tsa/tests/test_arima.py
--- a/statsmodels/tsa/tests/test_arima.py
+++ b/statsmodels/tsa/tests/test_arima.py
@@ -1891,6 +1891,11 @@ def test_small_data():
     mod = ARIMA(y, (1, 0, 2))
     res = mod.fit(disp=0, start_params=[.1, .1, .1, .1])
 
+def test_arima00():
+    y = np.random.random(10)
+    assert_raises(ValueError, ARMA, y, (0,0))
+    assert_raises(ValueError, ARIMA, y, (0,1,0))
+    assert_raises(ValueError, ARIMA, y, (0,0,0))
 
 if __name__ == "__main__":
     import nose