Repository: statsmodels/statsmodels

Issue:
Issue in removing nan's from the trend variable
Hi,
I am trying to use your code given in the link: https://github.com/statsmodels/statsmodels/blob/master/statsmodels/tsa/seasonal.py
to remove the nan's which occur due to the moving averages. When i put the variable extrapolate_trend >0 then i get a value error saying: 
ValueError: could not broadcast input array from shape (26,1) into shape (26)
I want to extrapolate the trend values for a time series at a week level


Discussion:
Can you make up a full example that I can run, and show the full traceback, at least the end with the exception line.

There might be something wrong if the series is a one-column array, but I cannot see anything related from skimming the code.
If i use the data mentioned in the end of the code itself (which is a one dimensional array):
x = np.array([-50, 175, 149, 214, 247, 237, 225, 329, 729, 809,
                  530, 489, 540, 457, 195, 176, 337, 239, 128, 102,
                  232, 429, 3, 98, 43, -141, -77, -13, 125, 361, -45, 184])
    results = seasonal_decompose(x, freq=4, extrapolate_trend = 4)
I get an error saying ValueError: could not broadcast input array from shape (2,1) into shape (2)

Would i have to convert it to a 2 column array?
Thanks for reporting this. It needs to be fixed before the upcoming release.

With two column it works, but this is a bug in `_extrapolate_trend` which doesn't work with 1-D array.
Because the x is squeezed in the function, using a 2-D single column is treated the same way as a 1-D and doesn't provide a workaround.

```
from statsmodels.tsa.seasonal import seasonal_decompose, _extrapolate_trend

_extrapolate_trend(np.arange(12), 4)
Traceback (most recent call last):

  File "<ipython-input-38-235087ac0eb0>", line 1, in <module>
    _extrapolate_trend(np.arange(12), 4)

  File "m:\...\statsmodels\tsa\seasonal.py", line 37, in _extrapolate_trend
    trend[:front] = (np.arange(0, front) * np.c_[k] + np.c_[n]).T

ValueError: could not broadcast input array from shape (0,1) into shape (0)


_extrapolate_trend(np.arange(12)[:, None], 4)
Out[39]: 
array([[ 0],
       [ 1],
       [ 2],
       ..., 
       [ 9],
       [10],
       [11]])
```
I'm not sure how to make `trend[:front] = (np.arange(0, front) * np.c_[k] + np.c_[n]).T` to work for both 1-D and 2-D, so I will fix it by squeezing if trend is 1-D.
That makes the change relatively simple and shouldn't be "dangerous" in causing another bug.
Thanks, please let me know when you make the changes.

I will finish up in a few minutes.

The unit test don't fail because the freq=2, and for freq 2 and 3 the ValueError doesn't show up. It shows up with freq=4. I'm changing the unit test to use a larger example.

Resolving patch:
diff --git a/statsmodels/tsa/seasonal.py b/statsmodels/tsa/seasonal.py
--- a/statsmodels/tsa/seasonal.py
+++ b/statsmodels/tsa/seasonal.py
@@ -34,12 +34,18 @@ def _extrapolate_trend(trend, npoints):
     k, n = np.linalg.lstsq(
         np.c_[np.arange(front, front_last), np.ones(front_last - front)],
         trend[front:front_last])[0]
-    trend[:front] = (np.arange(0, front) * np.c_[k] + np.c_[n]).T
+    extra = (np.arange(0, front) * np.c_[k] + np.c_[n]).T
+    if trend.ndim == 1:
+        extra = extra.squeeze()
+    trend[:front] = extra
 
     k, n = np.linalg.lstsq(
         np.c_[np.arange(back_first, back), np.ones(back - back_first)],
         trend[back_first:back])[0]
-    trend[back + 1:] = (np.arange(back + 1, trend.shape[0]) * np.c_[k] + np.c_[n]).T
+    extra = (np.arange(back + 1, trend.shape[0]) * np.c_[k] + np.c_[n]).T
+    if trend.ndim == 1:
+        extra = extra.squeeze()
+    trend[back + 1:] = extra
 
     return trend
 


Test patch:
diff --git a/statsmodels/tsa/tests/test_seasonal.py b/statsmodels/tsa/tests/test_seasonal.py
--- a/statsmodels/tsa/tests/test_seasonal.py
+++ b/statsmodels/tsa/tests/test_seasonal.py
@@ -261,22 +261,26 @@ def test_2d(self):
         assert_equal(trend, expected)
 
     def test_interpolate_trend(self):
-        x = np.arange(6)
-        trend = seasonal_decompose(x, freq=2).trend
+        x = np.arange(12)
+        freq = 4
+        trend = seasonal_decompose(x, freq=freq).trend
         assert_equal(trend[0], np.nan)
 
-        trend = seasonal_decompose(x, freq=2, extrapolate_trend=1).trend
+        trend = seasonal_decompose(x, freq=freq, extrapolate_trend=5).trend
+        assert_almost_equal(trend, x)
+
+        trend = seasonal_decompose(x, freq=freq, extrapolate_trend='freq').trend
         assert_almost_equal(trend, x)
 
-        trend = seasonal_decompose(x, freq=2, extrapolate_trend='freq').trend
+        trend = seasonal_decompose(x[:, None], freq=freq, extrapolate_trend=5).trend
         assert_almost_equal(trend, x)
 
         # 2d case
-        x = np.tile(np.arange(6), (2, 1)).T
-        trend = seasonal_decompose(x, freq=2, extrapolate_trend=1).trend
+        x = np.tile(np.arange(12), (2, 1)).T
+        trend = seasonal_decompose(x, freq=freq, extrapolate_trend=1).trend
         assert_almost_equal(trend, x)
 
-        trend = seasonal_decompose(x, freq=2, extrapolate_trend='freq').trend
+        trend = seasonal_decompose(x, freq=freq, extrapolate_trend='freq').trend
         assert_almost_equal(trend, x)
 
     def test_raises(self):