internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: statsmodels/statsmodels

Issue:
BUG: get index location for quarterly date string unreliable
Specifying a quarterly date string sometimes breaks, e.g.:

```python
dta = sm.datasets.macrodata.load_pandas().data
dta.index = pd.date_range('1959Q1', '2009Q3', freq='QS')

mod = sm.tsa.SARIMAX(dta.infl)
res = mod.fit()

res.forecast('2012Q4')
```

yields:

```
2009-10-01    3.042447
2010-01-01    2.600136
2010-04-01    2.222128
2010-07-01    1.899074
2010-10-01    1.622987
2011-01-01    1.387037
Freq: QS-JAN, dtype: float64
```


Discussion:

Resolving patch:
diff --git a/statsmodels/tsa/base/tsa_model.py b/statsmodels/tsa/base/tsa_model.py
--- a/statsmodels/tsa/base/tsa_model.py
+++ b/statsmodels/tsa/base/tsa_model.py
@@ -341,7 +341,7 @@ def _get_index_loc(self, key, base_index=None):
                 if index_class is PeriodIndex:
                     date_key = Period(key, freq=base_index.freq)
                 else:
-                    date_key = Timestamp(key)
+                    date_key = Timestamp(key, freq=base_index.freq)
 
                 # Out-of-sample
                 if date_key > base_index[-1]:
@@ -355,6 +355,10 @@ def _get_index_loc(self, key, base_index=None):
                                          periods=len(index) + 1,
                                          freq=base_index.freq)
 
+                    # To avoid possible inconsistencies with `get_loc` below,
+                    # set the key directly equal to the last index location
+                    key = index[-1]
+
         # Get the location
         if date_index:
             # (note that get_loc will throw a KeyError if key is invalid)


Test patch:
diff --git a/statsmodels/tsa/base/tests/test_tsa_indexes.py b/statsmodels/tsa/base/tests/test_tsa_indexes.py
--- a/statsmodels/tsa/base/tests/test_tsa_indexes.py
+++ b/statsmodels/tsa/base/tests/test_tsa_indexes.py
@@ -1137,3 +1137,14 @@ def test_nonfull_periodindex():
                ' forecasting.')
     with pytest.warns(ValueWarning, match=message):
         tsa_model.TimeSeriesModel(endog)
+
+
+def test_get_index_loc_quarterly():
+    # See GH#6339
+
+    ix = pd.date_range('2000Q1', periods=8, freq='QS')
+    endog = pd.Series(np.zeros(8), index=ix)
+
+    mod = tsa_model.TimeSeriesModel(endog)
+    loc, index, _ = mod._get_index_loc('2003Q2')
+    assert_equal(index[loc], pd.Timestamp('2003Q2'))
Repository: statsmodels/statsmodels

Issue:
ENH Docs: example page needs section TOC
The notebook example page is pretty long and it is not quick to find a specific section
http://www.statsmodels.org/dev/examples/index.html

It would be better to have a table of content on top with section headers, e.g. Linear Regression Models, ... User Notes 



Discussion:
aside: `PCA` is in User Notes instead of a "Multivariate" section

no notebooks for other multivariate classes AFAICS, Manova, canonical, ...


Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -13,9 +13,9 @@
 # serve to show the default.
 
 import contextlib
-import json
 import os
 import sys
+import yaml
 from os.path import dirname, join
 
 import sphinx_material
@@ -393,7 +393,7 @@
 # ghissue config
 github_project_url = 'https://github.com/statsmodels/statsmodels'
 
-example_context = json.load(open('examples/landing.json'))
+example_context = yaml.safe_load(open('examples/landing.yml'))
 html_context.update({'examples': example_context})
 
 # --------------- DOCTEST -------------------
@@ -424,8 +424,6 @@ def rstjinja(app, docname, source):
     if app.builder.format != "html":
         return
     src = source[0]
-    import copy
-    orig = copy.deepcopy(src)
     # Skip converted notebooks
     if 'nbconvert_exporter' in src:
         return


Test patch:
diff --git a/statsmodels/tsa/arima/tests/test_model.py b/statsmodels/tsa/arima/tests/test_model.py
--- a/statsmodels/tsa/arima/tests/test_model.py
+++ b/statsmodels/tsa/arima/tests/test_model.py
@@ -157,7 +157,7 @@ def test_statespace():
     mod = ARIMA(endog, order=(1, 0, 1), trend='n')
     res = mod.fit(method='statespace')
     # Note: atol is required only due to precision issues on Windows
-    assert_allclose(res.params, desired_p.params, atol=1e-5)
+    assert_allclose(res.params, desired_p.params, atol=1e-4)
 
     # ARMA(1, 2), with trend
     desired_p, _ = statespace(endog, order=(1, 0, 2),
@@ -165,7 +165,7 @@ def test_statespace():
     mod = ARIMA(endog, order=(1, 0, 2), trend='c')
     res = mod.fit(method='statespace')
     # Note: atol is required only due to precision issues on Windows
-    assert_allclose(res.params, desired_p.params, atol=1e-5)
+    assert_allclose(res.params, desired_p.params, atol=1e-4)
 
     # SARMA(1, 0)x(1, 0)4, no trend
     desired_p, _spec = statespace(endog, order=(1, 0, 0),
@@ -174,7 +174,7 @@ def test_statespace():
     mod = ARIMA(endog, order=(1, 0, 0), seasonal_order=(1, 0, 0, 4), trend='n')
     res = mod.fit(method='statespace')
     # Note: atol is required only due to precision issues on Windows
-    assert_allclose(res.params, desired_p.params, atol=1e-5)
+    assert_allclose(res.params, desired_p.params, atol=1e-4)
 
 
 def test_low_memory():
Repository: statsmodels/statsmodels

Issue:
Robust skewness SK4 calculation "error"

SK4 calculation in robust skewness is  SK_{4}=\\frac{\\mu-\\hat{q}_{0.5}}{\\hat{\\sigma}} where sigma should be standard deviation. But the source code shows that variance was used instead: 


```python
    sigma = np.mean(((y - mu_b)**2), axis)

    sk1 = stats.skew(y, axis=axis)
    sk2 = (q1 + q3 - 2.0 * q2) / (q3 - q1)
    sk3 = (mu - q2) / np.mean(abs(y - q2_b), axis=axis)
    sk4 = (mu - q2) / sigma
```
It should instead be 

```python
    sk4 = (mu - q2) / sigma ** 0.5
```
Thank you.




Discussion:

Resolving patch:
diff --git a/statsmodels/stats/stattools.py b/statsmodels/stats/stattools.py
--- a/statsmodels/stats/stattools.py
+++ b/statsmodels/stats/stattools.py
@@ -193,7 +193,7 @@ def robust_skewness(y, axis=0):
     mu_b = np.reshape(mu, shape)
     q2_b = np.reshape(q2, shape)
 
-    sigma = np.mean(((y - mu_b)**2), axis)
+    sigma = np.sqrt(np.mean(((y - mu_b)**2), axis))
 
     sk1 = stats.skew(y, axis=axis)
     sk2 = (q1 + q3 - 2.0 * q2) / (q3 - q1)


Test patch:
diff --git a/statsmodels/stats/tests/test_statstools.py b/statsmodels/stats/tests/test_statstools.py
--- a/statsmodels/stats/tests/test_statstools.py
+++ b/statsmodels/stats/tests/test_statstools.py
@@ -2,13 +2,15 @@
 # TODO: Test robust kurtosis
 import numpy as np
 import pandas as pd
-from numpy.testing import (assert_almost_equal, assert_raises, assert_equal)
+from numpy.testing import (assert_almost_equal, assert_raises, assert_equal,
+                           assert_allclose)
+
+from statsmodels.stats._adnorm import normal_ad
 from statsmodels.stats.stattools import (omni_normtest, jarque_bera,
                                          durbin_watson, _medcouple_1d, medcouple,
                                          robust_kurtosis, robust_skewness)
-from statsmodels.stats._adnorm import normal_ad
 
-#a random array, rounded to 4 decimals
+# a random array, rounded to 4 decimals
 x = np.array([-0.1184, -1.3403, 0.0063, -0.612, -0.3869, -0.2313, -2.8485,
               -0.2167, 0.4153, 1.8492, -0.3706, 0.9726, -0.1501, -0.0337,
               -1.4423, 1.2489, 0.9182, -0.2331, -0.6182, 0.183])
@@ -259,6 +261,15 @@ def test_robust_skewness_3d(self, reset_randomstate):
         for sk in sk_3d:
             assert_almost_equal(sk, result)
 
+    def test_robust_skewness_4(self, reset_randomstate):
+        x = np.random.standard_normal(1000)
+        x[x > 0] *= 3
+        m = np.median(x)
+        s = x.std(ddof=0)
+        expected = (x.mean() - m) / s
+        _, _, _, sk4 = robust_skewness(x)
+        assert_allclose(expected, sk4)
+
     def test_robust_kurtosis_1d_2d(self, reset_randomstate):
         x = np.random.randn(100)
         y = x[:, None]
Repository: statsmodels/statsmodels

Issue:
DOC/BUG acorr_ljungbox does not assume mean zero
https://stats.stackexchange.com/questions/442741/statsmodel-ljung-box-test-for-non-zero-mean

#6079  incorrect sentence "assumed to have mean zero"

```
x = 1 + np.random.randn(500)
res1 = acorr_ljungbox(x)
res2 = acorr_ljungbox(x - x. mean(0))

np.max(np.abs(res1[0] - res2[0])), np.max(np.abs(res1[1] - res2[1]))
(1.4210854715202004e-14, 3.3306690738754696e-16)
```

ljungbox uses acf which uses acovf which demeans



Discussion:

Resolving patch:
diff --git a/statsmodels/sandbox/stats/diagnostic.py b/statsmodels/sandbox/stats/diagnostic.py
--- a/statsmodels/sandbox/stats/diagnostic.py
+++ b/statsmodels/sandbox/stats/diagnostic.py
@@ -234,8 +234,8 @@ def acorr_ljungbox(x, lags=None, boxpierce=False, model_df=0, period=None,
     Parameters
     ----------
     x : array_like
-        The data series. Assumed to have mean zero, e.g., regression residuals
-        when used as diagnostic test.
+        The data series. The data is demeaned before the test statistic is
+        computed.
     lags : {None, int, array_like}
         If lags is an integer then this is taken to be the largest lag
         that is included, the test result is reported for all smaller lag


Test patch:
Repository: statsmodels/statsmodels

Issue:
BUG GLM fit regularized ridge version has wrong "weights", vector alpha
fit_ridge has
```
        def fun(x):
            return -(self.loglike(x) / self.nobs - alpha * np.sum(x**2) / 2)

        def grad(x):
            return -(self.score(x) / self.nobs - alpha * x)
```

which is wrong if alpha is a vector, which is allowed especially for the case when we don't want to penalize the constant.

the negloglike needs to multiply by alpha before computing the squared sum, i.e.

`return -(self.loglike(x) / self.nobs - np.sum(alpha * x**2) / 2)`


Discussion:
@kshedden @bashtage 
This bug is already in 0.10+, and so while it shoudl should be fixed, it doesn't seem to be a blocker.  Moreover if alpha is an array, then fun(x) is a vector and it is likely that it will blow up. 
> Moreover if alpha is an array, then fun(x) is a vector and it is likely that it will blow up.

ok, I didn't think of that. Then this is a regular bug that can wait, not a high priority bug that silently returns wrong numbers

Resolving patch:
diff --git a/statsmodels/genmod/generalized_linear_model.py b/statsmodels/genmod/generalized_linear_model.py
--- a/statsmodels/genmod/generalized_linear_model.py
+++ b/statsmodels/genmod/generalized_linear_model.py
@@ -1286,7 +1286,7 @@ def _fit_ridge(self, alpha, start_params, method="newton-cg"):
             start_params = np.zeros(self.exog.shape[1])
 
         def fun(x):
-            return -(self.loglike(x) / self.nobs - alpha * np.sum(x**2) / 2)
+            return -(self.loglike(x) / self.nobs - np.sum(alpha * x**2) / 2)
 
         def grad(x):
             return -(self.score(x) / self.nobs - alpha * x)


Test patch:
diff --git a/statsmodels/genmod/tests/test_glm.py b/statsmodels/genmod/tests/test_glm.py
--- a/statsmodels/genmod/tests/test_glm.py
+++ b/statsmodels/genmod/tests/test_glm.py
@@ -1984,12 +1984,19 @@ def test_tweedie_EQL():
 
     # Series of ridge fits using gradients
     ev = (np.array([1.00186882, -0.99213087, 0.00717758, 0.50610942]),
-          np.array([0.98560143, -0.96976442,  0.00727526,  0.49749763]),
+          np.array([0.98560143, -0.96976442, 0.00727526, 0.49749763]),
           np.array([0.20643362, -0.16456528, 0.00023651, 0.10249308]))
     for j, alpha in enumerate([0.05, 0.5, 0.7]):
         model3 = sm.GLM(y, x, family=fam)
         result3 = model3.fit_regularized(L1_wt=0, alpha=alpha)
         assert_allclose(result3.params, ev[j], rtol=rtol, atol=atol)
+        result4 = model3.fit_regularized(L1_wt=0, alpha=alpha * np.ones(x.shape[1]))
+        assert_allclose(result4.params, result3.params, rtol=rtol, atol=atol)
+        alpha = alpha * np.ones(x.shape[1])
+        alpha[0] = 0
+        result5 = model3.fit_regularized(L1_wt=0, alpha=alpha)
+        assert not np.allclose(result5.params, result4.params, rtol=rtol, atol=atol)
+
 
 def test_tweedie_EQL_poisson_limit():
     # Test the limiting Poisson case of the Nelder/Pregibon/Tweedie
Repository: statsmodels/statsmodels

Issue:
Fix simple typo: variuos -> various
# Issue Type

[x] Bug (Typo)

# Steps to Replicate

1. Examine statsmodels/sandbox/multilinear.py.
2. Search for `variuos`.

# Expected Behaviour

1. Should read `various`.




Discussion:

Resolving patch:
diff --git a/statsmodels/sandbox/multilinear.py b/statsmodels/sandbox/multilinear.py
--- a/statsmodels/sandbox/multilinear.py
+++ b/statsmodels/sandbox/multilinear.py
@@ -284,7 +284,7 @@ def multigroup(pvals, groups, exact=True, keep_all=True, alpha=0.05):
     >>> url = url + "Rdatasets/csv/HistData/Guerry.csv"
     >>> df = pd.read_csv(url, index_col='dept')
 
-    evaluate the relationship between the variuos paramenters whith the Wealth
+    evaluate the relationship between the various paramenters whith the Wealth
     >>> pvals = multiOLS('Wealth', df)['adj_pvals', '_f_test']
 
     define the groups


Test patch:
Repository: statsmodels/statsmodels

Issue:
BUG: GLMGAM predict broken with pandas exog_smooth
https://stackoverflow.com/questions/60027463/how-do-i-get-predictions-on-new-data-using-a-fitted-statsmodel-glmgam-model

example works when using asarray for exog_smooth


Discussion:

Resolving patch:
diff --git a/statsmodels/gam/generalized_additive_model.py b/statsmodels/gam/generalized_additive_model.py
--- a/statsmodels/gam/generalized_additive_model.py
+++ b/statsmodels/gam/generalized_additive_model.py
@@ -158,6 +158,8 @@ def _tranform_predict_exog(self, exog=None, exog_smooth=None,
         exog_transformed : ndarray
             design matrix for the prediction
         """
+        if exog_smooth is not None:
+            exog_smooth = np.asarray(exog_smooth)
         exog_index = None
         if transform is False:
             # the following allows that either or both exog are not None
@@ -526,7 +528,7 @@ def __init__(self, endog, exog=None, smoother=None, alpha=0, family=None,
         if xnames_linear is None:
             xnames_linear = self.data_linear.xnames
         if exog is not None:
-            exog_linear = np.asarray(exog)
+            exog_linear = self.data_linear.exog
             k_exog_linear = exog_linear.shape[1]
         else:
             exog_linear = None


Test patch:
diff --git a/statsmodels/gam/tests/test_penalized.py b/statsmodels/gam/tests/test_penalized.py
--- a/statsmodels/gam/tests/test_penalized.py
+++ b/statsmodels/gam/tests/test_penalized.py
@@ -327,7 +327,7 @@ def setup_class(cls):
         nobs = data_mcycle['times'].shape[0]
         cc = CyclicCubicSplines(data_mcycle['times'].values, df=[6],
                                 constraints='center')
-        gam_cc = GLMGam(data_mcycle['accel'], np.ones((nobs, 1)),
+        gam_cc = GLMGam(data_mcycle['accel'], np.ones(nobs),
                         smoother=cc, alpha=1 / s_scale / 2)
         cls.res1 = gam_cc.fit(method='bfgs')
 
@@ -585,7 +585,8 @@ def test_predict(self):
 
         # linpred = res1.predict(self.exog[2:4], res1.model.smoother.x[2:4],
         #                        linear=True)
-        linpred = res1.predict(df_autos.iloc[2:4], res1.model.smoother.x[2:4],
+        xp = pd.DataFrame(res1.model.smoother.x[2:4])
+        linpred = res1.predict(df_autos.iloc[2:4], xp,
                                linear=True)
         assert_allclose(linpred, res2.linear_predictors[2:4],
                         rtol=self.rtol_fitted)
Repository: statsmodels/statsmodels

Issue:
pip doesn't recognize py2 deprecation from pypi
#### Describe the bug
As one [sees on pypi, v0.10.2](https://pypi.org/project/statsmodels/0.10.2/) is the last supported py2-version, under python 2 [version 0.11](https://pypi.org/project/statsmodels/0.11.0/) shouldn't be installed.



#### Code Sample, a copy-pastable example if possible


```bash
(my-env) $ pip install statsmodels
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. …
Collecting statsmodels
  Downloading statsmodels-0.11.0.tar.gz (15.4 MB)
```

#### Expected Output

Expected would be to having 0.10.2 installed.

#### Further Information
please compare [this issue on pandas-gbq](https://github.com/pydata/pandas-gbq/issues/306#issuecomment-568487470) with similar problems.



Discussion:

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -101,6 +101,7 @@
                'Programming Language :: Python :: 3.5',
                'Programming Language :: Python :: 3.6',
                'Programming Language :: Python :: 3.7',
+               'Programming Language :: Python :: 3.8',
                'Operating System :: OS Independent',
                'Intended Audience :: End Users/Desktop',
                'Intended Audience :: Developers',
@@ -344,7 +345,8 @@ def is_pure(self):
       setup_requires=SETUP_REQUIRES,
       install_requires=INSTALL_REQUIRES,
       extras_require=EXTRAS_REQUIRE,
-      zip_safe=False
+      zip_safe=False,
+      python_requires=">=3.5",
       )
 
 # Clean-up copied files


Test patch:
Repository: statsmodels/statsmodels

Issue:
BUG: acorr_breusch_godfrey error in regressions without const or exogs
In ARMA models without constant (or ARMAX without constant and without exogs) the acorr_breusch_godfrey method don't check if res.model.exog is None and try to use exogs returning a ValueError.

#### Code Sample, a copy-pastable example if possible


```python
from statsmodels.tsa.arima_model import ARMA
import pandas as pd
from statsmodels.datasets import sunspots

data = sunspots.load_pandas().data['SUNACTIVITY']

res = ARMA(data, (1,0)).fit(disp=False, trend = 'nc')

from statsmodels.stats.diagnostic import acorr_breusch_godfrey

acorr_breusch_godfrey(res, nlags=1)
```

#### Expected Output

N/A

#### Output of ``import statsmodels.api as sm; sm.show_versions()``

0.11.0 


Discussion:

Resolving patch:
diff --git a/statsmodels/stats/diagnostic.py b/statsmodels/stats/diagnostic.py
--- a/statsmodels/stats/diagnostic.py
+++ b/statsmodels/stats/diagnostic.py
@@ -802,7 +802,10 @@ def acorr_breusch_godfrey(res, nlags=None, store=False):
     nobs = xdall.shape[0]
     xdall = np.c_[np.ones((nobs, 1)), xdall]
     xshort = x[-nobs:]
-    exog = np.column_stack((exog_old, xdall))
+    if exog_old is None:
+        exog = xdall
+    else:
+        exog = np.column_stack((exog_old, xdall))
     k_vars = exog.shape[1]
 
     resols = OLS(xshort, exog).fit()


Test patch:
diff --git a/statsmodels/stats/tests/test_diagnostic.py b/statsmodels/stats/tests/test_diagnostic.py
--- a/statsmodels/stats/tests/test_diagnostic.py
+++ b/statsmodels/stats/tests/test_diagnostic.py
@@ -28,6 +28,7 @@
 from statsmodels.tools.tools import Bunch
 from statsmodels.tools.tools import add_constant
 from statsmodels.tsa.ar_model import AutoReg
+from statsmodels.tsa.arima_model import ARMA
 
 cur_dir = os.path.abspath(os.path.dirname(__file__))
 
@@ -287,6 +288,11 @@ def test_acorr_breusch_godfrey_multidim(self):
         with pytest.raises(ValueError, match='Model resid must be a 1d array'):
             smsdia.acorr_breusch_godfrey(res)
 
+    def test_acorr_breusch_godfrey_exogs(self):
+        data = sunspots.load_pandas().data['SUNACTIVITY']
+        res = ARMA(data, (1, 0)).fit(disp=False, trend='nc')
+        smsdia.acorr_breusch_godfrey(res, nlags=1)
+
     def test_acorr_ljung_box(self):
 
         # unit-test which may be useful later
Repository: statsmodels/statsmodels

Issue:
Note rather than warn about heteroscedastic-robust standard errors in regression summary
#### Is your feature request related to a problem? Please describe
Specifying heteroscedastic-robust errors doesn't warrant a warning in my view:
![image](https://user-images.githubusercontent.com/6076111/85773608-e3d50f80-b6d2-11ea-8a54-e95e9dcabf7f.png)

I understand that this is for equivalence with non-robust standard errors:
![image](https://user-images.githubusercontent.com/6076111/85774835-159aa600-b6d4-11ea-8f83-4ed3bad4e6bb.png)


#### Describe the solution you'd like
Make this a note rather than a warning, at least for robust (if not for non-robust as well).

#### Describe alternatives you have considered
None

#### Additional context
N/A


Discussion:
That's a bug. The warning note should show up only for nonrobust cov_type.
If the users chooses a robust cov_type, then we should assume they know what they are doing.
I misunderstood. It's not a bug. The message is correct.

The main reason I added it was as "educational tool" given that robust standard errors are not very common outside of economics and a very few other fields

But still, the message is not really necessary for robust cov_types
I agree that warning is the wrong term.

Resolving patch:
diff --git a/statsmodels/regression/linear_model.py b/statsmodels/regression/linear_model.py
--- a/statsmodels/regression/linear_model.py
+++ b/statsmodels/regression/linear_model.py
@@ -2680,7 +2680,7 @@ def summary(self, yname=None, xname=None, title=None, alpha=.05):
         if etext:
             etext = ["[{0}] {1}".format(i + 1, text)
                      for i, text in enumerate(etext)]
-            etext.insert(0, "Warnings:")
+            etext.insert(0, "Notes:")
             smry.add_extra_txt(etext)
 
         return smry


Test patch:
diff --git a/statsmodels/regression/tests/test_regression.py b/statsmodels/regression/tests/test_regression.py
--- a/statsmodels/regression/tests/test_regression.py
+++ b/statsmodels/regression/tests/test_regression.py
@@ -1015,7 +1015,7 @@ def test_summary_as_latex():
 %\\caption{OLS Regression Results}
 \\end{center}
 
-Warnings: \\newline
+Notes: \\newline
  [1] Standard Errors assume that the covariance matrix of the errors is correctly specified. \\newline
  [2] The condition number is large, 4.86e+09. This might indicate that there are \\newline
  strong multicollinearity or other numerical problems."""