internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: statsmodels/statsmodels

Issue:
DOC/BUG: duplicate section in docstrings (build error on Debian)
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=868977
last comment:
```
reading sources... [ 19%] generated/statsmodels.genmod.families.family.Family

Exception occurred:
  File "/usr/lib/python2.7/dist-packages/numpydoc/docscrape.py", line 343, in _parse
    raise ValueError(msg)
ValueError: The section Notes appears twice in the docstring of None in None.
```


AFAICS: `families.Family.weights` docstring has two section headers for  "Notes". The second should be deleted.

all other methods have at most one Notes section header.


I assume that usually this results in a warning and didn't stop the doc build.


Discussion:
@yarikoptic    this should be an easy fix for Debian

Resolving patch:
diff --git a/statsmodels/genmod/families/family.py b/statsmodels/genmod/families/family.py
--- a/statsmodels/genmod/families/family.py
+++ b/statsmodels/genmod/families/family.py
@@ -105,8 +105,6 @@ def starting_mu(self, y):
 
            \mu_0 = (Y + \overline{Y})/2
 
-        Notes
-        -----
         Only the Binomial family takes a different initial value.
         """
         return (y + y.mean())/2.
@@ -638,7 +636,7 @@ def resid_anscombe(self, endog, mu, scale=1.):
             The Anscombe residuals for the Gaussian family defined below
 
         Notes
-        --------
+        -----
         For the Gaussian distribution, Anscombe residuals are the same as
         deviance residuals.
 
@@ -765,7 +763,7 @@ def loglike(self, endog, mu, iweights=1., scale=1.):
             (endog,mu,iweights,scale) as defined below.
 
         Notes
-        --------
+        -----
         .. math::
 
            llf = -1 / scale * \sum_i *(Y_i / \mu_i+ \log(\mu_i)+
@@ -1001,7 +999,7 @@ def loglike(self, endog, mu, iweights=1, scale=1.):
             (endog,mu,iweights,scale) as defined below.
 
         Notes
-        --------
+        -----
         If the endogenous variable is binary:
 
         .. math::
diff --git a/statsmodels/genmod/families/links.py b/statsmodels/genmod/families/links.py
--- a/statsmodels/genmod/families/links.py
+++ b/statsmodels/genmod/families/links.py
@@ -77,11 +77,6 @@ def inverse_deriv(self, z):
         """
         Derivative of the inverse link function g^(-1)(z).
 
-        Notes
-        -----
-        This reference implementation gives the correct result but is
-        inefficient, so it can be overriden in subclasses.
-
         Parameters
         ----------
         z : array-like
@@ -92,6 +87,10 @@ def inverse_deriv(self, z):
         g'^(-1)(z) : array
             The value of the derivative of the inverse of the link function
 
+        Notes
+        -----
+        This reference implementation gives the correct result but is
+        inefficient, so it can be overriden in subclasses.
         """
         return 1 / self.deriv(self.inverse(z))
 
@@ -664,7 +663,7 @@ class probit(CDFLink):
     The probit (standard normal CDF) transform
 
     Notes
-    --------
+    -----
     g(p) = scipy.stats.norm.ppf(p)
 
     probit is an alias of CDFLink.


Test patch:
Repository: statsmodels/statsmodels

Issue:
BUG: medcouple fails for int arrays
reported on the mailing list

assigns inf to zero values
I guess we have to make `standardization` at least float dtype

```
>>> from statsmodels.stats.stattools import medcouple
>>> medcouple(np.arange(9))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "...\statsmodels\stats\stattools.py", line 451, in medcouple
    return np.apply_along_axis(_medcouple_1d, axis, y)
  File "C:\...\python-3.4.4.amd64\lib\site-packages\numpy\lib\shape_base.py", line 91, in apply_along_axis
    res = func1d(arr[tuple(i.tolist())], *args, **kwargs)
  File "...\statsmodels\stats\stattools.py", line 417, in _medcouple_1d
    standardization[is_zero] = np.inf
OverflowError: cannot convert float infinity to integer

>>> medcouple(np.arange(9)*1.0)
array(0.0)
>>> medcouple(np.random.randn(9))
array(0.03126347622708405)
>>> medcouple(np.random.randn(8))
array(0.0072095663649526616)
```


Discussion:

Resolving patch:
diff --git a/statsmodels/stats/stattools.py b/statsmodels/stats/stattools.py
--- a/statsmodels/stats/stattools.py
+++ b/statsmodels/stats/stattools.py
@@ -445,6 +445,7 @@ def medcouple(y, axis=0):
        distributions" Computational Statistics & Data Analysis, vol. 52, pp.
        5186-5201, August 2008.
     """
+    y = np.asarray(y, dtype=np.double)  # GH 4243
     if axis is None:
         return _medcouple_1d(y.ravel())
 


Test patch:
diff --git a/statsmodels/stats/tests/test_statstools.py b/statsmodels/stats/tests/test_statstools.py
--- a/statsmodels/stats/tests/test_statstools.py
+++ b/statsmodels/stats/tests/test_statstools.py
@@ -2,7 +2,7 @@
 # TODO: Test robust kurtosis
 import numpy as np
 import pandas as pd
-from numpy.testing import (assert_almost_equal, assert_raises)
+from numpy.testing import (assert_almost_equal, assert_raises, assert_equal)
 from statsmodels.stats.stattools import (omni_normtest, jarque_bera,
                                          durbin_watson, _medcouple_1d, medcouple,
                                          robust_kurtosis, robust_skewness)
@@ -192,11 +192,15 @@ def test_medcouple_symmetric(self):
         mc = medcouple(np.arange(5.0))
         assert_almost_equal(mc, 0)
 
-
     def test_medcouple_nonzero(self):
         mc = medcouple(np.array([1, 2, 7, 9, 10.0]))
         assert_almost_equal(mc, -0.3333333)
 
+    def test_medcouple_int(self):
+        # GH 4243
+        mc1 = medcouple(np.array([1, 2, 7, 9, 10]))
+        mc2 = medcouple(np.array([1, 2, 7, 9, 10.0]))
+        assert_equal(mc1, mc2)
 
     def test_medcouple_symmetry(self):
         x = np.random.standard_normal(100)
@@ -204,7 +208,6 @@ def test_medcouple_symmetry(self):
         mcn = medcouple(-x)
         assert_almost_equal(mcp + mcn, 0)
 
-
     def test_durbin_watson(self):
         x = np.random.standard_normal(100)
         dw = sum(np.diff(x)**2.0) / np.dot(x, x)
Repository: statsmodels/statsmodels

Issue:
BUG:  MICEData breaks if only one missing value
part of problems reported in #4369 
https://github.com/statsmodels/statsmodels/issues/4369#issuecomment-374942409
`TypeError: len() of unsized object` also occurs with pandas 0.19

The crucial line in mice is

File "/.../statsmodels/imputation/mice.py", line 395, in _store_changes
self.data[col].iloc[ix] = vals

There might be a problem with squeezing to a scalar, if there is only a single missing value in a column.

We need to add atleast_1d, i.e.
self.data[col].iloc[ix] = np.atleast_1d(vals)




Discussion:

Resolving patch:
diff --git a/statsmodels/imputation/mice.py b/statsmodels/imputation/mice.py
--- a/statsmodels/imputation/mice.py
+++ b/statsmodels/imputation/mice.py
@@ -392,7 +392,7 @@ def _store_changes(self, col, vals):
 
         ix = self.ix_miss[col]
         if len(ix) > 0:
-            self.data[col].iloc[ix] = vals
+            self.data[col].iloc[ix] = np.atleast_1d(vals)
 
 
     def update_all(self, n_iter=1):


Test patch:
diff --git a/statsmodels/imputation/tests/test_mice.py b/statsmodels/imputation/tests/test_mice.py
--- a/statsmodels/imputation/tests/test_mice.py
+++ b/statsmodels/imputation/tests/test_mice.py
@@ -339,6 +339,29 @@ def test_combine(self):
         assert_allclose(result.tvalues, tvalues, atol=1e-5)
 
 
+def test_micedata_miss1():
+    # test for #4375
+    np.random.seed(0)
+    data = pd.DataFrame(np.random.rand(50, 4))
+    data.columns = ['var1', 'var2', 'var3', 'var4']
+    # one column with a single missing value
+    data.iloc[1, 1] = np.nan
+    data.iloc[[1, 3], 2] = np.nan
+
+    data_imp = mice.MICEData(data)
+    data_imp.update_all()
+
+    assert_equal(data_imp.data.isnull().values.sum(), 0)
+
+    ix_miss = {'var1': np.array([], dtype=np.int64),
+                 'var2': np.array([1], dtype=np.int64),
+                 'var3': np.array([1, 3], dtype=np.int64),
+                 'var4': np.array([], dtype=np.int64)}
+
+    for k in ix_miss:
+        assert_equal(data_imp.ix_miss[k], ix_miss[k])
+
+
 if  __name__=="__main__":
     import pytest
     pytest.main([__file__, '-vvs', '-x', '--pdb'])
Repository: statsmodels/statsmodels

Issue:
BUG: coint ignores autolag
This was supposed to be fixed by #3966 which was closed without merging the fix.

currently autolag is hard coded to None in the coint call to adfuller and the keyword is ignored




Discussion:
I'm not sure how to unit test this, the only information that we get is a different test statistic, and no information about which number of lags are used.
We can still assume that autolag in adfuller is correct.  (i.e. no functional testing)
Note, the critical values depend on nobs and not on the number of lags used.
So, the critical values do not depend on `autolag`.

Resolving patch:
diff --git a/statsmodels/tsa/stattools.py b/statsmodels/tsa/stattools.py
--- a/statsmodels/tsa/stattools.py
+++ b/statsmodels/tsa/stattools.py
@@ -933,6 +933,10 @@ def coint(y0, y1, trend='c', method='aeg', maxlag=None, autolag='aic',
     Constant or trend is included in 1st stage regression, i.e. in
     cointegrating equation.
 
+    **Warning:** The autolag default has changed compared to statsmodels 0.8.
+    In 0.8 autolag was always None, no the keyword is used and defaults to
+    'aic'. Use `autolag=None` to avoid the lag search.
+
     Parameters
     ----------
     y1 : array_like, 1d
@@ -952,6 +956,13 @@ def coint(y0, y1, trend='c', method='aeg', maxlag=None, autolag='aic',
         keyword for `adfuller`, largest or given number of lags
     autolag : string
         keyword for `adfuller`, lag selection criterion.
+        * if None, then maxlag lags are used without lag search
+        * if 'AIC' (default) or 'BIC', then the number of lags is chosen
+          to minimize the corresponding information criterion
+        * 't-stat' based choice of maxlag.  Starts with maxlag and drops a
+          lag until the t-statistic on the last lag length is significant
+          using a 5%-sized test
+
     return_results : bool
         for future compatibility, currently only tuple available.
         If True, then a results instance is returned. Otherwise, a tuple
@@ -1017,7 +1028,7 @@ def coint(y0, y1, trend='c', method='aeg', maxlag=None, autolag='aic',
     res_co = OLS(y0, xx).fit()
 
     if res_co.rsquared < 1 - 100 * SQRTEPS:
-        res_adf = adfuller(res_co.resid, maxlag=maxlag, autolag=None,
+        res_adf = adfuller(res_co.resid, maxlag=maxlag, autolag=autolag,
                            regression='nc')
     else:
         import warnings


Test patch:
diff --git a/statsmodels/tsa/tests/test_stattools.py b/statsmodels/tsa/tests/test_stattools.py
--- a/statsmodels/tsa/tests/test_stattools.py
+++ b/statsmodels/tsa/tests/test_stattools.py
@@ -349,6 +349,13 @@ def test_coint():
             r1 = res1[i][2]
             assert_allclose(r1, r2, rtol=0, atol=6e-7)
 
+    # use default autolag #4490
+    res1_0 = coint(y[:, 0], y[:, 1], trend='ct', maxlag=4)
+    assert_allclose(res1_0[2], res_egranger['ct'][0][1:], rtol=0, atol=6e-7)
+    # the following is just a regression test
+    assert_allclose(res1_0[:2], [-13.992946638547112, 2.270898990540678e-27],
+                    rtol=1e-10, atol=1e-27)
+
 
 def test_coint_identical_series():
     nobs = 200
Repository: statsmodels/statsmodels

Issue:
BUG: UnboundLocalError: local variable 'mu' referenced before assignment
```
y = np.random.random(size=10)
model = sm.tsa.AR(y).fit(maxlag=1, method="mle", trend="nc", disp=False)
model.fittedvalues
```

Causes a: `UnboundLocalError: local variable 'mu' referenced before assignment`

The error appears in tsa/ar_model.py (line 224). Since k_trend is zero, mu is never set.
Adding an else clause to line 219 with mu=0 resolved the issue on my end.

```
if k_trend:
    mu = params[0]/(1-np.sum(params[k_trend:]))
else:
    mu = 0
```


Discussion:
@jf2 Thanks for the report

This needs to be fixed for 0.9. 
There might/should be an existing test case that we can reuse for this.

Resolving patch:
diff --git a/statsmodels/tsa/ar_model.py b/statsmodels/tsa/ar_model.py
--- a/statsmodels/tsa/ar_model.py
+++ b/statsmodels/tsa/ar_model.py
@@ -211,6 +211,8 @@ def predict(self, params, start=None, end=None, dynamic=False):
         if method == 'mle':  # use Kalman Filter to get initial values
             if k_trend:
                 mu = params[0]/(1-np.sum(params[k_trend:]))
+            else:
+                mu = 0
 
             # modifies predictedvalues in place
             if start < k_ar:


Test patch:
diff --git a/statsmodels/tsa/tests/test_ar.py b/statsmodels/tsa/tests/test_ar.py
--- a/statsmodels/tsa/tests/test_ar.py
+++ b/statsmodels/tsa/tests/test_ar.py
@@ -4,6 +4,7 @@
 import statsmodels.api as sm
 from statsmodels.compat.python import range
 from statsmodels.tsa.ar_model import AR
+from statsmodels.tsa.arima_model import ARMA
 from numpy.testing import (assert_almost_equal, assert_allclose, assert_)
 from statsmodels.tools.testing import assert_equal
 from .results import results_ar
@@ -85,7 +86,7 @@ class TestAROLSNoConstant(CheckARMixin):
     @classmethod
     def setup_class(cls):
         data = sm.datasets.sunspots.load()
-        cls.res1 = AR(data.endog).fit(maxlag=9,method='cmle',trend='nc')
+        cls.res1 = AR(data.endog).fit(maxlag=9, method='cmle', trend='nc')
         cls.res2 = results_ar.ARResultsOLS(constant=False)
 
     def test_predict(self):
@@ -114,7 +115,18 @@ def test_predict(self):
         assert_almost_equal(model.predict(params, start=308, end=327),
                 self.res2.FVOLSn15start312, DECIMAL_4)
 
-        #class TestARMLEConstant(CheckAR):
+    def test_mle(self):
+        # check predict with no constant, #3945
+        res1 = self.res1
+        endog = res1.model.endog
+        res0 = AR(endog).fit(maxlag=9, method='mle', trend='nc', disp=0)
+        assert_allclose(res0.fittedvalues[-10:], res0.fittedvalues[-10:],
+                        rtol=0.015)
+
+        res_arma = ARMA(endog, (9, 0)).fit(method='mle', trend='nc', disp=0)
+        assert_allclose(res0.params, res_arma.params, atol=5e-6)
+        assert_allclose(res0.fittedvalues[-10:], res_arma.fittedvalues[-10:],
+                        rtol=1e-4)
 
 
 class TestARMLEConstant(object):
Repository: statsmodels/statsmodels

Issue:
violinplot modification
Modifed violinplot in boxplots.py to add data check and gaussian_kde bw_method option



Discussion:

Resolving patch:
diff --git a/statsmodels/graphics/boxplots.py b/statsmodels/graphics/boxplots.py
--- a/statsmodels/graphics/boxplots.py
+++ b/statsmodels/graphics/boxplots.py
@@ -58,6 +58,8 @@ def violinplot(data, ax=None, labels=None, positions=None, side='both',
           - 'label_fontsize', MPL fontsize.  Adjusts fontsize only if given.
           - 'label_rotation', scalar.  Adjusts label rotation only if given.
                 Specify in degrees.
+          - 'bw_factor', Adjusts the scipy gaussian_kde kernel. default: None.
+                Options for scalar or callable.
 
     Returns
     -------
@@ -123,6 +125,11 @@ def violinplot(data, ax=None, labels=None, positions=None, side='both',
     .. plot:: plots/graphics_boxplot_violinplot.py
 
     """
+
+    if np.size(data) == 0:
+        msg = "No Data to make Violin: Try again!"
+        raise ValueError(msg)
+
     fig, ax = utils.create_mpl_ax(ax)
 
     data = list(map(np.asarray, data))
@@ -150,6 +157,7 @@ def violinplot(data, ax=None, labels=None, positions=None, side='both',
 
 def _single_violin(ax, pos, pos_data, width, side, plot_opts):
     """"""
+    bw_factor = plot_opts.get('bw_factor', None)
 
     def _violin_range(pos_data, plot_opts):
         """Return array with correct range, with which violins can be plotted."""
@@ -170,7 +178,7 @@ def _violin_range(pos_data, plot_opts):
 
     pos_data = np.asarray(pos_data)
     # Kernel density estimate for data at this position.
-    kde = gaussian_kde(pos_data)
+    kde = gaussian_kde(pos_data, bw_method=bw_factor)
 
     # Create violin for pos, scaled to the available space.
     xvals = _violin_range(pos_data, plot_opts)


Test patch:
diff --git a/statsmodels/graphics/tests/test_boxplots.py b/statsmodels/graphics/tests/test_boxplots.py
--- a/statsmodels/graphics/tests/test_boxplots.py
+++ b/statsmodels/graphics/tests/test_boxplots.py
@@ -33,6 +33,16 @@ def test_violinplot_beanplot():
 
     plt.close(fig)
 
+    fig = plt.figure()
+    ax = fig.add_subplot(111)
+    violinplot(age, ax=ax, labels=labels,
+               plot_opts={'cutoff_val':5, 'cutoff_type':'abs',
+                          'label_fontsize':'small',
+                          'label_rotation':30,
+                          'bw_factor':.2})
+
+    plt.close(fig)
+
 
     fig = plt.figure()
     ax = fig.add_subplot(111)
Repository: statsmodels/statsmodels

Issue:
BUG: GLM fit_irls  wls_method hard coded
I don't know what's going on here.

In current master fit method is hard coded as 'lstsq'

```
            wls_results = reg_tools._MinimalWLS(
                    wlsendog,
                    wlsexog,
                    self.weights).fit(method='lstsq')
```

I thought I had changed this to using wls_method option

https://github.com/statsmodels/statsmodels/pull/4021/files#diff-724e15f4019b2a24906f6b271d67e370L1103


needs checking to see what's going on (or if my memory is wrong)


Discussion:
I guess this is a mistake in a rebase, maybe merge conflict resolution

this commit reverted the wls_method
https://github.com/statsmodels/statsmodels/commit/78dcce22993de5b11e07f8636e3e474c535707ac

Whoops. I have no idea why I would’ve done that.

I’ll make a PR ago revert.

Resolving patch:
diff --git a/statsmodels/genmod/generalized_linear_model.py b/statsmodels/genmod/generalized_linear_model.py
--- a/statsmodels/genmod/generalized_linear_model.py
+++ b/statsmodels/genmod/generalized_linear_model.py
@@ -1132,7 +1132,7 @@ def _fit_irls(self, start_params=None, maxiter=100, tol=1e-8,
             wls_results = reg_tools._MinimalWLS(
                     wlsendog,
                     wlsexog,
-                    self.weights).fit(method='lstsq')
+                    self.weights).fit(method=wls_method)
             lin_pred = np.dot(self.exog, wls_results.params)
             lin_pred += self._offset_exposure
             mu = self.family.fitted(lin_pred)


Test patch:
Repository: statsmodels/statsmodels

Issue:
BUG  VAR.predict is broken
The following error says that TimeSeriesModel (super or super's super object) has no `_get_predict_start`. But VAR needs it. I only find `_get_predict_index` in TimeSeriesModel.

 File "/Users/william/Folders/临时文件夹/script.py", line 17, in <module>
    print(results.model.predict(['2015']))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/statsmodels/tsa/vector_ar/var_model.py", line 535, in predict
    start = self._get_predict_start(start, lags)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/statsmodels/tsa/vector_ar/var_model.py", line 529, in _get_predict_start
    return super(VAR, self)._get_predict_start(start)
AttributeError: 'super' object has no attribute '_get_predict_start'



Discussion:
I think, just change one of them
@Freakwill Thanks for reporting.  This is most likely a refactoring victim, and AFAICS, there is no test coverage for it.

Note `predict` is a model method and the first argument is params. There is no corresponding results `predict` method for insample dynamic prediction. I only have seen the `forecast` method in use for VAR.
We need a unit test case to check if the function has just been renamed or has now a incompatible behavior.

You should be able to use VARMAX as workaround for dynamic insample prediction in a VAR model. VARMAXResults has predict and get_prediction and a dynamic keyword.


Yes, unfortunately the use of the new date / time handling in https://github.com/statsmodels/statsmodels/commit/6c1f892c6032ea0e3422534e4bc83d6a7735af21#diff-e2d6024b9d0ed6c0410bf2fffc1f1254R342 was overridden subsequently. I can make a PR to replace the working behavior.

Resolving patch:
diff --git a/statsmodels/tsa/vector_ar/var_model.py b/statsmodels/tsa/vector_ar/var_model.py
--- a/statsmodels/tsa/vector_ar/var_model.py
+++ b/statsmodels/tsa/vector_ar/var_model.py
@@ -523,17 +523,18 @@ def __init__(self, endog, exog=None, dates=None, freq=None,
         self.neqs = self.endog.shape[1]
         self.n_totobs = len(endog)
 
-    def _get_predict_start(self, start, k_ar):
-        if start is None:
-            start = k_ar
-        return super(VAR, self)._get_predict_start(start)
-
     def predict(self, params, start=None, end=None, lags=1, trend='c'):
         """
         Returns in-sample predictions or forecasts
         """
-        start = self._get_predict_start(start, lags)
-        end, out_of_sample = self._get_predict_end(end)
+        params = np.array(params)
+
+        if start is None:
+            start = lags
+
+        # Handle start, end
+        start, end, out_of_sample, prediction_index = (
+            self._get_prediction_index(start, end))
 
         if end < start:
             raise ValueError("end is before start")


Test patch:
Repository: statsmodels/statsmodels

Issue:
BUG: k_params in SARIMAX
statsmodels.tsa.statespace.sarimax.py (version 0.9.0)

class SARIMA
def __init__(...)
...
        # Number of parameters
        self.k_params = (
            self.k_ar_params + self.k_ma_params +
            self.k_seasonal_ar_params + self.k_seasonal_ar_params +
            self.k_trend +
            self.measurement_error + 1
        )
...

self.k_seasonal_ar_params + self.k_seasonal_ar_params

should be self.k_seasonal_ar_params + self.k_seasonal_ma_params ?


Discussion:
Thanks very much! You're absolutely right.

Resolving patch:
diff --git a/statsmodels/tsa/statespace/sarimax.py b/statsmodels/tsa/statespace/sarimax.py
--- a/statsmodels/tsa/statespace/sarimax.py
+++ b/statsmodels/tsa/statespace/sarimax.py
@@ -476,7 +476,7 @@ def __init__(self, endog, exog=None, order=(1, 0, 0),
         # Number of parameters
         self.k_params = (
             self.k_ar_params + self.k_ma_params +
-            self.k_seasonal_ar_params + self.k_seasonal_ar_params +
+            self.k_seasonal_ar_params + self.k_seasonal_ma_params +
             self.k_trend +
             self.measurement_error +
             int(not self.concentrate_scale)


Test patch: