internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: statsmodels/statsmodels

Issue:
ENH: GLM add warning if negbin parameter is default
I have seen several examples, e.g. on stackoverflow, where users do not set the GLM-negbin parameter.
I guess users often assume that the extra parameter is estimated.

A warning if the default parameter is used would be useful to reduce the risk that user misunderstand the model.




Discussion:

Resolving patch:
diff --git a/statsmodels/genmod/families/family.py b/statsmodels/genmod/families/family.py
--- a/statsmodels/genmod/families/family.py
+++ b/statsmodels/genmod/families/family.py
@@ -8,6 +8,7 @@
 
 
 import inspect
+import warnings
 
 import numpy as np
 from scipy import special, stats
@@ -1310,6 +1311,9 @@ class NegativeBinomial(Family):
 
     def __init__(self, link=None, alpha=1.):
         self.alpha = 1. * alpha  # make it at least float
+        if alpha is self.__init__.__defaults__[1]:
+            warnings.warn("Negative binomial dispersion parameter alpha not "
+                          f"set. Using default value alpha={alpha}.")
         if link is None:
             link = L.Log()
         super(NegativeBinomial, self).__init__(


Test patch:
diff --git a/statsmodels/discrete/tests/test_discrete.py b/statsmodels/discrete/tests/test_discrete.py
--- a/statsmodels/discrete/tests/test_discrete.py
+++ b/statsmodels/discrete/tests/test_discrete.py
@@ -1636,6 +1636,22 @@ def test_issue_341():
     np.testing.assert_equal(res1.predict(x[None]).shape, (1,7))
 
 
+def test_negative_binomial_default_alpha_param():
+    with pytest.warns(UserWarning, match='Negative binomial'
+                      ' dispersion parameter alpha not set'):
+        sm.families.NegativeBinomial()
+    with pytest.warns(UserWarning, match='Negative binomial'
+                      ' dispersion parameter alpha not set'):
+        sm.families.NegativeBinomial(link=sm.families.links.nbinom(alpha=1.0))
+    with warnings.catch_warnings():
+        warnings.simplefilter("error")
+        sm.families.NegativeBinomial(alpha=1.0)
+    with warnings.catch_warnings():
+        warnings.simplefilter("error")
+        sm.families.NegativeBinomial(link=sm.families.links.nbinom(alpha=1.0),
+                                     alpha=1.0)
+
+
 def test_iscount():
     X = np.random.random((50, 10))
     X[:,2] = np.random.randint(1, 10, size=50)
Repository: statsmodels/statsmodels

Issue:
Gaussian Copula: Add parameter allow_singular
The Gaussian Copula module is using scipy.stats package for the multivariate normal distribution. There is a parameter allow_singular=False which allows the covariance matrix to not have full rank. Unfortunately this parameter is not passed to the Copula interface.

It could be added here:
https://github.com/statsmodels/statsmodels/blob/55aff1e2268f56cc26b45ddcc1f44e5f924f8d05/statsmodels/distributions/copula/elliptical.py#L153

I currently wrote an inherited class, and singular matrixes for the copula seem to work fine.


Discussion:

Resolving patch:
diff --git a/statsmodels/distributions/copula/elliptical.py b/statsmodels/distributions/copula/elliptical.py
--- a/statsmodels/distributions/copula/elliptical.py
+++ b/statsmodels/distributions/copula/elliptical.py
@@ -124,6 +124,8 @@ class GaussianCopula(EllipticalCopula):
         matrix is the identity matrix.
     k_dim : int
         Dimension, number of components in the multivariate random variable.
+    allow_singular : bool
+        Allow singular correlation matrix
 
     Notes
     -----
@@ -141,7 +143,7 @@ class GaussianCopula(EllipticalCopula):
 
     """
 
-    def __init__(self, corr=None, k_dim=2):
+    def __init__(self, corr=None, k_dim=2, allow_singular=False):
         super().__init__(k_dim=k_dim)
         if corr is None:
             corr = np.eye(k_dim)
@@ -150,7 +152,8 @@ def __init__(self, corr=None, k_dim=2):
 
         self.corr = np.asarray(corr)
         self.distr_uv = stats.norm
-        self.distr_mv = stats.multivariate_normal(cov=corr)
+        self.distr_mv = stats.multivariate_normal(
+            cov=corr, allow_singular=allow_singular)
 
     def dependence_tail(self, corr=None):
         """


Test patch:
Repository: statsmodels/statsmodels

Issue:
DIckey Fuller test breaks on constant values
#### Describe the bug

Dickey-Fuller test gives error when testing with constant values.

#### Code Sample, a copy-pastable example if possible


```python
adfuller(np.full(8, 5.0))
```
`RuntimeWarning: divide by zero encountered in log
  llf = -nobs2*np.log(2*np.pi) - nobs2*np.log(ssr / nobs) - nobs2`

#### Expected Output

I have expected that this series is stationary

#### Output of ``import statsmodels.api as sm; sm.show_versions()``

<details>

INSTALLED VERSIONS
------------------
Python: 3.10.6.final.0
OS: Linux 5.15.0-50-generic #56-Ubuntu SMP Tue Sep 20 13:23:26 UTC 2022 x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

statsmodels
===========

Installed: 0.13.2 (/home/michael/.local/lib/python3.10/site-packages/statsmodels)

Required Dependencies
=====================

cython: Not installed
numpy: 1.23.3 (/home/michael/.local/lib/python3.10/site-packages/numpy)
scipy: 1.9.2 (/home/michael/.local/lib/python3.10/site-packages/scipy)
pandas: 1.5.0 (/home/michael/.local/lib/python3.10/site-packages/pandas)
    dateutil: 2.8.2 (/home/michael/.local/lib/python3.10/site-packages/dateutil)
patsy: 0.5.3 (/home/michael/.local/lib/python3.10/site-packages/patsy)

Optional Dependencies
=====================

matplotlib: 3.6.1 (/home/michael/.local/lib/python3.10/site-packages/matplotlib)
    backend: module://matplotlib_inline.backend_inline 
cvxopt: Not installed
joblib: Not installed

Developer Tools
================

IPython: 8.5.0 (/home/michael/.local/lib/python3.10/site-packages/IPython)
    jinja2: 3.1.2 (/home/michael/.local/lib/python3.10/site-packages/jinja2)
sphinx: Not installed
    pygments: 2.13.0 (/home/michael/.local/lib/python3.10/site-packages/pygments)
pytest: Not installed
virtualenv: 20.16.5 (/home/michael/.local/lib/python3.10/site-packages/virtualenv)

</details>



Discussion:
The test isn't well defined if the residual variance is 0. Ideally this would be caught and a meaningful exception raised. 

Resolving patch:
diff --git a/statsmodels/tsa/stattools.py b/statsmodels/tsa/stattools.py
--- a/statsmodels/tsa/stattools.py
+++ b/statsmodels/tsa/stattools.py
@@ -268,6 +268,9 @@ def adfuller(
     store = bool_like(store, "store")
     regresults = bool_like(regresults, "regresults")
 
+    if x.max() == x.min():
+        raise ValueError("Invalid input, x is constant")
+
     if regresults:
         store = True
 


Test patch:
diff --git a/statsmodels/tsa/tests/test_stattools.py b/statsmodels/tsa/tests/test_stattools.py
--- a/statsmodels/tsa/tests/test_stattools.py
+++ b/statsmodels/tsa/tests/test_stattools.py
@@ -188,6 +188,12 @@ def test_store_str(self):
         )
 
 
+@pytest.mark.parametrize("x", [np.full(8, 5.0)])
+def test_adfuller_resid_variance_zero(x):
+    with pytest.raises(ValueError):
+        adfuller(x)
+
+
 class CheckCorrGram:
     """
     Set up for ACF, PACF tests.
Repository: statsmodels/statsmodels

Issue:
DOC: Add a better description of the plot generated by plot_fit
#### Is your feature request related to a problem? Please describe
The documentation of the function `plot_fit` does not describe the features of the plots it generates. The value in the y-axis and the fitted value are covered by the legend but there's no description of the vertical black lines.

#### Describe the solution you'd like
I'd like the docstring to include the meaning of the features of the plot, and especially the one of the vertical black lines.

#### Describe alternatives you have considered
I had to look for the information somewhere else. There's an answer here: https://stackoverflow.com/questions/30327329/vertical-lines-in-plot-fit-graphic-of-python-ststsmodels-package


Discussion:
We are always happy to have a PR to improve documentation.

Resolving patch:
diff --git a/statsmodels/graphics/regressionplots.py b/statsmodels/graphics/regressionplots.py
--- a/statsmodels/graphics/regressionplots.py
+++ b/statsmodels/graphics/regressionplots.py
@@ -94,7 +94,7 @@ def plot_fit(results, exog_idx, y_true=None, ax=None, vlines=True, **kwargs):
         If given, this subplot is used to plot in instead of a new figure being
         created.
     vlines : bool, optional
-        If this not True, then the uncertainty of the fit is not
+        If this not True, then the uncertainty (pointwise prediction intervals) of the fit is not
         plotted.
     **kwargs
         The keyword arguments are passed to the plot command for the fitted
@@ -123,7 +123,8 @@ def plot_fit(results, exog_idx, y_true=None, ax=None, vlines=True, **kwargs):
     >>> model = sm.OLS(y, X)
     >>> results = model.fit()
 
-    Create a plot just for the variable 'Poverty':
+    Create a plot just for the variable 'Poverty.'
+    Note that vertical bars representing uncertainty are plotted since vlines is true
 
     >>> fig, ax = plt.subplots()
     >>> fig = sm.graphics.plot_fit(results, 0, ax=ax)


Test patch:
Repository: statsmodels/statsmodels

Issue:
BUG/Maint: numpy additional pre-failures 
we have two new pre-failures, AFAICS after the numpy main version was updated

FAILED statsmodels/base/tests/test_generic_methods.py::TestGenericNegativeBinomial::test_zero_collinear
FAILED statsmodels/discrete/tests/test_discrete.py::test_unchanging_degrees_of_freedom

The second test fails most likely because of (assigning a non-scalar array to a single array element in negbin _hessian_nb2
https://github.com/numpy/numpy/issues/10404

The first test has deprecation warnings in a unit test that no warnings should show up

It's not clear to me why negbin _hessian_nb2 does not cause many more test failures.
(I will try to look into this later today)


Discussion:
numpy only added a deprecation

The discrete unit test raises explicitly by setting `warnings.simplefilter('error')`
I don't know why the warnings filter is set to error.
I had added it in #4235 but no explanation in PR comments. I don't know which warning it wanted to catch as test failure.

quickfix
add `.squeeze` in hessian_nb2
proper fix would be to rewrite the loop.




Resolving patch:
diff --git a/statsmodels/discrete/discrete_model.py b/statsmodels/discrete/discrete_model.py
--- a/statsmodels/discrete/discrete_model.py
+++ b/statsmodels/discrete/discrete_model.py
@@ -3543,7 +3543,7 @@ def _hessian_nb2(self, params):
                 if j > i:
                     continue
                 hess_arr[i,j] = np.sum(-exog[:,i,None] * exog[:,j,None] *
-                                       const_arr, axis=0)
+                                       const_arr, axis=0).squeeze()
         tri_idx = np.triu_indices(dim, k=1)
         hess_arr[tri_idx] = hess_arr.T[tri_idx]
 


Test patch:
Repository: statsmodels/statsmodels

Issue:
DOC: Statespace differencing tool docstrings are incorrect
Very minor, but the arguments to the function (`k_diff`, `k_seasonal_diff`) are not accurately reflected in the [docstring](https://github.com/statsmodels/statsmodels/blob/main/statsmodels/tsa/statespace/tools.py#L259-L263).


Discussion:

Resolving patch:
diff --git a/statsmodels/tsa/statespace/tools.py b/statsmodels/tsa/statespace/tools.py
--- a/statsmodels/tsa/statespace/tools.py
+++ b/statsmodels/tsa/statespace/tools.py
@@ -256,9 +256,9 @@ def diff(series, k_diff=1, k_seasonal_diff=None, seasonal_periods=1):
     ----------
     series : array_like
         The series to be differenced.
-    diff : int, optional
+    k_diff : int, optional
         The number of simple differences to perform. Default is 1.
-    seasonal_diff : int or None, optional
+    k_seasonal_diff : int or None, optional
         The number of seasonal differences to perform. Default is no seasonal
         differencing.
     seasonal_periods : int, optional


Test patch:
Repository: statsmodels/statsmodels

Issue:
Typo in CanCorr documentation and docstring
CanCorr's documentation and docstring say that CanCorr has attributes x_cancoeff and y_cancoeff. However, they should say x_cancoef and y_cancoef. Should I submit a PR?


Discussion:
Please do submit a PR.

Resolving patch:
diff --git a/statsmodels/multivariate/cancorr.py b/statsmodels/multivariate/cancorr.py
--- a/statsmodels/multivariate/cancorr.py
+++ b/statsmodels/multivariate/cancorr.py
@@ -34,9 +34,9 @@ class CanCorr(Model):
         See Parameters.
     cancorr : ndarray
         The canonical correlation values
-    y_cancoeff : ndarray
+    y_cancoef : ndarray
         The canonical coefficients for endog
-    x_cancoeff : ndarray
+    x_cancoef : ndarray
         The canonical coefficients for exog
 
     References


Test patch:
Repository: statsmodels/statsmodels

Issue:
DOC: `linear_regression_diagnostics_plots` in `examples` broken
https://www.statsmodels.org/stable/examples/notebooks/generated/linear_regression_diagnostics_plots.html

<img width="939" alt="image" src="https://github.com/statsmodels/statsmodels/assets/77875500/3e9720e3-2b80-45f1-b73f-a954d0599bef">



Discussion:

Resolving patch:
diff --git a/examples/python/linear_regression_diagnostics_plots.py b/examples/python/linear_regression_diagnostics_plots.py
--- a/examples/python/linear_regression_diagnostics_plots.py
+++ b/examples/python/linear_regression_diagnostics_plots.py
@@ -109,7 +109,7 @@ def __init__(
         >>> df = pd.DataFrame({'x':x, 'y':y})
         >>> res = smf.ols(formula= "y ~ x", data=df).fit()
         >>> cls = Linear_Reg_Diagnostic(res)
-        >>> cls(plot_context="seaborn-paper")     
+        >>> cls(plot_context="seaborn-v0_8")     
 
         In case you do not need all plots you can also independently make an individual plot/table
         in following ways
@@ -143,7 +143,7 @@ def __init__(
         self.cooks_distance = influence.cooks_distance[0]
         self.nparams = len(self.results.params)
 
-    def __call__(self, plot_context='seaborn-paper'):
+    def __call__(self, plot_context='seaborn-v0_8'):
         # print(plt.style.available)
         with plt.style.context(plot_context):
             fig, ax = plt.subplots(nrows=2, ncols=2, figsize=(10, 10))


Test patch:
Repository: statsmodels/statsmodels

Issue:
DOC: ztest documentation says that it returns pvalue of the t-test .
Just a minor mistype I think. Was a little confused when read that z-test function returns t-test p-value.
From the ztest docstring:

Returns
-------
tstat : float
    test statistic
pvalue : float
    pvalue of the t-test


Discussion:
Hi @Shakoth, could you please paste the specific document page you mentioned? If it still exists, I'd like to fix it.
Sure, here it is:
https://www.statsmodels.org/dev/generated/statsmodels.stats.weightstats.ztest.html

The docstring in the code says the same (my version is 0.14.0)

Resolving patch:
diff --git a/statsmodels/stats/weightstats.py b/statsmodels/stats/weightstats.py
--- a/statsmodels/stats/weightstats.py
+++ b/statsmodels/stats/weightstats.py
@@ -498,7 +498,7 @@ def ztest_mean(self, value=0, alternative="two-sided"):
         tstat : float
             test statistic
         pvalue : float
-            pvalue of the t-test
+            pvalue of the z-test
 
         Notes
         -----
@@ -1524,7 +1524,7 @@ def ztest(
     tstat : float
         test statistic
     pvalue : float
-        pvalue of the t-test
+        pvalue of the z-test
 
     Notes
     -----


Test patch:
Repository: statsmodels/statsmodels

Issue:
Should it not be `positive`?
Should it not be `positive`?

https://github.com/statsmodels/statsmodels/blob/c2194ff6052362ffc09da7d00a5a9e99ecb11205/statsmodels/tsa/statespace/dynamic_factor_mq.py#L2828


Discussion:

Resolving patch:
diff --git a/statsmodels/tsa/statespace/dynamic_factor_mq.py b/statsmodels/tsa/statespace/dynamic_factor_mq.py
--- a/statsmodels/tsa/statespace/dynamic_factor_mq.py
+++ b/statsmodels/tsa/statespace/dynamic_factor_mq.py
@@ -2825,7 +2825,7 @@ def _em_maximization_step(self, res, params0, mstep_method=None):
                 f_A = cho_solve(cho_factor(A), B.T).T
             except LinAlgError:
                 # Fall back to general solver if there are problems with
-                # postive-definiteness
+                # positive-definiteness
                 f_A = np.linalg.solve(A, B.T).T
 
             f_Q = (C - f_A @ B.T) / nobs
@@ -2895,7 +2895,7 @@ def _em_maximization_obs_nonmissing(self, res, Eaa, a, compute_H=False):
                 Lambda[i, factor_ix] = cho_solve(cho_factor(A), B.T).T
             except LinAlgError:
                 # Fall back to general solver if there are problems with
-                # postive-definiteness
+                # positive-definiteness
                 Lambda[i, factor_ix] = np.linalg.solve(A, B.T).T
 
         # Compute new obs cov
@@ -2949,7 +2949,7 @@ def _em_maximization_obs_missing(self, res, Eaa, a, compute_H=False):
                 Lambda[i, factor_ix] = cho_solve(cho_factor(Ai), Bi.T).T
             except LinAlgError:
                 # Fall back to general solver if there are problems with
-                # postive-definiteness
+                # positive-definiteness
                 Lambda[i, factor_ix] = np.linalg.solve(Ai, Bi.T).T
 
         # Compute unrestricted design for quarterly
@@ -2993,7 +2993,7 @@ def _em_maximization_obs_missing(self, res, Eaa, a, compute_H=False):
                     restricted = unrestricted - AiiRT @ RAiiRTiR @ unrestricted
                 except LinAlgError:
                     # Fall back to slower method if there are problems with
-                    # postive-definiteness
+                    # positive-definiteness
                     Aii = np.linalg.inv(Ai)
                     unrestricted = (BiQ @ Aii)[0]
                     RARi = np.linalg.inv(R @ Aii @ R.T)


Test patch:
Repository: statsmodels/statsmodels

Issue:
Suggesting updates on the doc of `statsmodels.tsa.ar_model.AutoReg`
Hi,

We are an academic team of software engineering researchers from a university working on automated program analysis techniques to improve API documentation quality, ultimately contributing to improving data science software development practices. we would like to keep anonymity for the purpose of double-blind paper reviewing.

We discover an inconsistency issue between documentation and code in the class [`statsmodels.tsa.ar_model.AutoReg`](https://www.statsmodels.org/devel/generated/statsmodels.tsa.ar_model.AutoReg.html#statsmodels.tsa.ar_model.AutoReg). As mentioned in the description of parameter `deterministic`:

> **deterministic** : DeterministicProcess
A deterministic process. If provided, trend and seasonal are ignored. **A warning is raised if trend is not “n” and seasonal is not False.**

This means that the warning is triggered only when both conditions are met at the same time. However, the most relevant source code snippet looks like this:

```python
if deterministic is not None and (
    self._trend != "n" or self._seasonal
):
    warnings.warn(
        'When using deterministic, trend must be "n" and '
        "seasonal must be False.",
        SpecificationWarning,
        stacklevel=2,
    )
```

The actual code execution logic is that as long as one of the conditions is met, a warning will be triggered.

Maybe you can change the line in the aforementioned doc to "A warning is raised if trend is not “n” **or** seasonal is not False.".



Discussion:
Happy to accept a PR.

Resolving patch:
diff --git a/statsmodels/tsa/ar_model.py b/statsmodels/tsa/ar_model.py
--- a/statsmodels/tsa/ar_model.py
+++ b/statsmodels/tsa/ar_model.py
@@ -144,7 +144,7 @@ class AutoReg(tsa_model.TimeSeriesModel):
         If 'raise', an error is raised. Default is 'none'.
     deterministic : DeterministicProcess
         A deterministic process.  If provided, trend and seasonal are ignored.
-        A warning is raised if trend is not "n" and seasonal is not False.
+        A warning is raised if trend is not "n" or seasonal is not False.
     old_names : bool
         Flag indicating whether to use the v0.11 names or the v0.12+ names.
 


Test patch:
Repository: facebookresearch/fairscale

Issue:
Async broadcast request in OSS
## 🚀 Feature
Make sure that the broadcast request to sync shards are async, and not blocking the main thread

## Motivation
OSS is currently too slow

## Pitch
Easy fix, no compromise

## Alternatives
Stay slow for no reason

## Additional context
Clear bottleneck when tracing execution


Discussion:

Resolving patch:
diff --git a/fairscale/optim/oss.py b/fairscale/optim/oss.py
--- a/fairscale/optim/oss.py
+++ b/fairscale/optim/oss.py
@@ -106,11 +106,14 @@ def step(self, closure: Optional[Callable[[], float]] = None, **kwargs: Any) ->
         # Run the optimizer step on this shard only
         loss = self.optim.step(closure=closure, **kwargs)  # type: ignore
 
-        # Sync all the states
+        # Sync all the states. Broadcast requests are issued async, we check completeness before moving on
+        requests = []
         for rank, param_groups in enumerate(self.partition_parameters()):
             for param_group in param_groups:
                 for param in param_group["params"]:
-                    dist.broadcast(tensor=param, src=rank, group=self.group)
+                    requests.append(dist.broadcast(tensor=param, src=rank, group=self.group, async_op=True))
+
+        _ = list(map(lambda x: x.wait(), requests))
         return loss
 
     def local_state_dict(self) -> dict:


Test patch: