Repository: statsmodels/statsmodels

Issue:
BUG/TST: recursive_ls test failures, pandas incompatibility
recursive_ls has been failing for some time, first in pre and now in many environments which seem all to use pandas 2.0.0


AFAICS, the problem is with indexing into dataframe.values in 
https://github.com/statsmodels/statsmodels/blob/main/statsmodels/regression/recursive_ls.py#L99
`endog.values[:, 1:] = self._q_matrix[:, 0]`

It looks like this is not an inplace assignement anymore.

If I replace the affine constraint by a linear constraint in test_constraints_stata, then I get the same results as in the failing unit tests
`mod = RecursiveLS(endog, exog, constraints='m1 + unemp = 0')`
instead of `mod = RecursiveLS(endog, exog, constraints='m1 + unemp = 1')`

However, I don't know how to fix this.
I tried replacing indexing with `values` by `iloc`, but I did not get it to work with pandas 1.4.0 (my current dev environment.



Discussion:
@ChadFulton @bashtage 

any ideas how to fix this.

(I also don't really understand what that assignment is supposed to do and what shapes it is supposed to support.)
Thanks, and sorry I didn’t see these test failures earlier! I’ll take a look tomorrow 

Resolving patch:
diff --git a/statsmodels/regression/recursive_ls.py b/statsmodels/regression/recursive_ls.py
--- a/statsmodels/regression/recursive_ls.py
+++ b/statsmodels/regression/recursive_ls.py
@@ -91,12 +91,15 @@ def __init__(self, endog, exog, constraints=None, **kwargs):
             self._r_matrix, self._q_matrix = LC.coefs, LC.constants
             self.k_constraints = self._r_matrix.shape[0]
 
-            constraint_endog = np.zeros((len(endog), len(self._r_matrix)))
+            nobs = len(endog)
+            constraint_endog = np.zeros((nobs, len(self._r_matrix)))
             if endog_using_pandas:
                 constraint_endog = pd.DataFrame(constraint_endog,
                                                 index=endog.index)
                 endog = concat([endog, constraint_endog], axis=1)
-                endog.values[:, 1:] = self._q_matrix[:, 0]
+                # Complexity needed to handle multiple version of pandas
+                # Pandas >= 2 can use endog.iloc[:, 1:] = self._q_matrix.T
+                endog.iloc[:, 1:] = np.tile(self._q_matrix.T, (nobs, 1))
             else:
                 endog[:, 1:] = self._q_matrix[:, 0]
 


Test patch:
diff --git a/statsmodels/regression/tests/test_recursive_ls.py b/statsmodels/regression/tests/test_recursive_ls.py
--- a/statsmodels/regression/tests/test_recursive_ls.py
+++ b/statsmodels/regression/tests/test_recursive_ls.py
@@ -4,22 +4,22 @@
 Author: Chad Fulton
 License: Simplified-BSD
 """
+import os
 
 import numpy as np
+from numpy.testing import assert_allclose, assert_equal, assert_raises
 import pandas as pd
 import pytest
 from scipy.stats import norm
-import os
 
 from statsmodels.datasets import macrodata
-from statsmodels.regression.linear_model import OLS
 from statsmodels.genmod.api import GLM
-from statsmodels.tools.eval_measures import aic, bic
+from statsmodels.regression.linear_model import OLS
 from statsmodels.regression.recursive_ls import RecursiveLS
 from statsmodels.stats.diagnostic import recursive_olsresiduals
 from statsmodels.tools import add_constant
+from statsmodels.tools.eval_measures import aic, bic
 from statsmodels.tools.sm_exceptions import ValueWarning
-from numpy.testing import assert_equal, assert_raises, assert_allclose
 
 current_path = os.path.dirname(os.path.abspath(__file__))