Repository: statsmodels/statsmodels

Issue:
BUG:  base t_test incorrect ravel for MNLogit, cov_names are not strings
see #8904

It looks like t_test and wald test ravel params incorrectly, should be "F" order.
res.model.data.cov_names is raveled by columns ("F" order), i.e stacked by equation not by exog var 
(pr #6012)
brief check `t_test(np.eye(...))` does not match res.tvalues.

another bug: 
cov_names is a multi-index, i.e tuple and not a list of strings.
I don't see how this could be used in a string constraints. (all my tries errored)

another bug:
column labes in params, bse do not correspond to endog names/levels as used in summary and in cov_names
e.g. hanes test case: summary reports y-values from 1 to 6, (0 is reference), while params columns are indexed from 0 to 5.
```
res2.bse.columns, res2.params.columns
(RangeIndex(start=0, stop=6, step=1), RangeIndex(start=0, stop=6, step=1))
```

Also
Current hanes version is not a good test case, because `res1.params.shape` is square (6, 6).
indexing, or reshaping bugs will not raise if rows and columns are mixed up.


usage of t_test and f_test/wald_test:

currently we can only use contrast/restriction matrices directly.
However, params and cov_params don't match up.
We need example and test cases that wald test specific parameters, and helper functions to test manova type constraints L P M.
(I started something like that for my experimental code for MultivariateLS)




Discussion:
multi-index to string names
Do we have a convention for combining equation names and exog names?

```
['y{}_{}'.format(i[0], i[1]) for i in res2.model.data.cov_names]
['y1_logpopul',
 'y1_selfLR',
 'y1_age',
 'y1_educ',
 'y1_income',
 'y1_const',
 'y2_logpopul',
 'y2_selfLR',
 'y2_age',
 'y2_educ',
 'y2_income',
```

using this in `base` t_test, then t_test works without exception, and correctly after a ravel with order "F"

```
tt = res2.t_test("y1_age")
tt, tt.effect, tt.sd, tt.tvalue
(<class 'statsmodels.stats.contrast.ContrastResults'>
                              Test for Constraints                             
 ==============================================================================
                  coef    std err          z      P>|z|      [0.025      0.975]
 ------------------------------------------------------------------------------
 c0            -0.0249      0.007     -3.823      0.000      -0.038      -0.012
 ==============================================================================,
 array([-0.024945]),
 array([[0.00652486]]),
 array([[-3.82307077]]))

col = 0  # different col indexing in params, ...
res2.params.loc["age", col], res2.bse.loc["age", col], res2.tvalues.loc["age", col]
(-0.024944995441998526, 0.006524858401442056, -3.8230707713879957)
```
for anes test case: we can drop logpopul so that params is not square array
(lopopul is mainly important for small popul for strongly republican (last level of endog PID)

```
c = ["y{}_logpopul".format(i) for i in range(1, 7)]
cs = ", ".join(c)
tt = res2.t_test(cs)
print(c)
print(tt.summary(xname=c))
['y1_logpopul', 'y2_logpopul', 'y3_logpopul', 'y4_logpopul', 'y5_logpopul', 'y6_logpopul']
                              Test for Constraints                             
===============================================================================
                  coef    std err          z      P>|z|      [0.025      0.975]
-------------------------------------------------------------------------------
y1_logpopul    -0.0115      0.034     -0.336      0.736      -0.079       0.056
y2_logpopul    -0.0888      0.039     -2.266      0.023      -0.166      -0.012
y3_logpopul    -0.1060      0.057     -1.858      0.063      -0.218       0.006
y4_logpopul    -0.0916      0.044     -2.091      0.037      -0.177      -0.006
y5_logpopul    -0.0933      0.039     -2.371      0.018      -0.170      -0.016
y6_logpopul    -0.1409      0.042     -3.343      0.001      -0.223      -0.058
===============================================================================

wt = res2.wald_test(cs, scalar=True)
wt
<class 'statsmodels.stats.contrast.ContrastResults'>
<Wald test (chi2): statistic=16.092245564731254, p-value=0.013267319745202254, df_denom=6>
```

However, `wald_test_terms` is still broken for MNLogit.
It would be convenient for row or column hypothesis, e.g.
`res2.wald_test_terms(skip_single=True, combine_terms=["logpopul"])`
I don't see an obvious quickfix.
wald_test_terms handles more cases and term definition in DesignInfo is only for exog and does not take multivariate endog into account.
It might be possible to add an option to skip all parts in wald_test_terms except the pattern matching.

new issue #8906 expand partial constraint matrices to full constraint matrices.




another test: test that all slope coefficients (params except const) are equal across equations/levels 
(not quite right?, this ignores reference level 0)
test sequential pairs

```
c = ["y{}_{} - y{}_{}".format(j, i, j-1, i) for j in range(2, 7) for i in res2.model.exog_names[:-1]]
cs = ", ".join(c)
tt = res2.t_test(cs)
print(c)
print(tt.summary(xname=c))
['y2_logpopul - y1_logpopul', 'y2_selfLR - y1_selfLR', 'y2_age - y1_age', 'y2_educ - y1_educ', 'y2_income - y1_income', 
'y3_logpopul - y2_logpopul', 'y3_selfLR - y2_selfLR', 'y3_age - y2_age', 'y3_educ - y2_educ', 'y3_income - y2_income', 
'y4_logpopul - y3_logpopul', 'y4_selfLR - y3_selfLR', 'y4_age - y3_age', 'y4_educ - y3_educ', 'y4_income - y3_income', 
'y5_logpopul - y4_logpopul', 'y5_selfLR - y4_selfLR', 'y5_age - y4_age', 'y5_educ - y4_educ', 'y5_income - y4_income', 
'y6_logpopul - y5_logpopul', 'y6_selfLR - y5_selfLR', 'y6_age - y5_age', 'y6_educ - y5_educ', 'y6_income - y5_income']
                                     Test for Constraints                                    
=============================================================================================
                                coef    std err          z      P>|z|      [0.025      0.975]
---------------------------------------------------------------------------------------------
y2_logpopul - y1_logpopul    -0.0772      0.039     -1.979      0.048      -0.154      -0.001
y2_selfLR - y1_selfLR         0.0940      0.105      0.892      0.372      -0.113       0.300
y2_age - y1_age               0.0020      0.008      0.253      0.800      -0.014       0.018
y2_educ - y1_educ             0.0986      0.086      1.152      0.249      -0.069       0.266
y2_income - y1_income         0.0427      0.022      1.928      0.054      -0.001       0.086
y3_logpopul - y2_logpopul    -0.0172      0.060     -0.289      0.772      -0.134       0.099
y3_selfLR - y2_selfLR         0.1818      0.163      1.112      0.266      -0.139       0.502
y3_age - y2_age               0.0080      0.012      0.659      0.510      -0.016       0.032
y3_educ - y2_educ            -0.1882      0.133     -1.420      0.156      -0.448       0.072
y3_income - y2_income         0.0097      0.036      0.270      0.787      -0.061       0.080
y4_logpopul - y3_logpopul     0.0144      0.061      0.234      0.815      -0.106       0.135
y4_selfLR - y3_selfLR         0.7053      0.172      4.095      0.000       0.368       1.043
y4_age - y3_age               0.0062      0.012      0.501      0.616      -0.018       0.030
y4_educ - y3_educ             0.2070      0.136      1.523      0.128      -0.059       0.473
y4_income - y3_income         0.0269      0.038      0.713      0.476      -0.047       0.101
y5_logpopul - y4_logpopul    -0.0017      0.042     -0.042      0.967      -0.083       0.080
y5_selfLR - y4_selfLR         0.0682      0.120      0.568      0.570      -0.167       0.303
y5_age - y4_age              -0.0092      0.008     -1.117      0.264      -0.025       0.007
y5_educ - y4_educ             0.0171      0.090      0.190      0.850      -0.160       0.194
y5_income - y4_income        -0.0035      0.026     -0.134      0.893      -0.055       0.048
y6_logpopul - y5_logpopul    -0.0476      0.036     -1.306      0.192      -0.119       0.024
y6_selfLR - y5_selfLR         0.7231      0.124      5.816      0.000       0.479       0.967
y6_age - y5_age               0.0085      0.007      1.148      0.251      -0.006       0.023
y6_educ - y5_educ             0.1050      0.080      1.311      0.190      -0.052       0.262
y6_income - y5_income         0.0279      0.024      1.188      0.235      -0.018       0.074
=============================================================================================

wt = res2.wald_test(cs, scalar=True)
wt
<class 'statsmodels.stats.contrast.ContrastResults'>
<Wald test (chi2): statistic=236.40573788169553, p-value=2.559570053943716e-36, df_denom=25>
```
Note, endog PID is ordinal left to right. So, we could also use the OrderedModel with it. But constant slopes is strongly rejected.

Resolving patch:
diff --git a/statsmodels/base/model.py b/statsmodels/base/model.py
--- a/statsmodels/base/model.py
+++ b/statsmodels/base/model.py
@@ -1639,7 +1639,11 @@ def t_test(self, r_matrix, cov_p=None, use_t=None):
         """
         from patsy import DesignInfo
         use_t = bool_like(use_t, "use_t", strict=True, optional=True)
-        names = self.model.data.cov_names
+        if self.params.ndim == 2:
+            names = ['y{}_{}'.format(i[0], i[1])
+                     for i in self.model.data.cov_names]
+        else:
+            names = self.model.data.cov_names
         LC = DesignInfo(names).linear_constraint(r_matrix)
         r_matrix, q_matrix = LC.coefs, LC.constants
         num_ttests = r_matrix.shape[0]
@@ -1649,7 +1653,7 @@ def t_test(self, r_matrix, cov_p=None, use_t=None):
                 not hasattr(self, 'cov_params_default')):
             raise ValueError('Need covariance of parameters for computing '
                              'T statistics')
-        params = self.params.ravel()
+        params = self.params.ravel(order="F")
         if num_params != params.shape[0]:
             raise ValueError('r_matrix and params are not aligned')
         if q_matrix is None:
@@ -1853,8 +1857,12 @@ def wald_test(self, r_matrix, cov_p=None, invcov=None,
             use_f = (hasattr(self, 'use_t') and self.use_t)
 
         from patsy import DesignInfo
-        names = self.model.data.cov_names
-        params = self.params.ravel()
+        if self.params.ndim == 2:
+            names = ['y{}_{}'.format(i[0], i[1])
+                     for i in self.model.data.cov_names]
+        else:
+            names = self.model.data.cov_names
+        params = self.params.ravel(order="F")
         LC = DesignInfo(names).linear_constraint(r_matrix)
         r_matrix, q_matrix = LC.coefs, LC.constants
 


Test patch:
diff --git a/statsmodels/discrete/tests/test_discrete.py b/statsmodels/discrete/tests/test_discrete.py
--- a/statsmodels/discrete/tests/test_discrete.py
+++ b/statsmodels/discrete/tests/test_discrete.py
@@ -2586,16 +2586,15 @@ def test_cov_confint_pandas():
     assert isinstance(ci.index, pd.MultiIndex)
 
 
-def test_t_test():
+def test_mlogit_t_test():
     # GH669, check t_test works in multivariate model
-    data = load_anes96()
+    data = sm.datasets.anes96.load()
     exog = sm.add_constant(data.exog, prepend=False)
     res1 = sm.MNLogit(data.endog, exog).fit(disp=0)
     r = np.ones(res1.cov_params().shape[0])
     t1 = res1.t_test(r)
     f1 = res1.f_test(r)
 
-    data = sm.datasets.anes96.load()
     exog = sm.add_constant(data.exog, prepend=False)
     endog, exog = np.asarray(data.endog), np.asarray(exog)
     res2 = sm.MNLogit(endog, exog).fit(disp=0)
@@ -2604,3 +2603,20 @@ def test_t_test():
 
     assert_allclose(t1.effect, t2.effect)
     assert_allclose(f1.statistic, f2.statistic)
+
+    tt = res1.t_test(np.eye(np.size(res2.params)))
+    assert_allclose(tt.tvalue.reshape(6,6, order="F"), res1.tvalues.to_numpy())
+    tt = res2.t_test(np.eye(np.size(res2.params)))
+    assert_allclose(tt.tvalue.reshape(6,6, order="F"), res2.tvalues)
+
+    wt = res1.wald_test(np.eye(np.size(res2.params))[0], scalar=True)
+    assert_allclose(wt.pvalue, res1.pvalues.to_numpy()[0, 0])
+
+
+    tt = res1.t_test("y1_logpopul")
+    wt = res1.wald_test("y1_logpopul", scalar=True)
+    assert_allclose(tt.pvalue, wt.pvalue)
+
+    wt = res1.wald_test("y1_logpopul, y2_logpopul", scalar=True)
+    # regression test
+    assert_allclose(wt.statistic, 5.68660562, rtol=1e-8)