Repository: statsmodels/statsmodels

Issue:
BUG: f_test and t_test shape mismatch in MNLogit
MNLogit cov_params is for flattened array, params is 2d

It looks like this is not adjusted for f_test and t_test. 

```
>>> mlogit_res.t_test(np.ones(36))
Traceback (most recent call last):
  File "<pyshell#13>", line 1, in <module>
    mlogit_res.t_test(np.ones(36))
  File "e:\josef\eclipsegworkspace\statsmodels-git\statsmodels-all-new\statsmodels\statsmodels\base\model.py", line 1078, in t_test
    raise ValueError('r_matrix and params are not aligned')
ValueError: r_matrix and params are not aligned
>>> mlogit_res.t_test(np.ones(6))
Traceback (most recent call last):
  File "<pyshell#14>", line 1, in <module>
    mlogit_res.t_test(np.ones(6))
  File "e:\josef\eclipsegworkspace\statsmodels-git\statsmodels-all-new\statsmodels\statsmodels\base\model.py", line 1095, in t_test
    _sd = np.sqrt(self.cov_params(r_matrix=r_matrix, cov_p=cov_p))
  File "e:\josef\eclipsegworkspace\statsmodels-git\statsmodels-all-new\statsmodels\statsmodels\base\model.py", line 1014, in cov_params
    tmp = np.dot(r_matrix, np.dot(cov_p, np.transpose(other)))
ValueError: matrices are not aligned
```

Note, the proper test for discrete should be Wald test based on asymptotic normality, but it has the same structure as f_test and t_test.

I don't know which other models might be affected (VAR has 2d params, sysreg works with 1d params)

my suggestion callback for params transform, on mailing list



Discussion:
Briefly until I have time to look at this in detail. cov_params should stay 2d, we just need to be consistent about how we ravel and reshape. 'F' order now (I think, but I'd have to check again). I think what we should do is have MultivariateModel that does the reshaping and then pushes things up to t_test/f_test or whatever we need rather than model specific callbacks. That way, it's handled consistently and in one place, and I think will cover all multivariate / multiequation models.

I think working with 2d params should be an exception (or I would even prefer it raveled)

The problem with multivariate / multiequation models is that 2d only works if we have the same number of exog (and maybe same exog) in each equation. 

Alexandre's sysreg works mostly (or exclusively IIRC) with 1d params because or when there is no restriction that the number of parameters is the same for each equation. (he reshaped data and residuals when necessary)

Similar for panel data, the versions I worked with don't assume balanced panels and work with stacked data only.

Same for two-stage models (not available yet), Heckman, ZeroInflated with explanatory variable in the zero equation.

And even multinomial/conditional choice models can have a richer structure across endog-choices, that doesn't fit the 2d params shape.

There are a few models where 2d params can be more convenient, VAR, SUR, pure multivariate model, balanced panel (?)

needs checking, this might have been fixed by the L1 penalization merge

I'm a bit curious about API here. I don't think patsy yet handles equations (or it's been discussed). Or how to signify for which equation you'd like to test a parameter?A dictionary ?

I think each variable name will need to have an equation reference/qualifier. 
Internally, I think it's just the flattened params corresponding to the cov_params which are also for flattened params.

MNLogit and VAR are still simpler because of the rectangular params, but for sysreg or other models with different number of parameters per equation, we need explicit mapping.

We should check what Stata is doing.

This is still broken, now it fails already in the call to patsy see #669 

summary() works because it has special code for MNLogit that doesn't require the names for all parameters.

One possibility is to add an extra attribute `params_names` so we don't have to patch exog_names when k_exog is not equal to the number of params. (mentioned once in another issue)

I removed high priority and moved to 0.6 milestone, because t_test and f_test are currently t-distribution and not normal distribution based.

I'm kind of wanting to punt on this for 0.6. It's not a regression, so shouldn't be a release blocker I don't think and people haven't been exactly clamoring for this feature to be fixed. I am willing to move this up my priority list and get this working though. Thoughts?

Food for thought, this is how Stata works syntactically.

```
Give the outcome name. Test all coefficients are zero.

[Outcome1]

Test specific coefficient of equation.

[Outcome1]coef

Test that all coefficients are equal across models

[Outcome1=Outcome2]

Test within an outcome

[Outcome1]: coef1 coef2

Arbitrary test

[Outcome1]coef1 + [Outcome2]coef2)/2 = 2 - [Outcome2]coef3
```

We can make this more Pythonic I think by switching the bracketing. To be like `__getitem__` access for variables within equations. Either we'll need to do some pre-processing or patsy will need to get some machinery for multiple equations.

Punting to 0.7


Resolving patch:
diff --git a/statsmodels/base/model.py b/statsmodels/base/model.py
--- a/statsmodels/base/model.py
+++ b/statsmodels/base/model.py
@@ -1498,7 +1498,7 @@ def t_test(self, r_matrix, cov_p=None, scale=None, use_t=None):
                           DeprecationWarning)
 
         from patsy import DesignInfo
-        names = self.model.data.param_names
+        names = self.model.data.cov_names
         LC = DesignInfo(names).linear_constraint(r_matrix)
         r_matrix, q_matrix = LC.coefs, LC.constants
         num_ttests = r_matrix.shape[0]
@@ -1508,7 +1508,8 @@ def t_test(self, r_matrix, cov_p=None, scale=None, use_t=None):
                 not hasattr(self, 'cov_params_default')):
             raise ValueError('Need covariance of parameters for computing '
                              'T statistics')
-        if num_params != self.params.shape[0]:
+        params = self.params.ravel()
+        if num_params != params.shape[0]:
             raise ValueError('r_matrix and params are not aligned')
         if q_matrix is None:
             q_matrix = np.zeros(num_ttests)
@@ -1524,10 +1525,7 @@ def t_test(self, r_matrix, cov_p=None, scale=None, use_t=None):
             # switch to use_t false if undefined
             use_t = (hasattr(self, 'use_t') and self.use_t)
 
-        _t = _sd = None
-
-        _effect = np.dot(r_matrix, self.params)
-        # nan_dot multiplies with the convention nan * 0 = 0
+        _effect = np.dot(r_matrix, params)
 
         # Perform the test
         if num_ttests > 1:
@@ -1718,7 +1716,8 @@ def wald_test(self, r_matrix, cov_p=None, scale=1.0, invcov=None,
             use_f = (hasattr(self, 'use_t') and self.use_t)
 
         from patsy import DesignInfo
-        names = self.model.data.param_names
+        names = self.model.data.cov_names
+        params = self.params.ravel()
         LC = DesignInfo(names).linear_constraint(r_matrix)
         r_matrix, q_matrix = LC.coefs, LC.constants
 
@@ -1727,7 +1726,7 @@ def wald_test(self, r_matrix, cov_p=None, scale=1.0, invcov=None,
             raise ValueError('need covariance of parameters for computing '
                              'F statistics')
 
-        cparams = np.dot(r_matrix, self.params[:, None])
+        cparams = np.dot(r_matrix, params[:, None])
         J = float(r_matrix.shape[0])  # number of restrictions
 
         if q_matrix is None:


Test patch:
diff --git a/statsmodels/discrete/tests/test_discrete.py b/statsmodels/discrete/tests/test_discrete.py
--- a/statsmodels/discrete/tests/test_discrete.py
+++ b/statsmodels/discrete/tests/test_discrete.py
@@ -782,7 +782,7 @@ def test_t_test(self):
         assert_almost_equal(t_unreg.effect, t_reg.effect[:m], DECIMAL_3)
         assert_almost_equal(t_unreg.sd, t_reg.sd[:m], DECIMAL_3)
         assert_almost_equal(np.nan, t_reg.sd[m])
-        assert_almost_equal(t_unreg.tvalue, t_reg.tvalue[:m, :m], DECIMAL_3)
+        assert_almost_equal(t_unreg.tvalue, t_reg.tvalue[:m], DECIMAL_3)
 
     @pytest.mark.skip("Skipped test_f_test for MNLogit")
     def test_f_test(self):
@@ -2390,3 +2390,23 @@ def test_cov_confint_pandas():
     assert_index_equal(ci.index, cov.index)
     assert_index_equal(cov.index, cov.columns)
     assert isinstance(ci.index, pd.MultiIndex)
+
+
+def test_t_test():
+    # GH669, check t_test works in multivariate model
+    data = sm.datasets.anes96.load(as_pandas=True)
+    exog = sm.add_constant(data.exog, prepend=False)
+    res1 = sm.MNLogit(data.endog, exog).fit(disp=0)
+    r = np.ones(res1.cov_params().shape[0])
+    t1 = res1.t_test(r)
+    f1 = res1.f_test(r)
+
+    data = sm.datasets.anes96.load(as_pandas=True)
+    exog = sm.add_constant(data.exog, prepend=False)
+    endog, exog = np.asarray(data.endog), np.asarray(exog)
+    res2 = sm.MNLogit(endog, exog).fit(disp=0)
+    t2 = res2.t_test(r)
+    f2 = res2.f_test(r)
+
+    assert_allclose(t1.effect, t2.effect)
+    assert_allclose(f1.statistic, f2.statistic)