Repository: statsmodels/statsmodels

Issue:
statsmodels.sandbox.regression.gmm.IV2SLS Unable to print summary
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-193-559ab9b682c5> in <module>()
---> 16 model.summary()

/econ/lib/python3.6/site-packages/statsmodels/sandbox/regression/gmm.py in summary(self, yname, xname, title, alpha)
    324         top_right = [('R-squared:', ["%#8.3f" % self.rsquared]),
    325                      ('Adj. R-squared:', ["%#8.3f" % self.rsquared_adj]),
--> 326                      ('F-statistic:', ["%#8.4g" % self.fvalue] ),
    327                      ('Prob (F-statistic):', ["%#6.3g" % self.f_pvalue]),
    328                      #('Log-Likelihood:', None), #["%#6.4g" % self.llf]),

/econ/lib/python3.6/site-packages/statsmodels/tools/decorators.py in __get__(self, obj, type)
     92         if _cachedval is None:
     93             # Call the "fget" function
---> 94             _cachedval = self.fget(obj)
     95             # Set the attribute in obj
     96 #            print("Setting %s in cache to %s" % (name, _cachedval))

/econ/lib/python3.6/site-packages/statsmodels/sandbox/regression/gmm.py in fvalue(self)
    216         restriction = np.eye(k_vars)
    217         idx_noconstant = lrange(k_vars)
--> 218         del idx_noconstant[self.model.data.const_idx]
    219         fval = self.f_test(restriction[idx_noconstant]).fvalue # without constant
    220         return fval

TypeError: list indices must be integers or slices, not NoneType


Discussion:
Is this with version 0.6.1 or with 0.8.0?
I have tried both 0.6.1 and 0.8.0, and receive the error on both versions.
I am having exactly the same error. Cannot find a way to print the summary. Everything else works on IV2SLS. Version 0.8.0
Based on the traceback in the issue description, this happens if const_idx is None.
This needs to be fixed, but can you check if it works with your dataset if you include an explicit constant.

IIRC I fixed this case for the regression results, but I didn't check GMM for it. #2444, #2477
But that code path should be for the case for cov_type is not 'nonrobust'. However that's in regression results, IVRegressionResults overrides the inherited fvalue and always uses Waldtest, even if that is not possible because of the missing constant column.

#3723 possible solution if const_idx is None.

Using inherited fvalue: I don't know if the ssr based f-test is valid. Given that I didn't use it and wrote the class specific fvalue, I guess not.

A "cheap" way to rescue `summary` is to set fvalue to nan if const_idx is None, similar to OLS/WLS with a robust cov_type.
Hi, 

I added a constant to the exogenoues variable : X['const_idx'] = 1
and to the instrument: Z['const_idx'] = 1

That solved the problem. 

Thanks a lot for the feedback.


@tiagocaruso  Thanks for checking.

I have the fix of summary by setting fvalue to nan locally but still need unit tests. 
A PR should be merged into master within a few days.

Resolving patch:
diff --git a/statsmodels/sandbox/regression/gmm.py b/statsmodels/sandbox/regression/gmm.py
--- a/statsmodels/sandbox/regression/gmm.py
+++ b/statsmodels/sandbox/regression/gmm.py
@@ -213,18 +213,22 @@ class IVRegressionResults(RegressionResults):
 
     @cache_readonly
     def fvalue(self):
-        k_vars = len(self.params)
-        restriction = np.eye(k_vars)
-        idx_noconstant = lrange(k_vars)
-        del idx_noconstant[self.model.data.const_idx]
-        fval = self.f_test(restriction[idx_noconstant]).fvalue # without constant
-        return fval
+        const_idx = self.model.data.const_idx
+        # if constant is implicit or missing, return nan see #2444, #3544
+        if const_idx is None:
+            return np.nan
+        else:
+            k_vars = len(self.params)
+            restriction = np.eye(k_vars)
+            idx_noconstant = lrange(k_vars)
+            del idx_noconstant[const_idx]
+            fval = self.f_test(restriction[idx_noconstant]).fvalue # without constant
+            return fval
 
 
     def spec_hausman(self, dof=None):
         '''Hausman's specification test
 
-
         See Also
         --------
         spec_hausman : generic function for Hausman's specification test


Test patch:
diff --git a/statsmodels/sandbox/regression/tests/test_gmm.py b/statsmodels/sandbox/regression/tests/test_gmm.py
--- a/statsmodels/sandbox/regression/tests/test_gmm.py
+++ b/statsmodels/sandbox/regression/tests/test_gmm.py
@@ -741,3 +741,12 @@ def test_input_dimensions(self):
                     else:
                         assert_allclose(res.params, res2.params)
 
+def test_noconstant():
+    exog = exog_st[:, :-1]  # with const removed at end
+
+    mod = gmm.IV2SLS(endog, exog, instrument)
+    res = mod.fit()
+
+    assert_equal(res.fvalue, np.nan)
+    # smoke test
+    res.summary()