Repository: statsmodels/statsmodels

Issue:
how to specify columns in conf_int()?
On statsmodels 0.5, the docstring to the `RegressionResultsWrapper.conf_int()` method states

```
cols : array-like, optional
    `cols` specifies which confidence intervals to return
```

However, when I use this kwarg as

```
result2.conf_int(cols=["omega"])
```

I get

```
IndexError: arrays used as indices must be of integer (or boolean) type
```

When trying an integer /col/, 

```
result2.conf_int(cols=[1])
```

I get

```
ValueError: Shape of passed values is (2, 1), indices imply (2, 37)
```

So somehow I'm not using this kwarg correctly; most probably due to my interpretation of the docstring.



Discussion:
The integer index works in master for OLS

```
>>> res_ols.conf_int(cols=[1])
array([[ 0.11143502,  1.38877782]])
>>> res_ols.conf_int(cols=[1, 2])
array([[ 0.11143502,  1.38877782],
       [-0.01228462,  1.19657773]])
```

my guess is that the names are not connected to the integer index. Internally we don't have pandas. But I haven't tried a pandas wrapped case yet.

Since the return should be a pandas DataFrame, the indexing by names can also be done there.

Should just be a matter of hooking in existing convenience functions. https://github.com/statsmodels/statsmodels/blob/master/statsmodels/graphics/utils.py#L94

I'm not sure we really need to support column selection. I thought it is mostly noise (unnecessary code complication) in cov_params.

However, it might also become useful when we add dropped variables to the results.

@ssakatausa Do you want to submit a PR?
Sorry, my "fix", which I have already removed from this thread, was totally incorrect. But I think the problem is essentially in attach_columns in data.py defined between lines 553 and 560. The function assumes that the given argument, _result_, is an array whose rows correspond to self.param_names. But when conf_int is called with a non-None _cols_ argument, _result_ only contains the elements specified by _cols_, so that self.param_names contains the names of the elements not contained in _result_.  This makes the dimension mismatch between _result_ and self.param_names. Unfortunately, I do not understand the structure of the statsmodels package well enough to find the best way to address the issue (without breaking many other parts). Anyone has a good idea?
I went over the source code to see how the confidence intervals are calculated and processed to yield a pandas dataframe. Here are my findings. First, the confidence intervals are calculated for all parameters, regardless of what are specified in _cols_, and then the parameters not in _cols_ are stripped off from the result if _cols_ is not None. So, _cols_ has absolutely no effects on the compuation burden (well, the burden is negligible in any case, though). Second, the pandas dataframe of conf_int's output is generated by some wrappers, which essentially add row names to the matrix containing the upper and lower bounds of the confidence intervals. The problem is that the wrappers are not written in such a way that they can access _cols_. To make _cols_ work correctly, we would need to do something a little tricky in a way or another, at the cost of making the code more difficult to maintain.  So, I did not touch the code.

A good news is that what _cols_ option is supposed to do can be done easily by using .loc and .iloc methods of pandas dataframe objects. E.g.:
`
result.conf_int(0.01).loc[['smoker', 'nprevist']]
`
or
`
result.conf_int(0.05).iloc[[0, 2]]
`
So, I propose dropping the _cols_ option from the conf_int method.

Resolving patch:
diff --git a/statsmodels/base/model.py b/statsmodels/base/model.py
--- a/statsmodels/base/model.py
+++ b/statsmodels/base/model.py
@@ -2120,6 +2120,13 @@ def conf_int(self, alpha=.05, cols=None):
         cols : array_like, optional
             Specifies which confidence intervals to return.
 
+        .. deprecated: 0.13
+
+           cols is deprecated and will be removed after 0.14 is released.
+           cols only works when inputs are NumPy arrays and will fail
+           when using pandas Series or DataFrames as input. You can
+           subset the confidence intervals using slices.
+
         Returns
         -------
         array_like
@@ -2167,6 +2174,14 @@ def conf_int(self, alpha=.05, cols=None):
         lower = params - q * bse
         upper = params + q * bse
         if cols is not None:
+            warnings.warn(
+                "cols is deprecated and will be removed after 0.14 is "
+                "released. cols only works when inputs are NumPy arrays and "
+                "will fail when using pandas Series or DataFrames as input. "
+                "Subsets of confidence intervals can be selected using slices "
+                "of the full confidence interval array.",
+                FutureWarning
+            )
             cols = np.asarray(cols)
             lower = lower[cols]
             upper = upper[cols]


Test patch: