Repository: statsmodels/statsmodels

Issue:
'MixedLMResults' object has no attribute 're_params'
In difference to the documentation [1] the class MixedLMResults has no attribute re_params.

```
In [449]: result.re_params
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-449-6c53a17ec896> in <module>()
----> 1 result.re_params

/home/moebius/software/python-environments/nmf/lib/python3.4/site-packages/statsmodels/b
ase/wrapper.py in __getattribute__(self, attr)
     33             pass
     34
---> 35         obj = getattr(results, attr)
     36         data = results.model.data
     37         how = self._wrap_attrs.get(attr)

AttributeError: 'MixedLMResults' object has no attribute 're_params'
```

I am using version

```
In [453]: sm.version.full_version
Out[453]: '0.6.1'
```

Minimal example

```
import statsmodels.formula.api as smf
rfit = smf.mixedlm('value ~ 1', groups=dat['officer'], data=dat).fit()
print(rfit.summary(alpha=.01))

rfit.fe_params
rfit.re_params
```

where dat is a DataFrame which looks as follows

```
   officer  value
0        0     76
1        0     65
2        0     85
3        0     74
4        1     59
5        1     75
6        1     81
7        1     67
8        2     49
9        2     63
10       2     61
11       2     46
12       3     74
13       3     71
14       3     85
15       3     89
16       4     66
17       4     84
18       4     80
19       4     79
```

How do I get the random effects parameters? I am in particular interested in estimating the intraclass correlation. If you already happened to code this somewhere, let me know!

Cheers,
Thomas

[1] http://statsmodels.sourceforge.net/stable/generated/statsmodels.regression.mixed_linear_model.MixedLMResults.html

[2] https://en.wikipedia.org/wiki/Intraclass_correlation#Modern_ICC_definitions:_simpler_formula_but_positive_bias



Discussion:
Hello Thomas,

Does rfit.random_effects work for you?

MixedLM is fairly new and we made some major modifications about a year ago
when we added support for variance components.  If possible it would be
better to use the code from statsmodels master on github.

Kerby

On Fri, May 20, 2016 at 10:11 AM, Thomas W. D. Möbius <
notifications@github.com> wrote:

> In difference to the documentation [1] the class MixedLMResults has no
> attribute re_params.
> 
> ## In [449]: result.re_params
> 
> AttributeError                            Traceback (most recent call last)
> <ipython-input-449-6c53a17ec896> in <module>()
> ----> 1 result.re_params
> 
> /home/moebius/software/python-environments/nmf/lib/python3.4/site-packages/statsmodels/b
> ase/wrapper.py in **getattribute**(self, attr)
>      33             pass
>      34
> ---> 35         obj = getattr(results, attr)
>      36         data = results.model.data
>      37         how = self._wrap_attrs.get(attr)
> 
> AttributeError: 'MixedLMResults' object has no attribute 're_params'
> 
> I am using version
> 
> In [453]: sm.version.full_version
> Out[453]: '0.6.1'
> 
> Minimal example
> 
> import statsmodels.formula.api as smf
> rfit = smf.mixedlm('value ~ 1', groups=dat['officer'], data=dat).fit()
> print(rfit.summary(alpha=.01))
> 
> rfit.fe_params
> rfit.re_params
> 
> where dat is a DataFrame which looks as follows
> 
>    officer  value
> 0        0     76
> 1        0     65
> 2        0     85
> 3        0     74
> 4        1     59
> 5        1     75
> 6        1     81
> 7        1     67
> 8        2     49
> 9        2     63
> 10       2     61
> 11       2     46
> 12       3     74
> 13       3     71
> 14       3     85
> 15       3     89
> 16       4     66
> 17       4     84
> 18       4     80
> 19       4     79
> 
> How do I get the random effects parameters? I am in particular interested
> in estimating the intraclass correlation. If you already happened to code
> this somewhere, let me know!
> 
> Cheers,
> Thomas
> 
> [1]
> http://statsmodels.sourceforge.net/stable/generated/statsmodels.regression.mixed_linear_model.MixedLMResults.html
> 
> [2]
> https://en.wikipedia.org/wiki/Intraclass_correlation#Modern_ICC_definitions:_simpler_formula_but_positive_bias
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly or view it on GitHub
> https://github.com/statsmodels/statsmodels/issues/2961

I was able to replicate his error on master. 

``` python
In [2]: import pandas as pd

In [3]: dat = pd.read_csv('data.csv')

In [4]: import statsmodels.formula.api as smf

In [5]: rfit = smf.mixedlm('value ~ 1', groups=dat['officer'], data=dat).fit()

In [6]: rfit.re_params
Traceback (most recent call last):

  File "<ipython-input-6-9d93bff9fa5f>", line 1, in <module>
    rfit.re_params

  File "C:\Anaconda3\lib\site-packages\statsmodels-0.8.0-py3.5-win-amd64.egg\statsmodels\base\wrapper.py", line 35, in __getattribute__
    obj = getattr(results, attr)

AttributeError: 'MixedLMResults' object has no attribute 're_params'


In [7]: import statsmodels.api as sm

In [8]: sm.version.full_version
Out[8]: '0.8.0.dev0+fd90132'
```

I don't remember the history on this but I think at some point we switched
from re_params to random_effects.  We probably just need to update the docs.

Kerby

On Fri, May 20, 2016 at 10:24 AM, Peter Quackenbush <
notifications@github.com> wrote:

> I was able to replicate his error master.
> 
> In [2]: import pandas as pd
> 
> In [3]: dat = pd.read_csv('data.csv')
> 
> In [4]: import statsmodels.formula.api as smf
> 
> In [5]: rfit = smf.mixedlm('value ~ 1', groups=dat['officer'], data=dat).fit()
> 
> In [6]: rfit.re_params
> Traceback (most recent call last):
> 
>   File "<ipython-input-6-9d93bff9fa5f>", line 1, in <module>
>     rfit.re_params
> 
>   File "C:\Anaconda3\lib\site-packages\statsmodels-0.8.0-py3.5-win-amd64.egg\statsmodels\base\wrapper.py", line 35, in **getattribute**
>     obj = getattr(results, attr)
> AttributeError: 'MixedLMResults' object has no attribute 're_params'
> 
> In [7]: import statsmodels.api as sm
> 
> In [8]: sm.version.full_version
> Out[8]: '0.8.0.dev0+fd90132'
> 
> —
> You are receiving this because you commented.
> Reply to this email directly or view it on GitHub
> https://github.com/statsmodels/statsmodels/issues/2961#issuecomment-220619889

Given the random one-way ANOVA model of the example:
  y_ij = μ + μ_i + ε_ij  with  μ_i ~ N(0, λ^2)  and  ε_ij ~ N(0, σ^2)

I am looking for the variance estimates of λ^2 (often called the homogeneity) and σ^2 (variance of the residuals). The `rfit.random_effects` gives me the estimates of the μ_i. It seems to be that σ^2 = `result.params[-1:]`. Is this correct? How do I get an estimate of λ^2?

PS. I may be able to switch to github-master; I will do so soon.

Below is an example that illustrates how this works.  The `scale` value given in the top of the summary output, or as `result.scale` is the variance of the independent error terms (epsilon in your notation).  In the summary output below, `Group RE` is the variance of the group random effect (a.k.a. random intercept, lambda^2 in your notation).  MixedLM uses a reparameterized likelihood internally, so `result.params` is hard to interpret.  To get the random effects variance parameters, use `result.cov_re`.

The ICC would be `result.cov_re[0,0] / (result.cov_re[0,0] + result.scale)`, which should be around 0.7 in the example below.

```
import statsmodels.api as sm
import numpy as np

ngrp = 1000 # Number of groups                                                                                                                       
gsize = 20  # Number of observations per group                                                                                                       
rsd = 2     # SD of errors                                                                                                                           
gsd = 3     # SD of group means                                                                                                                      
mn = gsd*np.random.normal(size=ngrp)
gmn = np.kron(mn, np.ones(gsize))
y = gmn + rsd*np.random.normal(size=ngrp*gsize)
gr = np.kron(np.arange(ngrp), np.ones(gsize))
x = np.ones(ngrp * gsize)
model = sm.MixedLM(y, x, groups=gr)
result = model.fit()
```

```
          Mixed Linear Model Regression Results
=========================================================
Model:            MixedLM Dependent Variable: y          
No. Observations: 20000   Method:             REML       
No. Groups:       1000    Scale:              4.0166     
Min. group size:  20      Likelihood:         -44222.7198
Max. group size:  20      Converged:          Yes        
Mean group size:  20.0                                   
----------------------------------------------------------
             Coef.  Std.Err.    z    P>|z|  [0.025  0.975]
----------------------------------------------------------
const        0.009     0.098  0.088  0.930  -0.184   0.202
Group RE     9.501     0.222                              
=========================================================
```

Just a note that I'm getting an error on master when I run `result.random_effects` in the above example.  It looks like the names are not being set.

Thanks for the explanation of the summary output. I can see why you call the residual variance 'Scale', but I wouldn't have guessed it. To summarise: `result.scale` and `result.cov_re` are my friends instead of `result.params`.

I have realised that `result.cov_re` in your example is either a pandas.DataFrame or numpy.ndarray depending on whether y is a pandas.Series or a numpy.ndarray. This means that reference to the homogeneity changes: From `result.cov_re[0,0]` to `result.cov_re.ix[0,0]`. Is this as intended?

PS. Now using sm.version.full_version: '0.8.0.dev0+fd90132'

I don't understand the wrappers that handle type conversions of return values very well.

@josef-pkt -- should `result.cov_re` always be a data frame?

about the pandas wrapper.
in general: 
numpy in -> numpy out
pandas in -> pandas out

this uses pandas if at least one of endog or exog is a pandas object IIRC. 
(there could be other data types also, but I think we only have numpy and pandas fully supported right now)

The advantage of returning numpy ndarrays is for users that don't use pandas directly, and for us when we use a model as internal tool for, for example, auxiliary regression.

I think there's still a documentation issue: http://www.statsmodels.org/dev/generated/statsmodels.regression.mixed_linear_model.MixedLMResults.random_effects.html#statsmodels.regression.mixed_linear_model.MixedLMResults.random_effects says `random_effects` is a function, and http://www.statsmodels.org/dev/generated/statsmodels.regression.mixed_linear_model.MixedLMResults.html#statsmodels.regression.mixed_linear_model.MixedLMResults says that `re_params` is an attribute of `MixedLMResults`, but I get the following (`m` is a model):

    m.random_effects()

> TypeError: 'dict' object is not callable

    m.re_params

> AttributeError: 'MixedLMResults' object has no attribute 're_params'

    m.random_effects  # Produces the dictionary of random effects.

I'm on version 0.8.0.
Thanks for the report.

I think the issue with `random_effects` being misdocumented as a function
is due to it being a cached attribute.  I'm not sure if we have a way to
fix that right now.

I think the docstring that says `re_params` should say `cov_re` in its
place.  I will look into it and fix if needed.

On Thu, Jan 25, 2018 at 1:35 PM, Max Ghenis <notifications@github.com>
wrote:

> I think there's still a documentation issue: http://www.statsmodels.org/
> dev/generated/statsmodels.regression.mixed_linear_model.
> MixedLMResults.random_effects.html#statsmodels.regression.
> mixed_linear_model.MixedLMResults.random_effects says random_effects is a
> function, and http://www.statsmodels.org/dev/generated/statsmodels.
> regression.mixed_linear_model.MixedLMResults.html#
> statsmodels.regression.mixed_linear_model.MixedLMResults says that
> re_params is an attribute of MixedLMResults, but I get the following (m
> is a model):
>
> m.random_effects()
>
> TypeError: 'dict' object is not callable
>
> m.re_params
>
> AttributeError: 'MixedLMResults' object has no attribute 're_params'
>
> m.random_effects  # Produces the dictionary of random effects.
>
> I'm on version 0.8.0.
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/statsmodels/statsmodels/issues/2961#issuecomment-360558274>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ACiww98rWqkXFv5PJwKEyUTTr0HMqYjEks5tOMl9gaJpZM4IjPF_>
> .
>


Resolving patch:
diff --git a/statsmodels/regression/mixed_linear_model.py b/statsmodels/regression/mixed_linear_model.py
--- a/statsmodels/regression/mixed_linear_model.py
+++ b/statsmodels/regression/mixed_linear_model.py
@@ -187,7 +187,7 @@ def _multi_dot_three(A, B, C):
     cost1 = (A.shape[0] * A.shape[1] * B.shape[1] +  # (AB)
              A.shape[0] * B.shape[1] * C.shape[1])   # (--)C
     # cost2 = cost((AB)C)
-    cost2 = (B.shape[0] * B.shape[1] * C.shape[1] +  #  (BC)
+    cost2 = (B.shape[0] * B.shape[1] * C.shape[1] +  # (BC)
              A.shape[0] * A.shape[1] * C.shape[1])   # A(--)
 
     if cost1 < cost2:
@@ -227,6 +227,7 @@ def _get_exog_re_names(self, exog_re):
     defnames = ["x_re{0:1d}".format(k + 1) for k in range(exog_re.shape[1])]
     return defnames
 
+
 class MixedLMParams(object):
     """
     This class represents a parameter state for a mixed linear model.
@@ -256,7 +257,6 @@ def __init__(self, k_fe, k_re, k_vc):
         self.k_tot = self.k_fe + self.k_re2 + self.k_vc
         self._ix = np.tril_indices(self.k_re)
 
-
     def from_packed(params, k_fe, k_re, use_sqrt, has_fe):
         """
         Create a MixedLMParams object from packed parameter vector.
@@ -319,7 +319,8 @@ def from_packed(params, k_fe, k_re, use_sqrt, has_fe):
 
     from_packed = staticmethod(from_packed)
 
-    def from_components(fe_params=None, cov_re=None, cov_re_sqrt=None, vcomp=None):
+    def from_components(fe_params=None, cov_re=None, cov_re_sqrt=None,
+                        vcomp=None):
         """
         Create a MixedLMParams object from each parameter component.
 
@@ -377,7 +378,6 @@ def copy(self):
         obj.vcomp = self.vcomp.copy()
         return obj
 
-
     def get_packed(self, use_sqrt, has_fe=False):
         """
         Return the model parameters packed into a single vector.
@@ -574,7 +574,8 @@ class MixedLM(base.LikelihoodModel):
     A mixed model with fixed effects for the columns of ``exog`` and
     independent random coefficients for the columns of ``exog_re``:
 
-    >>> free = MixedLMParams.from_components(fe_params=np.ones(exog.shape[1]), \
+    >>> free = MixedLMParams.from_components(
+                     fe_params=np.ones(exog.shape[1]),
                      cov_re=np.eye(exog_re.shape[1]))
     >>> model = sm.MixedLM(endog, exog, groups, exog_re=exog_re)
     >>> result = model.fit(free=free)
@@ -599,7 +600,8 @@ def __init__(self, endog, exog, groups, exog_re=None,
         _allowed_kwargs = ["missing_idx", "design_info", "formula"]
         for x in kwargs.keys():
             if x not in _allowed_kwargs:
-                raise ValueError("argument %s not permitted for MixedLM initialization" % x)
+                raise ValueError(
+                    "argument %s not permitted for MixedLM initialization" % x)
 
         self.use_sqrt = use_sqrt
 
@@ -632,7 +634,8 @@ def __init__(self, endog, exog, groups, exog_re=None,
 
         self._init_keys.extend(["use_sqrt", "exog_vc"])
 
-        self.k_fe = exog.shape[1] # Number of fixed effects parameters
+        # Number of fixed effects parameters
+        self.k_fe = exog.shape[1]
 
         if exog_re is None and exog_vc is None:
             # Default random effects structure (random intercepts).
@@ -668,7 +671,7 @@ def __init__(self, endog, exog, groups, exog_re=None,
             # HACK: could've been set in from_formula already
             # needs refactor
             (param_names, exog_re_names,
-                 exog_re_names_full) = self._make_param_names(exog_re)
+             exog_re_names_full) = self._make_param_names(exog_re)
             self.data.param_names = param_names
             self.data.exog_re_names = exog_re_names
             self.data.exog_re_names_full = exog_re_names_full
@@ -680,7 +683,7 @@ def __init__(self, endog, exog, groups, exog_re=None,
         group_labels = list(set(groups))
         group_labels.sort()
         row_indices = dict((s, []) for s in group_labels)
-        for i,g in enumerate(groups):
+        for i, g in enumerate(groups):
             row_indices[g].append(i)
         self.row_indices = row_indices
         self.group_labels = group_labels
@@ -717,7 +720,6 @@ def __init__(self, endog, exog, groups, exog_re=None,
         # Precompute this
         self._lin, self._quad = self._reparam()
 
-
     def _setup_vcomp(self, exog_vc):
         if exog_vc is None:
             exog_vc = {}
@@ -727,7 +729,6 @@ def _setup_vcomp(self, exog_vc):
         vc_names.sort()
         self._vc_names = vc_names
 
-
     def _make_param_names(self, exog_re):
         """
         Returns the full parameter names list, just the exogenous random
@@ -752,7 +753,6 @@ def _make_param_names(self, exog_re):
 
         return exog_names + param_names + vc_names, exog_re_names, param_names
 
-
     @classmethod
     def from_formula(cls, formula, data, re_formula=None, vc_formula=None,
                      subset=None, use_sparse=False, missing='none', *args,
@@ -853,7 +853,8 @@ def from_formula(cls, formula, data, re_formula=None, vc_formula=None,
         """
 
         if "groups" not in kwargs.keys():
-            raise AttributeError("'groups' is a required keyword argument in MixedLM.from_formula")
+            raise AttributeError("'groups' is a required keyword argument " +
+                                 "in MixedLM.from_formula")
         groups = kwargs["groups"]
 
         # If `groups` is a variable name, retrieve the data for the
@@ -864,9 +865,11 @@ def from_formula(cls, formula, data, re_formula=None, vc_formula=None,
             groups = np.asarray(data[groups])
         del kwargs["groups"]
 
-        # Bypass all upstream missing data handling to properly handle variance components
+        # Bypass all upstream missing data handling to properly handle
+        # variance components
         if missing == 'drop':
-            data, groups = _handle_missing(data, groups, formula, re_formula, vc_formula)
+            data, groups = _handle_missing(data, groups, formula, re_formula,
+                                           vc_formula)
             missing = 'none'
 
         if re_formula is not None:
@@ -883,7 +886,8 @@ def from_formula(cls, formula, data, re_formula=None, vc_formula=None,
                     eval_env = EvalEnvironment({})
                 exog_re = patsy.dmatrix(re_formula, data, eval_env=eval_env)
                 exog_re_names = exog_re.design_info.column_names
-                exog_re_names = [x.replace("Intercept", group_name) for x in exog_re_names]
+                exog_re_names = [x.replace("Intercept", group_name)
+                                 for x in exog_re_names]
                 exog_re = np.asarray(exog_re)
             if exog_re.ndim == 1:
                 exog_re = exog_re[:, None]
@@ -911,8 +915,9 @@ def from_formula(cls, formula, data, re_formula=None, vc_formula=None,
                 for group_ix, group in enumerate(kylist):
                     ii = gb.groups[group]
                     vcg = vc_formula[vc_name]
-                    mat = patsy.dmatrix(vcg, data.loc[ii, :], eval_env=eval_env,
-                                        return_type='dataframe')
+                    mat = patsy.dmatrix(
+                        vcg, data.loc[ii, :], eval_env=eval_env,
+                        return_type='dataframe')
                     if use_sparse:
                         exog_vc[vc_name][group] = sparse.csr_matrix(mat)
                     else:
@@ -921,12 +926,9 @@ def from_formula(cls, formula, data, re_formula=None, vc_formula=None,
         else:
             exog_vc = None
 
-        mod = super(MixedLM, cls).from_formula(formula, data,
-                                               subset=None,
-                                               exog_re=exog_re,
-                                               exog_vc=exog_vc,
-                                               groups=groups,
-                                               *args, **kwargs)
+        mod = super(MixedLM, cls).from_formula(
+            formula, data, subset=None, exog_re=exog_re,
+            exog_vc=exog_vc, groups=groups, *args, **kwargs)
 
         # expand re names to account for pairs of RE
         (param_names,
@@ -940,7 +942,6 @@ def from_formula(cls, formula, data, re_formula=None, vc_formula=None,
 
         return mod
 
-
     def predict(self, params, exog=None):
         """
         Return predicted values from a design matrix.
@@ -972,7 +973,6 @@ def predict(self, params, exog=None):
 
         return np.dot(exog, params)
 
-
     def group_list(self, array):
         """
         Returns `array` split into subarrays corresponding to the
@@ -989,7 +989,6 @@ def group_list(self, array):
             return [np.array(array[self.row_indices[k], :])
                     for k in self.group_labels]
 
-
     def fit_regularized(self, start_params=None, method='l1', alpha=0,
                         ceps=1e-4, ptol=1e-6, maxit=200, **fit_kwargs):
         """
@@ -1093,7 +1092,8 @@ def fit_regularized(self, start_params=None, method='l1', alpha=0,
                     ex_r, ex2_r = self._aex_r[group_ix], self._aex_r2[group_ix]
 
                     resid = resid_all[self.row_indices[group]]
-                    solver = _smw_solver(scale, ex_r, ex2_r, cov_re_inv, 1 / vc_var)
+                    solver = _smw_solver(scale, ex_r, ex2_r, cov_re_inv,
+                                         1 / vc_var)
 
                     x = exog[:, j]
                     u = solver(x)
@@ -1125,7 +1125,7 @@ def fit_regularized(self, start_params=None, method='l1', alpha=0,
         ii[self.k_fe:] = True
         ii = np.flatnonzero(ii)
         hess1 = hess[ii, :][:, ii]
-        pcov[np.ix_(ii,ii)] = np.linalg.inv(-hess1)
+        pcov[np.ix_(ii, ii)] = np.linalg.inv(-hess1)
 
         params_object = MixedLMParams.from_components(fe_params, cov_re=cov_re)
 
@@ -1145,7 +1145,6 @@ def fit_regularized(self, start_params=None, method='l1', alpha=0,
 
         return MixedLMResultsWrapper(results)
 
-
     def get_fe_params(self, cov_re, vcomp):
         """
         Use GLS to update the fixed effects parameter estimates.
@@ -1164,7 +1163,7 @@ def get_fe_params(self, cov_re, vcomp):
             return np.array([])
 
         if self.k_re == 0:
-            cov_re_inv = np.empty((0,0))
+            cov_re_inv = np.empty((0, 0))
         else:
             cov_re_inv = np.linalg.inv(cov_re)
 
@@ -1172,7 +1171,9 @@ def get_fe_params(self, cov_re, vcomp):
         if not hasattr(self, "_endex_li"):
             self._endex_li = []
             for group_ix, _ in enumerate(self.group_labels):
-                mat = np.concatenate((self.exog_li[group_ix], self.endog_li[group_ix][:, None]), axis=1)
+                mat = np.concatenate(
+                    (self.exog_li[group_ix],
+                     self.endog_li[group_ix][:, None]), axis=1)
                 self._endex_li.append(mat)
 
         xtxy = 0.
@@ -1188,7 +1189,6 @@ def get_fe_params(self, cov_re, vcomp):
 
         return fe_params
 
-
     def _reparam(self):
         """
         Returns parameters of the map converting parameters from the
@@ -1230,7 +1230,7 @@ def _reparam(self):
 
         # Quadratic terms for random effects covariance.
         ii = np.tril_indices(k_re)
-        ix = [(a,b) for a,b in zip(ii[0], ii[1])]
+        ix = [(a, b) for a, b in zip(ii[0], ii[1])]
         for i1 in range(k_re2):
             for i2 in range(k_re2):
                 ix1 = ix[i1]
@@ -1249,7 +1249,6 @@ def _reparam(self):
 
         return lin, quad
 
-
     def _expand_vcomp(self, vcomp, group):
         """
         Replicate variance parameters to match a group's design.
@@ -1270,13 +1269,13 @@ def _expand_vcomp(self, vcomp, group):
         vc_var = []
         for j, k in enumerate(self._vc_names):
             if group in self.exog_vc[k]:
-                vc_var.append(vcomp[j] * np.ones(self.exog_vc[k][group].shape[1]))
+                vc_var.append(
+                    vcomp[j] * np.ones(self.exog_vc[k][group].shape[1]))
         if len(vc_var) > 0:
             return np.concatenate(vc_var)
         else:
             return np.empty(0)
 
-
     def _augment_exog(self, group_ix):
         """
         Concatenate the columns for variance components to the columns
@@ -1290,7 +1289,7 @@ def _augment_exog(self, group_ix):
         group = self.group_labels[group_ix]
         ex = [ex_r] if self.k_re > 0 else []
         any_sparse = False
-        for j,k in enumerate(self._vc_names):
+        for j, k in enumerate(self._vc_names):
             if group not in self.exog_vc[k]:
                 continue
             ex.append(self.exog_vc[k][group])
@@ -1306,7 +1305,6 @@ def _augment_exog(self, group_ix):
 
         return ex
 
-
     def loglike(self, params, profile_fe=True):
         """
         Evaluate the (profile) log-likelihood of the linear mixed
@@ -1384,7 +1382,8 @@ def loglike(self, params, profile_fe=True):
             resid = resid_all[self.row_indices[group]]
 
             # Part 1 of the log likelihood (for both ML and REML)
-            ld = _smw_logdet(1., ex_r, ex2_r, cov_re_inv, 1 / vc_var, cov_aug_logdet)
+            ld = _smw_logdet(1., ex_r, ex2_r, cov_re_inv, 1 / vc_var,
+                             cov_aug_logdet)
             likeval -= ld / 2.
 
             # Part 2 of the log likelihood (for both ML and REML)
@@ -1398,7 +1397,7 @@ def loglike(self, params, profile_fe=True):
 
         if self.reml:
             likeval -= (self.n_totobs - self.k_fe) * np.log(qf) / 2.
-            _,ld = np.linalg.slogdet(xvx)
+            _, ld = np.linalg.slogdet(xvx)
             likeval -= ld / 2.
             likeval -= (self.n_totobs - self.k_fe) * np.log(2 * np.pi) / 2.
             likeval += ((self.n_totobs - self.k_fe) *
@@ -1412,7 +1411,6 @@ def loglike(self, params, profile_fe=True):
 
         return likeval
 
-
     def _gen_dV_dPar(self, ex_r, solver, group, max_ix=None):
         """
         A generator that yields the element-wise derivative of the
@@ -1439,8 +1437,9 @@ def _gen_dV_dPar(self, ex_r, solver, group, max_ix=None):
             for j2 in range(j1 + 1):
                 if max_ix is not None and jj > max_ix:
                     return
-                mat_l, mat_r = ex_r[:,j1:j1+1], ex_r[:,j2:j2+1] # Need 2d
-                vsl, vsr = axr[:,j1:j1+1], axr[:,j2:j2+1]
+                # Need 2d
+                mat_l, mat_r = ex_r[:, j1:j1+1], ex_r[:, j2:j2+1]
+                vsl, vsr = axr[:, j1:j1+1], axr[:, j2:j2+1]
                 yield jj, mat_l, mat_r, vsl, vsr, j1 == j2
                 jj += 1
 
@@ -1454,7 +1453,6 @@ def _gen_dV_dPar(self, ex_r, solver, group, max_ix=None):
                 yield jj, mat, mat, axmat, axmat, True
                 jj += 1
 
-
     def score(self, params, profile_fe=True):
         """
         Returns the score vector of the profile log-likelihood.
@@ -1467,17 +1465,19 @@ def score(self, params, profile_fe=True):
         """
 
         if type(params) is not MixedLMParams:
-            params = MixedLMParams.from_packed(params, self.k_fe,
-                                               self.k_re, self.use_sqrt,
-                                               has_fe=False)
+            params = MixedLMParams.from_packed(
+                params, self.k_fe, self.k_re, self.use_sqrt,
+                has_fe=False)
 
         if profile_fe:
             params.fe_params = self.get_fe_params(params.cov_re, params.vcomp)
 
         if self.use_sqrt:
-            score_fe, score_re, score_vc = self.score_sqrt(params, calc_fe=not profile_fe)
+            score_fe, score_re, score_vc = self.score_sqrt(
+                params, calc_fe=not profile_fe)
         else:
-            score_fe, score_re, score_vc = self.score_full(params, calc_fe=not profile_fe)
+            score_fe, score_re, score_vc = self.score_full(
+                params, calc_fe=not profile_fe)
 
         if self._freepat is not None:
             score_fe *= self._freepat.fe_params
@@ -1489,7 +1489,6 @@ def score(self, params, profile_fe=True):
         else:
             return np.concatenate((score_fe, score_re, score_vc))
 
-
     def score_full(self, params, calc_fe):
         """
         Returns the score with respect to untransformed parameters.
@@ -1563,7 +1562,7 @@ def score_full(self, params, calc_fe):
 
         # V^{-1} exog' dV/dQ_jj exog V^{-1}, where Q_jj is the jj^th
         # covariance parameter.
-        xtax = [0.,] * (self.k_re2 + self.k_vc)
+        xtax = [0., ] * (self.k_re2 + self.k_vc)
 
         # Temporary related to the gradient of log |V|
         dlv = np.zeros(self.k_re2 + self.k_vc)
@@ -1591,7 +1590,8 @@ def score_full(self, params, calc_fe):
 
             # Contributions to the covariance parameter gradient
             vir = solver(resid)
-            for jj, matl, matr, vsl, vsr, sym in self._gen_dV_dPar(ex_r, solver, group):
+            for (jj, matl, matr, vsl, vsr, sym) in\
+                    self._gen_dV_dPar(ex_r, solver, group):
                 dlv[jj] = _dotsum(matr, vsl)
                 if not sym:
                     dlv[jj] += _dotsum(matl, vsr)
@@ -1644,7 +1644,6 @@ def score_full(self, params, calc_fe):
 
         return score_fe, score_re, score_vc
 
-
     def score_sqrt(self, params, calc_fe=True):
         """
         Returns the score with respect to transformed parameters.
@@ -1690,7 +1689,6 @@ def score_sqrt(self, params, calc_fe=True):
 
         return score_fe, score_re, score_vc
 
-
     def hessian(self, params):
         """
         Returns the model's Hessian matrix.
@@ -1737,7 +1735,7 @@ def hessian(self, params):
 
         rvir = 0.
         xtvix = 0.
-        xtax = [0.,] * (self.k_re2 + self.k_vc)
+        xtax = [0., ] * (self.k_re2 + self.k_vc)
         m = self.k_re2 + self.k_vc
         B = np.zeros(m)
         D = np.zeros((m, m))
@@ -1761,7 +1759,8 @@ def hessian(self, params):
             vir = solver(resid)
             rvir += np.dot(resid, vir)
 
-            for jj1, matl1, matr1, vsl1, vsr1, sym1 in self._gen_dV_dPar(ex_r, solver, group):
+            for (jj1, matl1, matr1, vsl1, vsr1, sym1) in\
+                    self._gen_dV_dPar(ex_r, solver, group):
 
                 ul = _dot(viexog.T, matl1)
                 ur = _dot(matr1.T, vir)
@@ -1788,14 +1787,19 @@ def hessian(self, params):
                 if not sym1:
                     E.append((vsr1, matl1))
 
-                for jj2, matl2, matr2, vsl2, vsr2, sym2 in self._gen_dV_dPar(ex_r, solver, group, jj1):
+                for (jj2, matl2, matr2, vsl2, vsr2, sym2) in\
+                        self._gen_dV_dPar(ex_r, solver, group, jj1):
 
-                    re = sum([_multi_dot_three(matr2.T, x[0], x[1].T) for x in E])
-                    vt = 2 * _dot(_multi_dot_three(vir[None, :], matl2, re), vir[:, None])
+                    re = sum([_multi_dot_three(matr2.T, x[0], x[1].T)
+                              for x in E])
+                    vt = 2 * _dot(_multi_dot_three(vir[None, :], matl2, re),
+                                  vir[:, None])
 
                     if not sym2:
-                        le = sum([_multi_dot_three(matl2.T, x[0], x[1].T) for x in E])
-                        vt += 2 * _dot(_multi_dot_three(vir[None, :], matr2, le), vir[:, None])
+                        le = sum([_multi_dot_three(matl2.T, x[0], x[1].T)
+                                  for x in E])
+                        vt += 2 * _dot(_multi_dot_three(
+                            vir[None, :], matr2, le), vir[:, None])
 
                     D[jj1, jj2] += vt
                     if jj1 != jj2:
@@ -1846,7 +1850,6 @@ def hessian(self, params):
 
         return hess
 
-
     def get_scale(self, fe_params, cov_re, vcomp):
         """
         Returns the estimated error variance based on given estimates
@@ -1898,7 +1901,6 @@ def get_scale(self, fe_params, cov_re, vcomp):
 
         return qf
 
-
     def fit(self, start_params=None, reml=True, niter_sa=0,
             do_cg=True, fe_pen=None, cov_pen=None, free=None,
             full_output=False, method='bfgs', **kwargs):
@@ -1971,17 +1973,16 @@ def fit(self, start_params=None, reml=True, niter_sa=0,
             else:
                 # It's a packed array
                 if len(start_params) == self.k_fe + self.k_re2 + self.k_vc:
-                    params = MixedLMParams.from_packed(start_params, self.k_fe,
-                                                       self.k_re, self.use_sqrt,
-                                                       has_fe=True)
+                    params = MixedLMParams.from_packed(
+                        start_params, self.k_fe, self.k_re, self.use_sqrt,
+                        has_fe=True)
                 elif len(start_params) == self.k_re2 + self.k_vc:
-                    params = MixedLMParams.from_packed(start_params, self.k_fe,
-                                                       self.k_re, self.use_sqrt,
-                                                       has_fe=False)
+                    params = MixedLMParams.from_packed(
+                        start_params, self.k_fe, self.k_re, self.use_sqrt,
+                        has_fe=False)
                 else:
                     raise ValueError("invalid start_params")
 
-
         if do_cg:
             kwargs["retall"] = hist is not None
             if "disp" not in kwargs:
@@ -2012,8 +2013,8 @@ def fit(self, start_params=None, reml=True, niter_sa=0,
         # Convert to the final parameterization (i.e. undo the square
         # root transform of the covariance matrix, and the profiling
         # over the error variance).
-        params = MixedLMParams.from_packed(params, self.k_fe, self.k_re,
-                                           use_sqrt=self.use_sqrt, has_fe=False)
+        params = MixedLMParams.from_packed(
+            params, self.k_fe, self.k_re, use_sqrt=self.use_sqrt, has_fe=False)
         cov_re_unscaled = params.cov_re
         vcomp_unscaled = params.vcomp
         fe_params = self.get_fe_params(cov_re_unscaled, vcomp_unscaled)
@@ -2022,8 +2023,9 @@ def fit(self, start_params=None, reml=True, niter_sa=0,
         cov_re = scale * cov_re_unscaled
         vcomp = scale * vcomp_unscaled
 
-        if (((self.k_re > 0) and (np.min(np.abs(np.diag(cov_re))) < 0.01)) or
-            ((self.k_vc > 0) and (np.min(np.abs(vcomp)) < 0.01))):
+        f1 = (self.k_re > 0) and (np.min(np.abs(np.diag(cov_re))) < 0.01)
+        f2 = (self.k_vc > 0) and (np.min(np.abs(vcomp)) < 0.01)
+        if f1 or f2:
             msg = "The MLE may be on the boundary of the parameter space."
             warnings.warn(msg, ConvergenceWarning)
 
@@ -2044,7 +2046,8 @@ def fit(self, start_params=None, reml=True, niter_sa=0,
         else:
             pcov = np.linalg.inv(-hess)
         if np.any(hess_diag >= 0):
-            msg = "The Hessian matrix at the estimated parameter values is not positive definite."
+            msg = ("The Hessian matrix at the estimated parameter values " +
+                   "is not positive definite.")
             warnings.warn(msg, ConvergenceWarning)
 
         # Prepare a results class instance
@@ -2086,18 +2089,21 @@ class MixedLMResults(base.LikelihoodModelResults, base.ResultMixin):
     **Attributes**
 
     model : class instance
-        Pointer to PHreg model instance that called fit.
+        Pointer to MixedLM model instance that called fit.
     normalized_cov_params : array
         The sampling covariance matrix of the estimates
     fe_params : array
         The fitted fixed-effects coefficients
-    re_params : array
+    cov_re : array
         The fitted random-effects covariance matrix
     bse_fe : array
         The standard errors of the fitted fixed effects coefficients
     bse_re : array
         The standard errors of the fitted random effects covariance
-        matrix
+        matrix and variance components.  The first `k_re * (k_re + 1)`
+        parameters are the standard errors for the lower triangle of
+        `cov_re`, the remaining elements are the standard errors for
+        the variance components.
 
     See Also
     --------
@@ -2111,7 +2117,6 @@ def __init__(self, model, params, cov_params):
         self.nobs = self.model.nobs
         self.df_resid = self.nobs - np_matrix_rank(self.model.exog)
 
-
     @cache_readonly
     def fittedvalues(self):
         """
@@ -2137,7 +2142,6 @@ def fittedvalues(self):
 
         return fit
 
-
     @cache_readonly
     def resid(self):
         """
@@ -2148,7 +2152,6 @@ def resid(self):
         """
         return self.model.endog - self.fittedvalues
 
-
     @cache_readonly
     def bse_fe(self):
         """
@@ -2158,20 +2161,24 @@ def bse_fe(self):
         p = self.model.exog.shape[1]
         return np.sqrt(np.diag(self.cov_params())[0:p])
 
-
     @cache_readonly
     def bse_re(self):
         """
-        Returns the standard errors of the variance parameters.  Note
-        that the sampling distribution of variance parameters is
+        Returns the standard errors of the variance parameters.
+
+        The first `k_re x (k_re + 1)` elements of the returned array
+        are the standard errors of the lower triangle of `cov_re`.
+        The remaining elements are the standard errors of the variance
+        components.
+
+        Note that the sampling distribution of variance parameters is
         strongly skewed unless the sample size is large, so these
         standard errors may not give meaningful confidence intervals
-        of p-values if used in the usual way.
+        or p-values if used in the usual way.
         """
         p = self.model.exog.shape[1]
         return np.sqrt(self.scale * np.diag(self.cov_params())[p:])
 
-
     def _expand_re_names(self, group):
         names = list(self.model.data.exog_re_names)
 
@@ -2182,7 +2189,6 @@ def _expand_re_names(self, group):
                 names.extend(na)
         return names
 
-
     @cache_readonly
     def random_effects(self):
         """
@@ -2197,7 +2203,8 @@ def random_effects(self):
         try:
             cov_re_inv = np.linalg.inv(self.cov_re)
         except np.linalg.LinAlgError:
-            raise ValueError("Cannot predict random effects from singular covariance structure.")
+            raise ValueError("Cannot predict random effects from " +
+                             "singular covariance structure.")
 
         vcomp = self.vcomp
         k_re = self.k_re
@@ -2207,7 +2214,8 @@ def random_effects(self):
 
             endog = self.model.endog_li[group_ix]
             exog = self.model.exog_li[group_ix]
-            ex_r, ex2_r = self.model._aex_r[group_ix], self.model._aex_r2[group_ix]
+            ex_r = self.model._aex_r[group_ix]
+            ex2_r = self.model._aex_r2[group_ix]
             vc_var = self.model._expand_vcomp(vcomp, group)
 
             # Get the residuals relative to fixed effects
@@ -2216,18 +2224,19 @@ def random_effects(self):
                 expval = np.dot(exog, self.fe_params)
                 resid = resid - expval
 
-            solver = _smw_solver(self.scale, ex_r, ex2_r, cov_re_inv, 1 / vc_var)
+            solver = _smw_solver(self.scale, ex_r, ex2_r, cov_re_inv,
+                                 1 / vc_var)
             vir = solver(resid)
 
             xtvir = _dot(ex_r.T, vir)
 
             xtvir[0:k_re] = np.dot(self.cov_re, xtvir[0:k_re])
             xtvir[k_re:] *= vc_var
-            ranef_dict[group] = pd.Series(xtvir, index=self._expand_re_names(group))
+            ranef_dict[group] = pd.Series(
+                xtvir, index=self._expand_re_names(group))
 
         return ranef_dict
 
-
     @cache_readonly
     def random_effects_cov(self):
         """
@@ -2252,11 +2261,13 @@ def random_effects_cov(self):
         ranef_dict = {}
         for group_ix in range(self.model.n_groups):
 
-            ex_r, ex2_r = self.model._aex_r[group_ix], self.model._aex_r2[group_ix]
+            ex_r = self.model._aex_r[group_ix]
+            ex2_r = self.model._aex_r2[group_ix]
             label = self.model.group_labels[group_ix]
             vc_var = self.model._expand_vcomp(vcomp, group_ix)
 
-            solver = _smw_solver(self.scale, ex_r, ex2_r, cov_re_inv, 1 / vc_var)
+            solver = _smw_solver(self.scale, ex_r, ex2_r, cov_re_inv,
+                                 1 / vc_var)
 
             n = ex_r.shape[0]
             m = self.cov_re.shape[0]
@@ -2276,8 +2287,8 @@ def random_effects_cov(self):
 
         return ranef_dict
 
-
-    # Need to override since t-tests are only used for fixed effects parameters.
+    # Need to override since t-tests are only used for fixed effects
+    # parameters.
     def t_test(self, r_matrix, scale=None, use_t=None):
         """
         Compute a t-test for a each linear hypothesis of the form Rb = q
@@ -2307,15 +2318,16 @@ def t_test(self, r_matrix, scale=None, use_t=None):
         """
 
         if r_matrix.shape[1] != self.k_fe:
-            raise ValueError("r_matrix for t-test should have %d columns" % self.k_fe)
+            raise ValueError("r_matrix for t-test should have %d columns"
+                             % self.k_fe)
 
         d = self.k_re2 + self.k_vc
         z0 = np.zeros((r_matrix.shape[0], d))
         r_matrix = np.concatenate((r_matrix, z0), axis=1)
-        tst_rslt = super(MixedLMResults, self).t_test(r_matrix, scale=scale, use_t=use_t)
+        tst_rslt = super(MixedLMResults, self).t_test(
+            r_matrix, scale=scale, use_t=use_t)
         return tst_rslt
 
-
     def summary(self, yname=None, xname_fe=None, xname_re=None,
                 title=None, alpha=.05):
         """
@@ -2407,7 +2419,7 @@ def summary(self, yname=None, xname_fe=None, xname_re=None,
 
         sdf = pd.DataFrame(index=self.model.data.param_names, data=sdf)
         sdf.columns = ['Coef.', 'Std.Err.', 'z', 'P>|z|',
-                          '[' + str(alpha/2), str(1-alpha/2) + ']']
+                       '[' + str(alpha/2), str(1-alpha/2) + ']']
         for col in sdf.columns:
             sdf[col] = [float_fmt % x if np.isfinite(x) else ""
                         for x in sdf[col]]
@@ -2416,12 +2428,10 @@ def summary(self, yname=None, xname_fe=None, xname_re=None,
 
         return smry
 
-
     @cache_readonly
     def llf(self):
         return self.model.loglike(self.params_object, profile_fe=False)
 
-
     @cache_readonly
     def aic(self):
         if self.reml:
@@ -2432,7 +2442,6 @@ def aic(self):
             df = self.params.size + 1
         return -2 * (self.llf - df)
 
-
     @cache_readonly
     def bic(self):
         if self.reml:
@@ -2443,7 +2452,6 @@ def bic(self):
             df = self.params.size + 1
         return -2 * self.llf + np.log(self.nobs) * df
 
-
     def profile_re(self, re_ix, vtype, num_low=5, dist_low=1., num_high=5,
                    dist_high=1.):
         """
@@ -2609,6 +2617,7 @@ def _handle_missing(data, groups, formula, re_formula, vc_formula):
     for fml in forms:
         # Unicode conversion is for Py2 compatability
         rl = StringIO(fml)
+
         def rlu():
             line = rl.readline()
             return asunicode(line, 'ascii')


Test patch:
diff --git a/statsmodels/regression/tests/test_lme.py b/statsmodels/regression/tests/test_lme.py
--- a/statsmodels/regression/tests/test_lme.py
+++ b/statsmodels/regression/tests/test_lme.py
@@ -586,7 +586,7 @@ def do1(reml, irf, ds_ix):
                 mdf = md.fit(gtol=1e-7, reml=reml)
         else:
             mdf = md.fit(gtol=1e-7, reml=reml)
-    
+
     else:  # Independent random effects
         k_fe = rslt.exog_fe.shape[1]
         k_re = rslt.exog_re.shape[1]