statsmodels/statsmodels — pull request #1398 by kshedden (2014-02-17 16:53:59+00:00)
Title: WIP linear mixed models

This is an implementation of linear mixed models, in the style of Laird & Ware (as found in R lme4, SAS proc mixed, etc.). I don't know of other work on this in statsmodels, so this starts from scratch.

Lots of work is still needed, but it converges, and the results agree with R/lme4 on a number of examples.

Major issues:

* Only implements MLE, still needs REML.

* Uses numeric second derivative to get standard errors. This saves a lot of work and the results agree with lme4 at least 1-2 sig figs.

* EM and conjugate gradient are implemented. A few EM iterations (hopefully) get a good starting value, then CG takes over. There are many other algorithms for fitting LME's that we can explore. I don't know the details of what is done in other packages.

* Speed can probably be improved, both from streamlining the linear algebra, and from improving the handoff between EM and CG iterations.

* Very bare results class.

* Behavior when the MLE of a variance component is on the boundary needs improvement (will always be difficult).

* Almost no post-estimation results (AIC, LRT, prediction, etc.) are implemented. Some of these are tricky for mixed models. BLUP's for the random effects aren't too hard.

* Doesn't work with formulas. We need to devise a way for two formulas to be passed (one for the fixed effects structure, one for the random effects structure).

* Testing coverage is minimal.