Three ways to run Linear Mixed Effects Models in Python Jupyter Notebooks | by Jin Hyun Cheong, PhD | Towards Data Science

One of the reasons I could not fully switch out of R to Python for data analyses was that linear mixed effects models were only available in R. Linear mixed effects models are a strong statistical method that is useful when you are dealing with longitudinal, hierarchical, or clustered data. Simply put, if your data has repeated samples, correlated data, or natural “groupings”, such as repeated responses from the same individual, data clustered by different geographical locations, or even data from a group of people interacting, you would probably want to use linear mixed effects models in your analyses.
You can learn more about exactly how and why linear mixed effects models or linear mixed effects regressions (LMER) are effective from these resources (Lindstrom & Bates, 1988) (Bates et al., 2015), but in this tutorial, we will focus on how you can run these models in a Python Jupyter Notebook environment. In the early days, one had save the data from Python, open up the data in R and run the LMER model. Over the years, R & Python got to know each other a little better and several options have emerged for running LMER analyses in Python. Here are three options we will be exploring for which I provide sample code for each:
LMER in StatsmodelsAccessing LMER in R using rpy2 and %RmagicPymer4 for seamless access to LMER in R
LMER in Statsmodels
Accessing LMER in R using rpy2 and %Rmagic
Pymer4 for seamless access to LMER in R
Here is a Google Colab Jupyter Notebook to follow along these methods!
Written by Jin Hyun Cheong
# Install Statsmodels
# Adapted from https://www.statsmodels.org/stable/examples/notebooks/generated/mixed_lm_example.html
!pip install -q statsmodels

# Load packages
import numpy as np
import pandas as pd
import statsmodels.api as sm
import statsmodels.formula.api as smf

# Load dataset
data = sm.datasets.get_rdataset('dietox', 'geepack').data

# Run LMER
md = smf.mixedlm("Weight ~ Time", data, groups=data["Pig"], re_formula="~Time")
mdf = md.fit(method=["lbfgs"])
print(mdf.summary())

           Mixed Linear Model Regression Results
===========================================================
Model:             MixedLM  Dependent Variable:  Weight    
No. Observations:  861      Method:              REML      
No. Groups:        72       Scale:               6.0372    
Min. group size:   11       Likelihood:          -2217.0475
Max. group size:   12       Converged:           Yes       
Mean group size:   12.0                                    
-----------------------------------------------------------
                 Coef.  Std.Err.   z    P>|z| [0.025 0.975]
-----------------------------------------------------------
Intercept        15.739    0.550 28.603 0.000 14.660 16.817
Time              6.939    0.080 86.925 0.000  6.783  7.095
Group Var        19.503    1.561                           
Group x Time Cov  0.294    0.153                           
Time Var          0.416    0.033                           
===========================================================


# Install R and Rpy2
!apt-get install r-base
!pip install -q rpy2

# Install LMER packages (THIS TAKES ABOUT 3~5 minutes)
packnames = ('lme4', 'lmerTest', 'emmeans', "geepack")
from rpy2.robjects.packages import importr
from rpy2.robjects.vectors import StrVector
utils = importr("utils")
utils.chooseCRANmirror(ind=1)
utils.install_packages(StrVector(packnames))

%load_ext rpy2.ipython
# Enable cell magic for Rpy2 interface

%%R 
# load LMER libraries
library(lme4)
library(lmerTest)
# load dataset in R
data(dietox, package='geepack')
# LMER model in R
m<-lmer('Weight ~ Time + (1+Time|Pig)', data=dietox)
print(summary(m))

R[write to console]: Loading required package: Matrix

R[write to console]: 
Attaching package: ‘lmerTest’


R[write to console]: The following object is masked from ‘package:lme4’:

    lmer


R[write to console]: The following object is masked from ‘package:stats’:

    step



Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: "Weight ~ Time + (1+Time|Pig)"
   Data: dietox

REML criterion at convergence: 4434.1

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-6.4286 -0.5529 -0.0416  0.4841  3.5624 

Random effects:
 Groups   Name        Variance Std.Dev. Corr
 Pig      (Intercept) 19.4929  4.415        
          Time         0.4161  0.645    0.10
 Residual              6.0375  2.457        
Number of obs: 861, groups:  Pig, 72

Fixed effects:
            Estimate Std. Error       df t value Pr(>|t|)    
(Intercept) 15.73865    0.55012 71.03978   28.61   <2e-16 ***
Time         6.93901    0.07983 71.06517   86.93   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
     (Intr)
Time 0.005 

Exchange variables between Python and R using the -i and -o arguments.
%%R -i data -o betas
# Load data from Python into R
m<-lmer('Weight ~ Time + (1|Pig)', data=data)
print(summary(m))
betas <- fixef(m)

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: "Weight ~ Time + (1|Pig)"
   Data: data

REML criterion at convergence: 4809.6

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-4.7118 -0.5696 -0.0943  0.4877  4.7732 

Random effects:
 Groups   Name        Variance Std.Dev.
 Pig      (Intercept) 40.39    6.356   
 Residual             11.37    3.371   
Number of obs: 861, groups:  Pig, 72

Fixed effects:
             Estimate Std. Error        df t value Pr(>|t|)    
(Intercept)  15.72352    0.78805  83.00293   19.95   <2e-16 ***
Time          6.94251    0.03339 788.03850  207.94   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
     (Intr)
Time -0.275

betas

array([15.72352307,  6.94250501])
Specify variables as a factor, set contrats, or relevel categorical variables.
%%R -i data
# Specify Evit as a factor and set a polynomial contrast.
data$Evit <- as.factor(data$Evit)
contrasts(data$Evit) <- contr.poly 
m<-lmer('Weight ~ Time * Evit + (1|Pig)', data=data)
print(summary(m))

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: "Weight ~ Time * Evit + (1|Pig)"
   Data: data

REML criterion at convergence: 4796.1

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-4.9037 -0.5661 -0.0972  0.4843  5.0245 

Random effects:
 Groups   Name        Variance Std.Dev.
 Pig      (Intercept) 39.73    6.304   
 Residual             11.21    3.347   
Number of obs: 861, groups:  Pig, 72

Fixed effects:
             Estimate Std. Error        df t value Pr(>|t|)    
(Intercept)  15.72208    0.78212  80.68619  20.102  < 2e-16 ***
Time          6.94538    0.03316 786.03373 209.422  < 2e-16 ***
Evit.L       -0.02168    1.35495  80.65930  -0.016  0.98727    
Evit.Q       -1.04967    1.35440  80.71312  -0.775  0.44060    
Time:Evit.L  -0.12420    0.05732 786.01871  -2.167  0.03054 *  
Time:Evit.Q  -0.16742    0.05757 786.04862  -2.908  0.00374 ** 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
            (Intr) Time   Evit.L Evit.Q Tm:E.L
Time        -0.275                            
Evit.L      -0.034  0.009                     
Evit.Q       0.001  0.001 -0.024              
Time:Evit.L  0.009 -0.030 -0.275  0.006       
Time:Evit.Q  0.001 -0.006  0.006 -0.275 -0.021

%%R -i data
data$Evit <- as.factor(data$Evit)
m<-lmer('Weight ~ Time * relevel(Evit, ref="Evit100") + (1|Pig)', data=data)
print(summary(m))

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: "Weight ~ Time * relevel(Evit, ref=\"Evit100\") + (1|Pig)"
   Data: data

REML criterion at convergence: 4793.9

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-4.9037 -0.5661 -0.0972  0.4843  5.0245 

Random effects:
 Groups   Name        Variance Std.Dev.
 Pig      (Intercept) 39.73    6.304   
 Residual             11.21    3.347   
Number of obs: 861, groups:  Pig, 72

Fixed effects:
                                            Estimate Std. Error        df
(Intercept)                                 16.57914    1.35412  80.74008
Time                                         7.08207    0.05770 786.06338
relevel(Evit, ref = "Evit100")Evit000       -1.27025    1.93539  80.68667
relevel(Evit, ref = "Evit100")Evit200       -1.30091    1.89561  80.71319
Time:relevel(Evit, ref = "Evit100")Evit000  -0.11722    0.08207 786.03367
Time:relevel(Evit, ref = "Evit100")Evit200  -0.29287    0.08057 786.04900
                                           t value Pr(>|t|)    
(Intercept)                                 12.243  < 2e-16 ***
Time                                       122.749  < 2e-16 ***
relevel(Evit, ref = "Evit100")Evit000       -0.656 0.513482    
relevel(Evit, ref = "Evit100")Evit200       -0.686 0.494505    
Time:relevel(Evit, ref = "Evit100")Evit000  -1.428 0.153611    
Time:relevel(Evit, ref = "Evit100")Evit200  -3.635 0.000296 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:
                 (Intr) Time   r(E,r="E100")E0 r(E,r="E100")E2 T:(E,r="E100")E0
Time             -0.275                                                        
r(E,r="E100")E0  -0.700  0.193                                                 
r(E,r="E100")E2  -0.714  0.197  0.500                                          
T:(E,r="E100")E0  0.193 -0.703 -0.275          -0.138                          
T:(E,r="E100")E2  0.197 -0.716 -0.138          -0.275           0.503          

"""pymer4 is down due to recent updates in rpy2. 
It's necessary to install via source until this issue is resolved. 
"""
!git clone https://github.com/ejolly/pymer4.git
!cd pymer4 && git pull origin dev && git checkout dev && pip install -q .

Cloning into 'pymer4'...
remote: Enumerating objects: 127, done.
remote: Counting objects: 100% (127/127), done.
remote: Compressing objects: 100% (87/87), done.
remote: Total 1712 (delta 45), reused 104 (delta 36), pack-reused 1585
Receiving objects: 100% (1712/1712), 5.80 MiB | 13.55 MiB/s, done.
Resolving deltas: 100% (962/962), done.
From https://github.com/ejolly/pymer4
 * branch            dev        -> FETCH_HEAD
Updating 7e98fa2..23d0282
Fast-forward
 .travis.yml                 |  2 +-
 conda/meta.yaml             |  2 +-
 docs/new.rst                |  8 ++++++++
 pymer4/io.py                |  4 ++--
 pymer4/models/Lm.py         | 11 +++++++----
 pymer4/models/Lm2.py        |  2 +-
 pymer4/models/Lmer.py       | 21 ++++++++++++++++-----
 pymer4/simulate.py          | 10 +++++-----
 pymer4/stats.py             | 31 ++++++++++++++++---------------
 pymer4/tests/test_models.py |  3 ++-
 pymer4/utils.py             | 11 +++++------
 pymer4/version.py           |  2 +-
 requirements.txt            |  2 +-
 13 files changed, 66 insertions(+), 43 deletions(-)
Branch 'dev' set up to track remote branch 'dev' from 'origin'.
Switched to a new branch 'dev'
  Building wheel for pymer4 (setup.py) ... done

# Install pymer4
!pip install -q pymer4
# load pymer4
from pymer4.models import Lmer
model = Lmer('Weight ~ Time + Evit + (1 + Time|Pig)', data=data)
display(model.fit())
# ANOVA results from fitted model
display(model.anova())
# Plot estimated model coefficients 
model.plot_summary()

Formula: Weight~Time+Evit+(1+Time|Pig)

Family: gaussian	 Inference: parametric

Number of observations: 861	 Groups: {'Pig': 72.0}

Log-likelihood: -2214.080 	 AIC: 4428.160

Random effects:

                 Name     Var    Std
Pig       (Intercept)  19.701  4.439
Pig              Time   0.416  0.645
Residual                6.037  2.457

             IV1   IV2   Corr
Pig  (Intercept)  Time  0.081

Fixed effects:


SS Type III Analysis of Variance Table with Satterthwaite approximated degrees of freedom:
(NOTE: Using original model contrasts, orthogonality not guaranteed)

<matplotlib.axes._subplots.AxesSubplot at 0x7faebd04de10>
Currently, the simplest out-of-the-box solutions is to use the LMER implementation in the Statsmodels package (examples here). Installation is the easiest and as simple as pip install statsmodels. After installation, you can run LMER like the following.
To provide a bit more context, we are analyzing the dietoxdataset (learn more about the dataset here) to predict the weight of pigs as a function of time with a random slope specified by re_formula="~Time" and a random intercept automatically specified by groups=data["Pig"]. The outputs look like the figure below with the coefficients, standard errors, z stats, p values, and 95% confidence intervals.
While this is splendid, specifying the random effects with this syntax is somewhat inconvenient and diverges from the traditional formulaic expressions used in LMER in R. For example in R, the random slopes and intercepts are specified within the model formula, in a single line, such as:
lmer('Weight ~ Time + (1+Time|Pig)', data=dietox)
This leads to our second option of using LMER in R through a direct interface between Python and R through rpy2.
The second option is to directly access the original LMER packages in R through the rpy2 interface. The rpy2 interface allows users to toss data and results back and forth between your Python Jupyter Notebook environment and your R environment. rpy2 used to be notoriously finicky to install, but it has gotten more stable over the years. To use this option, you’ll need to install R in your machine as well as rpy2 which can be achieved in Google Colab with the following code:
The first line installs R using Linux syntax. If you are using a Mac or Windows you can achieve this by simply following the R installation instructions. The next set of lines install rpy2 then uses rpy2 to install the lme4 andlmerTest packages.
Next, you’ll need to activate the Rmagic through the code, in you Jupyter Notebook cell by running the following code.
%load_ext rpy2.ipython
After this, any Jupyter Notebook cell starting with %%R will allow you to run R command from your notebook. For example, to run the same model that we ran in statsmodels, you would do the following:
Note that this code starts with %%R which signals that this cell contains R code. We are also using a much simpler formulaic expression than statsmodels, We were able to specify that our groupings is Pigs and that we are estimating both random slopes and intercepts through (1+Time|Pig). This will give you the results that you would be more familiar with if you had been using LMER in R.
And as I mentioned earlier, you can also pass your Pandas DataFrames to R. Remember that data DataFrame we loaded earlier in the statsmodels section? We can pass that to R, run the same LMER model, and retrieve the coefficients like this:
The -i data sends our Python Pandas DataFramedata into R and we use that to estimate our model m . Next, we retrieve the betas coefficients from the model with beta <- fixef(m) which is exported back to our notebook, because we specified in the first line -o betas.
The best part of this approach is that you can add additional code in the R environment before running the model. For example, maybe you want to make sure a column called Evit is recognized as a factor with data$Evit <- as.factor(data$Evit), specify a new contrast for this categorical variable using contrasts(data$Evit) <- contr.poly , or even relevel your categorical data in the formula itself. All this can be easily achieved when using rpy2 to access LMER from R.
To summarize, rpy2 interface gives you the most flexibility and the ability to access LMER and additional functions in R which you might be more familiar with if you are transitioning from R to Python. Lastly, we’ll touch a middle ground option using the Pymer4 package.
Pymer4 (Jolly, 2018) can be a convenient middle ground between using LMER directly through rpy2 and using an LMER implementation in Statsmodels. This package basically brings you the convenience of using R formulaic syntax but in a more Pythonic way, without having to deal with the R magic cells.
We covered 3 ways to run Linear Mixed Effects Models from a Python Jupyter Notebook environment. Statsmodels can be the most convenient but the syntax might be unfamiliar to users already experienced with LMER in R syntax. Using rpy2 gives you the most flexibility and power but this can get messy as you need to use Rmagic to switch between Python and R cells. Pymer4 is the great compromise solution which provides convenient access to LMER in R while minimizing the switch cost between languages.
Here is a Google Colab Jupyter Notebook to run all the tutorials.
Thank you for reading and feel free to check out my other Data Science tutorials!