Generating interpretable predictions about antidepressant treatment stability using supervised topic models
Authors: Hughes Michael C. , Pradier Melanie F. , Ross Andrew Slavin , McCoy Thomas H. , Perlis Roy H. , Doshi-Velez Finale 
Published: March 2020

Importance

In the absence of readily-assessed and clinically-validated predictors of treatment response, pharmacologic management of major depressive disorder (MDD) often relies on trial and error.

Objective

To utilize electronic health records to identify predictors of treatment response, while preserving interpretability of predictions despite large numbers of covariates.

Design

Retrospective cohort study.

Setting

Two academic medical centers in Boston, including outpatient primary and specialty care clinics.

Participants

81,630 adults with a coded diagnosis of MDD.

Exposure

Treatment with 1 or more of 11 standard antidepressants.

Main Outcomes and Methods

Stable treatment, intended as a proxy for treatment effectiveness, defined as continued prescription of an antidepressant for 90 days. We trained supervised topic models to extract 10 interpretable covariates from coded clinical data for stability prediction. Then, using data from one hospital system (Site A) we trained generalized linear models and ensembles of decision trees to predict stability outcomes from topic features that summarize patient history. We evaluated on held-out patients from Site A as well as all individuals from a second hospital system (B).

Results

Among the 81,630 adults (31% male; age 18-80 with mean 48.46), we identified 55,303 who reached a stable treatment regimen during follow-up. For held-out patients from Site A, mean area-under-the-receiver-operating-characteristic-curve (AUC) discrimination for general stability outcome was 0.627 (95% confidence interval (CI) 0.615 - 0.639) for our supervised topic model with 10 covariates. In evaluation on site B, our approach achieved similar AUC of 0.619 (95% CI 0.610 - 0.627). Building models to predict stability specific to a particular drug did not improve upon predicting general stability, even when using a harder-to-interpret ensemble classifier and 9,256 coded covariates (specific AUC = 0.647, 95% CI 0.635-0.658; general AUC = 0.661, 95% CI 0.648-0.672). Topics coherently captured clinical concepts associated with treatment response.

Conclusions and Relevance

Coded clinical data available in electronic health records facilitated prediction of general treatment response, but not response to specific medications. While greater discrimination is likely required for clinical application, our results provide a simple and transparent baseline for such studies.

Funding

Oracle Labs, Harvard SEAS, and National Institute of Mental Health.

Links

Supplement document providing more results, links to interactive visualizations, and detailed procedures for reproducibility https://www.michaelchughes.com/papers/HughesEtAl_medRxiv2020_Supplement.pdf

STROBE checklist: https://www.michaelchughes.com/papers/HughesEtAl_medRxiv2020_STROBE_checklist.pdf

Open-source code for our proposed machine learning methods https://github.com/dtak/prediction-constrained-topic-models

Key Points

Question

How well can coded clinical data from electronic health records be used to predict achievement of a stable antidepressant regimen in major depressive disorder?

Findings

In this in silico cohort study of 81,630 adults, we identified 55,303 who reached a stable antidepressant treatment regimen. Predictions using generalized linear models or ensembles of decision trees applied to diagnosis, procedure, and medication codes, as well as low-dimensional summaries of these codes via supervised topic models, achieved area under receiver operating characteristic curve values of ∼0.62-0.65; treatment-specific models performed no better than general treatment outcome models.

Meaning

Coded clinical data can facilitate prediction of antidepressant treatment outcomes, but medication-specific models do not outperform general response prediction models.