Creation of a ustekinumab external control arm for Crohn’s disease using electronic health records data: a pilot study
Authors: Rudrapatna Vivek A. , Cheng Yao-Wen , Feuille Colin , Mosenia Arman , Shih Jonathan , Shi Yongmei , Roberson Olivia , Rubin Benjamin , Butte Atul J. , Mahadevan Uma , Skomrock Nicholas , Erondu Ngozi , Chehoud Christel , Rahim Saquib , Apfel David , Curran Mark , Khan Najat S. , O’Brien Christopher , Terry Natalie , Martini Benjamin D 
Published: November 2021

Objectives

The use of external control arms to study treatment effects is growing in interest among drug sponsors and regulators. However, experience with performing these kinds of studies for complex, immune-mediated diseases is limited. We sought to analyze a retrospective cohort of Crohn’s patients to predict the outcome of a prospective cohort.

Methods

We queried electronic health records databases and screened records at the University of California, San Francisco to identify patients meeting the eligibility criteria of TRIDENT, a concurrent trial involving ustekinumab as a reference arm. Timepoints were defined to balance the tradeoff between missing disease activity and bias. We compared two imputation models by their impacts on cohort membership and outcomes. We compared the results of ascertaining disease activity using structured data algorithms against manual review. We used these data to estimate ustekinumab’s real-world effectiveness.

Results

Screening identified 183 patients. 30% of the cohort had missing baseline data. Two imputation models were tested and had similar effects on cohort definition and outcomes. Algorithms for ascertaining non-symptom-based elements of disease activity were similar in accuracy to manual review. The final cohort consisted of 56 patients. 34% of the cohort was in steroid-free clinical remission by week 24.

Conclusions

We predict that a third of the ustekinumab-treated cohort in TRIDENT will be in steroid-free remission by week 24. However, our prediction is limited by substantial missing data. Efforts to improve real-world data capture and align trial design with clinical practice may enable more robust future studies and improve trial efficiency.

STUDY HIGHLIGHTS

WHAT IS KNOWN

External control arm studies are receiving growing interest from drug sponsors and regulators as a potential source of real-world evidence

However, the feasibility and robustness of this approach is currently limited

Ustekinumab is an FDA-approved treatment of moderately to severely active Crohn’s disease

WHAT IS NEW HERE

We derived a retrospective cohort of patients designed to resemble the participants of TRIDENT, a concurrent phase 2b trial using ustekinumab as a reference arm

We predict that about 34% of the ustekinumab-assigned participants in TRIDENT will be in steroid-free clinical remission by week 24.

EHR structured data algorithms may be an accurate and less laborious alternative to manual abstraction of non-symptom-based components of the Crohn’s Disease Activity Index

Real-world practice differs from controlled studies in important ways, including the treatment goals and the timing of encounters

These differences can pose challenges to the feasibility of external control arm studies, and must be addressed to enable this novel study design

The methods, data, and code used in this pilot study are shared here for reproducibility and enhancement by others