Using R in a Regulatory Environment: 

some FDA perspectives

Paul Schuette

“IieミtifiI Coマputiミg Cooヴdiミatoヴ

Food and Drug Administration

Center for Drug Evaluation and Research 

OffiIe of Tヴaミslatioミal “IieミIes

OffiIe of BiostatistiIs 

Paul.Schuette@fda.hhs.gov

www.fda.gov
2

Disclaimer

This presentation reflects the views of 

the author and should not be construed 

to represent the FDA's views or policies.
3

Statistical Software Clarifying Statement

さFDA does ミot ヴeケuiヴe use of aミy speIifiI soft┘aヴe foヴ statistiIal aミalyses, 

and statistical software is not explicitly discussed in Title 21 of the Code 

of Federal Regulations [e.g., in 21CFR part 11]. However, the software 

package(s) used for statistical analyses should be fully documented in the 

submission, including version and build identification. 

As noted in the FDA guidance, E9 Statistical Principles for Clinical Trials, 

けThe Ioマputeヴ soft┘aヴe used foヴ data マaミageマeミt aミd statistiIal 

analysis should be reliable, and documentation of appropriate software 

testiミg pヴoIeduヴes should He a┗ailaHle.げ “poミsoヴs aヴe eミIouヴaged to 

consult with FDA review teams and especially with FDA statisticians 

regarding the choice and suitability of statistical software packages at an 

eaヴly stage iミ the pヴoduIt de┗elopマeミt pヴoIess. ざ

https://www.fda.gov/downloads/forindustry/datastandards/studydatasta

ndards/ucm587506.pdf
4

R for Regulatory Review

How is R used for regulatory review work?

• Reviewers may opt to perform their analyses 

using R rather than commercial packages.

• R is used for graphics and data visualization.

• Simulations in general.

• Bayesian Methods

– JAGS

– Stan

• Complex, Innovative Clinical Designs (PDUFA VI)
5

Some R packages for Biostatistics

• survival, Therneau

• Hmisc, Harrell et al

• DoseFinding, Bornkamp, Pinheiro, and Bretz

• gsDesign, Anderson

• Beanz, Wang et al

• ORCI, Sun

IDE RStudio is used extensively at FDA.
6

Product Label 

https://www.accessdata.fda.gov/drugsatfda_docs/label/2016/208073s000lbl.pdf
7

Another Product Label

R Graphic. Drug for the reduction of elevated intraocular pressure (IOP) in 

patients with open-angle glaucoma or ocular hypertension 

https://www.accessdata.fda.gov/drugsatfda_docs/label/2017/208254lbl.pdf
8

Data Anomaly Detection

Use open source software to 

detect potential data problems

1. DABERS: Data Anomalies in 

BioEquivalence R Shiny 

app. Used for PK/PD 

profiles.

2. Cooperative Research and 

Development Agreement 

(CRADA) with CluePoints 

for detecting anomalous 

clinical trial sites.

Example of CRADA software 

output

www.fda.gov
9

R Shiny Apps

Internal to FDA

• Waterfall Plot

• Hepatotoxicity

• Demographics

• PRO

• DABERS

External to FDA (openFDA)

• LRT app for Adverse Event analyses
10

Waterfall Plot
11

Hepatotoxicity

The Hepatotoxicity tool bolsters 

analysis of Drug Induced Liver 

Injury (DILI) through a composite 

visualization that includes both 

pre-treatment and on-treatment 

prevalence of ALT and BILI in 

terms of Hyげs Law candidate 

laboratory Upper Limit Normal 

(ULN) thresholds as well as the 

magnitude of these elevations 

normalized by respective baseline 

test results. This analysis is 

particularly useful for studies in 

which subjects have elevated liver 

enzyme test results at baseline 

(e.g., subjects with Chronic 

Hepatitis C).

www.fda.gov
12

Demographic Tool

The Demographic Tool 

provides targeted 

descriptive statistics and 

safety endpoint analysis 

for demographic 

subgroups, including age, 

sex, race, and ethnicity. 

The tool has a simple user 

interface that dynamically 

walks end-users through 

the process of executing 

the analysis. The example 

deals with a safety 

endpoint analysis.
13

FAERS data, OpenFDA

https://openfda.shinyapps.io/LRTest/
14

Text Plot from LRT app,

Drug: aspirin 
15

Birthdate Problem
16

R for Research

• Data Mining and Machine Learning (also with 

Python)

• Simulations

• Evaluation of methodology

• Oak Ridge Institute for Science and Education 

(ORISE) Internships

• Broad Agency Agreements (BAA)

• Cooperative Research and Development 

Agreements (CRADA)

• PhUSE, DIA, and ASA working groups
17

Research, Pediatric vs Adult ADRs
18

Concluding Observations

• Open source tools such as R offer cost effective ways for 

FDA to carry out its public health mission, and to 

enhance communications with the public, health care 

providers and regulated industry. 

• R is widely used in academe, and is the first choice for 

many recent graduates.

• Managing packages and dependencies can be 

challenging.

• IミteヴaIti┗e tools suIh as ‘ “hiミy Iaミ eミhaミIe useヴsげ 

experience and understanding. 

• We still need subject matter experts to help frame 

questions and draw appropriate conclusions.
19

Acknowledgements

• Yan Wang

• Jimmy Wong

• Daniel Choi

• Austin Taylor

• Xiaomei Liu