Drug Safety Evidence that is Fit For
Purpose with National Healthcare
Databases
Sebastian Schneeweiss, MD, ScD
Professor of Medicine and Epidemiology

Division of Pharmacoepidemiology and Pharmacoeconomics,
Dept1

Database Study

followed by

RCT

Drug safety assessment

Drug utilization monitoring

Adaptive Biomedical Innovation

Database Study

followed by

RCT

Database Study

followed by

RCT

Why did these database studies get it right?
RCT

followed by

Database Study

RCT

無断転載・無断使用を禁止します。

followed by

Database StudyDatabase Study

followed by

Guidelines to build Confidence in Database Studies

RCTHow confident that the next study will get it right?
RCT

followed by

Database Study

Decision-relevant Evidence: The right evidence isgenerated the right way at the right time1) Meaningful evidence
2) Valid evidence3) Expedited evidence
4) Transparent evidence

1) Meaningful evidence

Data Quality

2) Valid evidence

Fit for purpose

3) Expedited evidence

Data Flexibility4) Transparent evidence

Meaningful statistics

Match data type to the question
Metrics that matter

Data quality: Fit for purpose

Data quality: Fit for purpose
Observational data

Accurate assessment of Exposure:Completeness of repeated uses
Prescribing vs. dispensing vs. use of drugs

Accurate assessment of Outcome:High specificity of outcome assessment when estimating relative
effect measures: risk ratio, rate ratio, hazard ratio
Reasonable sensitivity to preserve event counts

Research data

Transactional data

Data collected PRIMARILY for research

Data used SECONDARILY for research

For purpose
Data specifically forstudy purpose

Other purpose
Data intended forother studies

Reduced unobserved confounding
Pre-exposure measurement, avoid adjustment for intermediates

ExampleComplete assessment of Confounders:
Framingham Study
Cardiovas Health Study
Slone Birth Defects Study
Some registries

EOC

無断転載・無断使用を禁止します。

Health Study 1
Some registries

Other purpose

Clinicaldocumentation

Administrative

EHR-based studies
NDI linkage
Lab test databases
Some registries

Claims data studies
Geocoding/census

Basic epidemiologic measures are key!

Data flexibility: Match data type to the question

Counts of users
Duration of use
Population at risk
Incidence rates of events

Coordinating Node
Coordinating Node
Coordinates local analyses
Standardizes local analyses
Pools results from multiple
nodes to increase speed andprecision of analysis

PoolingAnalyses

Rapid-cycle Analytics

Node 3: Region Z

Rapid-cycle Analytics

Rapid-cycle Analytics

Node 1: Region X

Node 2: Plan Y

Schneeweiss et al. CP&T 2016

Local analytics guided by coordinatedstandards
To allow local linkage of data toenrich information
To inform local care management in
rapid cycles
To benchmark local care outcomes
To facilitate local data linkages

Net benefit calculation: We need risk differences!
Myocardialinfarction

Exposures:
Clopidogrel vs. prasogrel
Outcomes:
Benefit: MI prevention
Harm: Major bleed

1) Meaningful evidence
2) Valid evidence3) Expedited evidence
4) Transparent evidence

Avoid design flaws
Major bleed

Think of target trial

Approach confounding in context
Question & Data type dictate approach

Reduce investigator error
Structured approaches

Overallmortality

Report Risk Differences
15

Gagne et al. Drug Saf 2014

From transactional data to
study implementation

Today

VALUE OF EVIDENCE

Most Healthcare Decisions Need To Be Based On
Insights Above A Critical Evidence Threshold

Evidence Threshold
Hospital stay
May 1, 2016

DxRx

LabJul 1

Cohort Entry Date

Schneeweiss CPT 2016

無断転載・無断使用を禁止します。

Rx
Sept 1Rx
Nov 1V Dx
Jan 118

The many ways to implement a cohort study

Clarity in implementation reduces massive flaws

EDCohort Entry Date
New drugfirst fill

Avoidable design flaws:
Study design flaws= substantial bias

Reverse Causation
Immortal time bias
Adjustment for causal intermediates
Depletion of susceptible
Unobservable time bias
20

Intrinsic Study Characteristics
Internal validity (bias)
External validity (generalizability, transportability)
Precision
Heterogeneity in risk or benefit (personalized evidence)
Ethical consideration (equipoise)

User guidance reduces investigator error

External Study Characteristics
Timeliness (rapidly changing technology, policy needs)
Logistical constraints (study size, complexity, cost)
Data availability, quality, completeness

Schneeweiss et al. CP&T 2016

Intrinsic Study Characteristics
Internal validity (bias)
External validity (generalizability, transportability)
Precision
Heterogeneity in risk or benefit (personalized evidence)
Ethical consideration (equipoise)

Dealing with Confounding
Confounding

External Study Characteristics
Timeliness (rapidly changing technology, policy needs)
Logistical constraints (study size, complexity, cost)
Data availability, quality, completeness

Huge improvements
over past 2 decades

Unmeasured
Confounders

Schneeweiss et al. CP&T 2016

Schneeweiss, PDS 2006

無断転載・無断使用を禁止します。

Sensitivity analyses

Why we like propensity score matching
Database
Studies

Propensity scores:
Few outcomes
Many covariates

FDA likes to see sensitivity analyses to check the robustness of findings
However, rarely done because too labor-intensiveNo control ofcovariate
measurement
Large numbersof covariates canbe generated

Matching:
Transparency in the achieved balance
Trimming of subjects that cannot be matched(areas of no support)
Schneeweiss, Rassen et al. Epidemiol 2009;
Rassen et al. Am J Epidemiol 2014

Sensitivity analyses with a validated platform

Sensitivity analyses with a validated platform

CONFIDENTIALCONFIDENTIAL

Sensitivity analyses with a validated platform

Misspecified Exposure Risk Window
The ERW is shorter than the biologic effect or too long
The ERW may or may not overlap with a grace period
Sensitivity analysis:
Shorten and lengthen ERW

TimeCED
Schneeweiss S & Avorn J, JClinEpi 2005

(4) Varying length ofexposure risk window
MacMahon AD et al., PDS 1998

CONFIDENTIAL

無断転載・無断使用を禁止します。

Sensitivity analyses with a validated platform

Sensitivity analyses help build confidence

CONFIDENTIAL

Sensitivity analyses help build confidence

1) Meaningful evidence
2) Valid evidence3) Expedited evidence
4) Transparent evidence

Speed through closeness to data
Being embedded in the data stream

Speed through data standardsSentinel common data model

Speed through analytic tools
Speed without compromising science

How can we optimize validity at high speed
across multiple data systems?

Rapid-cycle analytics without compromisingvalidity across multiple data systems

無断転載・無断使用を禁止します。

When is a benefit real?
9++

Benefit

Benefit7--

PS-match
PS-match

PS-match++

Benefit

Benefit13--

PS-match
PS-match

PS-match
PS-match++
BenefitPS-match

BenefitPS-match--PS-match
PS-match
PS-match

PS-match
PS-match
PS-match

無断転載・無断使用を禁止します。

1) Meaningful evidence
2) Valid evidence3) Expedited evidence
4) Transparent evidence

Decision making in a monitoring system
PromisingSuperior?

better

Questionable

Transparency -> Reproducibility
Be able to reproduce in same data

Shared analytics & auditability
worseWithstand detailed audits of past data

Accepted statistical procedure
We have plenty statistical tools

Questionable:- Investigate subgroupeffects- Continue evaluation

Promising:- Continue program- Continue evaluation- Moderately expand
programSuperior:- Widely disseminate

43

Schneeweiss, Shrank, Ruhl, Maclure, IJTAHC 2015

Scientific validation of a platform against FDA standard
Wang S. et al. CP&T 2016:

Extremely highagreement with current
standard (FDA ModularProgram)
Kim S. et al. A&R in press:

Fralick M. et al. Ann Int Med in press:

Patorno E. et al. in progress:

CONFIDENTIAL

Analytic tools are build for transparency
Tabular format

Standardized Text

Analytic tools are build for transparency
Tabular format

Standardized Text

International Society for Pharmacoeconomics
and Outcomes Research

無断転載・無断使用を禁止します。

Case study: TCZ

Depositing codes for reproducibility

Tocilizumab (TCZ) inhibits the IL-6 receptor
TCZ was approved by the FDA in 2010 for treatment of rheumatoid
arthritis
Early studies showed an increase in LDL and triglycerides
FDA wanted to ensure cardiovascular (CV) safety by comparing TCZ against
current standard of care, TNF inhibitors

Case study: Non-randomized database study

Case study: ENTRACTE trial

Case study: Comparison

Decision-relevant Evidence

Observational study

ENTRACTE RCTMulti-database cohort study

Parallel group RCT

TCZ vs. any TNFi

TCZ vs. etanercept

8,790 TCZ patients

1,538 TCZ patients

HR = 0.90 (0.60-1.36)

HR = 0.89 (0.54-1.49)(FDA CV safety: rule out HR of 1.40)

Oct. 2015 Apr. 2016 (6 months)

Aug 2011 May 2016 (57 months)

Cost: $y

Cost: 100 X $y

Full transparency via Aetion report

Full transparency via GCP

MeaningfulValid

Expedited

Transparent

無断転載・無断使用を禁止します。