Welcome to today’s 

FDA/CDRH Webinar

Thank you for your patience while we register all of 

today’s participants.

If you have not connected to the audio portion of the 

webinar, please do so now:

Dial: 888-456-0327

International: 1-312-470-0074 

Passcode: 8872340 

Conference number: PW9776689
Adaptive Designs for Medical 

Device Clinical Studies

Final Guidance

Presenters: 

Gerry Gray, Ph.D

Deputy Director for Division of Biostatistics, Office of Surveillance and Biometrics 

Center for Devices and Radiological Health (CDRH)

Estelle Russek-Cohen, Ph.D

Division Director for Biostatistics, Office of Biostatistics and Epidemiology

Center for Biologics Evaluation and Research (CBER)

September 22, 2016
3 

Agenda

• Guidance Background

• Adaptive Design definition, benefits and limitations

• When to Choose an Adaptive Design?

• Adaptive Design Considerations and Approaches

• Adaptive Design in Centers for Devices and 

Biologics

• Question & Answer
www.fda.gov 4 
5 

What are Adaptive Designs?

• Studies that include a prospectively planned 

opportunity(ies) to change the study design based 

on accumulating data during the course of the 

study without undermining study integrity and 

validity.

– This potentially includes modifications after a trial is 

underway but before unmasking/unblinding

– Does NOT include retrospective or ad hoc changes 

introduced after outcomes are known, nor attempts to 

salvage failed trials

www.fda.gov
6 

Why Consider an Adaptive Design?

• Clinical study designs are based on expected 

outcomes, including event rates for treatment and 

control groups, accrual rates, dropout rates, 

variances, etc.

• If everything goes exactly according to plan, then a 

fixed trial design is optimal.

• But these original assumptions are really only 

guesses and are often far off.

• Adaptive designs can address moderate levels of 

uncertainty in these assumptions.

www.fda.gov
7 

Adaptive Designs

• Can be frequentist or Bayesian

– CDRH and CBER have experience with both types

• Adaptive designs are usually analyzed in stages, 

with potential adaptations pre-specified for 

each stage.

• For device submissions, the adaptation needs to 

be planned in great detail before any unblinded

data are observed.

www.fda.gov
8 

Examples of adaptations

• Sample size adjustment, including: 

– Group sequential designs

– Sample size re-estimation

– Adaptive recruitment (Bayesian)

• Adaptive randomization

• Changes to study eligibility criteria (patient 

inclusion/exclusion) 

• Drop/add/change treatment arms

• Changes to Statistical Analysis Plan or Hypotheses

• Endpoint changes

• Study duration

www.fda.gov
9 

Differences between a Fixed and 

Adaptive Trial

FACTOR FIXED TRIAL ADAPTIVE TRIAL

Number of patients Fixed Variable

Patient population Fixed Can be narrowed

Randomization Constant probability Can be adjusted

Primary hypothesis Fixed Can be changed

Decision rules Simple Complex 

www.fda.gov
10

Adaptive Design Pros and Cons

PRO CON

Efficiency 

 (patients, duration, money)

Complexity

May be shorter duration Unknown duration, possibly 

longer

More likely to succeed Results may be harder to 

interpret

Fail quickly Harder to conduct

Patient protection Potential operational bias

Flexibility Not as flexible

www.fda.gov
11

Should a trial be adaptive?

• Feasibility:

– If there is time to adapt.

• i.e. if all the patients are enrolled before any outcomes 

are known there is no time to make any changes.

– If there are multiple important endpoints it may not 

be possible to simultaneously adapt for all of them.

– If sample size is driven by safety concerns then 

adapting on effectiveness won’t be feasible.

www.fda.gov
12

Should a trial be adaptive? Contd.

• Advantageous if:

– There is some uncertainty in design parameters that 

can be addressed by adapting.

– The operating characteristics of the adaptive design 

are favorable over a wide range of plausible 

scenarios.

• Especially consider “anticipated regret” scenarios

– There are clear advantages over the alternative fixed 

design.

www.fda.gov
13

Should a trial be adaptive? Contd.

• NOT advantageous if:

– There are too many unknowns that affect which 

design is optimal

– Design parameters are not known with sufficient 

precision to allow efficient adaptation

– The gain over a fixed design is small

www.fda.gov
14

What phases?

• Adaptive designs can be used at any phase of 

development

– Dose finding studies are often adaptive

– Seamless phase II/III studies

– Pivotal studies 

• CDRH experience is almost entirely with pivotal 

studies. CBER has more experience with early 

phases. 

• From regulatory perspective, most 

concern/interest is in pivotal studies

www.fda.gov
15

Regulatory concerns

• Type I error control

– Operating characteristic of design

• Estimation of treatment effect

– Appropriate estimates & confidence intervals

• Operational bias

– Who knows what, when, and how it affects behavior 

that might bias the results of the study

• Power considerations

• Practical considerations

– Trial infrastructure, conduct

www.fda.gov
www.fda.gov 16

CDRH experience

• 251 adaptive studies 2007-2013*

– Mostly designs (IDEs)

– Some (32) product submissions (PMA, 510(k))

• Overwhelming majority are sample size related 

adaptations

– Frequentist (156/176 sample size related adaptations)

• Group sequential, sample size re-estimation 

– Bayesian (67/75 sample size related)

• Sample size re-estimation 

• Adaptive recruitment

*Yang et. al., 2016. “Adaptive Design Practice at the Center for Devices and Radiological Health 

(CDRH), January 2007 to May 2013” Therapeutic Innovation and Regulatory Science 1-8.
17

Blinded studies

• If blinding is scrupulously maintained:

– Adaptations are more easily accomplished.

– There are less questions about statistical or 

operational bias.

– Changes based on aggregate results can often be 

incorporated.

www.fda.gov
18

Adaptive Sample Size

• Why plan to adapt the sample size?

– Estimates of important parameters that are used to 

size a study are often (usually) different than what is 

observed in the study and results in either over- or 

under-powered studies.

• Sample size re-estimation can correct this 

problem to some degree, using estimates 

obtained at interim looks to revise the sample 

size for the study.

www.fda.gov
19

Frequentist Adaptive Sample Size

• Group sequential designs

• Sample size re-estimation (frequentist)

– Conditional power

– Conditional type I error

– P-value combination

– Etc.

• Internal pilots

– Maintain blind, only estimate nuisance parameters

www.fda.gov
20

Bayesian Adaptive Sample Size

• Bayesian approaches:

– Often employs frequent interim analyses

• Decision rules based on predictive probability 

– Includes adaptive recruitment 

• Recruitment ends when predictive probability of success 

is sufficiently large

– Includes group sequential designs

www.fda.gov
21

General Approach to 

Adaptive Sample Size

• Fix sample size for stage 1, calculate appropriate 

summary 

– test statistics/conditional power/predictive distribution/etc.

• Decision rule: stop (futility), stop (success), recruit more 

patients

– Potentially re-estimate sample size based on outcome 

– Continue to next stage

– Repeat as necessary (with pre-specified max # of looks)

• Compute overall test statistic using an appropriate 

method that controls type I error

www.fda.gov
22

Adaptive enrichment

• At interim analysis, evaluate results overall as 

well as within pre-specified subgroups

• Interim decision options:

– Stop study (futility, success)

– Continue with entire population

– Continue, but restrict to some subset of prespecified subgroup(s)

• Use appropriate design/analysis/decision rules 

(frequentist or Bayesian) that control type I 

error

www.fda.gov
23

Sample size vs. Enrichment

• Sample size adaptations can protect against 

poor estimates of important parameters (e.g. 

effect size) at design stage.

• Adaptive enrichment can protect against poor 

choices of appropriate patient population.

– False positives in preliminary studies

– Perhaps covariate influence on outcome is not clear

www.fda.gov
24

Simulations

• For complex designs, extensive simulations are 

needed to fully characterize operating 

characteristics of the design.

• Simulations can help inform operational plans – 

identify and plan for possibilities before they arise

• The details matter, should design the simulation 

like an experiment (need appropriate coverage of 

the parameter space)

• Basic programming mistakes do sometimes occur – 

importance of independent coding, etc.

www.fda.gov 24
25

Bias control

• Operational bias can occur if information 

regarding the progress of the trial leaks out.

• If participants (patients, investigators, analysts) 

change their behavior based on emerging 

results then the trial results will be biased.

Solution: maintain appropriate “firewalls” so that 

only those who need to know have access to 

randomization and outcome data.

www.fda.gov
26

Operational Challenges

• Adaptation/steering committee 

– Determines when to adapt trial 

• Data Monitoring Committees (DMCs)

– Primary role is protecting patients 

– Sometimes charged with determining adaptations

– Require DMC Statistician with Adaptive Design 

knowledge 

• Operational bias

– A concern in all trials, but especially adaptive trials

– Try to reduce ‘information leakage’ as far as possible 

www.fda.gov
27

Operational Challenges Contd.

• Institutional Review Boards (IRBs)

– Primarily concerned with safety

– Need advance planning to minimize delays in IRB 

approval of adaptations

• Logistics

– Need smooth data flow and checking for timely 

adaptation 

– Need common understanding of how trial will be 

conducted 

– Need supplies, etc. to accommodate changes to 

treatment

www.fda.gov
28

Interactions with CDRH/CBER

• Earlier is better

– Use pre-submission process as outlined in guidance.

• At design stage, operating characteristics will be of 

concern to FDA

– If using simulation, cover a wide range of possible 

scenarios.

• Monitoring plan will be important.

• Other techniques (blinding, firewalls, etc.) to 

reduce operational bias

www.fda.gov
29

CBER’s Devices 

• A few are therapeutic

• Products to process blood, cells and tissues

• Some trials are similar to trials for biologics

• Most adaptive and/or Bayesian submissions 

have been for biologic products

• Protocol assessments similar for Investigational 

Device Exemptions (IDEs) and Investigational 

New Drugs (INDs)
30

CBER’s Survey of Adaptive Designs

• Survey* covers 2008-2013; seeing more now. 

• Bayesian methods common in phase 1 and 2 (oncology) 

• Proposals for Bayesian adaptive studies in phase 3

• Proposals for adaptive designs with type 1 error via simulation

• May take more than one round of discussion to finalize 

• Clinical as well as statistical considerations are important

*Lin, M., S. Lee, B. Zhen, J. Scott, A. Horne, G. Solomon, E. Russek-Cohen (2016) CBER’s 

Experience with Adaptive Design Clinical Trials. Therapeutic Innovation and Regulatory 

Science 50:195-203. 
31

Diagnostics

• Prevalence of rare conditions will often drive 

sample size in diagnostic performance studies.

• Protocols for prospective studies can define a 

rule to add samples based on observed 

prevalence as defined by a reference method.

– No alpha penalty. 

• Banked specimens are sometimes allowed. 

– First contact review division. 
32

Thank you
Questions?

Division of Industry and Consumer Education: 

DICE@fda.hhs.gov

Slide Presentation, Transcript and Webinar 

Recording will be available at:

http://www.fda.gov/training/cdrhlearn

Under Heading: How to Market Your Device; 

Sub-heading: Clinical Studies/IDE