FDA Guidance: “Design 

Considerations for Pivotal 

Clinical Investigations for 

Medical Devices”

Greg Campbell, Ph.D.

Director, Division of Biostatistics

Center for Devices and Radiological Health

U.S. Food and Drug Administration

greg.campbell@fda.hhs.gov

FDA Stakeholder Webinar December 4, 2013
Design Considerations for 

Pivotal Clinical Investigations 

• http://www.fda.gov/medicaldevices/deviceregulation

andguidance/guidancedocuments/ucm373750.htm

• Draft issued Aug. 15, 2011; Final issued Nov. 7, 

2013. 

• It is guidance for industry, clinical investigators, 

institutional review boards and FDA staff.

2
Design Considerations for 

Pivotal Clinical Investigations 

• Guidance should help manufacturers select 

appropriate trial design.

• Better trial design and improve the quality of data 

that may better support the safety and effectiveness 

of a device. 

• Better quality data may lead to timelier FDA approval 

or clearance of premarket submissions and speed 

U.S. patient access to new devices.

3
Outline

• Regulatory Considerations

• Principles for the Choice of Clinical Study Design

• Clinical Outcome Studies

– RCTs

– Controls

– Non-Randomized Controls 

– Observational Uncontrolled One-Arm Studies

• OPCs and Performance Goals

– Diagnostic Clinical Outcome Studies

• Diagnostic Clinical Performance Studies 

• Sustaining the Level of Evidence of Studies

• Protocol and Statistical Analysis Plan

• Closing Remarks 4
Stages of Medical Device 

Clinical Studies

1) Exploratory Stage – first-in-human and 

feasibility/pilot studies, iterative learning and 

product development.

2) Pivotal Stage – definitive study to support the safety 

and effectiveness evaluation of the medical device 

for its intended use.

3) Postmarket Stage – includes studies intended to 

better understand the long-term effectiveness and 

safety of the device, including rare adverse events.

Not all products will go through all stages.

5
Importance of Exploratory 

Studies in Device Development

• Exploratory stage evaluation helps better 

understand how the device may perform, its 

intended use, and how to select an 

appropriate pivotal study design.

6
Kinds of Devices

• Therapeutic Devices – Intended to treat a 

specific condition or disease

• Aesthetic Devices – Provide a desired change 

in a subject’s appearance through physical 

modification of the structure of the body

• Diagnostic Devices – Provides information 

when used alone or in the context of other 

information to help assess a subject’s 

condition

7
Unique Features for 

Some Medical Devices

• Device complexity

• User skill level and training

• Learning curve

• Human factor considerations

8
Valid Scientific Evidence for PMAs

• Statutory directive:

• valid scientific evidence to determine whether there is reasonable 

assurance that the device is safe and effective. 

• Valid scientific evidence comes from:

• well controlled studies

• partially controlled studies

• objective trials without matched controls

• well documented case histories

• reports of significant human experience (21 CFR 860.7)

9
Valid Scientific Evidence for PMAs

The valid scientific evidence used to determine the 

effectiveness of a device shall consist principally of wellcontrolled investigations.

21 CFR 860.7(e)(2)

10
Principles for Designing a 

Clinical Study

• Types of Device Studies

• Bias and Variance

• Study Objectives

• Subject Selection

• Stratification for Subject Selection

• Site Selection

• Comparative Study Designs

11
Principles for the Choice of 

Clinical Study Design

• Types of Device Studies

• Bias and Variance

• Study Objectives

• Subject Selection

• Stratification for Subject Selection

• Site Selection

• Comparative Study Designs

12
Types of Device Studies

• Clinical outcome 

– All therapeutic and aesthetic devices and some diagnostic 

devices

• Diagnostic clinical performance

– Some diagnostic devices

13
Principles for the Choice of 

Clinical Study Design

• Types of Device Studies

• Bias and Variance

• Study Objectives

• Subject Selection

• Stratification for Subject Selection

• Site Selection

• Comparative Study Designs

14
Bias and Variance

• Bias -- Systematic (non-random) error in the estimate 

of a treatment effect

– One objective of a clinical study: eliminate, reduce or 

estimate the bias, so as to characterize in a accurate manner 

the safety and effectiveness of the device

• Variance can be reduced by a more efficient design 

or by a larger sample size.

15
Principles for Designing a 

Clinical Study

• Types of Device Studies

• Bias and Variance

• Study Objectives

• Subject Selection

• Stratification for Subject Selection

• Site Selection

• Comparative Study Designs

16
Principles for Designing a 

Clinical Study

• Types of Device Studies

• Bias and Variance

• Study Objectives

• Subject Selection

• Stratification for Subject Selection

• Site Selection

• Comparative Study Designs

17
Comparative Study Designs

• Parallel

• Paired

• Crossover

18
Clinical Outcome Study Designs

• Double-masked (blinded), randomized, controlled, 

multi-center clinical trials (RCTs) are the “gold 

standard” for clinical outcome studies (but not 

necessarily for diagnostic clinical performance 

studies). Such studies tend to minimize bias.

• But these studies may be impractical, unethical or 

infeasible in certain situations. For example, there 

are some devices where it is impossible to mask 

anyone to the treatment.

• Manufacturers should talk to FDA before finalizing a 

study design and provide a rationale for the proposed 

study design.

19
What if RCT is not Masked?

• Without masking, bias needs to be addressed.

• Usually bias occurs if the control is no treatment.

• This bias is usually very difficult to estimate and can 

be quite large and extend over a long period of time, 

even if the endpoints are “hard”.

• When the study is not doubly masked, the influence 

of the patient or the investigator can be subconscious 

or unconscious.

20
Controls

• Concurrent Control (Randomized Clinical Trial)

• Placebo 

• Active 

• No treatment 

• Non-randomized Concurrent Control (Observational Study)

• Historical Control (Observational Study)

• Patient as Own Control

21
Placebo Effect and Choice of Control

l Placebo control—a totally ineffective treatment 

but almost always masked.

l Placebo controls (sometimes called “shams”) for 

devices can sometimes be much more difficult to 

devise than placebo pills.

l Placebo effect is well-known for endpoints of pain 

or function and can be long-lasting.

l Enhanced placebo effects for devices (vs. drugs)

Kaptchuk TJ et al. J Clin Epidemiol. 2000 53:786-792.

22
The Placebo Effect

• Sources of the placebo effect

– Expectation of benefit

– Regression to the mean

– Showering of (medical) attention (Hawthorne effect)

• It exists for objective as well as subjective endpoints 

and can be very long lasting.

• The sponsor should consider asking the subject and 

physician during the study which arm they think that 

subject is in.

23
Control of No Treatment 

• Called Standard of Care or Best Medical Management.

• Study is unmasked; patients get no experimental 

treatment. There is often a bias if they know they 

did not receive a treatment, whereas in the 

experimental arm there is the expectation of 

benefit.

• Further, in the experimental arm there is the 

expectation of benefit, for the controls there is no 

such expectation.

• For unmasked trials where there is a measurable 

difference between placebo and no treatment, 

usually based on patients’ expectation of therapy, 

this is not a preferred control. 24
Non-Randomized Controls

• Two types:

– use of a concurrent, non-randomized control. 

– historically controlled studies

• There is a concern about how comparable 

the groups are without randomization. Even 

if comparable, is there the same expectation 

of benefit?

• Such studies are observational and 

comparative statistical inference is 

compromised.

• Example: some early hip studies 25
Non-randomized Controls

l From a scientific standpoint, the randomized 

controlled trial (RCT) offers the strongest form of 

evidence, the least amount of bias and is generally 

preferred.

l It may be possible to match the non-concurrent 

control group to the investigational arm in all 

observed measures but there is no assurance for any 

unobserved ones. In contrast, randomization 

balances for observed as well as unobserved 

measures.

l Historical controls can be especially problematic 

due to temporal bias. 26
Historical Controls

l A one-arm study is an observational study—not a clinical 

trial or an experiment. 

l Since there is no randomization, all comparative 

statistical inference is compromised.

l It may be possible to match the historical control group 

to the current study in all observed measures but there 

is no assurance for any unobserved ones.

l In contrast, randomization balances for observed as 

well as unobserved measures.

l Statistical techniques such as propensity scores can be 

used to investigate comparability and perform analysis.27
One-arm Studies

l Objective Performance Criteria (OPC)

l Usually a very well-described and publicly available control 

used to set the criterion.

l Examples in CDRH:

lIOLs, heart valves 

l Performance Goals

In some instances, the OPC or PG may be based 

on the upper (or lower) bound of the 

confidence interval of an effectiveness (or 

safety) endpoint. 28
Objective Performance Criteria

• An OPC is a numerical target value derived 

from historical data from clinical studies 

and/or registries which may be use for 

comparison with safety or effectiveness 

endpoints.

• Currently very few OPCs since device 

technology needs to be fairly mature.

• An OPC can be carefully constructed from all 

available patient-level data on a particular 

type of device. This is usually not done by 

FDA nor a particular company. 29
Performance Goals

• A Performance Goal (PG) is a numerical value (point 

estimate) that is considered sufficient by FDA for use 

as a comparison for a safety or effectiveness 

endpoint.

• Generally PGs are inferior level of evidence to OPCs.

• It is recommended that (like OPCs) that PGs not 

originate from a particular sponsor nor from the FDA. 

It is often helpful if it is recommended by a scientific 

or medical society.

• A fundamental regulatory question is: When a device 

passes an effectiveness (safety) PG (or an OPC), does 

that provide evidence that the device is effective 

(safe)? 30
Diagnostic Clinical 

Outcome Studies 

• Here the intervention is the outcome of the 

diagnostic test and the primary endpoints are 

clinical.

• It is usually always unmasked to the investigator but 

hopefully not the third party evaluator.

31
Diagnostic Clinical 

Performance Studies

• Object: 

– to characterize the performance of the diagnostic 

device based on the test results from the subjects

– to provide evidence to be used in the assessment 

of the benefit risk associated with the use of the 

diagnostic device for the intended population.

32
Diagnostic Clinical 

Performance Studies

• Consideration of Intended Use

• Study Population 

• Study Planning, Subject Selection and Specimen 

Collection

• Comparison Studies using the Clinical Reference 

Standard

• Masking

• Skill and Behavior of Persons Interacting with the 

Device (Total Test Concept)

• Other Sources of Bias 33
Sustaining Quality of 

Clinical Studies 

• Handling Clinical Data

• Study Conduct

• Study Analysis

• Anticipating Changes to the Pivotal Study 

34
The Protocol

• Scientific rationale (and hypotheses if approp.)

• Definition of subject populations

• Proposed intended use

• Listing of study endpoints

• Statistical Analysis Plan

35
Statistical Analysis Plan

• As much detail as possible at the protocol 

stage

• Should include the statistical analysis for the 

primary endpoint on which the sample size is 

based, anticipate if the assumptions are not 

valid and plan for missing data.

• Final detailed Statistical Analysis Plan needs 

to be submitted before any outcome data are 

available or unmasked

– For non-masked (open label) studies this means before any 

patient reaches an outcome. 36
Closing Remarks

• Device studies allow for a broad range of designs 

(much broader than the pharmaceutical world).

• Companies are encouraged to meet with the FDA in 

the planning phase for a pre-submission meeting or 

IDE meeting.

37
Questions?

38