Contains Nonbinding Recommendations 
 
 
Use of Real-World Evidence to 
Support Regulatory Decision-Making 
for Medical Devices 
Guidance for Industry and  
Food and Drug Administration Staff  
Document issued on August 31, 2017.  
 
The draft of this document was issued on July 27, 2016 
 
For questions about this document regarding CDRH-regulated devices, contact the Office of 
Surveillance and Biometrics (OSB) at 301-796-5997 or CDRHClinicalEvidence@fda.hhs.gov.  
For questions about this document regarding CBER-regulated devices, contact the Office of 
Communication, Outreach, and Development (OCOD) at 1-800-835-4709 or 240-402-8010. 
 
 
 
 
 
 
 
 
 
 
 
     U.S. Department of Health and Human Services 
Food and Drug Administration 
 
Center for Devices and Radiological Health 
Center for Biologics Evaluation and Research 
 
1 
  Contains Nonbinding Recommendations 
 
 
Preface 
 
Public Comment 
You may submit electronic comments and suggestions at any time for Agency consideration to 
http://www.regulations.gov . Submit written comments to the Dockets Management Staff, Food 
and Drug Administration, 5630 Fishers Lane, Room 1061, (HFA-305), Rockville, MD 20852. 
Identify all comments with the docket number FDA-2016-D-2153. Comments may not be acted 
upon by the Agency until the document is next revised or updated. 
Additional Copies 
CDRH 
Additional copies are available from the Internet.  You may also send an e-mail request to CDRH-
Guidance@fda.hhs.gov to receive a copy of the guidance. Please use the document number 
1500012 to identify the guidance you are requesting. 
CBER 
Additional copies are available from the Center for Biologics Evaluation and Research (CBER), 
Office of Communication, Outreach, and Development (OCOD), 10903 New Hampshire Ave., 
Bldg. 71, Room 3128, Silver Spring, MD 20993-0002, or by calling 1-800-835-4709 or 240-402-
8010, by email, ocod@fda.hhs.gov or from the Internet at 
http://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guida
nces/default.htm. 
2 
  Contains Nonbinding Recommendations 
 
 
Table of Contents
 
I.  Introduction and Scope .........................................................................................................4
II.  Background..........................................................................................................................5
III.  Real-World Evidence...........................................................................................................8
IV.  Regulatory Context in Which RWE May be Used ...............................................................9
A.  General considerations for the use of RWE .........................................................................9
B.  Application of Investigational Device Exemption (IDE) Requirements in 21 CFR 812 to 
the Collection of RWD .............................................................................................................. 11
V.  Characteristics of RWD ...................................................................................................... 12
A.  Relevance ........................................................................................................................... 13
B.  Reliability ........................................................................................................................... 15
(1)  Data accrual ................................................................................................................ 15
(2)  Data assurance - Quality Control ................................................................................ 16
VI.  Examples Where RWE is Used .......................................................................................... 17
A.  Expanded Indications for Use ............................................................................................ 17
B.  Postmarket Surveillance Studies (Section 522) ................................................................. 18
C.  Post-Approval Device Surveillance as Condition of Approval ......................................... 19
D.  Control Group .................................................................................................................... 19
E.  Supplementary Data ........................................................................................................... 20
F.  Objective Performance Criteria and Performance Goals ................................................... 20
VII.  Glossary .............................................................................................................................. 21
3 
  Contains Nonbinding Recommendations 
Use of Real-World Evidence to 
Support Regulatory Decision-Making 
for Medical Devices  
Guidance for Industry and  
Food and Drug Administration Staff 
This guidance represents the Food and Drug Administrations (FDAs) current thinking on 
this topic. It does not create or confer any rights for or on any person and does not operate to 
bind FDA or the public. You can use an alternative approach if the approach satisfies the 
requirements of the applicable statutes and regulations. If you want to discuss an alternative 
approach, contact the FDA staff responsible for implementing this guidance. If you cannot 
identify the appropriate FDA staff, call the appropriate number listed on the title page of this 
guidance. 
I.  Introduction and Scope 
FDA is issuing this guidance to clarify how we evaluate real-world data to determine whether  
they are sufficient for generating the types of real-world evidence that can be used in FDA 
regulatory decision-making for medical devices.  This guidance is applicable to all devices, as 
that term is defined under section 201(h) of the Federal Food, Drug, and Cosmetic Act (the 
FD&C Act), including software that meets the definition of a device.   
Real-World Data (RWD) are data relating to patient health status and/or the delivery of health 
care routinely collected from a variety of sources.   
Examples of RWD include data derived from electronic health records (EHRs), claims and 
billing data, data from product and disease registries, patient-generated data including in home-
use settings, and data gathered from other sources that can inform on health status, such as 
mobile devices.  RWD sources (e.g., registries, collections of EHRs, and administrative and 
healthcare claims databases) can be used as data collection and analysis infrastructure  to support 
many types of trial designs, including, but not limited to, randomized trials, such as large simple 
trials, pragmatic clinical trials, and observational studies (prospective and/or retrospective ). 
Real-World Evidence (RWE) is the clinical evidence regarding the usage, and potential 
benefits or risks, of a medical product derived from analysis of RWD.   
4  Contains Nonbinding Recommendations 
 
 
Under the right conditions, data derived from real world sources can be used to support 
regulatory decisions.  RWD and associated RWE may constitute valid scientific evidence 
depending on the characteristics of the data.  This guidance should not be construed to alter, or 
change in any way, the existing evidentiary standards applicable to FDAs regulatory decision-
making; rather, it describes the circumstances under which RWD may be used to support a 
variety of FDA decisions based on the existing evidentiary standards. While FDA encourages the 
use of relevant and reliable RWD, this guidance neither mandates its use nor restricts other 
means of providing evidence to support regulatory decision-making. This guidance highlights 
some of the potential uses of RWD, and describes the factors that FDA considers when 
evaluating whether specific RWD is of sufficient quality to inform or support a regulatory 
decision.  It also clarifies when an Investigational Device Exemption (IDE) may be needed to 
prospectively collect and use RWD for purposes of determining the safety and effectiveness of a 
device.   
This document does not address the use of non-clinical data, adverse event reports, secondary 
use of clinical trial data (e.g., post hoc analyses), or systematic literature reviews.  Nor does it 
address study design/conduct or analytical methodologies.  While it does describe the factors that 
FDA considers when evaluating RWD or RWE, it does not provide a specific set of pass/fail 
criteria or other scoring tools for making a determination about the suitability of RWD or RWE 
for a particular regulatory decision. 
This guidance does not affect any federal, state or local laws or regulations, or foreign laws or 
regulations that may be applicable to the use or collection of RWD, or that provide protections 
for human subjects (including informed consent requirements) or patient privacy.  This guidance 
should be used to complement, but not supersede, other device-specific and good clinical 
practice guidance documents. 
FDA's guidance documents, including this guidance, do not establish legally enforceable 
responsibilities.  Instead, guidance documents describe the Agencys current thinking on a topic 
and should be viewed only as recommendations, unless specific regulatory or statutory 
requirements are cited.  The use of the word should in Agency guidance means that something is 
suggested or recommended, but not required.  
II.  Background 
To protect and promote the public health, FDA needs to understand and evaluate the available 
evidence related to regulated products.1  For medical devices, available evidence is traditionally 
comprised of non-clinical and, in some cases, clinical studies conducted and provided to FDA by 
the device manufacturer or sponsor.  However, FDA recognizes that a wealth of RWD covering 
medical device experience exists and is routinely collected in the course of treatment and 
management of patients.  Data collected during clinical care or in the home setting may not have 
the same quality controls as data collected within a clinical trial setting.  Even so, under certain 
 
1 FDAs What We Do  (http://www.fda.gov/AboutFDA/WhatWeDo/default.htm).  
5 
  Contains Nonbinding Recommendations 
 
 
circumstances RWD may be of sufficient quality to help inform or augment FDAs 
understanding of the benefit-risk profile of devices at various points in their life cycle.  RWD, 
which are typically collected for non-regulatory purposes in EHRs, registries, and administrative 
and claims data, may provide new insights into the performance and clinical outcomes associated 
with medical device use.  This information can potentially be used by sponsors to demonstrate 
compliance with regulatory requirements and to aid FDA in our regulatory decision-making.  
FDA has issued guidance documents on premarket and postmarket data collection,2 benefit-risk 
determinations,3 patient preference information,4 and expedited access to medical devices for 
unmet medical needs in order to streamline the process for bringing new technologies to market 
while maintaining the assessment of reasonable assurance of safety and effectiveness of medical 
devices.  FDA has also issued plans for, and has begun implementation of, the National 
Evaluation System for health Technology (NEST)5,6,7,8 to leverage RWD in order to more 
quickly identify safety problems  and to better understand the benefit-risk profile of devices used 
in clinical care.  FDA believes that, if leveraged correctly, the NEST may also help to reduce the 
time and cost of generating the types of evidence used to support the marketing authorization of 
FDA-regulated products and to meet postmarket study and reporting requirements. 
Devices are often used in routine clinical practice for uses that are not within their cleared or 
approved indications for use.  However, the knowledge gained from all uses of a device in 
medical practice is often not realized because the data collected are not systematically 
characterized, aggregated, and analyzed in a way that can be relied upon to inform regulatory 
decision-making.  By recognizing the value of RWE as an important contributing factor for 
understanding and regulating medical devices, we hope to encourage the medical community to 
learn more from routine clinical care than we do today. 
 
2 Balancing Premarket and Postmarket Data Collection for Devices Subject to Premarket Approval 
(http://www.fda.gov/ucm/groups/fdagov-public/@fdagov-meddev-gen/documents/document/ucm393994.pdf).  
3 Factors to Consider When Making Benefit-Risk Determinations in Medical Device Premarket Approval and De 
Novo Classifications (http://www.fda.gov/ucm/groups/fdagov-public/@fdagov-meddev-
gen/documents/document/ucm517504.pdf) 
4  Patient Preference Information  Voluntary Submission, Review in Premarket Approval Applications, 
Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision Summaries and 
Device Labeling  (Patient Preference Information  Voluntary Submission, Review in Premarket Approval 
Applications, Humanitarian Device Exemption Applications, and De Novo Requests, and Inclusion in Decision 
Summaries and Device Labeling 
http://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm446680.pdf).  
5 Strengthening Our National System for Medical Device Postmarket Surveillance 
(http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHRe
ports/UCM301924.pdf). 
6 Strengthening Our National System for Medical Device Postmarket Surveillance: Update and Next Steps - April 
2013 (http://www.fda.gov/downloads/MedicalDevices/Safety/CDRHPostmarketSurveillance/UCM348845.pdf).  
7 Strengthening Patient Care: Building a National Postmarket Medical Device Surveillance System 
(http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHRe
ports/UCM435112.pdf).  
8 Recommendations for a National Medical Device Evaluation System: Strategically Coordinated Registry Networks 
to Bridge the Clinical Care and Research - August 2015 
(http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHRe
ports/UCM459368.pdf).  
6 
  Contains Nonbinding Recommendations 
 
 
FDA will use the criteria described in this guidance to evaluate whether RWD are of sufficient 
quality to support regulatory decision-making, including potentially generating valid scientific 
evidence.9  FDA relies only upon valid scientific evidence to determine whether there is a 
reasonable assurance that a device is safe and effective.  However, it is possible that RWE can 
meet this threshold under conditions where the underlying RWD were accurately and reliably 
captured at clinically relevant time intervals throughout the device lifecycle.  Under the right 
conditions, RWE may be suitable to support the clearance or approval of a new device, or the 
expansion of the indications for use of devices that are already on the market.  RWE may also be 
used to supplement the total evidence required for such clearances or approvals.  Other 
applications of RWE in premarket decision-making may also be possible, particularly as RWD 
systems and analysis methodology advance.   
Additionally, aggregation of RWD (e.g., in medical device registries) may prove useful as a 
postmarket control suitable for providing ongoing device safety surveillance and additional 
evidence for effectiveness.  FDA has long applied postmarket controls as a way to reduce 
premarket data collection, while still ensuring that the statutory standard of reasonable assurance 
of safety and effectiveness is met.10  FDA believes that applying postmarket controls to reduce 
premarket data collection, when appropriate, can help improve patient access to safe and 
effective medical devices.11  
In some cases, a traditional clinical trial may be impractical or excessively challenging to 
conduct.  Ethical issues regarding treatment assignment, and other similar challenges, may 
present themselves when developing and attempting to execute a high quality clinical trial.  
Analyses of RWD, using appropriate methods, may in some cases provide similar information 
with comparable or even superior characteristics to information collected and analyzed through a 
traditional clinical trial.  For example, RWD collected using a randomized exposure assignment 
within a registry can provide a sufficient number of patients for powered subgroup analyses, 
which could be used to expand the devices indications for use.  However, not all RWD are 
collected and maintained in a way that provides sufficient reliability.  As such, the use of RWE 
for specific regulatory purposes will be evaluated based on criteria that assess their overall 
 
9 Valid scientific evidence is evidence from well-controlled investigations, partially controlled studies, studies and 
objective trials without matched controls, well-documented case histories conducted by qualified experts, and 
reports of significant human experience with a marketed device, from which it can fairly and responsibly be 
concluded by qualified experts that there is reasonable assurance of the safety and effectiveness of a device under its 
conditions of use. The evidence required may vary according to the characteristics of the device, its conditions of 
use, the existence and adequacy of warnings and other restrictions, and the extent of experience with its use. [21 
CFR 860.7(c)(2)] 
(http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/CFRSearch.cfm?FR=860.7) 
10 The Least Burdensome Provisions of the FDA modernization Act of 1997: Concept and Principles, 
(http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm085999.p
df)  
11  Balancing Premarket and Postmarket Data Collection for Devices Subject to Premarket Approval, 
(http://www.fda.gov/ucm/groups/fdagov-public/@fdagov-meddev-gen/documents/document/ucm393994.pdf).  
7 
  Contains Nonbinding Recommendations 
 
 
relevance and reliability.  If a sponsor is considering using RWE to satisfy a particular FDA 
regulatory requirement, the sponsor should contact FDA through the pre-submission process.12 
III.  Real-World Evidence 
RWE can exist across a wide spectrum, ranging from observational studies within an existing 
dataset to studies that incorporate planned interventions with or without randomization at the 
point of care.13 Because of the rapidly advancing methodology for generating and interpreting 
RWD, this guidance will not elaborate on the methodological approaches that can be used.  
However, when reviewing the use of RWE to support a regulatory decision, FDA will rely on 
scientifically robust methods and approaches to determine whether submitted RWE is of 
sufficient quality to support a particular regulatory decision. 
Clinical trials are designed to control variability through detailed eligibility criteria and carefully 
designed clinical protocols performed by specialized research personnel.  They require intensive 
monitoring and data auditing to demonstrate that use of a device produces the expected results. 
Although useful in establishing a baseline for device performance, clinical trials may be narrow 
in scope due to practical challenges.  In contrast, studies leveraging RWD can potentially provide 
information on a wider patient population, thus providing information that cannot be obtained 
through a traditional clinical trial alone.  An existing RWD source, however, may have some 
inherent bias that could limit its value for drawing causal inferences between medical device 
exposures and outcomes.  Therefore, to mitigate potential bias, careful study design is needed, 
and a study protocol and analysis plan should be created prior to accessing, retrieving, and 
analyzing RWD, regardless of whether the RWD are already collected (retrospective) or if they 
are to be collected in the future (prospective design).  Protocols and analysis plans for RWD 
should address the same elements that a traditional clinical trial protocol and statistical analysis 
plan would cover.  FDA recommends use of the pre-submission process when considering the 
development of a study using RWD in a regulatory submission. 
When considering a prospective study design, one should consider whether RWD collection 
instruments and analysis infrastructure are sufficient to serve as the mechanism for conducting 
the study, and if they are not, whether it is possible to modify them for such a purpose.  
Ultimately, if the sources of bias can be mitigated, RWD collected using a prospective study 
design may be used to generate or contribute to the totality of evidence needed to assess medical 
device performance.   
Because of its nature, the quality of RWE can vary greatly across different data types and 
sources.  Likewise, there are many types of FDA regulatory decisions spanning the Total Product 
Life Cycle (TPLC) that necessitate different levels of evidence.  This guidance does not change 
 
12 Requests for Feedback on Medical Device Submissions: The Pre-Submission Program and Meetings with Food 
and Drug Administration Staff 
(http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM311176.
pdf) 
13 Sherman RE, Steven A, Dal Pan GJ, et.al.; Real-World Evidence  What Is It and What Can it Tell Us?; NEJM, 
Dec 2016; 375; 23 
8 
  Contains Nonbinding Recommendations 
 
 
FDAs evidentiary standards for regulatory decision-making, and in each context we will 
evaluate whether the available RWE is of sufficient quality to address the specific regulatory 
decision being considered.  FDA believes that the increased use of electronic data systems in the 
healthcare setting has the potential to generate substantial amounts of RWD.  Because these 
systems can vary greatly in terms of quality, not all RWD can by itself generate sufficient 
evidence to support an FDA regulatory decision.  Nevertheless, these RWD may still provide a 
valuable contribution to the totality of evidence considered for the decision. Furthermore, in 
order to use RWD, the device must be sufficiently identified with the level of detail necessary to 
address the regulatory question. For example, to evaluate a particular device, the Unique Device 
Identifier (UDI) or serial/model number may be necessary to identify the device within a RWD 
source that contains data on many similar devices.   
When RWE is intended to be used for purposes of evaluating a regulatory issue, it is important 
that the RWD not only follow the criteria described in section V, but are also presented to FDA 
according to recognized data standards (i.e. in standardized file formats and data structures, 
utilizing standardized variables and definitions, etc.) when applicable.  This includes discussions 
of the methodology used to analyze RWD and assess clinically relevant differences as well as 
statistical significance. 
IV.  Regulatory Context in Which RWE May be Used 
A. General considerations for the use of RWE   
FDA will consider the use of RWE to support regulatory decision-making for medical devices 
when it concludes that the RWD used to generate the RWE are of sufficient quality to inform or 
support a particular regulatory decision. The threshold for sufficient quality will depend on the 
specific regulatory use of the evidence.  For example, a specific registry might be leveraged for 
postmarket surveillance, but not be adequate to support a premarket determination of reasonable 
assurance of safety and effectiveness or substantial equivalence. 
The collection or aggregation of RWD sources outside of medical records is usually performed 
for specific pre-determined non-regulatory purposes. For example, medical administrative claims 
data are typically kept for purposes of billing/payment for medical care.  Disease-specific RWD 
sources sponsored by patient advocacy organizations may be useful for tracking progression or 
outcomes of specific rare or poorly understood diseases.  Treatment-specific RWD sources 
coordinated by one or more professional societies may have several purposes, including 
assessment and tracking of overall outcomes, providing data for quality assessment (QA), 
informing performance improvement (PI) initiatives, or providing risk prediction and 
benchmarking data for specific therapies. Therefore, the use of RWD for specific regulatory 
decisions necessitates an understanding of the strengths and limitations of the RWD, and how 
these qualities impact the relevance and reliability factors described below.  
RWD may potentially be used as some or all of the evidence necessary for understanding 
medical device performance at different points in the TPLC.  Some purposes for which RWD 
may potentially be used include the following: 
9 
  Contains Nonbinding Recommendations 
 
 
  for generating hypotheses to be tested in a prospective clinical study; 
  as a historical control, a prior in a Bayesian trial14, or as one source of data in a 
hierarchical model or a hybrid data synthesis; 
  as a concurrent control group or as a mechanism for collecting data related to a clinical 
study to support device approval or clearance in a setting where a registry or some other 
systematic data collection mechanism exists;   
  as evidence to identify, demonstrate, or support the clinical validity of a biomarker; 
  as evidence to support approval or granting of an Humanitarian Device Exemption,  
Premarket Approval Application (PMA), or  De Novo request; 
  as support for a petition for reclassification of a medical device under section 513(e) or 
(f)(3) of the FD&C Act; 
  as evidence for expanding the labeling of a device to include additional indications for 
use or to update the labeling to include new information on safety and effectiveness15,16  
  for public health surveillance efforts.  Through ongoing surveillance, signals are at times 
identified that suggest there may be a safety issue with a medical device.  RWE may be 
used to refine these signals for purposes of informing appropriate corrective actions and 
communication;17,18 
  to conduct post-approval studies that are imposed as a condition of device approval or to 
potentially preclude the need for postmarket surveillance studies ordered under section 
522 of the FD&C Act;  
 
14 Guidance for Industry and FDA Staff: Guidance for the Use of Bayesian Statistics in Medical Device Clinical 
Trials, 
(https://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm071121.
pdf) 
15 What We Mean When We Talk About EvGen Part II: Building Out a National System for Evidence Generation, 
(http://blogs.fda.gov/fdavoice/index.php/2016/04/what-we-mean-when-we-talk-about-evgen-part-i-laying-the-
foundation-for-a-national-system-for-evidence-generation/) 
16  Guidance for Industry: General/Specific Intended Use 
(http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm073945.p
df).  
17 Public Notification of Emerging Postmarket Medical Device Signals (Emerging Signals) Guidance for Industry 
and Food and Drug Administration Staff 
(http://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm479248.pdf).  
18 Strengthening Patient Care: Building an Effective National Medical Device Surveillance System  
(http://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHRe
ports/UCM435112.pdf).  
10 
  Contains Nonbinding Recommendations 
 
 
  in certain circumstances, for use in generating summary reports of Medical Device 
Reports (MDRs); and 
  to provide postmarket data in lieu of some premarket data.  
B. Application of Investigational Device Exemption (IDE) 
Requirements in 21 CFR 812 to the Collection of RWD 
An approved IDE permits a device to be shipped lawfully for the purpose of conducting 
investigations of the device without complying with certain other requirements of the FD&C Act 
that would apply to devices in commercial distribution.  The purpose of this, per 21 CFR 812.1, 
is to encourage, to the extent consistent with the protection of public health and safety and with 
ethical standards, the discovery and development of useful devices intended for human use, and 
to that end to maintain optimum freedom for scientific investigators in their pursuit of this 
purpose.  As explained in Part 812, the IDE regulations apply to all clinical investigations of 
devices to determine safety and effectiveness, with certain limited exceptions.  In many cases, an 
approved IDE is required before initiating a clinical investigation.  An investigation is defined as 
a clinical investigation or research involving one or more subjects to determine the safety or 
effectiveness of a device.19    
Whether the collection of RWD for a legally-marketed device requires an IDE depends on the 
particular facts of the situation.  Specifically, if the device is being used in the normal course of 
medical practice, an IDE would likely not be required.  Because FDA does not regulate the 
practice of medicine, this could include administration of a legally marketed device for uncleared 
or unapproved uses as long as the device is being administered under the authority of a 
healthcare practitioner within a legitimate practitioner-patient relationship. 20   However, if data 
are being gathered to determine the safety and effectiveness of the device, and the process for 
gathering the data would influence treatment decisions, such administration would likely not be 
within the normal course of medical practice, and an IDE may be required.  For example, a 
registry designed to determine the safety and effectiveness of an approved device for a new 
intended use would likely be subject to IDE requirements if physicians are instructed to treat 
specific patients or otherwise administer the device in a prescribed way for purposes of data 
generation, or when certain follow-up activities are performed for the purpose of research.  
Because the gathering of RWD is different from traditional investigations, we recommend that 
you contact FDA if you have questions about whether an IDE is required. 
As stated above, FDA does not regulate the practice of medicine, and recognizes that RWD may 
be collected during the routine care of patients that provides information related to the actual use 
of an approved or cleared device.  Such observations may include RWD from a use of a medical 
device that was not within the cleared or approved indications for use.   If such RWD collection 
 
19 See 21 CFR 812.3(h) 
20 FDA will not limit or interfere with the authority of a health care practitioner to prescribe or administer any 
legally marketed device to a patient for any condition or disease within a legitimate health care practitioner-patient 
relationship.  Section 1006 of the FD&C Act, 21 USC 396. 
11 
  Contains Nonbinding Recommendations 
 
 
does not impact how the device is administered, and the administration is within the normal 
course of medical care, an IDE would likely not be required.  For example, retrospective 
analyses of existing RWD involving the use of a device that was not within the cleared or 
approved indications for use would generally not be subject to IDE regulations.  In such cases, 
treatment decisions were made in the best interest of patients according to their clinicians 
medical judgment at that time.  However, if the plan to conduct such analyses impacts patient 
care, then the study may be subject to IDE requirements.  
Should a sponsor or Institutional Review Board (IRB) be unclear regarding the applicability of 
the IDE regulations to a particular RWD collection activity or use, the sponsor or IRB should 
contact FDA.  If an IDE is determined to be required, FDA will work with the IDE sponsor to 
develop the least burdensome approach to facilitate the efficient generation of high-quality 
RWE.  Note that regardless of the applicability of 21 CFR 812, FDA regulations at 21 CFR 56 
(IRB review), 21 CFR 50 (Informed Consent) and 21 CFR 54 (Financial Disclosure) may apply 
to RWE generation activities, as may other federal, state, and local laws regarding human subject 
protections.   
V.  Characteristics of RWD 
FDA does not endorse one type of RWD over another.  Sponsors should select appropriate RWD 
sources based on their suitability to address specific regulatory questions.  Whether the RWD 
resides within paper or electronic medical records, is collected by administrative databases, is 
abstracted, aggregated and stored in disease- or treatment-specific databases (i.e., registries), or 
collected and aggregated through other means, accuracy when compared to verifiable source 
documentation is essential.  Verifiable source documentation for RWD elements includes, but is 
not limited to: paper or electronic inpatient and outpatient medical records and case histories, 
diagnostic laboratory and imaging data, patient preference information, patient-reported outcome 
measures, UDI and other device identifiers, and performance data that exist within the device 
such as self-diagnostics, error codes, and patient diagnoses/treatments delivered. Requirements 
and needs for individual source data verification will vary with specific regulatory questions, and 
will contribute to the overall understanding of data quality for that source.  
In order to determine the suitability of RWD for regulatory decision-making, FDA will assess 
the relevance and reliability of the source and its specific elements.  This assessment will be used 
to determine whether the RWD source(s) and the proposed analysis can generate evidence that is 
sufficiently robust to be used for a given regulatory purpose.  Whether evidence is sufficiently 
relevant and reliable for use will, in part, depend on the level of quality necessary to make a 
particular regulatory decision.  FDA will evaluate the same factors to assess RWD across all data 
sources and regulatory decisions.  In cases where RWE is derived from multiple RWD sources, 
each RWD source will be evaluated individually and together in the aggregate to determine the 
relevance and reliability of the RWD. When developing a new RWD source, consultation with 
FDA and other stakeholders is recommended to ensure that relevance and reliability are 
addressed in the initial design. 
12 
  Contains Nonbinding Recommendations 
 
 
A.  Relevance   
The relevance of RWD, RWD sources, and resultant analysis is assessed by evaluating several 
factors as outlined below.  These factors can help determine if the data adequately address the 
applicable regulatory question or requirement, in part or in whole.  Questions about the 
applicability of RWD to a specific case should be discussed with FDA through the pre-
submission process.21  Relevance of RWD for regulatory decision-making can be assessed prior 
to a regulatory submission, such as via the pre-submission process, or during the regulatory 
review process. The overall assessment of relevance should determine whether the existing 
RWD source is adequate for evaluating the performance of a device in the identified regulatory 
context (as a sole source or partial source of evidence).   
Since RWD sources are usually developed for non-regulatory purposes (e.g., to document care in 
the case of EHRs or to submit insurance claims for reimbursement in administrative and claims 
data), FDA will assess whether the individual data elements contained within an existing RWD 
source are sufficient to be used for a regulatory purpose. The data should be accurate, as 
complete as possible, and have an appropriate scope to address the question at hand (i.e., data 
adequacy).  The need for review or adjudication of specific outcomes of interest (e.g. stroke or 
major bleeding) at the patient level may also be assessed.  For analysis and interpretation of 
RWD, it is important to have a pre-defined common set of data elements, a common definitional 
framework (i.e., data dictionary), and pre-specified time intervals for data element collection and 
outcome analyses.  In assessing the relevance of RWD, FDA will also consider, if warranted, the 
ability to supplement the available RWD through linkage with other data sources to provide 
additional or confirmatory data, e.g., with EHR and/or administrative claims data.  
Important relevance factors that FDA will assess to determine if the RWD are suitable for 
regulatory use include, but are not limited to, whether:  
  the RWD contain sufficient detail to capture the use of the device, exposures, and the 
outcomes of interest in the appropriate population (i.e. the data apply to the question at 
hand); 
  the data elements available for analysis are capable of addressing the specified question 
when valid and appropriate analytical methods are applied (i.e. the data are amenable to 
sound clinical and statistical analysis); and 
  the RWD and RWE they provide are interpretable using informed clinical/scientific 
judgment.  Important considerations for the assessment of this factor include: 
 
21 Requests for Feedback on Medical Device Submissions: The Pre-Submission Program and Meetings with Food 
and Drug Administration Staff: Guidance for Industry and Food and Drug Administration Staff 
(http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM311176.
pdf) 
13 
  Contains Nonbinding Recommendations 
 
 
o  whether the use of the device in a real-world population is representative as 
captured within the data source, and is generalizable to the relevant population 
being evaluated;  
o  whether the RWD source is used regionally, nationally and/or internationally;  
o  the overall percentage of patient exposure to the  device that are captured in the 
RWD source;  
o  the validation protocols and resultant data that are used to evaluate how well the 
RWD source reflects the patient populations experience;   
o  whether the RWD study design, study protocol, and/or analysis plan is 
appropriate to address the regulatory question and capable of being accomplished 
in a sufficiently timely manner; 
o  whether the RWD contains elements to capture specific device identification 
information (e.g., unique device identifier);   
o  whether the RWD adequately captures patient medical history and preexisting 
conditions, as well as  follow-up information needed evaluate the question being 
addressed (e.g., whether administrative claims data have adequate continuity of 
coverage); 
o  whether sufficient data elements are collected to adjust for confounding factors 
that may impact the exposure or outcomes of interest; 
o  whether any linkages performed are scientifically appropriate and account for 
differences in coding and reporting across sources; 
o  the RWD source reporting schedule, including time interval between database 
close and release, and length of reporting periods;  
o  the prior documented (e.g., peer reviewed publications or practice guidelines) use 
of the RWD source for determining outcomes-based quality assessments, 
validated predictive risk modeling, signal detection, performance improvement, 
benchmarking, and other clinically-meaningful uses; 
o  whether the data elements collected are sufficient for assessing outcomes 
(including adjudication, if necessary); and 
o  whether supplemental data sources are available and sufficient to provide any 
missing information or evidence required for an informed decision. 
14 
  Contains Nonbinding Recommendations 
 
 
B. Reliability  
The reliability of RWD, RWD sources, and resultant analyses is assessed by evaluating several 
factors as outlined below.  The primary factors FDA considers for assessing the reliability of 
RWD include how the data were collected (data accrual), and whether the people and processes 
in place during data collection and analysis provide adequate assurance that errors are minimized 
and that data quality and integrity are sufficient (data assurance).  Additionally, the RWD 
analysis protocol should be prospectively defined as described in section III above. FDA will 
consider existing data accrual, data assurance, and analysis approaches in its assessment of the 
fitness of a given RWD source and its data. 
(1)  Data accrual 
To ensure the reliability of RWD, the RWD source should have an operational manual or other 
documentation that pre-specifies the data elements to be collected, data element definitions (i.e., 
data dictionary to provide a common definitional framework), methods for data aggregation and 
documentation (e.g., common case report form, abstraction from verifiable sources), and the 
relevant time windows for data element collection (i.e., common temporal framework).  Some 
RWD sources such as EHRs or claims data may not fulfill all of these characteristics, but may 
still demonstrate sufficient reliability to support regulatory decision-making.  Factors FDA will 
consider in making this determination include (but are not limited to):  
o  the preparedness of individual sites for complete and accurate collection of RWD 
(e.g., whether there are defined processes, site training and support, and qualified 
personnel);  
o  whether a common data capture form was used;  
o  whether a common definitional framework (i.e., data dictionary) was used;   
o  adherence to a common temporal framework for collection of key data points; 
o  the timing of establishing the study plan, protocol, and/or analysis plan relative to 
collection or retrieval of the RWD; 
o  the sources and technical methods used for data element capture (e.g., chart 
abstraction, point of care entry, EHR integration, UDI capture, data records from 
the device, and linkage to claims data);  
o  whether patient selection and enrollment criteria minimize bias and ensure a 
representative real-world population (e.g., all-comers design, consecutive patient 
enrollment);  
o  the timeliness of data entry, transmission, and availability; and 
15 
  Contains Nonbinding Recommendations 
 
 
o  whether necessary and adequate patient protections were in place (e.g., methods 
to protect patient privacy, and need for informed consent as determined by the 
reviewing IRB and in compliance with FDA regulations).   
(2)  Data assurance - Quality Control  
Data quality control is essential for providing confidence in the reliability of RWD and RWE 
sources. RWD quality can generally be improved by following published recommendations 
concerning registries, such as those by the Agency for Health Care Quality, Patient-Centered 
Outcomes Research Institute,22 the National Medical Device Registry Task Force,23 and the 
Regulators Forum (IMDRF) Registry Working Group. 24  However, certain sources of RWD, 
such as some administrative and healthcare claims databases or EHRs, may not have established 
data quality control processes and may not be capable of fully implementing or following the 
above recommendations.  When considering a source of RWD for regulatory purposes, it is 
important to consider any methods and systems used to help ensure sufficient data quality.  
Potential RWD sources should be evaluated in accordance with the data QA plan and procedures 
developed for the data source itself.  Since evaluation of RWD sources may not always permit 
specific line item source verification, important factors for consideration include: 
o  the quality of data element population (e.g., whether abstracted from a verifiable 
source to assess transcription errors or automatically populated through a data 
extraction algorithm);  
o  adherence to source verification procedures and data collection and recording 
procedures for completeness and consistency; 
o  completeness (i.e., minimized missing or out of range values) of data necessary 
for specified analyses, including adjustment for confounding factors;  
o  data consistency across sites and over time; 25 
 
22 AHRQ Registries for Evaluating Patient Outcomes: A Users Guide 3rd Edition, Volume 1 Chapter 11, Data 
Collection and Quality Assurance and Volume 2 Chapter 25, Assessing Quality. 
 (http://www.effectivehealthcare.ahrq.gov/search-for-guides-reviews-and-
reports/?pageaction=displayproduct&mp=1&productID=1897) 
23 A Report from the Medical Device Registry Task Force & the Medical Devices Epidemiology Network. 
Recommendations for a National Medical Device Evaluation System. Strategically Coordinated Registry Networks 
to Bridge Clinical Care and Research, August 2015. Available at: 
http://www.fda.gov/downloads/aboutfda/centersoffices/officeofmedicalproductsandtobacco/cdrh/cdrhreports/ucm45
9368.pdf. Accessed August 6, 2016. 
24 IMDRF Registry Essential Principles. Available at: 
http://www.imdrf.org/docs/imdrf/final/consultations/imdrf-cons-essential-principles-151124.pdf. Accessed August 
3, 2016 
25 PCORI Conduct of Registry Studies 
http://www.pcori.org/sites/default/files/Standards-in-the-Conduct-of-Registry-Studies-for-Patient-Centered-
Outcomes-Research1.pdf 
16 
  Contains Nonbinding Recommendations 
 
 
o  evaluation of on-going training programs for data collection and use of data 
dictionaries at participating sites;  
o  evaluation of site and data monitoring practices; and  
o  the use of data quality audit programs.   
The use of routine medical care data for additional analyses often relies on data cleaning and 
cross-referencing.  These techniques can confirm the datas internal consistency and identify 
missing values, but cannot fully determine data accuracy and authenticity.  In traditional clinical 
research, an audit that compares source documents to entered data is often an essential step to 
verify the accuracy and completeness of the data.  Study monitoring through various methods, 
such as remote monitoring, also plays an important role.26  These types of data verification 
activities are equally important for RWD that is intended to be used for regulatory purposes. 
Regardless of the original purpose for collecting the RWD, procedures for data collection and 
quality assurance should be put into place during the data source design and development stages 
(when applicable) to optimize the reliability, quality and usefulness of the data.  The data 
collection procedures used for data platforms such as registries should be clearly defined and 
described in a detailed data management standard operating procedures (SOP) manual.  When 
implementing a retrospective study using RWD, standardizing procedures to ensure the use of 
uniform and systematic methods for collecting and cleaning data are vital to ensuring data 
quality.  For example, a quality system approach with a risk-based quality assurance and 
monitoring plan is a practical strategy for data platforms such as registries that can be 
challenging to audit.  The RWD source organization or entity considering using the RWD for 
regulatory purposes should retain records regarding the assessment of adherence to the RWD 
sources established data quality assurance and quality control policies and procedures. 
VI.  Examples Where RWE is Used 
The following examples are generalized from actual uses of RWE in support of regulatory 
decision making.  These examples do not represent a comprehensive list of all potential uses or 
sources of RWD, but do describe some situations where RWE might be used to support 
regulatory decision-making. 
A.  Expanded Indications for Use 
The National Cardiovascular Data Registry (NCDR) was created in 1997 by the American 
College of Cardiology (ACC) as an exploration into strategies for improving cardiovascular 
care through the use and application of clinical data.  These registries are designed to help 
participants measure, benchmark, and improve cardiovascular care.  In particular, the Registry 
for diagnostic cardiac CATHeterization and Percutaneous Coronary Intervention (Cath-PCI 
 
26 Oversight of Clinical Investigations: A Risk-Based Approach to Monitoring (PCORI Conduct of Registry Studies 
(http://www.pcori.org/sites/default/files/Standards-in-the-Conduct-of-Registry-Studies-for-Patient-Centered-
Outcomes-Research1.pdf) 
17 
  Contains Nonbinding Recommendations 
 
 
Registry) assesses the characteristics, treatments and outcomes of cardiac disease patients who 
receive diagnostic catheterization and/or percutaneous coronary intervention (PCI) procedures, 
measuring adherence to ACC/AHA clinical practice guideline recommendations, procedure 
performance standards and appropriate use criteria for coronary revascularization.  As a registry 
collecting RWD on patients treated in routine clinical practice with an approved or cleared 
device, an IDE is not required even though a substantial volume of RWD is generated for uses 
outside of the devices cleared or approved indications for use.  RWD from this registry could be 
used to identify opportunities to expand the labeling of devices captured in the registry.  Should a 
manufacturer wish to expand indications, this type of RWD might provide sufficient evidentiary 
support, depending on the specific devices, indications, and analyses. 
Another example is a Class III device that, due to technological advances in its design, has seen 
an expansion of clinically acceptable use that is outside of the devices approved indications for 
use.  There is little published data to support a reasonable assurance of safety and effectiveness 
of the new use.  To address the lack of data to support new indications for use, relevant medical 
societies have established a national registry to collect safety and effectiveness information for 
all patients implanted with this specific device at participating institutions.  This registry also 
uses a validated matching algorithm to link registry records with administrative healthcare claims 
as a supplemental dataset for capturing long-term outcomes.  A study using the registry data 
collection and analysis infrastructure was initiated with an approved IDE application since the 
study focused on a use of this device that was not within the approved indications for use and 
imposed changes to the collection regiment of specific follow-up data that might not otherwise 
have occurred as part of routine medical care.  FDA is hopeful that the RWD may be of 
sufficient quality to address critical safety questions and to potentially support labeling changes 
or other regulatory decisions for this device.   
B.  Postmarket Surveillance Studies (Section 522)  
FDA has issued a series of postmarket surveillance orders, related to investigating patient safety 
issues for a type of class II device, under the authority of section 522 of the FD&C Act.  These 
522 orders covered multiple devices from different manufacturers that are similar in intended 
use, design, and other characteristics, such that the surveillance questions were identical.  To 
comply with the orders, many manufacturers decided to collaborate with a clinical professional 
society in this field and with FDA to develop a patient registry that would collect the needed data 
to address the public health questions.  An IDE was not required because the 522 order was 
focused on device uses consistent with the labeling.  The resultant registry was designed to 
collect RWD on all patients with the relevant condition, including those treated with the devices 
of interest, with other devices, and through medical management.  Manufacturers are able to 
share the comparator group consisting of treatments that do not use the devices of interest.  In 
addition, because the registry was designed at the outset to produce regulatory-quality RWD in 
addition to meeting research and quality improvement purposes, appropriate data quality checks 
and electronic controls were a part of the initial design and implementation.  Since this registry 
development process took a substantial amount of time, FDA was willing to grant extensions to 
manufacturers to respond to the 522 orders as long as progress was being made.  The registry 
was also designed to allow for its use (with additional protocols and other traditional study 
18 
  Contains Nonbinding Recommendations 
 
 
operational elements) in conducting premarket studies that could support future premarket 
submissions.  
C. Post-Approval Device Surveillance as Condition of Approval 
Permanent implants are typically designed to serve patients for a time period that is much longer 
than what can reasonably be captured in a premarket clinical trial.  For example, a trial that 
follows patients for two years after implantation would not produce data for the designed life 
span of 7 to10 years for that implanted device.  Traditionally, FDA would require extended 
follow-up of the premarket patient cohort and an additional new-enrollment study designed to 
capture hundreds to thousands of patients with follow-up for the life of the implanted device.  
Some clinical professional societies have developed registries that collect RWD on patients 
receiving these devices.  FDA has worked with manufacturers and professional societies to 
evaluate the registries and has found that they can be reliable for certain health outcomes of 
interest.  Should sufficient RWD exist that are capable of addressing the questions for which a 
Post-Approval Study (PAS) requirement may be issued, FDA may instead issue a condition of 
approval that requires collection/reporting of these RWD on the device. 
For example, a new breakthrough Class III medical device was approved based on prospective, 
randomized, and controlled clinical trial data.  Early in the PMA review process, the 
manufacturer began to consider postmarket commitments, and began discussions with FDA and 
other stakeholders. A registry was launched that generated RWD that could meet FDAs data 
requirements, as well as others. Because the new registry was constructed early enough to collect 
information about all patients receiving this device upon approval, FDA could provide an earlier 
device approval conditioned on further robust RWD collection and reporting in the postmarket 
setting.  This registry has since been used to a) collect surveillance data on subsequent devices 
with similar designs and indications, b) collect and retrospectively analyze RWD on all uses of 
the devices to support new expanded indications for use, and c) support embedded prospective 
clinical investigations under IDE for new devices and new generations of approved devices.  No 
IDE is necessary for the general data collection activities of the registry, as it collects RWD on 
all uses of otherwise approved medical devices and it does not influence the treatment decisions 
and/or follow-up care that patients receive.  The retrospective analysis of RWD for uses that are 
outside the approved indications for use did not require an IDE because treatment decisions were 
not influenced by the expectation of conducting the future analysis, but was still reviewed by an 
IRB for human subject protection issues.  However, prospective enrollment in a clinical 
investigation using the registry infrastructure to study a new, unapproved significant risk device 
would require an IDE.  Similarly, a prospective, non-observational clinical investigation of a new 
indication for an approved device may require an IDE, depending on the risk determination.   
D. Control Group 
A manufacturer approached FDA during the development of a next generation medical device 
that had substantial technological changes from previous iterations of that specific device and 
other similar devices from other manufacturers.  FDA determined that clinical evidence was 
needed to support an approval decision for this device modification.  A registry exists that 
captures RWD on all uses of medical devices with a similar intended use.  The manufacturer 
19 
  Contains Nonbinding Recommendations 
 
 
designed a clinical study that compared the use of the new device to a non-randomized 
concurrent control group derived from the registry. The existing registry was evaluated by FDA 
and the manufacturer according to the factors cited in this guidance and was found to provide 
sufficiently relevant and reliable RWD on the control population, such that the manufacturer did 
not have to collect additional data from these patients or influence the course of their clinical 
care in any way. The patients who received the investigational device were enrolled under an 
approved IDE. However, the patients who contributed to the control group were not considered 
part of the IDE because they were enrolled in a national registry that collected RWD on FDA 
approved devices, and their treatment was not influenced by the existence of the study. 
E. Supplementary Data  
Periodically, FDA identifies an issue related to the safety of a marketed medical device that was 
not detected in premarket trials.  In the case where RWD has been systematically collected, FDA 
has used these RWD, in combination with case reports, publications, adverse event reports, 
engineering and nonclinical test data, and other sources of information available to FDA to gain 
a better understanding of the severity of the issue, precipitating factors, affected populations, and 
alternative therapies.  The addition of RWD has proven extremely valuable as a means to 
develop a course of action that best protects public health in these instances. 
For example, a class III device was under review for a new indication.  The manufacturer 
provided data from a prospective clinical trial with limited follow-up information and inadequate 
data from the control group, which made interpretation of results difficult.  However, a pre-
existing registry was already collecting and reporting RWD on the control therapies.  The 
registry data were used to supplement and help interpret the clinical trial data, allowing FDA to 
come to an appropriate regulatory decision without requiring additional clinical trial data.  
Without the RWE, additional study subjects could have been exposed to a device with a 
questionable risk-benefit balance.  Coming to a final decision more quickly in this case protected 
subjects health while also spurring development of new designs for the medical device. 
F. Objective Performance Criteria and Performance Goals    
An Objective Performance Criterion (OPC) refers to a numerical target value derived from 
historical data from clinical studies and/or registries and may be used in a dichotomous 
(pass/fail) manner for the review and comparison of safety or effectiveness endpoints.27 An OPC 
is usually developed when device technology has sufficiently matured and can be based on 
publicly available information or on information pooled from all available studies on a particular 
kind of device.  Similar to OPC, a performance goal (PG) refers to a numerical value that is 
considered sufficient for use in the evaluation of an investigational device regarding a safety 
and/or effectiveness endpoint.  But, generally, the device technology is not as well-developed or 
 
27 Design Considerations for Pivotal Clinical Investigations for Medical Devices - Guidance for Industry, Clinical 
Investigators, Institutional Review Boards and Food and Drug Administration Staff for more information on OPCs 
and PGs. 
http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM373766.
pdf).  
20 
  Contains Nonbinding Recommendations 
 
 
mature for use of a PG as for an OPC, and the data used to generate a PG are not considered as 
robust as those used to develop an OPC.  A PG might be considered for patient populations that 
are difficult to study or if there is no clinical equipoise for any control.  From a sufficiently 
relevant and reliable RWD source, a PG can be constructed using appropriate statistical methods, 
such as a subject-level meta-analysis.  As technology evolves over time, an OPC or PG could be 
updated using RWD.   
VII.  Glossary 
The following definitions are supplied to provide the reader with an understanding of the specific 
terms used in this guidance.  These definitions should not be construed to be new interpretations 
or clarification of the use of similar words or phrases in the FD&C Act, related code or 
regulation, other federal, state, or local laws, or other guidance documents. 
  BiasBias is any systematic error in the design, conduct, analysis, interpretation, 
publication, or review of a study and its data that results in a mistaken estimate of a 
treatments effect on disease.  This systematic error results from flaws in the method of 
selecting study participants, in the procedures for gathering data, and in the decision of 
how and whether to publish the results.  These flaws can lead to observed study results 
that tend to be different from the true results.  Bias can be minimized by ensuring that 
the study design is appropriate for addressing the study hypotheses and establishing and 
carefully monitoring procedures of data collection that are valid and reliable.28  
  ConfoundingA situation in which a non-causal association between a given exposure 
or treatment and an outcome is observed as a result of the influence of a third variable 
designated as a confounder.  The confounding variable needs to be related to both the 
treatment and the outcome under study.  Confounding is distinct from bias because this 
association, while not causal, is real.29   
  Electronic Health Record (EHR)An electronic record of health-related information 
on an individual that conforms to nationally recognized/utilized interoperability standards 
and that can be created, managed, and consulted by authorized clinicians and staff across 
more than one health care organization.30 
  Electronic Medical Record (EMR)An electronic record of health-related 
information on an individual that can be created, gathered, managed, and consulted by 
authorized clinicians and staff within one health care organization.31 
 
28 JM Last.  A dictionary of Epidemiology (3rd edition).  New York:  Oxford University Press, 1995) (M Szklo & FJ 
Nieto.  Epidemiology: Beyond the basics. Gaithersburg, MD:  Aspen Publishers, Inc., 2000 
29 L Gordis.  Epidemiology. Philadelphia:  WB Saunders, Co., 1996 
30 The National Alliance for Health Information Technology Report to the Office of the National Coordinator for 
Health Information Technology on Defining Key Health Information Technology Terms April 28, 2008. 
(http://www.himss.org/national-alliance-health-information-technology-report-office-national-coordinator-health).  
31 The National Alliance for Health Information Technology Report to the Office of the National Coordinator for 
Health Information Technology on Defining Key Health Information Technology Terms April 28, 2008. 
(http://www.himss.org/national-alliance-health-information-technology-report-office-national-coordinator-health) 
21 
  Contains Nonbinding Recommendations 
 
 
  Interventional StudyA clinical study in which participants are assigned to receive 
one or more interventions (or no intervention) so that researchers can evaluate the effects 
of the interventions on biomedical or health-related outcomes. The assignments are 
determined by the study protocol. Participants may receive diagnostic, therapeutic, or 
other types of interventions.32 
  Large Simple Trial  A large simple trial (LST) is a type of randomized clinical trial 
(RCT) ideally suited to answer many important clinical questions and because it typically 
answers only one or two questions in a broader patient population, is generally more 
efficient and less expensive than other large RCTs. LSTs have a large sample size and 
statistical power to detect clinically relevant treatment effects, providing unambiguous 
results and minimizing the effects of random errors. 33 
  Medical Administrative Claims DataClaims data arise from a persons use of the 
health care system [and reimbursement of health care providers for that care].34 
  Medical Device Registry  Organized system that continuously and consistently 
collects relevant data in conjunction with routine clinical care, evaluates meaningful 
outcomes and comprehensively covers the population defined by exposure to particular 
device(s) at a reasonably generalizable scale (e.g. international, national, regional, and 
health system) with a primary aim to improve the quality of patient care. 35  
  Medically Recognized Standards of CareMedically recognized standards of care 
are treatments or procedures that have been accepted by medical experts as appropriate 
treatments or procedures for a given type of disease or condition and are commonly used 
by health care professionals.  The medical recognition of standards of care is typically 
represented by publication in a peer-reviewed journal or some form of recognition by a 
professional medical society.  The evidentiary bases for these recognized standards of 
care vary.36  
  Observational Study A study that does not involve any intervention (experimental or 
otherwise) on the part of the investigator; e.g., a population study in which changes in 
health status are studied in relation to changes in other characteristics. Most analytical 
epidemiological designs (notably, case-control and cohort studies) are properly called 
observational because investigators observe without intervening other than to record, 
classify, count, and analyze results. 37 
 
32  https://www.clinicaltrials.gov/ct2/about-studies/glossary    
33 Project: Large Simple Trials; Clinical Trials Transformation Initiative (https://www.ctti-
clinicaltrials.org/projects/large-simple-trials)  
34 Strom, Brian. Pharmacoepidemiology. Chichester, England: John Wiley and Sons, 2005. 
35 IMDRF Registry Essential Principles. Available at: 
http://www.imdrf.org/docs/imdrf/final/consultations/imdrf-cons-essential-principles-151124.pdf. Accessed August 
3, 2016 
36 Ethical Review and Oversight Issues in Research Involving Standard of Care Interventions: Workshop in Brief 
2015, Institute of Medicine (https://www.nap.edu/read/21668/chapter/1)  
37A Dictionary of Epidemiology (6 ed.) Oxford Reference http://irea.ir/files/site1/pages/dictionary.pdf 
22 
  Contains Nonbinding Recommendations 
 
 
  Postmarket surveillance Postmarket surveillance is the active, systematic, 
scientifically valid collection, analysis and interpretation of data or other information 
about a marketed device.38 
   Pragmatic clinical trial (PCT) -  A clinical trial designed for the primary purpose of 
informing decision-makers regarding the comparative balance of benefits, burdens and 
risks of a biomedical or behavioral health intervention at the individual or population 
level.39 
  Prospective StudyA prospective study design (also called a concurrent cohort study) 
defines the original population of interest at the start of the study and collects 
exposure/treatment and outcome data from that time point forward.  The start of the study 
is defined as the time the research protocol for the specific study question was 
initiated.40   
  Real-World Data (RWD) are data relating to patient health status and/or the delivery of 
health care routinely collected from a variety of sources.   
  Real-World Evidence (RWE) is the clinical evidence regarding the usage and potential 
benefits or risks of a medical product derived from analysis of RWD.     
  RegistryAn organized system that uses observational study methods to collect 
uniform data (clinical and other) to evaluate specified outcomes for a population defined 
by a particular disease, condition, or exposure, and that serves one or more predetermined 
scientific, clinical or policy purposes.41 
  Retrospective StudyA retrospective study design (also called a retrospective cohort 
study, a historical cohort, or non-concurrent prospective study) defines the population 
and determines the exposure/treatment from historical data (i.e., data generated prior to 
the initiation of the study).  The variables and outcomes of interest are determined at the 
time the study is initiated.  Some studies are a combination of concurrent and 
retrospective cohort designs where the exposure/treatment is ascertained from existing 
objective records (e.g., medical records, claims data), and follow up and measurement of 
the outcome continues into the future.42  
 
38 21 CFR 822.3 
39 Adapted from Califf R and Sugarman J; Exploring the Ethical and Regulatory Issues in Pragmatic Clinical Trials; 
Clin Trials, 2015 Oct;12(5):436-41. doi: 10.1177/1740774515598334. Epub 2015 Sep 15 
40 JM Last.  A dictionary of Epidemiology (3rd edition).  New York:  Oxford University Press, 1995) (M Szklo & FJ 
Nieto.  Epidemiology: Beyond the basics. Gaithersburg, MD:  Aspen Publishers, Inc., 2000 
 
41 Registries for Evaluating Patient Outcomes: A User's Guide 
(https://www.effectivehealthcare.ahrq.gov/ehc/products/420/1897/registries-guide-3rd-edition-vol-1-140430.pdf) 
42 JM Last.  A dictionary of Epidemiology (3rd edition).  New York:  Oxford University Press, 1995) (M Szklo & FJ 
Nieto.  Epidemiology: Beyond the basics. Gaithersburg, MD:  Aspen Publishers, Inc., 2000 
23 
  Contains Nonbinding Recommendations 
 
 
  SurveillanceSurveillance is a continuous and systematic process of collection, 
analysis, interpretation, and dissemination of descriptive information for monitoring 
health problems.43   
  Traditional clinical trialTraditional clinical trials are typically conducted in 
specialized research settings with specific populations. They often utilize measures 
designed to control variability and ensure data quality, such as detailed eligibility criteria, 
detailed case report forms that exist apart from ordinary medical records, and intensive 
monitoring and auditing designed to ensure precise adherence to study procedures and 
rigorous precision in data collection. They typically also include substantial efforts to 
ensure compliance with treatments and to avoid concomitant treatments that might 
influence the randomized treatment effect. 
 
 
43 JW Buehler. Surveillance (Ch. 22) pages 435-458 in KJ Rothman & S Greenland (editors) Modern Epidemiology 
2nd edition. Philadelphia:  Lippincott-Raven Publishers, 1998 
24