Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Use of Real-World Evidence to 
Support Regulatory Decision-Making 
for Medical Devices
 
 
Draft Guidance for Industry and 
Food and Drug Administration Staff 
 
DRAFT GUIDANCE 
 
This draft guidance document is being distributed for comment purposes 
only. 
 
Document issued on December 19, 2023. 
 
You should submit comments and suggestions regarding this draft document within 60 days of 
publication in the Federal Register of the notice announcing the availability of the draft 
guidance. Submit electronic comments to https://www.regulations.gov. Submit written 
comments to the Dockets Management Staff, Food and Drug Administration, 5630 Fishers Lane, 
Room 1061, (HFA-305), Rockville, MD 20852-1740. Identify all comments with the docket 
number listed in the notice of availability that publishes in the Federal Register. 
 
For questions about this document regarding CDRH-regulated devices, contact the Office of 
Clinical Evidence and Analysis at CDRHClinicalEvidence@fda.hhs.gov. For questions about 
this document regarding CBER-regulated devices, contact the Office of Communication, 
Outreach, and Development (OCOD) at 1-800-835-4709 or 240-402-8010, or by email at 
ocod@fda.hhs.gov. 
 
When final, this guidance will supersede Use of Real-World Evidence to 
Support Regulatory Decision-Making for Medical Devices, issued August 
2017. 
 
U.S. Department of Health and Human Services 
Food and Drug Administration 
Center for Devices and Radiological Health 
Center for Biologics Evaluation and Research  
 
 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Preface 
 
 
Additional Copies 
 
CDRH 
Additional copies are available from the Internet. You may also send an email request to CDRH-
Guidance@fda.hhs.gov to receive a copy of the guidance. Please include the document number 
GUI00500012 and complete title of the guidance in the request. 
 
CBER 
Additional copies are available from the Center for Biologics Evaluation and Research (CBER), 
Office of Communication, Outreach, and Development (OCOD), 10903 New Hampshire Ave., 
Bldg. 71, Room 3128, Silver Spring, MD 20993-0002, or by calling 1-800-835-4709 or 240-402-
8010, by email, ocod@fda.hhs.gov, or from the Internet at https://www.fda.gov/vaccines-blood-
biologics/guidance-compliance-regulatory-information-biologics/biologics-guidances. 
 
 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Table of Contents
 
I.  Introduction ............................................................................................................................. 1 
 
II.  Background ............................................................................................................................. 2 
III.  Scope ....................................................................................................................................... 5 
IV.  Regulatory Context in Which Use of RWE May be Appropriate .......................................... 6 
A.  General considerations for the use of RWE ..................................................................... 6 
B.  Application of Investigational Device Exemption (IDE) Requirements in 21 CFR 812 to 
the Collection of RWD ............................................................................................................... 9 
C.  Application of RWD from devices authorized for emergency use under section 564 of 
the FD&C Act ........................................................................................................................... 10 
V.  Assessing Data Relevance and Reliability ............................................................................ 11 
A.  Relevance ....................................................................................................................... 12 
(1)  Data availability .......................................................................................................... 12 
(2)  Linkages...................................................................................................................... 13 
(3)  Timeliness ................................................................................................................... 14 
(4)  Generalizability of data .............................................................................................. 14 
B.  Reliability ....................................................................................................................... 14 
(1)  Data Accrual ............................................................................................................... 14 
(2)  Data Quality and Integrity .......................................................................................... 16 
VI.  Considerations for Methodologies for Collection and Analysis of RWD to Generate RWE18 
A.  Methods for study designs using RWD.......................................................................... 19 
B.  Defining study design elements ..................................................................................... 20 
(1)  Study time, relative to index date ............................................................................... 20 
(2)  Development of conceptual and operational definitions for the study population, 
device, comparator, outcome, and covariates ....................................................................... 22 
(3)  Appropriate integration of data elements within study design and analysis .............. 24 
VII. Documentation for FDA Review .......................................................................................... 25 
A.  Regulatory Submission Cover Letters............................................................................ 25 
B.  Fit-For-Purpose Assessment .......................................................................................... 26 
C.  Protocol .......................................................................................................................... 26 
D.  Report ............................................................................................................................. 28 
E.  Additional Information ................................................................................................... 28 
 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Appendix A. Recommended Elements for Documentation and FDA Review ............................. 29 
Appendix B. Examples Where RWE is Used ............................................................................... 33 
 
 
 
 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Use of Real-World Evidence to 
1 
Support Regulatory Decision-Making 
2 
for Medical Devices
3   
4   
Draft Guidance for Industry and 
5 
Food and Drug Administration Staff 
6 
7   
8  This draft guidance, when finalized, will represent the current thinking of the Food and Drug 
9  Administration (FDA or Agency) on this topic.  It does not establish any rights for any person 
10  and is not binding on FDA or the public.  You can use an alternative approach if it satisfies 
11  the requirements of the applicable statutes and regulations.  To discuss an alternative 
12  approach, contact the FDA staff or Office responsible for this guidance as listed on the title 
13  page.  
14   
15  I.  Introduction  
16  FDA is issuing this draft guidance to clarify how FDA evaluates real-world data to determine 
17  whether they are of sufficient quality for generating real-world evidence that can be used in FDA 
18  regulatory decision-making for medical devices. This draft guidance also provides expanded 
19  recommendations to sponsors considering using real-world evidence to support a regulatory 
20  submission for medical devices.1  
21   
22  Real-world data (RWD) are data relating to patient health status and/or the delivery of health 
23  care routinely collected from a variety of sources. 
24   
25  Examples of RWD sources include data derived from electronic health records (EHRs),2 medical 
26  claims data, data from product and disease registries, and data gathered from other sources (such 
27  as digital health technologies) that can inform on health status. RWD sources can be used as data 
 
1 For more information on regulatory submissions for medical devices, see Sections III. and VII.  
2 For the purposes of this draft guidance, an EHR" is an electronic record of health-related information on an 
individual that conforms to nationally recognized/utilized interoperability standards and that can be created, 
managed, and consulted by authorized clinicians and staff across more than one health care organization. Definition 
adapted from The National Alliance for Health Information Technology Report to the Office of the National 
Coordinator for Health Information Technology on Defining Key Health Information Technology Terms April 28, 
2008, available at webarchive.library.unt.edu/eot2008/20080920183033/http:/www.hhs.gov 
/healthit/documents/m20080603/10_2_hit_terms.pdf  
 
1 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
28  collection and analysis infrastructure to support many types of study designs, including, but not 
29  limited to, randomized and non-randomized controlled trials; single-arm studies with or without 
30  comparison to an objective performance criterion, performance goal, or extended control; 
31  observational studies; and hybrid designs which combine elements of multiple study designs. 
32   
33  Real-world evidence (RWE) is the clinical evidence regarding the usage, and potential benefits 
34  or risks, of a medical product derived from analysis of RWD. 
35   
36  This draft guidance includes factors that FDA considers important to demonstrate whether the 
37  RWD are fit-for-purpose for a particular regulatory decision relating to medical devices, as well 
38  as FDAs recommendations on how FDA intends to assess these factors. When finalized, the 
39  recommendations and considerations in this draft guidance will apply regardless of the RWD 
40  source and encompass processes for conducting studies to generate RWE. A fit-for-purpose 
41  assessment should evaluate both the relevance and reliability of the RWD, discussed in more 
42  detail in Section V. FDA recognizes that there may be other approaches to address the 
43  considerations identified in this document. We encourage sponsors to discuss their approach with 
44  FDA, especially if the approach diverges from the recommendations in this draft guidance, when 
45  finalized.3 
46   
47  FDA recognizes and anticipates that the Agency and industry may need up to 60 days to perform 
48  activities to operationalize the recommendations within the final guidance. At this time, the 
49  Agency anticipates that, for regulatory submissions that will be currently pending with FDA after 
50  publication of the final guidance, as well as those submissions received within 60 days following 
51  publication of the final guidance, FDA generally would not anticipate that sponsors will be ready 
52  to include the newly recommended information outlined in the final guidance in their 
53  submission. FDA, however, would intend to review any such information if submitted at any 
54  time.  
55   
56  In general, FDAs guidance documents do not establish legally enforceable responsibilities. 
57  Instead, guidances describe the Agencys current thinking on a topic and should be viewed only 
58  as recommendations, unless specific regulatory or statutory requirements are cited. The use of 
59  the word should in Agency guidances means that something is suggested or recommended, but 
60  not required. 
61   
62  II.  Background 
63  To protect and promote public health, FDA needs to understand and evaluate the available 
64  evidence related to regulated products. For medical devices, common sources of available 
65  evidence include non-clinical and clinical studies4 provided to FDA by a device manufacturer or 
 
3 See FDAs guidance, Requests for Feedback and Meetings for Medical Device Submissions: The Q-Submission 
Program, available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/requests-
feedback-and-meetings-medical-device-submissions-q-submission-program 
4 For the purposes of this draft guidance, we use the term clinical studies in this guidance as a broad term to 
capture clinical research regarding the safety or effectiveness of a device, regardless of study design. We use the 
term traditional clinical studies to refer to clinical studies that do not utilize RWD. 
2 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
66  sponsor of a device premarket or postmarket submission. FDA recognizes that a wealth of 
67  clinical data in the form of RWD are routinely collected in the course of clinical practice during 
68  the treatment and management of patients. Although these data typically have different quality 
69  controls compared to data collected within a traditional clinical study, under certain 
70  circumstances RWD may be used to generate RWE to help inform or augment FDAs 
71  understanding of the benefit-risk profile of devices at various points in their life cycle. Per 21 
72  CFR 860.7(c)(1), [a]lthough [a] manufacturer may submit any form of evidence to the Food and 
73  Drug Administration in an attempt to substantiate the safety and effectiveness of a device, the 
74  agency relies upon only valid scientific evidence to determine whether there is reasonable 
75  assurance that the device is safe and effective. RWE derived from relevant and reliable RWD 
76  may constitute valid scientific evidence,5 depending on the study question, regulatory decision, 
77  data source(s), and design and analysis of the specific dataset derived from RWD source(s), and 
78  thus may be used to support regulatory decisions. FDA intends to use the considerations 
79  described in this draft guidance, when finalized, to evaluate whether RWD are relevant and 
80  reliable to support regulatory decision-making, including potentially generating valid scientific 
81  evidence. The use of RWE for specific regulatory purposes will include assessment of the overall 
82  relevance and reliability of the RWD used to generate the RWE.  
83   
84  When appropriate, use of RWD may provide an efficient means of generating the necessary 
85  clinical evidence to support regulatory decisions. Information specific to the clinical performance 
86  of a device can be generated through a number of methodological and operational approaches. In 
87  general, traditional clinical studies tend to be narrow in scope but allow for more control of 
88  sources of error and bias. In comparison, studies that leverage RWD may be able to evaluate 
89  broader questions but are subject to sources of bias which may be more difficult to control. 
90  Clinical evidence can be generated from studies using RWD, alone or in combination with data 
91  from more traditional clinical studies. Using appropriate design and methodologies, sponsors can 
92  leverage the strengths of these approaches while minimizing potential weaknesses.  
93   
94  RWD that includes patient experience data6 may provide new insights into the performance of a 
95  device. In addition, RWD may foster inclusion of target populations that are otherwise 
96  underrepresented in clinical studies. Similarly, leveraging RWD may allow for studies of a 
97  longer period of time than would be practical in a traditional clinical study and so may allow for 
98  data to be gathered on longer term outcomes. Clinical evidence generated from fit-for-purpose 
 
5 Under 21 CFR 860.7(c)(2), valid scientific evidence is considered evidence from well-controlled investigations, 
partially controlled studies, studies and objective trials without matched controls, well-documented case histories 
conducted by qualified experts, and reports of significant human experience with a marketed device, from which it 
can fairly and responsibly be concluded by qualified experts that there is reasonable assurance of the safety and 
effectiveness of a device under its conditions of use. The evidence required may vary according to the characteristics 
of the device, its conditions of use, the existence and adequacy of warnings and other restrictions, and the extent of 
experience with its use. 
6 For the purposes of this draft guidance, patient experience data means data that are (1) collected by any persons 
(including patients, family members and caregivers of patients, patient advocacy organizations, disease research 
foundations, researchers, and drug manufacturers); and (2) are intended to provide information about patients 
experiences with a disease or condition, including (A) the impact of such disease or condition, or a related therapy, 
on patients lives; and (B) patient preferences with respect to treatment of such disease or condition. See section 
3001 of the 21st Century Cures Act, Pub. L. No. 114-255 (December 2016). 
3 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
99  RWD informs device benefit-risk profiles assessment from a real-world environment, allows 
100  evaluation of outcomes which may not be feasible in traditional clinical studies, and better aligns 
101  with device innovation cycles to inform future device modifications and new technology 
102  development. Finally, RWD may include information from broader clinical experiences than is 
103  usually represented in traditional clinical studies. RWE is an important factor for understanding 
104  and regulating medical devices, and therefore, FDA encourages the medical community to learn 
105  more from routine clinical care to help support safety and effectiveness of medical devices. Use 
106  of relevant and reliable RWD to generate RWE can benefit stakeholders throughout the 
107  ecosystem, including but not limited to, patients, health care providers, manufacturers, and FDA.  
108   
109  Additionally, in some cases, a traditional clinical study may be impractical or excessively 
110  challenging to conduct. Ethical issues regarding treatment assignment, and other similar 
111  challenges, may present themselves when developing and attempting to execute such a study. 
112  Analyses of RWD, using appropriate methods, may in some cases provide similar information 
113  with comparable or even superior characteristics to information collected and analyzed through a 
114  traditional clinical study. Under the right conditions, RWE may be suitable to support the 
115  marketing authorization of a new device or the expansion of the indications for use of devices 
116  that are already on the market. Aggregation of RWD (e.g., in a medical device registry7) may 
117  prove useful as a postmarket control suitable for providing ongoing device safety surveillance 
118  and additional evidence for effectiveness. FDA has long applied postmarket controls as a way to 
119  reduce premarket data collection, while still ensuring that the statutory standard of reasonable 
120  assurance of safety and effectiveness is met.8 FDA believes that applying postmarket controls to 
121  reduce premarket data collection, when appropriate, can help improve patient access to safe and 
122  effective medical devices.9 
123   
124  Many of the considerations and best practices for generating RWE are derived from the same 
125  principles that govern generation of clinical evidence from traditional clinical studies, which are 
126  generally referred to as good clinical practice (GCP). Additionally, as with all clinical evidence 
127  FDA evaluates, FDAs assessment of RWE evaluated in support of a particular regulatory 
128  decision will be included as part of the totality of information available to FDA. Further, as with 
129  all types of clinical data, FDA recognizes there may be uncertainty of the benefits and risks of a 
130  device that remain after completion of a study using RWD. Some of these aspects are similar to 
131  those also present in more traditional forms of clinical data, but some are unique to RWD. 
 
7 For the purposes of this draft guidance, medical device registry means an organized system with a primary aim 
to increase the knowledge on medical devices contributing to improve the quality of patient care that continuously 
collects relevant data, evaluates meaningful outcomes and comprehensively covers the population defined by 
exposure to particular device(s) at a reasonably generalizable scale (e.g., international, national, regional, and health 
system). Definition is cited from the IMDRF document Principles of International System of Registries Linked to 
Other Data Sources and Tools, available at https://www.imdrf.org/documents/principles-international-system-
registries-linked-other-data-sources-and-tools 
8 For more information, see FDAs guidance The Least Burdensome Provisions: Concept and Principles, available 
at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/least-burdensome-provisions-
concept-and-principles 
9 For more information, see FDAs guidance Balancing Premarket and Postmarket Data Collection for Devices 
Subject to Premarket Approval, available at https://www.fda.gov/regulatory-information/search-fda-guidance-
documents/balancing-premarket-and-postmarket-data-collection-devices-subject-premarket-approval  
4 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
132  Assessment of the relevance and reliability of the RWD, as outlined in this draft guidance, can 
133  identify uncertainty that should be considered during the benefit-risk determinations10 for the 
134  device for a given regulatory purpose. 
135   
136  In 2017, FDA issued the guidance document, Use of Real-World Evidence to Support 
137  Regulatory Decision-Making for Medical Devices,11 in which we described the relevance and 
138  reliability factors of RWD that FDA assesses to determine if RWD are sufficient for generating 
139  RWE. Subsequently, on December 29, 2022, the Food and Drug Omnibus Reform Act of 2022 
140  (FDORA) was signed into law as part of the Consolidated Appropriations Act, 2023, Pub. L. 
141  No. 117-328. Section 3629 of FDORA Facilitating the Use of Real World Evidence directs 
142  FDA to issue or revise existing guidance on considerations for the use of RWD and RWE to 
143  support regulatory decision-making. FDA is issuing this draft guidance to propose revisions to 
144  the 2017 guidance, Use of Real-World Evidence to Support Regulatory Decision-Making for 
145  Medical Devices, to satisfy the requirement under section 3629(a)(2). This draft guidance also 
146  fulfills a commitment in Section V.F. of the Medical Device User Fee Amendments Performance 
147  Goals and Procedures, Fiscal Years 2023 Through 2027 (MDUFA V).12 This draft guidance is 
148  intended to provide expanded and updated recommendations to industry and FDA staff for 
149  conducting an assessment of relevance and reliability to demonstrate that RWD may be fit-for-
150  purpose to generate clinical evidence for regulatory decision-making. This includes 
151  recommendations to provide clarity on least burdensome general expectations related to 
152  demonstrating that RWD is fit-for-purpose for premarket regulatory purposes.  
153   
154  III.  Scope 
155  This draft guidance is applicable for the use of RWE to support regulatory submissions for 
156  medical devices.13 
157   
 
10 See FDAs guidances, Factors to Consider When Making Benefit-Risk Determinations in Medical Device 
Premarket Approval and De Novo Classifications available at https://www.fda.gov/regulatory-information/search-
fda-guidance-documents/factors-consider-when-making-benefit-risk-determinations-medical-device-premarket-
approval-and-de; Consideration of Uncertainty in Making Benefit-Risk Determinations in Medical Device 
Premarket Approvals, De Novo Classifications, and Humanitarian Device Exemptions available at 
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/consideration-uncertainty-making-
benefit-risk-determinations-medical-device-premarket-approvals-de; Benefit-Risk Factors to Consider When 
Determining Substantial Equivalence in Premarket Notifications (510(k)) with Different Technological 
Characteristics, available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/benefit-
risk-factors-consider-when-determining-substantial-equivalence-premarket-notifications-510k; and Factors to 
Consider Regarding Benefit-Risk in Medical Device Product Availability, Compliance, and Enforcement 
Decisions, available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/factors-
consider-regarding-benefit-risk-medical-device-product-availability-compliance-and   
11 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/use-real-world-
evidence-support-regulatory-decision-making-medical-devices    
12 Available at https://www.fda.gov/media/158308/download 
13 This guidance does not apply to drugs and biological products. For information on the RWE program for drugs 
and biological products, see https://www.fda.gov/science-research/real-world-evidence/center-biologics-evaluation-
and-research-center-drug-evaluation-and-research-real-world-evidence  
5 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
158  The topics covered within this draft guidance are framed specifically for the use of RWD and 
159  RWE in regulatory submissions for medical devices (e.g., Investigational Device Exemption 
160  (IDE), premarket notification under section 510(k) of the FD&C Act, Premarket Approval 
161  Application (PMA), Humanitarian Device Exemption (HDE), De Novo classification request, 
162  post-approval study, postmarket surveillance under Section 522 of the FD&C Act (522 
163  submissions), Clinical Laboratory Improvement Amendments (CLIA) Waiver by Applications 
164  (CW), Dual De Novo/510(k) and CLIA Waiver by Application Submissions (Duals)). The 
165  considerations included in this draft guidance may be applicable to supporting uses of RWD 
166  across the medical device total product life cycle (TPLC). 
167   
168  This draft guidance does not address the use of non-clinical data, adverse event reports, 
169  secondary use of clinical study data, or systematic literature reviews. Nor does it address all 
170  possible study design/conduct or analytical methodologies. While it does describe the factors that 
171  FDA considers when evaluating relevance and reliability of RWD, it does not provide a specific 
172  set of criteria or other scoring tools for determining the suitability of any specific RWD source 
173  for generating RWE for a particular regulatory decision. 
174   
175  This guidance, when finalized, should not be construed to alter or change in any way the existing 
176  evidentiary standards applicable to FDAs regulatory decision-making. Rather, this guidance 
177  describes the circumstances under which clinical evidence generated from RWD may be used to 
178  support a variety of FDA decisions based on the existing evidentiary standards. While FDA 
179  encourages the use of relevant and reliable data to generate clinical evidence, including RWE, 
180  this draft guidance neither mandates use of RWD and RWE nor restricts other means of 
181  providing evidence to support regulatory decision-making. This draft guidance does not affect 
182  any federal, state, or local laws or regulations, or foreign laws or regulations that may be 
183  applicable to the use or collection of RWD, or that provide protections for human subjects 
184  (including informed consent requirements) or patient privacy. When finalized, this guidance 
185  should be used to complement, but not supersede, other device-specific and GCP and guidance 
186  documents.  
187   
188  IV.  Regulatory Context in Which Use of RWE May be 
Appropriate  
189 
190  A.  General considerations for the use of RWE  
191  In general, FDA considers the use of RWD to be fit-for-purpose to support generation of clinical 
192  evidence for regulatory decision-making for medical devices when we conclude that the RWD 
193  used to generate the RWE are relevant to and reliable for informing or supporting a particular 
194  regulatory decision. It is important to understand the strengths and limitations of the underlying 
195  RWD and how these qualities impact their relevance and reliability. Similarly, the context of the 
196  specific regulatory decision for which the RWE is being proposed is central to FDAs evaluation.  
197   
6 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
198  FDA recognizes that RWE can be generated from a variety of RWD sources that are primarily 
199  intended for another purpose. For example, administrative claims data14 are typically collected 
200  for purposes of billing or payment for medical care. Disease-specific RWD sources may be 
201  useful for tracking progression or outcomes of specific rare or poorly understood diseases. 
202  Treatment-specific RWD sources may have several purposes, including assessment and tracking 
203  of overall outcomes, providing assessment of hospital operations, informing performance 
204  improvement initiatives, or providing risk prediction and benchmarking data for specific 
205  therapies. The suitability of the RWD source may be determined by the factors outlined in 
206  Section V. and the availability of sufficient data to address the study question of interest. 
207   
208  FDA does not endorse one type of RWD over another. Sponsors should select the appropriate 
209  RWD sources based on their suitability to address the specific study questions. Data sources that 
210  may be considered RWD sources include the following:15 
211   
212    Registries;16 
213    EHRs; 
214    Administrative claims data; 
215    Patient-generated data17 created, reported, or gathered by patients including in-home use 
216  settings (e.g., data from digital health technologies (DHTs)18 such as wearables); 
217    Device-generated data (e.g., implantable devices, physiological monitoring devices);  
218    Public health surveillance data (e.g., COVID-1919 case surveillance); 
 
14 For the purposes of this draft guidance, administrative claims data means claims data that arise from a persons 
use of the health care system and reimbursement of health care providers for that care. Definition adapted from 
Strom et al., Textbook of Pharmacoepidemiology 6th ed. 2022, page 137. 
15 Generally, FDA does not consider published literature to be RWD. Literature may report data from an RWD 
source, in which case sponsors should specify the RWD source type (e.g., if a journal article presents a retrospective 
analysis of EHRs, the RWD source should be specified as EHR in the cover sheet). 
16 For the purposes of this draft guidance, registry means an organized system that uses observational study 
methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a 
particular disease, condition, or exposure, and that serves one or more stated scientific, clinical, or policy purposes. 
Definition adapted from the Agency for Healthcare Research and Qualitys (AHRQs) Registries for Evaluating 
Patient Outcomes: A Users Guide, available at https://effectivehealthcare.ahrq.gov/products/registries-guide-4th-
edition/users-guide 
17 The use of the term patient-generated data is consistent with the use of this term in the Framework for FDAs 
Real-World Evidence Program document, available at https://www.fda.gov/media/120060/download. Patient-
generated data includes patient-generated health data. For the purposes of this draft guidance, patient-generated 
health data means health-related data created, recorded, or gathered by or from patients, family members, or other 
caregivers to help address a health concern. Definition adapted from https://www.healthit.gov/topic/scientific-
initiatives/pcor/patient-generated-health-data-pghd   
18 For the purposes of this draft guidance, digital health technology means a system that uses computing platforms, 
connectivity, software, and/or sensors for health care and related uses. These technologies span a wide range of uses, 
from applications in general wellness to applications as a medical device. They include technologies intended for 
use as a medical product, in a medical product, or as an adjunct to other medical products (devices, drugs, and 
biologics). They may also be used to develop or study medical products.  
19 In 2019, an outbreak of respiratory disease caused by a novel coronavirus began. The virus has been named 
SARS-CoV-2, and the disease it causes has been named Coronavirus Disease 2019 (COVID-19). On January 
31, 2020, the Secretary of Health and Human Services (HHS) issued a declaration of a Public Health Emergency 
 
7 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
219    Clinically annotated biobanks; and 
220    Medical device data repositories (e.g., imaging, electrocardiography databases). 
221   
222  Some purposes for which use of RWD may potentially be applicable in a regulatory submission 
223  include the following: 
224   
225    To generate hypotheses to be tested in a clinical study; 
226    As a historical control, an informative prior in a Bayesian analysis of a 
227  clinical trial,20 or as one source of data in a hierarchical model or a hybrid 
228  data synthesis; 
229    As a concurrent control group or as a mechanism for collecting data to support 
230  marketing authorization when a registry, EHR, claims data, or some other systematic 
231  data collection mechanism exists; 
232    As a mechanism for re-training artificial intelligence/machine learning-enabled 
233  medical devices; 
234    To generate evidence to identify, demonstrate, or support the clinical validity of a 
235  biomarker or clinical outcome assessment; 
236    To generate (primary) clinical evidence to support marketing authorization (e.g., 
237  HDE, PMA, 510(k) or De Novo request); 
238    To generate evidence directly by the subject device to provide new information 
239  on safety or effectiveness; 
240    To generate evidence to support a determination on whether the subject device 
241  meets the statutory criteria for a CLIA waiver21 (e.g., CW and Duals22); 
242    To generate evidence to support the interpretability of the primary clinical 
243  evidence (e.g., to demonstrate that the study population for an investigation 
244  conducted outside the United States (OUS) is representative of the US 
245  population, or to provide context for an adverse event observed in the clinical 
246  study);  
247    To generate evidence to support a petition for reclassification of a medical device 
248  under section 513(e) or (f)(3) of the FD&C Act; 
 
(PHE) related to COVID-19 in accordance with section 319 of the Public Health Service Act (PHS Act) (hereinafter 
referred to as section 319 PHE declaration) and mobilized the Operating Divisions of HHS. In addition, on March 
13, 2020, the President declared a national emergency in response to COVID-19. The section 319 PHE declaration 
related to COVID-19 expired on May 11, 2023. 
20 For more information on Bayesian trails, see FDAs guidance, Guidance for the Use of Bayesian Statistics in 
Medical Device Clinical Trials, available at https://www.fda.gov/regulatory-information/search-fda-guidance-
documents/guidance-use-bayesian-statistics-medical-device-clinical-trials-pdf-version 
21 For more information on CLIA waiver by applications, see FDAs guidance, Recommendations for Clinical 
Laboratory Improvement Amendments of 1988 (CLIA) Waiver Applications for Manufacturers of In  
Vitro Diagnostic Devices, available at https://www.fda.gov/regulatory-information/search-fda-guidance-
documents/recommendations-clinical-laboratory-improvement-amendments-1988-clia-waiver-applications  
22 For more information on the Dual 510(k) and CLIA Waiver by Application pathway, see FDAs guidance, 
Recommendations for Dual 510(k) and CLIA Waiver by Application Studies, available at 
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/recommendations-dual-510k-and-clia-
waiver-application-studies. Dual De Novo classification requests and CLIA Waiver by Application for certain 
devices (see section 3301 of FDORA) should also consult this guidance. 
8 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
249    To generate evidence for expanding the labeling of a device to include additional 
250  indications for use or to update the labeling to include new information on safety and 
251  effectiveness;23 
252    To generate evidence for postmarket surveillance. Through ongoing surveillance, 
253  signals are at times identified that suggest there may be a safety issue with a medical 
254  device. RWE may be generated using RWD to refine these signals for purposes of 
255  informing appropriate corrective actions and communication; 
256    To conduct post-approval studies that are imposed as a condition of device approval 
257  or to potentially preclude the need for 522 submissions; and 
258    To provide postmarket data in lieu of some premarket data, consistent with FDAs policy 
259  on balancing premarket and postmarket data collection.24 
260   
261  B.  Application of Investigational Device Exemption (IDE) 
262  Requirements in 21 CFR 812 to the Collection of RWD 
263  An approved IDE permits a device to be shipped lawfully for the purpose of conducting 
264  investigations of the device without complying with certain other requirements of the FD&C Act 
265  that would apply to devices in commercial distribution. The purpose of this investigational 
266  exemption, per 21 CFR 812.1, is to encourage, to the extent consistent with the protection of 
267  public health and safety and with ethical standards, the discovery and development of useful 
268  devices intended for human use, and to that end to maintain optimum freedom for scientific 
269  investigators in their pursuit of this purpose. As explained in 21 CFR Part 812, the IDE 
270  regulations apply to all clinical investigations of devices to determine safety and effectiveness, 
271  with certain limited exceptions.25 In many cases, an approved IDE is required before initiating a 
272  clinical investigation. An investigation is defined as a clinical investigation or research 
273  involving one or more subjects to determine the safety or effectiveness of a device.26 
274   
275  Whether the collection of RWD for a legally marketed device requires an IDE depends on the 
276  particular facts of the situation. Specifically, if the device is being used in the normal course of 
277  medical practice, an IDE would likely not be required. FDA recognizes that in clinical practice  
278  this could include use of a legally marketed device for uncleared or unapproved uses, where the 
279  device is being administered or prescribed under the authority of a health care practitioner within 
280  a legitimate practitioner-patient relationship. If data collection does not impact how the device is 
281  administered, and the administration is within the normal course of medical care, an IDE would 
282  likely not be required. For example, analyses of extant RWD (i.e., RWD already collected) 
283  involving the use in medical care of a device that was not within the cleared or approved 
284  indications for use would generally not be subject to IDE regulations. However, similar to 
 
23 See FDAs guidance, General/Specific Intended Use, available at https://www.fda.gov/regulatory-
information/search-fda-guidance-documents/generalspecific-intended-use-guidance-industry 
24 See FDAs guidance, Balancing Premarket and Postmarket Data Collection for Devices Subject to Premarket 
Approval, available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/balancing-
premarket-and-postmarket-data-collection-devices-subject-premarket-approval   
25 See 21 CFR 812.2(a). 
26 See 21 CFR 812.3(h). 
9 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
285  traditional clinical studies, if data are being gathered to determine the safety and effectiveness of 
286  the device, and the process for gathering the data would influence treatment decisions, such 
287  administration would likely not be within the normal course of medical practice and an IDE may 
288  be required. For example, a study using a registry infrastructure designed to determine the safety 
289  and effectiveness of an approved device for a new intended use would likely be subject to IDE 
290  requirements if physicians are instructed to treat specific patients or otherwise administer the 
291  device in a prescribed way for purposes of data generation, or when certain follow-up activities 
292  are performed for the purpose of research. 
293   
294  Should a sponsor or Institutional Review Board (IRB) be unclear regarding the applicability of 
295  the IDE regulations to a particular RWD collection activity or use, the sponsor or IRB should 
296  contact FDA. If an IDE is determined to be required, FDA intends to work with the IDE sponsor 
297  to develop the least burdensome approach to facilitate the efficient generation of RWE. Note that 
298  regardless of the applicability of 21 CFR Part 812, FDA regulations at 21 CFR Part 56 (IRB 
299  review), Part 50 (Protection of Human Subjects) and Part 54 (Financial Disclosure) may apply to 
300  RWE generation activities, as may other federal, state, and local laws regarding human subject 
301  protections. 
302   
303  C.  Application of RWD from devices authorized for 
304  emergency use under section 564 of the FD&C Act 
305  Section 564 of the FD&C Act provides that FDA may, after the HHS Secretary has made a 
306  declaration of emergency or threat justifying authorization of emergency use (an EUA 
307  declaration), authorize the emergency use of an unapproved product27 or an unapproved use of 
308  an approved product for certain emergency circumstances.  
309   
310  The routine clinical use of a device authorized under an EUA, when used within the scope of its 
311  authorization, is not considered to be a clinical investigation (see section 564(k) of the FD&C 
312  Act and Section IV.B. for more information on the application of IDE requirements in 21 CFR 
313  Part 812 to the collection of RWD). Clinical data routinely collected from the use of a device 
314  authorized under an EUA may be considered RWD and may be used to support regulatory 
315  decision-making, if determined to be fit-for-purpose. Generally, the recommendations in this 
316  draft guidance may apply to RWD from devices authorized under an EUA. Additionally, 
317  Appendix B includes an example of RWD from a device authorized under an EUA that was used 
318  in a subsequent premarket submission. 
319   
320  Device use pursuant to EUAs may lead to additional sources and novel uses of RWD to support 
321  FDA decision-making. We encourage sponsors to consider the recommendations in this guidance 
322  for devices authorized under an EUA (e.g., devices authorized under an EUA during the 
323  COVID-19 pandemic).  
324    
 
27 Under sections 564(a)(2)(A) and 564(a)(4)(D) of the FD&C Act, an unapproved product is one that is not 
approved, licensed, or cleared for commercial distribution under section 505, 510(k), 512, or 515 of [the FD&C] Act 
or section 351 of the [PHS] Act or conditionally approved under section 571 of [the FD&C] Act. 
10 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
325  V.  Assessing Data Relevance and Reliability 
326  To determine the potential suitability of RWD to generate RWE for regulatory decision-making, 
327  FDA assesses the relevance and reliability of the RWD source as well as the data elements, study 
328  design, and analytic components of the study. This section describes the elements that might be 
329  evaluated to determine if the data are fit-for-purpose. FDA recognizes that data, including RWD 
330  used to generate RWE, may have limitations. Sponsors should understand the strengths and 
331  limitations of generating evidence from RWD to address a specific study question and provide 
332  these limitations to FDA in their submission. If the RWD source appears relevant and reliable, 
333  then additional assessment of the study-specific derived dataset(s) may help demonstrate the 
334  RWD are fit-for-purpose to address the study question. This assessment may be used to 
335  determine whether the RWD source(s) and the proposed design and analysis can generate 
336  evidence that is sufficiently robust to be used for the given study question and regulatory 
337  purpose, i.e., whether the RWD are fit-for-purpose. 
338     
339  Whether data are sufficiently relevant and reliable for use will, in part, depend on the particular 
340  regulatory decision. FDA will evaluate the same factors to assess RWD across all data sources 
341  and regulatory decisions but will weigh each factor in accordance with the regulatory decision to 
342  be made. In cases where RWE is derived from multiple RWD sources, each RWD source will be 
343  evaluated individually and together in the aggregate to determine the relevance and reliability of 
344  the RWD.  
345   
346  The data should be accurate, as complete as possible, and of adequate data quality to credibly 
347  address the question at hand. Conducting a clinical investigation in accordance with GCP 
348  provides assurance that the data and results from the clinical investigation are credible and 
349  accurate and that the rights, safety, and well-being of subjects are protected.28 In traditional 
350  clinical studies, the best practices for incorporating GCP into the study design and execution are 
351  generally well established. However, because RWD are typically collected outside of a 
352  controlled research setting, additional precautions should be considered to ensure that the data 
353  are similarly credible and accurate, and that appropriate patient protections are in place. In 
354  order to determine whether data are credible and accurate, FDA assesses the relevance and 
355  reliability of the data.  
356   
357  Additionally, sponsors should ensure that RWD were collected using good data management 
358  practices and are sufficiently robust. Sponsors should also consider data related to various 
359  demographic characteristics (e.g., age, sex, race, and ethnicity)29 and other potentially relevant 
 
28 See, for example, 21 CFR 812.28(a)(1), which defines good clinical practice in the context of device 
investigations conducted outside the United States. 
29 For example, see the following FDA guidances: Guidance for the Use of Bayesian Statistics in Medical Device 
Clinical Trials; Evaluation of Sex-Specific Data in Medical Device Clinical Studies, available at 
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/evaluation-sex-specific-data-medical-
device-clinical-studies-guidance-industry-and-food-and-drug; and Evaluation and Reporting of Age-, Race-, and 
Ethnicity-Specific Data in Medical Device Clinical Studies, available at https://www.fda.gov/regulatory-
information/search-fda-guidance-documents/evaluation-and-reporting-age-race-and-ethnicity-specific-data-medical-
device-clinical-studies 
11 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
360  covariates, and whether the data are representative of the intended use population. The relevance 
361  and reliability factors listed below should be described to assess the RWD.   
362   
363  Studies using RWD should also be carefully designed to mitigate potential bias, and a study 
364  protocol and analysis plan should be created prior to analyzing RWD, regardless of whether the 
365  RWD are extant or if they are to be collected in the future. An existing RWD source may have 
366  some inherent sources of bias that could limit the relevance or reliability for drawing causal 
367  inferences between medical device exposures30 and outcomes. 
368   
369  To help ensure the relevance and reliability of the source data, FDA recommends sponsors 
370  consider the factors contained in this section. If considered, these factors should be referenced 
371  during study conduct, FDA inspection, or provided as additional information during FDA review 
372  of the applicable regulatory submission, as applicable. Appendix A sets forth the elements that 
373  FDA recommends sponsors document and have available for inspection, as well as 
374  recommended elements sponsors to include in the appropriate regulatory submission for FDA 
375  review.  
376   
377  A.  Relevance 
378  Relevance includes consideration of availability, timeliness, and generalizability of the RWD. 
379  When needed information is not available in one data source, sponsors may want to provide 
380  linkage of other data source(s). Important relevance factors that FDA will consider in 
381  determining whether RWD are suitable for generating RWE for regulatory use include the 
382  following: 
383   
384  (1)  Data availability 
385  The RWD should contain sufficient detail to capture the information needed to evaluate the 
386  question being addressed in the target population. Relevant considerations should include 
387  whether the RWD contains information on the following: 
388   
389    Use of the device (e.g., the device identifier (DI)31 portion of the unique device 
390  identifier (UDI),32 other structured data, clinical notes) or other exposure in the study 
391  population;  
392    Outcome(s) of interest in the study population;  
 
30 The exposure is the variable or data element whose causal effect is estimated in a study. For many studies 
supporting a regulatory submission, the exposure is the medical device. For some studies, especially those providing 
supportive clinical evidence, another exposure may be assessed to provide context for the use, safety, or 
effectiveness of the medical device within clinical practice. 
31 The device identifier is a mandatory, fixed portion of a UDI that identifies the specific version or model of a 
device and the labeler of that device. 21 CFR 830.3. For more information regarding UDI, see FDAs webpage 
https://www.fda.gov/medical-devices/unique-device-identification-system-udi-system/udi-rule-guidances-training-
and-other-resources 
32 For class I devices, the universal product code (UPC) may serve as the UDI (21 CFR 801.40(d)). In these  
instances, the UPC should be included. 
12 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
393    Covariates that may impact the exposure or outcomes of interest (e.g., RWD source 
394  contains signs, symptoms, treatments, procedures, diagnoses, patient and family 
395  history, pre-existing conditions, labs, demographics, and results which may be used to 
396  construct covariates that are relevant to the study question); and 
397  For example, the minimum set of data fields in a registry may be 
398  insufficient for a specific study question and additional data fields may 
399  be needed for the registry to be fit-for-purpose. The registry should 
400  retain information documenting the start or stop of collection during 
401  the  study  time  frame  for  data  fields  related  to  the  specific  study 
402  question. 
403    Longitudinality, including longevity (the length of time that data for an individual is 
404  captured within the RWD source) and continuity of care.   
405  Information across the continuum of care33 (i.e., data observability) 
406  may  aid  the  assessment  of  the  likelihood  that  all  exposures  and 
407  outcomes of interest will be captured for regulatory decision-making. 
408  For example, tertiary care hospitalization data may not have adequate 
409  data availability to study outcomes that are likely to be diagnosed in an 
410  emergency for all patients, because patients are likely to go to a nearby 
411  hospital  in  emergencies  but  may  travel  to  another  location  for  a 
412  specialty device procedure. 
413   
414  If the RWD source is insufficient on its own, the sponsor should determine whether 
415  supplemental data sources are available and sufficient to provide any missing information 
416  necessary to address the study question.  
417   
418  (2)  Linkages 
419  Sponsors should assess whether and how data from different sources can be obtained and 
420  integrated given the potential for heterogeneity in target population characteristics, clinical 
421  practices, and coding across data sources. A description of this assessment should be provided in 
422  the regulatory submission for FDA review. 
423   
424  Any linkages performed within and across RWD sources should use a predefined linkage 
425  methodology34 that is scientifically valid, protects the privacy of individuals whose data will be 
426  used, supports interoperability, and accounts for differences in coding and reporting across 
427  sources. The following considerations should be assessed by the sponsor: 
428   
429    Adequacy of line-level linkages (i.e., that the same individuals are being matched), 
430  including pre-defined rules to check for logical consistency and value ranges to 
431  confirm that data were retrieved accurately from a linked data source; and 
 
33 For the purposes of this guidance, continuum of care refers to the extent of the individuals pertinent health data 
which is captured across settings/environments of care is represented in the RWD source. 
34 See An Overview of Record Linkage Methods - Linking Data for Health Services Research - NCBI Bookshelf, 
available at https://www.ncbi.nlm.nih.gov/books/NBK253312/ 
13 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
432    Application of strategies to correct for redundant data, to resolve any inconsistencies, 
433  and assess the potential for missing data. 
434  Because patients typically visit multiple health care sites, especially in 
435  geographically contiguous areas, the inclusion of de-identified data 
436  from many sites creates the possibility that there will be multiple 
437  records from different health care sites for a single individual. This can 
438  result in overcounts of a particular data measure. Alternatively, if some 
439  site records are not available, this can result in a collection of histories 
440  that reflect only a fraction of the patients total health care history. 
441   
442  (3)  Timeliness  
443  As with traditional clinical studies, the time between data collection and release for research 
444  should be reasonable and the RWD considered for the study should reflect the current clinical 
445  environment (e.g., RWD from before a major change in clinical practice may not be timely). 
446  Sponsors should consider changes in clinical practice and guidelines over time (e.g., criteria for 
447  disease diagnosis, cancer staging), characteristics of a condition (e.g., prevalent strain of a 
448  pathogen) and health status of the population. If data are being collected within the RWD source 
449  during the study time frame, then the sponsors should update the availability of the RWD in a 
450  timely manner and should define the reporting schedule in the regulatory submission for FDA 
451  review.  
452   
453  (4)  Generalizability of data 
454  Once a study question is defined, the specific study sample meeting inclusion and exclusion 
455  criteria should be (1) representative of the population in the RWD source eligible for use of 
456  the device within the specified indication and (2) generalizable to the target population with 
457  the condition of interest. If upon quantitative assessment, the study sample is shown to not 
458  be representative of a subset of the target population, then analyses should be conducted to 
459  evaluate generalizability of the study findings. 
460   
461  B.  Reliability 
462  Reliability includes consideration of accrual, quality, and integrity of RWD. Important reliability 
463  factors that FDA considers in determining whether the RWD are suitable for generating RWE for 
464  regulatory use include the following:  
465   
466  (1)  Data Accrual 
467  To ensure the reliability of the RWD source, data should be collected and processed in a 
468  consistent and methodical manner. The manner of collection may differ for newly developed 
469  RWD sources which are actively collecting data (e.g., data dictionary to provide a common 
470  definitional framework in a registry), using nationally or internationally recognized coding 
471  systems (e.g., International Classification of Diseases, Tenth Revision, Clinical Modification 
472  (ICD-10-CM), Logical Observation Identifiers, Names, and Codes (LOINC), UDI, Current 
14 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
473  Procedural Terminology (CPT) in EHR or Claims) or custom-designed structured data capture 
474  (e.g., data capture within a device), or using unstructured data capture (e.g., narrative portion of 
475  clinical notes). Any of these approaches may be able to demonstrate sufficient reliability to 
476  support regulatory decision-making. Factors FDA will consider in making this determination 
477  include: 
478   
479    Adequacy of information and descriptors about data sources provided in the 
480  regulatory submission for FDA review, which should include information on:  
481    Data types;  
482    Health care settings/environment(s);  
483    Purpose of data collection;  
484    How data were obtained at point of data capture;  
485    How data are accessed by study team and sponsor;  
486    Any data transformations, including any modifications made for privacy 
487  protection;  
488    Full data dictionary or common data capture form, if applicable;  
489    Device information, including types of identifiers (e.g., DI) and indication for 
490  use;  
491    Completeness of fields that would typically be completed for all participants 
492  and needed for most study questions (e.g., age, sex, DI);  
493    Time frame (including common temporal framework for collection of data) 
494  and latency of the data (including the timeliness of data entry, transmission, 
495  and availability);  
496    Version control; and 
497    Key technical and privacy-related information. Sponsors should document 
498  routine migration of data between various sources over time (e.g., indicate the 
499  date and time of data transfers, linkages).  
500    Adequacy of information about data accrual methods and procedures provided in the 
501  regulatory submission for FDA review, which should include information on: 
502    Site collection procedures; 
503    Use of common data capture forms; 
504    Common definitional frameworks; 
505    Data cleaning and cross-referencing procedures; 
506    The sources and technical methods used for data element capture (e.g., chart 
507  abstraction, point of care entry, EHR integration, UDI capture, data records 
508  from the device, and linkage to administrative claims data); and 
509    Methods for data retrieval and processes to minimize missing data extraction, 
510  implausible values, and data quality checks in data captured at the point of 
511  care (e.g., during clinical practice for manual or automated health care data 
512  collection processes) to ensure accuracy and completeness of core data fields.  
513   
15 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
514  (2)  Data Quality and Integrity35 
515  When considering RWD sources for regulatory purposes, sponsors should consider the methods 
516  and systems used to help ensure sufficient data quality, including any data quality assurance 
517  plans and procedures developed for the RWD source itself. Since evaluation of RWD sources 
518  may not always permit specific line-item source verification, important factors for sponsors to 
519  consider include:  
520   
521    Quality control processes; 
522    Regardless of the original purpose for collecting the RWD, procedures for 
523  data collection and quality assurance should be put into place during the data 
524  source design and development stages to optimize the reliability, quality, and 
525  usefulness of the data, as appropriate. These procedures should be described 
526  in the regulatory submission for FDA review. 
527    Where appropriate, processes should include site and data monitoring, data 
528  quality audit programs, and evaluation of ongoing training programs for data 
529  collection.  
530    Records regarding the assessment of adherence to the RWD sources 
531  established data quality assurance and quality control policies and procedures 
532  should be retained. 
533    Assessment of completeness, accuracy, and consistency across sites and over time; 
534    Data should be captured in a manner designed to minimize missingness. 
535  Missingness and out of range values should be assessed for each data element. 
536  The amount of missingness per participant (across data elements) should also 
537  be assessed. The impact of missingness should be considered and thresholds 
538  for unacceptable levels of missingness should be pre-determined. 
539  Additionally, quantitative assessment of the potential bias associated with 
540  high missingness should be performed and included in the interpretation of the 
541  study. 
542    Data should be reflective of the actual patient experience (e.g., interactions 
543  with health care, disease trajectory, outcomes) with the condition of interest. 
544    Consistency of data capture should be used across sites and over time.36 If any 
545  changes are needed (e.g., where diagnostic criteria, definitions, or clinical 
546  practice change over several years), then sponsors should document those 
547  changes and assess their impact on the study question and provide summary 
548  information in the regulatory submission for FDA review. 
549    Auditing rules, methods, and the mitigation strategies used to reduce errors 
550  should be documented.37  
 
35 For more information on registry design and execution to better ensure data quality, sponsors can consult 
published literature such as: AHRQ Registries for Evaluating Patient Outcomes: A Users Guide, 4th Edition, 
Section 1 Chapter 3, Registry Design and Section 3 Chapter 11, Obtaining Data and Quality Assurance, available at 
https://effectivehealthcare.ahrq.gov/sites/default/files/pdf/registries-evaluating-patient-outcomes-4th-edition.pdf 
36 More information on PCORI Conduct of Registry Studies is available at http://www.pcori.org/sites/default/files/
Standards-in-the-Conduct-of-Registry-Studies-for-Patient-Centered-Outcomes-Research1.pdf 
37 For more information on documentation for FDA review, see Section VII. and Appendix A. 
16 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
551    Study sample size should be adequate to address the study question. 
552    If non-extant data are used (e.g., data for a newly marketed device will 
553  be captured in the future using the infrastructure of an existing data 
554  source), the sample size should be determined based upon adequate 
555  statistical power to detect a clinically meaningful difference. 
556    If extant RWD are used, adequate statistical power to detect a 
557  clinically meaningful difference should be determined based on the 
558  available sample size and should account for any sampling of 
559  participants from the data source. 
560    If there is inadequate statistical power based on the available 
561  sample size, sponsors should consider the use of multiple 
562  existing RWD sources to increase sample size. 
563    If the sample size could be expected to increase in the near 
564  future (e.g., device is new to market), sponsors should consider 
565  conducting interim analysis with extant data, monitoring 
566  uptake, and conducting final analysis when sufficient sample 
567  size is available.  
568    Sponsors should account for planned statistical analysis within the 
569  study size calculations (e.g., 1:1 matching of propensity scores in a 
570  study population where 10% of participants receive device would 
571  remove approximately 89% of participants with comparator from the 
572  analysis). 
573    Establishment and adherence to data collection, recording, and source verification 
574  procedures; 
575    As with all clinical evidence generation, data provenance and traceability are 
576  important. Sponsors should plan and document all aspects of data extraction, 
577  aggregation, curation, storage, and availability for research, as described 
578  below. 
579    Sponsors should ensure any automated electronic transmission of data fields 
580  to a repository (e.g., registry or data warehouse) occurs in a consistent and 
581  reproducible fashion. 
582    Adherence to source verification procedures and data collection and recording 
583  procedures should be documented for completeness and consistency.  
584    Data checks and procedures should be prespecified to help address identified 
585  errors (e.g., in coding or interpretation of the source documentation or 
586  transformation). 
587    Sponsors should describe the mitigations used to address audit findings, 
588  including data corrections.  
589    Sponsors should identify the source document(s) and first instance38 of data 
590  available to sponsor. Sponsors should generate data quality documentation 
 
38 For the purposes of this draft guidance, the first instance is considered to be the data as initially available to the 
sponsor. For example, if the raw data from an EHR is available, then the first instance is at the point of capture for 
the data element. However, if the sponsor only has access to curated data or a specific dataset, then the first 
instance is the initially ingested data by the sponsor, even though it is not the original data collected. 
17 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
591  from the first instance through RWD dataset(s) used to address the study 
592  question. 
593    If using a common data model, sponsors should ensure documentation of the 
594  transformation of data from the original source to the common data model is 
595  retained. 
596    Data audit trail, including assessment of discrepancies, should be included. 
597  For extant data sources, the sponsor may have access to the 
598  initial  capture  of  data  (e.g.,  direct  access  to  EHR  data  as 
599  obtained via data entry) or may only have access to a partially 
600  curated  data  source  (e.g.,  administrative  claims  data, 
601  aggregated  EHR).  We  recommend  sponsors  maintain 
602  information on the data audit trail from the first instance of data 
603  available to the sponsor through all aspects of data analysis. 
604  Further,  sponsors  should  obtain  as  much  information  as 
605  possible about the audit trail from the data holder if the first 
606  instance is not at the point of data entry. 
607    When the RWD source is not owned by the sponsor, the sponsor should 
608  attempt to obtain participant-level data for each participant. If not available, 
609  the sponsor should define the entity(ies) which do have access/permission for 
610  data entry, quality assurance, storage, aggregation or other linkage, and 
611  assessment of traceability from data entry to dataset, as applicable. Sponsors 
612  should consider the level of access which could be shared with FDA and the 
613  potential for third parties to provide participant-level data directly to FDA. 
614  The availability of data should be described in the regulatory submission for 
615  FDA review. 
616    As with all clinical evidence generation, FDA recommends that the sponsor 
617  have access to the RWD source from the first instance and to the RWD dataset 
618  used for the analyses throughout the regulatory decision-making process. FDA 
619  recognizes that some data sources will not allow sponsors to access the 
620  participant-level data. Although we do not discourage use of these data 
621  sources, FDA notes that uncertainty may arise if the sponsor does not have 
622  access to all of the necessary data.  
623    Adequate patient protections (e.g., methods to protect the privacy of individuals 
624  health data and adherence to applicable privacy and ethics standards) established in 
625  advance of executing the study protocol; and  
626    Prior demonstration of RWE generation from the data source. 
627    Sponsors should provide documentation (including fit-for-purpose 
628  assessment) of any previous use of the same RWD source for a similar target 
629  population and peer-reviewed literature of RWE generation from the data 
630  source. 
631   
632  VI.  Considerations for Methodologies for Collection and 
Analysis of RWD to Generate RWE 
633 
18 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
634  A study using relevant and reliable RWD in a well-designed and rigorously analyzed manner 
635  may be less burdensome than a traditional clinical study. Just as traditional clinical studies 
636  should be carefully designed, studies using RWD also should undergo careful assessment before 
637  embarking on the study or during the analysis to assure that the data are fit-for-purpose. FDA 
638  recognizes that some regulatory decisions may not be adequately supported using RWE, for 
639  various reasons, and we therefore recommend that sponsors consider the methodologies 
640  described below to address factors that can impact interpretability of a study using RWD.  
641   
642  Scientifically sound clinical study planning in advance of statistically valid analyses is important 
643  regardless of whether a study uses a traditional clinical study approach, uses only RWD, or 
644  incorporates a hybrid design. FDA recommends that any study be informed by the needs of the 
645  study question and regulatory decision driving the evidence generation. Further, just like for 
646  traditional clinical studies and in addition to the study design and analysis considerations 
647  described in Sections VI.A. and VI.B., FDA recommends that a sponsor document their 
648  decisions and the associated rationale for the following:  
649   
650    Whether to include randomization, concurrent, or historical controls;  
651    The choice of performance goals and objective performance criteria;  
652    Type I and type II error control;  
653    Data gathering or dependence on extant data;  
654    Bias mitigation strategies;  
655    Precision of outcome measures and other data elements, as applicable; and  
656    All other known factors pertinent to interpretation of the study results. 
657   
658  Although many of the considerations in this section for data collection and analysis are not novel 
659  in the context of clinical evidence generation, there may be unique aspects of these 
660  considerations for studies using RWD. Additionally, the information presented in this section is 
661  intended to augment, not replace, information in other FDA guidances on the design of clinical 
662  studies for regulatory decision-making. The information in this section is intended to clarify 
663  implementation of these concepts and practices when using RWD. In particular, the information 
664  below is intended to complement information in the FDA guidance, Design Considerations for 
665  Pivotal Clinical Investigations for Medical Devices.39 
666   
667  A.  Methods for study designs using RWD 
668  Generally, FDA does not endorse a specific type of study design for clinical studies, regardless 
669  of whether it is a traditional clinical study or uses RWD. As with all clinical evidence generation, 
670  choosing the appropriate design for studies using RWD depends on the study question, device, 
671  outcome, key covariates, and the specific study objectives or hypotheses. Additionally, sponsors 
672  should consider the regulatory purpose of the generated clinical evidence. FDA recognizes that 
673  multiple types of study designs may also be useful to generate RWE. These study designs may 
674  include:  
 
39 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/design-considerations-
pivotal-clinical-investigations-medical-devices 
19 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
675   
676    Single-arm studies with comparisons to external controls, in whole or part;  
677    Objective performance criteria or performance goals; 
678    Non-interventional studies (observational studies) (e.g., comparative cohort studies, 
679  case-control studies, self-controlled studies, and descriptive studies); and  
680    Randomized controlled trials using RWD to supplement one or more study arms. 
681   
682  Furthermore, FDA recognizes the utility of RWD in assessing device utilization, participant 
683  characteristics, natural history of disease or disease trajectory, treatment environment and 
684  treatment patterns, as well as background rates of outcomes.40  
685   
686  B.  Defining study design elements 
687  For studies using RWD, as with all clinical studies, after determining the overarching study 
688  design, the study time frame and collection of data elements should be defined, followed by a 
689  system to capture specific data elements (e.g., in a case report form). Additional data capture 
690  requirements may necessitate justification or adjudication, especially for study endpoints. FDA 
691  recommends clearly defining the individual data elements derived from the RWD source to 
692  develop study-specific RWD. Similarly, FDA recommends that sponsors show that the data 
693  elements, as defined and applied within the study design, are relevant, reliable, and fit for the 
694  regulatory purpose. For analysis of RWD and interpretation of RWE, sponsors should have a 
695  study design that describes the study time frame, the pre-defined set of data elements, and a 
696  systematic consideration that the proposed data elements are all necessary for inclusion and 
697  represent all the key data elements.  
698   
699  (1)  Study time, relative to index date 
700  In traditional clinical studies, a participant is often enrolled into the study, has a baseline visit, 
701  and then first uses the device or has a procedure on the index date (see Figure 1). After that, 
702  the participant is usually followed for a period of time until a final visit. Data elements are 
703  collected at each visit, although different information may be gathered at each visit, and 
704  additional data elements may be collected outside of clinical care (e.g., via a participant diary or 
705  wearable). The participant continues to be followed through a last study visit. If a visit or other 
706  data collection is missed, then the participant may be contacted, or additional questions may be 
707  asked at the next visit to gather key information.  
708   
 
40 Such uses of RWD, while beyond the scope of this draft guidance, may be valuable for describing the clinical 
context in which the device will be used, to support a diversity action plan. See section 520(g)(9) of the FD&C Act 
for more information on diversity action plans for medical devices.  
20 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
709  Figure 1. Traditional clinical studies - study time frame relative to index date 
710   
711   
712  In clinical studies using RWD where data are collected after the study is designed, it is possible 
713  that a similar visit structure and data collection will be available (e.g., within a registry). 
714  However, follow-up visits may not occur on a set schedule or more patient-generated data may 
715  be collected. For extant data such as EHR or administrative claims, baseline and follow-up data 
716  are not collected on a set schedule; rather data collection coincides with clinical care over a 
717  period of time (see Figure 2). Participants may also enter or exit the source database as their life 
718  situation changes (e.g., move out of a geographic area or a change in health insurance). Thus, 
719  continuity of care is an embedded part of the study. 
720   
721  A visual depiction, such as that exemplified in Figure 2, may be helpful in identifying the timing 
722  of collection for each data element relative to the index date, which will help to identify potential 
723  bias. Data elements that may impact the initial device use should be collected before or at the 
724  time of initiation. Outcomes of device use occur after use of the device is initiated. Additionally, 
725  the index date for the use of the comparator to the device would occur at a similar point in the 
726  progression of disease. As with a traditional clinical study, discussion of these ideas with the 
727  study team or with FDA may be aided by the visual depiction of when each data element will be 
728  assessed. 
729   
730  Figure 2. Clinical study using RWD - study time frame relative to index date 
731   
732   
733  Follow-up in a study using RWD typically extends from the index date of device use until either 
734  the end of the pre-planned follow-up time or the last time identified within the RWD source. 
735  FDA considers the study end date to be the last date that participant follow-up could occur. This 
736  date is set on a day when data checks/audits can assure that the underlying data are of sufficient 
737  quality for use in research. Any data in the RWD source indicating that a participant had 
21 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
738  subsequent care is no longer included in the study (i.e., study participation is censored on this 
739  date). Thus, similar to the first site being ready for enrollment in traditional clinical studies, FDA 
740  expects that the study time frame will be defined to begin on the earliest date that the first data 
741  element could be collected and extend through the latest date that the last data element could be 
742  collected.  
743   
744  In addition to the study design elements discussed above, any change in the standard of care, 
745  availability of the device or other treatments, or other relevant factors (e.g., change in hospital 
746  care due to a public health emergency) should be included on the graphical depiction. This 
747  additional information may aid in systematically capturing these time-dependent data elements 
748  and provide support for their inclusion as covariates in analyses or consideration of sensitivity 
749  analysis (e.g., assessing whether a change in ICD-CM coding from a prior edition or a major 
750  change in clinical practice affects study results). 
751   
752  The calendar time allotted for the study should be long enough to adequately measure all data 
753  elements in a study  from the beginning of the baseline period through the end of the follow-up 
754  needed to assess the outcome(s) of interest  in a sample of participants large enough to provide 
755  adequate statistical power to detect the minimal clinically important difference in the primary 
756  outcome.  
757   
758  (2)  Development of conceptual and operational definitions for 
759  the study population, device, comparator, outcome, and 
760  covariates 
761  As with any clinical study, all data elements should be defined before the start of a study using 
762  RWD and should address the specific study question when valid and appropriate analytical 
763  methods are applied (i.e., the data are amenable to sound clinical and statistical analysis). A 
764  conceptual definition that describes the construct or feature of each data element in general or 
765  quantitative terms should be generated using a shorthand name or notation. This conceptual 
766  definition should reflect the current medical and scientific thinking regarding the variable of 
767  interest, such as: (1) clinical criteria to define a condition for population selection or as an 
768  outcome of interest or a covariate; or (2) measurement of the device or procedure to define an 
769  exposure of interest. For example, a conceptual definition might be acute myocardial infarction 
770  (AMI) or AMI evidenced by increased troponin. For a traditional clinical study, the sponsor 
771  defines the collection and timing of each data element, whether at a visit or between visits, and 
772  usually has the ability to contact the participant to limit missing data or to solicit additional 
773  information if a visit was missed. In a study using RWD, the data elements may be collected in a 
774  similar fashion (e.g., registry) or need to be defined from clinical care visits (e.g., EHR or 
775  administrative claims data) or some other algorithm (e.g., combining unstructured EHR and 
776  patient-reported data).  
777   
778  An operational definition describing all of the components needed to identify complete and 
779  accurate data elements from the data source should also be generated. While an operational 
780  definition would typically be generated in a case report form in a traditional clinical study, 
781  operational definitions in a study using RWD frequently include combining structured codes or 
22 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
782  unstructured notes (e.g., clinical notes) in an algorithm to identify presence of the data element. 
783  FDA considers the operational definition to include three components, as applicable: 
784   
785    Time frame over which assessment occurs; 
786    Specific codes/component(s) assessed (e.g., via code lists); and 
787    Algorithm for combining the components (leading to positive identification or lack of 
788  identification). If machine learning is used to define criteria, sponsors should provide 
789  a full description of data management practices including the specifications of the 
790  model/algorithm (e.g., training, tuning, and testing), data collection and the data 
791  attributable to the proposed intended use population (e.g., with respect to race, 
792  ethnicity, disease severity, sex, gender, age, socioeconomic characteristics), as well as 
793  verification and validation information to indicate that the machine learning 
794  approaches are fit-for-purpose for defining criteria. 
795   
796  The availability of different data types in studies using RWD may make it possible to establish 
797  operational definitions that are different from those typically used in traditional clinical studies. 
798  These definitions may or may not be appropriate in the context of the study question being 
799  addressed depending on the study question and regulatory purpose. It is important to consider 
800  whether the operational definition will capture the intended concept for each data element and 
801  FDA notes that small differences in the choice of operational definition in a specific data source 
802  (e.g., requiring two diagnoses rather than one diagnosis of AMI in the example above) may have 
803  a large impact on study results (e.g., considerably decrease the identification of the disease or 
804  condition under study). FDA considers minimizing misclassification to be a critical part of the 
805  process of defining an operational definition. FDA recommends reviewing previous studies using 
806  the RWD source, including published literature, and gathering expert opinion when developing 
807  operational definitions. For some data elements, a rationale for the operational definition based 
808  on previous studies or expert opinion may be sufficient. Some data elements may warrant more 
809  scrutiny to ensure that the interpretation of study results is not substantially impacted by their 
810  misclassification or missingness.  
811   
812  In some cases, it may be appropriate to conduct a validation study in which quantitative 
813  measurements of the operational definition are compared to a ground truth reference standard. 
814  This may result in updating the operational definition to ensure that these critical data elements 
815  are accurately identified. When conducting a validation study, a protocol should be developed 
816  before initiating the data collection and analysis specific to the validation. The protocol generally 
817  contains the plan to compare the operational definition in the RWD (e.g., administrative claims 
818  data) with the ground truth in the reference standard (e.g., validating that administrative billing 
819  diagnosis accurately represents a point-of-care diagnosis by comparing an operational definition 
820  in administrative claims against an EHR) and prespecification of the acceptance criterion for 
821  each validation measure (e.g., sensitivity, specificity, or positive and negative predictive values) 
822  which is of interest. 
823   
824  As with traditional clinical studies, in choosing existing operational definitions or developing 
825  new ones, sponsors should maximize identification of those who have the condition and to 
826  minimize incorrectly identifying those without the condition as having it (i.e., minimize 
23 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
827  misclassification). Sponsors can check for misclassification, for example, by generating a table 
828  of the proportions of participants within the study who are at each level of each data element and 
829  performing a qualitative comparison to what is known from previous literature or expert opinion 
830  about that data element in the target population. Further exploration is recommended for data 
831  elements that are not aligned with expectation. This same exercise may help identify any under-
832  recording or missingness of data elements within the study. 
833   
834  In some RWD sources, the data elements that would be preferred for traditional clinical studies 
835  may not be available to the sponsor. However, a proxy for this missing information could be 
836  developed based on the information that is collected in the RWD source. Proxies can be 
837  developed for a wide range of uses, including identifying study participants (i.e., applying 
838  inclusion/exclusion criteria) as well as certain study endpoints. It may be possible to use the 
839  proxy, but sponsors should determine the suitability of a proxy by considering whether the proxy 
840  is clinically relevant and may call for additional data gathering or conducting validation of the 
841  proposed operational definition for the data element, if using the proxy adds too much 
842  uncertainty to the study interpretation. FDA encourages use of measures that participants or 
843  practicing clinicians deem meaningful as potential data elements for studies using RWD. 
844  Development of endpoints or potential consideration of proxy outcomes may be warranted to 
845  address some study questions. Additionally, development of proxies for key covariates may also 
846  be appropriate to address some study questions. 
847   
848  (3)  Appropriate integration of data elements within study 
849  design and analysis  
850  As with all clinical evidence, data elements in a study using RWD should be determined before 
851  conduct of the analysis and integrated into the study design and analysis in a manner which 
852  allows for assessment of the study question. Once the device and outcome are determined per the 
853  study question, variables which affect both the device and outcome (i.e., confounders) should be 
854  addressed within the study design and analysis to minimize bias and uncertainty. Differing levels 
855  of influence may exist and both direct and indirect influences on the device or outcome may exist 
856  for a confounder. Thus, only a subset of the confounders initially identified should be included in 
857  the study. Conversely, variables that are impacted by the device and subsequently impact the 
858  outcome (sometimes called mediators or intermediate variables) should be carefully 
859  considered before inclusion in evaluations of how the device impacts the outcome. Depending on 
860  the type of analysis, inclusion of such variables may dilute the totality of the device impact. As 
861  with all clinical studies, subgroup analyses for sex, age, racial or ethnic groups are expected 
862  because there may be differential effects of the outcome for participants across these 
863  subgroups.41 Other variables may also exhibit heterogeneity in risk of the outcome (i.e., 
864  modifiers or interaction terms) and stratified analyses for these variables may also be 
865  appropriate.   
866   
 
41 See FDAs guidances, including, Evaluation of Sex-Specific Data in Medical Device Clinical Studies; Collection 
of Race and Ethnicity Data in Clinical Trials, available at https://www.fda.gov/regulatory-information/search-fda-
guidance-documents/collection-race-and-ethnicity-data-clinical-trials; and Evaluation and Reporting of Age-, Race-, 
and Ethnicity-Specific Data in Medical Device Clinical Studies 
24 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
867  One way to identify which data elements fall into each of these types of variables is to generate 
868  and analyze causality diagrams. Causality diagrams (e.g., directed acyclic graphs, see Figure 3) 
869  and their subsequent assessment may provide a rationale for the design and analysis choices. 
870  Additionally, causality diagrams may provide a resource to aid discussions for a study design 
871  amongst the study team or with FDA. Covariates affecting and affected by both the exposure and 
872  outcome are noted within the causality diagram, irrespective of availability within RWD, and are 
873  assessed for potential relationships between the variables.  
874   
875  Figure 3. Directed acyclic graph to identify potential data elements and assess which are 
876  key for the study question 
877   
878   
879  VII. Documentation for FDA Review  
880  This section describes the documentation recommended to support the use of the RWD for 
881  generating RWE for regulatory purposes and applies to device regulatory submissions submitted 
882  to CDRH and CBER, including but not limited to, pre-submissions, 510(k)s, PMAs, BLAs, 
883  HDEs, De Novos, IDEs, post-approval study PMA supplements, 522 submissions, CLIA Waiver 
884  by Applications, and Duals. 
885   
886  A.  Regulatory Submission Cover Letters 
887  FDA recommends sponsors identify RWD and RWE as part of the regulatory submission cover 
888  letter to help facilitate review and internal tracking. Specifically, FDA recommends sponsors 
889  include the following elements in the cover letter for each submission that includes RWD: 
890   
891    Purpose of using RWE to support the submission (see list of examples in Section 
892  IV.A.); 
893    Description of where the RWE fits into the totality of clinical evidence submitted 
894  (e.g., to support interpretability of the primary evidence, to establish a performance 
895  goal, to supplement clinical evidence) (see list of examples in Section IV.A.); 
896    Study design (i.e., type of study) using RWD to generate RWE (see list of examples 
897  in Section VI.A.); 
25 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
898    RWD source(s) used to generate RWE (see list of examples in Section IV.A.); and 
899    Specific RWD source(s) and version, including the following information, if 
900  applicable:  
901    Data source name;  
902    Data source provider; 
903    Version number; and 
904    Date of extraction and date range of data extracted. 
905   
906  B.  Fit-For-Purpose Assessment 
907  If sponsors include RWE in support of regulatory submissions, they should include their fit-for-
908  purpose assessment of the relevance and reliability of the RWD to generate RWE with the 
909  following elements:  
910   
911    An assessment of the key relevance and reliability factors for the study using RWD 
912  (see Section V.), which may include, but is not limited to the following:  
913    Data availability, linkages, timeliness (see Section V.A.); 
914    Data accrual, quality and integrity (see Section V.B.); 
915    Study purpose, specific data elements, generalizability of data, assessment of 
916  confounding, timing of data availability (see Sections V.A. and VI.B.); and 
917    Completeness and accuracy of study sample reflecting the target population, 
918  study design and planning (see Section V.B.2.). 
919  In addition to the fit-for-purpose assessment of the RWD, we recommend that the sponsor 
920  provide the following contextual information regarding how the generated RWE fits into the 
921  totality of evidence: 
922   
923    A description of how the RWE is being used in the totality of clinical evidence 
924  submitted to FDA; 
925    A summary of how the totality of the relevance and reliability of the RWD is fit-for-
926  purpose to address the study question; and 
927    If unique considerations for the specific RWD source exist, sponsors should describe 
928  these considerations and how they impact the overall assessment of the data. 
929   
930  C.  Protocol 
931  As with traditional clinical studies, sponsors should submit the protocol as part of the regulatory 
932  submission to FDA. In studies designed to test a hypothesis, FDA recommends that sponsors 
933  finalize the protocol and analysis plan prior to reviewing the outcome data of a study and before 
934  performing the prespecified analyses. Sponsors should indicate in the regulatory submission 
935  whether or not the protocol and analysis plan were finalized prior to the analyses. In addition, 
936  individuals generating summary scores (e.g., propensity score modeling) should not have access 
26 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
937  to the outcomes within the dataset(s) used for the study. Any revisions to the protocol should be 
938  dated and time-stamped, and the rationale for each change should be provided.  
939   
940  Similar to protocols submitted with traditional clinical studies, sponsors should consider 
941  providing the following information in the protocol for the study generating the RWE, when the 
942  RWD or RWE is included in the regulatory submission: 
943   
944    Study synopsis; 
945    Background and study purpose; 
946    Explanation of how the source data is or is not representative of the general 
947  disease/population with the condition, including sufficient previous research 
948  to interpret study results within the context of the target population, disease 
949  trajectory, and current clinical care; and 
950    Description of the device included in the study, including the DI portion of the 
951  UDI, if available. For devices excepted from the UDI requirements, sponsors 
952  should include the version(s) of the device. 
953    Study aims and objectives; 
954    Study design, including study period; 
955    Study design diagrams are suggested to clarify (1) potential study entry dates within 
956  study period and (2) assessment of all other data elements in relation to cohort entry 
957  or index date (causality diagram recommended); 
958    Data source, including a description of how the setting/environment(s) of data capture 
959  provided adequate continuity of care (see Section V.B.); 
960    Identification of any common data model structure used for housing the RWD 
961  source or for transformed study-specific RWD, if applicable. 
962    Data elements (conceptual and operational definitions for all), including:  
963    Determination of initiation, continuance, and discontinuation of device 
964  exposure, if applicable (see Section V.B.); 
965    Study population, including inclusion and exclusion criteria; 
966    Device and comparator, if included in the study; 
967    Outcome and endpoints; and  
968    Covariates. 
969    Statistical/data analysis plan; 
970    Data management and quality control plans (see Section V.B.); 
971    Sample size and statistical power; 
972    Description of human subject protections, as appropriate, including informed consent, 
973  IRB determination, deidentification plan (e.g., to remove participant identifiers from 
974  patient-generated data or device-generated data), and data confidentiality plans; 
975    Plans for adverse event reporting;42 
 
42 See 21 CFR part 803. 
27 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
976    Milestones and timeline; 
977    Auditing and monitoring plans, as applicable; 
978    If validation or adjudication of data element(s) were conducted (see Section VI.B.), 
979  sponsors should include the study plan and results for validation or adjudication; and 
980    A copy of the data dictionary used, if one was used or developed. 
981   
982  D.  Report 
983  As with traditional clinical studies, sponsors should submit the report as part of the regulatory 
984  submission to FDA. The report should include the occurrence and rationale for any protocol 
985  deviations.  
986   
987  Additionally, in a regulatory submission with a report of a study using RWD, the study results, 
988  discussion, and conclusion should be included, including how the RWE supports the purpose of 
989  the submission. Specifically, sponsors should include the following information: 
990   
991    Date of data extraction (see Section V.B.); 
992    All elements from the protocol, updated to reflect how the study was conducted; and 
993    Justification that any changes or modifications to the protocol did not affect the 
994  validity of the resulting RWE. 
995   
996  E.  Additional Information 
997  As with traditional clinical studies, sponsors should also provide the following information in 
998  regulatory submissions that include RWE:  
999   
1000    ClinicalTrials.gov National Clinical Trial (NCT) Number,43 if applicable;  
1001    Informed consent and IRB documentation, as applicable; 
1002    Initial and continuing IRB review and approval; and 
1003    Initial and approved changes to informed consent.  
1004    List of investigational sites, if any, including mailing address, contact information, 
1005  and investigator name; and 
1006    Case Report Form templates, as developed by the sponsor, if applicable (e.g., these 
1007  templates may be helpful if EHR or claims data are not mapped to the dataset from 
1008  the source). 
 
1009   
 
43 ClinicalTrials.gov assigns a unique NCT number to each clinical study registered on their webpage. See 
https://clinicaltrials.gov/ for more information.  
28 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Appendix A. Recommended Elements for Documentation 
1010 
and FDA Review 
1011 
1012  The following is an example of the recommended elements to assist sponsors and FDA in 
1013  determining relevance and reliability of the RWD. The tables below summarize the 
1014  recommended elements identified throughout the guidance for sponsors to document and provide 
1015  in FDA submissions, and the recommended locations for where to include this information. The 
1016  tables are not intended to serve as either a mandatory or exclusive checklist. Rather, the tables 
1017  provide a simplified summary of the key elements that sponsors should use to assess relevance 
1018  and reliability.  
1019   
1020  Table 1- Recommended RWD Relevance Elements for Submission of RWE 
1021   
Item (Linked to Section V.)  Information  Information  Recommended 
for Sponsors  for Sponsors  Location in FDA 
to Document  to Provide to  Submission  
(e.g., to make  FDA in 
available for  Submission 
inspection) 
Determine RWD source contains sufficient  x (detailed)  x (rationale)  Protocol 
detail to capture data elements and address  (rationale for 
the study question  study question 
and data element 
definitions) 
 
Assess longitudinality of data source  x  Protocol  
 
Assess continuity of care in data source  x  Protocol and 
report 
 
Ensure reasonable time between data  x  Protocol and 
collection and release for research  report 
Consider changes in clinical  x  x  Protocol 
practice/guidelines over time 
 
Assess timing of availability of any new  x  Protocol and 
(i.e., updated) data after initial data  report 
availability 
 
Assess whether and how data from different  x   
sources can be obtained and integrated,  P  rotocol 
given the potential for heterogeneity in 
population characteristics, clinical 
practices, and coding across data sources 
If done, use of a predefined linkage  x (detailed)  x (high level)  Protocol 
methodology that is scientifically valid and 
accounts for differences in coding and 
reporting across sources 
29 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Item (Linked to Section V.)  Information  Information  Recommended 
for Sponsors  for Sponsors  Location in FDA 
to Document  to Provide to  Submission  
(e.g., to make  FDA in 
available for  Submission 
inspection) 
 
Assess adequacy of line-level linkages   x  Report 
   
Correct for redundant data, to resolve any  x 
inconsistencies, and assess the potential for 
missing data 
   
Demonstrate interoperability of the linked  x 
data systems 
Ensure study sample is representative and    x  Protocol and 
generalizable to RWD source and target  report 
population 
1022   
1023  Table 2- Recommended RWD Reliability Elements for Submission of RWE 
1024   
Item (Linked to Section V.)  Information  Information  Recommended 
for Sponsors  for Sponsors  Location in FDA 
to Document  to Provide to  Submission  
(e.g., to make  FDA in 
available for  Submission 
inspection) 
 
Establish information and descriptors about  x  Protocol 
data source(s) 
   
If applicable, describe defined processes,  x 
site training, support, qualified personnel 
for complete and accurate data collection 
   
Document routine migration of data from  x 
various sources over time 
   
Describe sources and technical methods  x 
used for data element capture  
   
Describe methods for data retrieval and  x 
processes to minimize missing data 
extraction, implausible values, and data 
quality checks 
 
If not data holder, describe level of access,  x  Protocol 
attempt to gain patient-level data, and 
consider access for FDA 
Describe the quality of the data captured   x (detailed)  x (high level)  Report 
30 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Item (Linked to Section V.)  Information  Information  Recommended 
for Sponsors  for Sponsors  Location in FDA 
to Document  to Provide to  Submission  
(e.g., to make  FDA in 
available for  Submission 
inspection) 
   
Plan and document the process of  x 
extraction, aggregation, curation, storage, 
and availability of data for research 
   
Describe data flow from first instance to  x 
data source instance as housed by sponsor 
   
Define and follow procedures for data  x 
collection and quality assurance 
 
Provide assessment of completeness,  x  Report 
accuracy, and consistency across sites and 
over time 
Assess consistency of data capture across  x (detailed)  x (high level)  Report 
sites and over time; if any changes are 
needed (e.g., diagnostic criteria or clinical 
definitions change in the course of clinical 
practice over several years), then document 
those changes and assess their impact on 
the study results 
Assess missingness and out of range values    x  Protocol and 
for each data element  report 
Ensure data elements captured and included    x  Protocol and 
in the study are reflective of the actual  report 
patient experience (e.g., interactions with 
health care, disease trajectory, outcomes) 
with the condition of interest 
   
Define the auditing rules and methods used  x 
and the mitigation strategies used to reduce 
errors  
Ensure study size is adequate to address the    x  Protocol and 
study question with adequate statistical  report 
power and accounting for planned analyses 
   
Document adherence to source verification  x 
procedures and data collection and 
recording procedures for completeness and 
consistency 
   
Prespecify data checks and procedures to  x 
help address identified errors  
31 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Item (Linked to Section V.)  Information  Information  Recommended 
for Sponsors  for Sponsors  Location in FDA 
to Document  to Provide to  Submission  
(e.g., to make  FDA in 
available for  Submission 
inspection) 
   
Describe mitigations to address audit  x 
findings, including data corrections  
   
Securely store data and ensure appropriate  x 
permissions/agreements for access 
 
Provide documentation of any previous  x  Protocol 
RWD source fit-for-purpose assessment for 
a similar target population and all peer 
reviewed literature of RWE generation 
from data source 
 
Ensure adequate patient protections (e.g.,  x  Protocol and 
methods to protect the privacy of  report 
individuals health data and adherence to 
applicable privacy and ethics standards) 
established in advance of executing the 
study protocol 
1025     
32 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
Appendix B. Examples Where RWE is Used 
1026 
1027  Most of the following examples are generalized from actual uses of RWD in support of FDA 
1028  regulatory decision-making. These examples do not represent a comprehensive list of all 
1029  potential uses or sources of RWD but do describe some situations where RWE might be used to 
1030  support regulatory decision-making. For additional examples of RWE used in regulatory 
1031  decisions, see the following FDA document: Examples of Real-World Evidence (RWE) Used in 
1032  Medical Device Regulatory Decisions.44 
1033   
1034  Example 1: New or Expanded Indications for Use 
1035  An implanted device, which was available outside the US (OUS), used RWE as the primary 
1036  clinical evidence to support the original PMA submitted to FDA. RWD from an OUS registry in 
1037  one country, which included nearly 300 patients with more than two years of follow-up, was 
1038  compared against a performance goal (PG) for device effectiveness. The PG was derived from a 
1039  prospectively defined, systematic meta-analysis of available published literature and registry data 
1040  for a control device legally marketed in the United States. The study prospectively evaluated the 
1041  functional outcome and patient satisfaction for multiple devices. The safety assessment was 
1042  based on a comparison of the serious device related adverse event rates for the subject device to 
1043  the rates extracted from the registry and the same literature studies used to derive the PG. These 
1044  analyses served as the primary basis supporting approval of the PMA. In this example, an IDE 
1045  was not needed for the study as the clinical data for the subject device was obtained from extant 
1046  data in an OUS registry.  
1047   
1048  Should a manufacturer wish to expand indications, this type of RWD might be used to generate 
1049  sufficient evidentiary support, depending on the specific devices, indications, and analyses. 
1050   
1051  Another example is an implanted device that was available on the US market for several years 
1052  for one indication for use. The sponsor submitted a PMA panel-track supplement to support 
1053  expanding indications for this device. Supplemental clinical evidence for safety of the indication 
1054  expansion was provided from the sponsors patient database, with more than 36,000 patients, 
1055  linked with Medicare claims. The RWE characterized the 12-month safety profile of patients 
1056  with the specific condition (i.e., new indication) implanted with the device and assessed the 
1057  safety among patients receiving the implant and diagnosed with the specific condition compared 
1058  to patients implanted with the device that did not have the respective condition. The sponsors 
1059  patient database and Medicare claims were linked using probabilistic methods (i.e., based on date 
1060  of birth, implant date, implant clinic). A systematic review of literature was also conducted to 
1061  identify diagnostic codes associated with safety outcomes. In this example, an IDE was not 
1062  needed for the study as the clinical data for the subject device was obtained from extant data. 
1063   
1064  Example 2: 522 Submissions  
 
44 Available at: https://www.fda.gov/media/146258/download 
33 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
1065  The manufacturer of a class II designated software-only device was subject to a postmarket 
1066  surveillance order for this device under section 52245 of the FD&C Act, as it met the statutory 
1067  criteria that its failure would be reasonably likely to have serious adverse health consequences, 
1068  and it also was expected to have significant use in pediatric populations. 
1069   
1070  Following discussions with the FDA, the postmarket surveillance study for this software-only 
1071  device was a single-arm, prospective study designed to assess the 12-month safety in a real-
1072  world setting and to support the continued assessment of the software for its intended use. It 
1073  included both pediatric and adult populations. The outcomes were evaluated versus a comparator 
1074  group, which was determined using evidence from a systematic literature review of similar use in 
1075  the intended use population. Device-related adverse events were recorded electronically from 
1076  both inbound reports from customers and assessed through outreach to patients. The safety 
1077  endpoints included the risk for adverse events as extracted in the literature review. Besides the 
1078  safety and effectiveness endpoints, the study also included a secondary endpoint of patient-
1079  reported satisfaction with and trust in the software-only device. At the end of the study, the 
1080  device had met its endpoints, and there were no unanticipated adverse device effects. 
1081   
1082  Example 3: Control group  
1083  A sponsor submitted a PMA panel track supplement to support an indication expansion 
1084  supported by a single-arm study compared to a control group of patients enrolled in a US registry 
1085  receiving alternative devices for the new disease condition. The alternative devices were 
1086  identified through the use of UDIs. To account for potential differences between the device and 
1087  control groups, the sponsor performed a propensity-score adjusted analysis using 20 pre-
1088  specified variables. The propensity score results were reviewed by FDA before the sponsor 
1089  performed the outcome analysis. Sensitivity analyses were conducted to assess the impact of 
1090  missing data. The registry did not routinely collect all relevant data for the control group, 
1091  therefore, additional information for patients in the registry was collected from extant EHRs. The 
1092  results of these analyses demonstrated that the success criteria were achieved. 
1093   
1094  Along with analyses of serious adverse events from the sponsors global study, this RWE was 
1095  the primary basis for supporting approval of the PMA supplement.  
1096   
1097  Example 4: Supplementary Data  
1098  A sponsor submitted a De Novo request for a device. Although the sponsor conducted a 
1099  prospective, repeated measures, single arm study as the primary clinical evidence to support 
1100  safety and effectiveness for their De Novo request, RWD from a chart review at a single site was 
1101  used as supplemental evidence. The overall objective of the study using RWD was to confirm the 
1102  performance of the subject device as part of the standard of care and to further investigate trends 
1103  in treatment outcomes for patients with different severities of the disease. The RWE generated 
1104  from this data provided clinical practice results for a patient population that was not included in 
 
45 FDA has issued a series of postmarket surveillance orders, related to investigating patient safety issues for a type 
of class II device, under the authority of section 522 of the FD&C Act. These 522 orders cover multiple devices 
from different manufacturers that are similar in intended use, design, and other characteristics, such that the 
surveillance questions are identical. 
34 
  Contains Nonbinding Recommendations 
 
Draft  Not for Implementation 
 
1105  one of the clinical studies and demonstrated consistent outcomes with the prospective single arm 
1106  study.  
1107   
1108  Example 5: RWE Obtained from Use of EUA device  
1109  In response to the COVID-19 outbreak, FDA authorized the emergency use of certain devices 
1110  under section 564 of the FD&C Act.46 A sponsor of a serology test authorized for emergency use 
1111  used RWD from a patient registry to support the expansion of the intended use (i.e., 
1112  asymptomatic testing when authorized only for symptomatic testing) in a subsequent Dual 
1113  510(k) and CLIA Waiver by Application. The RWD from the patient registry supplemented data 
1114  from a more traditional clinical study and from peer-reviewed literature. Following Agency 
1115  review, FDA considered the RWE to be valid scientific evidence; the RWD was fit-for-purpose 
1116  and potential sources of bias were minimized. The RWD was fit-for-purpose as it included 
1117  appropriate data elements that were captured in the data and was appropriately linked to the 
1118  specific assay in question (e.g., coding scheme using LOINC or Systematized Nomenclature of 
1119  Medicine Clinical Terms [SNOMED CT]) in addition to information about the setting in which 
1120  the assay was performed and the operators who performed the test.47 Additionally, it was 
1121  important that the sponsor ensured that the DI, patient identifier (ensuring appropriate patient 
1122  population and rationale for any exclusions (e.g., vaccination status)), relevant demographic and 
1123  clinical information, and their interrelationships were available in a single database or commonly 
1124  available RWD sources. In this example, the sponsor accounted for the use of different 
1125  instruments in generating the results as well as the use of different controls that make it difficult 
1126  to show equivalent performance or understand the comparator used. 
 
46 For more information on FDAs emergency use authorities under section 564 of the FD&C Act, see FDAs 
guidance Emergency Use Authorization of Medical Products and Related Authorities, available at 
https://www.fda.gov/regulatory-information/search-fda-guidance-documents/emergency-use-authorization-medical-
products-and-related-authorities 
47 For more information on the Dual 510(k) and CLIA Waiver by Application pathway, see FDAs guidance, 
Recommendations for Dual 510(k) and CLIA Waiver by Application Studies. 
 
35