Contains Nonbinding Recommendations 

Draft – Not for Implementation

Use of Real-World Evidence to 

Support Regulatory Decision-Making 

for Medical Devices

Draft Guidance for Industry and

Food and Drug Administration Staff

DRAFT GUIDANCE

This draft guidance document is being distributed for comment purposes 

only.

Document issued on December 19, 2023. 

You should submit comments and suggestions regarding this draft document within 60 days of 

publication in the Federal Register of the notice announcing the availability of the draft 

guidance. Submit electronic comments to https://www.regulations.gov. Submit written 

comments to the Dockets Management Staff, Food and Drug Administration, 5630 Fishers Lane, 

Room 1061, (HFA-305), Rockville, MD 20852-1740. Identify all comments with the docket 

number listed in the notice of availability that publishes in the Federal Register. 

For questions about this document regarding CDRH-regulated devices, contact the Office of 

Clinical Evidence and Analysis at CDRHClinicalEvidence@fda.hhs.gov. For questions about 

this document regarding CBER-regulated devices, contact the Office of Communication, 

Outreach, and Development (OCOD) at 1-800-835-4709 or 240-402-8010, or by email at 

ocod@fda.hhs.gov. 

When final, this guidance will supersede “Use of Real-World Evidence to 

Support Regulatory Decision-Making for Medical Devices,” issued August 

2017.

U.S. Department of Health and Human Services

Food and Drug Administration 

Center for Devices and Radiological Health

Center for Biologics Evaluation and Research 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

Preface

Additional Copies

CDRH

Additional copies are available from the Internet. You may also send an email request to CDRHGuidance@fda.hhs.gov to receive a copy of the guidance. Please include the document number 

GUI00500012 and complete title of the guidance in the request. 

CBER 

Additional copies are available from the Center for Biologics Evaluation and Research (CBER), 

Office of Communication, Outreach, and Development (OCOD), 10903 New Hampshire Ave., 

Bldg. 71, Room 3128, Silver Spring, MD 20993-0002, or by calling 1-800-835-4709 or 240-402-

8010, by email, ocod@fda.hhs.gov, or from the Internet at https://www.fda.gov/vaccines-bloodbiologics/guidance-compliance-regulatory-information-biologics/biologics-guidances.
Contains Nonbinding Recommendations 

Draft – Not for Implementation

Table of Contents

I. Introduction ............................................................................................................................. 1

II. Background ............................................................................................................................. 2

III. Scope ....................................................................................................................................... 5

IV. Regulatory Context in Which Use of RWE May be Appropriate .......................................... 6

A. General considerations for the use of RWE ..................................................................... 6

B. Application of Investigational Device Exemption (IDE) Requirements in 21 CFR 812 to 

the Collection of RWD ............................................................................................................... 9

C. Application of RWD from devices authorized for emergency use under section 564 of 

the FD&C Act ........................................................................................................................... 10

V. Assessing Data Relevance and Reliability ............................................................................ 11

A. Relevance ....................................................................................................................... 12

(1) Data availability .......................................................................................................... 12

(2) Linkages...................................................................................................................... 13

(3) Timeliness ................................................................................................................... 14

(4) Generalizability of data .............................................................................................. 14

B. Reliability ....................................................................................................................... 14

(1) Data Accrual ............................................................................................................... 14

(2) Data Quality and Integrity .......................................................................................... 16

VI. Considerations for Methodologies for Collection and Analysis of RWD to Generate RWE18

A. Methods for study designs using RWD.......................................................................... 19

B. Defining study design elements ..................................................................................... 20

(1) Study time, relative to index date ............................................................................... 20

(2) Development of conceptual and operational definitions for the study population, 

device, comparator, outcome, and covariates ....................................................................... 22

(3) Appropriate integration of data elements within study design and analysis .............. 24

VII. Documentation for FDA Review .......................................................................................... 25

A. Regulatory Submission Cover Letters............................................................................ 25

B. Fit-For-Purpose Assessment .......................................................................................... 26

C. Protocol .......................................................................................................................... 26

D. Report ............................................................................................................................. 28

E. Additional Information ................................................................................................... 28
Contains Nonbinding Recommendations 

Draft – Not for Implementation

Appendix A. Recommended Elements for Documentation and FDA Review ............................. 29

Appendix B. Examples Where RWE is Used ............................................................................... 33
Contains Nonbinding Recommendations 

Draft – Not for Implementation

1 

1 Use of Real-World Evidence to 

2 Support Regulatory Decision-Making 

3 for Medical Devices

4 

5 Draft Guidance for Industry and

6 Food and Drug Administration Staff

 7 

8 This draft guidance, when finalized, will represent the current thinking of the Food and Drug 

9 Administration (FDA or Agency) on this topic. It does not establish any rights for any person 

10 and is not binding on FDA or the public. You can use an alternative approach if it satisfies 

11 the requirements of the applicable statutes and regulations. To discuss an alternative 

12 approach, contact the FDA staff or Office responsible for this guidance as listed on the title 

13 page. 

 14 

15 I. Introduction 

16 FDA is issuing this draft guidance to clarify how FDA evaluates real-world data to determine 

17 whether they are of sufficient quality for generating real-world evidence that can be used in FDA 

18 regulatory decision-making for medical devices. This draft guidance also provides expanded 

19 recommendations to sponsors considering using real-world evidence to support a regulatory 

submission for medical devices.

1 20 

21 

22 Real-world data (RWD) are data relating to patient health status and/or the delivery of health 

23 care routinely collected from a variety of sources.

24 

Examples of RWD sources include data derived from electronic health records (EHRs), 25 2 medical

26 claims data, data from product and disease registries, and data gathered from other sources (such 

27 as digital health technologies) that can inform on health status. RWD sources can be used as data 

1 For more information on regulatory submissions for medical devices, see Sections III. and VII. 2 For the purposes of this draft guidance, an “EHR" is an electronic record of health-related information on an 

individual that conforms to nationally recognized/utilized interoperability standards and that can be created, 

managed, and consulted by authorized clinicians and staff across more than one health care organization. Definition 

adapted from The National Alliance for Health Information Technology Report to the Office of the National 

Coordinator for Health Information Technology on Defining Key Health Information Technology Terms April 28, 

2008, available at webarchive.library.unt.edu/eot2008/20080920183033/http:/www.hhs.gov

/healthit/documents/m20080603/10_2_hit_terms.pdf 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

2 

28 collection and analysis infrastructure to support many types of study designs, including, but not 

29 limited to, randomized and non-randomized controlled trials; single-arm studies with or without 

30 comparison to an objective performance criterion, performance goal, or extended control; 

31 observational studies; and hybrid designs which combine elements of multiple study designs. 

32 

33 Real-world evidence (RWE) is the clinical evidence regarding the usage, and potential benefits 

34 or risks, of a medical product derived from analysis of RWD. 

35 

36 This draft guidance includes factors that FDA considers important to demonstrate whether the 

37 RWD are fit-for-purpose for a particular regulatory decision relating to medical devices, as well 

38 as FDA’s recommendations on how FDA intends to assess these factors. When finalized, the 

39 recommendations and considerations in this draft guidance will apply regardless of the RWD 

40 source and encompass processes for conducting studies to generate RWE. A fit-for-purpose 

41 assessment should evaluate both the relevance and reliability of the RWD, discussed in more 

42 detail in Section V. FDA recognizes that there may be other approaches to address the 

43 considerations identified in this document. We encourage sponsors to discuss their approach with 

44 FDA, especially if the approach diverges from the recommendations in this draft guidance, when 

finalized.

3 45 

 46 

47 FDA recognizes and anticipates that the Agency and industry may need up to 60 days to perform 

48 activities to operationalize the recommendations within the final guidance. At this time, the 

49 Agency anticipates that, for regulatory submissions that will be currently pending with FDA after

50 publication of the final guidance, as well as those submissions received within 60 days following 

51 publication of the final guidance, FDA generally would not anticipate that sponsors will be ready 

52 to include the newly recommended information outlined in the final guidance in their

53 submission. FDA, however, would intend to review any such information if submitted at any 

54 time. 

55 

56 In general, FDA’s guidance documents do not establish legally enforceable responsibilities. 

57 Instead, guidances describe the Agency’s current thinking on a topic and should be viewed only 

58 as recommendations, unless specific regulatory or statutory requirements are cited. The use of 

59 the word should in Agency guidances means that something is suggested or recommended, but 

60 not required. 

 61 

62 II. Background 

63 To protect and promote public health, FDA needs to understand and evaluate the available 

64 evidence related to regulated products. For medical devices, common sources of available 

evidence include non-clinical and clinical studies4 65 provided to FDA by a device manufacturer or 

3 See FDA’s guidance, “Requests for Feedback and Meetings for Medical Device Submissions: The Q-Submission 

Program,” available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/requestsfeedback-and-meetings-medical-device-submissions-q-submission-program 4 For the purposes of this draft guidance, we use the term “clinical studies” in this guidance as a broad term to 

capture clinical research regarding the safety or effectiveness of a device, regardless of study design. We use the 

term “traditional clinical studies” to refer to clinical studies that do not utilize RWD. 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

3 

66 sponsor of a device premarket or postmarket submission. FDA recognizes that a wealth of 

67 clinical data in the form of RWD are routinely collected in the course of clinical practice during 

68 the treatment and management of patients. Although these data typically have different quality 

69 controls compared to data collected within a traditional clinical study, under certain 

70 circumstances RWD may be used to generate RWE to help inform or augment FDA’s 

71 understanding of the benefit-risk profile of devices at various points in their life cycle. Per 21 

72 CFR 860.7(c)(1), “[a]lthough [a] manufacturer may submit any form of evidence to the Food and 

73 Drug Administration in an attempt to substantiate the safety and effectiveness of a device, the 

74 agency relies upon only valid scientific evidence to determine whether there is reasonable 

75 assurance that the device is safe and effective.” RWE derived from relevant and reliable RWD 

may constitute valid scientific evidence,5 76 depending on the study question, regulatory decision, 

77 data source(s), and design and analysis of the specific dataset derived from RWD source(s), and 

78 thus may be used to support regulatory decisions. FDA intends to use the considerations 

79 described in this draft guidance, when finalized, to evaluate whether RWD are relevant and 

80 reliable to support regulatory decision-making, including potentially generating valid scientific 

81 evidence. The use of RWE for specific regulatory purposes will include assessment of the overall 

82 relevance and reliability of the RWD used to generate the RWE. 

 83 

84 When appropriate, use of RWD may provide an efficient means of generating the necessary 

85 clinical evidence to support regulatory decisions. Information specific to the clinical performance 

86 of a device can be generated through a number of methodological and operational approaches. In 

87 general, traditional clinical studies tend to be narrow in scope but allow for more control of 

88 sources of error and bias. In comparison, studies that leverage RWD may be able to evaluate

89 broader questions but are subject to sources of bias which may be more difficult to control.

90 Clinical evidence can be generated from studies using RWD, alone or in combination with data 

91 from more traditional clinical studies. Using appropriate design and methodologies, sponsors can 

92 leverage the strengths of these approaches while minimizing potential weaknesses. 

 93 

RWD that includes patient experience data6 94 may provide new insights into the performance of a 

95 device. In addition, RWD may foster inclusion of target populations that are otherwise 

96 underrepresented in clinical studies. Similarly, leveraging RWD may allow for studies of a 

97 longer period of time than would be practical in a traditional clinical study and so may allow for

98 data to be gathered on longer term outcomes. Clinical evidence generated from fit-for-purpose 

5 Under 21 CFR 860.7(c)(2), “valid scientific evidence” is considered “evidence from well-controlled investigations, 

partially controlled studies, studies and objective trials without matched controls, well-documented case histories 

conducted by qualified experts, and reports of significant human experience with a marketed device, from which it 

can fairly and responsibly be concluded by qualified experts that there is reasonable assurance of the safety and 

effectiveness of a device under its conditions of use. The evidence required may vary according to the characteristics 

of the device, its conditions of use, the existence and adequacy of warnings and other restrictions, and the extent of 

experience with its use.”

6 For the purposes of this draft guidance, “patient experience data” means data that are (1) collected by any persons 

(including patients, family members and caregivers of patients, patient advocacy organizations, disease research 

foundations, researchers, and drug manufacturers); and (2) are intended to provide information about patients’ 

experiences with a disease or condition, including (A) the impact of such disease or condition, or a related therapy, 

on patients’ lives; and (B) patient preferences with respect to treatment of such disease or condition. See section 

3001 of the 21st Century Cures Act, Pub. L. No. 114-255 (December 2016).
Contains Nonbinding Recommendations 

Draft – Not for Implementation

4 

99 RWD informs device benefit-risk profiles assessment from a real-world environment, allows 

100 evaluation of outcomes which may not be feasible in traditional clinical studies, and better aligns 

101 with device innovation cycles to inform future device modifications and new technology 

102 development. Finally, RWD may include information from broader clinical experiences than is 

103 usually represented in traditional clinical studies. RWE is an important factor for understanding 

104 and regulating medical devices, and therefore, FDA encourages the medical community to learn 

105 more from routine clinical care to help support safety and effectiveness of medical devices. Use 

106 of relevant and reliable RWD to generate RWE can benefit stakeholders throughout the 

107 ecosystem, including but not limited to, patients, health care providers, manufacturers, and FDA. 

 108 

109 Additionally, in some cases, a traditional clinical study may be impractical or excessively 

110 challenging to conduct. Ethical issues regarding treatment assignment, and other similar 

111 challenges, may present themselves when developing and attempting to execute such a study. 

112 Analyses of RWD, using appropriate methods, may in some cases provide similar information 

113 with comparable or even superior characteristics to information collected and analyzed through a 

114 traditional clinical study. Under the right conditions, RWE may be suitable to support the 

115 marketing authorization of a new device or the expansion of the indications for use of devices 

that are already on the market. Aggregation of RWD (e.g., in a medical device registry7 116 ) may 

117 prove useful as a postmarket control suitable for providing ongoing device safety surveillance 

118 and additional evidence for effectiveness. FDA has long applied postmarket controls as a way to 

119 reduce premarket data collection, while still ensuring that the statutory standard of reasonable 

assurance of safety and effectiveness is met.8 120 FDA believes that applying postmarket controls to 

121 reduce premarket data collection, when appropriate, can help improve patient access to safe and 

effective medical devices.9 122 

123 

124 Many of the considerations and best practices for generating RWE are derived from the same 

125 principles that govern generation of clinical evidence from traditional clinical studies, which are 

126 generally referred to as good clinical practice (GCP). Additionally, as with all clinical evidence 

127 FDA evaluates, FDA’s assessment of RWE evaluated in support of a particular regulatory 

128 decision will be included as part of the totality of information available to FDA. Further, as with 

129 all types of clinical data, FDA recognizes there may be uncertainty of the benefits and risks of a 

130 device that remain after completion of a study using RWD. Some of these aspects are similar to 

131 those also present in more traditional forms of clinical data, but some are unique to RWD. 

7 For the purposes of this draft guidance, “medical device registry” means an organized system with a primary aim 

to increase the knowledge on medical devices contributing to improve the quality of patient care that continuously 

collects relevant data, evaluates meaningful outcomes and comprehensively covers the population defined by 

exposure to particular device(s) at a reasonably generalizable scale (e.g., international, national, regional, and health 

system). Definition is cited from the IMDRF document “Principles of International System of Registries Linked to 

Other Data Sources and Tools,” available at https://www.imdrf.org/documents/principles-international-systemregistries-linked-other-data-sources-and-tools 8 For more information, see FDA’s guidance “The Least Burdensome Provisions: Concept and Principles,” available 

at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/least-burdensome-provisionsconcept-and-principles 9 For more information, see FDA’s guidance “Balancing Premarket and Postmarket Data Collection for Devices 

Subject to Premarket Approval,” available at https://www.fda.gov/regulatory-information/search-fda-guidancedocuments/balancing-premarket-and-postmarket-data-collection-devices-subject-premarket-approval 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

5 

132 Assessment of the relevance and reliability of the RWD, as outlined in this draft guidance, can 

identify uncertainty that should be considered during the benefit-risk determinations10 133 for the 

134 device for a given regulatory purpose. 

135 

136 In 2017, FDA issued the guidance document, Use of Real-World Evidence to Support 

Regulatory Decision-Making for Medical Devices,

11 137 in which we described the relevance and 

138 reliability factors of RWD that FDA assesses to determine if RWD are sufficient for generating 

139 RWE. Subsequently, on December 29, 2022, the Food and Drug Omnibus Reform Act of 2022 

140 (“FDORA”) was signed into law as part of the Consolidated Appropriations Act, 2023, Pub. L. 

141 No. 117-328. Section 3629 of FDORA “Facilitating the Use of Real World Evidence” directs 

142 FDA to issue or revise existing guidance on considerations for the use of RWD and RWE to 

143 support regulatory decision-making. FDA is issuing this draft guidance to propose revisions to 

144 the 2017 guidance, Use of Real-World Evidence to Support Regulatory Decision-Making for 

145 Medical Devices, to satisfy the requirement under section 3629(a)(2). This draft guidance also 

146 fulfills a commitment in Section V.F. of the Medical Device User Fee Amendments Performance 

Goals and Procedures, Fiscal Years 2023 Through 2027 (MDUFA V).

12 147 This draft guidance is 

148 intended to provide expanded and updated recommendations to industry and FDA staff for 

149 conducting an assessment of relevance and reliability to demonstrate that RWD may be fit-for150 purpose to generate clinical evidence for regulatory decision-making. This includes 

151 recommendations to provide clarity on least burdensome general expectations related to 

152 demonstrating that RWD is fit-for-purpose for premarket regulatory purposes. 

 153 

154 III. Scope

155 This draft guidance is applicable for the use of RWE to support regulatory submissions for 

medical devices.

13 156 

 157 

10 See FDA’s guidances, “Factors to Consider When Making Benefit-Risk Determinations in Medical Device 

Premarket Approval and De Novo Classifications” available at https://www.fda.gov/regulatory-information/searchfda-guidance-documents/factors-consider-when-making-benefit-risk-determinations-medical-device-premarketapproval-and-de; “Consideration of Uncertainty in Making Benefit-Risk Determinations in Medical Device 

Premarket Approvals, De Novo Classifications, and Humanitarian Device Exemptions” available at 

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/consideration-uncertainty-makingbenefit-risk-determinations-medical-device-premarket-approvals-de; “Benefit-Risk Factors to Consider When 

Determining Substantial Equivalence in Premarket Notifications (510(k)) with Different Technological 

Characteristics,” available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/benefitrisk-factors-consider-when-determining-substantial-equivalence-premarket-notifications-510k; and “Factors to 

Consider Regarding Benefit-Risk in Medical Device Product Availability, Compliance, and Enforcement 

Decisions,” available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/factorsconsider-regarding-benefit-risk-medical-device-product-availability-compliance-and 11 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/use-real-worldevidence-support-regulatory-decision-making-medical-devices 12 Available at https://www.fda.gov/media/158308/download 13 This guidance does not apply to drugs and biological products. For information on the RWE program for drugs 

and biological products, see https://www.fda.gov/science-research/real-world-evidence/center-biologics-evaluationand-research-center-drug-evaluation-and-research-real-world-evidence 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

6 

158 The topics covered within this draft guidance are framed specifically for the use of RWD and 

159 RWE in regulatory submissions for medical devices (e.g., Investigational Device Exemption 

160 (IDE), premarket notification under section 510(k) of the FD&C Act, Premarket Approval 

161 Application (PMA), Humanitarian Device Exemption (HDE), De Novo classification request, 

162 post-approval study, postmarket surveillance under Section 522 of the FD&C Act (522 

163 submissions), Clinical Laboratory Improvement Amendments (CLIA) Waiver by Applications 

164 (CW), Dual De Novo/510(k) and CLIA Waiver by Application Submissions (Duals)). The 

165 considerations included in this draft guidance may be applicable to supporting uses of RWD 

166 across the medical device total product life cycle (TPLC).

 167 

168 This draft guidance does not address the use of non-clinical data, adverse event reports, 

169 secondary use of clinical study data, or systematic literature reviews. Nor does it address all 

170 possible study design/conduct or analytical methodologies. While it does describe the factors that 

171 FDA considers when evaluating relevance and reliability of RWD, it does not provide a specific 

172 set of criteria or other scoring tools for determining the suitability of any specific RWD source 

173 for generating RWE for a particular regulatory decision. 

174 

175 This guidance, when finalized, should not be construed to alter or change in any way the existing 

176 evidentiary standards applicable to FDA’s regulatory decision-making. Rather, this guidance 

177 describes the circumstances under which clinical evidence generated from RWD may be used to 

178 support a variety of FDA decisions based on the existing evidentiary standards. While FDA 

179 encourages the use of relevant and reliable data to generate clinical evidence, including RWE, 

180 this draft guidance neither mandates use of RWD and RWE nor restricts other means of 

181 providing evidence to support regulatory decision-making. This draft guidance does not affect 

182 any federal, state, or local laws or regulations, or foreign laws or regulations that may be 

183 applicable to the use or collection of RWD, or that provide protections for human subjects 

184 (including informed consent requirements) or patient privacy. When finalized, this guidance 

185 should be used to complement, but not supersede, other device-specific and GCP and guidance 

186 documents. 

 187 

188 IV. Regulatory Context in Which Use of RWE May be 

189 Appropriate 

190 A. General considerations for the use of RWE 

191 In general, FDA considers the use of RWD to be fit-for-purpose to support generation of clinical 

192 evidence for regulatory decision-making for medical devices when we conclude that the RWD 

193 used to generate the RWE are relevant to and reliable for informing or supporting a particular 

194 regulatory decision. It is important to understand the strengths and limitations of the underlying 

195 RWD and how these qualities impact their relevance and reliability. Similarly, the context of the 

196 specific regulatory decision for which the RWE is being proposed is central to FDA’s evaluation. 

 197 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

7 

198 FDA recognizes that RWE can be generated from a variety of RWD sources that are primarily 

intended for another purpose. For example, administrative claims data14 199 are typically collected

200 for purposes of billing or payment for medical care. Disease-specific RWD sources may be 

201 useful for tracking progression or outcomes of specific rare or poorly understood diseases. 

202 Treatment-specific RWD sources may have several purposes, including assessment and tracking 

203 of overall outcomes, providing assessment of hospital operations, informing performance 

204 improvement initiatives, or providing risk prediction and benchmarking data for specific 

205 therapies. The suitability of the RWD source may be determined by the factors outlined in 

206 Section V. and the availability of sufficient data to address the study question of interest. 

 207 

208 FDA does not endorse one type of RWD over another. Sponsors should select the appropriate 

209 RWD sources based on their suitability to address the specific study questions. Data sources that

may be considered RWD sources include the following:15 210 

 211 

• Registries;

16 212 

213 • EHRs; 

214 • Administrative claims data; 

• Patient-generated data17 215 created, reported, or gathered by patients including in-home use 

settings (e.g., data from digital health technologies (DHTs)18 216 such as wearables); 

217 • Device-generated data (e.g., implantable devices, physiological monitoring devices); 

• Public health surveillance data (e.g., COVID-1919 218 case surveillance); 

14 For the purposes of this draft guidance, “administrative claims data” means claims data that arise from a person’s 

use of the health care system and reimbursement of health care providers for that care. Definition adapted from

Strom et al., Textbook of Pharmacoepidemiology 6th ed. 2022, page 137. 15 Generally, FDA does not consider published literature to be RWD. Literature may report data from an RWD 

source, in which case sponsors should specify the RWD source type (e.g., if a journal article presents a retrospective 

analysis of EHRs, the RWD source should be specified as EHR in the cover sheet). 16 For the purposes of this draft guidance, “registry” means an organized system that uses observational study 

methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a 

particular disease, condition, or exposure, and that serves one or more stated scientific, clinical, or policy purposes. 

Definition adapted from the Agency for Healthcare Research and Quality’s (AHRQ’s) “Registries for Evaluating 

Patient Outcomes: A User’s Guide,” available at https://effectivehealthcare.ahrq.gov/products/registries-guide-4thedition/users-guide 17 The use of the term “patient-generated data” is consistent with the use of this term in the “Framework for FDA’s 

Real-World Evidence Program” document, available at https://www.fda.gov/media/120060/download. Patientgenerated data includes patient-generated health data. For the purposes of this draft guidance, “patient-generated 

health data” means health-related data created, recorded, or gathered by or from patients, family members, or other 

caregivers to help address a health concern. Definition adapted from https://www.healthit.gov/topic/scientificinitiatives/pcor/patient-generated-health-data-pghd 18 For the purposes of this draft guidance, “digital health technology” means a system that uses computing platforms, 

connectivity, software, and/or sensors for health care and related uses. These technologies span a wide range of uses, 

from applications in general wellness to applications as a medical device. They include technologies intended for 

use as a medical product, in a medical product, or as an adjunct to other medical products (devices, drugs, and 

biologics). They may also be used to develop or study medical products. 19 In 2019, an outbreak of respiratory disease caused by a novel coronavirus began. The virus has been named 

“SARS-CoV-2,” and the disease it causes has been named “Coronavirus Disease 2019” (COVID-19). On January 

31, 2020, the Secretary of Health and Human Services (HHS) issued a declaration of a Public Health Emergency 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

8 

219 • Clinically annotated biobanks; and

220 • Medical device data repositories (e.g., imaging, electrocardiography databases). 

221 

222 Some purposes for which use of RWD may potentially be applicable in a regulatory submission 

223 include the following: 

224 

225 • To generate hypotheses to be tested in a clinical study;

226 • As a historical control, an informative prior in a Bayesian analysis of a 

clinical trial,20 227 or as one source of data in a hierarchical model or a hybrid 

228 data synthesis; 

229 • As a concurrent control group or as a mechanism for collecting data to support 

230 marketing authorization when a registry, EHR, claims data, or some other systematic 

231 data collection mechanism exists;

232 • As a mechanism for re-training artificial intelligence/machine learning-enabled 

233 medical devices; 

234 • To generate evidence to identify, demonstrate, or support the clinical validity of a 

235 biomarker or clinical outcome assessment; 

236 • To generate (primary) clinical evidence to support marketing authorization (e.g.,

237 HDE, PMA, 510(k) or De Novo request); 

238 • To generate evidence directly by the subject device to provide new information 

239 on safety or effectiveness; 

240 • To generate evidence to support a determination on whether the subject device 

meets the statutory criteria for a CLIA waiver21 (e.g., CW and Duals22 241 ); 

242 • To generate evidence to support the interpretability of the primary clinical 

243 evidence (e.g., to demonstrate that the study population for an investigation 

244 conducted outside the United States (OUS) is representative of the US 

245 population, or to provide context for an adverse event observed in the clinical 

246 study); 

247 • To generate evidence to support a petition for reclassification of a medical device

248 under section 513(e) or (f)(3) of the FD&C Act;

(PHE) related to COVID-19 in accordance with section 319 of the Public Health Service Act (PHS Act) (hereinafter 

referred to as “section 319 PHE declaration”) and mobilized the Operating Divisions of HHS. In addition, on March 

13, 2020, the President declared a national emergency in response to COVID-19. The section 319 PHE declaration 

related to COVID-19 expired on May 11, 2023. 20 For more information on Bayesian trails, see FDA’s guidance, “Guidance for the Use of Bayesian Statistics in 

Medical Device Clinical Trials,” available at https://www.fda.gov/regulatory-information/search-fda-guidancedocuments/guidance-use-bayesian-statistics-medical-device-clinical-trials-pdf-version 21 For more information on CLIA waiver by applications, see FDA’s guidance, “Recommendations for Clinical 

Laboratory Improvement Amendments of 1988 (CLIA) Waiver Applications for Manufacturers of In 

Vitro Diagnostic Devices,” available at https://www.fda.gov/regulatory-information/search-fda-guidancedocuments/recommendations-clinical-laboratory-improvement-amendments-1988-clia-waiver-applications 22 For more information on the Dual 510(k) and CLIA Waiver by Application pathway, see FDA’s guidance, 

“Recommendations for Dual 510(k) and CLIA Waiver by Application Studies,” available at 

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/recommendations-dual-510k-and-cliawaiver-application-studies. Dual De Novo classification requests and CLIA Waiver by Application for certain 

devices (see section 3301 of FDORA) should also consult this guidance.
Contains Nonbinding Recommendations 

Draft – Not for Implementation

9 

249 • To generate evidence for expanding the labeling of a device to include additional 

250 indications for use or to update the labeling to include new information on safety and

effectiveness;23 251 

252 • To generate evidence for postmarket surveillance. Through ongoing surveillance,

253 signals are at times identified that suggest there may be a safety issue with a medical 

254 device. RWE may be generated using RWD to refine these signals for purposes of 

255 informing appropriate corrective actions and communication;

256 • To conduct post-approval studies that are imposed as a condition of device approval 

257 or to potentially preclude the need for 522 submissions; and

258 • To provide postmarket data in lieu of some premarket data, consistent with FDA’s policy 

on balancing premarket and postmarket data collection.24 259 

 260 

261 B. Application of Investigational Device Exemption (IDE) 

262 Requirements in 21 CFR 812 to the Collection of RWD

263 An approved IDE permits a device to be shipped lawfully for the purpose of conducting 

264 investigations of the device without complying with certain other requirements of the FD&C Act 

265 that would apply to devices in commercial distribution. The purpose of this investigational

266 exemption, per 21 CFR 812.1, “is to encourage, to the extent consistent with the protection of 

267 public health and safety and with ethical standards, the discovery and development of useful 

268 devices intended for human use, and to that end to maintain optimum freedom for scientific 

269 investigators in their pursuit of this purpose.” As explained in 21 CFR Part 812, the IDE 

270 regulations apply to all clinical investigations of devices to determine safety and effectiveness, 

with certain limited exceptions.25 271 In many cases, an approved IDE is required before initiating a 

272 clinical investigation. An investigation is defined as “a clinical investigation or research 

involving one or more subjects to determine the safety or effectiveness of a device.”26 273 

274 

275 Whether the collection of RWD for a legally marketed device requires an IDE depends on the 

276 particular facts of the situation. Specifically, if the device is being used in the normal course of 

277 medical practice, an IDE would likely not be required. FDA recognizes that in clinical practice 

278 this could include use of a legally marketed device for uncleared or unapproved uses, where the 

279 device is being administered or prescribed under the authority of a health care practitioner within 

280 a legitimate practitioner-patient relationship. If data collection does not impact how the device is 

281 administered, and the administration is within the normal course of medical care, an IDE would 

282 likely not be required. For example, analyses of extant RWD (i.e., RWD already collected)

283 involving the use in medical care of a device that was not within the cleared or approved 

284 indications for use would generally not be subject to IDE regulations. However, similar to 

23 See FDA’s guidance, “General/Specific Intended Use,” available at https://www.fda.gov/regulatoryinformation/search-fda-guidance-documents/generalspecific-intended-use-guidance-industry 24 See FDA’s guidance, “Balancing Premarket and Postmarket Data Collection for Devices Subject to Premarket 

Approval,” available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/balancingpremarket-and-postmarket-data-collection-devices-subject-premarket-approval 25 See 21 CFR 812.2(a). 26 See 21 CFR 812.3(h). 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

10 

285 traditional clinical studies, if data are being gathered to determine the safety and effectiveness of 

286 the device, and the process for gathering the data would influence treatment decisions, such 

287 administration would likely not be within the normal course of medical practice and an IDE may 

288 be required. For example, a study using a registry infrastructure designed to determine the safety 

289 and effectiveness of an approved device for a new intended use would likely be subject to IDE 

290 requirements if physicians are instructed to treat specific patients or otherwise administer the 

291 device in a prescribed way for purposes of data generation, or when certain follow-up activities 

292 are performed for the purpose of research. 

293 

294 Should a sponsor or Institutional Review Board (IRB) be unclear regarding the applicability of 

295 the IDE regulations to a particular RWD collection activity or use, the sponsor or IRB should 

296 contact FDA. If an IDE is determined to be required, FDA intends to work with the IDE sponsor 

297 to develop the least burdensome approach to facilitate the efficient generation of RWE. Note that 

298 regardless of the applicability of 21 CFR Part 812, FDA regulations at 21 CFR Part 56 (IRB 

299 review), Part 50 (Protection of Human Subjects) and Part 54 (Financial Disclosure) may apply to 

300 RWE generation activities, as may other federal, state, and local laws regarding human subject 

301 protections. 

 302 

303 C. Application of RWD from devices authorized for 

304 emergency use under section 564 of the FD&C Act

305 Section 564 of the FD&C Act provides that FDA may, after the HHS Secretary has made a 

306 declaration of emergency or threat justifying authorization of emergency use (an “EUA 

declaration”), authorize the emergency use of an unapproved product27 307 or an unapproved use of 

308 an approved product for certain emergency circumstances. 

309 

310 The routine clinical use of a device authorized under an EUA, when used within the scope of its 

311 authorization, is not considered to be a clinical investigation (see section 564(k) of the FD&C 

312 Act and Section IV.B. for more information on the application of IDE requirements in 21 CFR 

313 Part 812 to the collection of RWD). Clinical data routinely collected from the use of a device 

314 authorized under an EUA may be considered RWD and may be used to support regulatory 

315 decision-making, if determined to be fit-for-purpose. Generally, the recommendations in this 

316 draft guidance may apply to RWD from devices authorized under an EUA. Additionally, 

317 Appendix B includes an example of RWD from a device authorized under an EUA that was used 

318 in a subsequent premarket submission. 

 319 

320 Device use pursuant to EUAs may lead to additional sources and novel uses of RWD to support 

321 FDA decision-making. We encourage sponsors to consider the recommendations in this guidance 

322 for devices authorized under an EUA (e.g., devices authorized under an EUA during the 

323 COVID-19 pandemic). 

 324 

27 Under sections 564(a)(2)(A) and 564(a)(4)(D) of the FD&C Act, an unapproved product is one that “is not 

approved, licensed, or cleared for commercial distribution under section 505, 510(k), 512, or 515 of [the FD&C] Act 

or section 351 of the [PHS] Act or conditionally approved under section 571 of [the FD&C] Act.”
Contains Nonbinding Recommendations 

Draft – Not for Implementation

11 

325 V. Assessing Data Relevance and Reliability

326 To determine the potential suitability of RWD to generate RWE for regulatory decision-making, 

327 FDA assesses the relevance and reliability of the RWD source as well as the data elements, study 

328 design, and analytic components of the study. This section describes the elements that might be 

329 evaluated to determine if the data are fit-for-purpose. FDA recognizes that data, including RWD 

330 used to generate RWE, may have limitations. Sponsors should understand the strengths and 

331 limitations of generating evidence from RWD to address a specific study question and provide 

332 these limitations to FDA in their submission. If the RWD source appears relevant and reliable, 

333 then additional assessment of the study-specific derived dataset(s) may help demonstrate the 

334 RWD are fit-for-purpose to address the study question. This assessment may be used to 

335 determine whether the RWD source(s) and the proposed design and analysis can generate 

336 evidence that is sufficiently robust to be used for the given study question and regulatory 

337 purpose, i.e., whether the RWD are fit-for-purpose. 

 338 

339 Whether data are sufficiently relevant and reliable for use will, in part, depend on the particular 

340 regulatory decision. FDA will evaluate the same factors to assess RWD across all data sources 

341 and regulatory decisions but will weigh each factor in accordance with the regulatory decision to 

342 be made. In cases where RWE is derived from multiple RWD sources, each RWD source will be 

343 evaluated individually and together in the aggregate to determine the relevance and reliability of 

344 the RWD. 

345 

346 The data should be accurate, as complete as possible, and of adequate data quality to credibly 

347 address the question at hand. Conducting a clinical investigation in accordance with GCP 

348 provides assurance that the data and results from the clinical investigation are credible and 

accurate and that the rights, safety, and well-being of subjects are protected.28 349 In traditional 

350 clinical studies, the best practices for incorporating GCP into the study design and execution are 

351 generally well established. However, because RWD are typically collected outside of a 

352 controlled research setting, additional precautions should be considered to ensure that the data 

353 are similarly “credible and accurate,” and that appropriate patient protections are in place. In 

354 order to determine whether data are “credible and accurate,” FDA assesses the relevance and 

355 reliability of the data. 

 356 

357 Additionally, sponsors should ensure that RWD were collected using good data management 

358 practices and are sufficiently robust. Sponsors should also consider data related to various 

demographic characteristics (e.g., age, sex, race, and ethnicity)29 359 and other potentially relevant 

28 See, for example, 21 CFR 812.28(a)(1), which defines good clinical practice in the context of device

investigations conducted outside the United States. 29 For example, see the following FDA guidances: Guidance for the Use of Bayesian Statistics in Medical Device 

Clinical Trials; “Evaluation of Sex-Specific Data in Medical Device Clinical Studies,” available at 

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/evaluation-sex-specific-data-medicaldevice-clinical-studies-guidance-industry-and-food-and-drug; and “Evaluation and Reporting of Age-, Race-, and 

Ethnicity-Specific Data in Medical Device Clinical Studies,” available at https://www.fda.gov/regulatoryinformation/search-fda-guidance-documents/evaluation-and-reporting-age-race-and-ethnicity-specific-data-medicaldevice-clinical-studies
Contains Nonbinding Recommendations 

Draft – Not for Implementation

12 

360 covariates, and whether the data are representative of the intended use population. The relevance 

361 and reliability factors listed below should be described to assess the RWD. 

 362 

363 Studies using RWD should also be carefully designed to mitigate potential bias, and a study 

364 protocol and analysis plan should be created prior to analyzing RWD, regardless of whether the 

365 RWD are extant or if they are to be collected in the future. An existing RWD source may have 

366 some inherent sources of bias that could limit the relevance or reliability for drawing causal 

inferences between medical device exposures30 367 and outcomes. 

368 

369 To help ensure the relevance and reliability of the source data, FDA recommends sponsors 

370 consider the factors contained in this section. If considered, these factors should be referenced 

371 during study conduct, FDA inspection, or provided as additional information during FDA review 

372 of the applicable regulatory submission, as applicable. Appendix A sets forth the elements that 

373 FDA recommends sponsors document and have available for inspection, as well as 

374 recommended elements sponsors to include in the appropriate regulatory submission for FDA 

375 review. 

376 

377 A. Relevance 

378 Relevance includes consideration of availability, timeliness, and generalizability of the RWD. 

379 When needed information is not available in one data source, sponsors may want to provide 

380 linkage of other data source(s). Important relevance factors that FDA will consider in 

381 determining whether RWD are suitable for generating RWE for regulatory use include the 

382 following: 

383 

384 (1) Data availability

385 The RWD should contain sufficient detail to capture the information needed to evaluate the 

386 question being addressed in the target population. Relevant considerations should include 

387 whether the RWD contains information on the following: 

 388 

• Use of the device (e.g., the device identifier (DI)31 389 portion of the unique device 

identifier (UDI),

32 390 other structured data, clinical notes) or other exposure in the study 

391 population; 

392 • Outcome(s) of interest in the study population; 

30 The exposure is the variable or data element whose causal effect is estimated in a study. For many studies 

supporting a regulatory submission, the exposure is the medical device. For some studies, especially those providing 

supportive clinical evidence, another exposure may be assessed to provide context for the use, safety, or 

effectiveness of the medical device within clinical practice. 31 The device identifier is a mandatory, fixed portion of a UDI that identifies the specific version or model of a 

device and the labeler of that device. 21 CFR 830.3. For more information regarding UDI, see FDA’s webpage 

https://www.fda.gov/medical-devices/unique-device-identification-system-udi-system/udi-rule-guidances-trainingand-other-resources 32 For class I devices, the universal product code (UPC) may serve as the UDI (21 CFR 801.40(d)). In these 

instances, the UPC should be included. 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

13 

393 • Covariates that may impact the exposure or outcomes of interest (e.g., RWD source 

394 contains signs, symptoms, treatments, procedures, diagnoses, patient and family 

395 history, pre-existing conditions, labs, demographics, and results which may be used to 

396 construct covariates that are relevant to the study question); and

397 For example, the minimum set of data fields in a registry may be 

398 insufficient for a specific study question and additional data fields may 

399 be needed for the registry to be fit-for-purpose. The registry should 

400 retain information documenting the start or stop of collection during 

401 the study time frame for data fields related to the specific study 

402 question. 

403 • Longitudinality, including longevity (the length of time that data for an individual is 

404 captured within the RWD source) and continuity of care. 

Information across the continuum of care33 405 (i.e., data observability) 

406 may aid the assessment of the likelihood that all exposures and 

407 outcomes of interest will be captured for regulatory decision-making. 

408 For example, tertiary care hospitalization data may not have adequate 

409 data availability to study outcomes that are likely to be diagnosed in an 

410 emergency for all patients, because patients are likely to go to a nearby

411 hospital in emergencies but may travel to another location for a 

412 specialty device procedure. 

 413 

414 If the RWD source is insufficient on its own, the sponsor should determine whether 

415 supplemental data sources are available and sufficient to provide any missing information 

416 necessary to address the study question. 

 417 

418 (2) Linkages

419 Sponsors should assess whether and how data from different sources can be obtained and 

420 integrated given the potential for heterogeneity in target population characteristics, clinical 

421 practices, and coding across data sources. A description of this assessment should be provided in 

422 the regulatory submission for FDA review. 

 423 

424 Any linkages performed within and across RWD sources should use a predefined linkage 

methodology34 425 that is scientifically valid, protects the privacy of individuals whose data will be 

426 used, supports interoperability, and accounts for differences in coding and reporting across 

427 sources. The following considerations should be assessed by the sponsor: 

 428 

429 • Adequacy of line-level linkages (i.e., that the same individuals are being matched), 

430 including pre-defined rules to check for logical consistency and value ranges to 

431 confirm that data were retrieved accurately from a linked data source; and

33 For the purposes of this guidance, “continuum of care” refers to the extent of the individual’s pertinent health data 

which is captured across settings/environments of care is represented in the RWD source. 34 See An Overview of Record Linkage Methods - Linking Data for Health Services Research - NCBI Bookshelf, 

available at https://www.ncbi.nlm.nih.gov/books/NBK253312/
Contains Nonbinding Recommendations 

Draft – Not for Implementation

14 

432 • Application of strategies to correct for redundant data, to resolve any inconsistencies, 

433 and assess the potential for missing data. 

434 Because patients typically visit multiple health care sites, especially in 

435 geographically contiguous areas, the inclusion of de-identified data 

436 from many sites creates the possibility that there will be multiple 

437 records from different health care sites for a single individual. This can 

438 result in overcounts of a particular data measure. Alternatively, if some 

439 site records are not available, this can result in a collection of histories 

440 that reflect only a fraction of the patient’s total health care history.

441 

442 (3) Timeliness 

443 As with traditional clinical studies, the time between data collection and release for research 

444 should be reasonable and the RWD considered for the study should reflect the current clinical 

445 environment (e.g., RWD from before a major change in clinical practice may not be timely). 

446 Sponsors should consider changes in clinical practice and guidelines over time (e.g., criteria for 

447 disease diagnosis, cancer staging), characteristics of a condition (e.g., prevalent strain of a 

448 pathogen) and health status of the population. If data are being collected within the RWD source 

449 during the study time frame, then the sponsors should update the availability of the RWD in a 

450 timely manner and should define the reporting schedule in the regulatory submission for FDA 

451 review. 

 452 

453 (4) Generalizability of data

454 Once a study question is defined, the specific study sample meeting inclusion and exclusion 

455 criteria should be (1) representative of the population in the RWD source eligible for use of 

456 the device within the specified indication and (2) generalizable to the target population with 

457 the condition of interest. If upon quantitative assessment, the study sample is shown to not 

458 be representative of a subset of the target population, then analyses should be conducted to 

459 evaluate generalizability of the study findings. 

460 

461 B. Reliability 

462 Reliability includes consideration of accrual, quality, and integrity of RWD. Important reliability 

463 factors that FDA considers in determining whether the RWD are suitable for generating RWE for 

464 regulatory use include the following: 

465 

466 (1) Data Accrual

467 To ensure the reliability of the RWD source, data should be collected and processed in a 

468 consistent and methodical manner. The manner of collection may differ for newly developed 

469 RWD sources which are actively collecting data (e.g., data dictionary to provide a common 

470 definitional framework in a registry), using nationally or internationally recognized coding 

471 systems (e.g., International Classification of Diseases, Tenth Revision, Clinical Modification 

472 (ICD-10-CM), Logical Observation Identifiers, Names, and Codes (LOINC), UDI, Current 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

15 

473 Procedural Terminology (CPT) in EHR or Claims) or custom-designed structured data capture 

474 (e.g., data capture within a device), or using unstructured data capture (e.g., narrative portion of 

475 clinical notes). Any of these approaches may be able to demonstrate sufficient reliability to

476 support regulatory decision-making. Factors FDA will consider in making this determination 

477 include: 

478 

479 • Adequacy of information and descriptors about data sources provided in the 

480 regulatory submission for FDA review, which should include information on: 

481 • Data types; 

482 • Health care settings/environment(s); 

483 • Purpose of data collection; 

484 • How data were obtained at point of data capture; 

485 • How data are accessed by study team and sponsor; 

486 • Any data transformations, including any modifications made for privacy 

487 protection; 

488 • Full data dictionary or common data capture form, if applicable; 

489 • Device information, including types of identifiers (e.g., DI) and indication for 

490 use; 

491 • Completeness of fields that would typically be completed for all participants

492 and needed for most study questions (e.g., age, sex, DI); 

493 • Time frame (including common temporal framework for collection of data) 

494 and latency of the data (including the timeliness of data entry, transmission, 

495 and availability); 

496 • Version control; and

497 • Key technical and privacy-related information. Sponsors should document 

498 routine migration of data between various sources over time (e.g., indicate the 

499 date and time of data transfers, linkages). 

500 • Adequacy of information about data accrual methods and procedures provided in the 

501 regulatory submission for FDA review, which should include information on: 

502 • Site collection procedures; 

503 • Use of common data capture forms; 

504 • Common definitional frameworks; 

505 • Data cleaning and cross-referencing procedures;

506 • The sources and technical methods used for data element capture (e.g., chart 

507 abstraction, point of care entry, EHR integration, UDI capture, data records 

508 from the device, and linkage to administrative claims data); and

509 • Methods for data retrieval and processes to minimize missing data extraction, 

510 implausible values, and data quality checks in data captured at the point of 

511 care (e.g., during clinical practice for manual or automated health care data 

512 collection processes) to ensure accuracy and completeness of core data fields. 

 513 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

16 

(2) Data Quality and Integrity35 514 

515 When considering RWD sources for regulatory purposes, sponsors should consider the methods 

516 and systems used to help ensure sufficient data quality, including any data quality assurance 

517 plans and procedures developed for the RWD source itself. Since evaluation of RWD sources 

518 may not always permit specific line-item source verification, important factors for sponsors to 

519 consider include: 

 520 

521 • Quality control processes; 

522 • Regardless of the original purpose for collecting the RWD, procedures for 

523 data collection and quality assurance should be put into place during the data 

524 source design and development stages to optimize the reliability, quality, and 

525 usefulness of the data, as appropriate. These procedures should be described 

526 in the regulatory submission for FDA review. 

527 • Where appropriate, processes should include site and data monitoring, data 

528 quality audit programs, and evaluation of ongoing training programs for data 

529 collection. 

530 • Records regarding the assessment of adherence to the RWD source’s 

531 established data quality assurance and quality control policies and procedures 

532 should be retained. 

533 • Assessment of completeness, accuracy, and consistency across sites and over time; 

534 • Data should be captured in a manner designed to minimize missingness. 

535 Missingness and out of range values should be assessed for each data element. 

536 The amount of missingness per participant (across data elements) should also 

537 be assessed. The impact of missingness should be considered and thresholds 

538 for unacceptable levels of missingness should be pre-determined.

539 Additionally, quantitative assessment of the potential bias associated with 

540 high missingness should be performed and included in the interpretation of the 

541 study. 

542 • Data should be reflective of the actual patient experience (e.g., interactions 

543 with health care, disease trajectory, outcomes) with the condition of interest. 

• Consistency of data capture should be used across sites and over time.

36 544 If any 

545 changes are needed (e.g., where diagnostic criteria, definitions, or clinical 

546 practice change over several years), then sponsors should document those 

547 changes and assess their impact on the study question and provide summary 

548 information in the regulatory submission for FDA review.

549 • Auditing rules, methods, and the mitigation strategies used to reduce errors 

should be documented.37 550 

35 For more information on registry design and execution to better ensure data quality, sponsors can consult 

published literature such as: “AHRQ Registries for Evaluating Patient Outcomes: A User’s Guide,” 4th Edition, 

Section 1 Chapter 3, Registry Design and Section 3 Chapter 11, Obtaining Data and Quality Assurance, available at 

https://effectivehealthcare.ahrq.gov/sites/default/files/pdf/registries-evaluating-patient-outcomes-4th-edition.pdf 36 More information on PCORI Conduct of Registry Studies is available at http://www.pcori.org/sites/default/files/

Standards-in-the-Conduct-of-Registry-Studies-for-Patient-Centered-Outcomes-Research1.pdf 37 For more information on documentation for FDA review, see Section VII. and Appendix A.
Contains Nonbinding Recommendations 

Draft – Not for Implementation

17 

551 • Study sample size should be adequate to address the study question. 

552 • If non-extant data are used (e.g., data for a newly marketed device will 

553 be captured in the future using the infrastructure of an existing data 

554 source), the sample size should be determined based upon adequate 

555 statistical power to detect a clinically meaningful difference. 

556 • If extant RWD are used, adequate statistical power to detect a 

557 clinically meaningful difference should be determined based on the 

558 available sample size and should account for any sampling of 

559 participants from the data source. 

560  If there is inadequate statistical power based on the available 

561 sample size, sponsors should consider the use of multiple 

562 existing RWD sources to increase sample size. 

563  If the sample size could be expected to increase in the near 

564 future (e.g., device is new to market), sponsors should consider 

565 conducting “interim” analysis with extant data, monitoring 

566 uptake, and conducting final analysis when sufficient sample 

567 size is available. 

568 • Sponsors should account for planned statistical analysis within the 

569 study size calculations (e.g., 1:1 matching of propensity scores in a 

570 study population where 10% of participants receive device would 

571 remove approximately 89% of participants with comparator from the 

572 analysis). 

573 • Establishment and adherence to data collection, recording, and source verification 

574 procedures; 

575 • As with all clinical evidence generation, data provenance and traceability are

576 important. Sponsors should plan and document all aspects of data extraction, 

577 aggregation, curation, storage, and availability for research, as described 

578 below. 

579 • Sponsors should ensure any automated electronic transmission of data fields 

580 to a repository (e.g., registry or data warehouse) occurs in a consistent and 

581 reproducible fashion. 

582 • Adherence to source verification procedures and data collection and recording 

583 procedures should be documented for completeness and consistency. 

584 • Data checks and procedures should be prespecified to help address identified 

585 errors (e.g., in coding or interpretation of the source documentation or 

586 transformation).

587 • Sponsors should describe the mitigations used to address audit findings, 

588 including data corrections. 

• Sponsors should identify the source document(s) and first instance38 589 of data 

590 available to sponsor. Sponsors should generate data quality documentation 

38 For the purposes of this draft guidance, the “first instance” is considered to be the data as initially available to the 

sponsor. For example, if the raw data from an EHR is available, then the first instance is at the point of capture for 

the data element. However, if the sponsor only has access to curated data or a specific dataset, then the “first 

instance” is the initially ingested data by the sponsor, even though it is not the original data collected.
Contains Nonbinding Recommendations 

Draft – Not for Implementation

18 

591 from the first instance through RWD dataset(s) used to address the study 

592 question. 

593 • If using a common data model, sponsors should ensure documentation of the 

594 transformation of data from the original source to the common data model is 

595 retained. 

596 • Data audit trail, including assessment of discrepancies, should be included. 

597 For extant data sources, the sponsor may have access to the 

598 initial capture of data (e.g., direct access to EHR data as 

599 obtained via data entry) or may only have access to a partially 

600 curated data source (e.g., administrative claims data, 

601 aggregated EHR). We recommend sponsors maintain 

602 information on the data audit trail from the first instance of data 

603 available to the sponsor through all aspects of data analysis. 

604 Further, sponsors should obtain as much information as 

605 possible about the audit trail from the data holder if the first 

606 instance is not at the point of data entry. 

607 • When the RWD source is not owned by the sponsor, the sponsor should 

608 attempt to obtain participant-level data for each participant. If not available, 

609 the sponsor should define the entity(ies) which do have access/permission for 

610 data entry, quality assurance, storage, aggregation or other linkage, and 

611 assessment of traceability from data entry to dataset, as applicable. Sponsors 

612 should consider the level of access which could be shared with FDA and the 

613 potential for third parties to provide participant-level data directly to FDA.

614 The availability of data should be described in the regulatory submission for 

615 FDA review.

616 • As with all clinical evidence generation, FDA recommends that the sponsor 

617 have access to the RWD source from the first instance and to the RWD dataset

618 used for the analyses throughout the regulatory decision-making process. FDA 

619 recognizes that some data sources will not allow sponsors to access the 

620 participant-level data. Although we do not discourage use of these data 

621 sources, FDA notes that uncertainty may arise if the sponsor does not have 

622 access to all of the necessary data. 

623 • Adequate patient protections (e.g., methods to protect the privacy of individuals’ 

624 health data and adherence to applicable privacy and ethics standards) established in 

625 advance of executing the study protocol; and 

626 • Prior demonstration of RWE generation from the data source. 

627 • Sponsors should provide documentation (including fit-for-purpose 

628 assessment) of any previous use of the same RWD source for a similar target 

629 population and peer-reviewed literature of RWE generation from the data 

630 source. 

 631 

632 VI. Considerations for Methodologies for Collection and 

633 Analysis of RWD to Generate RWE
Contains Nonbinding Recommendations 

Draft – Not for Implementation

19 

634 A study using relevant and reliable RWD in a well-designed and rigorously analyzed manner 

635 may be less burdensome than a traditional clinical study. Just as traditional clinical studies 

636 should be carefully designed, studies using RWD also should undergo careful assessment before 

637 embarking on the study or during the analysis to assure that the data are fit-for-purpose. FDA 

638 recognizes that some regulatory decisions may not be adequately supported using RWE, for 

639 various reasons, and we therefore recommend that sponsors consider the methodologies 

640 described below to address factors that can impact interpretability of a study using RWD. 

641 

642 Scientifically sound clinical study planning in advance of statistically valid analyses is important 

643 regardless of whether a study uses a traditional clinical study approach, uses only RWD, or 

644 incorporates a hybrid design. FDA recommends that any study be informed by the needs of the 

645 study question and regulatory decision driving the evidence generation. Further, just like for 

646 traditional clinical studies and in addition to the study design and analysis considerations 

647 described in Sections VI.A. and VI.B., FDA recommends that a sponsor document their 

648 decisions and the associated rationale for the following: 

 649 

650 • Whether to include randomization, concurrent, or historical controls; 

651 • The choice of performance goals and objective performance criteria; 

652 • Type I and type II error control; 

653 • Data gathering or dependence on extant data; 

654 • Bias mitigation strategies; 

655 • Precision of outcome measures and other data elements, as applicable; and 

656 • All other known factors pertinent to interpretation of the study results. 

657 

658 Although many of the considerations in this section for data collection and analysis are not novel 

659 in the context of clinical evidence generation, there may be unique aspects of these 

660 considerations for studies using RWD. Additionally, the information presented in this section is 

661 intended to augment, not replace, information in other FDA guidances on the design of clinical 

662 studies for regulatory decision-making. The information in this section is intended to clarify 

663 implementation of these concepts and practices when using RWD. In particular, the information 

664 below is intended to complement information in the FDA guidance, “Design Considerations for 

Pivotal Clinical Investigations for Medical Devices.”39 665 

666 

667 A. Methods for study designs using RWD

668 Generally, FDA does not endorse a specific type of study design for clinical studies, regardless 

669 of whether it is a traditional clinical study or uses RWD. As with all clinical evidence generation, 

670 choosing the appropriate design for studies using RWD depends on the study question, device, 

671 outcome, key covariates, and the specific study objectives or hypotheses. Additionally, sponsors 

672 should consider the regulatory purpose of the generated clinical evidence. FDA recognizes that 

673 multiple types of study designs may also be useful to generate RWE. These study designs may

674 include: 

39 Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/design-considerationspivotal-clinical-investigations-medical-devices
Contains Nonbinding Recommendations 

Draft – Not for Implementation

20 

675 

676 • Single-arm studies with comparisons to external controls, in whole or part; 

677 • Objective performance criteria or performance goals; 

678 • Non-interventional studies (observational studies) (e.g., comparative cohort studies, 

679 case-control studies, self-controlled studies, and descriptive studies); and 

680 • Randomized controlled trials using RWD to supplement one or more study arms. 

681 

682 Furthermore, FDA recognizes the utility of RWD in assessing device utilization, participant 

683 characteristics, natural history of disease or disease trajectory, treatment environment and

treatment patterns, as well as background rates of outcomes.40 684 

685 

686 B. Defining study design elements

687 For studies using RWD, as with all clinical studies, after determining the overarching study 

688 design, the study time frame and collection of data elements should be defined, followed by a 

689 system to capture specific data elements (e.g., in a case report form). Additional data capture 

690 requirements may necessitate justification or adjudication, especially for study endpoints. FDA 

691 recommends clearly defining the individual data elements derived from the RWD source to 

692 develop study-specific RWD. Similarly, FDA recommends that sponsors show that the data 

693 elements, as defined and applied within the study design, are relevant, reliable, and fit for the 

694 regulatory purpose. For analysis of RWD and interpretation of RWE, sponsors should have a 

695 study design that describes the study time frame, the pre-defined set of data elements, and a 

696 systematic consideration that the proposed data elements are all necessary for inclusion and 

697 represent all the key data elements. 

 698 

699 (1) Study time, relative to index date 

700 In traditional clinical studies, a participant is often enrolled into the study, has a baseline visit, 

701 and then first uses the device or has a procedure on the “index date” (see Figure 1). After that, 

702 the participant is usually followed for a period of time until a final visit. Data elements are 

703 collected at each visit, although different information may be gathered at each visit, and 

704 additional data elements may be collected outside of clinical care (e.g., via a participant diary or 

705 wearable). The participant continues to be followed through a last study visit. If a visit or other 

706 data collection is missed, then the participant may be contacted, or additional questions may be 

707 asked at the next visit to gather key information. 

708 

40 Such uses of RWD, while beyond the scope of this draft guidance, may be valuable for describing the clinical 

context in which the device will be used, to support a diversity action plan. See section 520(g)(9) of the FD&C Act 

for more information on diversity action plans for medical devices. 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

21 

709 Figure 1. Traditional clinical studies - study time frame relative to index date

710 

711 

712 In clinical studies using RWD where data are collected after the study is designed, it is possible 

713 that a similar visit structure and data collection will be available (e.g., within a registry).

714 However, follow-up visits may not occur on a set schedule or more patient-generated data may

715 be collected. For extant data such as EHR or administrative claims, baseline and follow-up data 

716 are not collected on a set schedule; rather data collection coincides with clinical care over a 

717 period of time (see Figure 2). Participants may also enter or exit the source database as their life 

718 situation changes (e.g., move out of a geographic area or a change in health insurance). Thus, 

719 continuity of care is an embedded part of the study. 

720 

721 A visual depiction, such as that exemplified in Figure 2, may be helpful in identifying the timing 

722 of collection for each data element relative to the index date, which will help to identify potential 

723 bias. Data elements that may impact the initial device use should be collected before or at the 

724 time of initiation. Outcomes of device use occur after use of the device is initiated. Additionally, 

725 the index date for the use of the comparator to the device would occur at a similar point in the 

726 progression of disease. As with a traditional clinical study, discussion of these ideas with the 

727 study team or with FDA may be aided by the visual depiction of when each data element will be 

728 assessed.

729 

730 Figure 2. Clinical study using RWD - study time frame relative to index date

731 

 732 

733 Follow-up in a study using RWD typically extends from the index date of device use until either 

734 the end of the pre-planned follow-up time or the last time identified within the RWD source. 

735 FDA considers the study end date to be the last date that participant follow-up could occur. This 

736 date is set on a day when data checks/audits can assure that the underlying data are of sufficient 

737 quality for use in research. Any data in the RWD source indicating that a participant had 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

22 

738 subsequent care is no longer included in the study (i.e., study participation is censored on this 

739 date). Thus, similar to the first site being ready for enrollment in traditional clinical studies, FDA 

740 expects that the study time frame will be defined to begin on the earliest date that the first data 

741 element could be collected and extend through the latest date that the last data element could be 

742 collected. 

743 

744 In addition to the study design elements discussed above, any change in the standard of care, 

745 availability of the device or other treatments, or other relevant factors (e.g., change in hospital 

746 care due to a public health emergency) should be included on the graphical depiction. This 

747 additional information may aid in systematically capturing these time-dependent data elements 

748 and provide support for their inclusion as covariates in analyses or consideration of sensitivity 

749 analysis (e.g., assessing whether a change in ICD-CM coding from a prior edition or a major 

750 change in clinical practice affects study results).

 751 

752 The calendar time allotted for the study should be long enough to adequately measure all data 

753 elements in a study – from the beginning of the baseline period through the end of the follow-up 

754 needed to assess the outcome(s) of interest – in a sample of participants large enough to provide 

755 adequate statistical power to detect the minimal clinically important difference in the primary 

756 outcome. 

757 

758 (2) Development of conceptual and operational definitions for 

759 the study population, device, comparator, outcome, and 

760 covariates

761 As with any clinical study, all data elements should be defined before the start of a study using 

762 RWD and should address the specific study question when valid and appropriate analytical 

763 methods are applied (i.e., the data are amenable to sound clinical and statistical analysis). A 

764 “conceptual definition” that describes the construct or feature of each data element in general or 

765 quantitative terms should be generated using a shorthand name or notation. This conceptual 

766 definition should reflect the current medical and scientific thinking regarding the variable of 

767 interest, such as: (1) clinical criteria to define a condition for population selection or as an 

768 outcome of interest or a covariate; or (2) measurement of the device or procedure to define an 

769 exposure of interest. For example, a conceptual definition might be “acute myocardial infarction 

770 (AMI)” or “AMI evidenced by increased troponin.” For a traditional clinical study, the sponsor 

771 defines the collection and timing of each data element, whether at a visit or between visits, and 

772 usually has the ability to contact the participant to limit missing data or to solicit additional 

773 information if a visit was missed. In a study using RWD, the data elements may be collected in a 

774 similar fashion (e.g., registry) or need to be defined from clinical care visits (e.g., EHR or 

775 administrative claims data) or some other algorithm (e.g., combining unstructured EHR and 

776 patient-reported data). 

777 

778 An “operational definition” describing all of the components needed to identify complete and 

779 accurate data elements from the data source should also be generated. While an operational 

780 definition would typically be generated in a case report form in a traditional clinical study, 

781 operational definitions in a study using RWD frequently include combining structured codes or 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

23 

782 unstructured notes (e.g., clinical notes) in an algorithm to identify presence of the data element. 

783 FDA considers the operational definition to include three components, as applicable: 

784 

785 • Time frame over which assessment occurs; 

786 • Specific codes/component(s) assessed (e.g., via code lists); and

787 • Algorithm for combining the components (leading to positive identification or lack of 

788 identification). If machine learning is used to define criteria, sponsors should provide 

789 a full description of data management practices including the specifications of the 

790 model/algorithm (e.g., training, tuning, and testing), data collection and the data 

791 attributable to the proposed intended use population (e.g., with respect to race, 

792 ethnicity, disease severity, sex, gender, age, socioeconomic characteristics), as well as 

793 verification and validation information to indicate that the machine learning 

794 approaches are fit-for-purpose for defining criteria. 

795 

796 The availability of different data types in studies using RWD may make it possible to establish 

797 operational definitions that are different from those typically used in traditional clinical studies. 

798 These definitions may or may not be appropriate in the context of the study question being 

799 addressed depending on the study question and regulatory purpose. It is important to consider 

800 whether the operational definition will capture the intended concept for each data element and 

801 FDA notes that small differences in the choice of operational definition in a specific data source 

802 (e.g., requiring two diagnoses rather than one diagnosis of AMI in the example above) may have 

803 a large impact on study results (e.g., considerably decrease the identification of the disease or 

804 condition under study). FDA considers minimizing misclassification to be a critical part of the 

805 process of defining an operational definition. FDA recommends reviewing previous studies using 

806 the RWD source, including published literature, and gathering expert opinion when developing 

807 operational definitions. For some data elements, a rationale for the operational definition based 

808 on previous studies or expert opinion may be sufficient. Some data elements may warrant more 

809 scrutiny to ensure that the interpretation of study results is not substantially impacted by their 

810 misclassification or missingness. 

 811 

812 In some cases, it may be appropriate to conduct a validation study in which quantitative 

813 measurements of the operational definition are compared to a “ground truth” reference standard. 

814 This may result in updating the operational definition to ensure that these critical data elements 

815 are accurately identified. When conducting a validation study, a protocol should be developed 

816 before initiating the data collection and analysis specific to the validation. The protocol generally 

817 contains the plan to compare the operational definition in the RWD (e.g., administrative claims

818 data) with the “ground truth” in the reference standard (e.g., validating that administrative billing 

819 diagnosis accurately represents a point-of-care diagnosis by comparing an operational definition 

820 in administrative claims against an EHR) and prespecification of the acceptance criterion for 

821 each validation measure (e.g., sensitivity, specificity, or positive and negative predictive values) 

822 which is of interest. 

 823 

824 As with traditional clinical studies, in choosing existing operational definitions or developing 

825 new ones, sponsors should maximize identification of those who have the condition and to 

826 minimize incorrectly identifying those without the condition as having it (i.e., minimize 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

24 

827 misclassification). Sponsors can check for misclassification, for example, by generating a table 

828 of the proportions of participants within the study who are at each level of each data element and 

829 performing a qualitative comparison to what is known from previous literature or expert opinion 

830 about that data element in the target population. Further exploration is recommended for data 

831 elements that are not aligned with expectation. This same exercise may help identify any under832 recording or missingness of data elements within the study. 

833 

834 In some RWD sources, the data elements that would be preferred for traditional clinical studies 

835 may not be available to the sponsor. However, a proxy for this missing information could be 

836 developed based on the information that is collected in the RWD source. Proxies can be 

837 developed for a wide range of uses, including identifying study participants (i.e., applying 

838 inclusion/exclusion criteria) as well as certain study endpoints. It may be possible to use the 

839 proxy, but sponsors should determine the suitability of a proxy by considering whether the proxy 

840 is clinically relevant and may call for additional data gathering or conducting validation of the 

841 proposed operational definition for the data element, if using the proxy adds too much 

842 uncertainty to the study interpretation. FDA encourages use of measures that participants or 

843 practicing clinicians deem meaningful as potential data elements for studies using RWD. 

844 Development of endpoints or potential consideration of proxy outcomes may be warranted to 

845 address some study questions. Additionally, development of proxies for key covariates may also 

846 be appropriate to address some study questions. 

847 

848 (3) Appropriate integration of data elements within study 

849 design and analysis 

850 As with all clinical evidence, data elements in a study using RWD should be determined before 

851 conduct of the analysis and integrated into the study design and analysis in a manner which 

852 allows for assessment of the study question. Once the device and outcome are determined per the 

853 study question, variables which affect both the device and outcome (i.e., confounders) should be 

854 addressed within the study design and analysis to minimize bias and uncertainty. Differing levels 

855 of influence may exist and both direct and indirect influences on the device or outcome may exist 

856 for a confounder. Thus, only a subset of the confounders initially identified should be included in 

857 the study. Conversely, variables that are impacted by the device and subsequently impact the 

858 outcome (sometimes called “mediators” or “intermediate variables”) should be carefully 

859 considered before inclusion in evaluations of how the device impacts the outcome. Depending on 

860 the type of analysis, inclusion of such variables may dilute the totality of the device impact. As 

861 with all clinical studies, subgroup analyses for sex, age, racial or ethnic groups are expected 

862 because there may be differential effects of the outcome for participants across these 

subgroups.41 863 Other variables may also exhibit heterogeneity in risk of the outcome (i.e., 

864 modifiers or “interaction” terms) and stratified analyses for these variables may also be 

865 appropriate. 

866 

41 See FDA’s guidances, including, Evaluation of Sex-Specific Data in Medical Device Clinical Studies; “Collection 

of Race and Ethnicity Data in Clinical Trials,” available at https://www.fda.gov/regulatory-information/search-fdaguidance-documents/collection-race-and-ethnicity-data-clinical-trials; and Evaluation and Reporting of Age-, Race-, 

and Ethnicity-Specific Data in Medical Device Clinical Studies
Contains Nonbinding Recommendations 

Draft – Not for Implementation

25 

867 One way to identify which data elements fall into each of these types of variables is to generate

868 and analyze causality diagrams. Causality diagrams (e.g., directed acyclic graphs, see Figure 3) 

869 and their subsequent assessment may provide a rationale for the design and analysis choices. 

870 Additionally, causality diagrams may provide a resource to aid discussions for a study design 

871 amongst the study team or with FDA. Covariates affecting and affected by both the exposure and 

872 outcome are noted within the causality diagram, irrespective of availability within RWD, and are 

873 assessed for potential relationships between the variables. 

874 

875 Figure 3. Directed acyclic graph to identify potential data elements and assess which are 

876 key for the study question

877 

 878 

879 VII. Documentation for FDA Review 

880 This section describes the documentation recommended to support the use of the RWD for 

881 generating RWE for regulatory purposes and applies to device regulatory submissions submitted 

882 to CDRH and CBER, including but not limited to, pre-submissions, 510(k)s, PMAs, BLAs, 

883 HDEs, De Novos, IDEs, post-approval study PMA supplements, 522 submissions, CLIA Waiver 

884 by Applications, and Duals. 

 885 

886 A. Regulatory Submission Cover Letters 

887 FDA recommends sponsors identify RWD and RWE as part of the regulatory submission cover 

888 letter to help facilitate review and internal tracking. Specifically, FDA recommends sponsors 

889 include the following elements in the cover letter for each submission that includes RWD: 

 890 

891 • Purpose of using RWE to support the submission (see list of examples in Section 

892 IV.A.); 

893 • Description of where the RWE fits into the totality of clinical evidence submitted 

894 (e.g., to support interpretability of the primary evidence, to establish a performance 

895 goal, to supplement clinical evidence) (see list of examples in Section IV.A.);

896 • Study design (i.e., type of study) using RWD to generate RWE (see list of examples 

897 in Section VI.A.); 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

26 

898 • RWD source(s) used to generate RWE (see list of examples in Section IV.A.); and

899 • Specific RWD source(s) and version, including the following information, if 

900 applicable: 

901 • Data source name; 

902 • Data source provider; 

903 • Version number; and

904 • Date of extraction and date range of data extracted.

905 

906 B. Fit-For-Purpose Assessment

907 If sponsors include RWE in support of regulatory submissions, they should include their fit-for908 purpose assessment of the relevance and reliability of the RWD to generate RWE with the 

909 following elements: 

 910 

911 • An assessment of the key relevance and reliability factors for the study using RWD 

912 (see Section V.), which may include, but is not limited to the following: 

913 • Data availability, linkages, timeliness (see Section V.A.); 

914 • Data accrual, quality and integrity (see Section V.B.); 

915 • Study purpose, specific data elements, generalizability of data, assessment of 

916 confounding, timing of data availability (see Sections V.A. and VI.B.); and

917 • Completeness and accuracy of study sample reflecting the target population, 

918 study design and planning (see Section V.B.2.). 

919 In addition to the fit-for-purpose assessment of the RWD, we recommend that the sponsor 

920 provide the following contextual information regarding how the generated RWE fits into the 

921 totality of evidence: 

 922 

923 • A description of how the RWE is being used in the totality of clinical evidence 

924 submitted to FDA;

925 • A summary of how the totality of the relevance and reliability of the RWD is fit-for926 purpose to address the study question; and

927 • If unique considerations for the specific RWD source exist, sponsors should describe 

928 these considerations and how they impact the overall assessment of the data. 

929 

930 C. Protocol

931 As with traditional clinical studies, sponsors should submit the protocol as part of the regulatory 

932 submission to FDA. In studies designed to test a hypothesis, FDA recommends that sponsors 

933 finalize the protocol and analysis plan prior to reviewing the outcome data of a study and before 

934 performing the prespecified analyses. Sponsors should indicate in the regulatory submission 

935 whether or not the protocol and analysis plan were finalized prior to the analyses. In addition,

936 individuals generating summary scores (e.g., propensity score modeling) should not have access 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

27 

937 to the outcomes within the dataset(s) used for the study. Any revisions to the protocol should be 

938 dated and time-stamped, and the rationale for each change should be provided. 

 939 

940 Similar to protocols submitted with traditional clinical studies, sponsors should consider 

941 providing the following information in the protocol for the study generating the RWE, when the 

942 RWD or RWE is included in the regulatory submission: 

943 

944 • Study synopsis; 

945 • Background and study purpose; 

946 • Explanation of how the source data is or is not representative of the general 

947 disease/population with the condition, including sufficient previous research 

948 to interpret study results within the context of the target population, disease 

949 trajectory, and current clinical care; and

950 • Description of the device included in the study, including the DI portion of the 

951 UDI, if available. For devices excepted from the UDI requirements, sponsors 

952 should include the version(s) of the device. 

953 • Study aims and objectives; 

954 • Study design, including study period; 

955 • Study design diagrams are suggested to clarify (1) potential study entry dates within 

956 study period and (2) assessment of all other data elements in relation to cohort entry 

957 or index date (causality diagram recommended); 

958 • Data source, including a description of how the setting/environment(s) of data capture 

959 provided adequate continuity of care (see Section V.B.); 

960 • Identification of any common data model structure used for housing the RWD 

961 source or for transformed study-specific RWD, if applicable. 

962 • Data elements (conceptual and operational definitions for all), including: 

963 • Determination of initiation, continuance, and discontinuation of device 

964 exposure, if applicable (see Section V.B.); 

965 • Study population, including inclusion and exclusion criteria;

966 • Device and comparator, if included in the study; 

967 • Outcome and endpoints; and 

968 • Covariates. 

969 • Statistical/data analysis plan;

970 • Data management and quality control plans (see Section V.B.);

971 • Sample size and statistical power;

972 • Description of human subject protections, as appropriate, including informed consent, 

973 IRB determination, deidentification plan (e.g., to remove participant identifiers from 

974 patient-generated data or device-generated data), and data confidentiality plans; 

• Plans for adverse event reporting;42 975 

42 See 21 CFR part 803.
Contains Nonbinding Recommendations 

Draft – Not for Implementation

28 

976 • Milestones and timeline;

977 • Auditing and monitoring plans, as applicable; 

978 • If validation or adjudication of data element(s) were conducted (see Section VI.B.), 

979 sponsors should include the study plan and results for validation or adjudication; and 

980 • A copy of the data dictionary used, if one was used or developed. 

981 

982 D. Report 

983 As with traditional clinical studies, sponsors should submit the report as part of the regulatory 

984 submission to FDA. The report should include the occurrence and rationale for any protocol 

985 deviations. 

 986 

987 Additionally, in a regulatory submission with a report of a study using RWD, the study results, 

988 discussion, and conclusion should be included, including how the RWE supports the purpose of 

989 the submission. Specifically, sponsors should include the following information: 

 990 

991 • Date of data extraction (see Section V.B.); 

992 • All elements from the protocol, updated to reflect how the study was conducted; and

993 • Justification that any changes or modifications to the protocol did not affect the 

994 validity of the resulting RWE. 

995 

996 E. Additional Information

997 As with traditional clinical studies, sponsors should also provide the following information in 

998 regulatory submissions that include RWE: 

 999 

• ClinicalTrials.gov National Clinical Trial (NCT) Number,

43 1000 if applicable; 

1001 • Informed consent and IRB documentation, as applicable; 

1002 • Initial and continuing IRB review and approval; and 

1003 • Initial and approved changes to informed consent. 

1004 • List of investigational sites, if any, including mailing address, contact information, 

1005 and investigator name; and 

1006 • Case Report Form templates, as developed by the sponsor, if applicable (e.g., these 

1007 templates may be helpful if EHR or claims data are not mapped to the dataset from 

1008 the source). 

1009 

43 ClinicalTrials.gov assigns a unique NCT number to each clinical study registered on their webpage. See

https://clinicaltrials.gov/ for more information. 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

29 

1010 Appendix A. Recommended Elements for Documentation 

1011 and FDA Review

1012 The following is an example of the recommended elements to assist sponsors and FDA in 

1013 determining relevance and reliability of the RWD. The tables below summarize the

1014 recommended elements identified throughout the guidance for sponsors to document and provide 

1015 in FDA submissions, and the recommended locations for where to include this information. The 

1016 tables are not intended to serve as either a mandatory or exclusive checklist. Rather, the tables 

1017 provide a simplified summary of the key elements that sponsors should use to assess relevance 

1018 and reliability. 

 1019 

1020 Table 1- Recommended RWD Relevance Elements for Submission of RWE

1021 

Item (Linked to Section V.) Information 

for Sponsors 

to Document 

(e.g., to make 

available for 

inspection)

Information 

for Sponsors 

to Provide to 

FDA in 

Submission

Recommended 

Location in FDA 

Submission 

Determine RWD source contains sufficient 

detail to capture data elements and address 

the study question 

x (detailed) x (rationale) Protocol 

(rationale for 

study question 

and data element 

definitions)

Assess longitudinality of data source x Protocol 

Assess continuity of care in data source x Protocol and 

report

Ensure reasonable time between data 

collection and release for research

x Protocol and 

report

Consider changes in clinical 

practice/guidelines over time

x x Protocol

Assess timing of availability of any new 

(i.e., updated) data after initial data 

availability

x Protocol and 

report

Assess whether and how data from different 

sources can be obtained and integrated, 

given the potential for heterogeneity in 

population characteristics, clinical 

practices, and coding across data sources

x

Protocol 

If done, use of a predefined linkage 

methodology that is scientifically valid and 

accounts for differences in coding and 

reporting across sources

x (detailed) x (high level) Protocol
Contains Nonbinding Recommendations 

Draft – Not for Implementation

30 

Item (Linked to Section V.) Information 

for Sponsors 

to Document 

(e.g., to make 

available for 

inspection)

Information 

for Sponsors 

to Provide to 

FDA in 

Submission

Recommended 

Location in FDA 

Submission 

Assess adequacy of line-level linkages x Report

Correct for redundant data, to resolve any 

inconsistencies, and assess the potential for 

missing data

x

Demonstrate interoperability of the linked 

data systems

x

Ensure study sample is representative and 

generalizable to RWD source and target 

population

x Protocol and 

report

 1022 

1023 Table 2- Recommended RWD Reliability Elements for Submission of RWE

1024 

Item (Linked to Section V.) Information 

for Sponsors 

to Document 

(e.g., to make 

available for 

inspection)

Information 

for Sponsors 

to Provide to 

FDA in 

Submission

Recommended 

Location in FDA 

Submission 

Establish information and descriptors about

data source(s)

x Protocol

If applicable, describe defined processes, 

site training, support, qualified personnel 

for complete and accurate data collection

x

Document routine migration of data from 

various sources over time

x

Describe sources and technical methods 

used for data element capture 

x

Describe methods for data retrieval and 

processes to minimize missing data 

extraction, implausible values, and data 

quality checks

x

If not data holder, describe level of access, 

attempt to gain patient-level data, and 

consider access for FDA

x Protocol

Describe the quality of the data captured x (detailed) x (high level) Report
Contains Nonbinding Recommendations 

Draft – Not for Implementation

31 

Item (Linked to Section V.) Information 

for Sponsors 

to Document 

(e.g., to make 

available for 

inspection)

Information 

for Sponsors 

to Provide to 

FDA in 

Submission

Recommended 

Location in FDA 

Submission 

Plan and document the process of 

extraction, aggregation, curation, storage, 

and availability of data for research

x

Describe data flow from first instance to 

data source instance as housed by sponsor

x

Define and follow procedures for data 

collection and quality assurance

x

Provide assessment of completeness, 

accuracy, and consistency across sites and 

over time

x Report

Assess consistency of data capture across 

sites and over time; if any changes are 

needed (e.g., diagnostic criteria or clinical 

definitions change in the course of clinical 

practice over several years), then document 

those changes and assess their impact on 

the study results

x (detailed) x (high level) Report

Assess missingness and out of range values 

for each data element

x Protocol and 

report

Ensure data elements captured and included 

in the study are reflective of the actual 

patient experience (e.g., interactions with 

health care, disease trajectory, outcomes) 

with the condition of interest

x Protocol and 

report

Define the auditing rules and methods used 

and the mitigation strategies used to reduce 

errors 

x

Ensure study size is adequate to address the 

study question with adequate statistical 

power and accounting for planned analyses

x Protocol and 

report

Document adherence to source verification 

procedures and data collection and 

recording procedures for completeness and 

consistency

x

Prespecify data checks and procedures to 

help address identified errors 

x
Contains Nonbinding Recommendations 

Draft – Not for Implementation

32 

Item (Linked to Section V.) Information 

for Sponsors 

to Document 

(e.g., to make 

available for 

inspection)

Information 

for Sponsors 

to Provide to 

FDA in 

Submission

Recommended 

Location in FDA 

Submission 

Describe mitigations to address audit 

findings, including data corrections 

x

Securely store data and ensure appropriate 

permissions/agreements for access

x

Provide documentation of any previous 

RWD source fit-for-purpose assessment for 

a similar target population and all peer 

reviewed literature of RWE generation 

from data source

x Protocol

Ensure adequate patient protections (e.g., 

methods to protect the privacy of 

individuals’ health data and adherence to 

applicable privacy and ethics standards) 

established in advance of executing the 

study protocol

x Protocol and 

report

1025 
Contains Nonbinding Recommendations 

Draft – Not for Implementation

33 

1026 Appendix B. Examples Where RWE is Used

1027 Most of the following examples are generalized from actual uses of RWD in support of FDA 

1028 regulatory decision-making. These examples do not represent a comprehensive list of all 

1029 potential uses or sources of RWD but do describe some situations where RWE might be used to 

1030 support regulatory decision-making. For additional examples of RWE used in regulatory 

1031 decisions, see the following FDA document: Examples of Real-World Evidence (RWE) Used in 

Medical Device Regulatory Decisions.

44 1032 

 1033 

1034 Example 1: New or Expanded Indications for Use

1035 An implanted device, which was available outside the US (OUS), used RWE as the primary 

1036 clinical evidence to support the original PMA submitted to FDA. RWD from an OUS registry in 

1037 one country, which included nearly 300 patients with more than two years of follow-up, was 

1038 compared against a performance goal (PG) for device effectiveness. The PG was derived from a 

1039 prospectively defined, systematic meta-analysis of available published literature and registry data 

1040 for a control device legally marketed in the United States. The study prospectively evaluated the 

1041 functional outcome and patient satisfaction for multiple devices. The safety assessment was 

1042 based on a comparison of the serious device related adverse event rates for the subject device to 

1043 the rates extracted from the registry and the same literature studies used to derive the PG. These 

1044 analyses served as the primary basis supporting approval of the PMA. In this example, an IDE 

1045 was not needed for the study as the clinical data for the subject device was obtained from extant 

1046 data in an OUS registry. 

 1047 

1048 Should a manufacturer wish to expand indications, this type of RWD might be used to generate 

1049 sufficient evidentiary support, depending on the specific devices, indications, and analyses. 

1050 

1051 Another example is an implanted device that was available on the US market for several years 

1052 for one indication for use. The sponsor submitted a PMA panel-track supplement to support 

1053 expanding indications for this device. Supplemental clinical evidence for safety of the indication 

1054 expansion was provided from the sponsor’s patient database, with more than 36,000 patients, 

1055 linked with Medicare claims. The RWE characterized the 12-month safety profile of patients 

1056 with the specific condition (i.e., new indication) implanted with the device and assessed the 

1057 safety among patients receiving the implant and diagnosed with the specific condition compared 

1058 to patients implanted with the device that did not have the respective condition. The sponsor’s 

1059 patient database and Medicare claims were linked using probabilistic methods (i.e., based on date 

1060 of birth, implant date, implant clinic). A systematic review of literature was also conducted to 

1061 identify diagnostic codes associated with safety outcomes. In this example, an IDE was not 

1062 needed for the study as the clinical data for the subject device was obtained from extant data.

1063 

1064 Example 2: 522 Submissions 

44 Available at: https://www.fda.gov/media/146258/download
Contains Nonbinding Recommendations 

Draft – Not for Implementation

34 

1065 The manufacturer of a class II designated software-only device was subject to a postmarket 

surveillance order for this device under section 52245 1066 of the FD&C Act, as it met the statutory 

1067 criteria that its failure would be reasonably likely to have serious adverse health consequences, 

1068 and it also was expected to have significant use in pediatric populations. 

 1069 

1070 Following discussions with the FDA, the postmarket surveillance study for this software-only 

1071 device was a single-arm, prospective study designed to assess the 12-month safety in a real1072 world setting and to support the continued assessment of the software for its intended use. It 

1073 included both pediatric and adult populations. The outcomes were evaluated versus a comparator 

1074 group, which was determined using evidence from a systematic literature review of similar use in 

1075 the intended use population. Device-related adverse events were recorded electronically from 

1076 both inbound reports from customers and assessed through outreach to patients. The safety 

1077 endpoints included the risk for adverse events as extracted in the literature review. Besides the 

1078 safety and effectiveness endpoints, the study also included a secondary endpoint of patient1079 reported satisfaction with and trust in the software-only device. At the end of the study, the 

1080 device had met its endpoints, and there were no unanticipated adverse device effects. 

 1081 

1082 Example 3: Control group 

1083 A sponsor submitted a PMA panel track supplement to support an indication expansion 

1084 supported by a single-arm study compared to a control group of patients enrolled in a US registry 

1085 receiving alternative devices for the new disease condition. The alternative devices were 

1086 identified through the use of UDIs. To account for potential differences between the device and 

1087 control groups, the sponsor performed a propensity-score adjusted analysis using 20 pre1088 specified variables. The propensity score results were reviewed by FDA before the sponsor 

1089 performed the outcome analysis. Sensitivity analyses were conducted to assess the impact of 

1090 missing data. The registry did not routinely collect all relevant data for the control group, 

1091 therefore, additional information for patients in the registry was collected from extant EHRs. The 

1092 results of these analyses demonstrated that the success criteria were achieved. 

 1093 

1094 Along with analyses of serious adverse events from the sponsor’s global study, this RWE was 

1095 the primary basis for supporting approval of the PMA supplement. 

1096 

1097 Example 4: Supplementary Data 

1098 A sponsor submitted a De Novo request for a device. Although the sponsor conducted a 

1099 prospective, repeated measures, single arm study as the primary clinical evidence to support 

1100 safety and effectiveness for their De Novo request, RWD from a chart review at a single site was 

1101 used as supplemental evidence. The overall objective of the study using RWD was to confirm the 

1102 performance of the subject device as part of the standard of care and to further investigate trends 

1103 in treatment outcomes for patients with different severities of the disease. The RWE generated 

1104 from this data provided clinical practice results for a patient population that was not included in 

45 FDA has issued a series of postmarket surveillance orders, related to investigating patient safety issues for a type 

of class II device, under the authority of section 522 of the FD&C Act. These 522 orders cover multiple devices 

from different manufacturers that are similar in intended use, design, and other characteristics, such that the 

surveillance questions are identical.
Contains Nonbinding Recommendations 

Draft – Not for Implementation

35 

1105 one of the clinical studies and demonstrated consistent outcomes with the prospective single arm 

1106 study. 

1107 

1108 Example 5: RWE Obtained from Use of EUA device 

1109 In response to the COVID-19 outbreak, FDA authorized the emergency use of certain devices 

under section 564 of the FD&C Act.46 1110 A sponsor of a serology test authorized for emergency use 

1111 used RWD from a patient registry to support the expansion of the intended use (i.e., 

1112 asymptomatic testing when authorized only for symptomatic testing) in a subsequent Dual 

1113 510(k) and CLIA Waiver by Application. The RWD from the patient registry supplemented data 

1114 from a more traditional clinical study and from peer-reviewed literature. Following Agency 

1115 review, FDA considered the RWE to be valid scientific evidence; the RWD was fit-for-purpose 

1116 and potential sources of bias were minimized. The RWD was fit-for-purpose as it included 

1117 appropriate data elements that were captured in the data and was appropriately linked to the 

1118 specific assay in question (e.g., coding scheme using LOINC or Systematized Nomenclature of 

1119 Medicine Clinical Terms [SNOMED CT]) in addition to information about the setting in which 

the assay was performed and the operators who performed the test.

47 1120 Additionally, it was 

1121 important that the sponsor ensured that the DI, patient identifier (ensuring appropriate patient 

1122 population and rationale for any exclusions (e.g., vaccination status)), relevant demographic and 

1123 clinical information, and their interrelationships were available in a single database or commonly 

1124 available RWD sources. In this example, the sponsor accounted for the use of different 

1125 instruments in generating the results as well as the use of different controls that make it difficult 

1126 to show equivalent performance or understand the comparator used. 

46 For more information on FDA’s emergency use authorities under section 564 of the FD&C Act, see FDA’s

guidance “Emergency Use Authorization of Medical Products and Related Authorities,” available at 

https://www.fda.gov/regulatory-information/search-fda-guidance-documents/emergency-use-authorization-medicalproducts-and-related-authorities 47 For more information on the Dual 510(k) and CLIA Waiver by Application pathway, see FDA’s guidance, 

Recommendations for Dual 510(k) and CLIA Waiver by Application Studies.