Considerations for the Design 
and Conduct of Externally 
Controlled Trials for Drug and 
Biological Products  
Guidance for Industry 
 
 
 
 
 
DRAFT GUIDANCE 
 
  This guidance document is being distributed for comment purposes only. 
 
Comments and suggestions regarding this draft document should be submitted within 90 days of 
publication in the Federal Register of the notice announcing the availability of the draft 
guidance.  Submit electronic comments to https://www.regulations.gov.  Submit written 
comments to the Dockets Management Staff (HFA-305), Food and Drug Administration, 5630 
Fishers Lane, Rm. 1061, Rockville, MD 20852.  All comments should be identified with the 
docket number listed in the notice of availability that publishes in the Federal Register. 
 
For questions regarding this draft document, contact (CDER) Dianne Paraoan, 301-796-2500, or 
(CBER) Office of Communication, Outreach and Development, 800-835-4709 or 240-402-8010.  
 
 
U.S. Department of Health and Human Services 
Food and Drug Administration 
Center for Drug Evaluation and Research (CDER) 
Center for Biologics Evaluation and Research (CBER)  
Oncology Center of Excellence (OCE) 
 
February 2023 
Real-World Data/Real-World Evidence (RWD/RWE) 
49539091dft   
Considerations for the Design 
and Conduct of Externally 
Controlled Trials for Drug and 
Biological Products  
Guidance for Industry 
 
 
Additional copies are available from: 
 
Office of Communications, Division of Drug Information  
Center for Drug Evaluation and Research 
Food and Drug Administration 
10001 New Hampshire Ave., Hillandale Bldg., 4th Floor  
Silver Spring, MD 20993-0002  
Phone: 855-543-3784 or 301-796-3400; Fax: 301-431-6353 
Email: druginfo@fda.hhs.gov  
https://www.fda.gov/drugs/guidance-compliance-regulatory-information/guidances-drugs  
 
and/or 
 
Office of Communication, Outreach and Development  
Center for Biologics Evaluation and Research 
 Food and Drug Administration 
10903 New Hampshire Ave., Bldg. 71, Room 3128 
Silver Spring, MD 20993-0002 
Phone: 800-835-4709 or 240-402-8010 
Email: ocod@fda.hhs.gov 
https://www.fda.gov/vaccines-blood-biologics/guidance-compliance-regulatory-information-biologics/biologics-guidances  
 
U.S. Department of Health and Human Services 
Food and Drug Administration 
Center for Drug Evaluation and Research (CDER) 
Center for Biologics Evaluation and Research (CBER)  
Oncology Center of Excellence (OCE) 
 
 
February 2023 
Real-World Data/Real-World Evidence (RWD/RWE) 
 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
TABLE OF CONTENTS 
   
I.  INTRODUCTION............................................................................................................. 1 
II.  BACKGROUND ............................................................................................................... 2 
III.  DESIGN AND ANALYSIS OF EXTERNALLY CONTROLLED TRIALS .............. 4 
A.  Design Considerations ................................................................................................................... 4 
1. Overview .......................................................................................................................................... 4 
2. Characteristics of Study Populations ............................................................................................... 6 
3. Attributes of Treatment .................................................................................................................... 6 
4. Designation of Index Date (Time Zero) ........................................................................................... 7 
5. Assessment of Outcomes .................................................................................................................. 8 
B.  Data Considerations for the External Control Arm ................................................................. 10 
1. Data from Clinical Trials............................................................................................................... 10 
2. Data from RWD Sources ................................................................................................................ 11 
3. Considerations for Assessing Comparability of Data Across Trial Arms ..................................... 11 
C.  Analysis Considerations .............................................................................................................. 13 
1. General Considerations ................................................................................................................. 13 
2. Missing Data .................................................................................................................................. 14 
3. Misclassification of Available Data ............................................................................................... 15 
4. Additional Analyses ....................................................................................................................... 16 
IV.  CONSIDERATIONS TO SUPPORT REGULATORY REVIEW ............................ 16 
A.  Communication with FDA .......................................................................................................... 16 
B.  Access to Data and Documents ................................................................................................... 16 
GLOSSARY................................................................................................................................. 17 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
1  Considerations for the Design and Conduct of Externally Controlled 
2  Trials for Drug and Biological Products1 
3   
4   
5  This draft guidance, when finalized, will represent the current thinking of the Food and Drug 
6  Administration (FDA or Agency) on this topic.  It does not establish any rights for any person 
7  and is not binding on FDA or the public.  You can use an alternative approach if it satisfies the 
8  requirements of the applicable statutes and regulations.  To discuss an alternative approach, 
9  contact the FDA staff responsible for this guidance as listed on the title page.  
10   
11   
12   
13   
14  I.  INTRODUCTION  
15   
16  This guidance provides recommendations to sponsors and investigators considering the use of 
17  externally controlled clinical trials2 to provide evidence of the safety and effectiveness of a drug 
18  product.3  In an externally controlled trial, outcomes in participants receiving the test treatment 
19  according to a protocol are compared to outcomes in a group of people external to the trial who 
20  had not received the same treatment.  The external control arm can be a group of people, treated 
21  or untreated, from an earlier time (historical control), or it can be a group of people, treated or 
22  untreated, during the same time period (concurrent control) but in another setting.4,5  
23   
24  The guidance addresses considerations for the design and analysis of externally controlled trials 
25  to study the effectiveness and safety of drugs, including discussion of threats to the validity of 
 
1 This guidance has been prepared by the Office of Medical Policy in the Center for Drug Evaluation and Research 
in cooperation with the Center for Biologics Evaluation and Research and the Oncology Center of Excellence at the 
Food and Drug Administration.  
 
2 In this guidance, the terms clinical trials, clinical studies, and clinical investigations are interchangeable.  
 
3 In this guidance, the term drug product includes both human drugs and biological products. 
 
4 FDA regulations under 21 CFR 314.126 outline the characteristics of adequate and well-controlled studies, and 
recognize various controls, including a historical control, which FDA considers to be a subset of a broader category 
of potential external controls.  FDA has accepted various types of external controls, when appropriate, for a specific 
drug development program.  See also the International Council for Harmonisation (ICH) guidance for industry E10 
Choice of Control Group and Related Issues in Clinical Trials (May 2001).  We update guidances periodically.  For 
the most recent version of a guidance, check the FDA guidance web page at https://www.fda.gov/regulatory-
information/search-fda-guidance-documents.  
 
5 Although multiple arms may be part of the overall trial design, this guidance discusses externally controlled trials 
involving analysis of a single treatment arm and a single control arm. 
 
 1 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
26  trial results from potential bias.6  Although various sources of data can serve as the control arm 
27  in an externally controlled trial, this guidance focuses on the use of patient-level data from other 
28  clinical trials or from real-world data (RWD) sources, such as registries as well as electronic 
29  health records (EHRs) and medical claims.7  The guidance also describes considerations related 
30  to communicating with FDA and ensuring access by FDA to data from an externally controlled 
31  trial. 
32   
33  This guidance does not address other types of external controls, such as using summary-level 
34  estimates instead of patient-level data.  This guidance does not discuss details of the design and 
35  analysis of a natural history study8 nor the reliability and relevance of various sources of RWD9 
36  that could be used in an externally controlled trial.  Finally, this guidance also does not discuss 
37  considerations for using external control data to supplement a control arm in a traditional 
38  randomized controlled clinical trial. 
39   
40  In general, FDAs guidance documents do not establish legally enforceable responsibilities.  
41  Instead, guidances describe the Agencys current thinking on a topic and should be viewed only 
42  as recommendations, unless specific regulatory or statutory requirements are cited.  The use of 
43  the word should in Agency guidances means that something is suggested or recommended, but 
44  not required. 
45   
46   
47  II.  BACKGROUND 
48   
49  The purpose of conducting clinical investigations of a drug product is to distinguish the effect of 
50  a drug on the target condition from other influences, such as spontaneous change in the course of 
51  the disease, placebo effect, or biased observation.10  When properly conducted, a clinical trial
52  with random assignment of participants either to a treatment arm or to a placebo (or other 
 
6 Words and phrases in bold italics are defined in the Glossary. 
 
7 Given that an external control arm can involve the use of RWD, FDA is issuing this guidance to satisfy, in part, the 
requirements of the 21st Century Cures Act to issue guidance on the use of real-world evidence (RWE) in 
regulatory decision-making, specifically to evaluate the potential use of RWE to help support the approval of a new 
indication for a drug already approved under section 505(c) of the FD&C Act or to help support or satisfy 
postapproval study requirements. 
 
8 See the draft guidance for industry Rare Diseases:  Natural History Studies for Drug Development (March 2019).  
When final, this guidance will represent FDAs current thinking on this topic.  Natural history studies can be used 
for purposes such as identifying a study population, developing clinical outcome assessments or biomarkers, and 
serving as a comparator group in an externally controlled trial.  
 
9 See the following draft guidances for industry:  Real-World Data:  Assessing Electronic Health Records and 
Medical Claims Data to Support Regulatory Decision-Making for Drug and Biological Products (September 2021); 
Real-World Data: Assessing Registries to Support Regulatory Decision-Making for Drug and Biological Products 
(November 2021); and Data Standards for Drug and Biological Product Submissions Containing Real-World Data 
(October 2021).  When final, these guidances will represent FDAs current thinking on these topics. 
 
10 See 21 CFR 314.126(a).  
 
 2 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
53  control) armoptimally promotes the similarity of compared groups regarding such influences, 
54  such that a conclusion can be made as to whether differences in outcomes observed between 
55  groups can be attributed to the treatment of interest.  Nevertheless, for decades FDA has 
56  recognized the potential value of other types of controls, including historical controls as a type of 
57  external control.11  Clinical trials using these other types of controls can, when appropriate, serve 
58  as the adequate and well-controlled clinical investigations generally required to provide 
59  substantial evidence of effectiveness under section 505(d) of the Federal Food, Drug, and 
60  Cosmetic Act (FD&C Act).12 
61   
62  Given that externally controlled trials do not involve randomization of the study population to 
63  the treatments being compared, the treatment and control arm populations should be as similar as 
64  possible regarding known factors that can affect the outcome being measured.  These factors, 
65  discussed in more detail in section III, include important baseline characteristics (e.g., 
66  demographic factors, comorbidities), disease attributes (e.g., severity, symptoms, duration of 
67  illness), start of follow-up for the treatment of interest, concomitant therapies, and the clinical 
68  observations collected.  Importantly, before choosing to conduct a clinical trial using an external 
69  control arm as a comparator, sponsors and investigators should consider the likelihood that such 
70  a trial design would be able to distinguish the effect of a drug from other factors that impact the 
71  outcome of interest and meet regulatory requirements.13   
72   
73  The suitability of an externally controlled trial design warrants a case-by-case assessment, 
74  informed by issues including heterogeneity of the disease (e.g., clinical presentation, severity, 
75  prognosis), preliminary evidence regarding the drug product under investigation, the approach to 
76  ascertaining the outcome of interest, and whether the goal of the trial is to show superiority or 
77  non-inferiority.14  Of note, if the natural history of a disease is well-defined and the disease is 
78  known not to improve in the absence of an intervention or with available therapies, historical 
79  information can potentially serve as the control group.  For example, objective response rate is 
80  often used as a single-arm trial endpoint in oncology given the established understanding that 
81  tumor shrinkage rarely occurs without an intervention.15,16   
82   
83  In many situations, however, the likelihood of credibly demonstrating the effectiveness of a drug 
84  of interest with an external control is low, and sponsors should choose a more suitable design, 
 
11 See 21 CFR 314.126(b)(2)(v).  
 
12 See section 505(d) of the FD&C Act (21 U.S.C. 355(d)).  
 
13 See 21 CFR 314.126. 
 
14 A non-inferiority approach is not recommended using an externally controlled trial design.  See the guidance for 
industry Non-Inferiority Clinical Trials to Establish Effectiveness (November 2016). 
 
15 See the ICH guidance for industry E10 Choice of Control Group and Related Issues in Clinical Trials (May 
2001).  
 
16 See the guidance for industry Clinical Trial Endpoints for the Approval of Cancer Drugs and Biologics 
(December 2018). 
 
 3 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
85  regardless of the prevalence of disease.  For example, when considering whether to use an 
86  externally controlled trial design, sponsors should decide whether it is possible to generate 
87  evidence capable of distinguishing the effect of the drug from outcomes attributable to the 
88  diseases natural history,17 prognostic differences in the study populations, knowledge of 
89  treatment assignment (lack of blinding), or other factors such as differences in concomitant 
90  therapies.  
91   
92  The remainder of this guidance is intended to assist sponsors in identifying and addressing 
93  commonly encountered challenges when considering the conduct of an externally controlled 
94  trial.  
95   
96   
97  III.  DESIGN AND ANALYSIS OF EXTERNALLY CONTROLLED TRIALS  
98   
99  A.  Design Considerations 
100   
101  1.  Overview 
102   
103  Reducing the potential for bias in externally controlled trials is best addressed in the design 
104  phase, in that well-chosen design elements increase confidence in the interpretability of study 
105  results when appropriate analytic methods are applied to estimate treatment effects.  Sponsors 
106  should finalize a study protocol before initiating the externally controlled trial, including 
107  selection of the external control arm and analytic approach, rather than selecting an external 
108  control arm after the completion of a single-arm trial.  Specific design elements to prespecify in 
109  the protocol (i.e., before conducting an externally controlled trial) include suitable study data 
110  sources,18 baseline eligibility (inclusion and exclusion) criteria,19 appropriate exposure 
111  definitions and windows, well-defined and clinically meaningful endpoints, cogent analytic 
112  plans, and approaches to minimize missing data and sources of bias. 
 
17 Scenarios that would not be suitable for externally controlled trials include when the natural history of the disease 
of interest is not understood sufficiently or when the disease course is considered well-understood but is variable. 
 
18 FDA recognizes that access to and evaluation of relevant data sources or databases are important steps in 
designing a control arm for externally controlled trials and in evaluating the trials feasibility.  Sponsors should 
document and describe in the trial protocol all data sources accessed when designing the control arm of the trial and 
the results of any feasibility evaluations or exploratory analyses.  Sponsors should provide a justification for 
selecting or excluding relevant data sources and demonstrate that the choice of a final analytic dataset for the control 
arm aligns with the research question of interest and was not chosen to favor particular study results.  FDA 
recommends that sponsors generate audit trails in their datasets that can track access to and analyses performed on 
relevant data sources.  See the draft guidance for industry Considerations for the Use of Real-World Data and Real-
World Evidence to Support Regulatory Decision-Making for Drug and Biological Products (December 2021).  
When final, this guidance will represent the FDAs current thinking on this topic.  
 
19 In this guidance, the term eligibility criteria refers to the requirements for entry into a clinical trial (i.e., the 
characteristics the participants must or must not have to be able to participate in the trial).  See the guidance for 
industry Enhancing the Diversity of Clinical Trial Populations  Eligibility Criteria, Enrollment Practices, and 
Trial Designs (November 2020). 
 
 4 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
113    
114  The estimand framework20involving a precise description of the treatment effect reflecting the 
115  clinical question posed by the study objectivecan be used to help design an externally 
116  controlled trial.  An estimand is comprised conceptually of the study population, treatment of 
117  interest and comparator, outcome of interest, handling of intercurrent events, and summary 
118  measures.  Many of the elements of the estimand framework are described individually in the 
119  subsections below, and considering the elements together promotes alignment of trial objectives, 
120  conduct, analysis, and interpretation of results.  
121   
122  A specific design consideration for externally controlled trials involves prespecifying plans 
123  regarding how to measure and analyze data on important confounding factors and sources of 
124  bias.  The ability to identify confounding factors in an externally controlled trial is limited by 
125  both conceptual and practical concerns.  Conceptually, when seeking to provide evidence of 
126  effectiveness using an externally controlled trial design, a thorough understanding is needed
127  but is often difficult to verifyregarding the natural history21 of the disease involved and 
128  relevant prognostic factors influencing outcomes.  For example, important prognostic factors for 
129  an outcome may not be known and therefore cannot be used in the process of developing the 
130  external control arm to match, as closely as possible, such factors in the treatment arm.   
131   
132  From a practical perspective, fit-for-use data on suspected confounding factors (e.g., history of 
133  cigarette smoking, performance status) may be missing for some patients or participants or may 
134  be measured differently in the external control arm compared to the treatment arm.  Accordingly, 
135  before deciding whether an externally controlled trial is a suitable design to answer the research 
136  question of interest, sponsors should confirm that recognized, important prognostic 
137  characteristics can be assessed in the data sources that will be used in an externally controlled 
138  trial.  Specifically, the source population for the external control arm should be as comparable as 
139  possible to the treatment arm population, given that controlling for differences between the two 
140  study arms (see section III.C) becomes more challenging with increasingly dissimilar 
141  populations.    
142   
143  Although unmeasured confounding, lack of blinding, and other sources of bias cannot be 
144  eliminated in externally controlled trials, an assessment of the extent of confounding and bias, 
145  along with analytic methods to reduce the impact of such bias, are critically important in the 
146  conduct of such trials.  Given the challenges outlined, externally controlled trials are more likely 
 
20 For further information, see the ICH guidance for industry E9(R1) Statistical Principles for Clinical Trials: 
Addendum: Estimands and Sensitivity Analysis in Clinical Trials (May 2021).  
 
21 Changes over time in the understanding of the natural history of a disease can also introduce bias in an externally 
controlled trial.  For example, diagnosis of patients with a genetic disorder may have been based historically on the 
development of signs and symptoms, whereas the development and increased use of genetic testing in clinical trials 
can diagnose patients at earlier stages of disease (see, for example, EA Nannenberg, IAW van Rijsingen, PA van der 
Zwaag, MP van den Berg, JP van Tintelen, MWT Tanck, MJ Ackerman, AAM Wilde, and I Christiaans, 2018, 
Effect of Ascertainment Bias on Estimates of Patient Mortality in Inherited Cardiac Diseases, Circ Genom Precis 
Med, 11(10):e001797).  In such situations, a historical control arm would have shorter diagnosis-to-death intervals 
than a treatment arm, even if the drug of interest has no impact on survival. 
 
 5 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
147  to provide convincing results when the effect size on a well-characterized outcome of interest is 
148  anticipated to be large.22  
149   
150  2.  Characteristics of Study Populations  
151   
152  In the absence of randomization, a major concern for externally controlled trials is that attributes 
153  of patients23 likely to influence outcomes in an external control arm will differ from 
154  corresponding attributes of participants in a treatment arm of the trial.  Examples of baseline 
155  attributes of participants or patients in treatment and control groups that can be dissimilar include 
156  demographic and related factors (e.g., age, sex, race, socioeconomic status, geographic region).  
157  Additional attributes that could be dissimilar but often are more challenging to address include 
158  disease characteristics (e.g., severity, duration, specific signs and symptoms, performance 
159  status), prognostic or predictive biomarkers,24 comorbidities, and prior and current treatments 
160  received.  When accounting for baseline characteristics, specific challenges can include (1) 
161  whether relevant confounding factors are known and well-characterized; (2) whether such 
162  confounding factors are captured; (3) whether these factors have been assessed with appropriate 
163  methods and measured similarly across compared groups; and (4) whether the studys analytic 
164  methods sufficiently address the differences in clinical characteristics between the compared 
165  groups.   
166   
167  A specific consideration involves how well the eligibility criteria can be applied to the external 
168  control arm in order to obtain a population comparable to the treatment arm.  In addition, unless 
169  a concurrent control group is being used, sponsors should consider whether diagnostic criteria for 
170  the condition of interest and other relevant baseline factors, or the approaches used to ascertain 
171  data on such factors, have changed during the time of data collection.  Accordingly, the protocol 
172  for an externally controlled trial should include specific plans for evaluating eligibility criteria to 
173  determine if the criteria can be applied in a manner that allows for selection of similar patients in 
174  the treatment and external control groups, recognizing the limitations of information available in 
175  many RWD sources.  
176   
177  3.  Attributes of Treatment 
178   
179  In properly designed and conducted randomized trials, observed differences in efficacy and 
180  safety outcomes can generally be attributed to the investigational drug, but confidence in such 
181  attribution is diminished in externally controlled trials because of concerns over potentially 
 
22 See the ICH guidance for industry E10 Choice of Control Group and Related Issues in Clinical Trials (May 2001) 
and 21 CFR 314.126(b)(v). 
 
23 In this guidance, the term patient refers to a person whose health care information (e.g., regarding a disease) is 
included in a study, whereas the term participant refers to a healthy person or a person with a disease who 
participates in a study. 
 
24 Prognostic and predictive biomarkers are used to assess the rate of disease progression or response to therapy, 
respectively.  For additional discussion, see BEST (Biomarkers, EndpointS, and other Tools) Resource, available at 
https://www.ncbi.nlm.nih.gov/books/NBK338448, as well as the guidance for industry Enrichment Strategies for 
Clinical Trials to Support Approval of Human Drugs and Biological Products (March 2019).  
 6 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
182  important imbalances with respect to treatment between the treatment arm and the external 
183  control arm that are either not documented or cannot be accounted for.  Such imbalances can 
184  involve factors related to the treatment of interest (e.g., adherence, dose, timing of initiation, and 
185  duration of treatment) and receipt of additional treatments.  These concerns are expected when 
186  the data in the external control arm are from an RWD source, and although the remainder of this 
187  section focuses on such data sources, potential imbalances can also exist when the data come 
188  from other clinical trials.  
189   
190  Clinical trial protocols typically include a plan for collecting data on use of concomitant or 
191  supportive therapies (including non-prescription products) that could affect the outcomes of 
192  interest, along with detailed data on the characteristics and administration of such therapies.  
193  Examples include drug formulation, dose, strength, route, timing, frequency, and durationand 
194  for certain medications, specific rules for dose modifications, interruptions, or discontinuations 
195  are specified in the protocol.  In contrast, documentation of such data in routine clinical care may 
196  not be complete or accurate, and RWD may therefore lack comprehensive details describing the 
197  administration of a treatment or information on the use of concomitant or supportive therapies.  
198  For example, suitable data on additional treatment modalities (e.g., radiotherapy and surgical 
199  interventions when treating patients with cancer) may not be available in certain data sources.  In 
200  addition, management of treatment- or disease-related adverse events may not be predefined or 
201  described consistently compared to a trial protocol.    
202   
203  Additional factors can influence the treatment and delivery of care that patients receive as well as 
204  the assessment of outcomes related to those treatments when data from clinical care are 
205  analyzed.  Examples include differences in health-seeking behaviors, insurance coverage 
206  (including prescription drug plans), adoption of clinical practice guidelines, availability of novel 
207  treatments, and use of companion diagnostic testing (e.g., a genetic test used in conjunction with 
208  a corresponding therapeutic product).  Access to emergency department or intensive care, 
209  availability and coordination of subspecialty care, and academic versus community health care 
210  settings can also be markedly different within or across health care systems or geographic areas.  
211  These and other health care delivery factorsat the level of the patient, provider, or health 
212  systemcan influence treatment selection.  Such factors should be identified and accounted for 
213  adequately in externally controlled trials; otherwise, a different design approach (e.g., 
214  randomized controlled trial) should be considered.  
215   
216  4.  Designation of Index Date (Time Zero)  
217   
218  A specific and difficult challenge when designing externally controlled trials is specifying the 
219  index date (also called time zero or zero time), which is the start of the observation period for 
220  assessing endpoints.  Given the lack of randomization in externally controlled trials, differences 
221  in the way the index date is determined across trial arms may lead to biased effect estimates.  
222  The index date for the treatment and control arms in a randomized trial is usually designated as 
223  the time when eligibility criteria are determined to have been met and a decision was made 
224  regarding the intended treatment strategy for each participant.  For an externally controlled trial 
225  that relies on RWD, however, the index date for the control arm can be assigned in various ways.  
 7 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
226  If there are temporal differences in this date relative to treatment initiation or other important 
227  landmark times by treatment arm, any observed treatment effects may be biased.   
228   
229  Determination of the index date in the treatment arm and the external control arm should avoid 
230  analyses that include a period of time (immortal time) during which the outcome of interest 
231  could not have occurred in one of the two arms.  If the index date is not established appropriately 
232  across compared arms in an externally controlled trial, bias due to immortal time can occur.  For 
233  example, consider an externally controlled trial that involves a time-to-event mortality endpoint 
234  and an index date established as the time of having failed prior therapy.  If analyses of 
235  participants in the treatment arm include only those who actually receive the drug of interest, 
236  then any period of time between eligibility determination (i.e., failed prior therapy) and treatment 
237  initiation is immortal time; that is, the person must survive the period to receive the drug and be 
238  accounted for in the analysis.  In contrast, if patients in the external control arm do not receive 
239  subsequent therapy after determination of eligibility (i.e., failed prior therapy), these patients 
240  would be included in the analysis regardless of survival.  Accordingly, patients with very short 
241  survival times would be included in the control arm but not in the treatment arm, introducing a 
242  bias that makes the drug seem more effective than it actually is.25    
243   
244  When assessing bias that may be introduced related to immortal time in an externally controlled 
245  trial, the clinical circumstances related to assigning the index date should be considered.  
246  Specifically, if a treatment strategy is assigned immediately after a discrete and identifiable 
247  clinical event, the index date for the compared groups may be reasonably determined by the time 
248  of occurrence of that event.  For example, if treatment is started after an acute myocardial 
249  infarction, stroke, or heart failure hospitalization, these events may be more suitable to identify 
250  the index date for both the treatment arm and external control arm.  In contrast, when the event 
251  that prompts the treatment of interest is not discrete and readily identifiable, such as worsening 
252  of heart failure symptoms or poor control of hypertension, determining a suitable index date can 
253  be difficult or may not be possible.  Identifying an index date can also be especially challenging 
254  in situations in which no treatment is the treatment strategy for the external control arm. 
255   
256  5.  Assessment of Outcomes 
257   
258  The lack of blinding to treatments in externally controlled trials can pose challenges when 
259  considering certain outcomes, in that knowledge of the particular treatment by patients, 
260  caregivers, clinicians, or investigators can potentially lead to a biased estimate of the effect of 
261  treatment.  Accordingly, whenever possible and for suitable endpoints, the outcome should be 
262  assessed blinded to treatment status.  In some cases, this activity may require re-adjudication of 
263  the externally controlled data, such as by blinded independent central review.  Bias can also be 
264  introduced if outcome assessments in the treatment arm and the external control arm differ based 
265  on the sources of data involved or the criteria used to establish outcomes.  Sponsors should seek 
266  to assess outcomes consistently across the treatment arm and the external control arm for the 
267  results of an externally controlled trial to be credible. 
268   
 
25 In a randomized trial, potential periods of immortal time are expected to be balanced across treatment groups. 
 8 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
269  Well-defined, reliable, and clinically meaningful outcomes that are typically used in randomized 
270  trials may be particularly difficult to ascertain and evaluate in an RWD source that is being 
271  considered for an externally controlled trial.  For example, radiologic endpoints in controlled 
272  oncology trials (e.g., objective response rate and progression-free survival) are based on 
273  prespecified imaging assessment frequency and standardized measurement criteria for response 
274  evaluation criteria in solid tumors (RECIST).  In routine clinical care, however, radiologic 
275  assessment frequency is variable, and formal tumor measurement may not routinely be 
276  performed or documented, making a valid assessment of progression-free survival or objective 
277  response rate using external control data, such as data from EHRs, challenging.26  A similar 
278  consideration applies to the assessment of motor milestones, such as the ability to sit or walk, 
279  which are usually not recorded with the same rigor during routine clinical care compared to 
280  approaches used in clinical trials.  As another example, a randomized trial may include specific 
281  testing to detect or confirm a particular clinical entity (e.g., severe inflammatory bowel disease 
282  activity confirmed by endoscopy), whereas various strategies may be used in clinical care to 
283  identify and confirm the same event.  In some cases, and depending on the outcome, the 
284  occurrence of an event (e.g., worsening heart failure status according to a specific classification 
285  system) may not have been evaluated in clinical care or, if evaluated, may not have been 
286  recorded.  As a general consideration, outcomes of interest are more likely to be recorded in 
287  clinical records when events are objective and/or require immediate medical attention (e.g., 
288  stroke or myocardial infarction).   
289   
290  When considering outcomes in externally controlled trials, sponsors should also evaluate the 
291  consistency of timing of outcome assessments in the treatment arm compared to the external 
292  control arm.  In general, the timing and frequency of outcome assessments in RWD will have 
293  been determined during clinical care27 and may have been influenced by the patients clinical 
294  status, whereas outcome assessments in the treatment arm are protocol-specified.  In addition, 
295  even when external control arm data are from another clinical trial rather than from an RWD 
296  source, the approach to outcome ascertainment may differ from the treatment arm.  Accordingly, 
297  sponsors should first establish for what total duration of time and at what intervals the outcome 
298  of interest should be assessed in the analysis of data from an externally controlled trial.  Based on 
299  such determinations, sponsors can then evaluate whether the availability and timing of outcome 
300  assessments are sufficient and comparable across both arms of the externally controlled trial for 
301  the research hypothesis being tested. 
302   
303  Additional challenges when considering the selection of outcomes to be assessed in an externally 
304  controlled trial include changing diagnostic criteria over time for what constitutes abnormal 
305  clinical, radiographic, serologic, or other outcomes.  Whereas both trial arms would be similarly 
306  affected in a traditional randomized trial, extensive heterogeneity or substantial changes in 
 
26 See EA Eisenhauer, P Therasse, J Bogaerts, LH Schwartz, D Sargent, R Ford, J Dancey, S Arbuck, S Gwyther, M 
Mooney, L Rubinstein, L Shankar, L Dodd, R Kaplan, D Lacombe, and J Verweij, 2009, New Response Evaluation 
Criteria in Solid Tumours: Revised RECIST Guideline (version 1.1), Eur J Cancer, 45(2):228247.   
 
27 Registries (one type of RWD) may collect data at predetermined and regular intervals, whereas EHRs and medical 
claims data would usually not.  
 
 9 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
307  diagnostic criteria can introduce bias when analyzing outcomes using a non-contemporaneous 
308  external control arm (or when using a reasonably contemporaneous external control arm that 
309  reflects a different diagnostic standard of care).  As another challenge that can introduce bias, 
310  biomarkers used as surrogate outcomes in clinical trials may be used for different purposes in 
311  clinical care, or biomarkers used in clinical care may not be well-characterized in terms of 
312  comparability to assays used in clinical trials. 
313    
314  Further challenges may arise from differential capture of intercurrent events that may preclude 
315  the measurement of or impair the interpretability of the treatment effect on the outcome of 
316  interest.  For example, initiation of ancillary therapy after treatment with the drug of interest is 
317  started may be protocol-determined and recorded during study visits in a clinical trial, whereas 
318  data from routine clinical care may not accurately capture additional therapies, which may 
319  confound interpretation of the effect of treatment on the study outcome.  
320   
321  Other considerations apply when an outcome in an externally controlled trial is based on certain 
322  clinical outcome assessments.28  For example, the potential lack of standardization and training 
323  in the definitions and use of such assessments in routine clinical care settingsif the assessments 
324  are used at allcompared to what occurs in clinical trial settings, can lead to higher variability 
325  or bias in the measurements from an external control arm.  Accordingly, clinical outcome 
326  assessments that are acceptable in randomized trials may not be fit for use in externally 
327  controlled trials.  
328   
329  B.  Data Considerations for the External Control Arm 
330   
331  1.  Data from Clinical Trials 
332   
333  Using data from another clinical trial for an external control arm can have advantages compared 
334  to using data collected during routine clinical care, based in part on the rigor of protocol-based 
335  (and therefore more consistent) data collection.  Such use would only be appropriate, however, 
336  when comparability exists between the two trial arms regarding participant eligibility criteria, 
337  treatment administration, patterns of care (e.g., location of treatment sites), recording of 
338  concomitant medications, and assessments of adverse events and outcomes.  A particular concern 
339  for bias would be the selection of an external control arm from a completed trial whose outcomes 
340  are already known.  This would be especially problematic if the results of the external control 
341  arm are inconsistent with prior experience.  Furthermore, when using data from other clinical 
342  trials as an external control arm, sponsors should consider the extent of and reason for any 
343  missing data and how the interpretability of study results may be affected. 
344   
345  In many situations, data for the treatment and control arms in an externally controlled trial will 
346  have been collected during different time periods.  Lack of concurrent data collection may be of 
 
28 A clinical outcome assessment is a measure that describes or reflects how a patient feels, functions, or survives.  
Types of such assessments include measures of patient-reported outcomes, observer-reported outcomes, clinician-
reported outcomes, and performance outcomes.  See BEST (Biomarkers, EndpointS, and other Tools) Resource, 
available at https://www.ncbi.nlm.nih.gov/books/NBK338448. 
 
 10 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
347  particular concern when the assessment and management of a disease (including supportive care) 
348  changes over time, such as use of predictive or prognostic biomarkers in the patient population.  
349  For example, prior trials involving certain cancers may not have information regarding newer 
350  biomarkers or specific gene alterations of interest or tumor mutational burden.  Accordingly, 
351  sponsors should assess whether use of data from a specific clinical trial is justified as an external 
352  control arm when planning an externally controlled trial.  
353   
354  2.  Data from RWD Sources 
355   
356  The concerns described in the preceding section regarding comparability of participant 
357  characteristics, timing and frequency of data collection, and patterns of care should be addressed 
358  when using RWD collected on patients for non-research purposes as external control arms.  In 
359  addition, specific concerns regarding missing data from RWD sources obtained as part of routine 
360  clinical practice can threaten the validity of the results of an externally controlled trial.  For 
361  example, patients who initially met eligibility criteria may be lost to follow-up (e.g., due to 
362  changing their health care provider) from the external control arm.  Furthermore, availability of a 
363  dataset containing patients with the disease of interest does not guarantee that there is sufficient 
364  information on relevant clinical characteristics (e.g., prognostic factors for the outcome of 
365  interest) to permit an appropriate comparison.  
366   
367  3.  Considerations for Assessing Comparability of Data Across Trial Arms 
368   
369  The table below summarizes important considerations, discussed above, regarding the 
370  comparability of data between the treatment arm and the external control arm.  The relevance of 
371  each consideration can vary on a case-by-case basis, depending on attributes of the treatment 
372  arm, the selected data source for the external control arm, and the stage of the trial (design, 
373  conduct, or analysis).    
 11 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
374  Table.  Summary of Considerations for Assessing Comparability of Data29 
375   
Focus of 
Considerations for Data Comparability 
Comparison 
Time periods  Various aspects of clinical care may change over time, such as the standard of care 
for the condition of interest, types of treatments, supportive care regimens, and 
criteria for determining disease response or progression.  Such temporal differences 
are difficult to address using statistical analyses alone.  It is important to consider 
whether and how different time frames in the treatment arm and the external control 
arm impact the interpretability of study findings. 
Geographic  Standards of care and other factors (e.g., access to care) that affect health-related 
region  outcomes can vary across geographic regions and health care systems.  A balance of 
participants or patients across geographic regions and health care systems in an 
externally controlled trial, when possible, can help reduce the impact of confounding 
based on such differences. 
Diagnosis  The criteria used to establish a diagnosis may differ based on practice variation or 
may have changed in the interval between when the treatment arm of the trial was 
conducted and when the data for the external control arm were collected.  Sponsors 
should consider the diagnostic standards used and whether relevant clinical tests to 
establish a diagnosis were conducted and reported equally across the compared arms. 
Prognosis  Based on demographic and clinical characteristicsand if sufficient knowledge of 
relevant prognostic factors is availableprognostic indicators for the participants or 
patients in each arm of the trial should be evaluated and shown to be of sufficient 
similarity to permit an unbiased assessment of the treatment-outcome association.   
Treatments   Attributes of the treatment of interestincluding drug formulation, dose, route of 
administration, timing, frequency, and duration as well as specific rules for dose 
modifications, interruptions, discontinuations, and adherencewill have been 
prespecified or measured in the treatment arm.  In contrast, specific aspects of a 
comparator treatment (as applicable) in the external control arm may not have been 
protocol-driven depending on the data source.  Accordingly, sponsors should assess 
whether the external control arm data can be meaningfully compared to the treatment 
arm data. 
Other  Various treatment-related considerations, when relevant, include (1) previous 
treatment- treatments received (e.g., lines of therapy in patients with cancer), (2) medications 
related factors  received concomitantly that can affect the outcome of interest, or (3) predictive 
biomarkers (e.g., genomic testing) related to the treatment of interest.  When 
differentially distributed across groups being compared, such factors can threaten an 
assessment of the drug-outcome association. 
Follow-up  Designation of the index date should be consistent between the treatment arm and the 
periods  external control arm, and the duration of follow-up periods should be comparable 
across compared arms. 
Intercurrent  The relevance of intercurrent events across treatment arms should be assessed, 
events  including differential use of additional therapies after initiation of the treatment of 
interest.   
Outcome   Whether endpoints used in an externally controlled trial can be reliably and 
consistently measured across the external control arm and the treatment arm will be 
influenced by several factors, including the definitions of the endpoints, the data 
 
29 Some of the considerations will be relevant to multiple rows. 
 12 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
Focus of 
Considerations for Data Comparability 
Comparison 
source for the external control arm, and the potential for the outcome to be influenced 
by knowledge of treatment received.  In addition, sponsors should be able to apply 
the same criteria for the evaluation and timing of outcome assessments across both 
arms of the externally controlled trial. 
Missing data  The extent of missing data in the external control arm should be assessed before 
conducting an externally controlled trial to evaluate feasibility (when such data are 
available).  When analyzing results from such a trial, the extent of missing data in 
both the treatment and external control arms should be assessed to examine the 
potential impact of missing data. 
376   
377  The considerations listed in the table above are directed at understanding and managing potential 
378  threats to the validity of externally controlled trials.  Additional considerations regarding the 
379  comparability of trial arms may be relevant for a specific externally controlled trial.  
380   
381  C.  Analysis Considerations  
382   
383  1.  General Considerations  
384   
385  Before conducting an externally controlled trial, sponsors should develop a statistical analysis 
386  plan that prespecifies analyses of interest, such as analyses of primary and secondary endpoints, 
387  calculations of statistical power and sample size, and plans to control the chance of erroneous 
388  conclusions (e.g., to control the overall type I error probability).  The statistical analysis plan 
389  should be submitted along with the protocol to the relevant review division before initiation of 
390  enrollment in the clinical trial for the experimental treatment.  In addition, decisions regarding 
391  the study design and statistical analysis plan for an externally controlled trial should be blinded 
392  to any observed external control data (e.g., from an existing RWD source), with the exception of 
393  planned feasibility analyses, such as evaluating the availability of key variables or missing data.  
394  During the conduct of an externally controlled trial, and specifically when analyzing data already 
395  collected, changes to the statistical analysis plan are discouraged.  If such changes are 
396  nonetheless implemented, any revisions should be date-stamped and the corresponding rationale 
397  provided and discussed with the relevant FDA review division. 
398   
399  FDA does not recommend a particular approach to analyzing data from externally controlled 
400  trials.  No single statistical or analytical method will be suitable for all trials involving external 
401  control arms, and potential approaches should be discussed with the appropriate FDA review 
402  division.  Sponsors should provide a justification for the analytic methods selected as well as a 
403  description of the strengths and limitations of the methods used to assess the effect of treatment.  
404  In general, the analytic method used should identify and manage sources of confounding and 
405  bias, including a strategy to account for differences in baseline factors and confounding variables 
406  between trial arms. 
407   
408  Various statistical methodologies may be appropriate for these types of comparisons, each with a 
409  corresponding level of complexity regarding approaches to account for bias.  The assumptions 
410  involved should be made explicit, and sensitivity analyses as well as model diagnostics should be 
 13 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
411  conducted to examine such assumptions.  Importantly, however, adding complexity to an 
412  analytical framework usually requires making additional assumptions, which often cannot be 
413  substantiated and may impair the interpretability of results.  
414   
415  Even when employing analytic methods to balance the trial arm populations, sponsors should 
416  propose additional analyses to evaluate the actual comparability between the external control and 
417  treatment arms for important covariates.  Determining similarity across trial arms will require 
418  selection of specific population characteristics to compare, a method for the comparison, and 
419  criteria to demonstrate similarity.  For example, an a priori threshold30 could be set to determine 
420  whether the external control population has a statistical distribution of covariates that is similar 
421  to the treatment arm population after a balancing method, such as weighting, has been applied.  
422   
423  Consideration should also be given, based on available scientific data, to the anticipated effect 
424  size for analyses of the primary endpoint.  Especially when the anticipated effect size is modest, 
425  an externally controlled trial may not be an appropriate study design because of concerns for bias 
426  affecting the results.  In addition, sponsors should develop a priori plans for assessing the impact 
427  of confounding factors and sources of bias, with quantitative or qualitative bias analyses used to 
428  evaluate these concerns.  Such prespecified analyses can assist in the interpretation of study 
429  results.  
430   
431  2.  Missing Data 
432   
433  The proposed analytical methods should include a strategy for dealing with missing data, 
434  including data that may not be available in a chosen data source based on the type and frequency 
435  of assessments conducted during the patient encounter, patients no longer being followed, or 
436  other reasons.  Analytical methods (such as strategies for imputing missing data) may be used in 
437  such situations, but these methods require assumptions regarding the pattern of missing 
438  information.31  Assumptions about missing data can be unverifiable and may be difficult to 
439  justify, in addition to other assumptions required for estimation of treatment effect in a non-
440  randomized setting.  
441   
 
30 FDA does not endorse a single approach for determining thresholds.  As one example, a threshold value could be 
selected for standardized mean differences as a metric that summarizes the statistical distribution of important 
prognostic covariates. 
 
31 The terms missing completely at random, missing at random, and missing not at random describe assumptions 
about why data are missing.  When observations of a variable are missing completely at random, the missing 
observations are a random subset of all observations, such that the missing and observed values have the same 
underlying distributions, and bias from missing data is not a threat to the study.  Missing at random indicates 
systematic differences may exist between the observed and unobserved values of a variable, but other observed 
variables could be used to address such differences and mitigate bias.  Missing not at random indicates that the 
missing data are directly related to the treatment or outcome under investigation, and bias can be introduced.  See 
AR Donders, GJ van der Heijden, T Stijnen, and KG Moons, 2006, Review: A Gentle Introduction to Imputation of 
Missing Values, J Clin Epidemiol, 59(10):10871091. 
 
 14 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
442  To understand the potential impact of missing data, externally controlled trials should be 
443  designed to capture and analyze information relevant to the missing data (e.g., available 
444  characteristics of patients with and without missing data).  Analytical methods may be used, as 
445  mentioned above, to address potential bias caused by the missing data in the primary analysis.  In 
446  addition, sensitivity analyses should be used to evaluate the potential impact of plausible 
447  violations in missing data assumptions on the results of the primary and other key analyses.  
448   
449  In some cases, data may be missing because of an intercurrent event, which may interfere with 
450  the measurement of outcomes and estimation of the treatment effect.  The study analysis plan 
451  and an appropriate estimand should account for any intercurrent event that can be considered 
452  potentially related to both the treatment and outcome of interest, recognizing that certain 
453  intercurrent events may be difficult to detect in external control datasets.  For example, in 
454  contrast to data collected according to a research protocol, RWD sources may not capture the 
455  time of occurrence of an intercurrent event, precluding accurate assessment of time-to-event 
456  endpoints such as progression-free survival.  
457   
458  3.  Misclassification of Available Data 
459   
460  Misclassification32 (mischaracterization) of data in externally controlled trials, especially in an 
461  external control arm using RWD sources, can occur when the value of a measurement is assigned 
462  to an incorrect category for subsequent analysis, potentially affecting estimates of the observed 
463  drug-outcome association.  For example, EHR data collected during routine clinical care may 
464  include information on lifestyle characteristics, such as alcohol use.  Beyond concerns about 
465  potentially inaccurate reporting by patients about their alcohol intake because of stigma or other 
466  factors, differences in the approach used to classify alcohol use within or across various sources 
467  of data can lead to misclassification.  In routine clinical practice, for example, different health 
468  care providers may use different quantitative or qualitative descriptions of alcohol use, such that 
469  two patients with the same actual intake may be assigned to two different categories in the RWD 
470  source.  
471   
472  If misclassification is extensiveespecially when information on treatments, outcomes, or 
473  confounding factors are involveda biased assessment of the drug-outcome association may 
474  occur.  For example, the scenario described above regarding misclassification of alcohol intake 
475  would be relevant when alcohol use is a potentially important confounding factor (covariate) in 
476  an analysis of an externally controlled trial.  Although analytical modeling methods could be 
477  used to assess the potential impact of misclassification, the best strategy to avoid bias is to use 
478  objective and reliable measurements for the data of interest.  For example, RWD sources that 
479  include information on alcohol intake collected using structured questionnaires are generally 
480  more reliable than patient-reported and clinician-documented values obtained during routine 
481  patient care.  
 
32 Misclassification errors can be non-differential when the probability of misclassification is equal across study 
arms or differential when the probability of misclassification differs across study arms.  Misclassification can 
introduce bias regarding the drug-outcome association when involving the drug of interest, covariates, or outcomes 
of interest. 
 
 15 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
482   
483  4.  Additional Analyses 
484   
485  Sponsors can also use specific sensitivity analyses to test the vulnerability of trial results to 
486  assumptions in the analysis plan.  For example, if the primary analysis of a time-to-event 
487  endpoint assumes proportional hazards, an appropriate sensitivity analysis could be estimation by 
488  a statistical method that does not assume proportional hazards.  Finally, prespecified 
489  supplementary analyses can provide further understanding of the treatment effect.  An example 
490  would be supplementary analyses in prespecified subgroups based on prognostic factors for the 
491  outcome.  
492   
493   
494  IV.  CONSIDERATIONS TO SUPPORT REGULATORY REVIEW 
495   
496  A.  Communication with FDA 
497    
498  Sponsors should consult with the relevant FDA review division early in a drug development 
499  program about whether it is reasonable to conduct an externally controlled trial instead of a 
500  randomized controlled trial.  As part of these discussions, sponsors should provide a detailed 
501  description of the (1) reasons why the proposed study design is appropriate, (2) proposed data 
502  sources for the external control arm and an explanation of why they are fit for use, (3) planned 
503  statistical analyses, and (4) plans to address FDAs expectations for the submission of data. 
504   
505  B.  Access to Data and Documents 
506    
507  Sponsors must include in their marketing applications relevant patient-level data (i.e., data on 
508  each participant and patient in the externally controlled trial), as required under FDA 
509  regulations,33 for both the treatment and external control arms.  If sponsors do not own the data 
510  used for the external control arm, they should structure their agreements with the data owners to 
511  ensure that patient-level data can be provided to FDA in support of the marketing application. 
512  Sponsors should also ensure that FDA has access to source documents and source data for the 
513  external control arm as part of an FDA inspection or upon request.34   
 
33 See 21 CFR 314.50(f) and 601.2.  
 
34 See the guidances for industry Use of Electronic Health Record Data in Clinical Investigations (July 2018) and 
Electronic Source Data in Clinical Investigations (September 2013). 
 16 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
514  GLOSSARY 
515   
516  Bias:  Any systematic error in the design, conduct, analysis, or interpretation of a study that 
517  results in an erroneous estimate of a treatments effect on the outcome of interest.  
518   
519  Confounding:  Distortion of the measure of the effect of a treatment on an outcome due to 
520  another factor that is associated with both the treatment and the outcome.  
521   
522  Intercurrent Event:  An event occurring after treatment initiation that affects either the 
523  interpretation or the existence of the measurements associated with the clinical question of 
524  interest.  Examples include switching or discontinuing treatment, using rescue medications, or 
525  experiencing terminal events such as death.    
526   
527  Real-World Data (RWD):  Data relating to patient health status and/or the delivery of health 
528  care routinely collected from a variety of sources.  
529   
530  Real-World Evidence (RWE):  Clinical evidence about the usage and potential benefits or risks 
531  of a medical product derived from analysis of RWD. 
532   
533  Source Data:  All information in original records and certified copies of original records of 
534  clinical findings, observations, or other activities (in a clinical investigation) used for the 
535  reconstruction and evaluation of the study.  Source data are contained in source documents (i.e., 
536  original records or certified copies).35 
537   
538  Source Documents:  Original documents, data, and records (e.g., hospital records; clinical and 
539  office charts; laboratory notes; memoranda; subjects diaries or evaluation checklists; pharmacy 
540  dispensing records; recorded data from automated instruments; copies or transcriptions certified 
541  after verification as being accurate copies; microfiches; photographic negatives; microfilm or 
542  magnetic media; x-rays; subject files; and records kept at the pharmacy, at the laboratories, and 
543  at the medico-technical departments involved in the clinical trial).36 
 
35 See the guidance for industry Electronic Source Data in Clinical Investigations (September 2013). 
 
36 See the guidance for industry E6(R2) Good Clinical Practice:  Integrated Addendum to ICH E6(R1) (March 
2018). 
 17