Rare Diseases:  Natural 
History Studies for 
Drug Development 
Guidance for Industry 
 
 
 
DRAFT GUIDANCE
 
 
This guidance document is being distributed for comment purposes only. 
 
Comments and suggestions regarding this draft document should be submitted within 90 days of 
publication in the Federal Register of the notice announcing the availability of the draft 
guidance.  Submit electronic comments to https://www.regulations.gov.  Submit written 
comments to the Dockets Management Staff (HFA-305), Food and Drug Administration, 5630 
Fishers Lane, Rm. 1061, Rockville, MD 20852.  All comments should be identified with the 
docket number listed in the notice of availability that publishes in the Federal Register. 
 
For questions regarding this draft document, contact (CDER) Lucas Kempf at 301-796-1140; 
(CBER) Office of Communication, Outreach, and Development at 800-835-4709 or 240-402-
8010; or Office of Orphan Products Development (OOPD) at 301-796-8660. 
 
 
 
 
 
 
U.S. Department of Health and Human Services 
Food and Drug Administration 
Center for Drug Evaluation and Research (CDER) 
Center for Biologics Evaluation and Research (CBER) 
Office of Orphan Products Development (OOPD) 
 
March 2019 
Rare Diseases 
26420dft.docx 
3/1/2019   
Rare Diseases:  Natural 
History Studies for 
Drug Development 
Guidance for Industry 
 
 
Additional copies are available from: 
 
Office of Communications, Division of Drug Information 
Center for Drug Evaluation and Research 
Food and Drug Administration 
10001 New Hampshire Ave., Hillandale Bldg., 4th Floor  
Silver Spring, MD  20993-0002  
Phone: 855-543-3784 or 301-796-3400; Fax: 301-431-6353; Email: druginfo@fda.hhs.gov 
https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/default.htm 
and/or 
Office of Communication, Outreach, and Development  
Center for Biologics Evaluation and Research 
Food and Drug Administration 
10903 New Hampshire Ave., Bldg. 71, rm. 3128 
Silver Spring, MD  20993-0002 
Phone: 800-835-4709 or 240-402-8010; Email: ocod@fda.hhs.gov 
https://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guidances/default.htm 
and/or 
Office of Orphan Products Development 
Office of the Commissioner 
Food and Drug Administration 
10903 New Hampshire Ave, Bldg. 32, Room 5295 
Silver Spring, MD  20993-0002 
Phone: 301-796-8660; Email: orphan@fda.hhs.gov 
https://www.fda.gov/ForIndustry/DevelopingProductsforRareDiseasesConditions/default.htm 
 
 
 
U.S. Department of Health and Human Services 
Food and Drug Administration 
Center for Drug Evaluation and Research (CDER) 
Center for Biologics Evaluation and Research (CBER) 
Office of Orphan Products Development (OOPD) 
 
March 2019 
Rare Diseases 
   
TABLE OF CONTENTS  
 
I.  INTRODUCTION............................................................................................................. 1 
II.  BACKGROUND ............................................................................................................... 2 
III.  USES OF A NATURAL HISTORY STUDY ................................................................. 3 
A.  Drug Development ......................................................................................................................... 3 
1. Identifying the Patient Population ................................................................................................... 3 
2. Identification or Development of Clinical Outcome Assessments ................................................... 3 
3. Identification or Development of Biomarkers .................................................................................. 4 
4. Design of Externally Controlled Studies:  Use of Natural History Study Data ............................... 4 
B.  Other Uses ...................................................................................................................................... 6 
IV.  TYPES OF NATURAL HISTORY STUDIES ............................................................... 7 
A.  Retrospective and Prospective Natural History Studies............................................................. 7 
B.  Cross-Sectional Studies and Longitudinal Natural History Studies ......................................... 8 
1. Cross-Sectional Studies ................................................................................................................... 9 
2. Longitudinal Studies ........................................................................................................................ 9 
V.  STUDY PROTOCOL, DATA ELEMENTS, AND RESEARCH PLANS .................. 9 
A.  Study Protocol ................................................................................................................................ 9 
B.  Data Elements .............................................................................................................................. 10 
C.  Protocol Elements ........................................................................................................................ 10 
D.  Statistical Analysis Plan .............................................................................................................. 11 
E.  Practical Considerations for Study Design ................................................................................ 12 
1. Early Planning and Implementation .............................................................................................. 12 
2. When to Start a Natural History Study .......................................................................................... 12 
3. Finding Patients and Maintaining Their Involvement ................................................................... 12 
4. Study Site and Local Data Collection ............................................................................................ 13 
VI.  DATA COLLECTION, STORAGE, AND DISSEMINATION ................................. 13 
A.  Data Collection ............................................................................................................................. 14 
B.  Data Storage ................................................................................................................................. 15 
C.  Data Dissemination ...................................................................................................................... 15 
VII.  HUMAN SUBJECT PROTECTION ............................................................................ 15 
A.  Confidentiality of Subjects and Data Protection ....................................................................... 16 
B.  IRB Review ................................................................................................................................... 16 
C.  Informed Consent ........................................................................................................................ 17 
VIII.  INTERACTING WITH FDA ........................................................................................ 17 
REFERENCES ............................................................................................................................ 19 
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
1  Rare Diseases:  Natural History Studies  
2  for Drug Development 
3  Guidance for Industry1 
4   
5   
6   
7  This draft guidance, when finalized, will represent the current thinking of the Food and Drug 
8  Administration (FDA or Agency) on this topic.  It does not establish any rights for any person and is not 
9  binding on FDA or the public.  You can use an alternative approach if it satisfies the requirements of the 
10  applicable statutes and regulations.  To discuss an alternative approach, contact the FDA staff responsible 
11  for this guidance as listed on the title page. 
12   
13   
14   
15  I.  INTRODUCTION 
16   
17  This guidance is intended to help inform the design and implementation of natural history studies 
18  that can be used to support the development of safe and effective drugs and biological products 
19  for rare diseases.2  Although existing FDA guidance considers common issues encountered in 
20  drug development for rare diseases,3 this draft guidance expands on the subject of natural history 
21  studies specifically.  The focus of this guidance is rare diseases; however, the recommendations 
22  in the guidance may be applicable to drug development for nonrare diseases.  For applicability to 
23  nonrare diseases, discuss with the FDA review division or office responsible for the review of 
24  the drug. 
25   
26  This guidance describes the broad potential uses of a natural history study in all phases of drug 
27  development for rare diseases, the strengths and weaknesses of various types of natural history 
28  studies, data elements and research plans, and a practical framework for the conduct of a natural 
29  history study.  This guidance also discusses some considerations for aligning the study design 
30  with study objectives and for enhancing the interpretability of study results; patient 
31  confidentiality and data protection issues in natural history studies; and potential interactions 
32  with FDA related to these studies.   
33   
                                                 
1 This guidance has been prepared by the Office of New Drugs, Rare Diseases Program, and the Office of 
Translational Sciences in the Center for Drug Evaluation and Research in cooperation with the Center for Biologics 
Evaluation and Research and the Office of Orphan Products Development, Office of the Commissioner, at the Food 
and Drug Administration.  
 
2 For the purposes of this guidance, all references to drugs include both human drugs and biological products unless 
otherwise specified. 
 
3 See the draft guidance for industry Rare Diseases:  Common Issues in Drug Development (January 2019).  When 
final, this guidance will represent the FDAs current thinking on this topic.  For the most recent version of a 
guidance, check the FDA guidance web page at https://www.fda.gov/RegulatoryInformation/Guidances/default.htm.  
 
1  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
34  In general, FDAs guidance documents do not establish legally enforceable responsibilities.  
35  Instead, guidances describe the Agencys current thinking on a topic and should be viewed only 
36  as recommendations, unless specific regulatory or statutory requirements are cited.  The use of 
37  the word should in Agency guidances means that something is suggested or recommended, but 
38  not required. 
39   
40   
41  II.  BACKGROUND 
42   
43  Section 526(a)(2)(A) of the Federal Food, Drug, and Cosmetic Act (FD&C Act) defines a rare 
44  disease, in part, as a disease or condition that affects less than 200,000 persons in the United 
45  States.4  There are approximately 7,000 recognized rare diseases.  Individually, each rare 
46  disease affects a small number of people, but cumulatively rare diseases affect about 1 in 10 
47  people in the United States.  Most rare diseases have no approved therapies, and thus, overall, 
48  this presents a significant unmet public health need.  
49   
50  The natural history of a disease is traditionally defined as the course a disease takes in the 
51  absence of intervention in individuals with the disease, from the diseases onset until either the 
52  diseases resolution or the individuals death.  A natural history study5 is a preplanned 
53  observational study intended to track the course of the disease.  Its purpose is to identify 
54  demographic, genetic, environmental, and other variables (e.g., treatment modalities, 
55  concomitant medications) that correlate with the diseases development and outcomes.  Natural 
56  history studies are likely to include patients receiving the current standard of care and/or 
57  emergent care, which may alter some manifestations of the disease.  Disease registries are a 
58  frequent platform to acquire the data for natural history studies.   
59   
60  Knowledge of a diseases natural history is important for planning drug development; however, 
61  there is only limited information about the natural history of most rare diseases.  In the following 
62  sections, this guidance describes major roles of natural history studies in planning controlled 
63  trials of investigational drugs to treat rare diseases.  It also touches briefly on the potential use of 
64  natural history data as an external6 control in a clinical trial, but not as the primary focus of this 
65  guidance.   
66   
                                                 
4 In addition, section 526(a)(2)(B) of the FD&C Act also defines a rare disease as any disease or condition that 
affects more than 200,000 in the United States and for which there is no reasonable expectation that the cost of 
developing and making available in the United States a drug for such disease or condition will be recovered from 
sales in the United States of such drug. 
 
5 See the National Cancer Institute Dictionary of Cancer Terms, available at 
https://www.cancer.gov/publications/dictionaries/cancer-terms/def/natural-history-study.   
 
6 The regulation at 21 CFR 314.126 uses the term historical control, which is a subset of external control.  An 
externally controlled trial compares a group of subjects receiving the test treatment with a group of patients external 
to the trial, rather than to an internal control group consisting of patients from the same population assigned to a 
different treatment.  The external control can be a group of patients treated at an earlier time (historical control) or a 
group treated during the same time period but in another setting.  See the ICH guidance for industry E10 Choice of 
Control Group and Related Issues in Clinical Trials (May 2001).  This guidance uses the term external control, 
except when referring to section 314.126. 
2  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
67   
68  III.  USES OF A NATURAL HISTORY STUDY 
69   
70  Information obtained from a natural history study can play an important role at every stage of 
71  drug development, from drug discovery to the design of clinical studies intended to support 
72  marketing approval of a drug and beyond into the postmarketing period. 
73   
74  A.  Drug Development 
75   
76  Comprehensive knowledge of a disease can help sponsors design and conduct adequate and well-
77  controlled clinical trials of adequate duration with clinically meaningful endpoints to support 
78  marketing applications for new drugs.  The following sections highlight important contributions 
79  of a natural history study to the clinical development program. 
80   
81  1.  Identifying the Patient Population 
82   
83  Some rare diseases have substantial genotypic and/or phenotypic heterogeneity, and the natural 
84  history of each subtype may be poorly understood or inadequately characterized.  For example, 
85  different phenotypes may present with involvement of different organ systems, with different 
86  severity or rate of deterioration.  A natural history study may uncover sentinel events or 
87  detectable physiologic changes that are important predictors of disease progression or that are 
88  clinically important in their own right.  A well-designed natural history study may be useful in 
89  understanding which patient subgroup(s) may benefit from a particular drug trial.  The 
90  information about subtype signs and symptoms and rates and patterns of progression are useful 
91  in deciding the inclusion criteria, the stage of disease to treat, the duration of a trial, the 
92  frequency of data collection, and the specific endpoints.   
93   
94  2.  Identification or Development of Clinical Outcome Assessments  
95   
96  A clinical outcome assessment is an assessment that describes or reflects how an individual feels, 
97  functions, or survives.  Clinical outcome assessments can be used during trials to assess the 
98  efficacy and safety of a drug.  There are four types of clinical outcome assessments (FDA-NIH 
99  Biomarker 2017): 
100   
101    Clinician-reported outcome 
102    Observer-reported outcome (e.g., reports by or from caregivers) 
103    Patient-reported outcome 
104    Performance outcome (e.g., tests of memory or walking ability)  
105   
106  A natural history study can help evaluate the ability of a new or existing clinical outcome 
107  assessment to detect change in a particular disease or a pattern of progression of a disease or 
108  symptoms of disease.  Natural history studies also can be used to evaluate the performance and 
109  reproducibility of a clinical outcome assessment for use in a clinical investigation.   
110   
111  We recommend that input is obtained from clinicians with expertise in caring for patients with 
112  the target rare disease, patients, caregivers, regulatory agencies, and experts in clinical outcome 
3  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
113  assessment measurement to ensure that the selected clinical outcome assessments are fit for 
114  regulatory use and are valid assessments of the important and relevant aspects of the disease.    
115   
116  3.  Identification or Development of Biomarkers  
117   
118  In general, the term biomarker refers to a characteristic that is objectively measured and 
119  evaluated as an indicator of normal biological processes, pathologic processes, or biological 
120  responses to a therapeutic intervention (see, for example, section 507(e)(1) of the FD&C Act, 
121  FDA-NIH Biomarker 2017).  Biomarkers include physiological measurements, blood tests and 
122  other chemical analyses of tissue or bodily fluids, genetic or metabolic data, and measurements 
123  from images (Institute of Medicine Committee 2010).  A natural history study can help identify 
124  or develop biomarkers that can be diagnostic of the disease, prognostic of the diseases course, 
125  predictive of treatment response, or useful in guiding patient selection and dose selection in drug 
126  development programs.7  
127   
128  Natural history studies provide an opportunity to collect specimens and images that can be used 
129  in an analytical validation program.  When robustly validated, these biomarkers can serve as 
130  endpoints or surrogate endpoints in clinical trials.   
131   
132  4.  Design of Externally Controlled Studies:  Use of Natural History Study Data 
133   
134  To qualify for marketing approval, an application submitted under section 505(b) of FD&C Act 
135  must, among other things, be supported by investigations showing the drug to be safe and 
136  effective under the conditions prescribed, recommended, or suggested in the product labeling and 
137  demonstrate a favorable benefit-risk profile in the specified patient population.8  To demonstrate 
138  effectiveness, sponsors must provide substantial evidence from adequate and well-controlled 
139  investigations, including clinical investigations,9 that include (among other factors) a valid 
140  comparison to a control.10  The sponsor uses data collected from an adequate control group to 
141  discriminate patient outcomes caused by the investigational drug from outcomes caused by other 
142  factors (i.e., what would have happened if similar patients had not received the investigational 
143  drug).  FDA regulations recognize historical controls as a possible control group (usually 
144  reserved for special circumstances); however, inability to control for certain biases could limit 
                                                 
7 See the guidance for industry and FDA staff Qualification Process for Drug Development Tools (January 2014).  
We update guidances periodically.  To make sure you have the most recent version of a guidance, check the FDA 
guidance web page at https://www.fda.gov/RegulatoryInformation/Guidances/default.htm.  
 
8 Section 505(d) of the FD&C Act (21 U.S.C. 355). 
 
9 Section 505(d) of the FD&C Act (21 U.S.C. 355).  FDA has also generally considered substantial evidence of 
effectiveness to be necessary to support licensure of a biological product under section 351 of the Public Health 
Service Act (PHS Act).  For a biological product to be licensed under section 351 of the PHS Act, a sponsor must 
demonstrate that its product is safe, pure, and potent.  Potency has long been interpreted to include effectiveness (21 
CFR 600.3(s)). 
 
10 The characteristics of an adequate and well-controlled investigation are detailed under FDA regulations at 21 CFR 
314.126.   
 
4  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
145  the ability of externally controlled trials to demonstrate substantial evidence of effectiveness.11  
146  However, bias may be mitigated in certain situations where the disease course is predictable and 
147  the treatment effect dramatic.  In some cases where the natural history data exist and are part of 
148  the general medical knowledge of the disease course, a baseline control study design can be used 
149  because the pathophysiology is well understood (e.g., tumors do not shrink in the absence of 
150  treatment; tumors are known to have a high probability of progression in a defined time period).  
151  In other cases, data and information from a natural history study may provide an untreated, 
152  external control group for use as the comparator to the treatment group(s) in an investigational 
153  drug trial.   
154   
155  The use of external controls requires careful planning and assessment, including the following 
156  considerations: 
157   
158    The external control group needs to be very similar to the treated group in all respects, 
159  including disease severity, duration of illness, prior treatments, and any other aspects of 
160  the disease that could affect outcomes and the timing of outcomes.  The availability of 
161  patient level data12 can help provide support for comparison between the control group 
162  and the group receiving the investigational drug.    
163   
164    Use of valid epidemiological approaches can reduce selection bias (e.g., 
165  inclusion/exclusion criteria, prespecified statistical analysis plan) (Ellenberg 1994).13  
166  Selection bias is a major concern when using external controls because there is no 
167  randomization and unrecognized baseline differences can affect outcomes.  Points to 
168  consider include the following: 
169   
170    Critical patient disease characteristics may not have been assessed or may have been 

171  assessed differently based on historical approaches, resulting in a lack of 
172  comparability (e.g., disease definitions, diagnostic techniques, and approaches to 
173  safety monitoring may have evolved).   
174   
175    Aspects of standard of care may have changed.  

176   
177    Data collection intervals and quality may lack consistency and not be comparable. 

178   
                                                 
11 See 21 CFR 314.126(b)(2)(v). 
 
12 Real-world data (i.e., data relating to patient health status and /or the delivery of health care that is routinely 
collected from a variety of sources) may be useful to collecting data for natural history studies.  See Framework for 
FDAs Real-World Evidence Program available at 
https://www.fda.gov/downloads/ScienceResearch/SpecialTopics/RealWorldEvidence/UCM627769.pdf. 
 
13 See the draft guidance for industry, FDA staff, and other stakeholders Patient-Focused Drug Development:  
Collecting Comprehensive and Representative Input (June 2018).  When final, this guidance will represent the 
FDAs current thinking on this topic.   
 
5  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
179    Use of an external control group is especially challenging if the outcome assessments 
180  used in the external control group are not well defined and reliable and, therefore, not 
181  suitable for regulatory use.   
182   
183  There are two types of external controls that provide varying strengths of evidence.14  
184  Nonconcurrent external controls15 consider the subject-level data from a different group 
185  (external) of subjects followed in the past for whom the individual subject-level data are 
186  available for the same outcomes and same covariates as in the current trial.  For example, 
187  subject-level data may be obtained from the comparator group from a prior clinical trial (e.g., 
188  placebo group) or a natural history study.  The stronger concurrent16 external control design 
189  considers subject-level data collected at the same time as the group being treated in the clinical 
190  trial.  However, in contrast with a completed natural history study, a concurrent control arm may 
191  not provide timely advice for planning the clinical trials. 
192   
193  Regardless of external control type, even for diseases with relatively predictable progression, an 
194  external control is most interpretable when a treatment effect:  (1) is large in comparison to 
195  potential biases and the known variability in progression,17 (2) is not affected by patient or 
196  investigator motivation or choice of subjects for treatment,18 (3) can be objectively measured, (4) 
197  is measured in a manner that reasonably manages and minimizes bias, (5) has a strong temporal 
198  association with administration of the investigational drug, and (6) is consistent with expected 
199  pharmacological activity based on the target and perhaps shown in animal models.  The pros and 
200  cons of various controls are discussed at length in the ICH guidance for industry E10 Choice of 
201  Control Group and Related Issues in Clinical Trials (ICH E10).  While not discussed in ICH 
202  E10, a hybrid approach of using external control data to add to a concurrent randomized control 
203  arm in a clinical trial may sometimes be useful.  
204   
205  B.  Other Uses 
206   
207  The benefits of planning, organizing, and implementing a natural history study may go beyond 
208  drug development.  A natural history study may benefit patients with rare diseases by 
209  establishing communication pathways, identifying disease-specific centers of excellence, 
210  facilitating the understanding and evaluation of the current standard of care practices, and 
                                                 
14 See ICH E10.  
 
15 For examples of nonconcurrent external control studies, see the Kanuma (sebelipase alfa) label (available at 
https://www.accessdata.fda.gov/drugsatfda_docs/label/2015/125561s000lbl.pdf) and the Brineura (cerliponase alfa) 
label (available at https://www.accessdata.fda.gov/drugsatfda_docs/label/2017/761052lbl.pdf).  
 
16 A concurrent control group is defined in ICH E10 as one chosen from the same population as the test group and 
treated in a defined way as part of the same trial that studies the test treatment, and over the same period of time.  
The test and control groups should be similar with regard to all baseline and on-treatment variables that could 
influence outcome, except for the study treatment.  
 
17 See ICH E10. 
 
18 A classic comparison of the results of randomized and externally controlled trials of a variety of treatments 
showed that externally controlled trials almost always showed a better effect, probably because the new treatment 
tended to be given to patients with a better prognosis (Sacks et al. 1982). 
6  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
211  identifying ways to improve patient care.  A natural history study may provide demographic data 
212  and epidemiologic estimates of the prevalence of the disease and disease characteristics and aid 
213  disease tracking.   
214   
215   
216  IV.  TYPES OF NATURAL HISTORY STUDIES 
217   
218  Natural history studies can be designed to collect data from case histories or ongoing clinical 
219  visits in a cross-sectional or longitudinal manner depending on the desired purpose.    
220   
221  A.  Retrospective and Prospective Natural History Studies 
222   
223  Retrospective and prospective natural history studies differ in the time at which patients are 
224  evaluated relative to when the study is planned and initiated.  In retrospective studies, the patient 
225  evaluations have already occurred.  In prospective studies, the evaluations occur in the future 
226  according to a prespecified data collection plan that may reflect current data standards.   
227   
228  Retrospective studies are often used as first steps in collecting natural history information.  This 
229  information is reviewed from existing medical records, such as patient charts, which were 
230  compiled for patient care rather than for use in a natural history study.  Retrospective study 
231  designs are informed by reviews of the following:  published scientific literature; the opinions 
232  and experience of disease experts; and other sources of information, such as data collected 
233  directly from patients (whether published or not).  These studies can collect and organize 
234  important information about a disease and identify information gaps that may be addressed by 
235  prospective data collection and analysis.  Because the data are already available, retrospective 
236  natural history studies may be performed more quickly than prospective natural history studies. 
237   
238  Retrospective natural history studies may be limited by several factors that affect their utility, 
239  including the following:  
240   
241    Data elements may not have been collected in existing records. 
242   
243    Data elements may lack comparability to more recently treated patients because the data 
244  elements were collected at variable time points or were obtained inconsistently.   
245   
246    Medical terminology may have changed over time or have been used inconsistently 
247  among health care providers and data collection sites.   
248   
249    Specialty clinics providing historical data may result in patient selection or referral bias 
250  (e.g., including only the most severely affected patients in a natural history study). 
251   
252    The patients medical record or longitudinal profile may not be sufficient to identify the 
253  onset of the disease or symptom.  
254   
255    If current patients from a database are used to select for study, those patients who have 
256  been in the database the longest may be overrepresented and the study may 
7  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
257  disproportionately fail to capture patients who enter and rapidly leave the database.  This 
258  is often called length-biased sampling (Delgado and Llorca 2004).  
259   
260    Natural history studies published in the medical literature may be biased toward reporting 
261  on patients with increased severity of illness or on successful outcomes.  Thus, literature 
262  reviews may not be adequate substitutes for natural history studies because literature 
263  reviews often do not characterize the full spectrum of the disease.  
264   
265    Retrospective natural history studies can be biased through patient selection criteria and 
266  through selection of dates of inception and cutoff. 
267   
268  These factors can result in variable and incomplete information about the disease and may limit 
269  the interpretability and generalizability of the available information.  Importantly, these factors 
270  can render retrospective natural history studies susceptible to bias. 
271   
272  Prospective studies can address many of the limitations encountered in the retrospective 
273  approach by, for example, doing the following: 
274   
275    Implementing standard, consistent, and up-to-date definitions of medical conditions and 
276  treatments.  These elements employ uniform medical language and are typically provided 
277  in advance in study protocols and procedure manuals.   
278     
279    Providing a consistent schedule of medical visits for the patient. 
280   
281    Providing standard operating procedures for investigators, which provides for greater 
282  consistency in the information collected (e.g., using the same clinical outcome 
283  assessments with comparable instructions for use). 
284   
285    Collecting additional data that may elucidate the pathogenesis and manifestations of a 
286  disease and the patients concomitant treatments. 
287   
288  However, prospective natural history studies will generally require more time, depending on 
289  needed duration of observation, than the collection of existing data, particularly for longitudinal 
290  studies (see section IV.B., Cross-Sectional Studies and Longitudinal Natural History Studies). 
291   
292  B.  Cross-Sectional Studies and Longitudinal Natural History Studies 
293   
294  In cross-sectional studies, data are collected from across a cohort of patients during a specified, 
295  limited time period, but in longitudinal studies, data are collected from patients at several points 
296  over time.  Either of these studies may be retrospective or prospective.  In general, data 
297  collection and analysis in a cross-sectional study take less time than a longitudinal study.  
298  Although data from cross-sectional studies may not be well suited to be used as an external 
299  control group in a clinical trial, the data may provide information that could be used to plan a 
300  future study. 
301   
8  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
302  1.  Cross-Sectional Studies 
303   
304  Cross-sectional studies collect and analyze data from a specified, limited time period (i.e., a 
305  specific date range or often a single point in time).  Cross-sectional data can be of value in drug 
306  development for a rare disease for reasons that include the following: 
307   
308    The general course of a particular disease may be inferred by sampling a cohort of 
309  patients at various stages of the disease 
310   
311    Studies can provide a description of the range and severity of manifestations of the 
312  disease and methods used to evaluate these manifestations  
313   
314    Studies can provide information for therapies intended to provide largely immediate 
315  benefits when given to patients with an acute episode or flare of the disease (e.g., sickle 
316  cell crises, rare venomous snake bites).  
317   
318  Although cross-sectional studies offer a quick and effective way to survey a current patient 
319  population, data collected from a specified, limited time period may not fully characterize the 
320  disease course and identify subtypes, including rapidly progressive subtypes that may be less 
321  well characterized because of length-biased sampling.    
322   
323  2.  Longitudinal Studies 
324   
325  Unlike cross-sectional studies that collect data at a specific time period, longitudinal studies 
326  collect data from all patients in a cohort over several time points.  Longitudinal natural history 
327  studies typically yield more comprehensive information about disease onset and progression over 
328  time than cross-sectional studies, and therefore longitudinal studies tend to be more useful as a 
329  source of natural history information.  In addition, longitudinal natural history studies are usually 
330  a better method to distinguish the variety of phenotypes and subgroups of a disease, especially in 
331  diseases with intermittent, variable, or unpredictable courses.  Longitudinal studies may also be 
332  useful to identify prognostic factors (e.g., to distinguish slow progressors from fast progressors) 
333  for the rare disease, particularly when the onset of the disease is difficult to identify.  The chief 
334  limitation of prospective longitudinal studies is that they typically require more time to conduct 
335  than cross-sectional studies and, therefore, are more resource intensive.   
336   
337   
338  V.  STUDY PROTOCOL, DATA ELEMENTS, AND RESEARCH PLANS 
339   
340  A.  Study Protocol 
341   
342  Natural history studies should have well-defined, carefully documented protocols, completed 
343  before initiation of the study.  These study protocols delineate who should be included in the 
344  study (inclusion and exclusion criteria), the information to be collected, how it is to be collected, 
345  the schedule for the data collections (if prospective), and the plan for analysis.  
346   
9  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
347  B.  Data Elements 
348   
349  When collecting natural history information, all of the potential uses of the information should 
350  be considered, including those uses pertinent to drug development.  FDA has specific data 
351  standards and terminology recommendations for marketing applications.19  Therefore, natural 
352  history data that will be used to support a marketing application should be collected according to 
353  these data standards.  Because rare disease drug development may take place in multiple 
354  countries, international data standards should also be considered.   
355   
356  C.  Protocol Elements 
357   
358  A prespecified natural history study protocol should include the following: 
359   
360    A description of methods for data collection. 
361   
362    Disease definition and diagnostic criteria for entry into the study and rationale.  
363   
364    List of demographic information to be collected.   
365   
366    A list of disease related information to collect including:  
367   
368    Signs and symptoms. 

369   
370    Age at onset of symptoms, age at diagnosis, and age at development of important 

371  morbidities and mortality. 
372   
373    Measures that can assess the severity and nature of involvement of the disease for 

374  potentially affected body systems.  The natural history study data should not be 
375  limited to the most severely affected body systems because treatment responses might 
376  be more reliably detected by evaluation of a less affected body system.    
377   
378    Documented genotypes and phenotypic features, which may be important in 

379  identifying disease subpopulations.  
380   
381    Clinically meaningful disease effects and outcomes including a focus on those 

382  important to patients and their families. 
383   
384    A description of any regional treatment guidelines or algorithms, including any changes 
385  in standard of care over time, if applicable. 
386   
387    Analytical plan. 
388   
                                                 
19 See the FDAs Study Data Standards Resources web page at 
https://www.fda.gov/forindustry/datastandards/studydatastandards/default.htm.  
10  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
389    A provision to record when protocol revisions are made and if that would alter the 
390  analysis.   
391   
392    Study duration. 
393   
394    Date of inception. 
395   
396    Date of cutoff. 
397   
398  Prospective protocols may also include the following: 
399   
400    Description of the data element to be collected, methods and procedures of measurement, 
401  and schedule of collection 
402   
403    Standardized procedures for evaluating patients including procedures for those that leave 
404  prematurely   
405   
406    Methods used for standardizing inter- and intra-rater reliability for clinical outcome 
407  assessments and performance requirements for biomarker measurement tests including 
408  reproducibility when multiple labs or testing sites are involved 
409   
410    An analytical plan including a plan for how protocol deviations and drop-outs will be 
411  considered in the analysis 
412   
413  D.  Statistical Analysis Plan 
414   
415  For natural history studies to be most informative, the Agency recommends that natural history 
416  studies have a prospectively defined statistical analysis plan (SAP).20  The ICH guidance for 
417  industry E9 Statistical Principles for Clinical Trials (September 1998) was written for 
418  interventional trials, but many of the elements described are of use when considering the 
419  development and use of natural history studies.  The Agency recommends the involvement of a 
420  statistician as part of the natural history study planning committee.  The SAP elements should 
421  delineate the analysis population, definition of endpoints, descriptive objectives, testable 
422  hypotheses and statistical methods to be employed in analysis of the data including the timing of 
423  the data analyses conducted in the study.  The SAP should include enough detail so that the 
424  analysis results can be replicated.  The SAP can also increase the studys efficiency by focusing 
425  on the most relevant data to be collected without imposing excessive rigidity (Thomas and 
426  Peterson 2012).  Preplanned interim analyses at certain intervals or milestones may suggest 
427  design changes to the protocol.  Protocol elements may be modified or dropped for reasons of 
428  relevancy, feasibility, and reliability based on interim analyses, but any such changes should be 
                                                 
20 In the ICH guidance for industry E9 Statistical Principles for Clinical Trials (September 1998), a SAP is defined 
as a document that contains a more technical and detailed elaboration of the principal features of the analysis 
described in the protocol, and includes detailed procedures for executing the statistical analysis of the primary and 
secondary variables and other data.  See also ICH E10. 
 
11  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
429  well documented as an amendment to the protocol, including the timing and rationale for the 
430  changes.    
431   
432  In any natural history study, consistency of procedures and data collection across data collection 
433  sites and across time is critical.  The analysis model may also need to make adjustments for the 
434  effects of sites within the country or region.  A natural history study that collects data in widely 
435  dispersed site locations needs to consider potential language and cultural differences in the 
436  patient perceptions, manifestations, and effects of a disease.21  Evaluation of intra- and inter-rater 
437  reliability of clinical outcome assessments and performance requirements of the biomarker 
438  measurement assays/tests should be considered.  
439   
440  E.  Practical Considerations for Study Design 
441   
442  1.  Early Planning and Implementation 
443   
444  The starting point of a natural history study is the collection, organization, and analysis of all 
445  currently available data.  These data may come from and be reviewed by a planning committee 
446  comprised of diverse stakeholder representatives such as patients and advocates, treating 
447  physicians, other health care providers, researchers, investigators, and drug developers.  The 
448  planning committee can consider the data to be collected, the need for potential adjustments to 
449  the natural history study, and the potential uses of the information obtained from the natural 
450  history study.  Natural history studies should include plans to formally monitor study conduct 
451  and approaches to make protocol adjustments when warranted. 
452   
453  2.  When to Start a Natural History Study 
454   
455  For many rare diseases, early initiation of a natural history study (even before an investigational 
456  therapeutic drug has been identified) can provide benefit by allowing time to collect data 
457  including a longer duration and larger patient population.  However, natural history studies need 
458  not delay drug development or delay approval of a needed treatment if drug development is 
459  already under way.  For some diseases, there might be adequate information available for 
460  planning and initiation of a drug development program; however, data obtained from a natural 
461  history study may contribute additional information.   
462   
463  Because natural history studies often face an array of unknowns, a small pilot study may be 
464  valuable at any stage of the natural history study process.  Pilot studies help clarify what data 
465  elements to collect, how to code the data, and how to standardize the information collection in a 
466  way to facilitate analysis (see section V.B., Data Elements).  For prospective studies, a pilot 
467  study can refine study procedures, logistics, and data collection. 
468   
469  3.  Finding Patients and Maintaining Their Involvement 
470   
471  For rare diseases, finding patients for inclusion in a natural history study can be a challenge and 
472  frequently requires participation of many sites across the United States or sites from multiple 
                                                 
21 See the ICH guidances for industry E5 Ethnic Factors in the Acceptability of Foreign Clinical Data (June 1998) 
and E17 General Principles for Planning and Design of Multi-Regional Clinical Trials (July 2018) (ICH E17). 
12  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
473  countries.  A retrospective literature review might identify referral centers or individual 
474  specialists that can help identify patients.  Consideration should be given to enlisting the help of 
475  disease-specific support groups or patient advocacy groups because they are invaluable resources 
476  for identifying and helping to recruit patients.  They also can contribute to study design and 
477  execution because of their unique perspectives.  Natural history studies can be registered in 
478  https://wwww.ClinicalTrials.gov to also increase participation and recruitment.  
479   
480  Patients continuing study participation ensures the robustness of follow-up data.  Patient 
481  advocacy or support groups can make an important contribution in keeping the patient 
482  community interested and engaged and in providing valuable perspectives both on minimizing 
483  burdens to patients and families and on the acceptability of proposed investigations.  
484  Importantly, to minimize missing data and to enhance study quality and interpretability, the 
485  reasons participants drop out of the study or choose not to participate at all should be 
486  investigated and addressed.  Approaches to increase patient participation may include providing 
487  support for travel and lodging expenses, issuing a study newsletter, and sharing the interim 
488  results of the study with the participants/community. 
489   
490  4.  Study Site and Local Data Collection 
491   
492  Natural history study data may be collected by various means and in a variety of locations.  
493  Specialty medical centers may have expertise and testing equipment for making medical 
494  diagnoses and performing clinical and laboratory measurements.  Data may be collected from 
495  patients or observers in the patients homes or by the local health care provider in person or 
496  remotely.  Remote data collection may be of special value for geographically dispersed patients 
497  with rare diseases.  Local observations (e.g., lab test results) may be sent to a centralized study 
498  center.  When multiple laboratories are used, each should be qualified for the study testing 
499  analysis and reporting; this is of particular importance for laboratory tests that are key to disease 
500  diagnosis or monitoring of disease manifestations.  Alternatively, to decrease variability, a single 
501  central laboratory can analyze samples.  Increased patient convenience leads to larger numbers of 
502  patients being able to participate in the study.  In international studies, some countries may 
503  restrict sending samples outside their borders, so a determination should be made if this is an 
504  issue and, if so, plans to address this issue should be made.  
505   
506  The tradeoffs between (1) the less convenient location for patients (at centralized facilities) 
507  potentially offering better standardization of data collection and (2) the more convenient location 
508  for patients (e.g., local doctors office, patients homes) potentially offering less standardized 
509  data collection should be carefully explored.  The particular approach used may need to be 
510  adjusted to collect data as understanding of the diseases natural history increases. 
511   
512   
513  VI.  DATA COLLECTION, STORAGE, AND DISSEMINATION 
514   
515  If the natural history study data are intended to provide essential support for a drug application 
516  (e.g., as a potential external control for comparison to patients treated with an investigational 
517  drug), FDA will likely find it necessary to have access to patient-level data and to evaluate the 
13  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
518  natural history data in detail.22  When the natural history data and information are used as an 
519  external control in a clinical trial, FDAs regulations covering investigational new drug 
520  applications under 21 CFR 312 may apply.23  The ICH guidance for industry E6(R2) Good 
521  Clinical Practice:  Integrated Addendum to ICH E6(R1) (March 2018) may provide additional 
522  information about data collection standards.   
523   
524  Even if the data from a natural history study are not to be used as essential support for a 
525  marketing application, the quality and integrity of the data obtained in these studies will be 
526  important for effective use of the study results.  
527   
528  A.  Data Collection24 
529   
530  Particularly for international studies, natural history studies should code data from patient 
531  experiences using a vocabulary that is internationally interpretable, is standardized for all 
532  participating health care providers, and is easily translatable to a database for analytical 
533  purposes.  FDA encourages the use of available standardized data dictionaries, terminology, and 
534  common data elements (e.g., Medical Dictionary for Regulatory Activities (MedDRA)).25  FDA 
535  encourages natural history study data to be coded using the same data standards they plan to use 
536  for the clinical trial data. 
537   
538  With the increasing use of electronic health records, standards development is evolving rapidly, 
539  and FDA encourages the use of data standards that are either government supported or produced 
540  by a standards developing organization (e.g., Systematized Nomenclature of Medicine  
541  Clinical Terms (SNOMED CT),26 Logical Observation Identifiers Names and Codes (LOINC),27 
542  Clinical Data Interchange Standards Consortium Study Data Tabulation Model (CDISC 
543  STDM)28).  
544   
                                                 
22 See 21 CFR 314.50(f). 
 
23 Researchers may discuss the applicability of these regulations to a specific natural history study with the 
appropriate review division. 
 
24 See the FDA Resources for Data Standards web page at 
https://www.fda.gov/forindustry/datastandards/default.htm.  
 
25 See also the guidance for industry Providing Regulatory Submissions in Electronic Format  Certain Human 
Pharmaceutical Product Applications and Related Submissions Using the eCTD Specifications (April 2018). 
 
26 SNOMED CT is one of a suite of designated standards for use in U.S. federal government systems for the 
electronic exchange of clinical health information and is a required standard interoperability specification of the 
Healthcare Information Technology Standards Panel. 
 
27 LOINC is one of a suite of designated standards for use in U.S. federal government systems for the electronic 
exchange of clinical health information and has been identified by the Health Level Seven International standards 
development organization as a preferred code set for laboratory test names in health information transactions.    
 
28 CDISC defines STDM as a standard structure for human clinical trial study data tabulations.   
 
14  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
545  B.  Data Storage 
546   
547  For maximal usefulness of the data, it is important to ensure from the outset of the natural history 
548  study that the data are maintained securely and will be accessible and stable for the duration of 
549  the study and its analyses.  A robust documentation and systematic auditing system to allow for 
550  traceability may be valuable for later regulatory purposes.  FDA encourages adherence to current 
551  recommendations for data security and engagement of technical experts in the use and 
552  availability of hardware and software programs for storing and accessing natural history study 
553  data.29  
554   
555  C.  Data Dissemination 
556   
557  Because of the general lack of clinical data available in rare diseases, FDA encourages 
558  dissemination of information as widely as possible (e.g., through peer-reviewed publications) on 
559  the methods used to conduct the natural history study, the practical aspects of conducting the 
560  study (including the studys limitations), and the results of the study in full consideration of any 
561  patient confidentiality and intellectual property rights issues.  A dissemination plan should be 
562  considered at the beginning of the study and, as feasible, with the participation of all of the 
563  interest groups.  
564   
565   
566  VII.  HUMAN SUBJECT PROTECTION 
567   
568  For all research studies, adequate provisions should be in place to ensure the privacy of patients 
569  and to maintain the confidentiality of data.  Those planning natural history studies should make 
570  detailed assessments of all regulatory requirements that may be applicable.    
571   
572  Natural history studies may be subject to several federal regulations designed to protect the 
573  rights, safety, and welfare of human subjects.  The core of FDAs human subject protection 
574  regulations is found at 21 CFR part 50 (Protection of Human Subjects) and 21 CFR part 56 
575  (Institutional Review Boards).  The FDA regulations at 21 CFR part 50 outline the requirements 
576  related to informed consent, and the FDA regulations at 21 CFR part 56 outline the requirements 
577  related to institutional review board (IRB) operations. 
578   
579  Natural history studies may be subject to FDA regulations at 21 CFR parts 50 and 56 if they 
580  meet the definitions of clinical investigation and other applicable definitions under those parts.30  
581  Other regulations may also apply, particularly the regulations of the U.S. Department of Health 
582  and Human Services (HHS) governing the protection of human subjects found at 45 CFR part 46 
583  (often referred to as the Common Rule).  These regulations apply to all nonexempt research 
584  involving human subjects that is conducted, supported, or otherwise subject to regulation by 
                                                 
29 Some current recommendations for data security are in the Health Insurance Portability and Accountability Act 
privacy and security rules (45 CFR part 160, subpart A and part 164, subpart C). 
 
30 See 21 CFR 50.3 and 21 CFR 56.102.  Researchers may discuss the relevant regulatory requirements for a specific 
natural history study with the appropriate review division.  
 
15  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
585  HHS.31  The Common Rule is administered by the HHS Office for Human Research Protections 
586  (OHRP).   
587   
588  As with all research studies, natural history studies conducted in multiple regions may also be 
589  subject to different regulatory requirements.32  
590   
591  A.  Confidentiality of Subjects and Data Protection 
592   
593  Protecting confidentiality is a critical human subject protection responsibility.  Study planning 
594  should include addressing the applicable requirements of the HIPAA (Health Insurance 
595  Portability and Accountability Act)33 Privacy, Security, and Breach Notification rules, which 
596  may include requirements to obtain authorization from study subjects for the use and disclosure 
597  of their health information or a waiver of authorization from an IRB or privacy board, as well as 
598  requirements to implement safeguards to protect that information.   
599   
600  Data protection and security are an important human subject right.  The study organizers should 
601  consider all applicable local, state, national, and international privacy or security laws (to include 
602  tribal law passed by an official governing body of a Native American or Alaskan Native tribe) 
603  when planning the management of requests for access to the individual or aggregate data or data 
604  analyses by other researchers.      
605   
606  B.  IRB Review34 
607   
608  IRB review is generally required for a natural history study that is subject to the Common Rule 
609  and may be subject to FDA regulations at 21 CFR part 56 as described above.35  When multiple 
610  study sites participate in a natural history study, researchers should consider using a single IRB 
                                                 
31 45 CFR 46.101(a).  Note that the Common Rule may also apply if the natural history study is conducted, supported, 
or otherwise subject to regulation by another federal department or agency that has adopted the Common Rule (45 
CFR 46.103).  Researchers should discuss the relevant regulatory requirements with the appropriate federal agency or 
department supporting the research. 
 
32 See ICH E17. 
 
33 Health Insurance Portability and Accountability Act of 1996, Pub. L. No. 104-191.  
 
34 The term IRB used here refers to organizations constituted to oversee human subject protections, such as IRBs in 
the United States or independent ethics committees (IECs) in the European Union. 
 
35 45 CFR part 46; 21 CFR 56.102. 
 
16  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
611  for most or all the study sites for the initial and any periodic continuing review.36, 37  In many 
612  cases, natural history studies may be suitable for an IRB expedited review procedure.38    
613   
614  C.  Informed Consent 
615   
616  Determining what information needs to be disclosed in an informed consent form will likely 
617  depend on the applicable law and regulations.  FDA regulations at 21 CFR part 50 outline the 
618  informed consent process for applicable clinical investigations.  If HHS supports or conducts the 
619  natural history study (to include studies funded by the FDA), then the requirements of the 
620  Common Rule must be met.39  OHRP has published guidance addressing informed consent 
621  requirements under 45 CFR part 46.40     
622   
623  In designing a natural history study, the study organizer should consider the possibility that the 
624  data and biospecimens collected may be useful in addressing a research question not considered 
625  during the development of the original natural history study.  In particular, biological samples 
626  and genetic testing results obtained during a natural history study might be of value in the future 
627  as biomedical knowledge about the disease increases (even though the specific future use of the 
628  data may not have been known).  When planning a natural history study the study organizer may 
629  want to work with an IRB to determine the best approach to obtain consent, when appropriate, 
630  for the study as well as for any possible future secondary research use of the data and 
631  biospecimens collected.   
632   
633   
634  VIII.  INTERACTING WITH FDA 
635   
636  Because natural history studies can have broad applicability to different modes of treatment, 
637  discussions with FDA do not need to be related to a specific drug or conducted in the context of 
638  a regulatory submission or application.  Discussions such as in a Critical Path Innovation 
639  Meeting with the Center for Drug Evaluation and Research or in presubmission discussions with 
640  the Center for Biologics Evaluation and Research may provide nonbinding, scientific, and 
641  medical advice on drug development issues.41  For a sponsor with an investigational drug for a 
                                                 
36 See the guidance for industry Using a Centralized IRB Review Process in Multicenter Clinical Trials (March 
2006). 
 
37 The Federal Policy for the Protection of Human Subjects (known as the Common Rule, 45 CFR part 46) requires 
cooperative research to rely on approval by a single IRB in certain circumstances (45 CFR 46.114(b)).  This 
provision of the Common Rule is effective January 20, 2020 (see the revised final rule published June 19, 2018 (83 
FR 28497 for details). 
 
38 45 CFR 46.110. 
 
39 45 CFR 46. 
 
40 See the OHRP Informed Consent web page at https://www.hhs.gov/ohrp/regulations-and-
policy/guidance/informed-consent/index.html.  
 
41 See the guidance for industry Critical Path Innovation Meetings (April 2015). 
 
17  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
642  rare disease, FDA encourages the sponsor to meet with the relevant drug review division or 
643  office42 about the use of natural history data to support development of a particular drug.    
644   
645  For those who plan to conduct natural history studies through the Office of Orphan Products 
646  Development (OOPD) Orphan Products Natural History Grants Program, the FDA encourages 
647  discussion with the OOPD program officers about proposals that support targeted studies that 
648  advance rare disease drug development through characterization of the natural history of rare 
649  diseases and conditions, identification of genotypic and phenotypic subpopulations, and 
650  development and/or validation of clinical outcome measures, biomarkers, and/or companion 
651  diagnostics.43 
652   
                                                 
42 See the draft guidance for industry Formal Meetings Between the FDA and Sponsors or Applicants of PDUFA 
Products (December 2017).  When final, this guidance will represent the FDAs current thinking on this topic. 
 
43 For more information, see the Orphan Products Natural History Grants Program web page at 
https://www.fda.gov/ForIndustry/DevelopingProductsforRareDiseasesConditions/OrphanProductsNaturalHistoryGr
antsProgram/default.htm.  
18  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
653  REFERENCES 
654   
655  Delgado-Rodriguez M and Llorca J, 2004, Bias, J Epidemiol Community Health 58(8):635641. 
656   
657  Ellenberg JH, 1994, Cohort Studies:  Selection Bias in Observational and Experimental Studies, 
658  Stat Med, 13:557567. 
659   
660  FDA-NIH Biomarker Working Group, 2017, BEST (Biomarkers, EndpointS, and other Tools) 
661  Resource, Silver Spring, Maryland: Food and Drug Administration, and Bethesda, Maryland: 
662  National Institutes of Health (available at 
663  https://www.ncbi.nlm.nih.gov/books/NBK326791/pdf/Bookshelf_NBK326791.pdf). 
664   
665  Institute of Medicine Committee on Qualification of Biomarker and Surrogate Endpoints in 
666  Chronic Disease, 2010, Evaluation of Biomarkers and Surrogate Endpoints in Chronic Disease, 
667  Micheel CM and Ball JR, editors, Washington, DC:  National Academies Press, 2. 
668   
669  Sacks H, Chalmers T, and Smith H, 1982, Randomized Versus Historical Controls for Clinical 
670  Trials, Am J Med, 72(2):233240. 
671   
672  Schoenfeld DA and Cudkowicz M, 2008, Design of Phase II ALS Clinical Trials, Amyotroph 
673  Lateral Scler, 9(1):1623 (available at http://dx.doi.org/10.1080/17482960701875896). 
674   
675  Thomas L and Peterson ED, 2012, The Value of Statistical Analysis Plans in Observational 
676  Research: Defining High-Quality Research from the Start, JAMA, 308(8):773774; 
19