REMS Assessment: 
Planning and Reporting 
Guidance for Industry 
 
 
 
 
 
 
DRAFT GUIDANCE
 
 
For questions regarding this draft document contact (CDER) Doris Auth 301-796-0487, or 
(CBER) Office of Communication, Outreach and Development, 800-835-4709 or 240-402-8010  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
U.S. Department of Health and Human Services 
Food and Drug Administration 
Center for Drug Evaluation and Research (CDER) 
Center for Biologics Evaluation and Research (CBER)  
 
January 2019 
Procedural
  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
REMS Assessment: 
Planning and Reporting 
Guidance for Industry 
 
Additional copies are available from: 
Office of Communications, Division of Drug Information  
Center for Drug Evaluation and Research 
Food and Drug Administration 
10001 New Hampshire Ave., Hillandale Bldg., 4th Floor  
Silver Spring, MD 20993-0002  
Phone: 855-543-3784 or 301-796-3400; Fax: 301-431-6353 
Email: druginfo@fda.hhs.gov  
https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/default.htm 
 
and/or 
 
Office of Communication, Outreach and Development  
Center for Biologics Evaluation and Research 
 Food and Drug Administration 
10903 New Hampshire Ave., Bldg. 71, Room 3128 
Silver Spring, MD 20993-0002 
Phone: 800-835-4709 or 240-402-8010 
Email: ocod@fda.hhs.gov 
https://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guidances/default.htm   
 
 
 
 
 
 
 
 
 
 
U.S. Department of Health and Human Services 
Food and Drug Administration 
Center for Drug Evaluation and Research (CDER) 
Center for Biologics Evaluation and Research (CBER) 
  
January 2019 
Procedural 
 
    Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
 
TABLE OF CONTENTS 
 
 
I.  INTRODUCTION............................................................................................................. 1 
II.  BACKGROUND ............................................................................................................... 2 
III.  REMS ASSESSMENTOVERVIEW ........................................................................... 3 
IV.  DEVELOPING THE REMS ASSESSMENT PLAN .................................................... 5 
A.   Assessment Categories ................................................................................................................... 5 
B.   Selecting Metrics ............................................................................................................................ 6 
C.  Selecting Sources of Assessment Data .......................................................................................... 7 
1. Applicants REMS Data ................................................................................................................... 8 
2. Surveys ............................................................................................................................................. 9 
3. Drug Utilization Data ...................................................................................................................... 9 
4. Postmarketing Adverse Event Data ................................................................................................ 10 
5. Observational/Epidemiology Data ................................................................................................. 11 
6. Data From Root Cause Analysis .................................................................................................... 12 
7. Data from Stakeholder Outreach ................................................................................................... 12 
D.   Specifying Thresholds for REMS Effectiveness ........................................................................ 12 
V.  CONSIDERATIONS FOR MEASURING BARRIERS TO PATIENT ACCESS 
AND BURDEN ON THE HEALTH CARE DELIVERY SYSTEM...................................... 13 
A.   Assessing Burden on the Health Care Delivery System ........................................................... 13 
B.  Assessing Barriers to Patient Access .......................................................................................... 14 
VI.   REMS ASSESSMENT SUBMISSIONS ....................................................................... 15 
A.   REMS Assessment Plan ............................................................................................................... 15 
1. Overview ......................................................................................................................................... 15 
2. Methodology or Protocols Describing Sources of Data and Analytical Tools .............................. 16 
B.   REMS Assessment Reports ......................................................................................................... 16 
APPENDIX 1: EXAMPLE - ASSESSMENT CATEGORY AND METRICS ..................... 19 
APPENDIX 2: EXAMPLE REMS ASSESSMENT PLAN OVERVIEW ............................. 20 
Appendix 2A: Drug X REMS Assessment Plan ................................................................................. 21 
 
 
    Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
REMS Assessment: Planning and Reporting 
1 
Guidance for Industry1 
2 
3   
4   
5  This draft guidance, when finalized, will represent the current thinking of the Food and Drug 
6  Administration (FDA or Agency) on this topic. It does not establish any rights for any person and is not 
7  binding on FDA or the public. You can use an alternative approach if it satisfies the requirements of the 
8  applicable statutes and regulations. To discuss an alternative approach, contact the FDA staff responsible 
9  for this guidance as listed on the title page.  
10   
11   
12   
13  I.  INTRODUCTION  
14   
15  This document provides guidance to industry on the assessment of risk evaluation and mitigation 
16  strategies (REMS) for prescription drug products, including biological products.2,3 This guidance 
17  describes how to develop a REMS Assessment Plan4, specifically, how the REMS program 
18  goals, objectives5 and REMS design may impact the selection of metrics6 and data sources, 
19  which will be used to assess whether the program is meeting its risk mitigation goals. The 
20  guidance also discusses considerations for assessing the impact of REMS on patient access to the 
21  drug and its burden to the healthcare delivery system. Finally, this guidance provides 
                                                           
1 This guidance has been prepared by the Division of Risk Management, Office of Medication Error Prevention and 
Risk Management, Office of Surveillance and Epidemiology in the Center for Drug Evaluation and Research 
(CDER) in cooperation with other divisions and offices within CDER and the Center for Biologics Evaluation and 
Research (CBER) at the Food and Drug Administration.  
2 For purposes of this guidance, unless otherwise specified, references to drugs and drug products include drugs 
approved under the FD&C Act and biological products licensed under the Public Health Service (PHS) Act, other 
than biological products that also meet the definition of a device in section 201(h) of the Federal Food, Drug and 
Cosmetic (FD&C) Act (21 U.S.C. 321(h)). 
3 This is one of several guidance documents being developed to fulfill performance goals under the fifth 
authorization of the prescription drug user fee program, the Prescription Drug User Fee Act V (PDUFA V), PDUFA 
V Reauthorization Performance Goals and Procedures Fiscal Years 2013 Through 2017, Section XI.A.1 
4 For purposes of this guidance, a REMS Assessment Plan is a specific plan for how the applicant intends to assess 
the performance of the REMS in meeting its risk mitigation goals and objectives. The REMS Assessment Plan is 
outlined in the REMS approval letter for NDAs and BLAs and described in detail in the REMS Supporting 
Document. 
5 For purposes of this guidance, REMS goals are the overall, safety-related health outcome(s) that the REMS are 
designed to achieve. REMS objectives are the metrics that indicate that the program is meeting its goals when the 
risk mitigation goal cannot be measured directly. 
6 For purposes of this guidance, REMS metrics are the measures (such as quantity, quality, duration, size, or 
frequency) of an aspect of the program that provide a systematic basis for assessing how well a program has 
performed.  
 
1  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
22  recommendations on a standardized approach for reporting REMS assessment findings to FDA 
23  using the REMS Assessment Report.7  
24   
25  This document does not address the design or development of REMS, methods for designing, 
26  conducting, and reporting surveys, pharmacoepidemiologic safety studies, or other studies when 
27  used as a component of a REMS assessment; however, in relevant sections, it references 
28  available FDA guidances that address these issues.  
29   
30  This guidance applies to certain drug and biological products submitted for approval or approved 
31  under sections 505(b) or 505(j) of the Federal Food, Drug, and Cosmetic Act (FD&C Act) (21 
32  U.S.C. 355(b) or 355(j)), or section 351 of the Public Health Service Act (PHS Act) (42 U.S.C. 
33  262), that are required by FDA to have REMS. These applications are considered covered 
34  applications and include new drug applications (NDAs), abbreviated new drug applications 
35  (ANDAs) and biologics license applications (BLAs). 
36   
37  In general, FDAs guidance documents do not establish legally enforceable responsibilities. 
38  Instead, guidances describe the Agencys current thinking on a topic and should be viewed only 
39  as recommendations, unless specific regulatory or statutory requirements are cited. The use of 
40  the word should in Agency guidances means that something is suggested or recommended, but 
41  not required.  
42   
43  II.  BACKGROUND 
44   
45  Section 505-1 of the FD&C Act (21 U.S.C. 355-1), as added by the Food and Drug 
46  Administration Amendments Act of 2007 (FDAAA) and later amended by the Food and Drug 
47  Administration Safety and Innovation Act of 2012 (FDASIA), authorizes FDA to require REMS 
48  for certain drugs8 if FDA determines that a REMS is necessary to ensure that the benefits of the 
49  drug outweigh its risks.9, 10, 11 
50   
                                                           
7 For purposes of this guidance, the REMS Assessment Report is the document applicants submit that contains 
information generated from the analysis of the metrics outlined in the REMS Assessment Plan. 
8 Section 505-1 of the FD&C Act applies to applications for prescription drugs submitted or approved under 
subsections 505(b) (i.e., new drug applications) or (j) (i.e., abbreviated new drug applications) of the FD&C Act and 
to applications submitted or approved under section 351 (i.e., biologics license applications) of the Public Health 
Service Act (42 U.S.C. 262). For the purposes of this document, unless otherwise specified, the term drug refers to 
human prescription drugs, including those that are licensed as biological products (biologics). 
9 Public Law 110-85, September 27, 2007, available at https://www.gpo.gov/fdsys/pkg/PLAW-
110publ85/html/PLAW-110publ85.htm, accessed November 19, 2018. 
10 Public Law 112-144, July 9, 2012, available at http://www.gpo.gov/fdsys/pkg/PLAW-112publ144/pdf/PLAW-
112publ144.pdf, accessed November 19, 2018. 
11 See Section 505-1(a) of the FD&C Act. 
 
2  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
51  REMS may include a Medication Guide, a patient package insert, and/or a communication 
52  plan.12 FDA also may require certain elements to assure safe use (ETASU) as part of REMS for a 
53  drug.13  
54   
55  Every proposed REMS for an NDA and BLA must have a timetable for submission of REMS 
56  assessments,14 that: 
57   
58    includes assessments submitted to the FDA by the dates that are 1)18 months, 2) 3 years 
59  after the strategy is initially approved, and 3) in the 7th year after the strategy is so 
60  approved, and 
61   
62    is at a frequency specified in the strategy and can be increased or reduced in frequency 
63  under certain circumstances and eliminated under certain circumstances.  
64   
65  With limited exceptions, REMS assessments are also required when submitting a supplemental 
66  application for a new indication for use, when required by the strategy, and whenever FDA 
67  determines that an assessment is needed to evaluate whether the strategy should be modified to 
68  ensure the benefits of the drug outweigh the risks or to minimize the burden on the healthcare 
69  delivery system of complying with the strategy.15 In addition to the required assessments, an 
70  applicant may voluntarily submit an assessment of an approved REMS at any time.16 
71   
72  Section 505-1(g)(3) of the FD&C Act specifies that a REMS assessment shall include, with 
73  respect to each goal in the strategy, an assessment of the extent to which the approved strategy, 
74  including the elements, is meeting the goal or whether the goal or elements should be modified. 
75  The FD&C Act does not specifically describe how an applicant should conduct this assessment.  
76   
77  III.  REMS ASSESSMENTOVERVIEW 
78   
79  The development of the REMS Assessment Plan should begin during the REMS design phase, 
80  with the development of a clear risk mitigation goal (i.e., REMS goal). The risk mitigation goal 
81  is the safety-related health outcome that the REMS will be designed to achieve. Because risk 
82  mitigation goals cannot always be measured directly, it is important to include one or more 
83  intermediate measurable objectives that, if achieved, indicate that the program is meeting its 
84  goals. For example, a REMS for a drug with a risk of renal toxicity may include a goal to 
85  mitigate the risk of renal failure, the success of which may be measured by the objectives that all 
86  patients undergo periodic testing of serum creatinine and that appropriate management steps are 
87  undertaken when laboratory values are out of range.  
                                                           
12 Section 505-1(e)(2)-(3) of the FD&C Act. 
13 See Section 505-1(f)(1) of the FD&C Act.  
14 See Section 505-1(c)-(d) of the FD&C Act. 
15 See Section 505-1(g)(2) of the FD&C Act. 
16 See Section 505-1(g)(1) of the FD&C Act. 
 
3  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
88   
89  Once the goals and objectives have been determined, the design of the REMS can begin, 
90  including the REMS requirements17 and REMS materials18, and if applicable an implementation 
91  system, that align with the goals and objectives. For example, if the REMS includes a goal to 
92  mitigate the risk of a drug and an objective to inform or educate patients about the safe use of 
93  that drug, prescribers could be required to counsel their patients using a REMS material such as a 
94  patient-prescriber agreement.  
95  While a comprehensive discussion regarding REMS design is beyond the scope of this guidance, 
96  it is important to consider how the REMS design corresponds to the goals and objectives of the 
97  REMS and to the development of the REMS Assessment Plan.  
98   
99  When designing a REMS program, applicants should consider: 
100     
101    The characteristics of the risk associated with the drug that the REMS is intended to 
102  mitigate (e.g., risk factors, timing, detectability, reversibility) 
103   
104    Any information demonstrating the effectiveness of the proposed strategy in mitigating 
105  the risk (e.g., results from premarket testing with stakeholders, effectiveness 
106  demonstrated during clinical trials or from the published literature, findings from 
107  qualitative or quantitative human factors studies, previous experience with similar REMS 
108  programs). 
109   
110    Which stakeholders within the existing healthcare delivery system may require additional 
111  support to effectively mitigate the risk, as well as the type and extent of the support that 
112  may be required (e.g., training about how to manage the risk, verification that laboratory 
113  monitoring was conducted).  
114   
115    The feasibility of implementing the proposed strategies, the potential burden of the 
116  proposed mitigation strategies on the healthcare delivery system, and the potential impact 
117  of the proposed strategies on patient access to the drug (e.g., strategies that have the 
118  potential to result in treatment interruption or delays, particularly where patients have 
119  serious or life-threatening conditions). 
120   
121  Applicants should document the rationale for their proposed REMS design in the REMS 
122  Supporting Document.19 The rationale should include how the REMS requirements, REMS 
                                                           
17 For purposes of this guidance, the term REMS requirements refers to the activities that both REMS participants 
(e.g., healthcare providers, patients, health care settings) and applicants must undertake in a REMS. 
18 For purposes of this guidance, the term REMS materials is used to describe any materials, processes, or system 
designed to operationalize one or more REMS requirements. 
19 For purposes of this guidance, the REMS Supporting Document provides additional information about the REMS, 
such as the rationale for, and supporting information about, the design, implementation, and assessment of the 
REMS. 
4  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
123  materials, and implementation system were selected or designed to achieve the goals and 
124  objectives.  
125   
126  The REMS Assessment Plan should include the metrics, data sources, and methodologies that 
127  the applicant intends to use to assess the performance of the REMS. It should also include a plan 
128  for assessing the impact of the REMS on the healthcare delivery system and patient access to the 
129  drug.  
130   
131  The data sources, methodologies, and metrics used to assess the effectiveness of REMS continue 
132  to evolve. As with any method or data source used to support program evaluation, there are 
133  limitations that should be considered with REMS assessments. For example, for a REMS that is 
134  established at the time of initial drug approval, there may not be relevant baseline data for 
135  comparison, such as the incidence of the risk associated with the drug or drug use patterns. Also, 
136  many safety-related health outcomes that are the focus of the REMS may occur rarely and thus 
137  be challenging to measure and accurately evaluate using available data sources. Additionally, for 
138  drugs that are infrequently prescribed some commonly used data sources may not have sufficient 
139  drug utilization information to make study of program impact possible. Finally, it is often 
140  difficult to distinguish the effect of the REMS from other healthcare or public health initiatives. 
141  Applicants should make every effort to develop a REMS Assessment Plan that enables them to 
142  assess the effectiveness of the REMS, while acknowledging these limitations. 
143   
144  Considering their limitations, no single metric, data source, or methodology should be relied 
145  upon to assess the effectiveness of REMS. Instead, several metrics, data sources, and 
146  methodologies should be considered, as appropriate. Each REMS Assessment Plan should 
147  include discussion of any anticipated challenges with conducting the assessment and limitations, 
148  if any, of the data that will be used.   
149   
150  FDA encourages applicants and the research community to develop novel methods for assessing 
151  REMS. Robust collaborations between FDA and other regulatory agencies, applicants, and the 
152  research community can help advance the science of post-market assessment of effectiveness of 
153  risk mitigation strategies.  
154   
155  IV.  DEVELOPING THE REMS ASSESSMENT PLAN 
156   
157  A.  Assessment Categories 
158   
159  The REMS Assessment Plan should include assessment of all aspects of program performance, 
160  including the individual REMS requirements (e.g., prescriber certification), REMS materials 
161  (e.g., prescriber-patient agreement), and the overall impact of the program.20,21 REMS can be 
                                                           
20 Gaglio B, Shoup JA, Glasgow RE. The RE-AIM Framework: A Systematic Review of Use Over Time. Am J 
Public Health. 2013;103(6):e38-e46. 
21 Practical Approaches to Risk Minimisation for Medicinal Products: Report of CIOMS Working Group IX; August 
2014. 
 
5  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
162  assessed using both process indicators22 and the intended outcomes (e.g., reduction in 
163  inappropriate prescribing), but can also include the unintended outcomes (e.g., barriers to patient 
164  access) of the program. Below is a set of assessment categories that are intended to capture both 
165  REMS program processes and outcomes. There may be some overlap between these categories, 
166  and each category may include both process and outcome metrics.  
167   
168    Program Outreach and CommunicationMeasures of the extent to which the REMS 
169  materials reached the intended stakeholders. 
170   
171    Program Implementation and OperationsMeasures of the extent to which the intended 
172  stakeholders are participating in the program; how effectively the REMS program is 
173  being implemented, including the extent of use of REMS materials and compliance with 
174  REMS requirements; and any unintended consequences that could affect patient access or 
175  potential burden to the healthcare system related to the program operations. 
176   
177    KnowledgeMeasures of the extent of stakeholders (e.g., patient/caregiver, prescriber, 
178  pharmacist) knowledge about the REMS-related risk or knowledge of any safe use 
179  conditions that are needed in order to mitigate the risk.23 
180   
181    Safe Use BehaviorsMeasures of the extent to which safe use conditions are being 
182  adopted or followed (e.g., how often a required laboratory test is conducted prior to 
183  dispensing of the medication). 
184   
185    Health Outcomes and/or Surrogates of Health OutcomesMeasures of the safety-related 
186  health outcome of interest (e.g., a reduction in the number of serious outcomes associated 
187  with a particular adverse event) or a surrogate of a health outcome (e.g., a reduction in 
188  the number or proportion of patients at greatest risk of an adverse event who are 
189  prescribed a drug).  
190   
191  B.  Selecting Metrics 
192   
193  REMS assessment metrics should be identified for all assessment categories that are relevant to 
194  the REMS program and that are feasible. Applicants should provide a rationale for all metrics 
195  selected and state whether the metric has been validated. The metrics should fall within the 
                                                           
22 For purposes of this guidance, process indicators directly measure the extent of compliance with required REMS 
processes, such as processes used to comply with REMS requirements, such as distribution of REMS materials. 
23 See the draft guidance for industry, Survey Methodologies to Assess REMS Goals That Relate to Knowledge. We 
update guidances periodically. To make sure you have the most recent version of a guidance, check the FDA Drugs 
guidance web page (available at 
https://www.fda.gov/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/default.htm) or Biologics 
guidances web page (available at 
https://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guidances/default.htm). 
When final, this guidance will represent FDAs current thinking on this topic. 
 
6  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
196  categories described above in section IV.A. More than one metric may be selected for each 
197  assessment category. Example metrics for the above assessment categories are provided below.  
198   
199    Metrics in the Program Outreach and Communication assessment category may include 
200  numbers of specific REMS materials that were distributed to, and the proportion of these 
201  that were subsequently opened or read by, the targeted audiences.  
202   
203    Metrics in the Program Implementation and Operations assessment category may include 
204  the number of prescribers, health care settings, and/or pharmacies that have certified or 
205  undergone training in the REMS program; the number of contacts to the call center and a 
206  summary of the reason for the contact; number and results of audits of certified health 
207  care settings; and the number of shipments of the drug to non-certified settings.  
208   
209    Metrics in the Knowledge assessment category may include stakeholder understanding of 
210  the risks and safe use of the drug. The draft guidance for industry Survey Methodologies 
211  to Assess REMS Goals that Relate to Knowledge provides further recommendations on 
212  using surveys to evaluate knowledge of REMS risks and safe use conditions.23 
213   
214    Metrics in the Safe Use Behaviors assessment category can include an evaluation of 
215  prescribing patterns and the proportion of patients who were counseled prior to initiating 
216  a drug, as evidenced by the use of a REMS material such as a patient counseling tool or 
217  patient-provider agreement form.  
218   
219    Metrics in the Health Outcomes and/or Surrogates of Health Outcomes assessment 
220  category can include numbers and/or rates of a specific adverse event of interest such as 
221  rates of serious bleeds or severe neutropenia. Surrogate metrics could include the number 
222  of inadvertent fetal exposures or the number of prevented fetal exposures to the 
223  teratogenic drug.  
224   
225  The metrics that are selected within each assessment category will depend on the goals and 
226  objectives of the program, the REMS requirements (e.g., education, dispensing requirements) 
227  and REMS material (e.g., prescriber-patient agreement), and the feasibility of the measurement.  
228   
229  See Appendix 1 for additional examples of some potential metrics for the different assessment 
230  categories and Appendix 2 for an example of how the development of a REMS Assessment Plan 
231  may be linked to the REMS goals, objectives, and requirements. Applicants may also consider 
232  other healthcare program assessment frameworks to help identify and organize REMS metrics.  
233   
234  C.  Selecting Sources of Assessment Data  
235   
236  Applicants are encouraged to identify complementary data sources that provide a combination of 
237  qualitative and quantitative information about the REMS and should select sources that provide 
238  data supporting the REMS assessment starting from the initial REMS implementation.  
239  In selecting the sources of data, applicants should take into consideration how accurately and 
240  completely each data source can capture the relevant population and important components 
7  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
241  necessary for the assessment. Some data sources may be used to assess multiple identified 
242  metrics. For example, drug utilization data may be used to assess changes in prescribing 
243  behaviors as well as to inform potential barriers to patient access that would require additional 
244  analyses. In other cases, multiple data sources may be needed to assess a single metric.  
245   
246  A detailed description of the data sources and methodological approaches, as well as the 
247  applicants evaluation of the adequacy of these in assessing the specific REMS requirement or 
248  REMS materials, should be provided in the REMS Supporting Document. For all studies, a 
249  protocol and statistical analysis plan should be submitted to FDA for review and comment prior 
250  to study initiation. If applicable, timelines for the submission of study protocol and interim and 
251  final reports should be provided by the applicant and agreed upon by FDA.  
252   
253  Described below are examples of data sources that may be used to inform REMS assessments. 
254  This list is not meant to be comprehensive, and additional sources or approaches may be 
255  appropriate.  
256   
257  1.  Applicants REMS Data  
258   
259  REMS with ETASU may include a requirement that the applicant maintain a database of 
260  certified/enrolled prescribers, dispensers, healthcare settings, distributors, or patients. This 
261  database can be a rich source of data for metrics that apply to several assessment categories. 
262  Applicants should carefully consider what data need to be collected for REMS assessment 
263  purposes when they design and develop REMS databases. 
264   
265  As an example, an applicants REMS database can collect program participation metrics, 
266  including the number of stakeholders or healthcare settings enrolled or certified in the program 
267  when an ETASU requires such enrollment or certification. Depending on what data are captured, 
268  the database might help to provide information about patient access to the drug, geographic 
269  location of prescribers, numbers of prescriptions dispensed, and prescriber specialties. The 
270  database can also include data that inform program operations and safe use conditions, such as 
271  the number of prescriptions dispensed with and without the proper authorization when the 
272  ETASU require such authorization. It may provide information about burden to the healthcare 
273  system through categorizing data from general complaints received through a call center or 
274  instances of delays in patient access that may be associated with the REMS. For REMS that 
275  require a post-training knowledge assessment, the database can capture poor performance on 
276  knowledge assessment questions, which can prompt a revision of the training program content to 
277  address knowledge gaps. The database may also collect information related to health outcomes 
278  of interest or results of laboratory monitoring (e.g., absolute neutrophil counts, pregnancy test 
279  results).  
280   
281  Many applicants also collect other REMS operations data, such as the results of audit findings. 
282  Applicants are encouraged to collect data across a wide range of program processes and 
283  functions to the extent it facilitates their assessment of their REMS.  
284     
8  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
285  2.  Surveys 
286   
287  When a REMS includes an objective to inform or educate patients or health care professionals 
288  about a serious risk associated with a drug, or about the safe use of a drug, the assessment plan 
289  should include an assessment of these stakeholders knowledge. The key messages for each 
290  targeted stakeholder should be defined prior to REMS approval (and documented in the REMS 
291  Supporting Document), should be consistent with the key messages in the counseling, education, 
292  or communication materials, and should relate to the REMS objective. Surveys are often used to 
293  assess knowledge of these key messages and the related safe use actions to be taken by various 
294  stakeholders. 
295   
296  Assessment of stakeholder understanding of REMS requirements may also be useful to 
297  determine whether safe use behaviors are being adopted and whether stakeholders adhere to 
298  certain REMS requirements. For example, if a REMS requires prescribers to counsel patients 
299  before prescribing the drug, prescribers may be surveyed on whether they did so with the initial 
300  and any subsequent prescriptions. Patients, in turn, may be surveyed on whether they received 
301  counseling from their provider before they were prescribed the drug.  
302   
303  In addition to the use of surveys to assess self-reported adherence with program requirements, 
304  surveys may also be designed to assess attitudes and beliefs and the potential burden associated 
305  with REMS program requirements. FDAs draft guidance, Survey Methodologies to Assess 
306  REMS Goals That Relate to Knowledge, provides recommendations to industry on conducting 
307  REMS assessment surveys to assess respondent knowledge of REMS-related information.23 
308   
309  3.  Drug Utilization Data  
310   
311  Drug utilization data not only provide descriptive information on the patterns of drug use but can 
312  also provide useful information on overall disease treatment patterns and healthcare market 
313  dynamics. Studies incorporating drug utilization data may be able to measure patient and 
314  provider characteristics; reasons for use; rates of drug uptake; concomitant drug use; and, in 
315  some cases, more detailed information such as duration of use and drug switching patterns. 
316  Finally, drug utilization studies may, in combination with other studies, inform barriers to patient 
317  access to the drug (see section V.B.). 
318   
319  If a REMS Assessment Plan includes a drug utilization study, it should describe the drug 
320  utilization data source, the rationale for the data source, and the data collection methodology, 
321  design and analytical approaches, and any limitations. The drug utilization study protocol should 
322  also describe the national representativeness of the utilization data analyzed, as well as the 
323  representativeness of the population evaluated in the drug utilization study relative to the overall 
324  patient population receiving the drug product, comprehensiveness of the capture of drug 
325  utilization across all settings of care, any linkages to other data sources as relevant/appropriate, 
326  and any relevant data projection methodologies employed.  
327   
9  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
328  For all drug utilization studies, a study protocol should be submitted to FDA for review and 
329  comment prior to study initiation. In some cases, a statistical analysis plan may also be 
330  necessary. 
331   
332  4.  Postmarketing Adverse Event Data  
333   
334  Adverse event data can provide qualitative information on adverse events and outcomes related 
335  to the risk that the REMS is intended to mitigate. A number of factors affect spontaneous adverse 
336  event reporting, including the nature and severity of the adverse event and length of time the 
337  product has been on the market. 
338   
339  The quality of the adverse event reports is critical for appropriate assessment of the REMS; 
340  therefore, we recommend collection of targeted information about specific adverse events of 
341  interest. The information collected should focus on further characterizing the risk, and capturing 
342  patient outcomes, as well as determining whether safe use conditions were met. The type of data 
343  the applicant plans to collect to further characterize the adverse event should be included in the 
344  REMS Assessment Plan. If feasible, it is helpful to link patient information in the applicants 
345  REMS database, when one exists, to adverse event reports in the applicants adverse event 
346  database.  
347   
348  A well-recognized limitation of spontaneous adverse event reporting is underreporting. It is 
349  possible that the extent of underreporting for products with REMS with ETASU is not as 
350  extensive as products without REMS, particularly when there is a mechanism to monitor patients 
351  for adverse events as part of a REMS requirement. However, because spontaneous adverse event 
352  reporting systems do not capture all adverse events, even adverse events for drugs with an 
353  approved REMS, the data from those reporting systems cannot be used to calculate the incidence 
354  of a particular adverse event.  
355   
356  Postmarketing adverse event data can be used to compare reporting rates of an adverse event 
357  before and after a REMS have been implemented. In certain circumstances, however, it may not 
358  be appropriate to do so because there may be differences in the way the adverse event 
359  information was obtained. For example, prior to the implementation of a REMS, adverse event 
360  information may be collected solely from spontaneous reports. If the REMS includes a prescriber 
361  attestation to report adverse events experienced by patients taking the drug, the number of 
362  reported events may be higher after REMS implementation (stimulated reporting).  
363   
364  To the extent possible, several sources should be employed to obtain information on adverse 
365  events and outcomes related to the risk that the REMS was intended to mitigate. Adverse event 
366  reports received or identified through REMS are still required to be evaluated and must be 
367  submitted to FDA per the regulations for postmarketing adverse event reporting.24 
368     
                                                           
24 21 CFR 314.80, 314.98, and 600.80. 
10  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
369  5.  Observational/Epidemiology Data 
370   
371  Studies analyzing observational data to evaluate outcomes associated with use of drug products 
372  (i.e., pharmacoepidemiology studies) can be considered to evaluate various aspects of REMS 
373  including safe use behaviors, prescribing patterns, barriers to patient access, and safety-related 
374  health outcomes or surrogates of those outcomes.  
375   
376  When designing epidemiology studies to assess a REMS, applicants may consider various 
377  population data sources and study designs. FDA has published best practices for conducting and 
378  reporting pharmacoepidemiology safety studies using electronic health care data.25 Additionally, 
379  the published literature contains guidelines for planning, conduct, analysis, and reporting of 
380  epidemiologic studies of drug safety, which could be used to assess the performance of 
381  REMS.26,27 
382   
383  There may be unique challenges in using pharmacoepidemiology data to assess the effectiveness 
384  of a REMS. In most instances, existing databases may not adequately capture important data 
385  elements, such as the outcome of interest and covariates, therefore limiting the adequacy of a 
386  pharmacoepidemiology study to evaluate the metrics of interest. In those instances, studies 
387  employing prospective data collection will need to be considered. Additionally, the utility of 
388  pharmacoepidemiology data is limited when a REMS is implemented at the time of approval and 
389  therefore no data are available on the use of the drug without a REMS. Nevertheless, the optimal 
390  design and methodology of studies using observational/epidemiologic data to assess the impact 
391  of REMS or specific REMS requirements on certain outcomes are evolving. FDA intends to 
392  exercise a flexible approach with regard to such studies.  
393   
394  This guidance does not recommend a specific pharmacoepidemiology study design or type for 
395  REMS assessments, nor does it address the specific population or data sources to be considered. 
396  The decision to use a pharmacoepidemiology study should be guided by the questions of interest, 
397  and the feasibility of the selected data to adequately evaluate the question of interest. A 
398  discussion of the challenges of conducting a proposed assessment and limitations of the data 
399  used should be included in the study proposal.   
400     
                                                           
25 FDA guidance for industry Best Practices for Conducting and Reporting Pharmacoepidemiologic Safety Studies 
Using Electronic Healthcare Data Sets. Available from: 
https://www.fda.gov/downloads/drugs/guidances/ucm243537.pdf.  
26 Von Elm E, Altman DG, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology 
(STROBE) statement: guidelines for reporting observational studies. BMJ. 2007; 335:806-808. 
27 European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) Guide on 
Methodological Standards in Pharmacoepidemiology (Revision 1) [Internet]. London (UK): European Medicines 
Agency. EMA/95098/2010. Available at 
http://www.encepp.eu/standards_and_guidances/documents/ENCePPGuideofMethStandardsinPE_2.pdf  
11  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
401  6.  Data from Root Cause Analysis 
402   
403  A root cause analysis (RCA) is a structured method used to analyze adverse events or root causes 
404  of program deficiencies and focuses on improving systems and processes to enhance patient 
405  safety.28 An RCA may need to be conducted to better understand the observed findings from a 
406  specific REMS assessment. RCAs may be conducted after a REMS assessment has been 
407  completed if it is unclear whether certain aspects of the program are performing as intended. 
408  RCAs can help identify determinants and underlying causes of REMS failure to meet its goals. 
409  RCAs can also help identify the factors responsible for a particular type of failure of the REMS, 
410  as well as burden of the program on the healthcare delivery system, and barriers to patient access 
411  to the drug. This analysis may help inform any necessary modifications to the REMS program 
412  goals and requirements. 
413   
414  RCA best practices include the development and use of a predefined protocol and a team-based 
415  reconstruction of each issue via retrospective review and interviews. The applicant should then 
416  assess the sequence of steps that led to each type of program failure (e.g., inadequate prescriber 
417  knowledge, lack of compliance) or unintended effect and determine how and why that event 
418  occurred. RCAs can also help assess other adverse events or unfavorable effects that may occur 
419  as an unintended consequence of the REMS requirements. 
420   
421  7.  Data from Stakeholder Outreach 
422   
423  Applicants should consider seeking input from the key stakeholders affected by the REMS, 
424  including prescribers, pharmacists, other healthcare professionals, and patients. Input from 
425  marketing research surveys, focus groups, and interviews could also inform the applicant and the 
426  Agency about the impact of the program on the healthcare delivery system and on patient access 
427  to the drug, as well as opportunities for program improvement.  
428   
429  D.  Specifying Thresholds for REMS Effectiveness 
430   
431  An additional consideration in REMS assessment planning is specifying performance thresholds 
432  for determining the effectiveness of the REMS. The specification of performance thresholds or 
433  performance levels over time provides criteria to help determine if REMS program performance 
434  is acceptable or if modifications to the REMS are needed. For example, a proposal and 
435  justification for a performance threshold should be provided in study protocols for knowledge 
436  surveys. In this case, the threshold would be the minimum knowledge rate that, if achieved, 
437  demonstrates that the REMS has met its goals of communicating the REMS key messages.23 
 
438   
439  The REMS Assessment Plan should specify a performance threshold for a health outcome of 
440  interest, if feasible. If the health outcomes of interest for the REMS are difficult to measure 
441  directly, performance thresholds should be specified for surrogate metrics.  
                                                           
28 Patient safety primers: root cause analysis [Internet]. Rockville (MD): Department of Health and Human Services 
(US). Agency for Healthcare Research and Quality. Available from: http://psnet.ahrq.gov/primer.aspx?primerID=10. 
See also Root Cause Analysis [Internet]. Washington (DC): Department of Veterans Affairs (US). National Center 
for Patient Safety. Available from: https://www.patientsafety.va.gov/professionals/onthejob/rca.asp  
12  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
442   
443  Applicants should describe potential limitations or challenges with selected performance 
444  thresholds. In cases in which prespecifying a performance threshold is deemed infeasible, 
445  applicants should explain the issues and considerations that resulted in this determination.  
446   
447  In lieu of prespecifying performance thresholds, applicants could consider providing alternatives 
448  such as: (1) a comparison of an adverse event for a drug with a REMS to a similar drug without a 
449  REMS, (2) a comparison of the reporting rate of an event from data obtained in a REMS registry 
450  to a background rate of that event in a similar patient population from a representative database, 
451  or (3) a comparison of the reported rate or the event rate from observational studies to the rate 
452  that was observed in the clinical trials. Each of these comparisons has limitations and should be 
453  interpreted with caution.  
454   
455  V.  CONSIDERATIONS FOR MEASURING BARRIERS TO PATIENT ACCESS 
456  AND BURDEN ON THE HEALTH CARE DELIVERY SYSTEM  
457   
458  Including ETASU in REMS allows patients safe access to drugs with known serious risks that 
459  would otherwise not be approved or would be withdrawn. Section 505-1(f)(2) of the FD&C Act 
460  states that such ETASU shall, considering the risk, not be unduly burdensome on patient access, 
461  and, to the extent practicable, minimize the burden on the health care delivery system. 
462   
463  A.  Assessing Burden on the Health Care Delivery System 
464   
465  In the context of REMS with ETASU, burden reflects the additional effort that healthcare 
466  professionals and other stakeholders expend in complying with the REMS requirements beyond 
467  what is required for good clinical care.29 This may include, for example, the effort expended to 
468  comply with program requirements to complete certification and training or to implement a 
469  process in a healthcare system for verifying documentation of laboratory monitoring as required 
470  by a REMS. Burden may also result when information on REMS requirements is not easily 
471  found or stakeholder roles and responsibilities under REMS are communicated in ways that 
472  stakeholders find confusing.  
473   
474  Identifying potential REMS burdens should begin during the REMS design phase, and applicants 
475  should make efforts to minimize potential burdens at this stage. When proposed REMS with 
476  ETASU are submitted, applicants should provide supportive information that demonstrates that 
477  they have considered the ways in which ETASU may introduce additional burden and that they 
478  have attempted to minimize that burden to the extent practicable. Applicants may use a range of 
479  methods to identify burdens and opportunities to reduce them, including interviews with 
480  stakeholders or use of focus groups, as well as assessing the workflows associated with 
481  implementing REMS requirements in various health care settings. For example, workflows may 
482  vary based on the outpatient or inpatient setting and, for the latter, whether the medication is 
483  formulary, nonformulary, or supplied by the patient. An analysis of workflow during the REMS 
                                                           
29 FDA Background Document: Impact of REMS on the Healthcare Delivery System and Patient Access. Public 
Meeting, October 5-6, 2015. Available from: https://www.fda.gov/Drugs/NewsEvents/ucm441308.htm 
13  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
484  design phase will also provide opportunities to identify postimplementation inefficiencies and 
485  make program improvements to reduce burden. Applicants are encouraged to explore additional 
486  methods to identify burdens associated with REMS with ETASU and describe these in their 
487  proposed REMS submission. For example, a time and motion study could potentially be used to 
488  evaluate the time required to complete a REMS activity and identify a more efficient operation 
489  thus reducing healthcare delivery burden. 30,31 
490    
491  The REMS Assessment Plan should describe the metrics, data sources, and analytical tools that 
492  applicants intend to use to assess REMS burdens following program implementation. Applicants 
493  should submit protocols for all studies assessing REMS burden to the Agency for review and 
494  comment prior to conducting the assessment (see section IV.C above).  
495   
496  In their REMS Assessment Reports, applicants should include the results of the assessment of 
497  REMS burden and evaluate the degree to which the observed burden compares to the expected 
498  burden across different categories of stakeholders or settings. The report should also include any 
499  additional areas of potential burden that may have been identified during program 
500  implementation. For example, applicants could demonstrate that they have assessed how REMS 
501  burden was reduced in certain settings by integrating verification of safe use requirements into 
502  electronic systems such as pharmacy practice management systems. The report could also 
503  include the settings that were unable to employ compatible electronic systems. Finally, the report 
504  could identify areas of opportunity and potential strategies for further reducing known or newly 
505  identified burdens.  
506   
507  B.  Assessing Barriers to Patient Access 
508   
509  Despite efforts during the REMS development to minimize barriers to patient access, certain 
510  patients may still find it difficult to access a drug that is subject to a REMS with ETASU. 
511  Assessing the impact of REMS with ETASU on barriers to patient access to the medication is an 
512  important part of the overall REMS performance assessment.   
513   
514  Identifying potential barriers to patient access should begin during the REMS design phase, and 
515  applicants should try to minimize any identified potential barriers to access. For example, when 
516  REMS place significant burdens on healthcare systems, some providers (e.g., prescribers, 
517  pharmacies, or clinics) may choose not to prescribe the drug because they may be unwilling to 
518  participate in the REMS; or, it may be difficult for a patient to find a participating prescriber in 
519  their geographical area, affecting the patients access to the drug. REMS program requirements 
520  may result in delays to patient access of a drug in ways that were not anticipated during the 
521  design phase.  
522   
                                                           
30 Time and Motion Studies Database US Department of Health and Human Services (US) Agency for Healthcare 
Research and Quality National Resource Center; Health Information Technology Available at: 
https://healthit.ahrq.gov/health-it-tools-and-resources/evaluation-resources/time-and-motion-studies-database  
31 Lo HG, Newmark LP, Yoon C, et al. Electronic Health Records in Specialty Care: a Time-Motion Study. JAMIN 
2007;14(5): 609-615. Available at: https://academic.oup.com/jamia/article/14/5/609/721654 
14  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
523  The REMS Assessment Plan should describe the metrics, data sources, and methodologies that 
524  applicants intend to use to assess barriers to patient access that are related to the REMS 
525  following implementation. This may be particularly challenging when a new product is approved 
526  with a REMS with ETASU, as uptake of the product may be slow. Patient interviews and focus 
527  groups may again be useful to identify any negative impact on patient access of REMS with 
528  ETASU.  
529   
530  Drug utilization data may be used to evaluate REMS impact on patient access. However, the use 
531  of these data should be planned carefully, because drug utilization data alone cannot be relied 
532  upon to fully describe the impact of REMS on patient access. For example, drug utilization data 
533  may show a reduction in the use of a drug product if a REMS with ETASU is imposed post-
534  approval. However, additional analysis may be needed to determine whether the reduction in use 
535  is consistent with the goals of the REMS or whether it suggests a reduction in the use of the 
536  product in patients for whom the benefit outweighs the risks. As with any data sources and 
537  methodologies used to assess REMS, protocols should be submitted to the Agency for review 
538  and comment prior to the assessment (see section IV.C above).  
539   
540  Applicants should include the results of the assessment of access as specified in the REMS 
541  Assessment Plan and the degree to which the observed barriers to access compare to those 
542  expected across different categories of stakeholders or settings. The assessment should also 
543  include any additional barriers to patient access that may have been identified during program 
544  implementation. Finally, the assessment could identify areas of opportunity and potential 
545  strategies for further reducing barriers to access.  
546   
547  VI.  REMS ASSESSMENT SUBMISSIONS 
548   
549  A.  REMS Assessment Plan  
550   
551  1.  Overview 
552   
553  The REMS Assessment Plan should be presented in a separate section in the REMS Supporting 
554  Document and submitted to the Agency with the proposed REMS submission.  
555   
556  The REMS Assessment Plan should include an overview that depicts the REMS goals and 
557  objectives, REMS requirements and the REMS materials, and how each requirement is going to 
558  be assessed, including the assessment category, selected metrics, related data sources, analytical 
559  tools, and the frequency of assessment. The overview may be in a tabular format, as shown in 
560  Appendix 2A. An additional column could be added that maps the assessment plan metric, data 
561  source, or analytical tool to the location in the REMS Supporting Document where the details of 
562  the methodology or protocol are described further.  
563   
564  The REMS Assessment Plan described in the FDA approval letter will include only the REMS 
565  assessment categories and their corresponding metrics. 
566   
15  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
567  2.  Methodology or Protocols Describing Sources of Data and Analytical Tools 
568   
569  The REMS Assessment Plan should also include a thorough description of and rationale for each 
570  of the various data sources that will be used to collect data for the REMS assessment. 
571  Additionally, the metrics used to measure each objective, the types of data that will be analyzed, 
572  the analytical tools that the applicant intends to use, and the rationale for the selection of the 
573  types and source of data should be provided. The assessment plan should include as much detail 
574  as possible at the time the REMS Supporting Document is submitted.  
575   
576  If the assessment instruments and methodology have not been determined prior to approval of 
577  the REMS and are not included in the REMS Supporting Document, the applicant should include 
578  a timeline for submission of the methodology or the study protocol in the REMS Supporting 
579  Document. For example, if the applicant is planning a drug utilization study or 
580  pharmacoepidemiology study to assess an aspect of the REMS but has not fully developed the 
581  study protocol, the planned timeframe for protocol submission should be indicated. Applicants 
582  are encouraged to submit a full protocol for specific studies (e.g., surveys, 
583  pharmacoepidemiology studies, RCA) at least 90 days before the assessments will be conducted 
584  to allow for Agency review and feedback.  
585   
586  If there are changes to a previously submitted assessment instrument or methodology, the REMS 
587  Supporting Document should be updated and submitted for review and comment at least 90 days 
588  before the assessments will be conducted. Updates to the REMS Supporting Document may be 
589  included in a new document that references previous REMS Supporting Document submission(s) 
590  for unchanged portions. Alternatively, updates may be made by modifying the complete previous 
591  REMS Supporting Document, with all changes marked and highlighted.  
592   
593  Prominently identify the submission containing the assessment instruments and methodology 
594  with the following wording in bold capital letters at the top of the first page of the submission:  
595   
596  NDA/BLA/ANDA [assigned #] REMS CORRESPONDENCE (insert concise description of 
597  content in bold capital letters, e.g., UPDATE TO REMS SUPPORTING DOCUMENT - 
598  ASSESSMENT METHODOLOGY)  
599   
600  B.  REMS Assessment Reports 
601   
602  Submit the REMS Assessment Report to FDA according to the timetable for submission of 
603  assessments in the approved REMS. FDA suggests that the REMS Assessment Report include 
604  the following sections:  
605   
606  Cover Page 
607  Table of contents 
608  1.  Executive summary 
609  2.  Introduction 
610  3.  Background 
611  (a) REMS goals and objectives, REMS requirements, and REMS materials 
16  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
612  (b) REMS history32  
613  (c) Pending supplements  
614  4.  REMS Assessment Plan 
615  (a) An overview in tabular format (see Appendix 2A) or other format 
616  (b) Details of the information summarized in the overview of the REMS 
617  Assessment Plan (see section VI.A above). Methodology used to support 
618  REMS assessment (e.g., survey, other methodology) can be included or 
619  referenced to prior submission  
620  5.  Brief summary of previous assessments 
621  6.  Results or summary of findings of each assessment metric 
622  7.  Discussion: overall assessment of whether the REMS goals and objectives are 
623  being met 
624  8.  Proposed modifications to the REMS or revisions to the REMS Assessment Plan 
625   
626  The cover page should include the reporting time point (e.g., 18-month assessment), the date of 
627  the REMS Assessment Report, and the assessment reporting period (e.g., September 30, 2015, 
628  reporting period August 1, 2014, to July 31, 2015). Following the table of contents, the report 
629  should include an executive summary of the findings and conclusions. This should be followed 
630  by the introduction, background section, which includes the REMS goals and objectives, REMS 
631  requirements, and REMS materials that were in place during the assessment reporting period, a 
632  REMS history, and any pending supplements.  
633   
634  If survey results are included in the REMS Assessment Report, the survey instruments should be 
635  included in the report or reference the submission in which these instruments were provided. 
636  Any new methodologies or protocols, as well as those not previously submitted to the Agency, 
637  should be included in the REMS Assessment Report. The REMS Assessment Report should also 
638  include a brief summary of the previous assessments, including the key results and the overall 
639  conclusions. 
640   
641  The results section should include the aggregate data collected for each metric, a written 
642  summary of the data that was analyzed, key results, and a description of limitations. When 
643  appropriate, the data should be reported for the reporting period and cumulatively, and trends in 
644  performance compared to previous periods should be reported and discussed.  
645   
646  The discussion section should provide an evaluation of whether each individual goal or objective 
647  is being met as well as an overall evaluation and conclusion as to whether the REMS is meeting 
648  its goals and objectives. In some REMS Assessment Reports, it may be clear that one goal or 
649  objective is being met while others are not, leading to the conclusion that the REMS is only 
650  partially meeting its goals. If data are not robust enough to determine whether a goal or objective 
                                                           
32 The REMS history outlines all changes made to the REMS since its approval. The REMS history should be 
similar in format to the summary that application holders include in labeling supplements that provides the history of 
changes made to the product label. The REMS history should be in a tabular format as described in the guidance for 
industry Risk Evaluation and Mitigation Strategies: Modifications and Revisions. Available from: 
https://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM441226.pdf 
 
17  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
651  is being met this should be stated along with recommendations for additional data sources or 
652  analyses that would allow a conclusion to be made in a future assessment. In addition, for REMS 
653  with ETASU, a conclusion should be made regarding whether the burden on the healthcare 
654  delivery system is being minimized to the extent practicable, whether the ETASU are unduly 
655  burdensome on patient access to medication, and an explanation for these conclusions.  
656   
657  The final section should include any proposed modifications to the REMS, as well as the basis 
658  for the proposed modifications (e.g., address REMS compliance issues, reduce burdens, 
659  overcome barriers to patient access, improve efficiencies). Any considerations to address 
660  reducing burdens that are identified through the REMS assessment should focus on how burden 
661  could be reduced without adversely affecting the overall impact of the program on the safe use of 
662  the drug. For example, if a laboratory test is required as part of a safe use condition, burden 
663  could be potentially minimized if issues identified in the ordering, scheduling, and reporting of 
664  the results are addressed. In this example, the requirement for testing would remain; however, 
665  the processes surrounding it might be reevaluated to reduce inefficiency. Recommendations for 
666  proposed REMS modification or elimination should be supported by an analysis of the expected 
667  impact or a plan to assess the impact of any proposed modification or elimination on the future 
668  safe use of the drug. 
669   
670  The applicant should also consider proposed revisions to the REMS Assessment Plan if 
671  additional information is needed to determine whether the goal of the REMS is being met or if 
672  there are aspects of the REMS that are no longer necessary to assess.  
673   
674  On the first page of the submission of a REMS Assessment Report of approved REMS, 
675  prominently identify its content in bold capital letters at the top of the page:  
676   
677  NDA/BLA/ANDA [assigned #]  
678  REMS ASSESSMENT  
679   
680  If a REMS Assessment Report is submitted as a part of another submission, it is critical to 
681  provide complete identifying information on the submission so that it can be tracked, routed, and 
682  reviewed appropriately. In each case, the first page of the submission should prominently 
683  identify the submission as providing a REMS ASSESSMENT in bold capital letters at the top 
684  of the page. This wording on the first page of the submission should be combined with any other 
685  applicable content identification.  
686 
18  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
687   APPENDIX 1: EXAMPLE - ASSESSMENT CATEGORY AND METRICS 
688   
689  The following table lists the two REMS assessment categories and examples of associated 
690  metrics for a REMS whose objective is to ensure that safe use conditions are documented prior to 
691  administration of a drug; and which requires that the drug can only be dispensed by in a certified 
692  healthcare setting after completion of a checklist by a health care provider confirming safe use 
693  conditions.  
694   
REMS Assessment Category  Example Metrics 
Program implementation and    Number and geographical location of certified 
operations  healthcare settings that are able to dispense the drug 
  Number of certified healthcare settings that have 
administered drug at least once during the reporting 
period  
  Number of healthcare settings that were unable to 
become certified and reasons why 
  Summary of regions in the United States that do not 
have certified healthcare settings that are able to 
dispense drug  
  Summary of audit findings including the number of 
certified settings that lacked protocols or order sets or 
policies to ensure completion of the checklist prior to 
administering drug 
  Analysis of call center data indicating: 
o  Difficulty locating a certified healthcare facility 
o  Number of and reason for delays in drug dispensings 
Safe use behavior    Number and percent of doses not administered because 
safe use condition on checklist was not performed 
  Number of patients for whom prescribers were 
contacted because of an issue identified by the pre-
infusion/pre-administration checklist and subsequent 
outcome of contact 
  Number of counseling visits before or during initiation 
of treatment (claims database) 
  Number of patients receiving a concomitant medication 
known to potentiate risk being mitigated (drug 
utilization database) 
695   
696 
19  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
697  APPENDIX 2: EXAMPLE REMS ASSESSMENT PLAN OVERVIEW 
698   
699  Appendix 2A shows a REMS Assessment Plan overview in tabular format. The overview is for a 
700  fictitious REMS and illustrates the relationship among the REMS program (goals, objectives, 
701  stakeholder requirements and materials) and the REMS Assessment Plan (data sources, metrics, 
702  methodologies, performance thresholds).  
703   
704  Drug X REMS: 
705   
706  Drug X is a fictitious new medication shown in clinical trials to improve survival and function in 
707  patients with a rare progressive cardiac condition for which there are no FDA-approved 
708  therapies. However, up to 10 percent of patients experienced an acute, serious neurologic adverse 
709  reaction within 30 minutes of dosing that, if detected could be rapidly treated with a 
710  corticosteroid injection. The likely prescribers of Drug X are cardiologists and the medication is 
711  to be administered by intravenous infusion once a month. No risk factors were identified to help 
712  predict which patients were at risk of experiencing the acute neurologic adverse event.  
713   
714  The sponsor filed a new drug application with a proposed REMS including a communication 
715  plan and elements to assure safe use, because of the importance of educating cardiologists and 
716  other Drug X prescribers about detecting acute neurologic symptoms, the need to ensure early 
717  observation and detection and access to corticosteroid treatment, and the possibility of long-term 
718  sequelae if the adverse event is not promptly treated.  
719   
720  The goal of the Drug X REMS is to mitigate the risk of developing long-term adverse neurologic 
721  effects. Objectives of the REMS include: (1) ensuring healthcare providers are educated about 
722  the risk (including that prior exposure with an associated adverse event is a risk factor), the 
723  neurologic symptoms they need to monitor following dosing, and how to treat the acute 
724  neurologic symptoms should they occur; (2) ensuring healthcare providers observe patients for at 
725  least 30 minutes following Drug X dosing and that the treatment setting has corticosteroid 
726  injection readily available; and (3) ensuring healthcare providers enroll all patients who 
727  experience the adverse event into a patient registry to avoid re-exposure. 
728   
729  The Drug X REMS program includes the following requirements and REMS materials: (1) a 
730  REMS letter, (2) healthcare provider training, (3) certification of the healthcare setting to ensure 
731  that the setting has corticosteroid treatment available (i.e., as a safe use condition), and (4) a 
732  registry for enrolling patients who experience the acute neurologic adverse event. The Drug X 
733  REMS assessment reporting frequency is 6 months, 12 months, and annually after initial 
734  approval of the REMS. 
735     
20  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
Appendix 2A: Drug X REMS Assessment Plan 
REMS 
Methodology/ 
Assessment Plan  Assessment 
Data Sources/  Performance  Protocol 
Objective  Requirement  REMS Materials   Category/  Metrics  Report: 
Analytical Tools  Threshold  Location and 
Domain  Frequency of 
Date Submitted 
metric reporting 
Goal: Mitigate the risk of long-term adverse neurological events 
Ensure healthcare  Communication  -REMS letter  Outreach and  -Number and date  -Sponsor REMS  In 6 and 12 month    REMS letters list 
providers are  plan  -REMS website  communication  of REMS letters  database  reports  obtained by 
educated about the  sent/opened/  -Sponsor REMS  [organization] 
risk, the  returned  website 
neurologic  -Website access 
symptoms that 
statistics 
must be monitored 
Prescriber  -Prescriber  Implementation  -Date of product  -Sponsor REMS  In 6 month, 12  100% of  -Healthcare 
following dosing, 
certification  training program  and operations  launch  database  month, and annual  prescribers trained  provider 
and how to treat 
acute neurologic    -Number and  - Successful  reports  -Score 100% on  knowledge survey 
symptoms should    specialty of  completion of  post training  protocol 
they occur    certified  Post-Training  knowledge  (submitted mo, 
prescribers  Knowledge  assessment  day, year) 
- Number of  assessment  -80% knowledge  - 
rate on evaluation 
prescription orders   
of healthcare 
written by  - Evaluation of 
provider 
noncertified  healthcare 
knowledge (e.g., 
prescribers and  provider 
surveys of 
disposition   knowledge (e.g., 
knowledge) 
surveys of 
knowledge)  
 
Ensure healthcare  Pharmacy and  Pharmacy and  Implementation  Number and type  -Sponsor REMS  In 12 month and  100% pharmacies   
setting has  Healthcare facility  Healthcare setting  and operations  of pharmacy and  database  annual reports  and healthcare 
corticosteroid  certification  attestation of safe    healthcare    settings that 
injection available    use conditions    facilities enrolled  receive and 
(authorized    administer drug 
representative  are certified  
enrollment form) 
Safe use    Evaluation of safe  -Results of audits  Audit protocol  In 12 month and  -100% of audited  -Audit plan 
conditions  use conditions  of healthcare    annual reports  healthcare settings  submitted 
facility including  have policies and  [month/day/year] 
21  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
REMS 
Methodology/ 
Assessment Plan  Assessment 
Data Sources/  Performance  Protocol 
Objective  Requirement  REMS Materials   Category/  Metrics  Report: 
Analytical Tools  Threshold  Location and 
Domain  Frequency of 
Date Submitted 
metric reporting 
Goal: Mitigate the risk of long-term adverse neurological events 
summary of  procedures in  -Survey 
findings and any  place to ensure  methodology 
corrective actions  corticosteroids are  submitted 
taken  available  [month/day/year] 
-Results of  -80% overall 
surveys of  knowledge score 
certified  on survey 
prescribers, 
pharmacies and 
healthcare settings 
regarding need for 
and availability of 
corticosteroids  
Ensure healthcare  Safe use  Postinjection  Evaluation of safe  -Number of  -Sponsor REMS  In 6 month, 12  100% of   
providers observe  conditions  observation form  use conditions  patients who  database  month, and annual  observation forms 
patients for at  received the drug  -FAERS  reports  completed 
least 30 minutes  -Number of  100% of audited 
following  healthcare settings 
completed forms 
Treatocil dosing  have policies and 
documenting 
procedures in 
observation period 
place to ensure 
of 30 minutes 
patients are 
-Reports of 
monitored for 30 
adverse events 
minutes following 
with and without 
infusion 
observation 
Ensure healthcare  Safe use  -Post-injection  -Program  -Total number of  -Sponsor REMS  In 12 month and    Registry protocol 
providers enroll  conditions  observation form  infrastructure and  patients who  database  annual reports 
all patients who  -Neurologic event  performance  received the drug  -Sponsor registry 
experience acute  registry  -Evaluation of  -Number of cases  -Surveys of 
neurologic 
safe use  of acute  certified 
adverse event 
conditions  neurologic events  prescribers, 
following dosing 
-Adverse event  following dosing  pharmacies, and 
into a registry 
surveillance  healthcare facilities 
22  Contains Nonbinding Recommendations 
Draft  Not for Implementation 
 
REMS 
Methodology/ 
Assessment Plan  Assessment 
Data Sources/  Performance  Protocol 
Objective  Requirement  REMS Materials   Category/  Metrics  Report: 
Analytical Tools  Threshold  Location and 
Domain  Frequency of 
Date Submitted 
metric reporting 
Goal: Mitigate the risk of long-term adverse neurological events 
-Sponsor's adverse 
event database/ 
FAERS 
 
23