Department of Health and Human Services

OFFICE OF

INSPECTOR GENERAL

FDA LACKS COMPREHENSIVE DATA 

TO DETERMINE WHETHER RISK 

EVALUATION AND MITIGATION 

STRATEGIES IMPROVE DRUG SAFETY

Daniel R. Levinson 

Inspector General

February 2013

OEI-04-11-00510
EXECUTIVE SUMMARY: FDA LACKS COMPREHENSIVE DATA TO

DETERMINE WHETHER REMS IMPROVE DRUG SAFETY, OEI-04-11-00510

WHY WE DID THIS STUDY 

The Food and Drug Administration (FDA) requires manufacturers to submit structured 

plans, known as Risk Evaluation and Mitigation Strategies (REMS), for drugs associated 

with known or potential risks that may outweigh the drugs’ benefits. If FDA does not 

properly monitor REMS’ performance, it cannot ensure that the public is provided 

maximum protection from a drug’s known or potential risks. However, FDA does not 

have the authority to require, but may request, drug manufacturers (i.e., sponsors) to 

submit specific information regarding REMS’ effectiveness. 

HOW WE DID THIS STUDY

We reviewed approved REMS since program inception in 2008 through 2011 and 

conducted structured interviews with FDA officials about FDA’s efforts to evaluate 

REMS components. We also reviewed 49 sponsors’ REMS assessments and FDA’s 

reviews of these assessments to determine the extent to which sponsors’ assessments 

were complete, were submitted to FDA within required timeframes, and indicated that 

REMS were meeting their goals. We also determined whether FDA evaluated the 

elements to assure safe use (ETASU) of one drug in each year of the program, as 

required by Federal law.

WHAT WE FOUND

FDA approved 199 REMS between 2008 and 2011, 99 of which were still required in 

2012. Nearly half of sponsor assessments for the 49 REMS we reviewed did not include 

all information requested in FDA assessment plans, and 10 were not submitted to FDA 

within required timeframes. FDA determined that 7 of the 49 REMS we reviewed met 

all of their goals. However, FDA has not identified reliable methods to assess the 

effectiveness of REMS. Finally, FDA’s assessment review times exceeded its goal of 

60 days for all but one sponsor assessment, which reduces sponsors’ time to make 

suggested changes before submitting subsequent assessments. 

 

WHAT WE RECOMMEND

Our findings raise concerns about the overall effectiveness of the REMS program. To 

address these concerns, we made seven recommendations regarding FDA’s evaluation 

and assessment of REMS and its review of sponsors’ REMS assessments. FDA 

concurred with six of our recommendations. For the remaining recommendation, to seek 

legislative authority to make FDA assessment plans enforceable, FDA did not state 

whether it concurred or did not concur. However, FDA agreed that this recommendation 

should be considered if another opportunity arises to pursue legislative changes to the 

statutory provisions that describe the requirements for REMS assessments. 
 

TABLE OF CONTENTS

Objectives .................................................................................................... 1 

Background .................................................................................................. 1 

Methodology .............................................................................................. 10 

Findings ..................................................................................................... 14 

FDA approved 199 REMS between 2008 and 2011, 99 of which 

were still required in 2012 ............................................................. 14 

Nearly half of the 49 sponsor assessments we reviewed did not 

include all information requested in FDA assessment plans, and 

10 were not submitted to FDA within required timeframes .......... 15 

FDA determined that 7 of the 49 REMS we reviewed met all of 

their goals ....................................................................................... 16 

FDA has not identified reliable methods to evaluate the 

effectiveness of REMS .................................................................. 18 

FDA’s assessment review times exceeded its 60-day goal for 

all but one sponsor assessment ...................................................... 20 

Conclusion and Recommendations ............................................................ 22 

Agency Comments and Office of Inspector General Response .... 25 

Appendixes ................................................................................................ 27 

A: Assessment Review Memorandums Provided by FDA ........... 27 

B: Agency Comments ................................................................... 29 

Acknowledgments ..................................................................................... 34 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 1

OBJECTIVES

1. To describe the Risk Evaluation and Mitigation Strategies (REMS) the 

Food and Drug Administration (FDA) approved for drugs between

program inception in 2008 and 2011. 

2. To determine the extent to which sponsors’ REMS assessments 

indicate that REMS are complete, are meeting their goals, and are 

submitted to FDA within required timeframes. 

3. To determine the extent to which FDA has evaluated the effectiveness 

of REMS.

4. To determine the extent to which FDA reviews sponsors’ REMS 

assessments within its goal of 60 days.

BACKGROUND 

The Federal Food, Drug, and Cosmetic Act (FDCA), as amended by the 

Food and Drug Administration Amendments Act of 2007 and the Food and 

Drug Administration Safety and Innovation Act of 2012, authorizes FDA 

to require REMS for certain drugs and biological products (hereinafter 

referred to as drugs) to assure that their benefits outweigh their risks.

1, 2

 

REMS are structured plans to manage specific risks of drugs that are 

effective but associated with known or potential risks (e.g., death, injury) 

that, without REMS, may outweigh benefits. When FDA requires a 

REMS, the drug manufacturer (i.e., sponsor) must develop, implement, 

and assess it.3 FDA reviews and approves each REMS. 

1 Biological products include vaccines, blood and blood products, gene therapies, and 

allergenic extracts. Therapeutic biological products generally fall under the definition of 

“drugs,” which include substances intended for the diagnosis, cure, mitigation, treatment, or 

prevention of disease and which meet certain other requirements. FDA, Drugs@FDA 

Glossary of Terms. Accessed at 

http://www.fda.gov/Drugs/informationondrugs/ucm079436.htm#B on June 15, 2012. 2 The Food and Drug Administration Amendments Act (P.L. 110-85, Sept. 27, 2007) added 

§ 505-1 to the FDCA. The Food and Drug Administration Safety and Innovation Act 

(FDASIA) (P.L. 112-144, July 9, 2012) further amended FDCA § 505-1. 3 Under FDCA § 505-1, the responsible person submitting a covered application to market a 

drug or the holder of such approved application is required to submit, implement, and assess 

REMS. In this report, we refer to the “responsible person” as the sponsor. Generally, a drug 

manufacturer submits and holds a drug’s application and is the drug’s sponsor. FDA, Drug 

Development and Review Definitions. Accessed at 

http://www.fda.gov/Drugs/DevelopmentApprovalProcess/HowDrugsareDevelopedandAp

proved/ApprovalApplications/InvestigationalNewDrugINDApplication/ucm176522.htm 

on April 24, 2012.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 2

FDA has the authority to, among other things, deem a drug misbranded or 

impose civil monetary penalties on sponsors that violate approved 

REMS.

4, 5

 

Structure of REMS

Each REMS includes one or more overall goals to achieve a particular 

safety-related health outcome or understanding by patients and/or health 

care providers of a drug’s risks.6 Examples of REMS goals include 

preventing fetal exposure to a drug and informing prescribers, patients, 

and pharmacists of the serious risks and safe-use conditions for a drug. 

REMS include one or more strategies to provide additional safety in the 

usage of certain drugs. For example, REMS may require sponsors to 

create additional safety information for patients or to ensure that health 

care providers that prescribe the drug are specially certified.

7

For brand-name drugs, each REMS must include a timetable for sponsors 

to submit REMS assessments.8, 9 The standard timetable requires sponsors 

to submit assessments of REMS’ effectiveness by 18 months, 3 years, and 

7 years after approval of the REMS.10 FDA may require sponsors to 

submit assessments at different intervals specified in the REMS or 

eliminate the timetable after 3 years if it determines that a drug’s risks 

4 A drug may be deemed to be misbranded if it is subject to a REMS and the sponsor fails to 

comply with a requirement of the REMS. FDCA § 502(y). Misbranded drugs may not be 

introduced into, delivered, or received into interstate commerce. FDCA § 301 (a) and (c). 5 Sponsors that violate REMS requirements may be subject to civil monetary penalties of up 

to $250,000 per violation. Civil monetary penalties are not to exceed $1 million in a single 

proceeding. Civil monetary penalties may increase to $10 million for continued violations. 

FDCA § 303(f)(4)(A). Additionally, a sponsor may not introduce or deliver for introduction 

into interstate commerce an approved drug if a REMS is required for the drug and the sponsor 

fails to maintain compliance with the requirements of the approved REMS or with other 

requirements of § 505-1 of FDCA. FDCA § 505(p). 6

 FDA, Draft Guidance for Industry: Format and Content of Proposed REMS, REMS 

Assessments, and Proposed REMS Modifications, p. 9. Accessed at 

http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guid

ances/UCM184128.pdf on June 23, 2011. This draft guidance will represent FDA’s current 

thinking on this topic when finalized, but will not bind FDA or the public.

7 FDCA §§ 505-1(c)(2), 505-1(e)(2), and 505-1(f)(3)(A). 8 FDCA §§ 505-1(c)(1) and (d). The timetable for assessments specifies when sponsors will 

submit the assessment to FDA, not when the assessment will be performed. 

9 REMS for drugs subject to abbreviated new drug applications (i.e., generics) do not require a 

timetable for assessments. FDCA § 505-1(i)(1)(A–B). 

10 FDA often requires more frequent assessments (e.g., at 6-month intervals or annually) for 

drugs associated with the most serious risks. 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 3

have been adequately identified and assessed and are being adequately 

managed.11 

FDA may require one or more of the following components in addition to 

the timetable for assessments: medication guide or patient package insert, 

a communication plan, or elements to assure safe use (ETASU).12 FDA 

determines which of these additional components to require according to a 

drug’s risks. 

Medication Guides and Patient Package Inserts. Medication guides are 

paper handouts that include FDA-approved information about the safe and 

effective use of a drug. Medication guides are not unique to REMS; FDA 

oversees their content and format under separate regulations and may

require them without requiring a REMS.13 Some REMS may require 

patient package inserts, which also contain drug safety information, in 

addition to medication guides.14, 15

Communication Plans. Communication plans describe how sponsors will 

inform health care providers about a drug’s risks and/or the components of 

a REMS. These plans may describe how a sponsor will send letters to 

health care providers and disseminate information through professional 

societies about a drug’s risks.16

11 FDCA §§ 505-1(d)(4)(A) and (C). When FDA eliminates a REMS’ timetable for 

assessments, it may continue to require the REMS but the sponsor is no longer required to 

submit assessments. 

12 FDCA §§ 505-1(e)(1–3) and 505-1(f)(1–3). 13 If FDA requires a medication guide not as a component of a REMS, sponsors are not 

required to assess them. See 21 CFR 208.

14 Patient package inserts are required for some prescription drugs, including estrogens and 

oral contraceptive products and are considered to be part of a drug’s labeling. Manufacturers 

of other drugs may voluntarily provide patient package inserts. Required patient package 

inserts must be dispensed to patients with the drug. 21 CFR 310.501, 21 CFR 310.515. See 

also, FDA, Guidance: Drug Safety Information – FDA’s Communication to the Public, p. 10. 

Accessed at 

http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guid

ances/ucm072281.pdf on December 19, 2012. 15 FDA does not expect to require both patient package inserts and medication guides 

frequently. FDA may allow sponsors to include an existing patient package insert in a REMS 

instead of a medication guide or require sponsors to convert a patient package insert into a 

medication guide, if it meets medication guide requirements. FDA, Guidance for Industry: 

Format and Content of Proposed REMS, REMS Assessments, and Proposed REMS 

Modifications, p. 10. Accessed at 

http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guid

ances/UCM184128.pdf on June 23, 2011. 16 FDCA § 505-1(e)(3).
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 4

ETASUs. FDA may require ETASUs for drugs that are associated with a 

serious adverse drug experience if, without the ETASUs, FDA would not 

approve the drug or would withdraw the drug’s approval.17, 18 If drugs 

were initially approved without ETASUs, FDA must determine that other 

REMS components are not sufficient to mitigate risks before it requires

ETASUs.19 

FDA can require one or more of the following ETASUs: 

• health care providers who prescribe the drug have certain training, 

experience, or certification; 

• pharmacies, practitioners, and/or other health care settings that 

dispense the drug are specially certified;

• the drug is dispensed only in certain health care settings (e.g., 

hospitals, appropriately equipped physicians’ offices); 

• the drug is dispensed only to patients with evidence or 

documentation (e.g., laboratory test results, signed 

acknowledgement of risks) that they can safely use the drug; 

• each patient using the drug is subject to monitoring (e.g., patients 

must periodically contact the prescriber or undergo laboratory 

tests); and

• each patient using the drug is enrolled in a registry.20

Each ETASU must (1) correspond to the specific serious risk listed in the 

labeling of a drug; (2) considering such risk, not be unduly burdensome on 

patient access to a drug, particularly considering patients with serious or 

life-threatening conditions and patients who have difficulty accessing 

health care; and (3) to the extent practicable, minimize the burden on the 

health care delivery system.21

17 A serious adverse drug experience is an adverse event associated with the use of a drug that 

results in death, the immediate risk of death, inpatient hospitalization or prolonging existing 

hospitalization, a persistent or significant incapacity or substantial disruption of the ability to 

conduct normal life functions, a congenital anomaly or birth defect, or a medical or surgical 

intervention to prevent these outcomes. FDCA § 505-1(b)(4). 18 FDCA § 505-1(f)(1)(A).

19 FDCA § 505-1(f)(1)(B). 20 FDCA § 505-1(f)(3)(A–F). 21 FDCA §§ 505-1(f)(2)(A), (C), and (D). Specifically, to minimize the burden on the health 

care system, ETASUs should conform with the ETASUs of drugs with similar serious risks 

and be designed to be compatible with established distribution, procurement, and dispensing 

systems for drugs. FDCA § 505-1(f)(2)(D). Additionally, each ETASU must be posted 

publicly by the Secretary within 30 days with an explanation of how it will mitigate the safety 

risk. FDCA § 505-1(f)(2)(B). 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 5

When FDA requires certain ETASUs, the REMS may also include an 

implementation system.22 Sponsors may use implementation systems to 

take reasonable steps to monitor, evaluate, and improve the 

implementation of ETASUs.23 Examples of implementation systems 

include requiring sponsors to conduct periodic audits of health care 

settings that use the drug or to maintain a database of all certified entities 

to ensure that certification requirements are met.24

REMS components are not exclusive. For example, REMS that require 

ETASUs generally also require a communication plan and/or a medication 

guide. FDA refers to three types of REMS according to the most intensive 

(i.e., primary) required component: medication guide, communication 

plan, and ETASU REMS. See Table 1 for examples of drugs that require 

the various REMS components. 

22 ETASUs that may include an implementation system are: requiring special certification for 

pharmacies, practitioners, or other health care providers that dispense the drug; dispensing the 

drug in certain settings; and dispensing the drug only to patients with certain evidence or 

documentation. FDCA § 505-1(f)(4). 23 FDCA § 505-1(f)(4). 24 FDA, Guidance for Industry: Format and Content of Proposed REMS, REMS Assessments, 

and Proposed REMS Modifications, pp. 14–15. Accessed at 

http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guid

ances/UCM184128.pdf on June 23, 2011.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 6

Table 1: Examples of Drugs Requiring REMS

Drug Approved 

Indication Risks Required REMS Components* Primary REMSComponent

Chantix Smoking cessation

Neuropsychiatric 

symptoms, including 

changes in behavior, 

agitation, depressed 

mood, and suicidal 

thoughts and actions 

Timetable for assessments, 

medication guide

Medication 

guide

Vibativ

Treatment of adult 

patients with 

complicated skin and 

skin structure 

infections caused by 

susceptible Grampositive bacteria

Unintended exposure 

of pregnant women

resulting in 

teratogenicity

Timetable for assessments, 

medication guide, communication 

plan

Communication 

plan

OxyContin

controlledrelease tablets

Management of 

moderate to severe 

pain when a 

continuous, aroundthe-clock, opioid 

analgesic is needed 

for an extended 

period

Potential for abuse, 

misuse, overdose, and 

addiction

Timetable for assessments, 

medication guide, ETASUs ETASU

Avandia Management of type 

II diabetes Myocardial infarction

Timetable for assessments;

medication guide; communication 

plan; ETASUs, including an 

implementation system

ETASU

Source: Office of Inspector General (OIG) analysis of approved REMS, 2011. 

* The required components of these REMS may change over time. This table includes REMS components required as of 

December 31, 2011. 

Criteria for Requiring REMS

FDA may require a REMS before approving an application to market a 

drug (i.e., preapproval REMS) or after a drug is on the market (i.e., 

postapproval REMS).25 FDA must consider the following factors to

determine whether a preapproval REMS is necessary:

• the estimated size of the population likely to use the drug, 

• the seriousness of the disease or condition that the drug treats, 

• the expected benefit of the drug, 

• the duration of treatment, 

25 FDCA § 505-1(a).
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 7

• the seriousness of any known or potential adverse events related 

to the drug and the background incidence (i.e., frequency) of 

these events in the population likely to use the drug, and 

• whether the drug is a new molecular entity.26, 27 

FDA may require a postapproval REMS if it becomes aware of new safety 

information after a drug is on the market and determines that a REMS is 

necessary to ensure that the benefits of the drug outweigh the risks.

28, 29 

For example, FDA required a postapproval REMS for the drug Wellbutrin 

in response to reports of suicide-related events in patients using the drug 

for smoking cessation.30 

REMS Approval Process

When FDA requires a REMS, it notifies the drug’s sponsor in a REMS 

notification letter. The REMS notification letter instructs the sponsor to 

submit a proposed REMS containing the components FDA determines are

necessary. FDA reviews the proposed REMS and approves it after an 

interactive process with the sponsor.31 After FDA approves a proposed 

REMS, it notifies the drug’s sponsor in a REMS approval letter.

FDA works with sponsors to develop an FDA assessment plan for each 

REMS, which is included in the REMS approval letter. Assessment plans 

describe the information that FDA requests sponsors to collect to complete 

a valid assessment of whether a REMS is meeting its goals.

32 For 

example, FDA may request that a sponsor’s assessment include the 

26 A new molecular entity is an active ingredient that has never been marketed in the United 

States in any form. FDA, Drugs@FDA Glossary of Terms. Accessed at 

http://www.fda.gov/Drugs/informationondrugs/ucm079436.htm#N on September 2, 2011. 27 FDCA § 505-1(a)(1)(A–F). 28 FDCA § 505-1(a)(2)(A). New safety information is derived from scientific data (e.g., 

clinical trials, adverse event reports) about serious or unexpected risks associated with a drug 

that FDA has become aware of since the drug was approved, since the REMS was required, or 

since the last assessment of the approved REMS. These scientific data may be based on a new 

analysis of existing information. FDCA § 505-1(b)(3). 

29 FDA also considers the factors listed in FDCA § 505-1(a)(1)(A–F) when requiring a 

postapproval REMS.

30 FDA, NDA 018644/S-041, NDA 023858/S-048 Approval Letter, February 26, 2010. 

Accessed at 

http://www.accessdata.fda.gov/drugsatfda_docs/appletter/2010/018644s041,020358s048l

tr.pdf on November 1, 2011. 31 FDA, CDER 21st Century Review Process Desk Reference Guide, p. 53. Accessed at 

http://www.fda.gov/downloads/AboutFDA/CentersOffices/CDER/ManualofPoliciesProc

edures/UCM218757.pdf on May 3, 2012. 32 FDA, Guidance for Industry: Format and Content of Proposed REMS, REMS Assessments, 

and Proposed REMS Modifications, p. 18. Accessed at 

http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guid

ances/UCM184128.pdf on June 23, 2011. 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 8

number of patients enrolled in a user registry or an evaluation of patients’ 

understanding of a drug’s risks.

Sponsors’ Assessments of REMS

Sponsors of brand-name drugs must submit their assessments of REMS

for FDA’s review according to the timetable in the approved REMS.33 

FDCA requires FDA to review sponsors’ assessments but does not require 

FDA to perform its own assessments of each REMS.

34 FDA may require 

sponsors to submit an additional assessment if one is needed to evaluate 

whether the REMS should be modified.35 Sponsors may also submit 

voluntary assessments of REMS at any time.36 Sponsors’ assessments 

must include information required by FDCA and should also include 

information requested in FDA assessment plans. 

Requirements for Sponsors’ Assessments. At the time of our review, 

assessments of REMS with ETASUs were required to include an 

evaluation of the extent to which each ETASU is meeting the goal stated 

in the REMS in addition to complying with the approved timetable. 

Under current amendments to FDCA, all REMS assessments must 

include, with respect to each goal of the REMS, an assessment of the 

extent to which REMS are meeting the goals or of whether the goals or 

REMS should be modified.37 FDCA does not require sponsors to include 

additional information about the effectiveness of REMS in assessments. 

FDA may take enforcement actions (e.g., deem a drug misbranded, impose 

civil monetary penalties) if sponsors fail to meet the assessment 

requirements outlined in FDCA. 38 However, FDA does not have authority 

to take enforcement actions if sponsors’ assessments do not include 

information requested in FDA assessment plans. 

FDA’s Reviews of Sponsors’Assessments. Multidisciplinary teams made 

up of staff in FDA’s Division of Risk Management within the Office of 

Surveillance and Epidemiology, the respective drug review division(s) in 

the Office of New Drugs, and the Office of Compliance review sponsors’ 

REMS assessments. Staff in the Division of Risk Management complete 

review memorandums that summarize the teams’ findings. The teams

33 FDCA § 505-1(d). Sponsors of generic drugs are not required to submit REMS 

assessments according to a timetable.

34 FDCA § 505-1(h)(1). FDA’s requirement to assess the ETASUs of at least one drug each 

year is distinct from its responsibility to review sponsors’ assessments. 

35 FDCA § 505-1(g)(2)(C). 36 FDCA § 505-1(g)(1). 

37 FDCA § 505-1(g)(3). Because this was not required at the time of our review, we 

determined only whether assessments of REMS with ETASUs included assessments of the 

extent to which each ETASU was meeting the stated goal in the REMS.

38 FDCA §§ 502(y) and 303(f)(4)(A). 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 9

review assessments for completeness and determine whether REMS are 

meeting their goals or whether modifications to the REMS or FDA

assessment plans are required. A Division of Risk Management review 

memorandum documents whether a sponsor’s assessment (1) is complete, 

(2) indicates the REMS is meeting its goals or needs modifications, and 

(3) identifies any deficiencies that may affect a REMS’s ability to mitigate 

risks.39

The Office of Compliance also completes review memorandums, which 

document whether sponsors submit complete assessments according to the 

approved timetable. However, the Office of Compliance defers the 

analysis of assessment data and of whether REMS are meeting their goals 

to the Division of Risk Management.40

FDCA requires FDA to promptly review sponsors’ assessments.41

Although FDA does not define “promptly,” FDA officials indicated that 

their goal is to complete all assessment reviews within 60 days of 

submission if it does not discuss REMS modifications with sponsors.42 

Figure 1 illustrates the relationship of the REMS, FDA assessment plan, 

sponsor’s assessment, and FDA review memorandum. 

Figure 1: Relationship of REMS Documents

Source: OIG analysis of REMS documents, 2012.

FDA’s Evaluation of REMS

Federal law (i.e., FDCA) requires FDA to evaluate the ETASUs of at least 

one REMS each year to determine whether the ETASUs (1) assure safe 

39 Federal regulations and FDA policy do not define a complete assessment. “Complete” is 

defined later for the purposes of this study.

40 Based on OIG review of Office of Compliance review memorandums. 41 FDCA § 505-1(h)(1). 42Since FDA set this goal, some FDCA provisions regarding the review of sponsors’ 

assessments have changed because of FDASIA. Before FDASIA was enacted, FDA opened a 

90-day discussion period to work with sponsors to determine whether REMS modifications 

were necessary and to discuss possible modifications. FDASIA removed the provision 

regarding the 90-day discussion period, but FDA may still discuss REMS modifications with 

sponsors and is still required to promptly review sponsors’ assessments.

REMS

•Enforceable 

FDA document 

outlining REMS 

components

FDA Assessment 

Plan

•Unenforceable 

FDA document 

outlining how 

sponsors 

should assess 

REMS' 

effectiveness

Sponsor's 

Assessment

•Document 

containing 

sponsor's 

assessments of 

REMS 

effectiveness

FDA Review 

Memorandum

•Document 

containing 

FDA's review of 

sponsor's 

assessment
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 10

use of the drug; (2) are unduly burdensome on patient access to the drug;

and (3) to the extent practicable, minimize the burden on the health care 

delivery system.43 

In 2008, FDA issued a 5-year plan (2008–2012) to enhance and modernize 

its drug safety activities. In the 5-year plan, FDA states that it will 

develop a plan to identify, develop, validate, and assess the effectiveness 

of REMS components.44 

METHODOLOGY

Scope

We reviewed and described the REMS that FDA approved between 

program inception in 2008 and 2011. Additionally, we reviewed sponsors’ 

most recent assessments to determine whether they were complete. We 

reviewed FDA’s review memorandums for each of the sponsor 

assessments to determine whether FDA notified sponsors about 

incomplete assessments, concluded that REMS were meeting their goals, 

and completed its reviews within 60 days. We also determined the extent 

to which FDA has evaluated REMS with ETASUs, as required by Federal 

law. 

Our review included 199 REMS. We reviewed sponsors’ assessments and 

FDA review memorandums for the 49 REMS that required assessments 

according to their approved timetables as of December 31, 2011.45 We 

also reviewed the second most recent assessment (i.e., prior assessment) 

and FDA review memorandums for the 14 REMS that required multiple 

assessments during our timeframe.

46 See Appendix A for a list of REMS 

and associated review memorandums provided by FDA. 

We limited our review of sponsors’ assessments to those required by the 

timetable in each REMS. We did not include REMS for generic drugs in 

our review because sponsors of these drugs are not required to submit 

43 FDCA § 505-1(f)(5)(B). FDA has been required to complete this evaluation for one drug 

with an ETASU REMS each year since 2008. 44 FDA, PDUFA Drug Safety 5-year Plan 2008–2012, December 2008, p. 8. Accessed at 

http://www.fda.gov/downloads/ForIndustry/UserFees/PrescriptionDrugUserFee/UCM119

244.pdf on September 23, 2011. The 5-year plan is distinct from FDA’s required evaluation 

and is not specific to ETASUs.

45 Fifty-seven REMS required at least one assessment by December 31, 2011. We did not

include eight REMS in this analysis because FDA did not provide review memorandums for 

them. This does not necessarily indicate that FDA failed to review these assessments. These 

assessments may have been under FDA review during our data collection period. 46 Twenty REMS required multiple assessments by December 31, 2011. We did not include 

six REMS in this analysis because FDA did not provide prior review memorandums for four 

REMS and did not provide the most recent review memorandums for two. 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 11

REMS assessments according to a timetable.

47 We did not review 

sponsors’ assessments for which FDA did not provide review 

memorandums. Additionally, we did not independently determine the 

extent to which REMS are meeting their goals and relied on FDA 

statements in review memorandums. 

Data Collection and Analysis

We reviewed relevant statutes, policies, and guidance documents related to 

REMS. We conducted structured interviews with and requested 

documentation from FDA officials. Finally, we analyzed REMS, FDA 

assessment plans, sponsors’ assessments, and FDA review memorandums. 

Describing REMS Approved Between 2008 and 2011. We used FDA’s 

online list of REMS to calculate the number of REMS that FDA approved 

between March 25, 2008, when FDA’s authority to require REMS began, 

and December 31, 2011, when we collected our data. We described the 

components of these REMS and determined how many REMS were no 

longer required as of December 31, 2011. 

Determining Completeness and Timeliness of Sponsors’ Assessments. We 

analyzed the 49 most recent sponsor assessments to determine whether

sponsors included information requested in FDA assessment plans and 

required by Federal law. We based our determination on an independent 

review of sponsor assessments and FDA assessment plans. We did not 

rely on statements in FDA review memorandums to determine whether 

assessments were complete. However, if FDA review memorandums 

indicated that FDA no longer requested sponsors to submit certain pieces 

of information, we did not consider the information to be missing. To 

determine whether sponsors submitted their assessments according to the 

approved timetable, we compared assessment submission dates to the 

timetable in each approved REMS. We also requested documentation of 

any enforcement actions FDA has taken in response to sponsor 

assessments that did not comply with Federal requirements. 

Determining Whether REMS Were Meeting Goals. We analyzed FDA 

review memorandums associated with the 49 most recent sponsor 

assessments to determine whether FDA concluded that REMS were 

47 Our review includes the drug isotretinoin, which was approved as an acne treatment and 

marketed as Accutane. FDA required a single, shared-system REMS for Accutane and its 

generic equivalents but Accutane’s approval was withdrawn at the request of its sponsor. 

Several of its generic equivalents remain on the market and are under the originally approved 

single, shared-system REMS. Sponsors of these generic drugs have submitted assessments in 

accordance with the timetable included the Accutane REMS. 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 12

meeting their goals.48 We considered determinations of whether REMS 

were meeting their goals to be in four categories: (1) the REMS was 

meeting its goals, (2) the REMS was not meeting its goals, (3) FDA 

reviewers could not determine whether the REMS was meeting its goals, 

and (4) FDA reviewers did not determine whether the REMS was meeting 

its goals. 

We did not independently determine whether REMS were meeting their 

goals; instead, we relied on statements from FDA reviewers in review 

memorandums. For example, a review memorandum for a REMS that 

FDA determined was not meeting its goals states that “... we find that the 

goals are not being met.” A review memorandum for a REMS for which 

FDA could not determine was meeting its goals states that “... due to a 

lack of valid data, we cannot comment on whether the REMS is meeting 

its educational goals.” We considered REMS to be in the fourth category 

if review memorandums did not contain a statement from the reviewer 

regarding whether the REMS was meeting all of its goals. 

For the 14 REMS that required at least one prior assessment, we analyzed 

the prior review memorandums to determine whether reviewers stated that 

the assessments identified deficiencies that may affect the REMS’ ability 

to mitigate risks.

49 If reviewers noted deficiencies in prior assessments, 

we determined whether the reviewers noted the same deficiencies in the 

most recent sponsor assessments. We also compared FDA’s 

determinations about whether REMS were meeting their goals in review 

memorandums for the prior assessments to determinations in the most 

recent review memorandums. 

Determining the Extent of FDA’s Evaluation of REMS. We conducted 

structured interviews with FDA officials regarding (1) the evaluation of 

ETASUs, as required by FDCA; and (2) FDA’s plan to identify, develop, 

validate, and assess the effectiveness of REMS components, as described 

in its 5-year plan (2008–2012) to enhance and modernize drug safety 

activities. 

Determining FDA’s Assessment Review Times. We determined FDA’s 

review times for sponsor assessments for which FDA did not discuss 

REMS modifications with sponsors. For these 29 sponsor assessments, 

we used the submission dates on the assessments and the dates on the 

48 Reviewers’ determinations regarding whether REMS are meeting each goal are found 

in the discussion section of review memorandums. Only Division of Risk Management 

review memorandums include this determination; however, the memorandums reflect the 

reviews of a multidisciplinary team.

49 Reviewers note deficiencies in the discussion and recommendations sections of review 

memorandums.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 13

associated FDA review memorandums to determine whether FDA 

completed reviews within its goal of 60 days.

50

Limitations

We did not review FDA communication with sponsors regarding their 

assessment reviews. We based reports of actions that FDA took after 

assessment reviews (e.g., discussing REMS modifications with sponsors) 

on the recommendations stated in FDA’s Division of Risk Management 

review memorandums, not on FDA communication with sponsors. 

Standards

This study was conducted in accordance with the Quality Standards for 

Inspection and Evaluation issued by the Council of the Inspectors General 

on Integrity and Efficiency.

50 For Division of Risk Management review memorandums, we used the dates of review as 

the review completion dates. For Office of Compliance review memorandums, with one 

exception, we used the electronic signature dates for the review completion dates.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 14

FINDINGS

FDA approved 199 REMS between 2008 and 2011, 

99 of which were still required in 2012 

FDA requires REMS to manage specific risks of drugs that are effective 

but associated with known or potential risks (e.g., death, injury) that, 

without REMS, may outweigh the drugs’ benefits. Of the 199 approved 

REMS, 119 required medication guides only and 48 required

communication plans. For drugs with the most serious risks, FDA 

approved 32 REMS with ETASUs.

51 These drugs would not be approved 

or their approval would be withdrawn without the ETASUs.52 Of the 199 

approved REMS, FDA approved 74 preapproval REMS in response to 

information in drug applications. The remaining 125 postapproval REMS 

were required in response to new safety information discovered after the 

drugs were already on the market or for drugs with ETASUs already in 

effect before REMS program initiation.

53 

As of December 31, 2011, FDA no longer required 100 of the 199 

approved REMS. Ninety-two of the REMS that are no longer required 

included medication guides only, seven required communication plans, 

and one required ETASUs. Medication guides existed prior to REMS and 

may be required for drugs without REMS. Following program inception, 

FDA required REMS for all drugs that required medication guides. 

However, in November 2011, FDA published a guidance document 

describing when medication guides would be required without also 

requiring REMS. It also provided sponsors of REMS requiring only 

medication guides with information about how to submit proposals to 

eliminate the REMS.

54 

51 REMS components are not exclusive. The 48 REMS that required communication plans 

may have also required medication guides, and the 32 REMS that required ETASUs may have 

also required communication plans and/or medication guides. We categorized the REMS 

according to their primary component as of December 31, 2011.

52 Of the 32 REMS with ETASUs, 28 also required implementation systems. 53 FDA worked with sponsors to implement ETASUs for certain drugs prior to the REMS 

program. After REMS program initiation, FDA required sponsors of these drugs to submit 

REMS proposals in accordance with § 909(b)(3) of the Food and Drug Administration 

Amendments Act (P.L. 110-85, Sept. 27, 2007). 54 FDA, Guidance for Industry: Medication Guides – Distribution Requirements and 

Inclusion in REMS, pp. 7–8, November 2011. Accessed at 

http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guid

ances/UCM244570.pdf on August 2, 2012. This guidance states that it represents FDA’s 

current thinking on this topic but does not bind FDA or the public. 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 15

According to the approved timetable in each REMS, 57 REMS required

sponsors to submit at least 1 assessment by December 31, 2011.55 FDA 

provided review memorandums for 49 of these REMS.56 Of these 

49 REMS, 11 required medication guides only, 19 required 

communication plans, and 19 required ETASUs.57 

Nearly half of the 49 sponsor assessments we 

reviewed did not include all information requested in 

FDA assessment plans, and 10 were not submitted to 

FDA within required timeframes

FDA assessment plans request sponsors to submit specific information to 

enable FDA to determine whether REMS are meeting their goals. FDA 

reviewed 49 sponsor assessments as of December 31, 2011. Almost half 

(23 of 49) of the sponsor assessments did not include all of the 

information requested in FDA assessment plans. See Table 2 for the 

number of assessments that lacked information requested in assessment 

plans by primary REMS component. For example, one sponsor did not 

include the number of pharmacies that were deauthorized to dispense the 

drug because of noncompliance with the REMS, as requested in the 

assessment plan. The same sponsor also did not include the amount of the 

drug shipped to health care providers compared to actual patient orders, as 

requested. In review memorandums for the 23 assessments that did not 

include information requested in assessment plans, reviewers 

recommended that FDA notify the sponsors of 6 that their assessments 

were incomplete. FDA review memorandums for the remaining 17 

assessments did not indicate that FDA no longer requested sponsors to 

submit the missing information. 

55 This number does not include REMS that were no longer required as of December 31, 

2011, or REMS for generic drugs, which are not subject to a timetable for assessments.

56 For the remaining eight assessments, FDA had not completed assessment reviews as of 

December 31, 2011. As of December 31, 2011, FDA’s assessment reviews did not meet its 

goal of 60 days for three of these eight assessments. 

57 Of the 19 REMS with ETASUs, 12 required implementation systems.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 16

Table 2: Sponsors’ Assessments Lacking Information Requested in 

Assessment Plans

Primary REMS Component

Number of Assessments 

Lacking Information 

Requested in Assessment 

Plans

ETASU 10

Communication plan 8 

Medication guide 5 

 Total 23

 Source: OIG analysis of sponsor assessments.

Sponsors and health care providers have expressed concerns about the 

challenges associated with collecting data on the compliance of third 

parties (e.g., patients, pharmacies, drug distributors). Specifically, 

concerns about patient confidentiality and the lack of a standardized 

format for sponsor assessments may contribute to the lack of data in some 

assessments.58 

FDA does not have authority to take enforcement actions against sponsors 

that do not include requested information in their REMS assessments. 

Some items requested in FDA assessment plans are also required by 

Federal law. An assessment of the extent to which the ETASUs met the 

goal stated in the approved REMS was required by Federal law, but was 

missing from seven assessments.

59 

Further, 10 of 49 of sponsor assessments were not submitted to FDA by 

the dates specified in the approved timetables, as required by FDCA. 

These assessments were submitted between 3 and 70 days after the dates 

specified in the timetables, with a median of 17 days. FDA has the 

authority to take enforcement actions against sponsors that do not meet 

Federal requirements for REMS assessments. However, since program 

inception in 2008, FDA has not done so. 

FDA determined that 7 of the 49 REMS we reviewed 

met all of their goals

Using limited information in sponsor assessments, FDA determined that 7 

of the 49 REMS we reviewed were meeting all of their goals and that 21 

58 FDA, Transcript of REMS Public Meeting, July 28, 2010, pp. 208 and 284. Accessed at 

http://www.fda.gov/downloads/Drugs/NewsEvents/UCM224950.pdf on July 3, 2012.

59 Assessments of the extent to which the ETASUs met the goals stated in the approved 

REMS were required only for the 19 REMS with ETASUs. FDASIA changed this 

requirement to no longer apply specifically to ETASUs. According to FDCA § 505-1(g)(3), 

all sponsor assessments must now include, with respect to each goal of the REMS, 

assessments of the extent to which the REMS are meeting the goals or of whether the 

goals or REMS should be modified. 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 17

were not. FDA review memorandums indicated that FDA could not 

determine whether 17 REMS were meeting all of their goals. FDA review 

memorandums did not contain statements regarding whether the remaining 

four REMS were meeting all of their goals.60 

As shown in Table 3, FDA determined that 1 of 19 REMS with ETASUs, 

which are required for the riskiest drugs, was meeting all of its goals. In 

review memorandums, FDA reviewers stated that eight REMS with 

ETASUs were not meeting all goals and that they could not determine 

whether eight others were meeting all goals. FDA did not determine 

whether two REMS with ETASUs were meeting their goals. 

 Table 3: Determinations of Whether REMS Were Meeting Goals 

Primary REMS 

Component

Meeting All 

Goals 

Not Meeting 

All Goals

Unable To 

Determine

Did Not 

Determine Total

ETASU 1 8 8 2 19 

Communication plan 2 10 5 2 19

Medication guide 4 3 4 0 11

Total 7 21 17 4 49

 Source: OIG analysis of FDA review memorandums, 2012. 

FDA most often determined that REMS were not meeting their 

goals because of deficiencies in patient and prescriber 

awareness of drug risks

When FDA determined that REMS were not meeting all of their goals, it 

most often identified deficiencies related to patient awareness of risks (14 

of 21 assessments) and/or prescriber awareness of risks (12 of 21 

assessments). For example, because a survey showed that patients had a 

low understanding of key risk messages, FDA determined that one REMS 

was not meeting its goal to inform patients of a drug’s risks. FDA 

discussed REMS modifications with sponsors for 16 of the 21 REMS it 

determined were not meeting all of their goals. 

FDA did not discuss REMS modifications with sponsors of the remaining 

five REMS it determined were not meeting their goals.

61 

60 If an FDA review memorandum did not contain a statement from the reviewer regarding 

whether the REMS was meeting its goals, we concluded that FDA did not make that 

determination.

61 In some cases, FDA may have suggested that sponsors modify REMS assessments or 

education materials rather than modifying the REMS. FDA indicated that these suggestions 

were not consistently included in review memorandums.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 18

Nearly all deficiencies identified by prior FDA assessment 

reviews were still present in the sponsors’ most recent 

assessments 

FDA identified 22 deficiencies that may affect a REMS’ ability to mitigate 

risks.

62 In 11 prior assessments we reviewed, 19 of 22 deficiencies 

identified by FDA’s prior assessment reviews were identified again in 

reviews of the most recent assessments. Reviewers identified deficiencies 

that included low patient exposure to medication guides, low 

understanding of risk messages among patients and/or prescribers, and 

poor compliance with the REMS among health care providers. 

FDA could not determine whether REMS were meeting their 

goals most often because of incomplete information in 

assessments

For 8 of 17 assessments, FDA could not determine whether REMS were

meeting their goals because of a lack of information included in the 

sponsors’ assessments. For example, FDA could not determine whether 

one REMS was meeting its goal to inform patients of the drug’s risks 

because the sponsor did not include an assessment of patients’ 

understanding of risks in the assessment. FDA could not determine 

whether an additional three REMS were meeting their goals because

assessments were required too early in the implementation of the REMS to 

draw conclusions.63

FDA could not determine whether five REMS were meeting their goals

because of concerns about the quality of the surveys used in the sponsor 

assessments.64 FDA’s concerns generally related to small sample sizes or 

the use of surveys that did not meet FDA’s standards. However, FDA does 

not specify in its assessment plans the sample size that would enable it to 

determine whether a REMS is meeting its goals.

FDA has not identified reliable methods to evaluate the 

effectiveness of REMS

FDA has not developed a plan to identify, develop, validate, and assess the 

effectiveness of REMS components, as stated in its 5-year plan 

(2008–2012) to enhance and modernize drug safety activities. 

62 A sponsor’s second most recent assessment is considered the prior assessment. FDA 

provided review memorandums for 14 prior assessments and identified deficiencies in 11.

63 Two of these assessments were required 6 months after REMS approval. The third 

assessment was required 12 months after approval, but the drug’s introduction to the market 

was delayed, which delayed data collection for the assessment.

64 Two of these five REMS assessments also lacked information. FDA reviewers did not 

explain why they were unable to determine whether REMS were meeting their goals for the 

remaining sponsors’ assessments.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 19

Additionally, FDA has not met the Federal requirement to evaluate the 

ETASUs of one REMS each year. Further, FDA has identified limitations 

in methods used to evaluate the effectiveness of REMS. 

FDA has evaluated the ETASUs of one REMS since program 

inception in 2008

According to FDCA, FDA must evaluate the ETASUs for at least one drug 

each year to determine whether they (1) assure safe use of the drug; (2) are 

unduly burdensome on patient access to the drug; and (3) to the extent 

practicable, minimize the burden on the health care delivery system. 

Between 2008 and 2011, FDA approved 32 REMS with ETASUs to 

address drugs with the most serious risks. As of December 31, 2011, FDA 

had completed an evaluation for one drug with an ETASU. 

On December 1, 2011, FDA’s Drug Safety and Risk Management 

Advisory Committee met to evaluate the ETASUs of isotretinoin.

65 At the 

meeting, FDA and stakeholders discussed the impact of the isotretinoin 

REMS on patients and the health care delivery system. FDA and industry 

representatives noted several challenges associated with completing the 

required evaluation, including the lack of baseline (pre-REMS) utilization 

data and the inability to account for changes in utilization because of 

factors not related to the REMS. 

FDA stated that it had not formally evaluated a REMS with ETASUs in 

the first 3 years of the REMS program because it was focused on 

developing REMS and implementing the program. Additionally, FDA 

stated that REMS must be implemented for a sufficient period of time 

before they can be properly evaluated.66 

Because FDA has completed this evaluation for just 1 of 32 drugs with 

ETASUs, it has limited data to demonstrate that the remaining REMS with 

ETASUs effectively ensure safe use of drugs or meet statutory 

requirements to minimize burdens on patients and the health care system. 

65 Isotretinoin was originally approved as an acne treatment and marketed as Accutane. 

Accutane is no longer on the market, but its generic equivalents remain on the market under a 

single, shared-system REMS for isotretinoin. The ETASUs require that prescribers and 

pharmacies be specially certified, patients have evidence of safe-use conditions, and sponsors 

maintain pregnancy registries. Minutes of the Advisory Committee meeting are available to 

the public. Accessed at 

http://www.fda.gov/AdvisoryCommittees/CommitteesMeetingMaterials/Drugs/DrugSafet

yandRiskManagementAdvisoryCommittee/ucm250295.htm on July 19, 2012. 66 Prior to the use of REMS, FDA used Risk Minimization Action Plans (RiskMAPs) to 

manage risks of certain drugs. The ETASUs for isotretinoin were fully implemented in 2006 

under a RiskMAP. Claudia Karwoski, Opening Remarks for Drug Safety and Risk 

Management and Ophthalmic Advisory Committee Meeting, December 1, 2011. Accessed at 

http://www.fda.gov/downloads/AdvisoryCommittees/CommitteesMeetingMaterials/Drug

s/DermatologicandOphthalmicDrugsAdvisoryCommittee/UCM285047.pdf on September 

20, 2012. 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 20

FDA has identified limitations with using survey data to assess 

REMS

FDA often requests that sponsors include results of surveys regarding

patients’ and/or health care providers’ knowledge of risks in REMS 

assessments.

67 FDA made such requests in each of the 49 sponsor 

assessments we reviewed. Of these 49 sponsor assessments, 40 included 

survey data in response to this request and 9 did not. 

However, in a White Paper issued in June 2012, FDA identified several 

limitations in some sponsors’ surveys. These included sample sizes too 

small to draw conclusions, survey populations that do not reflect the 

demographics of the target population, bias caused by convenience 

samples, and the lack of objective standards to measure knowledge of 

risks.68 FDA states that because of these limitations, surveys may not

always be the best method for assessing the effectiveness of REMS 

educational components.69 

FDA’s assessment review times exceeded its 60-day 

goal for all but one sponsor assessment

FDCA requires FDA to promptly review sponsor assessments.70 Although 

FDA does not define “promptly,” FDA officials indicated that their goal is 

to complete all assessment reviews within 60 days of a sponsor’s 

assessment submission if FDA does not discuss REMS modifications with 

sponsors. FDA did not discuss REMS modifications with sponsors for 

29 of the 49 sponsor assessments we reviewed.71 FDA’s assessment

review times exceeded 60 days for 28 of these 29 sponsor assessments. 

FDA’s average assessment review time was 73 days and the median 

review time was 69 days.72 Reviews of three sponsor assessments were 

completed at least 120 days after the sponsor submitted them.

73 If FDA 

67 FDA, Risk Evaluation and Mitigation Strategy Assessments: Social Science Methodologies 

to Assess Goals Related to Knowledge, pp. 1–2. Accessed at 

http://www.fda.gov/downloads/Drugs/NewsEvents/UCM301966.pdf on September 13, 

2012.

68 Convenience samples are selected on the basis of what is most feasible for the researcher 

and are therefore not always representative of the larger population.

69 Ibid, pp. 5–6. 70 FDCA § 505-1(h)(1). 71 FDA’s 60-day goal for reviewing sponsors’ assessments does not apply to the 

remaining 20 assessments for which it discussed REMS modifications with sponsors. 

72 The average and median review times do not include reviews that were not complete as 

of December 31, 2011.

73 The FDA review memorandum for one of these REMS indicated that it is included in 

the opioid class REMS, which was under development at the time of the review and may 

have contributed to delays in review completion. The FDA review memorandums for the 

remaining two REMS do not indicate a reason for delays.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 21

does not review sponsor assessments in a timely manner, sponsors may 

have limited time to implement suggested changes to a REMS before 

submitting the next assessment. 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 22

CONCLUSION AND RECOMMENDATIONS

FDA requires REMS for drugs associated with known or potential risks 

that, without REMS, may outweigh the drugs’ benefits. REMS are 

enforceable, structured plans to manage specific risks associated with 

drugs. However, FDA does not have the authority to take enforcement 

actions against sponsors that do not include all information requested in 

FDA assessment plans. If FDA does not have comprehensive data to

monitor the performance of REMS, it cannot ensure that the public is 

provided maximum protection from a drug’s known or potential risks. 

Between program inception in 2008 and 2011, FDA approved 199 REMS, 

99 of which were still required in 2012. However, FDA does not have 

comprehensive data from sponsor assessments to determine whether

REMS are meeting their goals because many sponsor assessments are 

incomplete or include data that do not meet FDA’s standards. 

Additionally, using the limited information in sponsor assessments, FDA 

determined that 7 of 49 REMS were meeting all of their goals and that 

21 were not, raising questions about the effectiveness of REMS. 

Further, FDA has completed the federally required evaluation for 1 of 

32 drugs with ETASUs since program inception. Therefore, FDA has 

limited data to demonstrate that the remaining REMS with ETASUs 

effectively ensure safe use of drugs or meet statutory requirements to 

minimize burdens on patients and the health care system. Additionally, 

FDA has not identified reliable methods for evaluating REMS. 

Finally, FDA’s assessment review times exceeded 60 days for all but one 

sponsor assessment, which may limit sponsors’ ability to implement 

suggested changes to the REMS before submitting the next assessment. 

In conclusion, our findings raise concerns about the overall effectiveness 

of the REMS program. To address these concerns, we recommend that 

FDA: 

Develop and implement a plan to identify, develop, validate, 

and assess REMS components

This plan should fulfill FDA’s commitment in its 5-year plan and outline 

ways that FDA will assess the effectiveness of REMS beyond reviewing 

sponsors’ assessments. 

FDA should also identify and implement reliable methods to assess the 

effectiveness of REMS. FDA should decrease its reliance on survey data 

in sponsors’ assessments and work with sponsors and health care providers 

to develop more accurate evaluation methods. 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 23

Additionally, FDA should continue to hold discussions with stakeholders, 

similar to the public meetings held in 2010 and 2012, about the issues and 

challenges associated with assessing the effectiveness of REMS 

components.74

Identify REMS that are not meeting their goals and take 

appropriate actions to protect the public health

FDA should consistently discuss potential REMS modifications with 

sponsors when it determines that a REMS is not meeting its goals. FDA 

should work with sponsors to determine the most appropriate 

modifications to address the REMS’ deficiencies. If a REMS undergoes 

multiple modifications and continues not to meet its goals, FDA should 

consider removing the drug from the market. 

If FDA cannot determine whether a REMS is meeting its goals, it should 

work with sponsors to obtain any additional information that it needs to 

make this determination. FDA should not wait until it reviews the next 

sponsor assessment to determine whether a REMS is meeting its goals. 

Evaluate the ETASUs of one REMS each year as required by 

Federal law

FDA should determine whether the ETASUs (1) assure safe use of the 

drug; (2) are unduly burdensome on patient access to the drug; and (3) to 

the extent practicable, minimize the burden on the health care delivery 

system. FDA fulfilled this requirement for the first time in 2011 and 

should continue to conduct formal evaluations of at least one REMS with 

ETASUs each year. 

Through these evaluations, FDA can determine how effective various 

ETASUs are, whether ETASUs cause barriers to patient access, and which 

ETASUs are the most burdensome for health care providers. Evaluations 

would also inform FDA’s decisions about the most effective ETASUs for 

mitigating specific risks. FDA should use this information to change or 

eliminate any ETASUs that are both burdensome and ineffective in 

assuring safe use of a drug.

Identify incomplete sponsor assessments and work with 

sponsors to obtain missing information 

FDA should consistently notify sponsors that their assessments are 

incomplete and request that sponsors provide the missing information as 

soon as possible. FDA should also work with sponsors to obtain the 

missing information.

74 75 Fed. Reg. 34453 (June 17, 2010) and 77 Fed. Reg. 26292 (May 3, 2012).
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 24

Clarify expectations for sponsors’ assessments in FDA 

assessment plans 

FDA should make assessment plans as specific as possible to ensure that 

data included in sponsors’ assessments will enable FDA to determine 

whether REMS are meeting their goals. For example, FDA should specify 

sample sizes needed to complete valid surveys. FDA should use 

assessment plans to establish quality standards for data to be submitted in 

sponsors’ assessments. If FDA cannot determine whether a REMS is 

meeting its goals because of the quality of data in a sponsor’s assessment, 

it should notify the sponsor of this concern and, if necessary, clarify data 

standards for future assessments. 

Seek legislative authority to enforce FDA assessment plans 

FDA currently does not have authority to take enforcement actions when 

sponsors do not submit all information requested in assessment plans. 

FDA should work with the appropriate stakeholders to seek legislative 

authority to enforce assessment plans. This would allow FDA to take 

regulatory actions when a sponsor’s assessment did not include all items 

requested in the FDA assessment plan. Further, enforcement authority 

should encourage sponsors to submit the amount and quality of 

information FDA needs to determine whether REMS effectively mitigate 

risks.

Ensure that assessment reviews are timely

FDCA requires FDA to promptly review sponsor assessments. FDA 

officials indicated that their goal is to complete all assessment reviews 

within 60 days of sponsors’ assessment submissions if FDA does not 

discuss REMS modifications with sponsors. Since FDA set this goal, 

some FDCA provisions regarding the review of sponsors’ assessments 

have changed because of FDASIA. If FDA chooses to change this goal of 

60 days, it should define “promptly” as it pertains to reviewing sponsor 

assessments and complete assessment reviews within this timeframe. 

FDA should identify ways to complete assessment reviews within its goal 

of 60 days or the newly established timeframe. This would allow time for 

informed discussions with sponsors about REMS modifications, if 

necessary. This would also enable sponsors to make timely changes to 

REMS and/or REMS assessments in response to FDA’s review. To 

maximize review resources, FDA could prioritize assessment reviews for 

REMS with ETASUs as these REMS are required for drugs with the most 

serious risks and generally require more frequent assessments.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 25

AGENCY COMMENTS AND OFFICE OF INSPECTOR GENERAL 

RESPONSE 

In its comments on the draft report, FDA concurred with six of our 

recommendations and, for the remaining recommendation, agreed that 

seeking legislative change should be considered if another opportunity 

arises to do so. FDA agreed with the need to improve future REMS 

assessments but stated that there would always be challenges in measuring 

the impact of REMS. FDA provided historical context to describe the 

evolving nature of pharmaceutical risk management. FDA also provided 

information about the REMS Integration Initiative which was created in 

2011. 

With regard to our first recommendation, FDA noted that the REMS 

Integration Initiative will include contributions from key stakeholders on 

the development, implementation, and effectiveness of REMS tools. 

With regard to our second recommendation, FDA stated that it has worked 

and will continue to work with sponsors to determine the best response 

when it determines that a REMS is not meeting its goals. Additionally, 

FDA stated that it plans to develop draft guidance concerning how to write 

REMS goals and the metrics for determining whether these goals have 

been met. 

With regard to our third recommendation, FDA noted that it faced 

challenges, including insufficient experience in evaluating REMS and 

insufficient assessment data available, in meeting the Federal requirement 

to evaluate the ETASUs of at least one drug each year. FDA also noted 

that while it did not formally meet this requirement until 2011, it discussed

certain REMS with ETASUs at various advisory committees prior to 2011. 

With regard to our fourth recommendation, FDA stated that it has worked 

and will continue to work with sponsors to obtain information missing 

from assessments if that information is necessary to determine whether a 

REMS is meeting its goals. If FDA determines that the information is not 

necessary to determine whether a REMS is meeting its goals, it will 

reevaluate the need for the information. 

With regard to our fifth recommendation, FDA stated that it is committed 

to holding public workshops to gather stakeholder input. FDA plans to 

issue draft guidance on methodologies for assessing whether REMS are 

meeting are their goals and the impact of REMS on patient access and 

burden on the health care system. Additionally, FDA noted that it has 

reviewed and will continue to review proposed assessment protocols if 

sponsors submit them prior to initiating the REMS assessment.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 26

FDA did not explicitly concur with our sixth recommendation but agreed 

that it should be considered if another opportunity arises to pursue 

legislative changes to the statutory provisions that describe REMS 

assessment requirements. 

Finally, with regard to our seventh recommendation, FDA stated that it is 

examining the internal assessment review process for ways to improve the 

timeliness of FDA assessment reviews. 

We support FDA’s efforts to address these issues and encourage it to 

continue making progress in these areas. For the full text of FDA’s 

comments, see Appendix B. We made minor changes to the report based 

on FDA’s technical comments.
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 27

APPENDIX A

Assessment Review Memorandums Provided by FDA

Name of Drug

Division of Risk 

Management

Review 

Memorandum

Office of 

Compliance

Review 

Memorandum

Prior Assessment 

and Review

Memorandum(s)*

ETASU

Abstral sublingual tablets X X

Erythropoiesis stimulating 

agents X X X

Butrans transdermal system X

Caprelsa tablets X X

Entereg capsules X X X

Exalgo extended-release 

tablets X X X

Isotretinoin X X

Letairis tablets X X

Lumizyme X X

Nplate for subcutaneous 

injection X X X

Onsolis buccal soluble film X

Oxycontin

controlled-release tablets X X

Promacta tablets X X X

Revlimid capsules X X X

Sabril tablets and oral 

solution X X

Soliris injection X X X

Suboxone sublingual film X X X

Thalomid capsules X

Tracleer tablets X X

Communication Plan

Actemra injection X X

Ampyra extended-release 

tablets X

Botox and Botox Cosmetic

injection X X

Dulera inhalation aerosol X X

Dysport injection X X

Effient tablets X X

continued on next page
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 28

Appendix A, continued

Assessment Review Memorandums Provided by FDA

Name of Drug

Division of Risk 

Management 

Review 

Memorandum

Office of 

Compliance 

Review 

Memorandum

Prior 

Assessment and 

Memorandum(s)

Communication Plan

Embeda extended-release 

tablets X X X

Forteo injection X X

Kalbitor injection X X

Krystexxa injection X X

Multaq tablets X X X

Myobloc injection X X

Samsca tablets X

Stelara injection X X

Tasigna capsules X

Vibativ injection X

Xenazine tablets X X X

Xiaflex injection X X

Zortress tablets X X

Medication Guide

Actoplus Met tablets X

Actoplus Met XR 

extended-release tablets X

Actos tablets X

Duetact tablets X

Androgel 1.62% gel X X

Chantix tablets X X

Nucynta immediate release 

tablets X

Testim gel X X

Venlafaxine hydrochloride 

extended-release tablets X X

Vivitrol extended-release 

injectable suspension X

Zyban sustained-release 

tablets, Wellbutrin, 

Wellbutrin 

sustained-release tablets, 

and Wellbutrin 

extended-release tablets X X

 Total 49 33 14

*Some REMS had not required multiple assessments as of December 31, 2011. A blank in this column does not necessarily 

indicate that a sponsor did not complete a required assessment.
APPENDIX B 

Agency Comments 

..,.,-.... 

( t#.E m<:PARTMJ:NT OF HEAI.T!IA...lii.D HU\iAN SERV,lCES 

....~~ 

Food and Drug Administration 

Silver Spring , MD 20993 

DATE: 	 December 1 0, 2012 

TO: 	 Daniel R. Levinson 

Inspector General 

FROM: 	 Peter Lurie, MD, MPH 

Acting Associate Commissioner for Policy and Planning 

SUBJECT: 	 Food and Drug Administration's Response to Office of Inspector General 

Draft Report: FDA Lacks Conrprellensive Data to Determ i11e Whether REMS 

Improve Drug Safety (OEI-04-11-00510) 

FDA is providing the attached response to the Office of Inspector General's draft report. 

FDA appreciates the opportunity to comment on this draft report. 

Peter Lurie 

Acting Associate Commissioner for Policy and Planning 

Attachment 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 29 
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 30
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 31
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 32
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 33
 

FDA Lacks Comprehensive Data To Determine Whether REMS Improve Drug Safety (OEI-04-11-00510) 34

ACKNOWLEDGMENTS

This report was prepared under the direction of Dwayne Grant, Regional 

Inspector General for Evaluation and Inspections in the Atlanta regional 

office, and Jaime Durley, Deputy Regional Inspector General. 

Sarah Langford served as the lead analyst for this study. Central office 

staff who provided support include Clarence Arnold, Debra Roush, and 

Talisha Searcy. 
 

Office of Inspector General

http://oig.hhs.gov

The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, as 

amended, is to protect the integrity of the Department of Health and Human Services 

(HHS) programs, as well as the health and welfare of beneficiaries served by those 

programs. This statutory mission is carried out through a nationwide network of audits, 

investigations, and inspections conducted by the following operating components:

Office of Audit Services

The Office of Audit Services (OAS) provides auditing services for HHS, either by conducting 

audits with its own audit resources or by overseeing audit work done by others. Audits 

examine the performance of HHS programs and/or its grantees and contractors in carrying 

out their respective responsibilities and are intended to provide independent assessments of 

HHS programs and operations. These assessments help reduce waste, abuse, and 

mismanagement and promote economy and efficiency throughout HHS.

Office of Evaluation and Inspections

The Office of Evaluation and Inspections (OEI) conducts national evaluations to provide 

HHS, Congress, and the public with timely, useful, and reliable information on significant 

issues. These evaluations focus on preventing fraud, waste, or abuse and promoting 

economy, efficiency, and effectiveness of departmental programs. To promote impact, OEI 

reports also present practical recommendations for improving program operations. 

Office of Investigations

The Office of Investigations (OI) conducts criminal, civil, and administrative investigations 

of fraud and misconduct related to HHS programs, operations, and beneficiaries. With 

investigators working in all 50 States and the District of Columbia, OI utilizes its resources 

by actively coordinating with the Department of Justice and other Federal, State, and local 

law enforcement authorities. The investigative efforts of OI often lead to criminal 

convictions, administrative sanctions, and/or civil monetary penalties.

Office of Counsel to the Inspector General

The Office of Counsel to the Inspector General (OCIG) provides general legal services to 

OIG, rendering advice and opinions on HHS programs and operations and providing all 

legal support for OIG’s internal operations. OCIG represents OIG in all civil and 

administrative fraud and abuse cases involving HHS programs, including False Claims Act, 

program exclusion, and civil monetary penalty cases. In connection with these cases, OCIG 

also negotiates and monitors corporate integrity agreements. OCIG renders advisory 

opinions, issues compliance program guidance, publishes fraud alerts, and provides other 

guidance to the health care industry concerning the anti-kickback statute and other OIG 

enforcement authorities.