Exhibit P

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 1 of 24
Guidance for Industry
 

Good Pharmacovigilance
 

Practices and
 

Pharmacoepidemiologic
 

Assessment
 

U.S. Department of Health and Human Services
 

Food and Drug Administration
 

Center for Drug Evaluation and Research (CDER)
 

Center for Biologics Evaluation and Research (CBER)
 

March 2005
 

Clinical Medical
 

J:\!GUIDANC\6359OCC.doc 

03/22/05 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 2 of 24
Guidance for Industry
 Good Pharmacovigilance
 

Practices and
 

Pharmacoepidemiologic
 

Assessment
 

Additional copies are available from:
 

Office of Training and Communication
 

Division of Drug Information, HFD-240
 

Center for Drug Evaluation and Research
 

Food and Drug Administration
 

5600 Fishers Lane
 

Rockville, MD 20857
 

(Tel) 301-827-4573
 

http://www.fda.gov/cder/guidance/index.htm
 

or
 

Office of Communication, Training, and
 

Manufacturers Assistance, HFM-40
 

Center for Biologics Evaluation and Research
 

Food and Drug Administration
 

1401 Rockville Pike, Rockville, MD 20852-1448
 

http://www.fda.gov/cber/guidelines.htm.
 

(Tel) Voice Information System at 800-835-4709 or 301-827-1800
 

U.S. Department of Health and Human Services
 

Food and Drug Administration
 

Center for Drug Evaluation and Research (CDER)
 

Center for Biologics Evaluation and Research (CBER)
 

March 2005
 

Clinical Medical
 

J:\!GUIDANC\6359OCC.doc 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 3 of 24
 

 

Contains Nonbinding Recommendations 

TABLE OF CONTENTS
 

I. INTRODUCTION..............................................................................................................................................1
 

II. BACKGROUND ................................................................................................................................................1
 

A. PDUFA III’S RISK MANAGEMENT GUIDANCE GOAL.......................................................................................2
 

B. OVERVIEW OF THE RISK MANAGEMENT GUIDANCES .......................................................................................2
 

III. THE ROLE OF PHARMACOVIGILANCE AND PHARMACOEPIDEMIOLOGY IN RISK
 

MANAGEMENT .........................................................................................................................................................3
 

IV. IDENTIFYING AND DESCRIBING SAFETY SIGNALS: FROM CASE REPORTS TO CASE
 

SERIES.........................................................................................................................................................................4
 

A. GOOD REPORTING PRACTICE ...........................................................................................................................4
 

B. CHARACTERISTICS OF A GOOD CASE REPORT ..................................................................................................5
 

C. DEVELOPING A CASE SERIES............................................................................................................................6
 

D. SUMMARY DESCRIPTIVE ANALYSIS OF A CASE SERIES ....................................................................................7
 

E. USE OF DATA MINING TO IDENTIFY PRODUCT-EVENT COMBINATIONS ...........................................................8
 

F. SAFETY SIGNALS THAT MAY WARRANT FURTHER INVESTIGATION ..............................................................10
 

G. PUTTING THE SIGNAL INTO CONTEXT: CALCULATING REPORTING RATES VS. INCIDENCE RATES ................10
 

V. BEYOND CASE REVIEW: INVESTIGATING A SIGNAL THROUGH OBSERVATIONAL
 

STUDIES ....................................................................................................................................................................12
 

A. PHARMACOEPIDEMIOLOGIC STUDIES .............................................................................................................12
 

B. REGISTRIES ....................................................................................................................................................15
 

C. SURVEYS........................................................................................................................................................16
 

VI. INTERPRETING SAFETY SIGNALS: FROM SIGNAL TO POTENTIAL SAFETY RISK................17
 

VII. BEYOND ROUTINE PHARMACOVIGILANCE: DEVELOPING A PHARMACOVIGILANCE
 

PLAN .......................................................................................................................................................................18
 

J:\!GUIDANC\6359OCC.doc 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 4 of 24
 

 

 

 

 

Contains Nonbinding Recommendations 

Guidance for Industry1
 

Good Pharmacovigilance Practices and Pharmacoepidemiologic
 

Assessment
 

This guidance represents the Food and Drug Administration's (FDA's) current thinking on this topic. It 

does not create or confer any rights for or on any person and does not operate to bind FDA or the public. 

You can use an alternative approach if the approach satisfies the requirements of the applicable statutes 

and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for 

implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate 

number listed on the title page of this guidance. 

I. INTRODUCTION 

This document provides guidance to industry on good pharmacovigilance practices and 

pharmacoepidemiologic assessment of observational data regarding drugs, including biological 

drug products (excluding blood and blood components).2 Specifically, this document provides 

guidance on (1) safety signal identification, (2) pharmacoepidemiologic assessment and safety 

signal interpretation, and (3) pharmacovigilance plan development. 

FDA's guidance documents, including this guidance, do not establish legally enforceable 

responsibilities. Instead, guidances describe the Agency's current thinking on a topic and should 

be viewed only as recommendations, unless specific regulatory or statutory requirements are 

cited. The use of the word should in Agency guidances means that something is suggested or 

recommended, but not required. 

II. BACKGROUND 

1 This guidance has been prepared by the PDUFA III Pharmacovigilance Working Group, which includes members 

from the Center for Drug Evaluation and Research (CDER) and the Center for Biologics Evaluation and Research 

(CBER) at the Food and Drug Administration. 

2 For ease of reference, this guidance uses the term product or drug to refer to all products (excluding blood and 

blood components) regulated by CDER and CBER. Similarly, for ease of reference, this guidance uses the term 

approval to refer to both drug approval and biologic licensure. 

Paperwork Reduction Act Public Burden Statement: This guidance contains information collection provisions 

that are subject to review by the Office of Management and Budget (OMB) under the Paperwork Reduction Act of 

1995 (PRA) (44 U.S.C. 3501-3520). The collection(s) of information in this guidance were approved under OMB 

Control No. 0910-0001 (until March 31, 2005) and 0910-0338 (until August 31, 2005). 

J:\!GUIDANC\6359OCC.doc 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 5 of 24
Contains Nonbinding Recommendations 

A. PDUFA III’s Risk Management Guidance Goal 

On June 12, 2002, Congress reauthorized, for the second time, the Prescription Drug User Fee 

Act (PDUFA III). In the context of PDUFA III, FDA agreed to satisfy certain performance 

goals. One of those goals was to produce guidance for industry on risk management activities 

for drug and biological products. As an initial step towards satisfying that goal, FDA sought 

public comment on risk management. Specifically, FDA issued three concept papers. Each 

paper focused on one aspect of risk management, including (1) conducting premarketing risk 

assessment, (2) developing and implementing risk minimization tools, and (3) performing 

postmarketing pharmacovigilance and pharmacoepidemiologic assessments. In addition to 

receiving numerous written comments regarding the three concept papers, FDA held a public 

workshop on April 9 – 11, 2003, to discuss the concept papers. FDA considered all of the 

comments received in developing three draft guidance documents on risk management activities. 

The draft guidance documents were published on May 5, 2004, and the public was provided with 

an opportunity to comment on them until July 6, 2004. FDA considered all of the comments 

received in producing the final guidance documents. 

1.	 Premarketing Risk Assessment (Premarketing Guidance) 

2.	 Development and Use of Risk Minimization Action Plans (RiskMAP Guidance) 

3.	 Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment 

(Pharmacovigilance Guidance) 

B. Overview of the Risk Management Guidances 

Like the concept papers and draft guidances that preceded them, each of the three final guidance 

documents focuses on one aspect of risk management. The Premarketing Guidance and the 

Pharmacovigilance Guidance focus on premarketing and postmarketing risk assessment, 

respectively. The RiskMAP Guidance focuses on risk minimization. Together, risk assessment 

and risk minimization form what FDA calls risk management. Specifically, risk management is 

an iterative process of (1) assessing a product’s benefit-risk balance, (2) developing and 

implementing tools to minimize its risks while preserving its benefits, (3) evaluating tool 

effectiveness and reassessing the benefit-risk balance, and (4) making adjustments, as 

appropriate, to the risk minimization tools to further improve the benefit-risk balance. This fourpart process should be continuous throughout a product’s lifecycle, with the results of risk 

assessment informing the sponsor’s decisions regarding risk minimization. 

When reviewing the recommendations provided in this guidance, sponsors and applicants should 

keep the following points in mind: 

•	 Many recommendations in this guidance are not intended to be generally applicable to all 

products. 

Industry already performs risk assessment and risk minimization activities for products 

during development and marketing. The Federal Food, Drug, and Cosmetic Act (FDCA) 

and FDA implementing regulations establish requirements for routine risk assessment 

and risk minimization (see e.g., FDA requirements for professional labeling, and adverse 

J:\!GUIDANC\6359OCC.doc	 2 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 6 of 24
 

Contains Nonbinding Recommendations 

event monitoring and reporting). As a result, many of the recommendations presented 

here focus on situations when a product may pose a clinically important and unusual type 

or level of risk. To the extent possible, we have specified in the text whether a 

recommendation is intended for all products or only this subset of products. 

•	 It is of critical importance to protect patients and their privacy during the generation of 

safety data and the development of risk minimization action plans. 

During all risk assessment and risk minimization activities, sponsors must comply with 

applicable regulatory requirements involving human subjects research and patient 

privacy.3 

3

 See 45 CFR part 46 and 21 CFR parts 50 and 56. See also the Health Insurance Portability and Accountability Act 

of 1996 (HIPAA) (Public Law 104-191) and the Standards for Privacy of Individually Identifiable Health 

Information (the Privacy Rule) (45 CFR part 160 and subparts A and E of part 164). The Privacy Rule specifically 

permits covered entities to report adverse events and other information related to the quality, effectiveness, and 

safety of FDA-regulated products both to manufacturers and directly to FDA (45 CFR 164.512(b)(1)(i) and (iii), and 

45 CFR 164.512(a)(1)). For additional guidance on patient privacy protection, see http://www.hhs.gov/ocr/hipaa. 

•	 To the extent possible, this guidance conforms with FDA’s commitment to harmonize 

international definitions and standards as appropriate. 

The topics covered in this guidance are being discussed in a variety of international 

forums. We are participating in these discussions and believe that, to the extent possible, 

the recommendations in this guidance reflect current thinking on related issues. 

•	 When planning risk assessment and risk minimization activities, sponsors should 

consider input from health care participants likely to be affected by these activities (e.g., 

from consumers, pharmacists and pharmacies, physicians, nurses, and third party payers). 

•	 There are points of overlap among the three guidances. 

We have tried to note in the text of each guidance when areas of overlap occur and when 

referencing one of the other guidances might be useful. 

III.	 THE ROLE OF PHARMACOVIGILANCE AND PHARMACOEPIDEMIOLOGY 

IN RISK MANAGEMENT 

Risk assessment during product development should be conducted in a thorough and rigorous 

manner; however, it is impossible to identify all safety concerns during clinical trials. Once a 

product is marketed, there is generally a large increase in the number of patients exposed, 

including those with co-morbid conditions and those being treated with concomitant medical 

products. Therefore, postmarketing safety data collection and risk assessment based on 

observational data are critical for evaluating and characterizing a product's risk profile and for 

making informed decisions on risk minimization. 

J:\!GUIDANC\6359OCC.doc	 3 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 7 of 24
 

Contains Nonbinding Recommendations 

This guidance document focuses on pharmacovigilance activities in the post-approval period. 

This guidance uses the term pharmacovigilance to mean all scientific and data gathering 

activities relating to the detection, assessment, and understanding of adverse events. This 

includes the use of pharmacoepidemiologic studies. These activities are undertaken with the 

goal of identifying adverse events and understanding, to the extent possible, their nature, 

frequency, and potential risk factors. 

Pharmacovigilance principally involves the identification and evaluation of safety signals. In 

this guidance document, safety signal refers to a concern about an excess of adverse events 

compared to what would be expected to be associated with a product's use. Signals can arise 

from postmarketing data and other sources, such as preclinical data and events associated with 

other products in the same pharmacologic class. It is possible that even a single welldocumented case report can be viewed as a signal, particularly if the report describes a positive 

rechallenge or if the event is extremely rare in the absence of drug use. Signals generally 

indicate the need for further investigation, which may or may not lead to the conclusion that the 

product caused the event. After a signal is identified, it should be further assessed to determine 

whether it represents a potential safety risk and whether other action should be taken. 

IV.	 IDENTIFYING AND DESCRIBING SAFETY SIGNALS: FROM CASE 

REPORTS TO CASE SERIES 

Good pharmacovigilance practice is generally based on acquiring complete data from 

spontaneous adverse event reports, also known as case reports. The reports are used to develop 

case series for interpretation. 

A.	 Good Reporting Practice 

Spontaneous case reports of adverse events submitted to the sponsor and FDA, and reports from 

other sources, such as the medical literature or clinical studies, may generate signals of adverse 

effects of drugs. The quality of the reports is critical for appropriate evaluation of the 

relationship between the product and adverse events. FDA recommends that sponsors make a 

reasonable attempt to obtain complete information for case assessment during initial contacts and 

subsequent follow-up, especially for serious events,4

 and encourages sponsors to use trained 

health care practitioners to query reporters. Computer-assisted interview technology, targeted 

questionnaires, or other methods developed to target specific events can help focus the line of 

questioning. When the report is from a consumer, it is often important to obtain permission to 

contact the health care practitioner familiar with the patient’s adverse event to obtain further 

medical information and to retrieve relevant medical records, as needed. 

(1) Safety Reporting Requirements for Human Drug and Biological Products, Proposed Rule, 68 FR 12406 (March 

14, 2003), (2) FDA guidance for industry on Postmarketing Reporting of Adverse Experiences, (3) FDA guidance 

for industry on E2C Clinical Safety Data Management: Periodic Safety Update Report (PSUR), (4) FDA guidance 

for industry on Postmarketing Adverse Experience Reporting for Human Drug and Licensed Biological Products: 

Clarification of What to Report. 

J:\!GUIDANC\6359OCC.doc	 4 

4

 Good reporting practices are extensively addressed in a proposed FDA regulation and guidance documents. See 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 8 of 24
 

Contains Nonbinding Recommendations 

FDA suggests that the intensity and method of case follow-up be driven by the seriousness of the 

event reported, the report's origin (e.g., health care practitioner, patient, literature), and other 

factors. FDA recommends that the most aggressive follow-up efforts be directed towards serious 

adverse event reports, especially of adverse events not known to occur with the drug. 

B. Characteristics of a Good Case Report 

Good case reports include the following elements: 

1.	 Description of the adverse events or disease experience, including time to onset of signs 

or symptoms; 

2.	 Suspected and concomitant product therapy details (i.e., dose, lot number, schedule, 

dates, duration), including over-the-counter medications, dietary supplements, and 

recently discontinued medications; 

3.	 Patient characteristics, including demographic information (e.g., age, race, sex), baseline 

medical condition prior to product therapy, co-morbid conditions, use of concomitant 

medications, relevant family history of disease, and presence of other risk factors; 

4.	 Documentation of the diagnosis of the events, including methods used to make the 

diagnosis; 

5.	 Clinical course of the event and patient outcomes (e.g., hospitalization or death);5 

6.	 Relevant therapeutic measures and laboratory data at baseline, during therapy, and
 

subsequent to therapy, including blood levels, as appropriate;
 

7.	 Information about response to dechallenge and rechallenge; and 

8.	 Any other relevant information (e.g., other details relating to the event or information on 

benefits received by the patient, if important to the assessment of the event). 

5

 Patient outcomes may not be available at the time of initial reporting. In these cases, follow-up reports can convey 

important information about the course of the event and serious outcomes, such as hospitalization or death. 

For reports of medication errors, good case reports also include full descriptions of the following, 

when such information is available: 

1.	 Products involved (including the trade (proprietary) and established (proper) name, 

manufacturer, dosage form, strength, concentration, and type and size of container); 

2.	 Sequence of events leading up to the error; 

3.	 Work environment in which the error occurred; and 

4.	 Types of personnel involved with the error, type(s) of error, and contributing factors. 

J:\!GUIDANC\6359OCC.doc	 5 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 9 of 24
 

Contains Nonbinding Recommendations 

FDA recommends that sponsors capture in the case narrative section of a medication error report 

all appropriate information outlined in the National Coordinating Council for Medication Error 

Reporting and Prevention (NCC MERP) Taxonomy.6 Although sponsors are not required to use 

the taxonomy, FDA has found the taxonomy to be a useful tool to categorize and analyze reports 

of medication errors. It provides a standard language and structure for medication error-related 

data collected through reports. 

6

 See http://www.nccmerp.org for the definition of a medication error and taxonomy of medication errors. 

C. Developing a Case Series 

FDA suggests that sponsors initially evaluate a signal generated from postmarketing spontaneous 

reports through a careful review of the cases and a search for additional cases. Additional cases 

could be identified from the sponsor’s global adverse event databases, the published literature, 

and other available databases, such as FDA’s Adverse Event Reporting System (AERS) or 

Vaccine Adverse Events Reporting System (VAERS), using thorough database search strategies 

based on updated coding terminology (e.g., the Medical Dictionary for Regulatory Activities 

(MedDRA)). When available, FDA recommends that standardized case definitions (i.e., formal 

criteria for including or excluding a case) be used to assess potential cases for inclusion in a case 

series.7

 In general, FDA suggests that case-level review occur before other investigations or 

analyses. FDA recommends that emphasis usually be placed on review of serious, unlabeled 

adverse events, although other events may warrant further investigation (see section IV.F. for 

more details). 

As part of the case-level review, FDA suggests that sponsors evaluate individual case reports for 

clinical content and completeness, and follow up with reporters, as necessary. It is important to 

remove any duplicate reports. In assessing case reports, FDA recommends that sponsors look for 

features that may suggest a causal relationship between the use of a product and the adverse 

event, including: 

1.	 Occurrence of the adverse event in the expected time (e.g., type 1 allergic reactions 

occurring within days of therapy, cancers developing after years of therapy); 

2.	 Absence of symptoms related to the event prior to exposure; 

3.	 Evidence of positive dechallenge or positive rechallenge; 

4.	 Consistency of the event with the established pharmacological/toxicological effects of the 

product, or for vaccines, consistency with established infectious or immunologic 

mechanisms of injury; 

5.	 Consistency of the event with the known effects of other products in the class; 

J:\!GUIDANC\6359OCC.doc	 6 

7

 See, for example, Institute of Medicine (IOM) Immunization Safety Review on Vaccines and Autism, 2004. 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 10 of 24
 

 

Contains Nonbinding Recommendations 

6.	 Existence of other supporting evidence from preclinical studies, clinical trials, and/or 

pharmacoepidemiologic studies; and 

7.	 Absence of alternative explanations for the event (e.g., no concomitant medications that 

could contribute to the event; no co- or pre-morbid medical conditions). 

Confounded cases are common, especially among patients with complicated medical conditions. 

Confounded cases (i.e., cases with adverse events that have possible etiologies other than the 

product of concern) could still represent adverse effects of the product under review. FDA 

recommends that sponsors carefully evaluate these cases and not routinely exclude them. 

Separate analyses of unconfounded cases may be useful. 

For any individual case report, it is rarely possible to know with a high level of certainty whether 

the event was caused by the product. To date, there are no internationally agreed upon standards 

or criteria for assessing causality in individual cases, especially for events that often occur 

spontaneously (e.g. stroke, pulmonary embolism). Rigorous pharmacoepidemiologic studies, 

such as case-control studies and cohort studies with appropriate follow-up, are usually employed 

to further examine the potential association between a product and an adverse event. 

FDA does not recommend any specific categorization of causality, but the categories probable, 

possible, or unlikely have been used previously.8

 If a causality assessment is undertaken, FDA 

suggests that the causal categories be specified and described in sufficient detail to understand 

the underlying logic in the classification. 

If the safety signal relates to a medication error, FDA recommends that sponsors report all 

known contributing factors that led to the event. A number of references are available to assist 

sponsors in capturing a complete account of the event.9 FDA recommends that sponsors follow 

up to the extent possible with reporters to capture a complete account of the event, focusing on 

the medication use systems (e.g., prescribing/order process, dispensing process, administration 

process). This data may be informative in developing strategies to minimize future errors. 

8

 See World Health Organization, the Uppsala Monitoring Center, 2000, Safety Monitoring of Medicinal Product, 

for additional categorizations of causality. 

9

 See Cohen MR (ed), 1999, Medication Errors, American Pharmaceutical Association, Washington DC; Cousins 

DD (ed), 1998, Medication Use: A Systems Approach to Reducing Errors, Joint Commission on Accreditation of 

Healthcare Organizations, Oakbrook Terrace, IL. 

D. Summary Descriptive Analysis of a Case Series 

In the event that one or more cases suggest a safety signal warranting additional investigation, 

FDA recommends that a case series be assembled and descriptive clinical information be 

summarized to characterize the potential safety risk and, if possible, to identify risk factors. A 

case series commonly includes an analysis of the following: 

1.	 The clinical and laboratory manifestations and course of the event; 

J:\!GUIDANC\6359OCC.doc	 7 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 11 of 24
 

Contains Nonbinding Recommendations 

2.	 Demographic characteristics of patients with events (e.g., age, gender, race); 

3.	 Exposure duration; 

4.	 Time from initiation of product exposure to the adverse event; 

5.	 Doses used in cases, including labeled doses, greater than labeled doses, and overdoses; 

6.	 Use of concomitant medications; 

7.	 The presence of co-morbid conditions, particularly those known to cause the adverse 

event, such as underlying hepatic or renal impairment; 

8.	 The route of administration (e.g., oral vs. parenteral); 

9.	 Lot numbers, if available, for products used in patients with events; and 

10. Changes in event reporting rate over calendar time or product life cycle. 

E. Use of Data Mining to Identify Product-Event Combinations 

At various stages of risk identification and assessment, systematic examination of the reported 

adverse events by using statistical or mathematical tools, or so-called data mining, can provide 

additional information about the existence of an excess of adverse events reported for a product. 

By applying data mining techniques to large adverse event databases, such as FDA’s AERS or 

VAERS, it may be possible to identify unusual or unexpected product-event combinations 

warranting further investigation. Data mining can be used to augment existing signal detection 

strategies and is especially useful for assessing patterns, time trends, and events associated with 

drug-drug interactions. Data mining is not a tool for establishing causal attributions between 

products and adverse events. 

The methods of data mining currently in use usually generate a score comparing (1) the fraction 

of all reports for a particular event (e.g., liver failure) for a specific drug (i.e., the “observed 

reporting fraction”) with (2) the fraction of reports for the same particular event for all drugs 

(i.e.,“the expected reporting fraction”).10 This analysis can be refined by adjusting for aspects of 

reporting (e.g., the reporting year) or characteristics of the patient (e.g., age or gender) that might 

influence the amount of reporting. In addition, it may be possible to limit data mining to an 

analysis for drugs of a specific class or for drugs that are used to treat a particular disease. 

The score (or statistic) generated by data mining quantifies the disproportionality between the 

observed and expected values for a given product-event combination. This score is compared to 

a threshold that is chosen by the analyst. A potential excess of adverse events is operationally 

defined as any product-event combination with a score exceeding the specified threshold. When 

10 Evans SJ, 2000, Pharmacovigilance: A science or fielding emergencies? Statistics in Medicine 19(23):3199-209; 

Evans SJW, Waller PC, and Davis S, 2001, Use of proportional reporting ratios (PRRs) for signal generation from 

spontaneous adverse drug reaction reports, Pharmacoepidemiology and Drug Safety 10:483-6. 

J:\!GUIDANC\6359OCC.doc	 8 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 12 of 24
 

 

Contains Nonbinding Recommendations 

applying data mining to large databases (such as AERS), it is not unusual for a product to have 

several product-event combinations with scores above a specified threshold. The lower the 

threshold, the greater the likelihood that more combinations will exceed the threshold and will 

warrant further investigation. 

Several data mining methods have been described and may be worth considering, such as the 

Multi-Item Gamma Poisson Shrinker (MGPS) algorithm11,12, the Proportional Reporting Ratio 

(PRR) method13,14and the Neural Network approach.15 Except when the observed number of 

cases with the drug event combination is small (e.g., less than 20) or the expected number of 

cases with the drug event combination is < 1, the MGPS and PRR methods will generally 

identify similar drug event combinations for further investigation.16 

Although all of these approaches are inherently exploratory or hypothesis generating, they may 

provide insights into the patterns of adverse events reported for a given product relative to other 

products in the same class or to all other products. FDA exercises caution when making such 

comparisons, because voluntary adverse event reporting systems such as AERS or VAERS are 

subject to a variety of reporting biases (e.g., some observations could reflect concomitant 

treatment, not the product itself, and other factors, including the disease being treated, other comorbidities or unrecorded confounders, may cause the events to be reported). In addition, AERS 

or VAERS data may be affected by the submission of incomplete or duplicate reports, underreporting, or reporting stimulated by publicity or litigation. As reporting biases may differ by 

product and change over time, and could change differently for different events, it is not possible 

to predict their impact on data mining scores. 

Use of data mining techniques is not a required part of signal identification or evaluation. If data 

mining results are submitted to FDA, they should be presented in the larger appropriate clinical 

epidemiological context. This should include (1) a description of the database used, (2) a 

description of the data mining tool used (e.g., statistical algorithm, and the drugs, events and 

11 DuMouchel W and Pregibon D, 2001, Empirical Bayes screening for multi-item associations, Seventh ACM 

SigKDD International Conference on Knowledge Discovery and Data Mining. 

12 Szarfman A, Machado SG, and O'Neill RT, 2002, Use of screening algorithms and computer systems to 

efficiently signal higher-than-expected combinations of drugs and events in the US FDA's spontaneous reports 

database, Drug Safety 25(6): 381-92. 

13 Evans SJW, Waller P, and Davis S, 1998, Proportional reporting ratios: the uses of epidemiological methods for 

signal generation [abstract], Pharmacoepidemiology and Drug Safety 7:S102. 

14 Evans SJ, 2000, Pharmacovigilance: A science or fielding emergencies? Statistics in Medicine 19(23):3199-209; 

Evans SJW, Waller PC, and Davis S, 2001, Use of proportional reporting ratios (PRRs) for signal generation from 

spontaneous adverse drug reaction reports, Pharmacoepidemiology and Drug Safety 10:483-6. 

15 Bate A et al., 1998, A Bayesian neural network method for adverse drug reaction signal generation, European 

Journal of Clinical Pharmacology 54:315-21. 

16 This conclusion is based on the experience of FDA and of William DuMouchel, Ph.D., Chief Scientist, Lincoln 

Technologies, Wellsley, MA, as summarized in an email communication from Dr. DuMouchel to Ana Szarfman, 

M.D., Ph.D., Medical Officer, OPaSS, CDER, on October 13, 2004. 

J:\!GUIDANC\6359OCC.doc 9 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 13 of 24
 

Contains Nonbinding Recommendations 

stratifications selected for the analyses) or an appropriate reference, and (3) a careful assessment 

of individual case reports and any other relevant safety information related to the particular drugevent combination of interest (e.g., results from preclinical, clinical, pharmacoepidemiologic, or 

other available studies). 

F.	 Safety Signals That May Warrant Further Investigation 

FDA believes that the methods described above will permit a sponsor to identify and 

preliminarily characterize a safety signal. The actual risk to patients cannot be known from these 

data because it is not possible to characterize all events definitively and because there is 

invariably under-reporting of some extent and incomplete information about duration of therapy, 

numbers treated, etc. Safety signals that may warrant further investigation may include, but are 

not limited to, the following: 

1.	 New unlabeled adverse events, especially if serious; 

2.	 An apparent increase in the severity of a labeled event; 

3.	 Occurrence of serious events thought to be extremely rare in the general population; 

4.	 New product-product, product-device, product-food, or product-dietary supplement 

interactions; 

5.	 Identification of a previously unrecognized at-risk population (e.g., populations with 

specific racial or genetic predispositions or co-morbidities); 

6.	 Confusion about a product's name, labeling, packaging, or use; 

7.	 Concerns arising from the way a product is used (e.g., adverse events seen at higher 

than labeled doses or in populations not recommended for treatment); 

8.	 Concerns arising from potential inadequacies of a currently implemented risk 

minimization action plan (e.g., reports of serious adverse events that appear to reflect 

failure of a RiskMAP goal);17 and 

9.	 Other concerns identified by the sponsor or FDA. 

G.	 Putting the Signal into Context: Calculating Reporting Rates vs. Incidence 

Rates 

If a sponsor determines that a concern about an excess of adverse events or safety signal warrants 

further investigation and analysis, it is important to put the signal into context. For this reason, 

calculations of the rate at which new cases of adverse events occur in the product-exposed 

population (i.e., the incidence rate) are the hallmark of pharmacoepidemiologic risk assessment. 

17 For a detailed discussion of risk minimization action plan evaluation, please consult the RiskMAP Guidance. 

J:\!GUIDANC\6359OCC.doc	 10 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 14 of 24
 

Contains Nonbinding Recommendations 

In pharmacoepidemiologic studies (see section V.A), the numerator (number of new cases) and 

denominator (number of exposed patients and time of exposure or, if known, time at risk) may be 

readily ascertainable. In contrast, for spontaneously reported events, it is not possible to identify 

all cases because of under-reporting, and the size of the population at risk is at best an estimate. 

Limitations in national denominator estimates arise because: 

1.	 Accurate national estimates of the number of patients exposed to a medical product 

and their duration of exposure may not be available; 

2.	 It may be difficult to exclude patients who are not at risk for an event, for example, 

because their exposure is too brief or their dose is too low;18 and 

3.	 A product may be used in different populations for different indications, but use 

estimates are not available for the specific population of interest. 

Although we recognize these limitations, we recommend that sponsors calculate crude adverse 

event reporting rates as a valuable step in the investigation and assessment of adverse events. 

FDA suggests that sponsors calculate reporting rates by using the total number of spontaneously 

reported cases in the United States in the numerator and estimates of national patient exposure to 

product in the denominator.19,20 FDA recommends that whenever possible, the number of 

patients or person time exposed to the product nationwide be the estimated denominator for a 

reporting rate. FDA suggests that other surrogates for exposure, such as numbers of 

prescriptions or kilograms of product sold, only be used when patient-level estimates are 

unavailable. FDA recommends that sponsors submit a detailed explanation of the rationale for 

selection of a denominator and a method of estimation. 

Comparisons of reporting rates and their temporal trends can be valuable, particularly across 

similar products or across different product classes prescribed for the same indication. However, 

such comparisons are subject to substantial limitations in interpretation because of the inherent 

uncertainties in the numerator and denominator used. As a result, FDA suggests that a 

comparison of two or more reporting rates be viewed with extreme caution and generally 

considered exploratory or hypothesis-generating. Reporting rates can by no means be considered 

incidence rates, for either absolute or comparative purposes. 

To provide further context for incidence rates or reporting rates, it is helpful to have an estimate 

of the background rate of occurrence for the event being evaluated in the general population or, 

ideally, in a subpopulation with characteristics similar to that of the exposed population (e.g., 

premenopausal women, diabetics). These background rates can be derived from: (1) national 

health statistics, (2) published medical literature, or (3) ad hoc studies, particularly of 

18 See Current Challenges in Pharmacovigilance: Pragmatic Approaches, Report of the Council for International 

Organizations of Medical Sciences (CIOMS) Working Group V, Geneva, 2001. 

19 See Rodriguez EM, Staffa JA, Graham DJ, 2001, The role of databases in drug postmarketing surveillance, 

Pharmacoepidemiology and Drug Safety, 10:407-10. 

20 In addition to U.S. reporting rates, sponsors can provide global reporting rates, when relevant. 

J:\!GUIDANC\6359OCC.doc	 11 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 15 of 24
 

Contains Nonbinding Recommendations 

subpopulations, using large automated databases or ongoing epidemiologic investigations with 

primary data collection. FDA suggests that comparisons of incidence rates or reporting rates to 

background rate estimates take into account potential differences in the data sources, diagnostic 

criteria, and duration of time at risk. 

While the extent of under-reporting is unknown, it is usually assumed to be substantial and may 

vary according to the type of product, seriousness of the event, population using the product, and 

other factors. As a result, a reporting rate higher than the background rate may, in some cases, 

be a strong indicator that the true incidence rate is sufficiently high to be of concern. However, 

many other factors affect the reporting of product-related adverse events (e.g., publicity, newness 

of product to the market) and these factors should be considered when interpreting a high 

reporting rate. Also, because of under-reporting, the fact that a reporting rate is less than the 

background rate does not necessarily show that the product is not associated with an increased 

risk of an adverse event. 

V.	 BEYOND CASE REVIEW: INVESTIGATING A SIGNAL THROUGH 

OBSERVATIONAL STUDIES 

FDA recognizes that there are a variety of methods for investigating a safety signal. Signals 

warranting additional investigation can be further evaluated through carefully designed nonrandomized observational studies of the product’s use in the “real world” and randomized trials. 

The Premarketing Guidance discusses a number of types of randomized trials, including the 

large simple safety study, which is a risk assessment method that could be used either pre- or 

post-approval. 

This document focuses on three types of non-randomized observational studies: (1) 

pharmacoepidemiologic studies, (2) registries, and (3) surveys. By focusing this guidance on 

certain risk assessment methods, we do not intend to advocate the use of these approaches over 

others. FDA encourages sponsors to consider all methods to evaluate a particular safety signal. 

FDA recommends that sponsors choose the method best suited to the particular signal and 

research question of interest. Sponsors planning to evaluate a safety signal are encouraged to 

communicate with FDA as their plans progress. 

A.	 Pharmacoepidemiologic Studies 

Pharmacoepidemiologic studies can be of various designs, including cohort (prospective or 

retrospective), case-control, nested case-control, case-crossover, or other models.21 The results 

of such studies may be used to characterize one or more safety signals associated with a product, 

or may examine the natural history of a disease or drug utilization patterns. Unlike a case series, 

a pharmacoepidemiologic study which is designed to assess the risk attributed to a drug exposure 

has a protocol and control group and tests prespecified hypotheses. Pharmacoepidemiologic 

studies can allow for the estimation of the relative risk of an outcome associated with a product, 

and some (e.g., cohort studies) can also provide estimates of risk (incidence rate) for an adverse 

21 Guidelines for Good Pharmacoepidemiology, , International Society for Pharmacoepidemiology, 2004 

(http://www.pharmacoepi.org/resources/guidelines_08027.cfm) 

J:\!GUIDANC\6359OCC.doc	 12 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 16 of 24
 

 

Contains Nonbinding Recommendations 

event. Sponsors can initiate pharmacoepidemiologic studies at any time. They are sometimes 

started at the time of initial marketing, based on questions that remain after review of the 

premarketing data. More often, however, they are initiated when a safety signal has been 

identified after approval. Finally, there may also be occasions when a pharmacoepidemiologic 

study is initiated prior to marketing (e.g., to study the natural history of disease or patterns of 

product use, or to estimate background rates for adverse events). 

For uncommon or delayed adverse events, pharmacoepidemiologic studies may be the only 

practical choice for evaluation, even though they can be limited by low statistical power. 

Clinical trials are impractical in almost all cases when the event rates of concern are less 

common than 1:2000-3000 (an exception may be larger trials conducted for some vaccines, 

which could move the threshold to 1:10,000). It may also be difficult to use clinical trials: (1) to 

evaluate a safety signal associated with chronic exposure to a product, exposure in populations 

with co-morbid conditions, or taking multiple concomitant medications, or (2) to identify certain 

risk factors for a particular adverse event. On the other hand, for evaluation of more common 

events, which are seen relatively often in untreated patients, clinical trials may be preferable to 

observational studies. 

Because pharmacoepidemiologic studies are observational in nature, they may be subject to 

confounding, effect modification, and other bias, which may make results of these types of 

studies more difficult to interpret than the results of clinical trials. Some of these problems can 

be surmounted when the relative risk to exposed patients is high. 

Because different products pose different benefit-risk considerations (e.g., seriousness of the 

disease being treated, nature and frequency of the safety signal under evaluation), it is impossible 

to delineate a universal set of criteria for the point at which a pharmacoepidemiologic study 

should be initiated, and the decision should be made on a case-by-case basis. When an important 

adverse event–product association leads to questions on the product’s benefit-risk balance, FDA 

recommends that sponsors consider whether the particular signal should be addressed with one 

or more pharmacoepidemiologic studies. If a sponsor determines that a pharmacoepidemiologic 

study is the best method for evaluating a particular signal, the design and size of the proposed 

study would depend on the objectives of the study and the expected frequency of the events of 

interest. 

When performing a pharmacoepidemiologic study, FDA suggests that investigators seek to 

minimize bias and to account for possible confounding. Confounding by indication is one 

example of an important concern in performing a pharmacoepidemiologic study.22 Because of 

the effects of bias, confounding, or effect modification, pharmacoepidemiologic studies 

evaluating the same hypothesis may provide different or even conflicting results. It is almost 

always prudent to conduct more than one study, in more than one environment and even use 

different designs. Agreement of the results from more than one study helps to provide 

reassurance that the observed results are robust. 

22 See, for example, Strom BL (ed), 2000, Pharmacoepidemiology, 3rd edition, Chichester: John Wiley and Sons, 

Ltd; Hartzema AG, Porta M, and Tilson HH (eds), 1998, Pharmacoepidemiology: An Introduction, 3rd edition, 

Cincinnati, OH: Harvey Whitney Books. 

J:\!GUIDANC\6359OCC.doc 13 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 17 of 24
 

Contains Nonbinding Recommendations 

There are a number of references describing methodologies for pharmacoepidemiologic studies, 

discussing their strengths and limitations,23 and providing guidelines to facilitate the conduct, 

interpretation, and documentation of such studies.24 Consequently, this guidance document does 

not comprehensively address these topics. However, a protocol for a pharmacoepidemiologic 

study generally includes: 

1.	 Clearly specified study objectives; 

2.	 A critical review of the literature; and 

3.	 A detailed description of the research methods, including: 

•	 the population to be studied; 

•	 the case definitions to be used; 

•	 the data sources to be used (including a rationale for data sources if from outside 

the U.S.); 

•	 the projected study size and statistical power calculations; and 

•	 the methods for data collection, management, and analysis. 

Depending on the type of pharmacoepidemiologic study planned, there are a variety of data 

sources that may be used, ranging from the prospective collection of data to the use of existing 

data, such as data from previously conducted clinical trials or large databases. In recent years, a 

number of pharmacoepidemiologic studies have been conducted in automated claims databases 

(e.g., HMO, Medicaid) that allow retrieval of records on product exposure and patient outcomes. 

In addition, recently, comprehensive electronic medical record databases have also been used for 

studying drug safety issues. Depending on study objectives, factors that may affect the choice of 

databases include the following: 

1.	 Demographic characteristics of patients enrolled in the health plans (e.g., age,
 

geographic location);
 

2.	 Turnover rate of patients in the health plans; 

3.	 Plan coverage of the medications of interest; 

4.	 Size and characteristics of the exposed population available for study; 

5.	 Availability of the outcomes of interest; 

6.	 Ability to identify conditions of interest using standard medical coding systems (e.g., 

International Classification of Diseases (ICD-9)), procedure codes or prescriptions that 

could be used as markers; 

23 Ibid. 

24 Guidelines for Good Pharmacoepidemiology, International Society for Pharmacoepidemiology, 2004 

(http://www.pharmacoepi.org/resources/guidelines_08027.cfm). 

J:\!GUIDANC\6359OCC.doc	 14 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 18 of 24
 

Contains Nonbinding Recommendations 

7. Access to medical records; and 

8. Access to patients for data not captured electronically. 

For most pharmacoepidemiologic studies, FDA recommends that sponsors validate diagnostic 

findings through a detailed review of at least a sample of medical records. If the validation of the 

specific outcome or exposure of interest using the proposed database has been previously 

reported, FDA recommends that the literature supporting the validity of the proposed study be 

submitted for review. 

FDA encourages sponsors to communicate with the Agency when pharmacoepidemiologic 

studies are being developed. 

B.	 Registries 

The term registry as used in pharmacovigilance and pharmacoepidemiology can have varied 

meanings. In this guidance document, a registry is “an organized system for the collection, 

storage, retrieval, analysis, and dissemination of information on individual persons exposed to a 

specific medical intervention who have either a particular disease, a condition (e.g., a risk factor) 

that predisposes [them] to the occurrence of a health-related event, or prior exposure to 

substances (or circumstances) known or suspected to cause adverse health effects.”25 Whenever 

possible, a control or comparison group should be included, (i.e., individuals with a disease or 

risk factor who are not treated or are exposed to medical interventions other than the intervention 

of interest).26 

Through the creation of registries, a sponsor can evaluate safety signals identified from 

spontaneous case reports, literature reports, or other sources, and evaluate factors that affect the 

risk of adverse outcomes, such as dose, timing of exposure, or patient characteristics.27 

Registries can be particularly useful for: 

1.	 Collecting outcome information not available in large automated databases; and 

2.	 Collecting information from multiple sources (e.g., physician records, hospital 

summaries, pathology reports, vital statistics), particularly when patients receive care 

from multiple providers over time. 

A sponsor can initiate a registry at any time. It may be appropriate to initiate the registry at or 

before initial marketing, when a new indication is approved, or when there is a need to evaluate 

25 See Frequently Asked Questions About Medical and Public Health Registries, The National Committee on Vital 

and Health Statistics, at http://www.ncvhs.hhs.gov. 

26 See for example, FDA Guidance for Industry, Establishing Pregnancy Exposure Registries, August 2002 

http://www.fda.gov/cder/guidance/3626fnl.pdf. 

27 Ibid. 

J:\!GUIDANC\6359OCC.doc	 15 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 19 of 24
 

 

Contains Nonbinding Recommendations 

safety signals identified from spontaneous case reports. In deciding whether to establish a 

registry, FDA recommends that a sponsor consider the following factors: 

1.	 The types of additional risk information desired; 

2.	 The attainability of that information through other methods; and 

3.	 The feasibility of establishing the registry. 

Sponsors electing to initiate a registry should develop written protocols that provide: (1) 

objectives for the registry, (2) a review of the literature, and (3) a summary of relevant animal 

and human data. FDA suggests that protocols also contain detailed descriptions of: (1) plans for 

systematic patient recruitment and follow-up, (2) methods for data collection, management, and 

analysis, and (3) conditions under which the registry will be terminated. A registry-based 

monitoring system should include carefully designed data collection forms to ensure data quality, 

integrity, and validation of registry findings against a sample of medical records or through 

interviews with health care providers. FDA recommends that the size of the registry and the 

period during which data will be collected be consistent with the safety questions under study 

and we encourage sponsors to discuss their registry development plans with FDA. 

C.	 Surveys 

Patient or health care provider surveys can gather information to assess, for example: 

1.	 A safety signal; 

2.	 Knowledge about labeled adverse events; 

3.	 Use of a product as labeled, particularly when the indicated use is for a restricted 

population or numerous contraindications exist; 

4.	 Compliance with the elements of a RiskMAP (e.g., whether or not a Medication 

Guide was provided at the time of product dispensing); and 28 

5.	 Confusion in the practicing community over sound-alike or look-alike trade (or 

proprietary) names. 

Like a registry, a survey can be initiated by a sponsor at any time. It can be conducted at the 

time of initial marketing (i.e., to fulfill a postmarketing commitment) or when there is a desire to 

evaluate safety signals identified from spontaneous case reports. 

FDA suggests that sponsors electing to initiate a survey develop a written protocol that provides 

objectives for the survey and a detailed description of the research methods, including: (1) 

patient or provider recruitment and follow-up, (2) projected sample size, and (3) methods for 

data collection, management, and analysis.29 FDA recommends that a survey-based monitoring 

28 For a detailed discussion of RiskMAP evaluation, please consult the RiskMAP Guidance. 

29 See 21 CFR parts 50 and 56 for FDA’s regulations governing the protection of human subjects. 

J:\!GUIDANC\6359OCC.doc	 16 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 20 of 24
Contains Nonbinding Recommendations 

system include carefully designed survey instruments and validation of survey findings against a 

sample of medical or pharmacy records or through interviews with health care providers, 

whenever possible. FDA recommends that survey instruments be validated or piloted before 

implementation. FDA suggests that sponsors consider whether survey translation and cultural 

validation would be important. 

Sponsors are encouraged to discuss their survey development plans with FDA. 

VI.	 INTERPRETING SAFETY SIGNALS: FROM SIGNAL TO POTENTIAL 

SAFETY RISK 

After identifying a safety signal, FDA recommends that a sponsor conduct a careful case level 

review and summarize the resulting case series descriptively. To help further characterize a 

safety signal, a sponsor can also: (1) employ data mining techniques, and (2) calculate reporting 

rates for comparison to background rates. Based on these findings and other available data (e.g., 

from preclinical or other sources), FDA suggests that a sponsor consider further study (e.g., 

observational studies) to establish whether or not a potential safety risk exists. 

When evaluation of a safety signal suggests that it may represent a potential safety risk, FDA 

recommends that a sponsor submit a synthesis of all available safety information and analyses 

performed, ranging from preclinical findings to current observations. This submission should 

include the following: 

1.	 Spontaneously reported and published case reports, with denominator or exposure 

information to aid interpretation; 

2.	 Background rate for the event in general and specific patient populations, if available; 

3.	 Relative risks, odds ratios, or other measures of association derived from
 

pharmacoepidemiologic studies;
 

4.	 Biologic effects observed in preclinical studies and pharmacokinetic or
 

pharmacodynamic effects;
 

5.	 Safety findings from controlled clinical trials; and 

6.	 General marketing experience with similar products in the class. 

After the available safety information is presented and interpreted, it may be possible to assess 

the degree of causality between use of a product and an adverse event. FDA suggests that the 

sponsor’s submission provide an assessment of the benefit-risk balance of the product for the 

population of users as a whole and for identified at-risk patient populations, and, if appropriate, 

(1) propose steps to further investigate the signal through additional studies, and (2) propose risk 

J:\!GUIDANC\6359OCC.doc	 17 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 21 of 24
 

 

Contains Nonbinding Recommendations 

minimization actions.30 FDA will make its own assessment of the potential safety risk posed by 

the signal in question, taking into account the information provided by the sponsor and any 

additional relevant information known to FDA (e.g., information on other products in the same 

class) and will communicate its conclusions to the sponsor whenever possible. Factors that are 

typically considered include: 

1.	 Strength of the association (e.g., relative risk of the adverse event associated with the 

product); 

2.	 Temporal relationship of product use and the event; 

3.	 Consistency of findings across available data sources; 

4.	 Evidence of a dose-response for the effect; 

5.	 Biologic plausibility; 

6.	 Seriousness of the event relative to the disease being treated; 

7.	 Potential to mitigate the risk in the population; 

8.	 Feasibility of further study using observational or controlled clinical study designs; 

and 

9.	 Degree of benefit the product provides, including availability of other therapies. 

As noted in section II, risk management is an iterative process and steps to further investigate a 

potential safety risk, assess the product’s benefit-risk balance, and implement risk minimization 

tools would best occur in a logical sequence, not simultaneously. Not all steps may be 

recommended, depending on the results of earlier steps.31 FDA recommends that assessment of 

causality and of strategies to minimize product risk occur on an ongoing basis, taking into 

account the findings from newly completed studies. 

VII.	 BEYOND ROUTINE PHARMACOVIGILANCE: DEVELOPING A 

PHARMACOVIGILANCE PLAN 

For most products, routine pharmacovigilance (i.e., compliance with applicable postmarket 

requirements under the FDCA and FDA implementing regulations) is sufficient for 

postmarketing risk assessment. However, in certain limited instances, unusual safety risks may 

become evident before approval or after a product is marketed that could suggest that 

consideration by the sponsor of a pharmacovigilance plan may be appropriate. A 

30 In the vast majority of cases, risk communication that incorporates appropriate language into the product’s 

labeling will be adequate for risk minimization. In rare instances, however, a sponsor may consider implementing a 

RiskMAP. Please refer to the RiskMAP Guidance for a complete discussion of RiskMAP development. 31 For additional discussion of the relationship between risk assessment and risk minimization, please consult the 

RiskMAP Guidance. 

J:\!GUIDANC\6359OCC.doc	 18 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 22 of 24
 

 

Contains Nonbinding Recommendations 

pharmacovigilance plan is a plan developed by a sponsor that is focused on detecting new safety 

risks and/or evaluating already identified safety risks. Specifically, a pharmacovigilance plan 

describes pharmacovigilance efforts above and beyond routine postmarketing spontaneous 

reporting, and is designed to enhance and expedite the sponsor’s acquisition of safety 

information.32 The development of pharmacovigilance plans may be useful at the time of 

product launch or when a safety risk is identified during product marketing. FDA recommends 

that a sponsor’s decision to develop a pharmacovigilance plan be based on scientific and 

logistical factors, including the following: 

1.	 The likelihood that the adverse event represents a potential safety risk; 

2.	 The frequency with which the event occurs (e.g., incidence rate, reporting rate, or 

other measures available); 

3.	 The severity of the event; 

4.	 The nature of the population(s) at risk; 

5.	 The range of patients for which the product is indicated (broad range or selected 

populations only); and 

6.	 The method by which the product is dispensed (through pharmacies or performance 

linked systems only).33 

A pharmacovigilance plan may be developed by itself or as part of a Risk Minimization Action 

Plan (RiskMAP), as described in the RiskMAP Guidance. Sponsors may meet with 

representatives from the appropriate Office of New Drugs review division and the Office of Drug 

Safety in CDER, or the appropriate Product Office and the Division of Epidemiology, Office of 

Biostatistics and Epidemiology in CBER regarding the specifics of a given product’s 

pharmacovigilance plan. 

FDA believes that for a product without safety risks identified pre- or post-approval and for 

which at-risk populations are thought to have been adequately studied, routine spontaneous 

reporting will be sufficient for postmarketing surveillance. On the other hand, 

pharmacovigilance plans may be appropriate for products for which: (1) serious safety risks have 

been identified pre- or post-approval, or (2) at-risk populations have not been adequately studied. 

32 As used in this document, the term “pharmacovigilance plan” is defined differently than in the ICH draft E2E 

document (version 4.1). As used in the ICH document, a “pharmacovigilance plan” would be routinely developed 

(i.e., even when a sponsor does not anticipate that enhanced pharmacovigilance efforts are necessary). In contrast, 

as discussed above, FDA is only recommending that pharmacovigilance plans be developed when warranted by 

unusual safety risks. This ICH guidance is available on the Internet at http://www.fda.gov/cder/guidance/index.htm 

under the topic ICH Efficacy. The draft E2E guidance was made available on March 30, 2004 (69 FR 16579). ICH 

agreed on the final version of the E2E guidance in November, 2004. 

33 For a detailed discussion of controlled access systems, please consult the RiskMAP Guidance. 

J:\!GUIDANC\6359OCC.doc	 19 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 23 of 24
 

 

Contains Nonbinding Recommendations 

Sponsors may discuss with the Agency the nature of the safety concerns posed by such a product 

and the determination whether a pharmacovigilance plan is appropriate. 

A pharmacovigilance plan could include one or more of the following elements: 

1.	 Submission of specific serious adverse event reports in an expedited manner beyond 

routine required reporting (i.e., as 15-day reports); 

2.	 Submission of adverse event report summaries at more frequent, prespecified
 

intervals (e.g., quarterly rather than annually);
 

3.	 Active surveillance to identify adverse events that may or may not be reported 

through passive surveillance. Active surveillance can be (1) drug based: identifying 

adverse events in patients taking certain products, (2) setting based: identifying 

adverse events in certain health care settings where they are likely to present for 

treatment (e.g., emergency departments, etc.), or (3) event based: identifying adverse 

events that are likely to be associated with medical products (e.g., acute liver failure); 

4.	 Additional pharmacoepidemiologic studies (for example, in automated claims 

databases or other databases) using cohort, case-control, or other appropriate study 

designs (see section V); 

5.	 Creation of registries or implementation of patient or health care provider surveys 

(see section V); and 

6.	 Additional controlled clinical trials.34 

As data emerges, FDA recommends that a sponsor re-evaluate the safety risk and the 

effectiveness of its pharmacovigilance plan. Such re-evaluation may result in revisions to the 

pharmacovigilance plan for a product. In some circumstances, FDA may decide to bring 

questions on potential safety risks and pharmacovigilance plans before its Drug Safety and Risk 

Management Advisory Committee or the FDA Advisory Committee dealing with the specific 

product in question. Such committees may be convened when FDA seeks: (1) general advice on 

the design of pharmacoepidemiologic studies, (2) comment on specific pharmacoepidemiology 

studies developed by sponsors or FDA for a specific product and safety question, or (3) advice 

on the interpretation of early signals from a case series and on the need for further investigation 

in pharmacoepidemiologic studies. While additional information is being developed, sponsors 

working with FDA can take interim actions to communicate information about potential safety 

risks (e.g., through labeling) to minimize the risk to users of the product. 

34 For a discussion of risk assessment in controlled clinical trials, please consult the Premarketing Guidance. 

J:\!GUIDANC\6359OCC.doc	 20 

03/22/05
 

Case 2:15-md-02641-DGC Document 8574-4 Filed 11/02/17 Page 24 of 24