Exhibit O

Case 2:15-md-02641-DGC Document 8574-3 Filed 11/02/17 Page 1 of 6
• 

• 

• 

CLINICAL THERAPEUTICSIIINOL. 20, SUPPLEMENTC, 1998 

Limitations and Strengths of Spontaneous Reports Data * 

Stephen A. Goldman, MD 

us Food and Drug Administration, Rockville, Maryland 

ABSTRACT 

US Food and Drug Administration (FDA) 

monitoring of the continued safety of marketed medical products depends greatly 

on spontaneous reporting of serious adverse events by health professionals. Despite its inherent limitations, the national 

postrnarketing surveillance system provides vital information of clinical importance. Key words: FDA, spontaneous reports, postmarketing surveillance, adverse 

drug event. 

INTRODUCTION 

As with clinical trials, there are important 

limitations to consider when using spontaneously reported adverse event information. These limitations include diffi-

""This manuscript is adapted from: Goldman SA. The 

clinical impact of adverse event reporting. MedWatch 

continuing education article. Rockville, Md: US Food 

and Drug Administration. October 1996. Available at 

www.fda.gov/medwatchlarticles.htm. where it first 

appeared as a section of that publication. 

C40 

culties with adverse event recognition, 

underreporting, biases, estimation of population exposure, and report quality. This 

paper addresses the limitations and 

strengths of data obtained from spontaneous reporting of adverse events. 

L~TATIONSOFSPONTANEOUS 

REPORTS DATA 

Adverse Event Recognition 

The recognition of adverse drug events, 

or any other medical product-associated 

adverse event, is quite subjective and imprecise.1 Although an attribution between 

the medical product and the observed 

event is assumed with all spontaneously 

reported events, every effort is made to 

rule out other explanations for the event 

in question. It is well known that placebos2 and even no treatment3 can be associated with adverse events. In addition, an 

underlying background rate almost always 

exists for any clinical event in a population, regardless of whether exposure to a 

medical product occurred. 

0149-2918/98f$19.00 

BPV-17-01-00190286 

Case 2:15-md-02641-DGC Document 8574-3 Filed 11/02/17 Page 2 of 6
• 

• 

• 

S.A. GOLDMAN 

Reaching a finn conclusion about the relationship between exposure to a medical 

product and the occurrence of an adverse 

event can be difficult In one study, clinical 

pharmacologists and treating physicians 

showed complete agreement less than half 

the time when detennining whether medication, alcohol, or ''recreational'' drug use 

. had caused hospitalization.4 

Such considerations emphasize the crucial need for careful, thoughtful review of 

adverse event reports on their receipt by 

the US Food and Drug Administration 

(FDA) or the manufacturer. It is through 

this process that causality, or at least a 

high degree of suspicion for a product-adverse event association, is put to the test. 

UndeTTeporting 

Another major concern with any spontaneous reporting system is underreporting of adverse events.S-8 It has been estimated that rarely more than 10% of 

serious adverse drug reactions (ADRs), 

and 2% to 4% of nonserious reactions, are 

reported to the British spontaneous reporting program.6 A similar estimate is 

that the FDA receives by direct report less 

than 1 % of suspected serious ADRs. 8 This 

means that cases reported spontaneously 

to any surveillance program, which comprise the numerator, generally represent 

only a small portion of the number that 

have actually occurred. The effect of underreporting can be somewhat lessened if 

submitted reports, irrespective of number, 

are of high quality. 

Biases 

Unlike clinical trial data, which are obtained under strictly controlled conditions, 

spontaneously reported infonnation is uncontrolled and therefore subject to the possible influence of a number of biases that 

can affect reporting. These biases include I 

the length of time a product has been on, 

the market. country, reporting environment, •• 

detailing time, and quality of the data. 9 A 

striking illustration of the impact that one 

such factor can have is the finding that the 

peak of spontaneous ADR reporting for a 

drug is at the end of the second year of' 

rnariceting. with a subsequent precipitous 

decline in reportioglO despite a lack of apparent decline in usage or change in ADR 

incidence.IO,9 In addition to these biases, it " 

is possible that reported cases might differ i, 

from nonreported cases in characteristics 

such as time to onset or severity. 11 

Estimation of Populolion Exposure 

Compounding these numemtor limitations is the lack of denominator data, such 

as user population and drug exposure patterns.1I These data· would provide the exact number of patients exposed to the 

medical product and thus at risk for the 

adverse event of interest. Numemtor and 

denominator limitations make incidence 

rates computed from spontaneously reported data problematic. 11 if not completely baseless. However, even if the precise number of exposed patients is not 

known, estimation of the exposure can be 

attempted by using drug utilization data. 12 ' 

This approach, whose basic methodologies are applicable to medical products 

in general, can be of great use. Major 

sources of data on the use of drugs by a 

defined population include market surveys based on sales or prescription data, 

third-party payers or health maintenance 

organizations, institutional/ambulatory 

settings, or specific. pharmacoepidemiologic studies.12 Coopemtive agreements 

C41 

BPV-17-01-00190287 

Case 2:15-md-02641-DGC Document 8574-3 Filed 11/02/17 Page 3 of 6
• 

• 

• 

and contracts with outside researchers enable the FDA to use such databases in its 

investigations. Device utilization studies 

use the same sources of data, as well as 

Medicare-derived information. 

Care must be taken in interpreting results from studies using these databases. 

That drug prescribing does not necessarily equal drug usage,12 and whether results derived from a specific population 

(such as Medicaid recipients) are applicable to the population at large, need to be 

weighed carefully. 

Report Quality 

The ability to assess, analyze, and act on 

safety issues based on spontaneous reporting is dependent on the quality of information submitted by health professionals 

in their reports. A complete adverse event 

report should include the following: product name (and information such as model 

and serial numbers in the case of medical 

devices); demographic data; succinct clinical description of adverse event, including 

confmnatory/relevant testllaboratory results; confounding factors (eg, concomitant medical products and medical history); 

temporal information, including date of 

event onset and start/stop dates for use of 

medical product; dose/frequency of use (as 

applicable); biopsy/autopsy results (as applicable); dechallenge/rechallenge information (if available); and outcome. 

STRENGTHS OF SPONTANEOUS 

REPORTS DATA 

Large-Scale and Cost-Effective 

Two vital advantages of surveillance 

systems based on spontaneous reports are 

that they potentially maintain ongoing surC42 

CLINICAL THERAPEUTICS-., 

veillance of all patients and are relatively 

inexpensive. 13 In fact, they are probably, 

the most cost-effective way to detect rare, ' 

serious adverse events not discovered during clinical trials. 

Generation of Hypotheses and Signals 

Making the best possible use of the data ,. 

obtained through monitoring underlies 

postmarketing surveillance. 14 Toward that 

goal, the great utility of spontaneous reports lies in hypothesis genemtion,7 with 

the need to explore possible explanations 

for the adverse event in question. By fostering suspicions, IS spontaneous reportbased surveillance programs perform an 

important function, which is to genemte 

signals of potential problems that warrant 

further investigation. 

Assessment of the medical product-adverse event relationship for a particular report or series of reports can be difficult 

The table lists factors that are helpful in 

evaluating the strength of association between a drug and a reported adverse event 16 

The stronger the drug-event relationship in each case and the lower the incidence of the adverse event occurring spontaneously, the fewer case reports are 

needed to perceive causality.17 It has been 

found that for rare events, coincidental 

drug-event associations are so unlikely 

that they merit little concern, with greater 

than three reports constituting a signal requiring further study.11 In fact, it has been 

suggested that a tempomi relationship between medical product and adverse event, 

coupled with positive dechallenge and 

rechallenge, can make isolated reports 

conclusive as to a product-event association.1S Biologic plausibility and reasonable strength of association aid in deeming any association as causal.6 However, 

BPV-17-01-00190288 

Case 2:15-md-02641-DGC Document 8574-3 Filed 11/02/17 Page 4 of 6
• 

• 

• 

S.A. GOLDMAN 

Table. Useful factors for assessing causal relationship between a drug and a reported 

adverse event. 16 

Chronology of administration of agent, including beginning and ending of treatment and 

adverse event onset 

Course of adverse event when suspected agent stopped (decballenge) or continued 

Etiologic roles of agents and diseases in regard to adverse event 

Response to readminislration (rechaIlenge) of agent 

Laboratory test results 

Previously known toxicity of agent 

achieving certain proof of causality 

through postmarketing surveillance is unusual. 11 Attaining a prominent degree of 

suspicion is much more likely, and may 

be considered a sufficient basis for regulatory decisions. 11 

Clinician Contribution 

The reliance of .postmarketing surveillance systems on health professional reporting enables an individual to help improve public health. One study showed that 

direct pmctitioner participation in the FDA 

spontaneous reporting system was the most 

effective source of new ADR reports that 

led to changes in product labeling.19 Postmarketing surveillance is further enhanced 

. by ensuring that the infonnation provided 

in the adverse event report is as complete 

and in depth as possible. 

CONCLUSIONS 

Although possessing inherent limitations, 

posbnarketing surveillance based on spontaneous reports data is a powerful tool for 

detecting adverse event signals of .direct 

clinical impact. It is dependent not only on 

health professional participation, but also 

on the quality of the reports submitted. 

Address correspondence to: Stephen A. 

Goldman, MD, Associate Director for 

Medicine, MedWatch, Office of the 

Commissioner, US Food and Drug 

Administration, S600 Fishers Lane, HF-2, 

Room 11-65, Rockville, MD 20857. 

REFERENCES 

1. Koch-Weser 1, Sellers EM, Zacest R. The 

ambiguity of adverse drug reactions. Eur 

J elin PhannacoL 1977;11:75-78 . 

2. Green OM. Pre-existing conditions, 

placebo reactions, and "side effects." Ann 

Intern Med. 1964;60:255-265. 

3. Reidenberg MM, Lowenthal DT. Adverse 

nondrug reactions. NEJM. 1968;279: 

678-679. 

4. Karch FE, Smith CL, Kerzner D, et at. 

Adverse drug reactions-a matter of opinC43 

BPV-17-01-00190289 

Case 2:15-md-02641-DGC Document 8574-3 Filed 11/02/17 Page 5 of 6
• 

• 

• 

ion. Clin Pharmacol Ther. 1976;19: 

489-492. 

5. Chen RT, Rastogi SC, Mullen JR, et al. 

The Vaccine Adverse Event Reporting 

System (VAERS). Vaccine. 1994;12: 

542-550. 

6. Rawlins MD. Pharmacovigilance: Par· 

adise lost, regained or postponed? The 

William Withering Lecture 1994. J R Coli 

Physicians Lond. 1995;29:41-49. 

7. Strom BL, Tugwell P. Pharmacoepidemi· 

ology: Current status, prospects, and prob· 

lems.Ann IntemMed 1990;113:179-181. 

8. Scott RD, Rosenbaum SE, Waters WJ, et 

al. Rhode Island physicians' recognition 

and reporting of adverse drug reactions. R 

I Med J. 1987;70:311-316. 

9. Sachs RM. Bortnichak EA. An evaluation 

of spontaneous adverse drug reaction 

monitoring systems. Am J Med. 1986;81 

(Suppl 5B):49-55. 

10. Weber JCP. Epidemiology of adverse reo 

actions to nonsteroidal antiinflammatory 

drugs. In: Rainsford KD. Vel0 OP, eds. 

Advances in Inflammation Research. Vol 

6. New York: Raven Press; 1984:1-7. 

11. Begaud B. Moride Y, Tubert-Bitter P, et 

al. False·positives in spontaneous report· 

ing: Should we worry about them? Br J 

Clin Pharmacal. 1994;38:401-404. 

C44 

CLINICAL THERAPEUTICS·· 

12. Serradell J, Bjornson DC, Hartzema AG. ' 

Drug utilization study methodologies: Na. ,i 

tional and international perspectives. Drug :" 

Intell Clin Pharm. 1987;21 :994-100 1. 

13. Fletcher AP. Spontaneous adverse drug reaction reporting vs event monitoring: A 

comparison. J R Soc Med. 1991;84: 

341-344. 

14. Finney OJ. The detection of adverse reactions to therapeutic drugs. Stat Med. 1982; 

1:153-161. 

15. Finney OJ. Statistical aspects of monitoring for dangers in drug therapy. Methods 

InfMed.1971;1O:1-8. 

16. Standardization of definitions and criteria 

of causality assessment of adverse drug 

reactions: Drug-induced liver disorders: 

Report of an international consensus meeting. Int I Clin Pharmacal Ther Toxicol. 

1990;28:317-322. 

17. Auriche M, Loupi E. Does proof of causal· 

ity ever exist in phannacovigilance? Drug 

Sa/. 1993;9:230-235. 

18. Temple RJ, Jones JK, Crout JR. Adverse 

effects of newly marketed drugs. NEJM. 

1979;300:1046-1047. 

19. Rossi AC, Knapp DE. Discovery of new 

adverse drug reactions: A review of the 

Food and Drug Administration's Sponta· 

neous Reporting System. JAMA. 1984; 

252:1030-1033. 

BPV-17-01-00190290 

Case 2:15-md-02641-DGC Document 8574-3 Filed 11/02/17 Page 6 of 6