This is the accessible text file for GAO report number GAO-03-494 
entitled 'Dietary Supplements: Review of Health-Related Call Records 
for Users of Metabolife 356' which was released on April 30, 2003.

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

Report to the Chairman, Subcommittee on Wellness and Human Rights, 
Committee on Government Reform, House of Representatives:

United States General Accounting Office:

GAO:

March 2003:

DIETARY SUPPLEMENTS:

Review of Health-Related Call Records for Users of Metabolife 356:

Metabolife 356 Adverse Event Reports:

GAO-03-494:

Contents:

Letter:

Results in Brief:

Background:

Consumer Information in Call Records Was Limited, Sometimes Difficult 
to Interpret, and Not Consistently Recorded:

Metabolife International Had Thousands of Call Records Reporting 
Adverse Events Associated with Metabolife 356:

Findings of Different Reviews of Metabolife International Call Records 
Vary:

Summary:

Agency and Metabolife International Comments and Our Evaluation:

Appendix I: Scope and Methodology for Categorizing the Call Records:

Call Records And Supplementary Information:

Exclusion of Duplicate and Nonhealth-Related Call Records:

Classification of Records and Data Entry Procedures:

Appendix II: Metabolife 356 Label:

Appendix III: Requirements for Reporting Adverse Events to FDA:

Appendix IV: Comments from the Food and Drug Administration:

Appendix V: GAO Contact and Staff Acknowledgments:

GAO Contact:

Acknowledgments:

Tables:

Table 1: Percentage of Call Records in Which Consumer and Response 
Details Were Recorded:

Table 2: Metabolife 356 Call Records Reporting Heart Attack, Stroke, 
Seizure, or Death:

Table 3: Metabolife 356 Call Records Reporting Adverse Events Described 
as Serious or Potentially Serious in FDA's 1997 Proposed Rule:

Table 4: Number of Call Records Containing Reports of Heart Attack, 
Stroke, Seizure, Death, or Cardiac Arrest Reported in Reviews of 
Metabolife International Call Records:

Table 5: Requirements for Reporting Adverse Events to FDA:

Abbreviations:

ANADA: Abbreviated New Animal Drug Application:

ANDA: Abbreviated New Drug Application:

FDA: Food and Drug Administration:

HHS: Department of Health and Human Services:

NADA: New Animal Drug Application:

NDA: New Drug Application:

This is a work of the U.S. Government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. It may contain 
copyrighted graphics, images or other materials. Permission from the 
copyright holder may be necessary should you wish to reproduce 
copyrighted materials separately from GAO's product.

United States General Accounting Office:

Washington, DC 20548:

March 31, 2003:

The Honorable Dan Burton
Chairman
Subcommittee on Wellness and Human Rights
Committee on Government Reform
House of Representatives:

Dear Mr. Chairman:

Medical experts have expressed concerns about the safety of dietary 
supplements containing ephedra or ephedrine alkaloids, which are used 
by millions of Americans annually.[Footnote 1] On February 28, 2003, 
the Food and Drug Administration (FDA) announced several proposed 
changes to its regulation of dietary supplements containing ephedra, 
including requiring a product label warning that "Heart attack, stroke, 
seizure, and death have been reported after consumption of ephedrine 
alkaloids."[Footnote 2] As of September 27, 2002, FDA had received 
approximately 1,800 adverse event reports regarding consumers of 
dietary supplements containing ephedra. Of these, 322 concerned 
Metabolife 356, a weight loss product first marketed in 1995 by 
Metabolife International, a large manufacturer of dietary supplements 
containing ephedra. Adverse event reports can signal potential health 
problems that deserve additional investigation, but, on their own, 
generally are not sufficient to establish that the reported problems 
were caused by use of the product.

Metabolife International has also received complaints about adverse 
health events among users of Metabolife 356.[Footnote 3] Between August 
and December 2002, Metabolife International made available to the 
public redacted[Footnote 4] copies of nearly 16,000 pages of 
documentation that it identified as containing reports of adverse 
events among consumers of Metabolife 356.[Footnote 5] These complaints, 
which were received between May 1997 and July 2002, had not been 
previously released to FDA. Most of them were records of calls received 
through a consumer health information phone line established by 
Metabolife International in 1998 and staffed by its nurses.[Footnote 6] 
Metabolife International officials told us that the phone line was 
established to provide information to consumers regarding appropriate 
use of Metabolife 356. In letters to the Texas Department of Health and 
FDA,[Footnote 7] company officials described the phone line as a 
"safety monitoring procedure" for the reporting of medical complaints. 
The call records ranged from handwritten notes to printed versions of 
records that had been entered into a database developed by Metabolife 
International. These call records have been the subject of six previous 
reviews: one by Metabolife International,[Footnote 8] three by 
consultants commissioned by Metabolife International,[Footnote 9] one 
by the minority staff of the Committee on Government Reform, House of 
Representatives,[Footnote 10] and one by the RAND Corporation.[Footnote 
11]

You asked us to review the content of all health-related call records 
made public by Metabolife International. Specifically, you asked us to 
answer the following questions. (1) To what extent was consumer 
information in the call records comprehensive, interpretable, and 
consistently recorded? (2) How many call records reported health-
related problems, and how many of those were serious? (3) How do our 
counts of reported serious adverse events compare with those of other 
reviews for those events counted by Metabolife International?

In responding to your request, we reviewed all the pages of 
documentation voluntarily provided to us by Metabolife International. 
We did not independently verify that we received all of the call 
records held by Metabolife International. We excluded from our review 
call records that duplicated other records.[Footnote 12] To determine 
the extent to which consumer information was comprehensive, 
interpretable, and consistently recorded in the call records, we 
recorded information about the adverse event, demographic information 
about the individual consumer, and other details in the call record. 
Specifically, we recorded background information similar to that used 
by FDA in the reporting of adverse events, including age, sex, weight, 
height, the amount of Metabolife 356 used, the duration of use, and 
whether any medical history was noted in the call record.

To assess how many call records reported health-related problems and 
how many of those were serious, we first counted the number of call 
records that reported at least one adverse event. Within this set of 
call records, we then counted the number of reports of heart attack, 
stroke, seizure, and death--the types of serious adverse events 
identified in FDA's proposed label warning. We also counted the number 
of reports of the 
23 other types of adverse events that were described as serious or 
potentially serious in FDA's 1997 proposed rule on dietary supplements 
containing ephedrine alkaloids.[Footnote 13] For call records that did 
not report any of the above adverse events, we counted the number of 
records, but did not count the number of other specific types of 
adverse events reported.

We classified events in the call records based solely on the words and 
phrases therein; we did not attempt to diagnose a consumer's condition 
or to otherwise interpret the information presented.[Footnote 14] We 
did not apply medical judgment in the process of classifying events and 
we did not independently verify the accuracy of the information in the 
records or determine the validity of the claims made in the call 
records. We also did not attempt to determine whether Metabolife 356 
caused the reported adverse events. Our results may either overestimate 
or underestimate the number and severity of adverse events because the 
call records generally do not include medical diagnoses made by 
qualified professionals.[Footnote 15]

To determine how our counts of reported serious adverse events compare 
with those of other reviews, we examined the six previous reviews of 
Metabolife International's call records. In addition, we interviewed 
Metabolife International and FDA officials. Appendix I describes our 
methodology in more detail. We conducted our work from September 2002 
through March 2003 in accordance with generally accepted government 
auditing standards.

Results in Brief:

The information in the Metabolife International call records was 
limited, sometimes difficult to understand and interpret, and consumer 
information was not consistently recorded. In some cases, the evidence 
for a report of an adverse event was limited to a single word on a call 
record. In addition, most call records did not record complete 
information about the consumer's age, sex, weight, and height. 
Information about both the amount of product used and duration of use 
was recorded for 60 percent of the call records. Further, handwritten 
call records were difficult to read and understand. Different versions 
of the call records sometimes contained different information about the 
consumer and the symptoms they reported. Nearly all of the reports of 
adverse events that contained information about the amount of 
Metabolife 356 used and duration of use were for consumers who reported 
following the usage guidelines on the product label, not for consumers 
who reported that they took too much Metabolife 356 or used it for too 
long a period.

We categorized 14,684 call records from Metabolife International as 
containing reports of at least one adverse event associated with 
Metabolife 356. We found that there were 92 reports of the serious 
adverse events identified in FDA's proposed label warning for dietary 
supplements containing ephedrine alkaloids: 18 reported heart attacks, 
26 reported strokes, 43 reported seizures, and 5 reported deaths. Among 
the other adverse events reported that were identified as serious or 
potentially serious in FDA's 1997 proposed rule, we found, for example, 
93 reports of significant elevation of blood pressure, 31 reports of 
abnormal heart rhythm, 47 reports of loss of consciousness, and 
181 reports of systemic rash. Because of the inherent limitations of 
adverse event reports and the incomplete nature of these call records, 
we cannot establish that the reported adverse events were caused by the 
use of Metabolife 356.

All of the reviews of the Metabolife International call records, 
including ours, counted reports of serious adverse events, although 
none of the reviews reported identical results. For those adverse 
events that Metabolife International counted--heart attacks, strokes, 
seizures, deaths, and cardiac arrests--our counts of reported events 
are similar to the counts from the other reviews. We counted 96 such 
reported events. Metabolife International counted 78, and the counts of 
the other reviews ranged from 65 to 107.

In commenting on a draft of this report, FDA discussed the value of 
reports of adverse events in helping to understand the causes of such 
events.

Background:

Metabolife 356, which claims to raise the body's metabolism and help 
dieters lose weight while maintaining high energy levels, contains 
32 ingredients, including ephedra, guarana (an herbal source of 
caffeine), bee pollen, and caffeine.[Footnote 16] The product label 
recommends that adults take one to two caplets two to three times per 
day or every 4 hours, not to exceed eight caplets per day. Warnings on 
the product label suggest that a health care professional be consulted 
by individuals who are using any other dietary supplement, prescription 
drug, or over-the-counter drug containing ephedrine alkaloids or who 
have, or have a family history of, any of 11 health conditions, 
including heart disease, high blood pressure, diabetes, recurrent 
headaches, and depression. The label also recommends that persons 
should not use the product for more than 
12 weeks and that exceeding the recommended amount may cause serious 
adverse health effects including heart attack or stroke. Other possible 
side effects mentioned on the label include rapid heartbeat, dizziness, 
severe headache, and shortness of breath. The complete product label is 
in appendix II.

The Dietary Supplement Health and Education Act of 1994 created a 
framework for FDA's regulation of dietary supplements as part of its 
oversight of food safety. Dietary supplements are generally marketed 
without prior FDA review of their safety and effectiveness.[Footnote 
17] Manufacturers of dietary supplements are responsible for ensuring 
the safety of the dietary supplements they sell. Therefore, FDA relies 
on voluntary reports of adverse events from consumers, health 
professionals, and others in its effort to oversee the safety of 
marketed dietary supplements.

Although there are no adverse event reporting requirements for 
manufacturers of dietary supplements, there are such requirements for 
many other products regulated by FDA. Various types of adverse events 
associated with the use of human drugs and biologics,[Footnote 18] 
animal drugs, animal feeds containing animal drugs, medical devices, 
infant formulas, and radiation-emitting devices must be reported to 
FDA. In addition to dietary supplements, other products regulated by 
FDA that do not require adverse event reporting are foods, cosmetics, 
and color additives. (See app. III for details about adverse event 
reporting requirements.):

Voluntary adverse event reporting systems can be valuable tools for 
identifying potentially serious health issues that may be associated 
with the use of a product and for maintaining ongoing surveillance. FDA 
has used adverse event reports to identify issues for further 
investigation and, as we previously reported, it has used adverse event 
reports to help identify dietary supplements for which evidence of harm 
existed, and has issued warnings and alerts for dietary 
supplements.[Footnote 19] However, by themselves, adverse event 
reporting systems generally are not sufficient to establish that a 
product caused the reported health problem. As we noted in 1999, all 
voluntary surveillance systems, including FDA's adverse event reporting 
system, have certain weaknesses.[Footnote 20] These include 
underreporting, reporting biases, difficulties estimating population 
exposure, and poor report quality. For example, the Department of 
Health and Human Services (HHS) Inspector General reported that a study 
commissioned by FDA estimated that FDA receives reports for less than 
1 percent of adverse events associated with dietary supplements. 
[Footnote 21] In addition, it is often difficult to rule out other 
possible explanations for the event; for example, the event may have 
been caused by preexisting medical conditions, or by the concurrent use 
of prescription drugs, over-the-counter drugs, or other supplements. 
For these reasons, data from adverse event reports alone cannot be used 
to determine if the occurrence of a symptom among product users is 
unusually high.

Between August and December 2002, Metabolife International released 
copies of 15,948 pages of documents that it said contained call records 
that reported adverse events associated with Metabolife 356 that the 
company had received from May 1997 through July 2002. Some pages of 
call records contained information about more than one call while 
others did not contain reports of adverse events. Some pages were 
photocopies or duplicates of other pages.

Consumer Information in Call Records Was Limited, Sometimes Difficult 
to Interpret, and Not Consistently Recorded:

The information about reported adverse events in the 14,684 health-
related call records we examined was limited. Most of the call records 
we reviewed did not completely record demographic or medical history 
information about the consumer. Information about age, sex, weight, 
height, the amount of product used, and the duration of use was 
frequently not recorded. Handwritten call records were difficult to 
read and interpret. Information was often inconsistent across different 
versions of the same call record.

The call records contained limited information about reported adverse 
events and consumers. In some cases the evidence for a report of an 
adverse event was a single health-related word on the call record, such 
as "seizure" or "stroke." In addition, demographic and medical history 
information was not consistently recorded in the call records. Most of 
the call records we reviewed did not record information about the 
consumers' sex, age, weight, or height. Eighty-eight percent of the 
call records did not record at least one of these variables. In 
addition, information about the amount of Metabolife 356 used and the 
duration of use was not recorded in 27 and 33 percent of the call 
records, respectively. (See table 1.) The absence of this information 
makes it difficult to assess whether the call records represent a 
signal of health concerns related to the consumption of Metabolife 
356.[Footnote 22]

Table 1: Percentage of Call Records in Which Consumer and Response 
Details Were Recorded:

Type of detail recorded: Age; Percentage of call records: with 
information: (n=14,684): 42%.

Type of detail recorded: Sex[A]; Percentage of call records: with 
information: (n=14,684): 41.

Type of detail recorded: Weight; Percentage of call records: with 
information: (n=14,684): 62.

Type of detail recorded: Height; Percentage of call records: with 
information: (n=14,684): 34.

Type of detail recorded: Amount of Metabolife used; Percentage of call 
records: with information: (n=14,684): 73.

Type of detail recorded: Duration of use; Percentage of call records: 
with information: (n=14,684): 67.

Type of detail recorded: Medical history; Percentage of call records: 
with information: (n=14,684): 45.

Source: GAO.

Note: Analysis of 14,684 health-related call records provided by 
Metabolife International. Where information was not recorded, we do not 
know if Metabolife International did not record information in the call 
records or if the caller did not provide the information.

[A] Metabolife International likely has more information about 
consumers' sex than we did because in many cases the company had access 
to the names of consumers to help make that determination. Consumers' 
names had been removed from the records Metabolife International 
provided us to protect consumer privacy.

[End of table]

Both the amount of product used and duration of use were recorded for 
60 percent of the calls reporting adverse events. Relatively few of 
these records involved consumers who reported taking too much 
Metabolife 356 or using it for too long a period. Specifically, among 
call records containing information on the amount of product used or 
duration of use, 99 and 91 percent of consumers, respectively, reported 
using the product within the guidelines recommended on the label.

The format of the call records varied from brief handwritten notes to 
typed notations to printed versions of a form used by Metabolife 
International. In general, less information was recorded for the one-
third of call records that were handwritten than all other types of 
records. For example, calls recorded on a typed form more frequently 
recorded additional information such as recommendations by Metabolife 
International to discontinue Metabolife 356 (62 percent) or contact a 
doctor (54 percent) than did those on handwritten forms (13 percent and 
8 percent, respectively).

Further, it was often difficult to read handwritten call records. We 
could not always determine how many calls were reported on a single 
page since there was rarely a clear delineation of events. Because 
handwritten call records did not follow a template, we were unable to 
determine if some information was medical history or symptom 
information, or if a number was a weight, heart rate, or blood 
pressure.

Information in call records was sometimes inconsistent. Where duplicate 
call records were available, information about consumers and their 
usage of Metabolife 356 was sometimes presented differently in the 
different records of the same consumer call. In addition, Metabolife 
International officials told us that its nurses sometimes used several 
different terms to document the same type of adverse event.

Metabolife International Had Thousands of Call Records Reporting 
Adverse Events Associated with Metabolife 356:

We found that 14,684 of the Metabolife International call records 
reported at least one adverse event. Ninety-two of these were for the 
serious adverse events identified in the proposed label warning for 
dietary supplements containing ephedra that FDA announced on February 
28, 2003. Other adverse events reported included significant elevation 
of blood pressure, abnormal heart rhythm, loss of consciousness, and 
systemic rash. We cannot establish that any of the reported adverse 
events were caused by the use of Metabolife 356.

Reports of Adverse Events Identified as Serious in FDA's Proposed Label 
Warning:

We counted 92 reports of heart attack, seizure, stroke, or death--the 
serious adverse events identified in FDA's proposed label warning for 
dietary supplements containing ephedra (see table 2).[Footnote 23]

Table 2: Metabolife 356 Call Records Reporting Heart Attack, Stroke, 
Seizure, or Death:

Type of adverse event: Heart attack; Number[A]: 18.

Type of adverse event: Stroke; Number[A]: 26.

Type of adverse event: Seizure; Number[A]: 43.

Type of adverse event: Death; Number[A]: 5.


Source: GAO.

Note: Analysis of 14,684 health-related call records provided by 
Metabolife International.

[A] The counts do not represent unique consumers because a single call 
record may have more than one complaint and because some consumers 
called the Metabolife health information phone line more than once.


[End of table]

Other Adverse Events:

In its 1997 proposed rule on dietary supplements, FDA also identified 
other types of adverse events as serious or potentially serious. Table 
3 shows our counts for almost all such events.[Footnote 24] The serious 
and potentially serious types of adverse events described in FDA's June 
4, 1997, proposed rule were reported to the agency prior to June 7, 
1996. FDA officials report that some other types of adverse events not 
included in the table may be considered serious or potentially serious 
but had not been reported to FDA during the time period considered by 
its proposed rule.

Table 3: Metabolife 356 Call Records Reporting Adverse Events Described 
as Serious or Potentially Serious in FDA's 1997 Proposed Rule:

Category of event: Cardiovascular.

Event reported: Cardiovascular: Category of event: Chest pain; 
Number[A]: Cardiovascular: Category of event: 433.

Event reported: Cardiovascular: Category of event: Significant 
elevation in blood pressure[B]; Number[A]: Cardiovascular: Category of 
event: 93.

Event reported: Cardiovascular: Category of event: Abnormal heart 
rhythm (alternative names for this event include dysrhythmia, 
ventricular tachycardia, ventricular fibrillation, atrial 
fibrillation, atrial flutter)[C]; Number[A]: Cardiovascular: Category 
of event: 31.

Event reported: Cardiovascular: Category of event: Cardiomyopathy; 
Number[A]: Cardiovascular: Category of event: 7.

Event reported: Cardiovascular: Category of event: Cardiac arrest; 
Number[A]: Cardiovascular: Category of event: 4.

Event reported: Cardiovascular: Category of eventNervous system: 
Angina; Number[A]: Cardiovascular: Category of eventNervous system: 3.

Category of event: Nervous system.

Event reported: Cardiovascular: Category of event: Loss of 
consciousness; Number[A]: Cardiovascular: Category of event: 47.

Event reported: Cardiovascular: Category of event: Psychosis; 
Number[A]: Cardiovascular: Category of event: 7.

Event reported: Cardiovascular: Category of event: Altered 
consciousness (including disorientation or confusion); Number[A]: 
Cardiovascular: Category of event: 4.

Event reported: Cardiovascular: Category of event: Suicidal; Number[A]: 
Cardiovascular: Category of event: 3.

Event reported: Cardiovascular: Category of event: Vestibular (inner 
ear) disturbance; Number[A]: Cardiovascular: Category of event: 2.

Event reported: Cardiovascular: Category of event: Severe depression; 
Number[A]: Cardiovascular: Category of event: 2.

Event reported: Cardiovascular: Category of eventOther: Mania; 
Number[A]: Cardiovascular: Category of eventOther: 1.

Category of event: Other.

Event reported: Cardiovascular: Category of event: Systemic rash; 
Number[A]: Cardiovascular: Category of event: 181.

Event reported: Cardiovascular: Category of event: Urinary infection; 
Number[A]: Cardiovascular: Category of event: 110.

Event reported: Cardiovascular: Category of event: Urinary retention; 
Number[A]: Cardiovascular: Category of event: 72.

Event reported: Cardiovascular: Category of event: Elevations of liver 
function tests; Number[A]: Cardiovascular: Category of event: 54.

Event reported: Cardiovascular: Category of event: Prostatitis; 
Number[A]: Cardiovascular: Category of event: 24.

Event reported: Cardiovascular: Category of eventCardiovascular: 
Hepatitis; Number[A]: Cardiovascular: Category of eventCardiovascular: 
1.

Source: GAO.

Note: Analysis of 14,684 health-related call records provided by 
Metabolife International.

[A] The counts do not represent unique consumers because a single call 
record may have more than one complaint and because some consumers 
called the Metabolife health information phone line more than once.

[B] We used the MEDLINE Plus Medical Encyclopedia to define significant 
elevations in blood pressure as a measurement of greater than 160 
millimeters of mercury systolic or 100 millimeters of mercury 
diastolic. This count does not include call records that mentioned 
"high blood pressure" or "elevated blood pressure" without specifying 
these levels.

[C] Alternative names for abnormal heart rhythm were determined using 
the MEDLINE Plus Medical Encyclopedia (www.nlm.nih.gov/medlineplus/
encyclopedia.html) (downloaded December 2002 through February 2003).

[End of table]

In addition, the 14,684 call records with health-related reports 
presented a broad range of types of adverse events. Many of the call 
records contained reports of jitters, insomnia, hair loss, bruising, 
menstrual irregularities, and sexual dysfunction, as well as vague 
references to events such as "side effect" or "felt sick." Some 
reported blood in stool, blood in urine, or blood clots. There were 
also some reports of visits to emergency departments and hospital 
admissions. Some call records contained reports of diseases such as 
pulmonary embolus (a blockage of an artery in the lungs), multiple 
myeloma, and inflammation of heart tissue.

Causal Role of Metabolife 356 Cannot Be Established:

We cannot establish that any of the adverse events reported in the 
Metabolife International call records were caused by the use of 
Metabolife 356. As we noted earlier, adverse event reports by 
themselves are generally not sufficient to establish that a health 
problem was caused by the use of a particular product. For example, for 
many adverse event reports it is difficult to rule out other possible 
explanations for the event--the event may have been caused by 
preexisting medical conditions, or by the concurrent use of 
prescription drugs, over-the-counter drugs, or other dietary 
supplements. In addition, the limited information available in the 
Metabolife International call records means that we cannot confirm that 
a particular adverse event occurred, much less identify a specific 
cause for it.

Findings of Different Reviews of Metabolife International Call Records 
Vary:

All the reviews of the Metabolife International call records, including 
ours, counted reports of serious adverse events. None of the reviews 
reported identical tabulations of these events. For the set of adverse 
events that Metabolife International counted--heart attack, stroke, 
seizure, death, and cardiac arrest--our counts are similar to those of 
the other reviews (see table 4). In total, we counted 96 such events, 
Metabolife International counted 78, and the counts of the other 
reviews ranged from 65 to 107.

Table 4: Number of Call Records Containing Reports of Heart Attack, 
Stroke, Seizure, Death, or Cardiac Arrest Reported in Reviews of 
Metabolife International Call Records:

Events: Heart attack; GAO: 18; Metabolife: 16[F]; Karch[A]: 17; 
Mozayani[B]: 13; Molgaard[C]: 13; Minority Staff, Committee on 
Government Reform, House of Representatives[D]: 20; RAND[E]: 22.

Events: Stroke; GAO: 26; Metabolife: 20; Karch[A]: 24; Mozayani[B]: 19; 
Molgaard[C]: 13; Minority Staff, Committee on Government Reform, House 
of Representatives[D]: 24; RAND[E]: 31.

Events: Seizure; GAO: 43; Metabolife: 35; Karch[A]: 40; Mozayani[B]: 
52; Molgaard[C]: 36; Minority Staff, Committee on Government Reform, 
House of Representatives[D]: 40; RAND[E]: 46.

Events: Death; GAO: 5; Metabolife: 3; Karch[A]: 2; Mozayani[B]: 4; 
Molgaard[C]: 3; Minority Staff, Committee on Government Reform, House 
of Representatives[D]: 3; RAND[E]: 5.

Events: Cardiac arrest; GAO: 4; Metabolife: 4[F]; Karch[A]: 4; 
Mozayani[B]: 5; Molgaard[C]: NC; Minority Staff, Committee on 
Government Reform, House of Representatives[D]: NC; RAND[E]: 3.

Events: Total; GAO: 96; Metabolife: 78; Karch[A]: 87; Mozayani[B]: 93; 
Molgaard[C]: 65; Minority Staff, Committee on Government Reform, House 
of Representatives[D]: 87; RAND[E]: 107.

Source: GAO and others.

Notes: "NC" indicates that these types of events were not counted by 
these reviews. The counts do not represent unique consumers because a 
single call record may have more than one complaint and because some 
consumers called the Metabolife health information phone line more than 
once.

[A] Steven B. Karch, An Analysis of Metabolife 356 HealthLine Contacts 
(August 2002), www.ephedrafacts.com/metabolife.html (downloaded Dec. 
12, 2002), and An Analysis of an Additional 3268 HealthLine Records 
(Jan. 17, 2003).

[B] Ashraf Mozayani, Analysis of Metabolife 356 Health Line Reports 
(August 2002), and Supplemental Report of Analysis of Metabolife 356 
Health Line Reports (January 2003).

[C] Craig A. Molgaard, Epidemiologic Assessment of Health Line Reports 
about a Dietary Supplement (August 2002), and An Analysis of Additional 
HealthLine Records (Jan. 17, 2003).

[D] Minority Staff Report, Special Investigations Division, Committee 
on Government Reform, House of Representatives, Adverse Event Reports 
from Metabolife (October 2002), www.house.gov/reform/min (downloaded 
Dec. 11, 2002). This review did not include at least 1,480 pages of 
call records Metabolife International later made available to us and 
other reviews.

[E] Paul Shekelle, Sally C. Morton, Margaret Maglione, and colleagues, 
Ephedra and Ephedrine for Weight Loss and Athletic Performance 
Enhancement: Clinical Efficacy and Side Effects. Evidence Report/
Technology Assessment No. 76 (Prepared by Southern California Evidence-
based Practice Center, RAND, under Contract No. 290-97-0001, Task Order 
No. 9). AHRQ Publication No. 03-E022, Rockville, Md: Agency for 
Healthcare Research and Quality, February 2003.

[F] Metabolife International provided to us a list of call records it 
considered to report cardiac events. Because the other reviews counted 
heart attacks and cardiac arrests separately, we examined the events 
that Metabolife International classified as cardiac events to 
categorize them as cardiac arrest or heart attack.

[End of table]

There are several possible reasons for the slightly different counts of 
serious adverse events in the different reviews. First, the call 
records themselves are often difficult to understand and interpret. 
Second, not all of the reviews included the same set of call records, 
both because some were completed before all of the Metabolife 
International call records were released and because the reviews 
adopted different procedures for identifying and discarding duplicate 
records. Third, the reviews used different definitions of particular 
events or established different thresholds for categorizing a 
particular event. For example, we included reports of "convulsions" in 
our count of seizures, while some other reviews may not have. 
Specifically, the counts we report in table 4 for our review and the 
reviews by Metabolife International and Karch include reports of 
convulsions, while it is not clear if the other reviewers' counts did. 
Similarly, we did not count as a report of a heart attack a call record 
that reported "heart attack?", while at least one other review did.

Summary:

The information in the Metabolife International call records was 
limited, sometimes difficult to understand and interpret, and consumer 
information was not consistently recorded. Most call records contained 
only limited information about a consumer and the event being reported, 
and handwritten records were difficult to read and understand. We 
categorized 14,684 call records from Metabolife International as 
containing reports of at least one adverse event associated with 
Metabolife 356. We found that there were 92 reports of the types of 
serious adverse events identified in FDA's proposed label warning for 
dietary supplements containing ephedrine alkaloids. All of the reviews 
of the Metabolife International call records, including ours, counted 
reports of serious adverse events, although none of the reviews 
reported identical results. We counted 96 reports of the types of 
events counted by Metabolife International--heart attack, stroke, 
seizure, death, and cardiac arrest--and the counts of the other reviews 
ranged from 65 to 107.

Agency and Metabolife International Comments and Our Evaluation:

We provided a draft of this report to FDA and Metabolife International 
for their review. FDA asked us to clarify that it has not conducted its 
own review of the Metabolife International call records, that we only 
reviewed reports of adverse events contained in the Metabolife 
International call records, and that we did not review other reports of 
adverse events among users of Metabolife 356 that have been received by 
FDA. In addition, FDA pointed out that, when combined with other 
information, adverse event reports can help establish that an adverse 
event was caused by a particular health product. FDA's comments are 
included as appendix IV. FDA also provided technical comments, which we 
incorporated as appropriate.

In its comments, Metabolife International was primarily concerned about 
our use of the term "adverse events" to describe the health-related 
complaints that were reported in the call records we reviewed. We 
believe that our use of the term is accurate and consistent with its 
use by FDA and others in the field. Metabolife International also 
wanted us to clarify that, while it did identify some call records as 
containing references to types of specific adverse events that have 
been categorized as serious by others, it has not identified any call 
records as reporting "serious adverse events." We have made revisions 
so as not to imply that Metabolife International labeled these events 
as serious adverse events. Metabolife International also made other 
comments, which we incorporated as appropriate.

As agreed with your office, unless you publicly announce its contents 
earlier, we plan no further distribution of this report until 30 days 
from the date of this report. At that time, we will send copies to the 
Secretary of HHS, the Commissioner of FDA, and others who are 
interested. We will also provide copies to others upon request. In 
addition, the report will be available at no charge on GAO's Web site 
at http://www.gao.gov.

If you or your staff have any questions, please contact me at (202) 
512-7119. Another contact and major contributors to this report are 
listed in appendix V.

Sincerely yours,

Marcia Crosse
Acting Director, Health Care--Public Health and Science Issues:

Signed by Marcia Crosse:

[End of section]

Appendix I: Scope and Methodology for Categorizing the Call Records:

We reviewed call records and supplementary information voluntarily 
provided to us by Metabolife International to (1) determine the extent 
to which information was comprehensive, interpretable, and consistently 
recorded in the call records, and (2) count the number of call records 
reporting health-related problems associated with Metabolife 356, and 
how many of them were serious. During our review we removed duplicate 
call records and records that did not report health-related events. For 
each record we recorded demographic information about the individual 
consumer, other details about the call record and the consumer, and 
categorized the reported events.

Call Records And Supplementary Information:

From August 2002 through December 2002, Metabolife International 
voluntarily provided to us 15,948 pages[Footnote 25] of documentation 
relating to reports of adverse events among consumers of Metabolife 
356. Most of these records were from calls made to the company's 
consumer health information phone line from May 1997 through July 2, 
2002.[Footnote 26] Other records included e-mail messages and letters 
that had been sent to the company. Nurses on the staff of Metabolife 
International documented the calls to the consumer HealthLine in a 
variety of formats. The records included handwritten notes on a page, 
typed and handwritten letters, forms with handwritten entries, e-mails, 
and printed versions of records that had been entered into a database 
developed by Metabolife International. Many kinds of forms were used to 
record calls, ranging from simple forms with few spaces or check boxes 
to full-page forms with multiple boxes for consumer and event-related 
information. Metabolife International officials told us that health 
complaints that were noted on product return forms that it received 
were not in the call records provided to us.

Metabolife International also provided to us copies of 46 redacted 
medical records and a list of corresponding call records. After 
reviewing these records we found 8 that were not associated with other 
call records. Five of these records contained enough information to 
determine the nature of the adverse event and were coded in the same 
way as other call records. The other medical records were used as 
additional sources of information for documenting the events and 
consumer information reported in their corresponding records.

While most pages of call records contained information about a single 
call, some included information about multiple calls on the same page, 
other calls spanned multiple pages, and some did not include any report 
of adverse events. Records that spanned multiple pages were often 
letters to the company, some of which were sent with additional 
information (such as medical bills). Records that did not report an 
adverse event were either incomplete printouts of other records from 
the database, product questions, complaints about not losing weight, or 
reports of consumer satisfaction. As a result, the number of pages of 
call records that we received from Metabolife International does not 
correspond to the number of reports of adverse events.

The call records and medical records we received were redacted by 
Metabolife International to remove personal identifying information 
such as name, phone number, address, fax number, and e-mail address to 
protect consumer privacy. Metabolife International officials told us 
that in the process of redacting the records, some relevant adverse 
event information was also inadvertently removed.

Exclusion of Duplicate and Nonhealth-Related Call Records:

Metabolife International officials told us that there were duplicate 
call records in the set of call records they provided to us. Some 
duplicate reports were photocopies of the same call record. In other 
cases, there were multiple versions of the same call record in 
different formats. Metabolife International officials reported these 
multiple versions were the result of nurses taking handwritten notes 
and later entering the same information directly into a database 
established in September 1999.

Metabolife International gave us lists of those call records it 
believed to be duplicates. Over the course of our review, it identified 
more than 
2,200 records for which there were at least one duplicate. Metabolife 
International officials reported that they identified the duplicates on 
the basis of the name of the consumer. Duplicates may have included 
subsequent calls about different events from the same individual. We 
examined the duplicate call records identified in the lists provided 
throughout our review by Metabolife International. Because identifying 
information was removed, we examined the date of the call record, 
demographic information about the consumer (such as age, height, 
weight, the amount of the product used, and duration of use), and event 
details to determine if they were duplicate records. Where this 
information was the same or similar, we considered the records to be 
duplicates and excluded the extra records from our review. We did, 
however, include in our analysis any additional information that 
appeared on the duplicate records. For example, if one version included 
height and another weight, we recorded both of these.

We agreed with Metabolife International that most of the more than 
2,200 records it identified as duplicates were, in fact, duplicates. 
However, we did not exclude records that represented multiple calls 
from the same consumer for different events if the dates on the call 
records differed by more than a few days or the symptoms were clearly 
different. During the course of our review, we also identified 
duplicates not previously identified by Metabolife International, 
including photocopied records and records that used identical language 
in event descriptions. We do not know if all duplicate call records 
were identified.

We also excluded from our analysis records in which there was no health 
complaint or the health complaint could not be clearly determined. We 
also excluded call records that reported third-hand knowledge of 
adverse events (such as a friend of a friend who experienced an adverse 
event). In addition, we did not count call records that clearly 
referred to nutrition bars or other ephedra-free products manufactured 
by Metabolife International. In total, we determined that the 15,948 
pages of documentation provided by Metabolife International contained 
14,684 separate health-related call records.

Classification of Records and Data Entry Procedures:

We classified the adverse events reported in each call record and 
entered the appropriate codes into a database. We classified the 
reported adverse events as either one of the events FDA identified as 
serious in its 
February 28, 2003, announcement regarding a proposed label warning for 
dietary supplements containing ephedra (heart attack, stroke, seizure, 
or death) or as an other adverse event. All serious events reported 
within a particular call record were counted. Therefore an individual 
could have reported multiple serious adverse events, though this 
happened in few records. For other adverse events, we documented 
whether the call record reported one or more adverse events. We did not 
count the number of reports for every type of event reported in the 
record. We did, however, count the number of all but 1 of the 24 other 
types of adverse events that were described as serious or potentially 
serious in FDA's June 4, 1997, proposed rule on dietary supplements 
containing ephedrine alkaloids.[Footnote 27] The set of events 
identified by FDA in the proposed rule is not an exhaustive list of the 
adverse events that may be associated with the use of dietary 
supplements containing ephedrine alkaloids. FDA officials told us that 
some other types of adverse events may be considered serious or 
potentially serious but had not yet been reported to FDA during the 
time period considered by its proposed rule.

We did not apply medical judgment in the process of identifying and 
classifying events. Our classification of events in the call records 
was based solely on the words and phrases therein; we did not diagnose 
a consumer's condition or otherwise interpret the information 
presented. For example, if a report said "poss. heart attack," "heart 
attack symptoms," or "heart attack?", we did not classify it as a heart 
attack since it was not clear that a heart attack was reported. Also, 
while we counted "blood pressure 210/120" as an instance of 
significantly elevated blood pressure because it reported measurements 
greater than 160 systolic or 100 diastolic, we did not place in the 
same category call records that reported only "high blood pressure" 
because they did not contain the specific measurements needed for that 
determination. We used MEDLINE Plus Medical Encyclopedia definitions to 
further clarify individual symptoms related to these 
categories.[Footnote 28] We also did not attempt to determine whether 
Metabolife 356 caused the reported adverse events.

[End of section]

Appendix II: Metabolife 356 Label:

[See PDF for image]

Source: Metabolife International, February 12, 2003.

[End of section]

Appendix III: Requirements for Reporting Adverse Events to FDA:

Adverse events about many types of products regulated by FDA are 
required to be reported to the agency. Such products include human 
drugs, biologics, animal drugs, animal feeds containing animal drugs, 
medical devices, infant formulas, and radiation-emitting devices. There 
are, however, no reporting requirements for adverse events associated 
with other products regulated by FDA, including food and food 
additives, dietary supplements, cosmetics, or color additives. (See 
table 5.):

Table 5: Requirements for Reporting Adverse Events to FDA:

Product: Human drugs (including over-the-counter drugs) with approved 
New Drug Application (NDA) or Abbreviated New Drug Application 
(ANDA)[A]; Adverse events that must be reported to FDA: Serious and 
unexpected adverse drug experiences from all sources (domestic and 
foreign).[B]; Who reports: NDA and ANDA applicants, and any person 
whose name is on the label of an approved drug as manufacturer, packer, 
or distributor (" nonapplicants").; When reported: As soon as possible 
but within 15 calendar days. Nonapplicants may, instead, submit reports 
to applicants within 5 calendar days.

Adverse events that must be reported to FDA: New information obtained 
as result of follow-up investigation of earlier reports.; Who reports: 
Same as above.; When reported: Within 15 calendar days of receipt of 
new information or as requested by FDA. Nonapplicants may, instead, 
submit reports to applicants within 5 calendar days.

Adverse events that must be reported to FDA: Adverse experiences that 
occur domestically and that are serious and expected or not serious 
(expected or unexpected).; Who reports: NDA and ANDA applicants.; When 
reported: At quarterly intervals for the first 3 years after approval 
and then annually or at different times upon written notice by FDA.

Adverse events that must be reported to FDA: Serious and unexpected 
adverse drug experiences described in scientific literature as case 
reports or as the result of a formal clinical trial, or from or during 
postmarketing studies where the applicant concludes that there is a 
reasonable possibility that drug caused reaction.[B]; Who reports: NDA 
and ANDA applicants and nonapplicants.; When reported: Within 15 
calendar days.

Product: Human drugs without approved NDAs/ANDAs[C]; Adverse events 
that must be reported to FDA: Serious and unexpected adverse drug 
experiences from all sources (domestic and foreign).[B]; Who reports: 
Any person whose name is on the label as a manufacturer, packer, or 
distributor; and the manufacturer even if its name does not appear on 
the label, when it receives adverse drug experience reports directly 
from a packer or distributor.; When reported: As soon as possible but 
within 15 calendar days; packers and distributors may, instead, submit 
reports to manufacturers within 
5 calendar days.

Adverse events that must be reported to FDA: Serious and unexpected 
adverse drug experiences from a postmarketing study where there is 
reasonable possibility that drug caused reaction.[B]; Who reports: Same 
as above.; When reported: Same as above.

Adverse events that must be reported to FDA: New information obtained 
as result of follow-up investigation of 15-day alert reports.; Who 
reports: Same as above.; When reported: Within 15 calendar days of 
obtaining the information or as requested by FDA.

Product: Biologics[D]; Adverse events that must be reported to FDA: 
Serious and unexpected adverse experiences from all sources described 
in scientific literature, or described in postmarketing clinical 
studies where there is a reasonable possibility product caused 
reaction.[B]; Who reports: Licensed manufacturerse and manufacturers, 
packers, distributors, or other manufacturing participants whose name 
appears on the label.; When reported: As soon as possible but no later 
than 15 calendar days. Packers, distributors, and other nonlicensees 
required to report may submit reports to licensed manufacturers within 
5 calendar days.

Adverse events that must be reported to FDA: New information obtained 
as a result of follow-up of 15-day alert reports.; Who reports: Same as 
above.; When reported: Within 15 days of receipt of new information or 
as requested by FDA. Packers, distributors, and other unlicensed firms 
required to report may submit reports to licensees within 5 calendar 
days.

Adverse events that must be reported to FDA: Adverse experiences that 
are expected or nonserious.; Who reports: Licensed manufacturers.; When 
reported: At quarterly intervals for the first 3 years after license 
approval and then annually or at different times upon written notice by 
FDA.

Adverse events that must be reported to FDA: Certain reactions 
associated with administration of vaccines listed in 42 U.S.C. 300aa-
14.; Who reports: Vaccine manufacturers and health care providers.; 
When reported: Within 7 days of the administration of listed vaccines 
or as specified.[F].

Adverse events that must be reported to FDA: Fatality resulting from 
blood collection or transfusion.[G]; Who reports: Collecting facilities 
in the event of donor reaction; facilities performing compatibility 
tests in the event of transfusion reaction.; When reported: As soon as 
possible by telephone, facsimile, express mail, or electronic 
transmission with a written report to follow within 7 days.

Product: Animal drugs[H]; Adverse events that must be reported to FDA: 
Unexpected side effects, injury, toxicity, sensitivity, reaction; 
unexpected incidence or severity, or unusual failure to exhibit 
expected pharmacological activities.; Who reports: Applicants for New 
Animal Drug Application (NADA) or Abbreviated New Animal Drug 
Application (ANADA), including those whose name appears on the labeling 
as a manufacturer, packer, distributor, or who are engaged in 
manufacturing, processing, packing, or labeling of drug.; When 
reported: As soon as possible but within 15 working days of receipt by 
the applicant.

Adverse events that must be reported to FDA: Mix-up in new animal drug 
or its labeling with another article, bacteriological or significant 
physical or other change or deterioration in the drug, or failure to 
meet specifications.; Who reports: Same as above.; When reported: 
Immediately (generally within 
3 days).

Product: Animal feeds bearing or containing animal drugs[I]; Adverse 
events that must be reported to FDA: Mix-up with another drug or its 
labeling with another article; bacteriological or significant chemical, 
physical, or other change or deterioration in the drug; or failure to 
meet specifications.; Who reports: NADA and ANADA applicants.; When 
reported: Immediately (generally within 
3 days).

Adverse events that must be reported to FDA: Information concerning 
unexpected side effect, injury, toxicity, sensitivity reaction, any 
unexpected incidences or severity, or unusual failure to exhibit 
expected pharmacological activities.; Who reports: Same as above.; When 
reported: As soon as possible but within 15 working days of receipt by 
the applicant.

Product: Medical devices[J]; Adverse events that must be reported to 
FDA: Device-related deaths or serious injuries.; Who reports: Device 
user facilities.[K]; When reported: Within 10 work days of receiving 
relevant information; annual reports must summarize all reported 
events.

Adverse events that must be reported to FDA: Device-related deaths or 
serious injuries.; Who reports: Importers.; When reported: Within 30 
days of becoming aware of event.

Adverse events that must be reported to FDA: Device-related deaths or 
serious injuries.[L]; Who reports: Device manufacturers.; When 
reported: Within 30 days of becoming aware of event, or within 5 days 
if the event requires remedial action to prevent an unreasonable risk 
of substantial harm to the public health or if FDA has made a written 
request.

Adverse events that must be reported to FDA: Information that would 
have had to have been reported earlier but was unknown or unavailable.; 
Who reports: Same as above.; When reported: Within 1 month of receiving 
information.

Product: Infant formula[M]; Adverse events that must be reported to 
FDA: Possible causal connection between consumption of an infant 
formula and infant death.; Who reports: Manufacturers.; When reported: 
Within 15 days, conduct an investigation and notify FDA.

Product: Radiation-emitting devices[N]; Adverse events that must be 
reported to FDA: Injurious or potentially injurious exposure to 
radiation from nonmedical electronic products.[O]; Who reports: 
Manufacturers.; When reported: Immediately.

Product: Food and food additives; Adverse events that must be reported 
to FDA: No requirements to report adverse events.; Who reports: Not 
applicable.; When reported: Not applicable.

Product: Dietary supplements; Adverse events that must be reported to 
FDA: No requirements to report adverse events.; Who reports: Not 
applicable.; When reported: Not applicable.

Product: Cosmetics; Adverse events that must be reported to FDA: No 
requirements to report adverse events.; Who reports: Not applicable.; 
When reported: Not applicable.

Product: Color additives; Adverse events that must be reported to FDA: 
No requirements to report adverse events.; Who reports: Not 
applicable.; When reported: Not applicable.

[A] 21 C.F.R.  314.80, 314.98 (2002). Over-the-counter drugs are subject 
to FDA's adverse event reporting requirements only to the extent they 
are covered by approved NDAs or ANDAs. On March 14, 2003, FDA published 
a proposed rule which includes requirements for reporting suspected 
adverse events associated with drugs and biological products ("Safety 
Reporting Requirements for Human Drug and Biological Products," 68 Fed. 
Reg. 12406).

[B] FDA refers to these as 15-day alert reports.

[C] 21 C.F.R.  310.305. Adverse events associated with investigational 
new drugs are required to be reported under sections 312.32 and 312.33 
of Title 21 of the Code of Federal Regulations. Also see FDA's proposed 
rule at 68 Fed. Reg. 12406 (Mar. 14, 2003).

[D] 21 C.F.R.  600.80. There are no reporting requirements for 
manufacturers of whole blood or components of whole blood. 21 C.F.R.  
600.80(k)(1). Also see FDA's proposed rule at 68 
Fed. Reg. 12406 (Mar. 14, 2003).

[E] In vitro diagnostic products are subject to the reporting 
requirements for devices. 
21 C.F.R.  600.80(k)(2).

[F] 42 U.S.C.  300aa-25(b).

[G] 21 C.F.R.  606.170(b).

[H] 21 C.F.R.  510.300. FDA is in the process of redrafting the 
adverse event reporting rules for approved animal drugs.

[I] 21 C.F.R.  510.301. Certain medicated items incorporated into 
animal feeds are also subject to the animal drug reporting 
requirements. See 21 C.F.R.  514.80(a)(4).

[J] 21 C.F.R. pt. 803. Not all medical device adverse events must be 
reported to FDA; user facilities are required to report serious 
injuries to FDA only if the manufacturers are not known. 21 C.F.R.  
803.30(a)(2). Adverse events associated with devices under 
Investigational Device Exemptions must be reported and summaries must 
be included in applications submitted to FDA for premarket approval. 21 
C.F.R.  812.150, 814.20.

[K] Device user facilities do not include physician offices, school 
nurse offices, and employee health units. 21 C.F.R.  803.3(f).

[L] Manufacturers must also report to FDA if a device has malfunctioned 
and such malfunction, were it to recur, would be likely to cause or 
contribute to a death or serious injury. 21 C.F.R.  803.50(a)(2).

[M] 21 C.F.R.  106.100(k)(3). Manufacturers must promptly report to 
FDA knowledge about an infant formula it has processed and that has 
left its establishment if the infant formula may be adulterated or 
misbranded and that may present a risk to human health. 21 C.F.R.  
106.120(b).

[N] 21 C.F.R. 1002.20.

[O] 21 C.F.R.  1000.3, 1002.20. Nonmedical electronic products 
include, for example, microwave ovens and infrared alarm systems. If a 
product is classified as a medical device, the normal medical device 
reporting requirements apply.

[End of table]

[End of section]

Appendix IV: Comments from the Food and Drug Administration:

DEPARTMENT OF HEALTH & HUMAN SERVICES:

Food and Drug Administration Rockville MD 20857:

March 20, 2003:

Marcia Crosse, Ph.D.

Acting Director, Health Care-Public Health and Science Issues United 
States General Accounting Office 441 G Street, NW:

Washington, DC 20548:

Dear Dr. Crosse:

Please find the enclosed comments from the Food and Drug Administration 
on the GAO draft report entitled, DIETARY SUPPLEMENTS: Review of 
Reports of Adverse Events Among Users of Metabolite 356 (GAO-03-494). 
The Agency provided extensive technical comments directly to your 
staff.

We appreciate the opportunity to review and comment on this draft 
report before its publication as well as the opportunity to work with 
your staff in developing this report.

Sincerely,

Mark B. McClellan, M.D., Ph.D Commissioner of Food and Drugs:

Signed for Mark B. McClellan

Enclosure:

General Comments by the Department of Health and Human Service's Food 
and Drug Administration (FDA) on General Accounting Office's (GAO) 
Draft Report DIETARY SUPPLEMENTS Review of Reports of Adverse Events 
among Users of Metabolife 356 (GAO-03-494):

FDA appreciates the opportunity to comment on GAO's draft report which 
focuses additional attention on the area of adverse event reporting 
associated with dietary supplements.

We have a few general comments regarding the overall report, as 
follows:

The draft report implies that FDA conducted its own review and analyses 
of the adverse event reports submitted by Metabolife. This is not 
accurate.

There are multiple sets of adverse event reporting systems and 
databases related to dietary supplements containing ephedrine alkaloids 
and Metabolife and multiple databases. The GAO report references two 
different reporting systems (FDA's and Metabolife's) and discusses 
various interpretations of data subsets from these reporting systems 
(RAND, Minority House staff, etc.). The draft report is not 
sufficiently clear about which subset of data was used for this review. 
FDA encourages GAO to make additional clarifications regarding these 
systems and databases in the final report.

We conclude with our concern about authoritative statements made 
against the use of adverse events to prove, determine, or establish 
causality. While it may be true that causality can only rarely be 
definitively established from a reported adverse event, this does not 
mean that causality can never be established in an individual adverse 
event. Aggregated adverse events can not be used to establish risks in 
a population because this requires more complete and accurate 
information about the size of population exposed to a particular agent, 
and the number of individuals experiencing a particular type of adverse 
event (in exposed and non-exposed persons); for these reasons, 
aggregated adverse events are used to signal a problem that requires 
further study.

Statements to the effect that "adverse event reports are not sufficient 
on their own to definitively establish causality" while technically 
true, are not an adequate reflection of current scientific standards 
for adverse event assessment. With enough supporting evidence, such as 
supporting medical documents, dechallenge, rechallenge, temporality, 
biological plausibility, dose response, etc., a causal association may 
be determined.

[End of section]

Appendix V: GAO Contact and Staff Acknowledgments:

GAO Contact:

Martin T. Gahart, (202) 512-3596:

Acknowledgments:

Carolyn Feis Korman, Chad Davenport, Julian Klazkin, and Roseanne Price 
also made major contributions to this report.

FOOTNOTES

[1] It has been estimated that 12 million Americans consumed dietary 
supplements with ephedra in 1999 (C. A. Haller and N. L. Benowitz, 
"Adverse Cardiovascular and Central Nervous System Events Associated 
with Dietary Supplements Containing Ephedra Alkaloids," The New England 
Journal of Medicine, vol. 343, no. 25 (2000)).

[2] See 68 Fed. Reg. 10417 (Mar. 5, 2003). FDA also announced that it 
is reopening the comment period for its June 4, 1997, proposed rule, 
"Dietary Supplements Containing Ephedrine Alkaloids" (62 Fed. Reg. 
30678). 

[3] There is no information available about the extent to which reports 
of particular adverse events may have been reported to both FDA and 
Metabolife International.

[4] The redaction consisted primarily of the removal of personal 
identifying information (such as names, phone numbers, addresses, and 
e-mail addresses) to protect consumer privacy. Although data relevant 
to the adverse event being reported were not supposed to be removed, 
Metabolife International officials noted that such information was 
occasionally accidentally removed. 

[5] The number of adverse event reports does not equal the pages of 
documentation because some pages contained reports of more than one 
call reporting an adverse health event, some reports of adverse health 
events spanned several pages, and some pages included reports not 
related to negative health consequences. 

[6] In addition to phone calls, some call records were letters and e-
mails sent to Metabolife International. 

[7] The letter to FDA is available at http://www.fda.gov/ohrms/dockets/
dockets/98n0148/2.htm (letter from Metabolife International received 
February 10, 1999) (downloaded March 24, 2003).

[8] Metabolife International has not issued a report on its review of 
the call records, but provided to us a list of the calls it believed to 
report heart attack, stroke, seizure, death, and cardiac arrest.

[9] Each of the consultants reviewed the first set of approximately 
12,700 pages of Metabolife International records released in August 
2002. Steven B. Karch, An Analysis of Metabolife 356 HealthLine 
Contacts (August 2002) www.ephedrafacts.com/metabolife.html 
(downloaded Dec. 12, 2002); Craig A. Molgaard, Epidemiologic Assessment 
of Health Line Reports about a Dietary Supplement (August 2002); Ashraf 
Mozayani, Analysis of Metabolife 356 Health Line Reports (August 2002). 
After more pages of call records were made available, each of the 
consultants completed updated reviews with these additional reports. 
Steven B. Karch, An Analysis of an Additional 3268 HealthLine Records 
(Jan. 17, 2003); Craig A. Molgaard, An Analysis of Additional 
HealthLine Records (Jan. 17, 2003); Ashraf Mozayani, Supplemental 
Report of Analysis of Metabolife 356 Health Line Reports (January 
2003).

[10] Minority Staff Report, Special Investigations Division, Committee 
on Government Reform, House of Representatives, Adverse Event Reports 
from Metabolife (October 2002). www.house.gov/reform/min (downloaded 
Dec. 11, 2002).

[11] Paul Shekelle, Sally C. Morton, Margaret Maglione, and colleagues, 
Ephedra and Ephedrine for Weight Loss and Athletic Performance 
Enhancement: Clinical Efficacy and Side Effects. Evidence Report/
Technology Assessment No. 76 (Prepared by Southern California Evidence-
based Practice Center, RAND, under Contract No. 290-97-0001, Task Order 
No. 9). AHRQ Publication No. 03-E022, Rockville, Md: Agency for 
Healthcare Research and Quality, February 2003. 

[12] Metabolife International officials identified call records they 
believed were duplicates of each other. We reviewed the relevant call 
records to determine which were duplicates. Call records identified by 
Metabolife International officials as duplicates were either 
photocopies of specific call records, multiple entries of the same call 
(such as handwritten notes that were later also entered into the 
database, creating two pages of call records for the same call), or 
multiple calls about the same consumer describing different events. We 
considered the first two instances, but not the third, to be 
duplicates. We did not include in our review reports that we considered 
duplicates. We also identified additional call records that were 
duplicates and removed them from our review.

[13] FDA's June 4, 1997, proposed rule identified serious or 
potentially serious adverse events associated with the use of ephedra 
based on a review of the literature and an analysis of 600 adverse 
event reports that FDA had received by June 7, 1996. See "Dietary 
Supplements Containing Ephedrine Alkaloids," 62 Fed. Reg. 30678. We did 
not count reports of one of the events FDA identified, "altered serum 
enzymes," because the proposed rule did not specify threshold values. 

[14] We required that certain words be in the call record for it to be 
counted as a specific type of event. For example, for a call record to 
meet the criteria for a stroke, it needed to specifically include the 
word "stroke," not related terms like "stroke-like symptoms." 

[15] Our findings may either overestimate or underestimate the number 
and severity of adverse events. Our findings may overestimate the 
number of adverse events because we accepted the events as they were 
reported on the page. For example, if a call record reported a stroke, 
we counted it as a stroke even though the consumer may not have 
actually had a stroke. Conversely, our findings may underestimate the 
number and severity of adverse events because individual adverse events 
we categorized as other adverse events may collectively suggest a more 
serious event. For example, we categorized a call record reporting 
left-side numbness and tingling and left-side face drooping as an other 
adverse event where a physician or other health professional might have 
determined that these symptoms actually represented a stroke. 

[16] According to Metabolife International officials, the only 
ingredient change since Metabolife 356 was placed on the market was 
made in early 2001, when bovine complex was removed from the product. 
Some other inactive ingredients may vary by manufacturing facility. 
Metabolife International officials told us that the same labels are 
used for products sold in all states. 

[17] FDA officials reported that the agency conducts a premarket review 
of safety information for certain supplements that contain new dietary 
ingredients. 

[18] Biologics are any virus, therapeutic serum, toxin, antitoxin, 
vaccine, blood, blood component or derivative, allergenic product, or 
analogous product applicable to the prevention, treatment, or cure of a 
human disease or condition. Biological products include, but are not 
limited to, bacterial and viral vaccines, human blood and plasma and 
their derivatives, and certain products produced by biotechnology, such 
as interferons and erythropoietins.

[19] U.S. General Accounting Office, Health Products for Seniors: 
"Anti-Aging" Products Pose Potential for Physical and Economic Harm, 
GAO-01-1129 (Washington, D.C.: Sep. 7, 2001).

[20] U.S. General Accounting Office, Dietary Supplements: Uncertainties 
in Analyses Underlying FDA's Proposed Rule on Ephedrine Alkaloids, GAO/
HEHS/GGD-99-90 (Washington, D.C.: July 2, 1999).

[21] HHS Office of Inspector General, Adverse Event Reporting for 
Dietary Supplements: An Inadequate Safety Valve, OEI-01-00-00180 
(Washington, D.C.: April 2001).

[22] We previously reported that adverse event reports should optimally 
include demographic data (GAO/HEHS/GGD-99-90). Such information is 
useful for determining whether or not the adverse events reported would 
be unexpected in a specific population of users, for example, heart 
attacks in young adults.

[23] See 68 Fed. Reg. 10417 (Mar. 5, 2003). 

[24] We counted all reports of 23 of the 24 other types of adverse 
events FDA identified as serious or potentially serious in its 1997 
proposed rule. We did not count reports of "altered serum enzymes" 
since the proposed rule did not specify threshold values. The other 
serious or potentially serious adverse events--coma, myopathies, 
exfoliative dermatitis, and epididymitis--are not reported in the table 
because we did not find any reports of them in the call records 
provided by Metabolife International.

[25] These 15,948 pages contained 14,684 call records that we 
categorized as reporting adverse events. The number of adverse event 
reports does not equal the pages of documentation because some pages 
contained reports of more than one call reporting an adverse health 
event, some reports of adverse health events spanned several pages, and 
some pages included reports not related to negative health 
consequences. 

[26] Metabolife International received the call records we reviewed 
primarily from mid 
1998 through July 2002, although 12 call records were from 1997 and 
some were from early 1998.

[27] FDA's June 4, 1997, proposed rule identified serious or 
potentially serious adverse events associated with the use of ephedra 
based on a review of the literature and an analysis of 600 adverse 
event reports that FDA had received by June 7, 1996. See "Dietary 
Supplements Containing Ephedrine Alkaloids," 62 Fed. Reg. 30678.

[28] www.nlm.nih.gov/medlineplus/encyclopedia.html (downloaded 
December 2002 through February 2003).

GAO's Mission:

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony:

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics.

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to daily E-mail alert for newly 
released products" under the GAO Reports heading.

Order by Mail or Phone:

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:

U.S. General Accounting Office

441 G Street NW,

Room LM Washington,

D.C. 20548:

To order by Phone: 	

	Voice: (202) 512-6000:

	TDD: (202) 512-2537:

	Fax: (202) 512-6061:

To Report Fraud, Waste, and Abuse in Federal Programs:

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470:

Public Affairs:

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.

General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.

20548: