United States General Accounting Office

GAO Report to the Chairman and Ranking

Minority Member, Committee on

Science, House of Representatives

July 1999 DIETARY

SUPPLEMENTS

Uncertainties in

Analyses Underlying

FDA’s Proposed Rule

on Ephedrine Alkaloids

GAO/HEHS/GGD-99-90

GAO United States

General Accounting Office

Washington, D.C. 20548

Health, Education, and

Human Services Division

B-281581 

July 2, 1999

The Honorable James Sensenbrenner, Jr.

Chairman

The Honorable George E. Brown, Jr.

Ranking Minority Member

Committee on Science

House of Representatives

The dietary supplement industry estimates that as many as 2 billion doses

of dietary supplements containing botanical ephedrine alkaloids are

consumed each year in the United States. The principal source of

botanical ephedrine alkaloids is the Chinese herb ma huang. Traditionally,

this substance was used by Chinese physicians to treat conditions such as

hay fever and asthma. In Western medicine, synthetic ephedrine alkaloids

have been used in products such as over-the-counter asthma and cold

medicines, while botanical ephedrine alkaloids have been used in dietary

supplements that are promoted to provide health benefits, including

helping individuals lose weight and increase energy levels.

In June 1997, the Food and Drug Administration (FDA) published a

proposed rule that would establish a dosing regimen, require warning

statements, and affect other aspects of product labeling for dietary

supplements containing ephedrine alkaloids. FDA developed its proposed

rule in response to what it termed “serious illnesses and injuries, including

multiple deaths, associated with the use of dietary supplement products

that contain ephedrine alkaloids.” FDA based its rule, in part, on a number

of adverse events reports (AER) it received that indicated that some health

problems could have been associated with use of dietary supplements

containing ephedrine alkaloids.

FDA determined that its proposed rule on dietary supplements containing

ephedrine alkaloids was an economically significant rule and, therefore,

was required to conduct a cost-benefit analysis pursuant to the

requirements in Executive Order 12866.1

 Other regulatory analysis

requirements for federal rulemaking are those under the Regulatory

Flexibility Act (RFA), which directs agencies to consider the potential

impact of regulation on small businesses and other small entities,2

 and the

Unfunded Mandates Reform Act (UMRA), which generally requires agencies

1

E.O. 12866 was issued on September 30, 1993, and covers all agencies except independent regulatory

agencies.

2

5 U.S.C. 601-612.

Page 1 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

to prepare cost-benefit analyses for any proposed regulation that would

impose mandates likely to result in expenditures of $100 million or more

in any 1 year.3

Industry groups and the Small Business Administration’s (SBA) Office of

Advocacy have challenged FDA’s proposed rule, claiming that the scientific

information and AERs supporting the rule were poor and unreliable.

Further, industry groups and the Office of Advocacy claimed that the

cost-benefit analysis performed by FDA understated the costs of the

regulation and overestimated the benefits. They also had concerns about

compliance with RFA’s and UMRA’s procedural requirements.

In light of the concerns expressed by SBA and industry groups, you asked

us to examine (1) the scientific basis for FDA’s proposed rule and (2) the

agency’s adherence to the regulatory analysis requirements for federal

rulemaking. In examining the basis for the rule, you asked that we

examine scientific evidence, FDA’s past use of and internal guidance on

AERs in rulemaking, and information contained in the AERs for determining

dosing regimens and benefits that would arise from the proposed rule.

Regarding the rulemaking requirements, you asked that we determine the

extent to which FDA’s analysis contained elements expected in a federal

agency’s cost-benefit analysis and to analyze FDA’s compliance with the

requirements in RFA and UMRA.

To meet these objectives, we interviewed representatives of and obtained

documents from FDA, SBA’s Office of Advocacy, the Office of Information

and Regulatory Affairs (OIRA) in the Office of Management and Budget

(OMB), and the dietary supplement industry. We also examined the public

docket of the proposed rule, performed a content analysis of a random

sample of AERs, and reviewed scientific literature and case reports of

adverse events from products containing ephedrine alkaloids. We

conducted our review from September 1998 to May 1999 in accordance

with generally accepted government auditing standards. (See app. I for

further information on our scope and methodology.)

Results in Brief FDA based its proposed rule on numerous reports of adverse events

associated with products thought to contain ephedrine alkaloids; it also

used evidence from scientific literature indicating that ingestion of

ephedrine alkaloids adversely affects some individuals. The number and

type of AERs warranted FDA’s consideration of steps to address safety

3

2 U.S.C. 638 and 1532.

Page 2 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

issues. However, we have concerns about the strength of some of the

information FDA used to support two aspects of the proposed rule: the

dosing level and duration of use limits. While there was scientific evidence

showing adverse events at levels above 20 mg per serving, FDA’s dosing

level proposal of 8 mg per serving was based on information associated

with only 13 AERs—the quality of which is questionable. For the duration

of use limits, FDA relied on scientific studies that showed problems

associated with extended use, well beyond the 7-day limit proposed.

Moreover, FDA did not establish a causal link between the ingestion of

ephedrine alkaloids and the occurrence of adverse events for either its

proposed dosing level or duration of use. FDA also based its estimate of the

benefits of the proposed rule on the annual number of adverse events

reported to FDA. However, because FDA did not document which AERs it

identified as containing “serious” events, we could not determine the

accuracy of FDA’s estimated benefits. In addition, FDA has no internal

guidance on the use of AERs for rulemaking related to foods and dietary

supplements, and the AERs were used differently in this proposed rule than

in prior rulemaking.

The agency generally complied with the statutory and executive order

requirements for rulemaking, but the cost-benefit analysis that

accompanied the rule does not reflect the full range of uncertainty

associated with the proposed rule. FDA’s cost-benefit analysis and other

analyses included the primary elements required under E.O. 12866 and

related “best practices” guidance and RFA. UMRA’s requirements did not

appear to apply to the rule. Although FDA disclosed the basic methodology,

data, and assumptions used in its cost-benefit analysis, the agency did not

always disclose why certain key assumptions were made or the degree of

uncertainty involved in those assumptions. It also did not disclose that

alternative assumptions would have had a dramatic effect on the agency’s

estimate of the benefits of the proposed actions.

While FDA’s conclusions regarding the desirability of the proposed action

may be valid, we believe these conclusions are open to question because

of limitations and uncertainties associated with the agency’s scientific and

economic analyses. Given these uncertainties, we recommend that FDA

obtain additional information to support the proposed dosing levels and

duration of use limits and improve the transparency of its cost-benefit

analysis before proceeding to final rulemaking.

Page 3 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

Background FDA is responsible for overseeing the federal government’s regulation of

drugs, medical devices, food safety, veterinary medicine, and biological

products. Unlike many of these products, dietary supplements do not have

to undergo preapproval by FDA to determine their safety or efficacy. The

Dietary Supplement Health and Education Act of 1994 created a new

framework for FDA’s regulation of dietary supplements as part of its

oversight of food safety.4

 The act allows dietary supplement products to

bear a statement describing how consumption of the supplement can

affect humans,5

 but manufacturers of dietary supplements cannot make a

drug claim for the product—that is, a statement claiming to diagnose,

mitigate, treat, cure, or prevent a specific disease or class of diseases.6

Dietary supplements containing botanical sources of ephedrine alkaloids

are currently marketed for weight loss and increased energy. These

products are marketed in a variety of forms, including pills, powders,

liquid drops, and teas. Currently, FDA has no requirements that the

packages for these products include dosing regimens and warning labels.

Over-the-counter products containing synthetic ephedrine alkaloids are

considered safe and effective for people ages 12 and older to temporarily

relieve shortness of breath, tightness of chest, and wheezing due to

bronchial asthma, if recommended dosages are followed. The

recommended dosages—which range from 12.5 mg to 25 mg every 4 hours,

not to exceed 150 mg per day—are provided on the label. Labels for

over-the-counter drug products containing ephedrine alkaloids list side

effects, such as nervousness, sleeplessness, nausea, and loss of appetite.

The labels also warn against using the product if, for example, the

potential user has high blood pressure, heart disease, thyroid disease, or

diabetes, or has been hospitalized for asthma or is taking a prescription

drug for asthma, unless directed by a doctor. Labels also note that if

symptoms persist or become worse, users should discontinue taking the

drug and consult a doctor. After reports of stroke in asthmatics who had

taken ephedrine along with prescription monoamine oxidase inhibitors,7

precautions against using both drugs concurrently were added to the label.

4

P.L. 103-417.

5

Such statements may (1) claim a benefit related to a classical nutritional deficiency disease,

(2) describe the role of a nutrient or dietary ingredient intended to affect the structure or function in

humans, (3) characterize the documented mechanism by which the ingredient acts to maintain such

structure or function, or (4) describe general well-being from the consumption of the ingredient.

6

21 U.S.C. 321 (g)(1).

7

Monoamine oxidase inhibitors are drugs used to treat depression, psychiatric or emotional

conditions, or Parkinson’s disease.

Page 4 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

FDA has no authority to require submission of reports of adverse events.

FDA must generally rely on consumers or their friends and family members,

physicians or other health care professionals, product manufacturers, and

state health agencies to voluntarily report adverse events. FDA uses these

AERs as a passive monitoring tool to identify potentially serious public

health issues that may be associated with the use of a particular product

or type of products. Like all passive surveillance systems, AERs have

certain limitations such as underreporting and poor report quality (see

app. II for a further discussion of the adverse event monitoring system).

FDA’s Proposed Rule on

Ephedrine Alkaloids

FDA’s analysis of AERs and information from the scientific literature relating

to ephedrine alkaloids indicated to the agency that there was cause for

concern regarding a potential public health problem associated with

dietary supplements containing ephedrine alkaloids. At the time of the

proposed rule, FDA had received over 800 AERs for products thought to

contain ephedrine alkaloids—more AERs than the agency had received for

any other dietary supplements.

According to agency officials, FDA found that, unlike most AERs related to

foods, the AERs relating to ephedrine alkaloids commonly involved visits to

a physician or emergency room. Some of these reports were very serious,

including effects such as strokes, and the events were occurring in a

population of young adults in which such serious events are not expected.

FDA officials also stated that the adverse physiological and

pharmacological effects from dietary supplements believed to contain

botanical sources of ephedrine alkaloids were similar to those reported for

drugs containing synthetic sources of ephedrine alkaloids, but these

effects were being seen at lower doses and potencies for botanicals than in

the drug products.

From 1994 through 1997, FDA took a series of steps to respond to what it

perceived was a public health concern related to these products. In 1994,

FDA’s Center for Food Safety and Applied Nutrition (CFSAN), which

oversees dietary supplements, issued a medical bulletin outlining potential

adverse reactions from dietary supplements containing ephedrine

alkaloids. The agency issued a press release warning consumers not to

purchase a particular brand of dietary supplement containing ephedrine

alkaloids that the agency determined could cause severe injury or death in

some people.

Page 5 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

In October 1995, FDA convened a special work group composed of

pharmacologists, physicians, and industry representatives to address

concerns related to the use of dietary supplements containing ephedrine

alkaloids, but no consensus developed on how to address the public health

concerns outlined by FDA. In August 1996, FDA’s Food Advisory Committee

was asked to provide opinions on and rationale for specific ways to

address problems associated with dietary supplements containing

ephedrine alkaloids. Over half the committee members concluded that no

safe level for ephedrine alkaloids could be identified and recommended

that these products be removed from the market. Most of the other

members felt that a fairly low level of ephedrine alkaloids would be

“reasonably safe.” The Food Advisory Committee was unable to identify a

benefit for ephedrine alkaloids in terms of supplementing the diet.

In June 1997, FDA published a proposed rule regarding dietary supplements

containing ephedrine alkaloids.8

 In two subsequent notices the agency

reopened and extended the comment period on the proposed rule until

December 1997.9

 The proposed rule

• defines levels of the amount of ephedrine alkaloids in a serving of dietary

supplements at and above which the product would be deemed

adulterated (8 mg),

• places restrictions on the frequency of use and daily dosages (24 mg or

more),

• requires that labels on these supplements contain a statement warning that

the product should not be used for more than 7 days,

• prohibits the use of ephedrine alkaloids with ingredients that have a

known stimulant effect (for example, caffeine),

• prohibits certain labeling claims that require long-term intake of the

supplements to achieve the purported purpose,

• requires a warning statement in conjunction with claims that encourage

short-term excessive intake, and

• requires a specific warning statement to appear on product labels.

Federal Regulatory

Analysis Requirements

E.O. 12866 establishes certain rulemaking responsibilities for covered

agencies. Among other things, the order states that, in deciding whether

and how to regulate, agencies should assess all costs and benefits of

available regulatory alternatives, including both quantifiable and

qualitative effects. The order also states that, in choosing among

8

62 Fed. Reg. 30678.

9

62 Fed. Reg. 44247 and 62 Fed. Reg. 48968.

Page 6 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

alternatives, an agency should select those approaches that maximize net

benefits and “base its decisions on the best reasonably obtainable

scientific, technical, economic, and other information concerning the need

for, and consequences of, the intended regulation.” The order requires

agencies to conduct cost-benefit analyses for all regulatory actions that are

likely to result in a $100 million annual effect on the economy or are

otherwise economically significant. In January 1996, the Administrator of

OIRA issued “best practices” guidance on preparing cost-benefit analyses

under the order. The guidance indicates that an analysis should contain

certain basic elements and should be “transparent”—disclosing how the

analysis was conducted, what assumptions were used, and the

implications of plausible alternative assumptions.

During the past 20 years, the Congress has attempted to improve the

federal rulemaking process by enacting a number of statutes that impose

certain analytical requirements on agencies issuing proposed regulations.

For example, RFA of 1980 was enacted in response to concerns about the

effect that federal regulations could have on small entities. RFA directs all

agencies to give particular attention to the potential impact of regulation

on small businesses and other small entities and requires consideration of

regulatory alternatives that are less burdensome to small entities. Under

RFA, an agency must prepare an initial regulatory flexibility analysis at the

time proposed rules are issued unless the head of the agency determines

that the proposed rule would not have a “significant economic impact

upon a substantial number of small entities.” The act also requires

agencies to ensure that small entities have an opportunity to participate in

the rulemaking process.

Other statutory rulemaking requirements are set forth in UMRA. UMRA

generally requires agencies (other than independent regulatory agencies)

to prepare cost-benefit and other analyses for any regulations imposing

mandates that are likely to result in expenditures of $100 million or more

in any 1 year either by state, local, and tribal governments, in the

aggregate, or by the private sector. Although UMRA’s scope and

requirements differ from E.O. 12866, the provisions on economic analysis

are very similar. Accordingly, the guidance for implementing the executive

order states that the economic analysis that the agency prepares should

also satisfy the requirements of UMRA.

Page 7 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

FDA Analyses Relied

on Poorly

Documented Reports

of Adverse Events

To develop its proposed rule, FDA used a combination of scientific

evidence on the effects of ephedrine alkaloids and a set of reports it

received on adverse events associated with dietary supplements

containing ephedrine alkaloids. While the signs and symptoms described

in the AERs were consistent with available scientific evidence and known

physiologic and pharmacologic effects of ephedrine alkaloids, the AERs

were poorly documented. FDA also used the AERs differently than in its past

rulemaking. Specifically, the agency used AERs as the sole source of

support for specific dosing levels, relied on weak information to set limits

on duration of use, and did not perform a causal analysis to determine

whether ingestion of a dietary supplement containing ephedrine alkaloids

caused or contributed to the adverse events. FDA also used these AERs to

determine the number of serious events that could be attributed to the

dietary supplements and the expected benefits that would arise if the

proposed rule were implemented. However, FDA did not document which

AERs it determined to be serious. Moreover, it did not establish criteria to

determine which events were serious and did not perform any reliability

assessments of its analyses.

Scientific Information

Indicates Ephedrine

Alkaloids Can Affect the

Cardiovascular and

Nervous Systems

Our review of the scientific literature found case reports that suggested

that ephedrine alkaloids could increase blood pressure in persons with

normal and high blood pressure;10 predispose certain individuals to

tachycardia (rapid heart rate);11 and cause cardiomyopathy (disease of the

heart muscle),12 stroke,13 or myocardial necrosis (death of cells in the

heart).14 We also found descriptions of adverse events associated with

10S. Chua and S. Benrimoj, “Non-Prescription Sympathomimetic Agents and Hypertension,” Medical

Toxicology, Vol. 3 (1988), pp. 387-417.

11D. McCleave and others, “Compartmental Shift of Potassium—A Result of Sympathomimetic

Overdose,” Australian and New Zealand Journal of Medicine, Vol. 8 (1978), pp. 180-83.

12J. Gualtieri and C. Harris, “Dilated Cardiomyopathy in a Heavy Ephedrine Abuser,” (abstract)

Journal of Toxicology, Clinical Toxicology, Vol. 34 (1996), pp. 581-82.

13A. Bruno and others, “Stroke Associated With Ephedrine Use,” Neurology, Vol. 43 (1993), pp.

1313-16.

14P. Pentel and others, “Myocardial Necrosis Due to Intraperitoneal Administration of

Phenylpropanolamine in Rats,” Fundamental and Applied Toxicology, Vol. 9 (1987), pp. 167-72.

Page 8 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

ephedrine alkaloids that affected the central nervous system, such as

mania,15 paranoid psychoses,16 and seizures.17

Our review found only one clinical study examining the effects of a

botanical source of ephedrine alkaloids, ma huang, on heart rate and

blood pressure. The study of adults with normal blood pressure found a

statistically—but not clinically—significant increase in heart rate but

variable effects on blood pressure.18 However, “in combination with other

stimulants and at higher doses, effects of magnification of the heart rate

and blood pressure response could be expected.” One report also

highlighted a case of mania as a result of ingestion of a dietary supplement

containing ma huang.

19

We also reviewed a number of studies using clinical trials that evaluated

the efficacy of ephedrine (a synthetic ephedrine alkaloid)—alone or in

combination with caffeine—for weight loss. Although these trials were not

designed to determine whether ephedrine is safe to use during weight loss

programs, FDA concluded that this body of literature showed “clinically

significant adverse effects in populations with no known risk factors with

the use of ephedrine and that synergistic adverse effects can result when

ephedrine and caffeine are combined.” FDA further concluded that “The

patterns and types of the adverse effects reported in these trials are

consistent with the known effects of sympathomimetic agents20—that is,

they mainly involved nervous and cardiovascular system effects.”

AERs Were Used

Differently for Developing

the Proposed Rule Than

for Prior Rulemaking

To develop FDA’s proposed rule on dietary supplements containing

ephedrine alkaloids, CFSAN used AERs differently than in prior FDA

rulemaking. FDA officials acknowledge that the agency does not have any

formal internal guidelines on the use of AERs for rulemaking and provided

five examples of past rulemaking in which CFSAN used several sources of

information in addition to AERs to support the agency’s proposals. For the

15W. Clovis, “Mania and Cough Syrup,” Journal of Clinical Psychiatry, Vol. 54 (1993), p. 200.

16M. T. Lambert, “Paranoid Psychoses After Abuse of Proprietary Cold Remedies,” British Journal of

Psychiatry, Vol. 151 (1987), pp. 548-50.

17S. M. Mueller and E. B. Solow, “Seizures Associated With a New Combination Pick-Me-Up Pill,”

Annals of Neurology, Vol. 11 (1982), p. 322.

18L. White and others, “Pharmacokinetics and Cardiovascular Effects of Ma-huang (Ephedra sinica) in

Normotensive Adults,” Journal of Clinical Pharmacology, Vol. 37 (1997), pp. 116-22.

19R. R. Capwell, “Ephedrine-Induced Mania From an Herbal Diet Supplement” (letter), American

Journal of Psychiatry, Vol. 152 (1995), p. 647.

20Such agents produce physiological responses resembling those that are caused by the action of the

sympathetic nervous system, as in tending to reduce digestive secretions and speeding up the heart.

Page 9 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

proposed rule on dietary supplements containing ephedrine alkaloids,

CFSAN used AERs alone to determine specific dosing levels.21

In a prior rulemaking, FDA used prevalence estimates among people with

asthma to determine sensitive populations who might be affected by

sulfiting agents22 and convened a scientific panel to examine exposure

estimates and evidence from clinical experiments.23 For its rulemaking on

protein supplements, FDA used data collected through a telephone survey,

conducted jointly with the Centers for Disease Control and Prevention

(CDC), and information on death rates from the National Center for Health

Statistics. FDA also initiated its own experimental protocols to examine the

basic metabolic mechanisms of protein diets.24 FDA’s proposed rule on

supplements and drugs containing iron relied on data from the American

Association of Poison Control Centers and the U.S. Consumer Product

Safety Commission to determine fatalities from accidental iron poisonings.

Data from the National Electronic Injury Surveillance System were also

incorporated to estimate the number of cases of hospital emergency room

treatment for iron ingestion.25 FDA did not establish dosing levels in any of

these rulemakings.

Although AERs were used as early warning signals in many of these cases,

the proposed rule on ephedrine alkaloids relied more heavily on AERs than

did prior rulemaking efforts. (See app. III for additional information on the

use of AERs in prior rulemaking.)

Shortcomings of AERs and

FDA’s Reliance on Them

Add Uncertainty to FDA’s

Proposed Rule

The inherent weaknesses of AERs—along with FDA’s heavy reliance on

them—lead to uncertainty regarding the dosing regimen outlined in the

proposed rule. The AERs used in the rule lacked or had inconsistent

information relevant to FDA’s analysis, such as the amount of product used,

how often it was used, or for how long it was used. The limitations of a

passive surveillance system such as AERs call into question FDA’s ability to

21FDA also used AERs to show evidence of dechallenge (signs and symptoms resolve or improve when

a consumer stops using a product) and rechallenge (symptoms recur when the consumer resumes

using the product).

22Sulfiting agents are one type of chemical preservative that serves to prevent or to delay the process

of browning and deterioration of raw fruits and vegetables.

2351 Fed. Reg. 25021.

2449 Fed. Reg. 13679.

2562 Fed. Reg. 2218.

Page 10 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

determine a specific dosing level based solely on these reports.26 FDA did

not perform a causal analysis to determine if, in fact, the 13 AERs it used to

set dosing levels were caused by supplements containing ephedrine

alkaloids. FDA indicated in the proposed rule that 10 to 73 percent of

reported adverse events might not be related to consumption of dietary

supplements containing ephedrine alkaloids. FDA’s support for its

recommended limit on duration of use was also weak.

FDA’s lack of documentation on which adverse events it classified as

serious makes it difficult to determine the validity of FDA’s estimates of the

benefits that would arise from the proposed rule. In addition, FDA did not

have specific criteria to determine which events should be considered

serious and did not perform a reliability assessment to ascertain the

validity of its estimates of serious events.

Adverse Event Reports Were

Incomplete and Inconsistent

Prior to the proposed rulemaking, FDA received 864 AERs on dietary

supplements containing ephedrine alkaloids—more AERs than for any

other single dietary supplement. However, the AERs that we examined

often lacked important information, and the information that they did

contain was sometimes inconsistent. These problems suggest that AERs

should be used with caution, and their use can contribute to uncertainty in

FDA’s conclusions.

In examining a random sample of the AERs (92 out of 864 reports), we

found that 39 percent lacked information on the amount of product

consumed, 41 percent lacked information on the frequency with which the

product was consumed, and 28 percent lacked information on the duration

for which the product was consumed. A total of 45 percent of the AERs

lacked information on either dose, frequency, or duration, and 24 percent

lacked information on all three dimensions. Finally, 62 percent of the AERs

in our sample did not contain medical records, which are important in

determining potential underlying conditions that might have caused the

adverse event (rather than assuming ingestion of dietary supplements

containing ephedrine alkaloids caused the event).

We also found cases where the amount of product consumed or the

duration for which it was consumed was listed differently in multiple

locations within an AER. In addition, the name of the product consumed

sometimes varied within an AER. As a result, it is difficult to identify the

26FDA evaluated the scientific literature to determine that ephedrine alkaloids could pose hazards at 20

mg per dose. However, FDA relied solely on AERs in setting a specific dose level of 8 mg per dose and

24 mg per day.

Page 11 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

correct information that corresponds to the event being reported and to

make inferences from such reports.27

FDA Relied Solely on AERs for

Setting Specific Dosing

Regimens and Did Not

Determine If These Events

Were Caused by Ephedrine

Alkaloids

In its proposed rule, FDA concluded that one possible strategy for

addressing these adverse events was to restrict the level of ephedrine

alkaloids in dietary supplements. To determine a possible dosing level, FDA

reviewed clinical trials on therapeutic uses for ephedrine alkaloids used

alone and in combination with other pharmaceutical substances to treat

obesity. These trials indicated that 20 mg ephedrine alkaloids per dose

could cause adverse events to occur in the subjects taking part in the

clinical trial.28 Thus, FDA concluded that 20 mg per serving of ephedrine

alkaloids presented potential risks for some individuals.

However, the studies did not provide information on risk at levels below

20 mg per serving for use in the general population, and lean or

moderately overweight persons have been shown to be more sensitive to

substances like ephedrine alkaloids than are obese individuals. To

determine if serious adverse events were occurring at levels below 20 mg

per serving, FDA obtained and tested the dietary supplement products

associated with AERs that were serious, where available.29 FDA then

identified the levels of ephedrine alkaloids in those AERs by performing

laboratory analyses of ephedrine alkaloid levels for dietary supplements

turned over to the agency by the consumers who suffered adverse events

(consumer samples). FDA also performed tests on samples obtained from

the marketplace for situations in which the consumer experiencing the

adverse event no longer possessed the product (consumer-related

samples). In all, from more than 800 AERs submitted to the agency between

1993 and the time of the proposed rule, FDA collected and tested 34

samples of consumer or consumer-related products.

From the 34 samples tested, 13 product samples met two criteria: the

results of the analytical tests were valid and supportable and the products

contained less than 20 mg of ephedrine alkaloids. FDA then set a specific

dosing level of 8 mg per serving. In other words, FDA relied on these 13

AERs where the products tested yielded ephedrine alkaloid levels below 20

mg per serving as the sole source of support for the specific dosing level

27While these types of problems do not occur uniquely among reports involving ephedrine alkaloids,

they reinforce the fact that there are inherent weaknesses of passive surveillance systems such as

AERs.

28See, for example, A. Astrup and others, “The Effect and Safety of an Ephedrine/Caffeine Compound

Compared to Ephedrine, Caffeine and Placebo in Obese Subjects on an Energy Restricted Diet. A

Double Blind Trial,” International Journal of Obesity, Vol. 16 (1992), pp. 269-77.

29Testing was necessary because there are no requirements that product contents be labeled.

Page 12 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

found in the proposed rule.30 Table 1 provides information on these 13

cases.

Table 1: Summary of 13 AERs Used for Specific Dosing Level

Total ephedrine

alkaloids (mg)a

AER number Initial Recheck Type of sample Symptoms as described in AER

10088 1.6 2.0 Consumer Elevated blood pressure, severe headache, nausea, heavy perspiration,

passed out

11233 1.8 Trace Consumer Hair loss, sleeplessness, chest pain, increased energy

11106 Trace 1.9 Consumer Sweating, trembling, high blood pressure, menstrual bleeding, suffered

stroke

9754 8.8 8.8 Consumer Heat stroke, chest and back pain, rapid heart rate, hyperthermia

8475 9.1 9.9 Consumer-related Tremors, tics, insomnia, hyperactive reflexes

10974 11.0 13.1 Consumer-related Seizure

11144b 10.0, 11.7 9.0, 15.3 Consumer-related Transient amnesia, lost consciousness

10946 10.5 8.3 Consumer Rash, increased blood pressure

9751 9.2 11.1 Consumer Attempted suicide

10919 11.2 14.4 Consumer Weakness, dizziness, elevated blood pressure

11134c 14.8 Consumer Death

11619 15.1 18.5 Consumer Rapid heart rate, headache, numbness, face droop, dizziness

11298 16.6 20.6 Consumer Blood in urine, elevated blood pressure

aLevels are based on the amount the consumer reported having used per serving. Where

consumer intake was unknown, the levels were based on the label directions for use. FDA

performed an initial analysis and a recheck analysis for the samples it obtained.

bTwo consumer-related samples were tested.

cRecheck analysis was not conducted because original analysis was performed by a national or

international expert.

While FDA used these 13 AERs to set a dosing level, the agency did not

perform a causal analysis to determine whether the reported events were,

in fact, caused by the ingestion of dietary supplements containing

ephedrine alkaloids. Our review of these 13 AERs found numerous

problems that raise questions about the causal relationship between

ingestion of the implicated product and the adverse events observed. For

example,

30FDA’s laboratory analysis of the ephedrine alkaloid levels in the 34 AERs showed levels from

approximately 1 to 50 mg per serving. According to FDA, these reports show a pattern of clinically

significant adverse events at levels approaching and above 10 mg. Given the variability in the testing

procedure and natural variability in the alkaloid content of botanical ingredients, FDA determined a

range around 10 mg per serving could be expected to deviate by 10 to 20 percent. From this, FDA

tentatively concluded that an 8 mg limit per serving could be associated with a serious adverse event.

Page 13 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

• three AERs included physician reports that stated the cause of the event

was not related to a dietary supplement;

• one consumer-related sample was obtained and tested 2 years after the

initial event, and possible reformulations of the product might have

resulted in different levels of ephedrine than the product implicated in the

reported adverse event;

• three individuals reporting adverse events had experienced similar

problems prior to or well after using the dietary supplement;

• one individual who experienced the event was eating only one meal a day;

and

• one AER contained no medical records.

Some of the 13 AERs had more than one of these problems. As a result,

there are uncertainties in FDA’s conclusions in setting a specific dosing

limit since this limit was based on a small number of adverse

events—events which may or may not have been a result of ingestion of

dietary supplements containing ephedrine alkaloids.

Weak Support for FDA’s

Proposed Limitation on

Duration of Use

In its proposed rule, FDA also recommended a 7-day limit on the use of

dietary supplements containing ephedrine alkaloids. FDA used the

recommendations of some members from its special work group and Food

Advisory Committee, scientific literature, and data from the AERs to

support this limit. However, we found several weaknesses in FDA’s

evidence.

First, FDA did not present scientific evidence specifically pointing to an

increase in adverse events beginning at 7 days and under normal use

conditions. Rather, the scientific information FDA used to support a 7-day

limit outlined problems associated with extended use (months and years)

of ephedrine alkaloids. The agency also cited support for its 7-day limit

from studies involving other sympathomimetic agents, such as cocaine and

methamphetamines, but these studies also involved long-term use of the

drug. Second, FDA did not demonstrate a causal relationship between

ingestion of dietary supplements containing ephedrine alkaloids and

adverse events reported to the agency. Since FDA indicated in the proposed

rule that 10 to 73 percent of the reported adverse events might not be

related to consumption of dietary supplements containing ephedrine

alkaloids, the use of the AERs to describe a pattern of response across time

Page 14 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

is questionable.31 Finally, the agency did not demonstrate the relationship

between both dose levels and frequency of ingestion and duration of use.

As a result, there are uncertainties in the agency’s analysis of the

relationship between duration of use of dietary supplements containing

ephedrine alkaloids and the occurrence of adverse events.

FDA Based Its Risk Reduction

Value on Poorly Documented

Estimates of Serious Adverse

Events

FDA noted in the proposed rule that there were an average of 174 adverse

events reported each year between January 1993 and June 1996 and that 81

were serious events. FDA divided the serious adverse events into five

categories and outlined examples for each type: (1) cardiovascular

(dysrhythmia or abnormal heart rhythms, severe hypertension, cardiac

arrest, heart attack, angina, myocardial infarction, stroke, and

cardiomyopathy or muscle damage to heart); (2) central nervous system

(psychosis, suicide or attempted suicide, altered or loss of consciousness

or disorientation or confusion, seizure, mania, and severe depression);

(3) liver damage or hepatitis; (4) death; and (5) other events (vestibular or

inner ear disturbances, altered serum enzymes, myopathies or muscle

disease, genitourinary system disturbances or urinary tract problems,

prostate problems, and dermatological syndromes or rashes).

However, FDA did not record which AERs it considered to be serious. FDA

officials stated that a count had been performed by medical doctors of the

agency on the number of serious events, but there was no record of the

specific cases that they had identified as serious events. In addition, rather

than providing AER reviewers a standard definition, FDA told them to use

their clinical judgment in determining what they considered a serious

event.32 Also, FDA did not perform any assessments of interrater reliability

among reviewers to ensure that their judgments were consistent.

Based on the yearly average of serious adverse events reported to FDA

from January 1993 to June 1996, FDA estimated an annual value of risk

reduction that would occur as a result of the proposed actions outlined in

the rule. However, because FDA did not record which AERs were deemed

31FDA pointed out in the proposed rule that the relationship of the reported adverse event to the

consumption of dietary supplements categorized as containing ephedrine alkaloids had been

corroborated in about 27 percent of cases where a consumer stopped taking a product and the

symptoms improved. FDA also noted that a certain number of false reports might also be expected.

Thus, they believed that the actual percentage of cases related to consumption of ephedrine alkaloids

was between 27 percent and 90 percent. FDA’s professional judgment was that 80 percent of the

reported cases were actually related to consumption of dietary supplements containing ephedrine

alkaloids.

32FDA noted that the medical doctors were well qualified—by training and experience—to make

decisions regarding the clinical significance of the event.

Page 15 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

serious, we could not determine the accuracy of FDA’s conclusions

regarding the actual benefits that would arise from the proposed rule.

FDA’s Analysis of

Impacts Generally

Adhered to

Rulemaking

Requirements but Was

Not Fully Transparent

FDA generally complied with executive order and statutory requirements

for rulemaking. It prepared a cost-benefit analysis containing the basic

elements expected for an economically significant rule under E.O. 12866.

It also prepared an initial regulatory flexibility analysis and provided

opportunities for small businesses to participate in the rulemaking

process, as required by RFA. FDA’s analysis of impacts would also likely

satisfy most, if not all, of the analytical requirements of UMRA, although its

proposed rule does not appear to trigger these requirements.

However, OIRA’s guidance on preparing cost-benefit analyses under E.O.

12866 states that an agency’s analysis must be “transparent”—that is,

disclose how the analyses were conducted and what assumptions were

used. FDA’s analysis was only partially transparent. Although FDA described

its key assumptions and identified substantial uncertainties regarding the

data supporting its proposed rule, it did not fully disclose why certain key

assumptions were made, the degree of uncertainty involved in those

assumptions, or that alternative assumptions would have had a dramatic

effect on the agency’s estimates of the costs and benefits of the proposed

regulatory actions. Because of these uncertainties and limitations, the

results of FDA’s analysis are still open to question.

FDA’s Analysis Addressed

the Basic Cost-Benefit

Elements but Was Only

Partially Transparent

Under E.O. 12866, agencies are required to conduct cost-benefit analyses

for all regulatory actions that are likely to result in a $100 million annual

effect on the economy or are otherwise economically significant. OIRA’s

“best practices” guidance states that a federal agency’s cost-benefit

analysis should contain three basic elements: (1) a statement of the need

for the proposed action, (2) an examination of alternative approaches, and

(3) an analysis of benefits and costs. Although the guidance provides for

flexibility and professional judgment in conducting the analysis, it

establishes a clear expectation that the analysis of the risks, benefits, and

costs associated with the regulation “must be guided by the principles of

full disclosure and transparency.” For example, the guidance says the

analysis should identify and explain the data or studies on which the

estimates of benefits and costs are based “with enough detail to permit

independent assessment and verification of the results.” The guidance

also states “where benefit or cost estimates are heavily dependent on

certain assumptions, it is essential to make those assumptions explicit

Page 16 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

and, where alternative assumptions are plausible, to carry out sensitivity

analyses based on the alternative assumptions.”33 (See app. IV for a more

detailed discussion of the analytical requirements under E.O. 12866 and

related guidance, as well as our assessment of FDA’s cost-benefit analysis

in terms of that guidance.)

FDA’s cost-benefit analysis contained the three basic elements stipulated in

the guidance, but certain other specific elements in the guidance appear to

be lacking:

• With regard to the first element, FDA stated that the proposed rule was

needed because of a significant market failure—specifically, inadequate

consumer information on the health risks associated with dietary

supplements containing ephedrine alkaloids. However, the OIRA guidance

indicates that even where a market failure exists, agencies should also

discuss the appropriateness of alternatives to federal regulation, such as

state or local regulation. FDA’s analysis did not recognize efforts in several

states to regulate these products or say why federal regulation was a

better approach.

• For the second element, FDA discussed seven regulatory options, including

(1) a baseline alternative of taking no action, (2) taking no action but

generating additional information, (3) taking the proposed action,

(4) taking the proposed action but allowing a higher potency limit for the

supplements, (5) banning dietary supplements that contain ephedrine

alkaloids, (6) taking the proposed action but not requiring the warning

statement, and (7) requiring only the warning statement. FDA also generally

discussed the agency’s reasoning for selecting the proposed regulation

over other alternatives. However, FDA’s options primarily focused on

alternative levels of stringency and informational measures—not on other

basic types of regulatory alternatives suggested in the guidance. Also,

while FDA included some contextual information in the proposed rule, it

did not provide a complete picture of the baseline risks that may be

associated with supplements containing ephedrine alkaloids.

• For the actual analysis of potential benefits and costs—the third

element—FDA generally adhered to the principles and specific

recommendations in OIRA’s guidance. For example, FDA discussed the

distributional effects of the proposed rule and alternatives in terms of lost

sales to the dietary supplement industry. FDA also identified nonmonetized

costs associated with the proposed rule. However, FDA did not provide

monetized or quantified benefit or cost estimates for all of its regulatory

33To conduct a sensitivity analysis, the analyst calculates the costs and benefits of a proposed action

using different assumptions.

Page 17 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

alternatives, and the agency’s discussion of those alternatives’ benefits and

costs were compared to the proposed rule—not the baseline condition of

no regulation as recommended by the guidance.

The OIRA guidance also stresses the importance of full disclosure and

transparency in agencies’ cost-benefit analyses. We assessed the

transparency of FDA’s cost-benefit analysis of the proposed rule using three

criteria suggested by the guidance:

• first, whether it identified the data, models, inferences, and assumptions

used to calculate the estimates of benefits and costs;

• second, whether it disclosed the reasons why those data, models,

inferences, and assumptions were selected; and

• third, whether it assessed the effects of plausible alternative assumptions

and choices on the results of the analysis—what is often referred to as a

“sensitivity analysis.”

Overall, we concluded that FDA’s analysis was only partially transparent.

Using the first criterion, FDA’s analysis was very transparent. The agency

provided a clear and lengthy description of the data, assumptions, and

methodology that it used to calculate the benefits and costs of the

proposed rule. Against the second criterion, FDA’s analysis was only

partially transparent. For most elements of the analysis, FDA identified the

underlying data sources used and the rationale for the assumptions and

conclusions reached by FDA’s analysts and experts. However, FDA did not

fully disclose the underpinnings of all of its assumptions and choices. For

example, FDA said that between 27 percent and 90 percent of adverse event

reports were “probably” related to the consumption of dietary

supplements suspected of containing ephedrine alkaloids and that it

assumed the value was 80 percent, but it did not indicate why it made this

point-estimate assumption. In response to our questions, FDA officials

acknowledged that their analysis of impacts was not as transparent as it

should have been in explaining how the agency arrived at some of the

assumptions regarding its treatment of uncertainty in the underlying data.

Against the third criterion, FDA’s cost-benefit analysis was also only

somewhat transparent. For example, FDA estimated that the benefits of the

proposed rule would be between $240 million and $670 million per year.

That estimate was driven by three factors: (1) FDA’s estimate of the actual

number of adverse events each year (1,110), (2) FDA’s estimate of the

degree to which the proposed rule would reduce these events (35 percent

to 100 percent across all types of proposed actions), and (3) the values FDA

Page 18 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

assigned to the estimated risk reduction per case (for example, $5 million

per death avoided).34 Changes in any of these values could have

dramatically changed FDA’s estimates of the proposed rule’s benefits. For

example, FDA’s estimate of 1,110 adverse events each year was based on an

average of 174 adverse events reported per year and three assumptions:

(1) that 80 percent of the adverse event reports involved consumption of

dietary supplements suspected of containing ephedrine alkaloids, (2) that

80 percent of the supplements actually contained ephedrine alkaloids, and

(3) that only 10 percent of all adverse events related to supplements

containing ephedrine alkaloids were reported. Using these point estimates

(instead of ranges) does not reflect the uncertainty FDA indicated was

possible regarding these values and, therefore, the uncertainty associated

with the agency’s final benefit estimates.

Had FDA used its own initial estimates of the possible ranges (instead of

the 80-percent point estimates) for the first two assumptions (27 to 90

percent for the first assumption and 25 to 90 percent for the second

assumption), its estimate of the number of adverse events each year would

have been 117 to 1,409 instead of a single estimate of 1,110. (See table 2.) If

FDA’s projected effects for the proposed actions were applied to this range,

the expected reduction in annual adverse events would be between 41 and

1,409 cases per year (not the 390 to 1,110 in the published analysis). This

ultimately results in a range of potential benefits of roughly $25 million to

$850 million per year—a much wider range of possible benefits than the

$240 million to $670 million per year estimates that FDA included in the

proposed rule.

34In its proposed rule, FDA did not report or round its estimated values consistently, so slightly

different numbers for the same element appear within the proposed rule. For this report, we are using

the values reported in table 6 of FDA’s proposed rule, such as 1,110 for the estimated annual number

of adverse events.

Page 19 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

Table 2: FDA’s Estimate of Expected Benefits Is Sensitive to Changes in Key Assumptions

A B C D E (AxBxCxD) F G H (FxG)

Assumptions made regarding uncertainty in the actual

number of adverse event cases

Average

annual

number of

reported

adverse

events

Proportion of

cases 

actually 

related 

to dietary

supplements

Proportion 

in which the 

supplements 

actually 

contained 

ephedrine alkaloids

Multiplier 

to reflect

underreporting 

of 

adverse 

eventsa

Estimated 

total 

number 

of adverse 

event 

cases

Estimated 

reduction 

in the 

number of 

annual 

casesb

Value of

estimated

risk

reduction

per 

case

Total 

dollar value 

(in millions) 

of estimated 

risk 

reduction

Scenario 1: FDA’s reported results, using a point estimate for total number of adverse events.c

174 0.8 0.8 10 1,110 390-1,110 d $240-670

Scenario 2: Results using the full range of FDA’s assumptions regarding uncertainty on number of adverse events.

174 0.27-0.9 0.25-0.9 10 117-1,409 41-1,409 d 25-850

Scenario 3: Results using assumption that 20 percent of cases are reported.

174 0.8 0.8 5 557 195-557 d 118-336

Scenario 4: Results using assumption that 5 percent of cases are reported.

174 0.8 0.8 20 2,227 779-2,227 d 470-1,343

aFor example, if the assumption is that 10 percent of events are reported, the number of reported

cases is multiplied by 10.

bThe combined effects of all proposed actions are assumed to reduce adverse events by

between 35 percent and 100 percent.

cFDA’s table in the proposed rule on the estimation of benefits displayed rounded numbers. The

actual calculated values may vary from the published figures used here. For example, if the figure

for estimated total annual adverse events were not rounded, it would equal about 1,114 rather

than 1,110.

dThe value of risk reduction varies by the type of event avoided, such as $5 million per death

avoided and $837,000 for each serious cardiovascular system event avoided. FDA converted the

individual values and estimated totals into 1996 dollars. Calculations for scenarios 2, 3, and 4

assume the same proportions of types of adverse events as FDA used for its estimate.

Source: Scenario 1 data are taken from table 6 of FDA’s proposed rule. Data for other scenarios

were GAO calculations using information in FDA’s proposal.

Table 2 also shows that changes in the third assumption yield similarly

dramatic changes in the estimated benefits. FDA indicated in its proposed

rule that reporting rates might be higher than 10 percent if, for example,

the potential health risks were widely publicized, or lower than 10 percent

if consumers and physicians assumed that dietary supplements are

incapable of producing adverse events. If, keeping all other factors

constant, FDA had assumed that 20 percent of serious adverse events were

reported, FDA’s benefits estimate would have been reduced by about half.

Page 20 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

Conversely, if FDA had assumed that only 5 percent of events were being

reported, the benefits estimate would have doubled.

FDA Met RFA

Requirements, Although

Questions Remain

Concerning Impacts on

Industry

RFA requires agencies to consider the effects of their rules on small entities

and to take certain actions during the rulemaking process. For example,

before publishing a proposed rule for which a notice of proposed

rulemaking is required, sections 603 and 605(b) of RFA require a federal

agency to prepare and make available for public comment an initial

regulatory flexibility analysis that describes the anticipated effects of the

proposed rules on small entities, unless the head of the agency certifies

that the rule will not have a “significant economic impact on a substantial

number of small entities.” (See app. V for a more detailed discussion of

RFA requirements and our analysis of the actions FDA took to comply with

RFA.)

FDA determined that its proposed rule on ephedrine alkaloids would have a

significant economic impact on a substantial number of small entities and

prepared an initial regulatory flexibility analysis to identify those impacts.

FDA’s proposed rule addressed the basic elements that RFA requires

agencies to include in an initial regulatory flexibility analysis. For

example, the rule describes the reasons the agency was considering the

action, states the purpose and legal basis of the rule, describes and

provides an estimate of the number of small entities to which FDA believed

the rule would apply, and describes the compliance requirements. In

addition to describing direct compliance costs of between $3 million and

$80 million, FDA said that the proposed rule could have significant

distributive effects in the form of reduced sales of as much as $230 million

a year. FDA explicitly stated that costs and sales reductions of this

magnitude might threaten the viability of many firms. FDA also discussed

significant regulatory alternatives in the rule’s regulatory flexibility

analysis section, noting that most of the regulatory alternatives discussed

in the cost-benefit analysis section would reduce the impact of the rule on

small businesses.

When a rule is promulgated that will have a significant economic impact

on a substantial number of small entities, section 609(a) of RFA further

requires agencies to ensure that small entities have been given an

opportunity to participate in the rulemaking process through the

“reasonable use” of outreach efforts. Our review of the regulatory docket

for this rulemaking, as well as information obtained during interviews with

dietary supplement industry representatives, indicated that FDA provided

Page 21 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

opportunities for small business participation during this rulemaking

process. For example, in addition to the proposed rule itself, FDA published

public notices; held public meetings, during which industry

representatives participated and provided testimony; and collected written

comments on the issue. Representatives of industry trade associations also

told us that FDA had placed notices about the proposed rule in trade

literature.

However, SBA’s Office of Advocacy criticized the quality of FDA’s regulatory

flexibility analysis. Along with generally criticizing FDA’s scientific analysis

for the proposed rule, the office contended that FDA did not consider the

large numbers of independent distributors of dietary supplements in its

analysis and, as a result, underestimated the number of affected small

businesses and the impacts of the proposed actions. In response to SBA’s

criticism, FDA officials told us that the agency did not need to consider the

impact of its proposed rule on distributors of ephedrine alkaloids. FDA

officials cited several court decisions that support the proposition that,

under RFA, an agency is under no obligation to conduct a small entity

impact analysis of effects on entities that the agency does not

regulate—that is, distributors.35

None of the major parties—FDA, SBA’s Office of Advocacy, or industry

associations—have information on the actual number of entities involved

in the market for dietary supplements containing ephedrine alkaloids. All

three parties acknowledge this as a limitation in attempts to analyze the

effects of the proposed rule. According to FDA, it has no authority to

require companies in the dietary supplement industry either to register

with or contact the agency unless they are seeking approval to make a

claim in product labeling. However, the industry representatives we met

with confirmed that manufacturing and labeling of these products are

generally limited to a relatively small number of manufacturing firms. Such

firms would bear almost all of the direct expenditures required for

compliance with the proposed rule. SBA’s Office of Advocacy

recommended that FDA develop an outreach strategy to obtain more

reliable industry data, and FDA officials said that they have contracted for a

marketing study of the dietary supplement industry that should provide

better information for future FDA analyses.

35For example, United Distribution Cos. v. FERC, 88 F.3d 1105, 1170 (D.C. Cir. 1996).

Page 22 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

Proposed Rule Does Not

Appear to Trigger UMRA

Requirements

UMRA generally requires covered agencies to prepare specific types of

analyses for certain rules that include a federal mandate and that may

result in the expenditure in any 1 year of $100 million or more by the

private sector.36 FDA did not explicitly address UMRA in its proposed rule on

ephedrine alkaloids because its economic analysis indicated that direct

expenditures imposed on the private sector would not rise to the level

requiring additional analysis under UMRA. FDA estimated that the total

compliance costs for the proposed action would be between $3 million

and $80 million, with, at most, $70 million of those costs in the first year of

implementation. Although FDA also estimated that the rule might result in

lost sales for the dietary supplement industry of as much as $230 million

per year, lost sales cannot be considered direct expenditures by the

private sector and, therefore, cannot be used to trigger UMRA’s analytical

requirements. (See app. VI for a more detailed discussion of UMRA

requirements and our analysis of FDA’s compliance.)

However, SBA’s Office of Advocacy contended that FDA overlooked the

UMRA requirements and commented that if the agency had properly

estimated the number of affected businesses and the costs that would be

imposed on those entities, “it would have been apparent that the economic

impact of the instant rule would impose in excess of $100 million in costs

to the industry,” triggering UMRA requirements. However, the Office of

Advocacy does not have data on the number of entities involved in the

market for dietary supplements containing ephedrine alkaloids that it

believes should have been included in FDA’s analysis. Therefore, the Office

of Advocacy could not demonstrate that FDA had underestimated the

number of manufacturers that would be affected by the relabeling and

reformulation requirements in the rule or that FDA’s cost estimates for

those manufacturers were in error. Furthermore, even if the expenditures

associated with the proposed rule had triggered UMRA’s analytical

requirements, FDA appears to have satisfied most, if not all, of those

requirements. For example, FDA quantitatively and qualitatively assessed

the anticipated costs and benefits of the rule. FDA also identified and

considered a number of regulatory alternatives, indicating that the other

alternatives would not be as effective as the proposed rule.

Conclusions FDA was justified in determining that the number of AERs relating to dietary

supplements containing ephedrine alkaloids warranted their attention and

consideration of steps to address safety concerns. The available scientific

information suggests that the use of products containing synthetic

36The statute provides for annual adjustments for inflation.

Page 23 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

ephedrine alkaloids can result in adverse experiences for some

individuals, and over-the-counter products containing ephedrine alkaloids

have dosing recommendations. Furthermore, dietary supplement trade

associations have suggested specific dosing limits for dietary supplements

containing ephedrine alkaloids.

However, while FDA’s conclusions regarding the desirability of the

proposed actions may be valid, we believe these conclusions are open to

question because of limitations and uncertainties associated with the

agency’s underlying scientific and economic analyses. We have concerns

about the strength of the information upon which FDA based specific

elements of its proposed rule. There is no scientific information on the

specific dosing levels and duration limits proposed. FDA, therefore, relied

heavily on its AERs to determine a dosing regimen and to outline benefits

that would accrue from the proposed rule. However, the AERs suffer from

several problems that weaken the conclusions drawn by FDA for their

specific dosing regimen; the number of AERs used to support the dosing

regimen is small, their quality is questionable, and FDA did not establish a

causal link between the ingestion of ephedrine alkaloids and the

occurrence of particular adverse events. Finally, because FDA did not

document which AERs it identified as involving serious adverse events, it is

impossible to verify FDA’s calculation of the number of these events and

the accuracy of the benefits that FDA estimated would occur as a result of

the proposed rule.

FDA’s analysis contained the basic elements expected in a federal agency’s

cost-benefit analysis, and the proposed rule complied with rulemaking

requirements under RFA. The proposed rule does not appear to trigger the

UMRA analytical requirements. However, FDA’s cost-benefit analysis was not

always transparent regarding why certain key assumptions were made, the

degree of uncertainty involved in those assumptions, or the effect that

alternative assumptions would have had on the agency’s estimates of the

costs and benefits of the proposed action.

Recommendations Given the uncertainties in the information upon which FDA based its

proposed rule, we recommend that the Secretary of Health and Human

Services direct the Commissioner of FDA to obtain additional information

to support conclusions regarding the specific requirements in the

proposed rule for dietary supplements containing ephedrine alkaloids

before proceeding to final rulemaking. Specifically, FDA needs to provide

stronger evidence on the relationship between the intake of dietary

Page 24 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

supplements containing ephedrine alkaloids and the occurrence of

adverse reactions that support the proposed dosing levels and duration of

use limits.

We also recommend that the Secretary direct the Commissioner to

improve the transparency of FDA’s cost-benefit analysis in its final

rulemaking. Specifically, FDA should more fully explain the bases of its

cost-benefit assumptions, the degree of uncertainty associated with those

assumptions, and the implications of plausible alternative assumptions to

the proposed action and other regulatory alternatives.

Agency and Other

Reviewer Comments

We sent a draft of this report to the Commissioner of FDA, the

Administrator of SBA, and the Director of OMB. FDA and SBA’s Office of

Advocacy provided written comments, which are reprinted in appendixes

VII and VIII; OMB did not have comments on the report.

In its comments, FDA concurred with our recommendation that it obtain

additional information to support its conclusions on specific requirements

relating to dietary supplements containing ephedrine alkaloids before

proceeding to final rulemaking. FDA also concurred with our

recommendation to improve the transparency of the agency’s cost-benefit

analysis, noting that it intends to take appropriate steps to correct the

deficiencies of the analysis prior to publication of a final rule. However,

FDA was concerned that our report (1) did not sufficiently highlight the

agency’s justification in examining safety concerns related to dietary

supplements containing ephedrine alkaloids, (2) implied that FDA relied

principally on AERs to develop the proposed rule, (3) overemphasized the

agency’s failure to conduct a causal analysis linking dietary supplements

containing ephedrine alkaloids to the AERs, and (4) was not clear on

whether it was appropriate to use AERs in rulemaking.

Throughout our report, we express the clear view that given the number

and type of AERs, it was reasonable for FDA to investigate the safety of the

supplements. We also acknowledge that FDA’s actions were based on

information from scientific literature as well as AERs. However, although

clinical trials suggest that adverse events could occur in some individuals

using ephedrine alkaloids at levels approaching 20 mg per serving, they did

not provide information relating to adverse events below the 20 mg level.

Therefore, to conclude that an 8 mg per serving level would be

appropriate, FDA turned to results of tests it performed on a small number

of products implicated in 13 AERs. In other words, information from these

Page 25 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

AERs was the sole source of support for the specific dosing level of 8 mg

per serving that FDA proposed. Finally, we continue to hold the view that

without a causal link between the AERs and the ingestion of products

containing ephedrine alkaloids, the exclusive use of AERs to support a

specific dosing regimen is questionable. FDA also asked that we clarify that

using AERs to develop a proposed rule of this type is neither inappropriate

nor unscientific. For this report, we examined whether the use of AERs in

the proposed rule on dietary supplements containing ephedrine alkaloids

was consistent with the use of AERs in prior rulemaking; however, we did

not take a position on the appropriateness of the general use of AERs in

rulemaking.

SBA’s Office of Advocacy stated that our recommendations are reasonable

and would result in a more rational—and possibly less burdensome—

regulation. However, the officials questioned how we could conclude that

FDA had complied with RFA while at the same time suggesting that the

agency had prepared an “inadequate analysis” by using unreliable data and

not clearly demonstrating that the benefits of the proposed rule exceed the

cost. Although our report raises concerns about several aspects of FDA’s

analysis, we did not conclude that FDA prepared an inadequate analysis.

Rather, we concluded that FDA’s proposed rule contained the primary

elements required under E.O. 12866 for a cost-benefit analysis and under

RFA for an initial regulatory flexibility analysis. Furthermore, RFA contains

no standards or criteria that define an “adequate” regulatory flexibility

analysis. As noted in SBA’s comments, section 607 of RFA notes that

agencies may provide “general descriptive statements” to comply with

the act’s analytical requirements if quantification is not practicable or

reliable. Determining whether a regulatory flexibility analysis was in

compliance with RFA would require analysis and judgments concerning the

totality of the circumstances relating to the specific regulation in question.

We have no basis and are not in a position to make such a determination

regarding the quality of FDA’s analysis, nor does our report do so.

SBA’s Office of Advocacy also expressed its concern that FDA’s analysis did

not identify the indirect effects of the rule on distributors. Office of

Advocacy officials said that, although certain court decisions stemming

from Mid-Tex Electric Cooperative, Inc. v. FERC37 indicated that agencies

need not prepare regulatory flexibility analyses if the effects of a rule on

an industry are indirect, they disagreed with those interpretations and

contended that agencies should be required to conduct analyses of

indirect effects. They also stated that the Office of Advocacy was unaware

37773 F.2d 327 (D.C. Cir. 1985).

Page 26 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

of any comparable case law with respect to direct and indirect effects as

they apply to E.O. 12866, so they said FDA should have prepared an analysis

of the rule’s effects on distributors. We disagree with these comments for

three reasons. First, as the Office of Advocacy officials acknowledged,

prior case law indicates that agencies need not take the indirect effects of

their rules into account when conducting a regulatory flexibility analysis.

That interpretation was recently reaffirmed in a May 14, 1999, decision by

the U.S. Court of Appeals for the District of Columbia Circuit.38 Second,

the absence of case law comparable to Mid-Tex with respect to direct and

indirect effects as they apply to E.O. 12866 says nothing about FDA’s

analytical obligations under the order. Third, our review shows that FDA

did address the potential indirect effects of the rule through its estimates

of total lost sales for the dietary supplement industry. FDA also explicitly

acknowledged in the proposed rule that such lost sales are “obviously

very significant to the affected parties” and might threaten the viability of

many firms in this industry.

We also obtained comments on a draft of this report from a professor of

pharmacology with expertise on dietary supplements. He agreed that

while there is adequate reason for FDA to be concerned about the safety of

products containing ephedrine alkaloids, he believed that there are too

few AERs of substantive quality to allow for the setting of a maximum safe

dose. He also concurred with our conclusions and recommendations.

FDA provided technical comments, which we incorporated as appropriate.

38American Trucking Associations, Inc., et al, v. United States Environmental Protection Agency, No.

97-1440 U.S. App. LEXIS 9064 (D.C. Cir. May 14, 1999).

Page 27 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
B-281581 

As agreed with your office, unless you publicly announce its contents

earlier, we plan no further distribution of this report until 30 days from the

date of this letter. At that time, we will send copies to the Honorable

Donna E. Shalala, Secretary of Health and Human Services; the Honorable

Jane E. Henney, Commissioner of FDA; the Honorable Aida Alvarez,

Administrator of SBA; the Honorable Jacob J. Lew, Director of OMB; and

others who are interested. We will also provide copies to others upon

request. GAO contacts and major contributors to this report are listed in

appendix IX.

William J. Scanlon

Director of Health Financing

 and Public Health Issues

L. Nye Stevens

Director of Federal Management

 and Workforce Issues

Page 28 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Page 29 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Contents

Letter 1

Appendix I 

Scope and

Methodology

32

Appendix II 

Adverse Event

Reporting

34

Appendix III 

FDA’s Prior Use of

AERs in Rulemaking

37

Appendix IV 

FDA’s Analysis of

Benefits and Costs

42

Appendix V 

Regulatory Flexibility

Act Requirements

56

Appendix VI 

Unfunded Mandates

Reform Act

Requirements

61

Appendix VII 

Comments From the

Department of Health

and Human Services

63

Page 30 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Contents

Appendix VIII 

Comments From the

Small Business

Administration

72

Appendix IX 

GAO Contacts and

Staff

Acknowledgments

79

Tables Table 1: Summary of 13 AERs Used for Specific Dosing Level 13

Table 2: FDA’s Estimate of Expected Benefits Is Sensitive to

Changes in Key Assumptions

20

Table III.1: Five Prior FDA Rulemaking Cases 37

Figure Figure IV.1: Estimation of Benefits Is Sensitive to Changes in

Underlying Assumptions

53

Abbreviations

AEMS adverse event monitoring system

AER adverse event report

CDC Centers for Disease Control and Prevention

CFSAN Center for Food Safety and Applied Nutrition

CPSC Consumer Product Safety Commission

EPA Environmental Protection Agency

FDA Food and Drug Administration

GRAS generally recognized as safe

OIRA Office of Information and Regulatory Affairs

OMB Office of Management and Budget

OSN Office of Special Nutritionals

RFA Regulatory Flexibility Act

SBA Small Business Administration

UMRA Unfunded Mandates Reform Act

Page 31 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix I 

Scope and Methodology

To determine whether the reported cardiovascular and nervous system

effects outlined in the proposed rule were relevant for dietary

supplements containing ephedrine alkaloids and whether there was

evidence that these ingredients could cause adverse experiences in some

individuals, we examined the scientific literature for case reports of

adverse events from ingestion of products containing ephedrine alkaloids

and reviewed findings from clinical trials using ephedrine alkaloids to

treat obesity. To determine whether the Food and Drug Administration

(FDA) had any internal guidance on the use of adverse events reports (AER)

in rulemaking and whether these reports had been used in prior

rulemaking, we sought information from the agency. We examined five

prior cases of rulemaking that FDA provided in which the Center for Food

Safety and Applied Nutrition (CFSAN) had used AERs as part of the basis for

conclusions that were drawn in a proposed rule to ascertain how the

agency had previously used AERs in rulemaking.

In examining the basis for FDA’s rule, we evaluated information contained

in a sample of AERs and FDA’s reliance on them to support a dosing regimen

and to estimate benefits from the proposed rule. To identify criteria for the

quality of the AERs, we obtained information on the use of passive

surveillance systems, including their methodological strengths and

weaknesses. We also examined the scientific literature for studies on

adverse event reporting and outcomes from such reporting systems. We

performed a content analysis of a random sample of 92 out of 864 AERs

filed prior to the proposed rule to determine what information was

available in the AERs, such as dose, frequency of ingestion, duration of use,

type of product, the adverse effect reported, and whether a medical record

was included in the report.

We examined the list of 13 AERs used by FDA to set the dosing regimen

outlined in the proposed rule. We compared the analytical results of tests

performed on the products implicated in these AERs to determine the

dosing level ingested by the patient describing the adverse event. We

examined the results of these analytical tests to determine the variability

in the amount of ephedrine alkaloids that could be found in individual

bottles and, thus, the possible range in the amount of ephedrine alkaloids

ingested by a complainant. We sought documentation from FDA officials on

the AERs that were classed as serious, the numbers and types of each

category of serious event, and how those cases had been identified.

We reviewed the extent to which FDA’s cost-benefit analysis for the

proposed rule contained the elements expected of a federal agency

Page 32 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix I 

Scope and Methodology

analysis. To identify criteria for this objective, we reviewed the

cost-benefit requirements in E.O. 12866, “Regulatory Planning and

Review,” and a guidance document issued by the Administrator of the

Office of Information and Regulatory Affairs (OIRA) entitled “Economic

Analysis of Federal Regulations Under Executive Order 12866.” OIRA’s

guidance was intended to describe “best practices” for agencies in

preparing cost-benefit analyses of regulatory actions under the executive

order. Using these best practices, we reviewed FDA’s cost-benefit analysis

in the “analysis of impacts” section of the agency’s June 1997 proposed

rule. To obtain additional information, we interviewed FDA officials

involved in the development of the cost-benefit analysis and the proposed

rule. We also reviewed supporting documents referenced in the proposed

rule or included in the electronic docket for this rulemaking. As part of

that task, we examined each comment related to this rulemaking in the

electronic docket to identify substantive remarks on elements of FDA’s

analysis. We also interviewed officials at OIRA, the Small Business

Administration (SBA), and trade associations representing the dietary

supplement industry to obtain their views on FDA’s cost-benefit analysis.

We used a similar approach to describe FDA’s actions to comply with the

requirements of the Regulatory Flexibility Act (RFA) and the Unfunded

Mandates Reform Act (UMRA). We identified criteria by reviewing the

specific requirements in the statutes and then examined the relevant

sections of FDA’s proposed rule to compare the agency’s actions with those

requirements. As with the previous task, we supplemented our review of

the proposed rule by interviewing officials at FDA, SBA, and industry trade

associations. We also reviewed public comments submitted on the

proposed rule, including comments submitted by SBA’s Office of Advocacy

on FDA’s compliance with RFA and UMRA.

In our evaluation of FDA’s cost-benefit analysis and initial regulatory

flexibility analysis, we relied primarily on the information FDA presented in

the proposed rule or included in the regulatory docket. We did not perform

an independent assessment of the accuracy and reliability of FDA’s

underlying data and estimates. We also did not examine issues relating to

restrictions outlined in the proposed rule relating to ephedrine alkaloids

and their use with other stimulants and labeling restrictions.

Page 33 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix II 

Adverse Event Reporting

The clinical research and review staff in CFSAN’s Office of Special

Nutritionals (OSN) is responsible for tracking and reviewing AERs on special

nutritional products.39 Special nutritional products include dietary

supplements, infant formulas, medical foods, and foods for special dietary

use. The adverse event monitoring system (AEMS) used by OSN is a

voluntary surveillance system established to collect information on AERs

associated with the use of special nutritional products. Only AERs

associated with an illness or injury are included in AEMS; reports of product

quality or consumer dissatisfaction are not included. AERs may enter this

system through a variety of mechanisms, including FDA’s drug quality

reporting system, FDA’s Medical Products Reporting Program (Medwatch),

the United States Pharmacopeia, FDA field offices (through the Office of

Regulatory Affairs’ consumer complaint system), and from other federal

and state public health agencies. Written and electronic correspondence

and telephone conversations are other sources of information about

adverse events reported with the use of special nutritional products.

When an AER is received, it is reviewed to evaluate the need for further

follow-up, including medical record documents, label and labeling, and

other types of information necessary for evaluation of the AER. Follow-up

is routinely attempted on all AERs of serious events or when the event is

deemed to be clinically significant. MedWatch provides guidelines for

determining if an adverse event is serious for reporting purposes. Events

deemed serious include those that are fatal, life-threatening, or

permanently or significantly disabling; require or prolong hospitalization;

result in congenital anomalies; or require intervention to prevent

permanent impairment or damage.40 The proposed rule on ephedrine

alkaloids includes as serious cardiovascular events abnormal heart

rhythms, stroke, heart attack, and cardiomyopathy. Serious nervous

system events that were listed in the proposed rule included seizures,

psychosis, mania, severe depression, vestibular (inner ear) disturbances,

and loss of consciousness. Other reported adverse effects that FDA deemed

clinically serious or potentially serious included elevations of liver

function tests or overt hepatitis, myopathies (disease of muscle),

disturbances of the genitourinary system, and dermatologic

manifestations.

AEMS is considered an invaluable tool for identifying potential serious

public health issues that may be associated with the use of a particular

39Information in this appendix was obtained from a continuing education article from FDA’s Center for

Drug Evaluation and Research, “The Clinical Impact of Adverse Event Reporting” (Oct. 1996).

40Health professionals can report any adverse event that they judge to be clinically significant.

Page 34 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix II 

Adverse Event Reporting

product or types of products. Although more formal epidemiological

studies, prospective trials, and retrospective case-control studies are more

rigorous, surveillance can be important as an early alert to potential

problems. As such, even isolated reports can be definitive in associating

products with an adverse effect if the product and effect are temporally

related and there is evidence of dechallenge (signs and symptoms resolve

or improve when a consumer stops using a product) and rechallenge

(symptoms recur when the consumer resumes using the product).

However, like all voluntary surveillance systems, AEMS has certain

weaknesses. These limitations include different interpretations in

determining an adverse event, underreporting, biases, estimation of

population exposure, and report quality. In regard to differing

interpretations, it has been well established that AERs are quite subjective

and imprecise.41 For example, in one study, clinical pharmacologists and

treating physicians showed complete agreement less than half the time

when determining whether medication, alcohol, or recreational drug use

had caused hospitalization.42 It is also well known that placebos43 and even

no treatment44 can be associated with adverse events. In addition, there is

almost always an underlying background rate for any clinical event in a

population, regardless of whether there was exposure to a particular

product.

Another major concern of passive surveillance systems is underreporting

of adverse events. For example, it was estimated that under a British

spontaneous reporting system, only 10 percent of serious events and 2 to

4 percent of nonserious events are reported.45 Another estimate has shown

that FDA may receive, by direct reporting, less than 1 percent of suspected

serious adverse drug reactions.46 Thus, the cases spontaneously reported

41J. Kock-Weser and others, “The Ambiguity of Adverse Drug Reactions,” European Journal of Clinical

Pharmacology, Vol. 11 (1977), pp. 75-78.

42F. E. Karch and others, “Adverse Drug Reactions—A Matter of Opinion,” Clinical Pharmacology

Therapy, Vol. 19 (1976), pp. 489-92.

43D. M. Green, “Pre-Existing Conditions, Placebo Reactions, and Side Effects,” Annals of Internal

Medicine, Vol. 60 (1964), pp. 255-65.

44M. M. Reidenberg and D. T. Lowenthal, “Adverse Nondrug Reactions,” New England Journal of

Medicine, Vol. 279 (1968), pp. 678-79.

45M. D. Rawlins, “Pharmacovigilance: Paradise Lost, Regained or Postponed?,” The William Withering

Lecture, 1994, Vol. 29 (London: J. R. Coll. Physicians, 1995), pp. 41-49.

46H. D. Scott and others, “Rhode Island Physicians’ Recognition and Reporting of Adverse Drug

Reactions,” Rhode Island Medical Journal, Vol. 70 (1987), pp. 311-16.

Page 35 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix II 

Adverse Event Reporting

to any surveillance program generally represent only a small portion of the

number that have actually occurred.

Biases also limit the usefulness of spontaneous reporting systems. Unlike

clinical trial data, which are obtained under strictly controlled conditions,

spontaneously reported information is uncontrolled and, therefore, subject

to the possible influence of a number of reporting biases that are brought

about by the length of time a product has been on the market, the

reporting environment, and the quality of the data. Compounding these

problems, passive surveillance systems lack a denominator for the data,

such as user population or drug exposure patterns. This is helpful to

provide the exact number of people exposed to a product and, thus, the

risk for the adverse event to occur. As a result, incidence rates cannot be

computed. Finally, the ability to analyze information contained in an AER is

dependent on the quality of information submitted. Optimally, an AER

should include product name, demographic data, a clinical description of

the adverse event (including confirmatory and laboratory test results),

confounding factors (such as medical products and medical history),

temporal information, dose and frequency of use, biopsy or autopsy

reports, dechallenge and rechallenge information, and outcome.47

Although spontaneous reporting systems have a number of weaknesses,

they also have advantages over more controlled systems. First, they

maintain ongoing surveillance of products that are marketed and, second,

they are relatively inexpensive. In fact, they may be the most cost-effective

way to detect rare, serious adverse events not discovered during clinical

trials.48 Their usefulness lies in hypothesis generation, to explore cause

and effect relationships concerning an adverse event, and as a warning

signal of a potential problem with a product. Useful factors for assessing a

causal relationship include a chronology of administration of a product,

including beginning and ending of treatment and adverse event onset;

dechallenge; rechallenge; laboratory test results; and previously known

toxicity of an agent or product.

47FDA, Center for Drug Evaluation and Research, “The Clinical Impact of Adverse Event Reporting”

(Oct. 1996), pp. 5-6.

48FDA, Center for Drug Evaluation and Research, “The Clinical Impact of Adverse Event Reporting,”

p. 6.

Page 36 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix III 

FDA’s Prior Use of AERs in Rulemaking

AERs have been used by FDA in several rulemaking procedures. In response

to a request for prior rules that had relied on AERs, FDA provided five cases,

including one in which the “generally recognized as safe” (GRAS) status of

an agent (sulfites) was revoked. These cases are outlined in table III.1.

Table III.1: Five Prior FDA Rulemaking

Cases Regulation Substance Action

51 Fed. Reg. 25021 Sulfiting agent Revoked GRAS status for use on fruits

and vegetables intended to be served

or sold raw to consumers.

51 Fed. Reg. 25012 Sulfiting agent Required food labeling for foods.

49 Fed. Reg. 13679 Protein supplements Required warning labeling.

62 Fed. Reg. 2218 Supplements and

drugs containing iron

Required label warning statements and

unit-dose packaging.

44 Fed. Reg. 37212 FD&Ca yellow no. 5 Required labeling in food and drugs for

human use.

aFood, Drug, and Cosmetic.

Sulfiting Agents and

GRAS Status

FDA’s proposed rule to revoke GRAS status for sulfiting agents resulted after

the agency received information on the use of sulfur dioxide, sodium

sulfite, sodium bisulfate, potassium bisulfate, sodium metabisulfate, and

potassium metabisulfate (collectively known as sulfiting agents, or

sulfites) on fruits and vegetables intended to be served raw or sold raw to

consumers and it concluded that such use was not safe. This rule, similar

to the ephedrine alkaloid ruling, used AERs, advisory panels, clinical

studies, and case reports to determine whether sulfiting agents should

maintain its GRAS status.

FDA had received over 500 consumer complaints, where individuals

reportedly suffered from a variety of adverse allergic-type responses,

including reports of 13 deaths; the proposed rule indicated that these

responses tended to occur in people with asthma.49 Among the 500 AERs,

approximately 40 percent mentioned the occurrence of the event after

eating raw fruits and vegetables in restaurants (where sulfiting agents are

used), while 15 percent specifically mentioned the occurrence of the event

after drinking wine or beer. The proposed rule delineated the clinical

outcomes from these adverse reactions, ranging from gastrointestinal

49FDA relied on prevalence estimates that indicated there are 10 million individuals with asthma in the

United States, with up to 10 percent sulfite-sensitive. See Select Committee on GRAS Substances,

“The Reexamination of the GRAS Status of Sulfiting Agents,” Life Sciences Research Office,

Federation of American Societies for Experimental Biology, prepared under FDA contract 223-83-2020

(Jan. 28, 1985).

Page 37 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix III 

FDA’s Prior Use of AERs in Rulemaking

problems to anaphylactic shock (hypersensitivity reactions characterized

by swelling, erythema, bronchoconstrictions, and hypotension).

A scientific panel also reviewed the AERs submitted to FDA and concluded

that the reports indicated an association between adverse responses and

the ingestion of meals that included foods containing sulfiting agents. The

panel also examined exposure estimates and evidence from clinical

experiments during its review on this topic.

While this rule and the ephedrine alkaloid rule are similar—because the

AERs used in the rulings simply highlighted the number of complaints and

provided a breakdown of the complaints’ origins and clinical

complications—the proposed rule on sulfiting agents outlined additional

information not found in the proposed rule for ephedrine alkaloids. This

included information on exposure estimates, prevalence rates within

sensitive populations, and clinical experiments involving the exposure of

sensitive and nonsensitive populations to sulfiting agents. In addition,

dosing regimens were not set for sulfiting agents because the proposed

rule revoked their GRAS status.

Sulfiting Agents and

Food Labeling

The proposed rule for the labeling of products containing sulfiting agents

was designed to clarify the circumstances in which the presence of

sulfiting agents must be declared on food labels. FDA noted in this

proposed rule that any detectable amount of a sulfiting agent would

require that the product be labeled because the agency was unaware of

any evidence that established a level below which these substances would

not cause a reaction in sensitive individuals. FDA relied on scientific data

on human sensitivity to sulfiting agents to establish that certain sensitive

individuals—in particular, those with asthma—could react to ingested

sulfiting agents. FDA also used estimates of the number of individuals with

asthma and clinical studies on those who were sensitive to sulfiting agents

to determine the number who might be adversely affected. Last, FDA

formed an ad hoc advisory committee on hypersensitivity to sulfiting

agents in food to review and evaluate available data on adverse reactions

in humans associated with the use of sulfiting agents in food. This rule did

not describe, in any appreciable detail, adverse events.

Labeling for Protein

Supplements

In December 1997, FDA proposed labeling requirements for protein

supplements that were used in weight reduction programs. The purpose of

the rule was to alert consumers to the potential health hazards associated

Page 38 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix III 

FDA’s Prior Use of AERs in Rulemaking

with consuming protein supplements to control weight and to inform

consumers that the advice of a physician should be sought before using

these products for weight control. The agency proposed these

requirements on the basis of evidence that, without proper medical

supervision, low-calorie diets consisting primarily of protein may cause

serious medical problems, including death. The proposed rule noted that

an ad hoc advisory group had been formed to examine several cases of

deaths associated with individuals using these protein products as their

primary source of nourishment.

A tentative final rule published on December 29, 1978, noted that more

than 165 reports of adverse reactions attributed to consumption of protein

products by consumers attempting to lose weight had been reported to the

FDA. These complaints—described as “very diverse”—included nausea,

tachycardia (rapid heart rate or palpitations), breathlessness, and

headaches.50 Review of these reports by FDA revealed no consistent pattern

of complaints or reactions, and the tentative final rule concluded that “a

cause-and-effect relationship can neither be affirmed nor rejected on the

basis of the existing information from these reports.” The rule further

stated that “There were sufficient details among the reported adverse

reactions . . . to warrant concern.”

FDA further asserted that imposing a warning on the labeling of these

protein products—even with the gaps in their understanding of the basic

mechanism of how these protein supplements affected the body—was

particularly appropriate because the consequence of indiscriminate use of

these products was death. FDA also noted that the protein products were

not inherently dangerous and that the label warning was proposed to curb

misuse of these products and because the products were being sold

directly to consumers who might not be under proper medical supervision.

Data were collected through a telephone survey, conducted by FDA and

CDC, to determine the extent of use of protein products. This information

was used to establish a death rate by combining this information with data

on deaths among liquid protein dieters from the same population group.

This information was compared with data from the National Center for

Health Statistics on annual death rates due to cardiac abnormalities among

a similar population group. FDA concluded that the observed death rate

was greater in those who used low-calorie protein diets. FDA also initiated

experimental protocols using laboratory rats in order to explore the basic

50After a court challenge from the National Nutritional Foods Association, a final rule was published on

April 4, 1984.

Page 39 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix III 

FDA’s Prior Use of AERs in Rulemaking

mechanisms for the cause of death. These studies were designed to assess

the correlation of cardiac arrhythmias (irregular heartbeat) and death with

various aberrant states of nutrition health and rate of weight loss.

Supplements and

Drugs Containing Iron

In October 1994, FDA proposed warning labels and unit-dose packaging

requirements51 for products taken in solid oral form to supplement the

dietary intake of iron or to provide iron for therapeutic purposes. The

labeling was proposed because of evidence of acute iron poisoning

attributable to accidental overdoses of iron-containing products and an

upsurge in reported accidental pediatric ingestion of iron-containing

products, which, in some cases, resulted in death.52

FDA relied on data from the American Association of Poison Control

Centers and the U.S. Consumer Product Safety Commission (CPSC) that

highlighted increases in reported fatalities from accidental iron poisonings

of children. Over 63,000 reports were taken at poison control centers

involving ingestion of adult iron-containing products, with over 47,000 of

these involving children under 6 years of age. One hundred and fifty-nine

cases were classified as major outcomes; that is, they were life threatening

or resulted in permanent injury. An additional 1,500 cases were classified

as “moderate outcomes”; that is, the patient had symptoms that required

some form of treatment. Over 76,000 reports to these centers involved

ingestion of pediatric iron-containing products with children under 6 years

of age.

Likewise, data from the National Electronic Injury Surveillance System—a

probability sample of hospital emergency rooms in the United States used

by CPSC to measure the magnitude of injury associated with consumer

products—found a significant upward trend in the estimated number of

hospital emergency-room-treated iron-ingestion cases involving children

under 5 years of age. The proposed rule also outlined data collected by

CPSC on case reports of pediatric deaths, including the number of tablets

taken, the potency (dosage), and total amount ingested. Additionally, a

conference was held by CPSC to examine the reason for the increase in iron

poisonings among children.

51Unit-dose packaging is designed to prevent the unintended ingestion by children, not to control the

recommended dose for adults.

52U.S. Consumer Product Safety Commission, “Pediatric Iron Poisonings and Fatalities” (May 1994),

p. 3.

Page 40 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix III 

FDA’s Prior Use of AERs in Rulemaking

The final rule expanded on the information that was used as the basis for

warning labels by outlining the results of animal toxicity studies, acute

toxicity in humans, comparison of animal toxicity data to human toxicity

data, and physiological factors that influence toxicity. Thus, information

used by FDA in its proposed and final rules on iron-containing products

included data from CPSC, case reports from adverse reactions, and

scientific literature.

FD&C Yellow No. 5 In its proposed rule of February 4, 1977, FDA noted that there was

accumulating evidence that FD&C yellow no. 5 (tartrazine) caused

allergic-type responses, especially in aspirin-intolerant individuals.53 Thus,

FDA was proposing to require a label declaration of FD&C yellow no. 5

when used to color foods and ingested drugs and to prohibit its use in

certain drugs for human use. FDA noted in the proposed rule that “the

precise incidence of intolerance of FD&C yellow no. 5 in the total

population or even in aspirin-intolerant patients is not known.” However,

FDA did outline a number of case reports and studies highlighting that

these effects did occur and that some quantification regarding

subpopulations at risk could be identified. Specifically, FDA estimated that

approximately 47,000 to 94,000 people would be intolerant to tartrazine.

But this proposed rule did not denote how many case reports or adverse

drug reactions formed the basis for FDA’s actions. Instead, the primary

information used in the proposed and final rule related to scientific studies

demonstrating individual intolerance to tartrazine.

53In the final rule, FDA required a label declaration of FD&C yellow no. 5 when used to color foods and

drugs for human use.

Page 41 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

E.O. 12866 and related guidance from OIRA state that cost-benefit analyses

should contain three basic elements: (1) a statement of the need for the

proposed action, (2) an examination of alternative approaches, and (3) an

analysis of benefits and costs. Overall, the guidance indicates that an

analysis should be “transparent.” We concluded that FDA’s cost-benefit

analysis contained, to some degree, all three of the recommended

elements in such analyses. FDA also disclosed the basic methodology, data,

and assumptions used in its analysis. However, the cost-benefit analysis

was not always transparent regarding why certain key assumptions were

made, the degree of uncertainty involved in those assumptions, or the

effect that alternative assumptions would have had on the agency’s

estimates of the costs and benefits of the proposed action.

E.O. 12866 and

Related Guidance Set

Agency

Responsibilities for

Analysis of Benefits

and Costs

Conceptually, a cost-benefit analysis is a rigorous and data-intensive

procedure of weighing the costs and benefits of various alternatives to a

proposed action, informing decisionmakers about the potential

consequences of each. However, the results of the analysis, by themselves,

do not determine whether or how an agency will regulate in response to a

perceived problem.

E.O. 12866 requires agencies to conduct cost-benefit analyses for all

regulatory actions that are likely to result in a $100 million annual effect

on the economy or are otherwise economically significant. The executive

order also makes OIRA responsible for reviewing all significant regulatory

actions and for providing guidance to agencies on issues covered by the

order. On January 11, 1996, the Administrator of OIRA issued guidance

describing “best practices” for preparing cost-benefit analyses.54 The

guidance makes clear that it is not a “mechanistic blueprint” and provides

for flexibility and the exercise of professional judgment in preparing

analyses. However, the guidance does establish one clear expectation of

all such analyses:

Analysis of the risks, benefits, and costs associated with regulation must be guided by the

principles of full disclosure and transparency. Data, models, inferences, and assumptions

should be identified and evaluated explicitly, together with adequate justifications of

choices made, and assessments of the effects of these choices on the analysis. The

existence of plausible alternative models or assumptions and their implications should be

identified. In the absence of adequate valid data, properly identified assumptions are

necessary for conducting an assessment.

54The OIRA guidance was developed as a result of a 2-year study by an interagency group that included

representatives of all the major regulatory agencies and was co-chaired by a member of the Council of

Economic Advisers.

Page 42 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

Within this broad framework, the OIRA guidance states that analyses of

economically significant rules should contain three basic elements: (1) a

statement of the need for the proposed action, (2) an examination of

alternative approaches, and (3) an analysis of benefits and costs. Within

each of these three elements, the guidance recommends that certain items

be considered or analytical approaches be used in preparing an agency’s

analysis. For example, in relation to the third requirement, the guidance

says the analysis should identify and explain the data or studies on which

the estimates of benefits and costs are based “with enough detail to

permit independent assessment and verification of the results.” The

guidance also states, “where benefit or cost estimates are heavily

dependent on certain assumptions, it is essential to make those

assumptions explicit and, where alternative assumptions are plausible, to

carry out sensitivity analyses based on the alternative assumptions.” The

purpose of such disclosure is to allow for a reasoned determination by

decisionmakers of the appropriate level of regulatory action.

FDA’s Analysis

Addressed the

Primary Elements

Suggested by OIRA

Guidance

FDA indicated in the proposed rule that the regulation was economically

significant, and FDA officials told us that they used the OIRA guidance in

developing the cost-benefit analysis for the rule. We compared FDA’s

cost-benefit analysis to the elements and practices outlined in that

guidance and concluded that FDA’s analysis addressed the primary

elements suggested by the OIRA guidance. However, certain specific

elements appeared to be lacking.

Need for Regulation The OIRA guidance states that in establishing the need for the proposed

action, agencies should discuss (1) whether the problem constitutes a

significant market failure that compels government action and (2) if there

is a market failure, the appropriateness of alternatives to federal

regulation that would resolve the problem adequately or better than the

proposed rule. Among the types of market failures that the guidance says

agencies can discuss are monopolies and inadequate information available

to consumers about product characteristics. The guidance states that

alternatives to federal regulation can include subsidies or fees that may be

more efficient than rigid mandates or state and local regulation that would

be more appropriate.

FDA stated in the proposed rule that the rule was needed because of a

significant market failure—specifically, inadequate information. FDA said

that despite the presence of warning labels of various types on many of the

products, “some consumers may not have sufficient information on the

Page 43 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

health risks associated with dietary supplements containing ephedrine

alkaloids to make informed choices concerning the consumption of these

products.” Among other points, FDA said that the level of information

currently used by consumers might be less than optimal because of

consumer perceptions that products marketed as foods or derived from

botanical sources are inherently safe.

However, FDA did not directly address the second major item in the

guidance under the need for action section—whether alternatives to

federal regulation would resolve the problem adequately or better than the

proposed federal regulation. In particular, FDA did not recognize that

several states were in the process of regulating these products and did not

discuss whether federal regulation was superior to those state regulations

(for example, because these products are marketed across state lines).

Examination of Alternative

Approaches

The OIRA guidance states that an agency’s cost-benefit analysis should

demonstrate that the agency has considered the most important

alternative approaches to the underlying problem and should provide the

agency’s reasoning for selecting the proposed regulation over such

alternatives. The guidance notes that the number and choice of

alternatives to be selected for a detailed cost-benefit analysis is a matter of

judgment but says the agency should nevertheless “explore modifications

of some or all of a regulation’s attributes or provisions to identify

appropriate alternatives.” The guidance also states that the agency should

explore a number of different types of regulatory alternatives, including

• more performance-oriented standards for health, safety, and

environmental regulations;

• different requirements for different segments of the regulated population;

• alternative levels of stringency;

• alternative effective dates of compliance;

• alternative methods of ensuring compliance;

• informational measures (for example, mandatory disclosure requirements,

such as labeling);

• more market-oriented approaches; and

• consideration of whether the agency should adopt a more stringent

standard than already established by statutory requirements.

In the proposed rule, FDA identified seven regulatory “options”: (1) take

no action (the baseline alternative); (2) take no action, but generate

additional information; (3) take the proposed action; (4) take the proposed

Page 44 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

action but with a higher potency limit; (5) ban dietary supplements that

contain ephedrine alkaloids; (6) take the proposed action, but do not

require the warning statement; and (7) require only the warning statement.

In general, FDA’s options focused primarily on the OIRA guidance categories

of alternative levels of stringency and informational measures. FDA did not

appear to explore performance-oriented standards, different requirements

for different segments of the regulated population, market-oriented

approaches, or alternative effective dates of compliance. However, it is

unclear whether these types of alternatives could legitimately be

considered for the proposed regulation. For example, it is unclear how FDA

could establish market approaches, such as additional fees on the dietary

supplement industry, or performance standards for health effects.

In comments on the rule, industry representatives and SBA’s Office of

Advocacy identified what they viewed as a weakness related to FDA’s

baseline description of the world absent the proposed regulation. FDA

presented information on the number of adverse events believed to be

associated with ephedrine alkaloid products and described an increasing

trend in reported adverse events. However, industry representatives and

SBA said FDA did not provide sufficient information to put these data in a

meaningful context. For example, they said there was no information on

general probabilities or risks of people suffering the types of adverse

events that FDA attributed to ephedrine alkaloid products. They also said

that FDA did not provide information to put the aggregate number and

growth in reported adverse events in the context of total consumption of

dietary supplements that might contain ephedrine alkaloids. SBA’s Office of

Advocacy and industry trade associations commented that consumption of

such supplements might be in the billions of doses. In that context, they

contended that adverse events associated with the consumption of dietary

supplements containing ephedrine alkaloids may occur at very low rates,

perhaps lower than those for some over-the-counter drugs.

On the other hand, FDA did provide some contextual information in the

proposed rule on the incidence of adverse events for dietary supplements

as a whole. FDA’s data indicated that AERs believed to be associated with

supplements containing ephedrine alkaloids accounted for approximately

50 to 60 percent of the total number of reports received by FDA when the

rulemaking was initiated. FDA officials told us that the number of AERs in

their system suspected of involving ephedrine alkaloids “jumped out”

compared with the numbers for any other substances, although they have

not done any detailed analysis or tally to document numbers for other

Page 45 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

substances. Representatives of industry trade associations who we met

with acknowledged that the incidence of adverse events for supplements

containing ephedrine alkaloids is greater than would be expected for most

other dietary supplements. Still, providing only this contextual information

in the proposed rule does not appear to provide a balanced or complete

picture of the baseline risks that may be associated with supplements

containing ephedrine alkaloids.

Analysis of Benefits and

Costs

The bulk of OIRA’s guidance is devoted to addressing the third element,

analysis of benefits and costs. In this section, the guidance first presents

general principles on (1) measuring the benefits and costs of each

alternative against a baseline condition of the way the world would look

absent the proposed regulation, (2) evaluation of alternatives,

(3) discounting costs and benefits that may occur at different points in

time, (4) treatment of risk and uncertainty, (5) assumptions used in the

analysis, (6) international trade effects, (7) nonmonetized benefits and

costs, and (8) distributional effects and equity. These general principles

are followed in the OIRA document by more detailed guidance on the

analysis of benefits and costs. For example, within the benefits section,

the guidance states that reductions in illnesses, injuries, and fatalities as a

result of government action are best monetized according to a “willingness

to pay” approach.55 The guidance also emphasizes that benefits and costs

should be “incremental,” representing changes from the baseline

condition of no regulation, and that the analysis should identify and

explain the data or studies on which the estimates are based “with enough

detail to permit independent assessment and verification of the results.”

In its analysis of benefits and costs, FDA adhered to a number of the

recommendations in the OIRA guidance. For example, the analysis

discussed distributional effects of the proposed rule and alternatives in

terms of lost sales to the dietary supplement industry. FDA also discussed

nonmonetized costs associated with the proposed rule, and it monetized

the benefits associated with reductions in various types of adverse events

using the willingness-to-pay approach recommended in the guidance. In

addition, the analysis presented the monetized costs of the proposed rule

against the baseline condition of no regulation. However, FDA did not

provide monetized or quantified benefit or cost estimates for all of the

other alternatives, and the agency’s discussion of those alternatives’

benefits and costs were compared to the proposed rule—not the baseline

55The principle of “willingness to pay” captures the notion of opportunity cost by providing an

aggregate measure of what individuals are willing to forgo to enjoy a particular benefit.

Page 46 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

condition of no regulation. For example, in relation to the benefits

associated with allowing higher potency levels (option 4) compared to

FDA’s proposal (option 3), FDA said that the effect would be to “reduce

those benefits below those generated under option 3.” (FDA officials told

us during our review that the agency attempted to reach a balance in

conducting the analysis and said the analysts try to quantify the effects of

alternatives when such information would be of value to decisionmakers.)

Also, some parts of FDA’s analysis of these alternatives were either unclear

or illogical. For example, in relation to the option of banning dietary

supplements that contain ephedrine alkaloids (option 5), FDA stated that

“banning these products would decrease access to these products by

consumers.” It is not clear why a total ban in the products would not

eliminate—not just decrease—access. Similarly, FDA also said “the total

reduction in the consumption of dietary supplements that now contain

ephedrine would probably be approximately 33 percent under this

option.” It is not clear why a ban would only reduce consumption by

33 percent. Another unclear element of the analysis concerns discounting.

The guidance recommends that dollar estimates should be discounted and

reported in same-year dollars. Although FDA noted that it converted dollar

figures for its estimates of monetized benefits to 1996 dollars, it did not

identify in the proposed rule whether its total cost estimates were also

expressed in 1996 dollars.

As a result of its analysis, FDA concluded that the proposed rule would

generate benefits of between $240 million and $670 million per year,

quantifiable costs of between $3 million and $70 million in the first year,

and quantifiable costs of between “a minimal amount” and $500,000 in

each subsequent year. FDA justified the selection of the proposed rule over

the other regulatory alternatives in terms of the greater net benefits

expected of the proposed actions.

Analysis Did Not Fully

Explain Some Key

Assumptions or

Assess Implications of

Alternative

Assumptions

The OIRA guidance emphasizes the importance of full disclosure and

transparency of key data and assumptions in cost-benefit analyses. As

noted previously, the guidance says that “data, models, inferences, and

assumptions should be identified and evaluated explicitly, together with

adequate justifications of choices made, and assessments of the effects of

these choices on the analysis.” Therefore, we assessed the transparency

of FDA’s cost-benefit analysis of the proposed rule in terms of three

criteria:

Page 47 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

• first, whether it identified the data, models, inferences, and assumptions

used to calculate the estimates of benefits and costs;

• second, whether it disclosed the reasons why those data, models,

inferences, and assumptions were selected; and

• third, whether it assessed the effects of plausible alternative assumptions

and choices on the results of the analysis—what is often referred to as a

“sensitivity analysis.”

Identification of Data and

Assumptions

In relation to the first criterion, FDA’s analysis was very transparent. The

agency provided a clear and lengthy description of the data, assumptions,

and methodology that it used to calculate the benefits and costs of the

proposed rule. For example, in calculating the annual monetized benefits

of the proposed rule, FDA identified for each of six categories of adverse

events (for example, deaths, serious cardiovascular system events, serious

nervous system events, and abnormal liver functions) (1) the annual

number of reported adverse events (based on the average number of AERs

received each year), (2) the estimated total number of such events each

year, (3) the estimated number of cases that would be reduced by

implementation of the proposed rule, (4) the monetized value of each

reduced case, and (5) the estimated value of the total risk reduction (the

number of reduced cases times the value of each case).

FDA also identified many of the underlying assumptions that it used in

developing these elements. For example, in developing its estimate of the

total number of annual cases, FDA noted that it assumed that (1) 80 percent

of the reported adverse events suspected of involving supplements

containing ephedrine alkaloids were actually related to the consumption

of dietary supplements, (2) 80 percent of these supplements actually

contained ephedrine alkaloids, and (3) only 10 percent of adverse events

related to ephedrine alkaloids were actually reported. In similar fashion,

FDA disclosed how it calculated the estimated compliance costs of the rule,

including the one-time costs associated with relabeling and reformulating

the affected supplements.

Explanation of

Assumptions

In relation to the second transparency criterion—explaining the reasons

for choices made for key assumptions and values—FDA’s analysis was only

partially transparent. For most elements of the analysis, FDA identified the

underlying data sources used and the rationale for the assumptions and

conclusions reached by FDA’s analysts and experts. The proposed rule

included a lengthy discussion about the scientific data and studies FDA and

Page 48 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

its advisory committees reviewed in developing the various provisions of

the rule. In estimating the effects of these provisions, FDA also identified

the underlying data sources and analysis in most cases. For example, in its

discussion on potential relabeling costs, FDA described how it used data

from a previous FDA rule on nutrition labeling of dietary supplements to

derive its estimate of the specific costs for this rule. In describing how it

estimated the effect of restrictions on including other stimulants in

supplements containing ephedrine alkaloids, FDA discussed the results of a

clinical study on combinations of ephedrine and caffeine and the results of

its informal review of adverse event reports.

However, FDA did not fully disclose the underpinnings of all of its

assumptions and choices. In particular, although FDA identified the values

it used to estimate the number and severity of adverse events for its

calculation of benefits, its explanation of the underlying data sources,

analyses, and basis for the selected values was not complete. In its

analysis, FDA identified three main areas of uncertainty in its estimation of

the actual number of adverse events believed to be attributable to use of

products containing ephedrine alkaloids. These areas were (1) the

proportion of reported adverse events that were actually related to

consumption of dietary supplements, (2) the proportion of reported

adverse events that involved supplements that actually contained

ephedrine alkaloids, and (3) the proportion of adverse events that were

actually reported. For each of these elements, FDA assumed a particular

value that drove up the estimated number of actual adverse events and, as

a direct consequence, the estimated benefits of the proposed rule.

However, FDA did not always disclose why it selected those values or why

it selected a single value instead of a range of values that would have more

clearly represented the degree of uncertainty that FDA believed was

present.

In relation to the first area of uncertainty, FDA estimated that between 27

and 90 percent of the reported adverse events were “probably” related to

the consumption of dietary supplements suspected of containing

ephedrine alkaloids. However, FDA did not clearly disclose the basis for

estimating this range. Within this range, FDA assumed that 80 percent of the

reported adverse events were actually related to the consumption of

dietary supplements. FDA did not disclose how it arrived at the choice of

80 percent as the “most likely” value. Furthermore, in calculating the

“range” of estimated benefits for the proposed rule, FDA used only this

point estimate, not the entire 27 to 90 percent range, or a more narrowly

focused range of values in calculating the expected number of cases. Use

Page 49 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

of a range of values for this element would have more clearly represented

the uncertainty regarding the number of AERs actually related to the

consumption of dietary supplements.56

The second source of uncertainty that FDA identified was the extent to

which all AERs involved products that actually contained ephedrine

alkaloids. FDA estimated that the proportion of reported adverse events

associated with dietary supplements containing ephedrine alkaloids was

“probably” between 25 and 90 percent based on (1) the labeling of the

products involved, (2) FDA’s own market study and laboratory analysis of

125 marketed products, and (3) the similarity of the reported adverse

events to the known effects of ephedrine alkaloids. Within this range, FDA

assumed that 80 percent of the reported adverse events associated with

the consumption of dietary supplements involved supplements that

contained ephedrine. FDA did not disclose why it selected 80 percent as the

“most likely” value. Finally, as was the case in the first area of

uncertainty, FDA used only this 80 percent point estimate, not the full range

or an attenuated range of values, in calculating the expected number of

annual cases.

FDA’s third identified source of uncertainty was the likelihood that all

adverse events related to ephedrine alkaloids were probably not being

reported. FDA noted in the analysis that typical reporting rates for passive

reporting systems on adverse events associated with pharmaceutical drugs

are “generally assumed to be on the order of 10 percent.” However, FDA

noted that reporting rates might be higher than usual if, for example, the

potential health risks are widely publicized, or lower than normal if

consumers and physicians assume that dietary supplements are incapable

of producing adverse events. FDA said that “to incorporate this

uncertainty,” it assumed that the reporting rate for the adverse events

related to ephedrine alkaloids was 10 percent. It was not clear how a

single point estimate of 10 percent would “incorporate” the uncertainty

that FDA identified for this value.

In response to our questions about these uncertainties, FDA officials

acknowledged that their analysis of impacts was not as transparent as it

should have been in explaining how the agency arrived at some of the

assumptions. The Director of FDA’s Market Studies Division, which

prepared the cost-benefit analysis, explained that the estimated

56FDA was somewhat inconsistent in its use of point estimates versus ranges. FDA used its full

estimated range of adverse events in projecting the results for other regulatory options, such as

banning supplements with ephedrine alkaloids. FDA officials said that they chose to use the full ranges

in discussing other options because of the regulatory actions in those options.

Page 50 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

proportions and the selected values represented the best professional

judgment of FDA’s analysts and experts. He noted that the upper-bound

estimate of 90 percent for the first two areas of uncertainty represented

what the analysts believed was a reasonable choice for the “absolute most

it could be.” He said that the lower-bound estimate of 27 percent for the

first category was derived from the cases in the AERs in which people who

stopped using products suspected of containing ephedrine alkaloids no

longer experienced the effects associated with the adverse event.

Similarly, FDA’s figure of 25 percent as the most reasonable low-end

estimate for the second category was based on FDA’s review of samples of

products mentioned in the AERs. He said that the choice of 80 percent as

the most likely value for both of these categories was a determination

made by FDA’s scientific experts, given all the available data and scientific

evidence they had reviewed, not just the AERs.

The director and other FDA officials indicated that FDA had much more

confidence in the estimate that only 10 percent of actual adverse events

were reported. They said that, compared to the other two sources of

uncertainty, FDA knows more about AEMS and is also familiar with studies

of other passive reporting systems used by FDA and other federal agencies.

The officials noted that, for other systems, they have done reviews for

specific sites (for example, going to hospitals to check records and do

testing) and found that reporting rates were about 10 percent. The director

said that FDA did not provide a range for this category in the proposed rule

because FDA was confident in the point estimate and because it did not

have any way—other than through an “artificial exercise”—to come up

with a range of possible values.

Implications of Alternative

Assumptions

FDA’s cost-benefit analysis was also only somewhat transparent with

regard to the third of our transparency criteria—assessment of the effects

of plausible alternative assumptions and choices on the results of the

analysis. The degree of transparency in this area and the sensitivity of

FDA’s results to alternative assumptions are most clearly illustrated in the

agency’s estimate of expected benefits.

FDA estimated that the benefits of the proposed rule would be between

$240 million and $670 million per year. That estimate was driven by three

factors: (1) FDA’s estimate of the number of adverse events each year

(1,110), (2) FDA’s estimate of the degree to which the proposed rule would

reduce these events (35 percent to 100 percent across all types of

proposed actions), and (3) the values FDA assigned to the estimated risk

Page 51 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

reduction per case (for example, $5 million per death avoided).57 Changes

in any of these values could have dramatically changed FDA’s estimates of

the proposed rule’s benefits. By using the 35 to 100 percent range for the

second factor (and even wider ranges for some of the individual actions),

FDA illustrated the sensitivity of the analysis to changes in particular

values. This second factor was the only component in FDA’s calculations

that generated a “range” in the estimated reduction in annual adverse

events (390 to 1,110) and, in turn, the size of the expected benefits

($240 million to $670 million).58

On the other hand, FDA’s estimate of the 1,110 estimated adverse events

each year (factor 1) was based on an average of 174 adverse events

reported per year and the three assumptions about adverse events—that

80 percent of the AERs involved dietary supplements, that 80 percent of the

supplements contained ephedrine alkaloids, and that only 10 percent of all

adverse events related to supplements containing ephedrine alkaloids

were reported. FDA’s use of these point estimates (instead of ranges) does

not reflect the sensitivity of the final benefit estimates to these

assumptions. As illustrated in figure IV.1, the results of FDA’s estimation of

expected benefits are, therefore, very sensitive to changes in the

underlying assumptions. If FDA had used other plausible assumptions,

based on the information it presented in the proposed rule, the ranges of

expected benefits would have varied noticeably from the values FDA

presented in its proposal.

57In its proposed rule, FDA did not report or round its estimated values consistently, so slightly

different numbers for the same element appear within the proposed rule. For this report, we are using

the values reported in table 6 of FDA’s proposed rule, such as 1,110 for the estimated annual number

of adverse events.

58Similarly, FDA’s $3 million to $80 million range in the estimated cost of the proposed rule also

reflects the sensitivity of the agency’s assumptions.

Page 52 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

Figure IV.1: Estimation of Benefits Is

Sensitive to Changes in Underlying

Assumptions

0

200

400

600

800

1000

1200

1400

(1) FDA's

Reported

Estimates

(2) Using Full

Range of FDA

Assumptions

(3) Assuming

20% of Cases

Are Reported

(4) Assuming

5% of Cases

Are Reported

670

240

850

336

118

470

1343

25

Estimated Annual Benefits (Dollars in Millions)

High Bound

Low Bound

Legend

Note: The data for scenario 1 are from FDA’s reported estimates in table 6 of the proposed rule.

Scenario 2 was calculated using the full range of assumptions FDA presented in the narrative of

its proposed rule regarding key assumptions about the number of adverse events each year.

Scenarios 3 and 4 were calculated changing only FDA’s selected assumption regarding the

reporting rate for adverse events. All other assumptions were held constant.

Source: Scenario 1 data are taken from table 6 of FDA’s proposed rule. Data for other scenarios

were GAO calculations using information in FDA’s proposal.

Given the uncertainty surrounding most of the key data and assumptions

used in FDA’s analysis, similar types of sensitivity analyses could be done

using alternative choices for other elements in the analysis, such as the

following:

Page 53 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

• FDA used published estimates of the value that consumers place on

reducing certain types of risk to estimate the monetary value benefits

associated with the proposed rule. However, other published estimates for

these values could also have been used. For example, FDA assumed in its

analysis that the value of avoiding a death was $5 million. We have

reported that the Environmental Protection Agency’s (EPA) assumptions

regarding the value of avoiding a death have ranged from $1.6 million to

$12 million.59

• FDA’s estimates of the benefits of the rule were also a function of the

distribution of the types of adverse events (for example, deaths,

cardiovascular system events, and neurological system events) reported

through the underlying AERs. Given that there is sufficient uncertainty

regarding the reports that different distributions of the numbers of events

are plausible, FDA’s analysis could have reflected these alternative

distributions and the effect they would have had on the benefit estimates.

• FDA’s estimates of the costs of the proposed rule were based, in part, on

assumptions regarding the number of dietary supplement products sold

and the number of businesses involved in the dietary supplement market.

As discussed in appendix V, “Regulatory Flexibility Act Requirements,”

there is considerable uncertainty about the actual number of businesses

and products involved in the market for dietary supplements containing

ephedrine alkaloids.

As FDA officials noted in our meetings with them, there are limits to the

amount of effort that can be expended on an analysis, and they attempt to

reach a balance in conducting a reasonable effort. They pointed out that

the proposed rule was long and that they had struggled to keep the length

of the narrative under control. However, the OIRA guidance recommends

that agencies carry out sensitivity analyses over the full range of plausible

values of key parameters, particularly when there are several easily

identifiable critical assumptions in the analysis and information is

inadequate to carry out a more formal probabilistic simulation for

assessment of risks. In this respect, FDA’s focus on presenting the agency’s

“best estimate” does not seem to fully reflect the uncertainty in the

underlying data and analysis or the expectations set by the OIRA guidance.

In explaining why FDA did not do a sensitivity analysis for this proposed

rule, the Director of the Market Studies Division said that, at this point in

time, risk assessors are ahead of cost-benefit analysts in doing sensitivity

analyses. He did note that, while FDA did not do a sensitivity analysis for

59Regulatory Reform: Agencies Could Improve Development, Documentation, and Clarity of

Regulatory Economic Analyses (GAO/RCED-98-142, May 26, 1998), and Air Pollution: Information

Contained in EPA’s Regulatory Impact Analyses Can Be Made Clearer (GAO/RCED-97-38, Apr. 14,

1997).

Page 54 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IV 

FDA’s Analysis of Benefits and Costs

this particular proposed rule, it would probably do more sensitivity

analyses in the near future.

Page 55 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix V 

Regulatory Flexibility Act Requirements

The Regulatory Flexibility Act requires agencies to consider the effects of

their rules on small entities. FDA determined that its proposed rule on

dietary supplements containing ephedrine alkaloids would have a

significant economic impact on a substantial number of small entities.

Therefore, FDA prepared an initial regulatory flexibility analysis to identify

the potential impacts of the proposed rule and alternative actions on

affected small businesses. Also, as required by RFA, FDA provided

opportunities for small business participation in the rulemaking process.

However, while FDA addressed the basic requirements of the act, SBA’s

Office of Advocacy criticized the quality of FDA’s regulatory flexibility

analysis. For example, the office contended that FDA underestimated the

number of affected small businesses and the impacts of the proposed

actions by not considering the large numbers of independent distributors

of dietary supplements in its analysis.

Regulatory Flexibility

Act Requires Agencies

to Consider Impact of

Regulation on Small

Businesses

During the past 20 years, the Congress has enacted a number of statutes

designed to improve the federal rulemaking process, one of which is the

Regulatory Flexibility Act of 1980, as amended (5 U.S.C. 601-612).

Congress enacted RFA in response to concerns about the effect that federal

regulations can have on small entities. According to SBA’s Office of

Advocacy, the major goals of the act are to (1) increase agency awareness

and understanding of the impact of their regulations on small business,

(2) require that agencies communicate and explain their findings to the

public, and (3) encourage agencies to use flexibility and to provide

regulatory relief to small entities.

RFA mandates certain actions on the part of agencies during the

rulemaking process. For example, before publishing a proposed rule for

which a notice of proposed rulemaking is required, sections 603 and

605(b) of the act state that a federal agency must prepare and make

available for public comment an initial regulatory flexibility analysis that

describes the anticipated effects of the proposed rule on small entities,

unless the head of the agency certifies that the proposed rule will not have

a “significant economic impact on a substantial number of small entities.”

Specifically, the act states that an agency’s initial analysis must contain

(1) a description of why the agency is considering the action; (2) the

objectives and legal basis for the proposed rule; (3) a description and,

where feasible, an estimate of the number of small entities to which the

proposed rule will apply; (4) a description of the projected reporting,

recordkeeping, and other compliance requirements of the proposed rule;

and (5) a description of all federal rules that may duplicate, overlap, or

Page 56 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix V 

Regulatory Flexibility Act Requirements

conflict with the proposed rule. The act also states that the initial analysis

shall discuss significant alternatives for small entities that accomplish the

objectives of applicable statutes, such as different compliance or reporting

requirements, clarified or consolidated compliance and reporting

requirements, and an exemption from coverage by the rule.

When a rule is promulgated that will have a significant economic impact

on a substantial number of small entities, section 609(a) of RFA requires

agencies to ensure that small entities have been given an opportunity to

participate in the rulemaking process through the “reasonable use” of

outreach techniques.60 The act delineates a number of specific types of

outreach techniques agencies could use, including (1) publishing the

notice of proposed rulemaking in publications likely to be obtained by

small entities, (2) directly notifying interested small entities, and

(3) conducting open conferences or public hearings concerning the rule.

Section 612 of the act requires the SBA Chief Counsel for Advocacy to

monitor agencies’ compliance with RFA and authorizes the Chief Counsel

to appear as amicus curiae (“friend of the court”) in court proceedings to

review rules under the act.

FDA’s Actions Met

Basic RFA

Requirements but

Attracted Criticism

FDA determined that the proposed rule would have a significant economic

impact on a substantial number of small entities. The agency concluded

that a total of 80 small manufacturers and distributors would be affected

by the proposed rule using (1) information in previous studies that

indicated that 95 percent of all dietary supplement manufacturers were

small businesses and (2) information from two market surveys that

identified 85 manufacturers and distributors of dietary supplements

suspected of containing ephedrine alkaloids. This determination triggered

the requirement in sections 603 and 609 of the act, respectively, that FDA

conduct an initial regulatory flexibility analysis and conduct outreach

efforts.

Regulatory Flexibility

Analysis Requirements

FDA’s proposed rule addressed the basic elements that RFA requires

agencies to include in an initial regulatory flexibility analysis. The rule

described the reasons the agency was considering the action (market

failure), stated the purpose and legal basis of the rule, described and

60FDA is not subject to other RFA requirements regarding small business participation, such as the

requirement to convene small business advocacy review panels that the act, as amended by the Small

Business Regulatory Enforcement Fairness Act, places on EPA and the Occupational Safety and

Health Administration. For a discussion of these panels, see Regulatory Reform: Implementation of the

Small Business Advocacy Review Panel Requirements (GAO/GGD-98-36, Mar. 18, 1998).

Page 57 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix V 

Regulatory Flexibility Act Requirements

provided an estimate of the number of small entities to which FDA believed

the rule would apply, and described the compliance requirements. In

addition to describing direct compliance costs of between $3 million and

$80 million, FDA also said that the proposed rule could have significant

distributive effects in the form of reduced sales of as much as $230 million

a year. FDA explicitly stated that costs and sales reductions of this

magnitude might threaten the viability of many firms in this industry.

FDA also discussed significant alternatives to the proposed rule in the rule’s

regulatory flexibility analysis section, noting that most of the regulatory

alternatives discussed in the cost-benefit analysis section would reduce

the impact of this rule on small businesses. For example, FDA noted that

taking no action or simply generating additional information would reduce

the impact on small businesses to zero. However, FDA said that raising the

proposed potency limit would have the same impact as the proposed rule

and banning dietary supplements would have the greatest negative impact.

The industry representatives we met with confirmed that manufacturing

and labeling of these products are generally limited to a relatively small

number of manufacturing firms. Such firms would bear almost all of the

expenditures required for compliance with the proposed rule. However,

the executive director of one of the major trade associations we contacted

pointed out that the proposed actions may also create some compliance

costs for downstream distributors of the many different brands of

products marketed. He said that these distributors might have to revise

their marketing and promotional materials (for example, checking and

revising claims and other information presented in the materials) to

comply with the proposed rule. It is not clear whether FDA’s estimated

costs for relabeling also address this cost element.

In comments submitted on the proposed rule, SBA’s Office of Advocacy

was critical of FDA’s initial regulatory flexibility analysis, stating that the

underlying scientific data and analysis did not adequately support the

proposed actions and the proposed rule did not provide enough

information for someone to “meaningfully comment” on the proposed

rule or alternatives. SBA also criticized FDA’s analysis of the impacts on

small businesses, saying it had underestimated the number of entities

affected because it focused solely on the manufacturers of dietary

supplements suspected of containing ephedrine alkaloids and did not take

independent distributors into account. The Office of Advocacy noted that

“many ephedra products are sold by tens of thousands of home-based

distributorships that are part of multilevel marketing companies. Many of

Page 58 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix V 

Regulatory Flexibility Act Requirements

these businesses, although part of a larger parent company, may

nevertheless be independently owned and operated and considered to be

‘small business concerns’ under the Small Business Act.” The office

pointed out the importance of good information about the regulated

industry, stating,

In order to determine the impact of any regulation, an agency must make a reasonable

effort to identify the type and number of entities likely to be affected by the regulation. This

process of learning about the regulated industry not only helps the agency determine

whether to certify a rule for regulatory flexibility purposes, it also helps the agency develop

an analysis of impacts and choose appropriate regulatory alternatives that minimize

economic burden.

In response to SBA’s criticism, FDA officials told us that the agency need not

consider the impact of its proposed rule on distributors of ephedrine

alkaloids. The FDA officials cited several court decisions to support the

proposition that, under RFA, an agency is under no obligation to conduct a

small entity impact analysis of effects on entities that the agency does not

regulate—that is, distributors.61

None of the major parties—FDA, SBA’s Office of Advocacy, or industry

associations—has information on the actual number of entities involved in

the market for dietary supplements containing ephedrine alkaloids. Each

acknowledge this as a limitation in attempts to analyze the effects of the

proposed rule. FDA officials told us that the agency did not do a true

market study for the proposed rule. The market review that FDA used in its

analysis was done primarily for the purposes of informing the agency

about supplements that might contain ephedrine alkaloids. SBA’s Office of

Advocacy recommended that FDA develop an outreach strategy to obtain

more reliable industry data, and officials of both the Office of Advocacy

and FDA noted that they have met on this issue since the comment period

on the proposed rule ended. In addition, FDA officials said that they have

contracted for a market study of the dietary supplement industry that

should provide better information for future FDA analyses.

Outreach Efforts With regard to the requirement in section 609(a) of the act requiring FDA to

engage in outreach efforts, the regulatory docket shows that FDA provided

opportunities during the development of the proposed rule for

participation by affected businesses. For example, in addition to

publishing the proposed rule, FDA published public notices and held public

61For example, United Distribution Cos. v. FERC, 88 F.3d 1105, 1170 (D.C. Cir. 1996).

Page 59 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix V 

Regulatory Flexibility Act Requirements

meetings, during which industry representatives participated and provided

testimony, and collected written comments on this issue. Representatives

of industry trade associations also told us that FDA had placed notices

about the proposed rule in the trade literature.

Page 60 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VI 

Unfunded Mandates Reform Act

Requirements

The Unfunded Mandates Reform Act of 1995 imposes specific analytical

requirements for certain rules that include a federal mandate that may

result in the expenditure of $100 million or more by the private sector in

any 1 year. FDA did not explicitly address UMRA in its proposed rule.

However, FDA’s economic analysis indicated that the direct expenditures

imposed by this rule on the private sector would not rise to the level of

$100 million and, thus, trigger the required additional analysis.

Furthermore, even if the proposed rule did trigger UMRA’s analytical

requirements, FDA appears to have already satisfied most, if not all, of

those requirements.

UMRA Requirements

Apply Under Certain

Circumstances

UMRA requires federal agencies to take certain actions during the

rulemaking process for certain types of rules. For example, section 202 of

UMRA requires agencies to provide a “written statement,” including a

qualitative and quantitative assessment of the anticipated costs and

benefits for any proposed rule that includes a federal mandate that may

result in the expenditure of $100 million or more by state, local, and tribal

governments, in the aggregate, or by the private sector, in any 1 year.62

Section 205 of the act states that, before promulgating any rule for which a

written statement is required under section 202, agencies must identify

and consider a reasonable number of regulatory alternatives and select the

one that is the least costly, most cost-effective, or least burdensome

alternative that achieves the objectives of the proposed rule. If the agency

does not select the least costly, most cost-effective, or least burdensome

option, and if the requirements of section 205 are not inconsistent with

law, UMRA requires the agency head to publish with the final rule an

explanation of why the least costly, most cost-effective, or least

burdensome method was not adopted.

FDA’s Proposed Rule

Does Not Appear to

Trigger Further Action

Under UMRA

FDA’s proposed rule did not mention UMRA. FDA officials said they did not

include a specific UMRA section because the proposed rule was quite

lengthy and they wanted to avoid adding any additional boilerplate. Our

previous work on federal agency rulemaking indicated that even when

their proposed rules do not trigger UMRA’s analytical requirements, and

although they are not required to do so, agencies sometimes include a

62Even if an agency’s rule required $100 million in expenditures by the public or private sectors,

UMRA’s written statement requirements would only apply if the rule also imposed an “enforceable

duty” that was not a condition of federal financial assistance or that did not arise from participation in

a voluntary federal program. See Unfunded Mandates: Reform Act Has Had Little Effect on Agencies’

Rulemaking Actions (GAO/GGD-98-30, Feb. 4, 1998).

Page 61 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VI 

Unfunded Mandates Reform Act

Requirements

brief statement to that effect explaining why they did not believe those

requirements were applicable.63

Although FDA did not specifically mention UMRA, the agency’s analysis of

the costs associated with the proposed rule indicated that the rule would

not result in expenditures of $100 million or more in any 1 year by the

private sector. FDA estimated that the total compliance costs for the

proposed action would be between $3 million and $80 million, with at

most $70 million of those costs in the first year of implementation. These

costs are primarily associated with relabeling and reformulating the

affected supplements and appear to be expenditures covered by UMRA but

do not rise to the $100-million threshold. Although FDA also estimated that

the rule might result in lost sales for the dietary supplement industry of as

much as $230 million per year, lost sales could not be considered direct

expenditures by the private sector and, therefore, could not be used to

trigger UMRA’s analytical requirements.

SBA’s Office of Advocacy contended that FDA overlooked the UMRA

requirements. In the written comments submitted on FDA’s proposed rule,

the office stated, “If the agency had performed an adequate [cost-benefit]

analysis, it would have been apparent that the economic impact of the

instant rule would impose in excess of $100 million in costs to the

industry.” Officials from the Office of Advocacy told us that, if FDA

properly estimated the number of affected businesses and the costs that

would be imposed on those entities, FDA’s estimate of expenditures by the

private sector might increase to the level that would trigger UMRA’s

requirements. However, the Office of Advocacy could not provide any data

showing that FDA had underestimated the number of manufacturers that

would be affected by the relabeling and reformulation requirements in the

rule or that FDA’s cost estimates for those manufacturers were in error.

Even if the expenditures associated with the proposed rule had triggered

UMRA’s analytical requirements, FDA appears to have satisfied most if not all

of those requirements. For example, FDA identified the provision of federal

law under which the rule was being promulgated, quantitatively and

qualitatively assessed the anticipated costs and benefits of the rule, and

identified and considered a number of regulatory alternatives. Although

FDA said that most of the regulatory alternatives would reduce the impact

of the rule on businesses, FDA also indicated that these other options

would not be as effective as the proposed rule.

63GGD-98-30 and Federal Rulemaking: Agencies Often Published Final Actions Without Proposed Rules

(GAO/GGD-98-126, Aug. 31, 1998).

Page 62 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VII 

Comments From the Department of Health

and Human Services

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

Page 63 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VII 

Comments From the Department of Health

and Human Services

See comment 1.

Now on p. 23.

Now on pp. 23-24.

See comment 2.

Now on pp. 3 and 8.

Now on pp. 8-9.

See comment 3.

Now on pp. 3, 8, 10, and

12.

Page 64 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VII 

Comments From the Department of Health

and Human Services

See comment 4.

Now on pp. 8, 11, 13, and

24.

See comment 5.

See comment 6.

Page 65 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VII 

Comments From the Department of Health

and Human Services

Page 66 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VII 

Comments From the Department of Health

and Human Services

Page 67 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VII 

Comments From the Department of Health

and Human Services

Page 68 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VII 

Comments From the Department of Health

and Human Services

Page 69 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VII 

Comments From the Department of Health

and Human Services

GAO Comments 1. We have added language to the Results-in-Brief section of the report

indicating that the number of AERs relating to dietary supplements

containing ephedrine alkaloids warranted FDA’s attention and

consideration of steps to address safety concerns.

2. Throughout our report, we note that FDA used information from

scientific literature to conclude that ingestion of ephedrine alkaloids can

result in serious adverse events for some individuals. Our review of the

scientific literature found case reports that suggest that ephedrine

alkaloids could increase blood pressure in persons with normal and high

blood pressure; predispose certain individuals to tachycardia (rapid heart

rate); and cause cardiomyopathy (disease of the heart muscle), stroke, or

myocardial necrosis (death of cells in the heart). We also found

descriptions of adverse events associated with ephedrine alkaloids that

affected the central nervous system, such as mania, paranoid psychoses,

and seizures. However, FDA relied on AERs in establishing key components

of its proposed rule—the serving and duration of use limits. We have

clarified some language in the report to better reflect our concerns.

3. FDA highlighted in the proposed rule that clinical trials examining the

efficacy of ephedrine to treat obesity had shown adverse events occurring

at levels approaching 20 mg per serving. However, these trials did not

provide information relating to adverse events below the 20 mg level.

Thus, FDA did rely solely on results of tests performed on a small number

of products implicated in 13 AERs to conclude that an 8 mg per serving

level would be appropriate.

4. We note in our report that FDA did not perform an analysis on the AERs to

determine which, if any, of the reported events were caused by dietary

supplements containing ephedrine alkaloids. Further, we outlined several

weaknesses inherent in any passive surveillance system, such as adverse

event reporting. We continue to hold the view that without a causal link

between the AERs and the ingestion of products containing ephedrine

alkaloids, the exclusive use of AERs to support a specific dosing regimen is

questionable.

5. FDA noted in its comments that we suggested that FDA’s use of AERs in

this rulemaking in some respects, such as establishing an adulteration

level, was unprecedented. For this report, we examined whether the use of

AERs for the proposed rule on dietary supplements containing ephedrine

alkaloids was consistent with the use of AERs in prior rulemaking by FDA’s

Center for Food Safety and Applied Nutrition. We found that AERs had

Page 70 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VII 

Comments From the Department of Health

and Human Services

been used in prior rulemaking, but they were used differently in this

proposed rule when the agency relied solely on these reports to set a

specific dosing level. However, we did not take a position on the

appropriateness of the general use of AERs in rulemaking.

6. While we do not disagree with FDA’s chronology of events detailed in

pages 2-5 of its letter, we do not believe it is fully reflective of the role of

the AERs in FDA’s proposed action. In addition to using the AERs to signal a

potential problem with products containing ephedrine alkaloids and to

identify types of adverse events associated with such products, FDA also

used the AERs as the key element in establishing the proposed dosing

regimen.

Page 71 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VIII 

Comments From the Small Business

Administration

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

Page 72 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VIII 

Comments From the Small Business

Administration

See comment 1.

Page 73 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VIII 

Comments From the Small Business

Administration

Page 74 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VIII 

Comments From the Small Business

Administration

See comment 2.

Page 75 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VIII 

Comments From the Small Business

Administration

Page 76 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VIII 

Comments From the Small Business

Administration

GAO Comments 1. Although we did raise several concerns about specific parts of FDA’s

analysis and recommended certain revisions to any final rulemaking by

FDA, we did not conclude that FDA failed to prepare an adequate analysis.

RFA contains no standards or criteria that define an “adequate” analysis.

The provisions of the act specify certain actions that an agency must take

and certain elements that must be included in a regulatory flexibility

analysis. Beyond those broad parameters, the act does not establish

specific measures of quality or adequacy for an agency’s analysis. Thus, we

have no basis and are not in a position to make such a determination

regarding the quality of FDA’s analysis. Such a determination might be

made if a specific rulemaking is judicially reviewed.64 We did recognize,

however, that FDA’s analysis in the proposed rule contained the primary

elements required under E.O. 12866, related OIRA guidance, and RFA.

2. We disagree with the Office of Advocacy’s comments on direct versus

indirect effects for three reasons.

• First, prior case law indicates that agencies need not take the indirect

effects of their rules into account when conducting a regulatory flexibility

analysis. Most recently, in a May 14, 1999, decision, the U.S. Court of

Appeals for the District of Columbia Circuit said, “we have consistently

interpreted the RFA . . . to impose no obligation upon an agency ‘to

conduct a small entity impact analysis of effects on entities which it does

not regulate.’”65 The court also quoted Mid-Tex in noting, “Congress did

not intend to require that every agency consider every indirect effect that

any regulation might have on small businesses in any stratum of the

national economy.”66 The court also rejected the petitioners’ contention

that it must defer to the interpretations of SBA’s Chief Counsel for

Advocacy regarding RFA, pointing out that SBA “neither administers nor

has any policymaking role under RFA; at most its role is advisory . . . .

Therefore, we do not defer to the SBA’s interpretation of the RFA.”67

• Second, we do not believe that it is logical to conclude that the absence of

case law comparable to Mid-Tex with respect to direct and indirect effects

64The Small Business Regulatory Enforcement Fairness Act of 1996 amended RFA to provide for

judicial review of an agency’s compliance with RFA.

65American Trucking Associations, Inc., et al, v. United States Environmental Protection Agency, No.

97-1440 U.S. App. LEXIS 9064, at *40 (D.C. Cir. May 14, 1999), citing Motor & Equip. Mfrs. Ass’n. v.

Nichols, 142 F.3d 449, 467 & n.18 (1988).

66American Trucking, 1999 U.S. App. LEXIS 9064, at *41.

67American Trucking, 1999 U.S. App. LEXIS 9064, at *42. The court referred to 5 U.S.C. sections 601(3),

602(b), 603(a), 605(b), 609(b)(1), and 612 and Scheduled Airlines Traffic Offices, Inc. v. Department of

Defense, 87 F.3d 1356, 1361 (D.C. Cir. 1996) (no deference owed to agency interpretation of statute it

does not administer).

Page 77 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix VIII 

Comments From the Small Business

Administration

as they apply to E.O. 12866 means that FDA should have prepared an

analysis of the rule’s impact on distributors. The absence of case law says

nothing about FDA’s analytical obligations under the executive order.

• Third, we believe that, by estimating total lost sales to the industry if the

proposed rule or other regulatory alternatives were implemented, FDA

identified the potential magnitude of impacts on distributors in both its

cost-benefit analysis and regulatory flexibility analysis. FDA identified such

lost sales as a distributive effect, rather than a true social cost, because the

loss only represents a transfer of resources within society (more for

consumers and less for the dietary supplement industry). In addition, the

total loss to the industry is the total loss to the industry; it should not be

double counted—once for manufacturers and again for distributors—in a

cost-benefit analysis. Finally, FDA explicitly acknowledged in its proposed

rule that such effects would be significant to affected parties and could

threaten the viability of many firms in this industry.

Page 78 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Appendix IX 

GAO Contacts and Staff Acknowledgments

GAO Contacts William Scanlon, (202) 512-7114

Nye Stevens, (202) 512-8676

Staff

Acknowledgments

In addition to those named above, Tim Bober, Curtis Copeland, Marcia

Crosse, Carolyn L. Feis, and Kurt Kroemer made key contributions to this

report.

(108392, 410378) Page 79 GAO/HEHS/GGD-99-90 FDA’s Proposed Rule on Ephedrine Alkaloids
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100