GAO

United States General Accounting Office

Report to the Ranking Minority Member 

Committee on Veterans Affairs, House of 

Representatives

December 1999 AGENT ORANGE

Actions Needed to 

Improve

Communications of 

Air Force Ranch Hand 

Study Data and 

Results

GAO/NSIAD-00-31

Page 1 GAO/NSIAD-00-31 Agent Orange

Letter 3

Appendixes Appendix I: Objectives, Scope, and Methodology 26

Appendix II: Statistical Power of the Ranch Hand Study 28

Appendix III: Duties of the Ranch Hand Advisory Committee 31

Appendix IV: Roles of Air Force, National Academy of Sciences, 

and Department of Veterans Affairs in Evaluating Health 

Effects of Herbicides 32

Appendix V: Comments From the Department of Defense 34

Appendix VI: Comments From the Food and Drug 

Administration 37

Tables Table 1: Minimum Detectable Effect in 1982 29

Table 2: Minimum Detectable Effect in 1992 29

Table 3: Minimum Detectable Effect in 2002 30

Table 4: Summary of Roles and Criteria in Evaluating

the Health Effects of Herbicides 33

Figures Figure 1: Ranch Hand Study Milestones and Published

Articles and Reports 10

Figure 2: Communication Flow From the Advisory

Committee to Air Force Scientists Conducting the

Ranch Hand Study 18
Page 2 GAO/NSIAD-00-31 Agent Orange
Page 3 GAO/NSIAD-00-31 Agent Orange

United States General Accounting Office

Washington, D.C. 20548

National Security and

International Affairs Division

B-282636 Letr

December 17, 1999

The Honorable Lane Evans

Ranking Minority Member

Committee on Veterans Affairs

House of Representatives

Dear Mr. Evans:

From 1962 to early 1971, the United States sprayed herbicides, including 

Agent Orange, over Vietnam. In the late 1970s, concerns began to emerge 

over the long-term health problems of Vietnam veterans. Although these 

veterans could have been exposed to many potential health hazards, 

including pesticides, infectious diseases, and treatments for tropical 

diseases, attention focused on herbicides. Several herbicides, including 

Agent Orange, contain dioxin,1

 which is known to cause a variety of 

adverse health effects in animals. While there is scientific evidence of some 

associations between exposure to herbicides (or the dioxin they contain) 

and adverse human health conditions, the effect of this exposure on human 

health remains controversial. 

There has been long-standing congressional interest in and concern about 

the effects of exposure to herbicides such as Agent Orange. Congress has 

held at least 26 hearings on the subject since 1978, some of them involving 

scrutiny of scientific studies. One key effort to examine the long-term 

health effects of servicemembers’ exposure to herbicides in Vietnam is an 

ongoing Air Force study known as the Ranch Hand study. This study was 

designed to investigate whether exposure to herbicides in Vietnam led or 

would lead to adverse health. To this end, the study follows the health 

(morbidity) and mortality rates of the Ranch Hands—the almost 1,300 Air 

Force personnel who sprayed herbicides from the air in Vietnam—in 

contrast to those of a comparison group. This comparison group consists 

of Air Force military personnel who were not involved in the spraying and 

who were matched to the Ranch Hands in terms of age, race, and military 

occupation. The 25-year study, which began in 1982 and is scheduled to end 

in fiscal year 2006, is projected to cost over $140 million.

1

The chemical 2, 3, 7, 8-tetrachlorodibenzo-p-dioxin.
B-282636

Page 4 GAO/NSIAD-00-31 Agent Orange

The study’s protocol, which outlines the study’s purpose, methods, and 

procedures, requires the Air Force to disclose periodically the study’s 

findings in official reports and in a journal of stature. The protocol also 

mandated a monitoring group, which currently consists of an Advisory 

Committee of nine scientists (three of whom were nominated by veterans’ 

organizations), to oversee the conduct of the study and provide an 

independent review of the study’s findings. 

Findings from the Ranch Hand study, along with those of other studies 

(e.g., on U.S. and foreign Vietnam veterans, chemical factory workers, or 

populations exposed environmentally to dioxin), are used by the 

Department of Veterans Affairs to determine whether veterans are eligible 

to receive disability compensation for conditions believed to be connected 

to military service. In 1984 and again in 1991, Congress revised the process 

used by the Department of Veterans Affairs to determine whether Vietnam 

veterans are eligible to receive compensation for health effects from 

exposures to herbicides.2

 Since 1991, the Department has been required to 

contract with the National Academy of Sciences to perform reviews of 

scientific literature, including the Ranch Hand study, on the associations 

between diseases and herbicide exposure. Largely on the basis of these 

reviews, the Department of Veterans Affairs determines which diseases 

show a positive association between herbicide exposure and the disease. 

Vietnam veterans with such diseases are then eligible for disability 

compensation.

Since its inception, the Ranch Hand study has been controversial. For 

instance, news articles and statements in the Congressional Record have 

alleged that government officials delayed and withheld information on the 

study’s findings, improperly influenced the study’s design and 

implementation, and failed to provide adequate veterans’ representation on 

the Advisory Committee. In addition, veterans’ organizations such as the 

American Legion and the Vietnam Veterans of America have criticized the 

Department of Defense’s conduct with regards to studies of Agent Orange. 

Recent congressional concerns have centered not on the scientific design 

or implementation of the study but on the proper dissemination and 

reporting of study results and on proper independent oversight. On the 

basis of your concerns, we examined the conduct and findings of the study 

and assessed the study’s impact on determinations of veterans’ 

2

P.L. 98-542 (1984) and P.L. 102-4 (1991).
B-282636

Page 5 GAO/NSIAD-00-31 Agent Orange

compensation. As agreed with your office, we did not evaluate the study’s 

scientific design. Specifically, we (1) assessed whether the study’s findings 

and data have been properly and promptly reported and disseminated; 

(2) examined the statistical limitations of the study and whether they have 

been adequately reported and communicated; (3) examined the measures 

established to monitor the study’s conduct and to prevent improper 

influence, particularly those involving the Advisory Committee; and 

(4) assessed the impact of the study on determinations of diseases for 

which Vietnam veterans are eligible to receive compensation benefits. 

To examine the conduct and findings of the Ranch Hand study, we obtained 

and reviewed the study protocol, various study memoranda and 

correspondence, published study reports and peer-reviewed journal 

articles, executive summaries and Air Force press releases, and other 

available documents related to the study. In addition, we interviewed 

Ranch Hand study investigators, Advisory Committee members, Air Force 

program officials, Department of Veterans Affairs officials, veterans’ 

representatives, and scientists involved in research on the health effects 

associated with exposure to herbicides and dioxin. We also visited Brooks 

Air Force Base to interview members of the Ranch Hand study team and to 

review data collection and reporting procedures. Our scope and 

methodology are described in more detail in appendix I.

Results in Brief To date, the Air Force has conducted all scheduled phases of the Ranch 

Hand study, including up to five full physical examinations of each 

participating Ranch Hand and comparison, and has periodically reported 

the results in official reports. However, there have been some delays over 

the years in the publication of morbidity and mortality findings in scientific 

journals and in the update of a key report on reproductive outcomes. The 

pace of publication has increased in the past few years. No journal articles 

on mortality or morbidity outcomes were published until 1990, even though 

the first mortality and morbidity reports were published in 1983 and 1984, 

respectively, and the Advisory Committee repeatedly recommended that 

such articles be published. Several reasons contributed to the publishing 

delays, including the need to collect and analyze data for five other 

morbidity and mortality reports published between 1985 and 1990. The Air 

Force reported preliminary reproductive outcomes in 1984 but did not 

publish a more detailed update until 1992 because it decided to verify the 

data extensively and perform additional data analyses without releasing 

any interim findings. Although the number of published reports has 

increased, public access to study data is limited by the relatively small 
B-282636

Page 6 GAO/NSIAD-00-31 Agent Orange

amount of data currently available and its storage format (magnetic tape), 

which is difficult to use. Scientists and veterans’ groups critical of the study 

want access to all study data to verify the Air Force’s findings and to 

perform additional analyses. To date, the public can only access data the 

Air Force analyzed in 1987. The Air Force intends to make all other data 

available by the end of year 2000. 

Like many epidemiological studies,3

 the Ranch Hand study has a number of 

inherent limitations, but the Air Force has not clearly or effectively 

communicated these limitations to the public. Two limitations are the 

difficulty in detecting increases in risks of rare diseases (because of the 

small size of the Ranch Hand population) and the fact that the study’s 

findings cannot be generalized to all Vietnam veterans (because Ranch 

Hands and ground troops were exposed to different levels of herbicides in 

different ways). In its first two morbidity reports in 1984 and 1987, 

however, the Air Force described the study’s lack of findings (i.e., few 

diseases with increased incidence among Ranch Hands) as “reassuring,” 

possibly leading to the misinterpretation that the study showed herbicides 

to be safe. Although the Air Force no longer uses such language, it still 

reports findings to the public through press releases and executive 

summaries that do not make the study’s limitations clear. 

During the study’s first several years, some measures intended to ensure 

that the study is conducted independently and without bias were not 

carried out as planned. To ensure independent review and prevent any 

appearance of Air Force management bias, the study protocol (published in 

1982) mandated: (1) that an independent group of scientists (currently the 

Advisory Committee) be established to monitor the study’s conduct and 

(2) that Air Force scientists, subject to review by the monitoring group, 

have primary responsibility over the scientific aspects of the study. 

Shortcomings in the Advisory Committee’s makeup and in the processes 

through which the Committee and others communicated scientific advice 

were not fully corrected until 1989. Until that year, the Committee did not 

include any veterans’ representatives, as required by the study protocol. In 

addition, according to documents we reviewed dating from 1984 and 1985, 

Air Force management and the White House at the time tried to direct 

certain aspects of the Air Force scientists’ research. These attempts 

deviated from the protocol’s requirement that Air Force scientists retain 

3

Scientific studies of the incidence, distribution, or control of diseases in human 

populations.
B-282636

Page 7 GAO/NSIAD-00-31 Agent Orange

primary responsibility over the study’s scientific conduct. Furthermore, the 

White House’s actions bypassed review by the Advisory Committee. 

Although these early problems were resolved through executive and 

congressional actions, some problems remain with the extent of the 

Committee’s outreach to veterans. Although the Committee’s public 

meetings are announced in the Federal Register, the Committee has not 

directly informed veterans’ organizations of these meetings. In addition, the 

Committee’s informal process for soliciting nominations has resulted in not 

all interested veterans’ organizations being notified of opportunities to 

nominate Committee members.

The Ranch Hand study has had significant impact on one decision 

regarding compensation to Vietnam veterans. Partly on the basis of the 

study’s results, Congress passed legislation providing compensation for 

veterans’ children born with spina bifida.4

 The study has not contributed to 

decisions by the Department of Veterans Affairs to compensate Vietnam 

veterans for any other diseases. Because the Ranch Hand study’s small 

sample size provides limited potential for addressing possible links 

between herbicide exposure and many forms of cancer, it has not 

contributed to any National Academy of Sciences or Department of 

Veterans Affairs recommendations regarding cancer. The finding of a 

possible association between herbicide exposure and diabetes mellitus 

(diabetes), first reported by the Ranch Hand study in 1991, has led to 

further review of the issue by the Academy, as well as by Air Force and 

other scientists. However, the Department of Veterans Affairs has not yet 

determined that Vietnam veterans with the disease are eligible for 

compensation. 

We are recommending several additional actions intended to improve 

communication of Air Force Ranch Hand study data and results.

Background Before the Ranch Hand study began, official government and 

nongovernment reviewers of the study protocol expressed concern that the 

public would perceive the study as not credible because the fact that the 

Air Force would conduct the study might give the appearance of conflict of 

interest. An interagency working group that reviewed the study concluded, 

however, that the appearance of conflict of interest could be properly and 

4

Spina bifida is a birth defect in which the vertebral column (backbone) fails to close and 

that may allow herniation of the spinal cord.
B-282636

Page 8 GAO/NSIAD-00-31 Agent Orange

adequately addressed through independent review and monitoring. Thus, 

an Advisory Committee was created to regularly review and assess the 

conduct of the study, review all changes to the study protocol, and provide 

written comments and recommendations on the study’s findings. The 

Committee, chartered by the Department of Health and Human Services, 

was initially responsible to the Agent Orange Working Group, an 

interagency group that monitored all government activities related to 

herbicides such as Agent Orange.5

 The Agent Orange Working Group was 

headed by the Department of Health and Human Services and included 

representatives from many government agencies and offices, including the 

Department of Defense, the Department of Veterans Affairs, and the White 

House Office of Science and Technology Policy. In 1989, the Department of 

Health and Human Services transferred administration of the Committee to 

the Food and Drug Administration. The duties of the Advisory Committee 

are described in more detail in appendix III.

The process of translating scientific evidence into policy decisions on 

compensation benefits for veterans has also been controversial. Scientific 

studies typically strive to produce evidence of a “cause-and-effect” 

relationship, and use a relatively high standard of proof (e.g., 95-percent 

confidence). In 1989, a federal court ruled that the Department of Veterans 

Affairs’ use of similar standards in making compensation decisions was too 

restrictive.6

 According to the court, Congress had intended the Department 

of Veterans Affairs to require only a “statistical association” with a disease. 

Furthermore, the court concluded that veterans were to be given the 

benefit of the doubt in compensation decisions. Subsequently, in 1991, 

Congress established the current process for these decisions, including 

biennial reviews of the scientific literature by the National Academy of 

Sciences.7

 The Academy has conducted three such reviews for Agent 

Orange, the most recent published in 1999. Additional information on the 

roles of the Ranch Hand study, the National Academy of Sciences, and the 

Department of Veterans Affairs is in appendix IV.

5

The Agent Orange Working Group was replaced by the Agent Orange Task Force in 1990. 

The final meeting of the Agent Orange Task Force occurred in 1994.

6

Nehmer v. U.S. Veterans’ Administration, 712 F. Supp. 1404 (N.D.Cal.1989).

7

P.L. 102-4 (1991).
B-282636

Page 9 GAO/NSIAD-00-31 Agent Orange

Publication of Study 

Results and Release of 

Study Data Were Slow 

in the Early Years

Publication of study findings in peer-reviewed journals was slow to begin, 

but the number of articles has increased considerably in recent years (see 

fig. 1). An update of a report on reproductive outcomes was delayed to 

allow the completion of additional analyses and data collection. The Air 

Force has also been slow to release study data to the public. Only analyzed 

data from one physical examination (not all five examinations conducted 

thus far or all collected data) is currently available. In addition, the data is 

available only on magnetic tape rather than a more easily accessible format 

such as compact disc. 
B-282636

Page 10 GAO/NSIAD-00-31 Agent Orange

Figure 1: Ranch Hand Study Milestones and Published Articles and Reports

Note: “Journal articles” are articles (not including letters, comments, and errata) included in the 

National Library of Medicine’s MEDLINE database as of October 1999. “Reports” are official 

government reports published by the Air Force.

*: Data currently publicly available.

Source: Air Force, MEDLINE.
B-282636

Page 11 GAO/NSIAD-00-31 Agent Orange

Publication of Study 

Findings Slow to Begin

In spite of the requirement in the study protocol as well as guidance from 

the Advisory Committee that study findings be disclosed periodically in 

scientific journals, no journal publications appeared until 1989, nearly 

7 years after the study had begun. Publication of study results on morbidity 

or mortality did not begin in scientific journals until 1990. The study 

protocol initially mandated that the Air Force issue official reports on 

morbidity after each physical examination and on mortality at least every 

5 years and that the results of the study be published in a journal of stature. 

The Air Force did not publish journal articles, however, until after it had 

obtained the third round of physical examinations in 1987 and after several 

morbidity and mortality reports had been released. Several reasons 

contributed to the delay, including the need to collect and analyze data for 

five other morbidity and mortality reports published between 1985 and 

1990. The Advisory Committee repeatedly urged publication of study 

findings in journals. In 1986, it recommended that results be published in 

the “open literature,” and in 1987 it repeated the recommendation, stating 

that “first and foremost, there is an urgent need to publish in widely read 

peer-reviewed medical journals.” The pace of article publication has 

accelerated since 1990. Eleven articles on morbidity and mortality had 

been published by mid-1999, and Air Force scientists told us that over a 

dozen articles are currently in submission or preparation for publication in 

peer-reviewed journals. 

Update of Report on 

Reproductive Outcomes 

Was Delayed

An update of a preliminary reproductive outcome report was delayed 

8 years because the Air Force performed additional data collection and 

analyses without releasing any interim information. The Air Force first 

reported preliminary reproductive outcomes in a baseline morbidity study 

in 1984. It subsequently began verifying all medical records to check for 

birth defects instead of relying on reports of interviews with Ranch Hands, 

comparisons, and their spouses, which can be inaccurate. The Air Force 

anticipated completion of the effort within 1 year. Although a draft interim 

update, using partially verified data, was prepared later in 1984, the 

Advisory Committee suggested it not be released before additional work 

was done. The Air Force later expanded its verification of medical records 

through age 18 (rather than up to age 1) to identify birth defects that may 

not have manifested themselves earlier. This expanded verification was not 

completed until late 1988. From 1987 through 1990, the Air Force (in 

collaboration with the Centers for Disease Control and Prevention) 

measured dioxin levels in the blood of all study participants as part of its 

morbidity study and decided to analyze birth defects also using these 

measurements before publishing a report. The reproductive outcome 
B-282636

Page 12 GAO/NSIAD-00-31 Agent Orange

update with all the additional analyses and data was finally published in 

1992. 

Although the verification and additional analyses ultimately strengthened 

the scientific basis of the reproductive outcome portion of the study, the 

length of the delay, in conjunction with the lack of any interim report, led to 

concerns by some veterans and scientists that the study had been 

purposely delayed to suppress information. One reason underlying the 

concern was the apparently high standard of verification. Through our 

review of other birth defect studies and interviews with scientists, we 

found that the degree of birth defects verification the Air Force 

accomplished was highly unusual and virtually unprecedented for a study 

of its size. We examined two other studies of birth defects in children of 

Vietnam veterans. One of them verified only the health status of babies at 

birth, the other verified only those with reported birth defects through age 

1, not those reported as healthy. Because the Ranch Hand study produced 

evidence that led Congress to authorize compensation for Vietnam 

veterans whose children have spina bifida, one veterans’ organization told 

us that earlier reporting, even just of preliminary findings, might have led to 

earlier compensation. In addition, one scientist told us that such 

preliminary information could have been useful for family planning or 

prenatal care decisions. 

Public Access to Study Data 

Is Currently Limited 

Two major factors currently limit public access to study data: (1) only one 

set of analyzed data—not all sets of analyzed data or all collected data—is 

available and (2) data is only available on magnetic tape, not on a more 

common format such as compact disc or the Internet. Two scientists and 

one veterans’ group told us they were interested in obtaining study data. 

They said that access to study data would enable them to verify the 

accuracy of the Air Force analyses and to perform additional analyses on 

the data. They cited the limited amount of study data currently available 

and its format among the barriers to their use of study data for these 

purposes. 

Currently, only data files from Air Force analyses of 1987 physical 

examination data are available, while data files from 1982, 1985, 1992, and 

1997 are not. The Air Force database consists of two types of files: (1) raw 

data files, which include all the data collected at physical examinations, 

and (2) analysis data files, which were used to perform analyses for study 

reports. The content of raw and analysis data files overlap, but each 

contains data not included in the other. The data currently available to the 
B-282636

Page 13 GAO/NSIAD-00-31 Agent Orange

public are analysis files from 1987. Raw data files from 1987 and 1992 have 

been processed by the Air Force for release and are in the process of 

becoming publicly available. Air Force officials report that they expect all 

raw and analysis data files, including reproductive outcome data, to be 

available by the end of year 2000. One scientist and one veterans’ group 

representative told us they believe the Air Force should release all the 

study’s data to permit additional analysis of the data. According to these 

individuals, release of the data is particularly important because of past 

allegations that government officials may have improperly influenced the 

study’s conduct.

Currently, Ranch Hand study scientists send data on computer diskettes to 

the Defense Technical Information Center, which transfers the data to 

magnetic tape and sends it to the National Technical Information Service of 

the Department of Commerce. The Service said it provides the data to the 

public in the same format in which it is received, and charges about $450 to 

cover costs. In comparison, two other major government health databases 

we examined are both available on compact disc. These are the National 

Cancer Institute’s “Surveillance, Epidemiology, and End Results” database 

of cancer incidence throughout the United States, available free of charge 

from the Department of Health and Human Services, and the National 

Health and Nutrition Examination Survey, a Centers for Disease Control 

and Prevention database of the health and nutritional status of the U.S. 

population, that can be purchased for $20 from the Government Printing 

Office. Several veterans’ representatives said that they do not have the 

necessary computer equipment to use magnetic tapes, and one scientist 

told us that other formats such as compact discs would be easier to use. Air 

Force officials also told us they are planning to release the data on the 

Internet but noted that the National Technical Information Service would 

remain their primary means of release.

Study Limitations Have 

Not Been Clearly 

Communicated to the 

Public

All scientific studies have inherent limitations. Two major limitations of the 

Ranch Hand study are the difficulty in detecting increases in risks of rare 

diseases (because of the small size of the Ranch Hand population) and the 

fact that the study’s findings cannot be generalized to all Vietnam veterans 

(because Ranch Hands and ground troops were exposed to different levels 

of herbicides in different ways). In spite of the study’s sensitive and 

controversial nature, early study reports contained language that may have 

been misinterpreted to mean that the study showed herbicides were safe. 

Although the Air Force no longer uses such language, recent press releases 
B-282636

Page 14 GAO/NSIAD-00-31 Agent Orange

and executive summaries still do not clearly communicate the study’s 

limitations to the public. 

Two Major Study 

Limitations

The small number of participants in the Ranch Hand study—currently 

fewer than 1,200 Ranch Hands and less than 1,800 comparisons—makes it 

very difficult to detect a statistically significant increase in the risk of rare 

diseases. For instance, the lifetime risk of non-Hodgkins lymphoma 

(a cancer often linked to herbicide exposure) in Caucasian males in the 

United States is about 2 percent. Even if the Ranch Hand study were to 

follow participants through their entire lives (which it does not), because 

of the requirement for statistical significance, it could not confidently 

report any adverse effect unless the incidence of the disease in Ranch 

Hands were at least 3.4 percent. Appendix II contains a more detailed 

explanation of the study’s statistical limitations as they relate to the size of 

its study population and the detectable increased risk of cancer.

Another limitation is that it is difficult to generalize the results of the study 

to all Vietnam veterans because Ranch Hands were exposed to herbicides 

in different ways than most ground troops in Vietnam. For instance, by 

comparing dioxin blood levels with Ranch Hands’ reported exposures to 

herbicides, Air Force scientists found evidence that Ranch Hands were 

exposed to herbicides primarily through their skin. Many ground-troop 

veterans, however, reported exposure to herbicides through other 

mediums such as contaminated food and water and contaminated clothing 

worn for extended periods of time. They believe that Ranch Hands were 

only minimally exposed through these mediums because they consumed 

uncontaminated food and water, wore clean clothes, and could wash 

regularly. These differences have not been extensively studied, so little is 

known about their potential effects. 

However, blood measurements of dioxin levels suggest, though not 

conclusively, that Ranch Hands’ exposure levels were significantly higher 

than both those of the study’s comparison group and of many ground 

troops who have been tested.8

 Because the comparison group was not 

involved in spraying herbicides, its low dioxin levels were consistent with 

8

Since 1991, the study’s morbidity reports have concentrated on analyzing the association 

between dioxin levels and health rather than the health contrast between Ranch Hands and 

the comparison group. Almost half of the Ranch Hands have dioxin levels similar to those of 

the comparison group, further reducing the study size when applying the analysis of dioxin 

levels versus health.
B-282636

Page 15 GAO/NSIAD-00-31 Agent Orange

lack of exposure to herbicides. Congress had proposed a program to test 

veterans’ blood for dioxin, under the condition that such a program would 

make a contribution to scientific knowledge, but the National Academy of 

Sciences stated that such testing would not be useful unless it were part of 

a specific scientific study. Therefore, most Vietnam veterans’ blood dioxin 

levels are untested. Furthermore, according to several scientists we 

interviewed and scientific documents we reviewed, low levels of dioxin in 

the blood today do not guarantee that an individual had low amounts of 

exposure to herbicides in Vietnam. Reasons include (1) different herbicides 

contain different amounts of dioxin; (2) the general population is also 

exposed to detectable levels of dioxin masking exposures during the 

Vietnam War; (3) dioxin is slowly excreted from the body, so subjects may 

have had higher dioxin levels in the past; and (4) the rate at which dioxin is 

excreted differs, depending on the amount of body fat, so individuals with 

similar dioxin levels today may have had different exposures in the past.

Early Reports Could Have 

Led to Misinterpretation of 

Findings to Mean 

Herbicides Were Safe

The importance of communicating the study’s limitations to the public has 

been stressed and highlighted by a number of scientists and officials over 

the years. As far back as 1980, the predecessor of the Agent Orange 

Working Group9

 recommended that the study proceed only on condition 

that the public “clearly understand that the stated health goal in the Air 

Force [Ranch Hand study] Protocol may not be fully realized” and that this 

“does not imply flaws in protocol design; it is to emphasize the inherent 

limitation of the study size.” In addition, in 1984 the White House Office of 

Science and Technology Policy urged the Air Force to respond to criticisms 

and perceived misunderstandings of the study by discussing more 

explicitly “the meaning of the number of cases found (or not found) ... for 

each disease included in the study,” adding that “the discussion should be 

in terms of the confidence interval of these numbers and their comparison 

to general population data on these diseases.” 

In its early reports, and particularly in executive summaries and press 

releases, the Air Force used language that may have led the results of the 

Ranch Hand study to be interpreted inaccurately. As far back as 1980, a 

review panel of government scientists emphasized that “it needs to be 

clearly understood that failure to identify increased risk in a variety of 

health parameters is to be interpreted as inconclusive and not necessarily a 

9

The Scientific Panel of the Interagency Work Group on Phenoxy Herbicides and 

Contaminants.
B-282636

Page 16 GAO/NSIAD-00-31 Agent Orange

true lack of effect” (emphasis in original). However, in describing the 

results of the first two morbidity reports, published in 1984 and 1987, the 

Air Force described the overall study conclusions as “reassuring” because 

they detected few statistically significant increased risks. The use of such 

language, as well as a statement by the Deputy Air Force Surgeon General 

at a press conference in 1984 saying that the results showed “nothing that 

would keep us from using it [herbicides such as Agent Orange] again,” may 

have led to the misinterpretation that the Ranch Hand study proved that 

herbicides were safe. Since 1990, the Air Force has not used this type of 

language. Current Air Force scientists acknowledge that the word 

“reassuring” in particular may have led to improper interpretations. 

Study Limitations Not Fully 

and Clearly Communicated 

to the Public in Executive 

Summaries and Press 

Releases

We reviewed all Air Force reports, executive summaries, and press releases 

on the Ranch Hand study and found that the Air Force still reports study 

results to the public without consistently making clear that any lack of 

conclusive evidence may be due to the study’s limitations—not necessarily 

a lack of long-term health effects from herbicides. While study limitations 

have been addressed in the full text of the reports, they have not been 

consistently explained in executive summaries and press releases, which 

are more accessible to the general public. For example, the executive 

summary and press release for the baseline morbidity study, published in 

1984, only stated that statistical limits exist, and gave no further 

description. The executive summary of the third morbidity report, 

published in 1990, stated that the study had a good chance of detecting a 

statistically significant effect only if a high number of study subjects 

(i.e., 5 percent or more in the comparison group, and 10 percent or more in 

the Ranch Hands) contracted a particular disease. In addition, only in the 

executive summary and press release of its 1991 morbidity report did the 

Air Force mention study limitations due to the differences in exposure 

routes and levels between Ranch Hands and ground troops. However, in 

this case, the Air Force only stressed the limits with which any positive 

evidence of ill health could be applied to other Vietnam veterans and did 

not mention the limits on what conclusions could be drawn from a lack of 

evidence in the study. All other press releases and executive summaries 

have not mentioned study limitations at all. 

Several scientists we spoke with also voiced concern about the Air Force’s 

public reporting of study results. One scientist with expertise in dioxin 

research told us that Air Force press releases and executive summaries 

continue to be misleading because they are not adequately balanced by an 

account of the limitations of the study. A scientist on the Advisory 
B-282636

Page 17 GAO/NSIAD-00-31 Agent Orange

Committee also commented at a Committee meeting in August 1999 that 

the draft of the upcoming Air Force morbidity report stressed the study’s 

lack of findings too strongly and did not adequately discuss the limits to 

inferring either positive or negative effects. A former Air Force scientist, 

however, told us that the Air Force’s public reporting of study findings 

should be viewed in the context of media and unsubstantiated reports of 

major incidences of disease and death caused by herbicides.

Many Early Oversight 

Problems Have Been 

Resolved, but Some 

Veterans Groups Want 

More Outreach

Two measures specified by the study protocol to ensure independent 

review and prevent any appearance of Air Force management bias were not 

followed prior to 1989. These were the requirements that the Advisory 

Committee include scientists nominated by veterans’ organizations and 

that Air Force scientists, subject to review by the Advisory Committee, 

retain primary responsibility over scientific aspects of the study. Executive 

agencies (in 1985 and 1989) and Congress (in 1988 and 1990) took actions 

to correct these problems. However, problems remain with how well the 

Advisory Committee keeps veterans’ organizations informed of its 

activities.

Early Problems With the 

Advisory Committee’s 

Composition and the 

Communication of 

Scientific Advice

Prior to 1989, the Advisory Committee did not fulfill a key requirement of 

the study protocol: that it include scientists nominated by veterans’ 

organizations. The requirement stated that about one-third of Committee 

members should be scientists nominated by veterans’ organizations. 

According to our review, the Committee did not include any such scientists 

from 1982 through 1989. Documents we reviewed indicate that one reason 

was that the Agent Orange Working Group, which was responsible for 

administering the Committee, appeared to view the protocol’s description 

of the Advisory Committee as only giving a suggestion.

The second problem that existed before 1989 was that outside entities tried 

to direct how Air Force scientists conducted certain aspects of the study, 

even though the study protocol specified that Air Force scientists “are 

responsible for ... all data analysis, and for all data interpretation of 

analyses subject to review by the outside monitoring group.” This problem 

stemmed from the fact that before 1989, the Committee did not 

communicate its formal recommendations directly to Air Force scientists 

conducting the study. Rather, the Committee gave its formal 

recommendations first to the Agent Orange Working Group for approval, 

creating opportunities for others to influence the study. Figure 2 shows the 
B-282636

Page 18 GAO/NSIAD-00-31 Agent Orange

flow of communications from the Committee to study scientists before and 

after 1989. 

Figure 2: Communication Flow From the Advisory Committee to Air Force Scientists Conducting the Ranch Hand Study

Note: Solid lines indicate formal communication (e.g., agency memoranda). Dotted lines indicate 

informal communication (e.g., verbal comments or meeting minutes).

Source: Air Force.

Documents we reviewed from the period before 1989 indicate that the Air 

Force and the White House Office of Science and Technology Policy 
B-282636

Page 19 GAO/NSIAD-00-31 Agent Orange

(a member of the Agent Orange Working Group) tried to direct certain 

aspects of the Air Force scientists’ research. For instance, in a 

memorandum sent in October 1984, the commander of the School of 

Aerospace Medicine (where the study is conducted), Brooks Air Force 

Base, ordered the Air Force scientists to perform certain analyses for 

mortality reports and to present the results in a specific format. The 

memorandum noted that the direction came from the Air Force Deputy 

Surgeon General and that it was “in accordance with the comments of the 

Advisory Committee.” However, according to the study protocol, Air Force 

scientists, not Air Force management, have ultimate responsibility for all 

data analysis and interpretations. One of the scientists involved at the time 

in the study expressed concern that the actions ordered by the commander 

would weaken the study or de-emphasize the significance of the results. In 

addition, in a letter also sent in 1984, the White House Office of Science and 

Technology Policy wrote that it had “directed that the Air Force undertake 

... actions,” including the extensive verification of reported birth defects. 

This verification contributed to the 8-year delay in updating study findings 

on reproductive outcomes. Again, this direction infringed on the Air Force 

scientists’ decision-making responsibilities in the scientific conduct of the 

study. Furthermore, the White House later provided guidance directly to 

the Air Force, requesting that the Advisory Committee ensure that this 

guidance be implemented. However, the study protocol only authorized the 

Advisory Committee to provide written comments and recommendations 

on the conduct of the Ranch Hand study to the White House Office of 

Science and Technology Policy, not the reverse. The protocol also required 

that all scientific aspects of the study undergo review by the Advisory 

Committee. Both Air Force management and the Advisory Committee later 

expressed concern about the appropriateness of the White House’s 

communication to the Air Force scientists and sought clarification from the 

Agent Orange Working Group. 

Many Early Problems 

Solved by 1990

In 1985, the Agent Orange Working Group decided that all 

recommendations intended for the Air Force scientists conducting the 

study should be communicated first to the Committee. This decision was 

made primarily in response to concerns about conflicting advice received 

by the Air Force scientists from bodies such as the White House Office of 

Science and Technology Policy. We found no evidence that any 

recommendations made since then have bypassed the Committee. In 1988, 

Congress mandated that veterans’ organizations be allowed to nominate
B-282636

Page 20 GAO/NSIAD-00-31 Agent Orange

three of the Committee’s nine members,10 and we verified the Committee’s 

compliance with this requirement. In 1989, the Department of Health and 

Human Services transferred administration of the Committee from the 

Agent Orange Working Group to the Food and Drug Administration, thus 

eliminating the Agent Orange Working Group as an intermediary in the 

chain of communication. Congress passed additional legislation in 

1990 stating that “no officer or employee of the Federal Government may 

interfere in or impair direct communication between the Advisory 

Committee and ... [the Air Force] scientists.”11

Remaining Problems in 

Informing Veterans of 

Committee Activities

Although procedural problems in the early years of the study have been 

resolved, veterans’ groups and several scientists we spoke with continue to 

question the credibility of the study. In addition, representatives of 

veterans’ groups told us that the Committee’s communication with 

veterans’ organizations has been intermittent and inconsistent. Two 

veterans’ groups told us that they had not been notified repeatedly by either 

the Air Force or the Advisory Committee of public Committee meetings. 

Although they acknowledged that public Committee meetings are 

announced in the Federal Register, they stated that similar committees, 

such as those run by the Department of Veterans Affairs or the National 

Academy of Sciences, are more proactive in keeping veterans informed of 

their activities. One group, the Vietnam Veterans of America, which 

represents veterans with the most interest in herbicides such as Agent 

Orange, told us that lack of notification persisted even after it had 

requested directly to the Committee that it be informed of meetings in 

advance. 

In addition, the process for soliciting nominations for members of the 

Advisory Committee has been relatively unstructured and has led to 

incomplete dissemination of Committee membership openings, as well as 

inaccuracies in the identification of veterans’ nominees. The Food and 

Drug Administration is responsible for soliciting nominations from 

veterans’ organizations. However, openings have not been widely 

publicized. Rather, the Food and Drug Administration reported using an 

informal process, which involves asking previous Committee consultants 

to become nominees and telephone calls to some veterans’ organizations. 

10P.L. 100-687 section 1205 (1988).

11P.L. 101-510 section 1468 (1990).
B-282636

Page 21 GAO/NSIAD-00-31 Agent Orange

The Vietnam Veterans of America told us that as a consequence, it has 

never been asked to nominate Committee members. Our review of 

Committee documents showed that veterans’ organizations that have 

nominated Committee members include the American Legion, the Veterans 

of Foreign Wars, and the Ranch Hand Association. In addition, our review 

revealed that one current Committee member was incorrectly identified as 

having been nominated by a veterans’ organization and that another 

Committee member who was nominated by a veterans’ organization was 

not identified as such.

Study Has Had Limited 

Impact on 

Determination of 

Diseases for Which 

Veterans Are Eligible to 

Receive Compensation

The Ranch Hand study had significant impact on the decision to provide 

compensation for veterans’ children born with spina bifida. The study has 

not contributed to decisions by the Department of Veterans Affairs to 

compensate veterans for any other diseases. The finding of a possible 

association between herbicide exposure and diabetes mellitus, first 

reported by the Ranch Hand study in 1991, has led to greater discussion by 

the National Academy of Sciences and further study by Air Force and other 

scientists, but Vietnam veterans with the disease are not yet eligible for 

compensation. 

Study Contributed to the 

Decision to Compensate for 

Spina Bifida

The most significant impact of the Ranch Hand study to date has been on a 

decision to allow compensation to Vietnam veterans’ children born with 

spina bifida. According to a scientist at the National Academy of Sciences, 

because of the high quality of the Ranch Hand birth defect study, the 

Academy decided in 1996 to upgrade its evaluation of evidence for an 

association between herbicide exposure and spina bifida from 

“insufficient/inadequate” to “limited/suggestive.” Subsequently, the 

Department of Veterans Affairs requested, and Congress approved, 

legislation allowing the Department of Veterans Affairs to provide 

compensation to Vietnam veterans’ children with the disease.
B-282636

Page 22 GAO/NSIAD-00-31 Agent Orange

Study Has Not Contributed 

to Decisions to Allow 

Compensation for Other 

Diseases 

The Ranch Hand study has had limited impact on either National Academy 

of Sciences reports or Department of Veterans Affairs decisions concerning 

associations between herbicide exposure and other diseases.12 Although 

findings from other studies have enabled veterans with any of several 

cancers to be eligible for compensation, the Ranch Hand study has had 

almost no impact on these decisions because of the small size of the Ranch 

Hand population and the relative rarity of many cancers. In addition, 

scientific studies such as the Ranch Hand study are involved only in 

determining for which diseases veterans may be eligible for compensation, 

not in any other part of the compensation process. There are other 

requirements, such as obtaining medical diagnosis of a disease and 

determination of disability, that Vietnam veterans must satisfy in order to 

receive compensation. Moreover, reports and articles by the Ranch Hand 

study comprise only a small fraction of the information the National 

Academy of Sciences reviews and the Department of Veterans Affairs then 

considers when weighing scientific evidence. 

A finding of a possible association between herbicide exposure and 

diabetes mellitus was first suggested in a Ranch Hand study report in 1991. 

This and subsequent findings by the study have led to increased research 

on the subject and more reviews by the National Academy of Sciences. 

Although in its February 1999 report the Academy described the evidence 

of an association between herbicide exposure and diabetes mellitus as 

“equivocal,” the Academy still concluded, as it did in its earlier reports, that 

there was “inadequate/insufficient” evidence for such an association. 

However, a Task Force within the Department of Veterans Affairs 

recommended in April 1999 to allow Vietnam veterans with diabetes 

mellitus to become eligible for compensation. The Task Force wrote that 

because the weights of positive and negative evidence were approximately 

equal, the requirement to give veterans the benefit of the doubt (as stated 

by law) necessitated providing compensation for diabetes mellitus. The 

Department of Veterans Affairs has asked the Academy to conduct an 

additional evaluation of the scientific literature on the subject. The 

evaluation is scheduled to be completed by the end of 1999. 

12 Currently, the Department of Veterans Affairs allows compensation for nine diseases 

(including six cancers) in veterans and one birth defect in children of veterans.
B-282636

Page 23 GAO/NSIAD-00-31 Agent Orange

Conclusions The Air Force has conducted many aspects of the Ranch Hand study in a 

rigorous manner. However, some past and present problems in the conduct 

of the study have led some veterans to view the study with suspicion and to 

express concerns about the study’s conduct. In particular, several veterans’ 

groups and scientists are critical of the study’s methods and results and 

want greater access to study data. Until all study data is publicly available, 

people critical of the study cannot fully verify the Air Force’s analyses. 

Making all study data publicly available would increase the credibility of 

Ranch Hand study results. In addition, without full access to data, outside 

scientists cannot perform alternative or additional analyses, which would 

facilitate open scientific debate on the merits of the Air Force’s 

methodologies and analyses. The Air Force anticipates making all its study 

data available to the public by the end of year 2000, but those with interest 

in the data say that unless the data is available in a more easily accessible 

format (such as compact disc or the Internet), they would have difficulty in 

using it.

In addition, while communication of study limitations to the public has 

improved over the years, additional improvements are possible. Lack of 

sufficient knowledge of the study’s limitations can lead to misleading or 

incorrect interpretations of the study’s findings. Language used in early 

reports that may have led to such interpretations is no longer being used, 

but because of the study’s sensitive and controversial nature, it is important 

that publicly accessible documents such as executive summaries and press 

releases provide accurate information on the study’s limitations. 

Early problems in the Advisory Committee’s composition and 

communications have been remedied, but problems remain with the 

Committee’s outreach to veterans. The Committee’s reliance on indirect 

means, such as the Federal Register, to notify veterans’ organizations of its 

activities perpetuates the impression that the Committee does not seek 

veterans’ input. Moreover, better notification of Committee openings would 

help ensure that veterans’ groups perceive the Committee as fulfilling its 

role as an independent and unbiased oversight body.

Recommendations To facilitate public access to study data and more effective communication 

of study limitations, we recommend that the Secretary of Defense direct 

the Air Force scientists in charge of the Ranch Hand study, in consultation 

with the Advisory Committee, to 
B-282636

Page 24 GAO/NSIAD-00-31 Agent Orange

• establish and publicize a timetable for the release of all study data and 

provisions to release the data in a format (such as compact disc or the 

Internet) that is easily accessible to the general public and 

• include more information on the study’s limitations in press releases and 

executive summaries, which should address the limited applicability of 

study results (especially negative results) to other Vietnam veterans and 

the limited ability of the study to detect small to moderate increases in 

risks of rare diseases.

In addition, to facilitate dissemination of information on the Advisory 

Committee’s activities to veterans, we recommend that the Secretary of 

Health and Human Services direct the Committee’s Executive Secretary at 

the Food and Drug Administration to provide direct and timely notification 

to veterans’ organizations of scheduled Advisory Committee meetings and 

of opportunities for veterans to nominate Committee members.

Agency Comments In written comments on a draft of this report, the Department of Defense 

concurred with our recommendations and indicated that it is taking steps 

to address them. The Department also commented on our finding that the 

publication of study findings in scientific journals was slow to begin, 

stating that the time required to write and publish journal articles is 

typically 3-5 years and that work on the papers published in 1990 actually 

started in 1986. While we recognize that the publication process can be 

lengthy, it should be noted that the journal articles published in 1990 only 

included analysis of data collected in 1987 and reported in 1990 in an 

official Air Force report. No journal articles were published earlier based 

on data collected in 1982 and 1985 and respectively released in Air Force 

reports in 1984 and 1987. Other comments from the Department were 

technical in nature, and we incorporated them where appropriate. 

Comments from the Department of Defense are reproduced in their 

entirety in appendix V.

The Food and Drug Administration also provided written comments on a 

draft of this report. It stated that in general it found the report well done 

and accurate. With regard to our recommendation that the Secretary of 

Health and Human Services direct the Advisory Committee’s Executive 

Secretary to facilitate dissemination of information to veterans, it indicated 

that they are working with the Office of Veterans Affairs and Military 

Liaison of the Department of Health and Human Services to ensure that 

veterans’ organizations are notified in a timely manner of Advisory 

Committee meetings and vacancies. The Food and Drug Administration 
B-282636

Page 25 GAO/NSIAD-00-31 Agent Orange

also provided technical comments that we incorporated where 

appropriate. Comments from the Food and Drug Administration are 

reproduced in their entirety in appendix VI.

As agreed with your office, unless you publicly announce its contents 

earlier, we plan no further distribution of this report until 30 days from its 

issue date. At that time, we will send copies of this report to other 

appropriate congressional committees. We will also send copies to the 

Honorable William S. Cohen, Secretary of Defense; the Honorable 

F. Whitten Peters, Secretary of the Air Force; and the Honorable 

Donna E. Shalala, Secretary of Health and Human Services. In addition, we 

will make copies available to others upon request. 

If you have any questions about this report, please call me at

(202) 512-3652 or John Oppenheim at (202) 512-3111. Weihsueh Chiu was a 

key contributor to this report.

Sincerely yours,

Kwai-Cheung Chan

Director

Special Studies and Evaluations
Page 26 GAO/NSIAD-00-31 Agent Orange

Appendix I

Objectives, Scope, and Methodology AAppenednxidxeIis

Our objectives were to (1) assess whether the study’s interim findings and 

data have been properly and promptly reported and disseminated, 

(2) examine the statistical limitations of the study and whether they have 

been adequately reported and communicated, (3) examine the measures 

established to monitor the study’s conduct and to prevent improper 

influence, and (4) assess the impact of the Ranch Hand study on National 

Academy of Sciences reports and Department of Veterans Affairs 

determinations regarding diseases for which Vietnam veterans are eligible 

to receive compensation benefits. 

To assess whether the study’s interim findings and data have been properly 

and promptly reported and disseminated, we reviewed the study protocol 

and memoranda and correspondence by the Air Force and the Advisory 

Committee to determine the schedule for reporting study findings. In 

addition, we reviewed these documents to examine the reasons for any 

delays. We also obtained and reviewed published study reports and 

peer-reviewed articles to determine the actual date of these publications. 

We conducted site visits to Brooks Air Force Base to interview scientists in 

charge of the Ranch Hand study and to review data collection and reporting 

procedures. Finally, we reviewed the current state of public access to study 

data and queried several scientists and veterans’ organizations about their 

ability to access the data.

To examine the statistical limitations of the study and the reporting and 

communication of these limitations, we reviewed the study protocol, the 

study reports, and the memoranda and correspondence obtained from the 

Air Force and the Advisory Committee. In addition, we reviewed literature 

on epidemiology and communication of health risks. We reviewed study 

reports, executive summaries, and Air Force press releases to assess how 

study limitations have been communicated. We also queried scientists 

about aspects of communicating study limitations. Using the National 

Cancer Institute’s Surveillance, Epidemiological, and End Results database, 

we determined the expected incidence of several cancers in Ranch Hands 

based on the rate in the U.S. population and performed a calculation of the 

statistical power of the study (see app. II). 

To examine measures established to monitor the study’s conduct and to 

prevent improper influence, we reviewed the study protocol, memoranda, 

and correspondence obtained from the Air Force and the Advisory 

Committee and documentation provided by the Food and Drug 

Administration on veterans’ representation on the Committee. We 

documented the roles and activities of key organizations involved in 
Appendix I

Objectives, Scope, and Methodology

Page 27 GAO/NSIAD-00-31 Agent Orange

making scientific recommendations to Air Force scientists and compared 

them with requirements outlined in the study protocol. We also reviewed 

requirements on the selection and composition of the Ranch Hand 

Advisory Committee and reporting requirements as stated by law. 

To assess the impact of the Ranch Hand study on National Academy of 

Sciences reports and Department of Veterans Affairs determinations of 

diseases for which veterans are eligible to receive compensation benefits, 

we reviewed laws outlining the process to determine for which diseases 

Vietnam veterans are eligible for compensation. We then obtained and 

reviewed reports by the Academy and the Department of Veterans Affairs. 

In addition, we interviewed officials from these organizations to determine 

how research results lead to compensation decisions. We also reviewed 

those portions of the Federal Register that announce and codify regulations 

governing compensation of Vietnam veterans.

In conducting our study, we contacted current and former members, 

affiliates, or representatives from the following organizations: Advisory 

Committee, Executive Secretary at National Center for Toxicological 

Research, Food and Drug Administration; Air Force Research Laboratory, 

Brooks Air Force Base, San Antonio, Texas; Air Force Surgeon General, 

Bolling Air Force Base, Washington, D.C.; American Legion, Washington, 

D.C.; Department of Veterans Affairs, Washington, D.C.; National Academy 

of Sciences, Washington, D.C.; Veterans of Foreign Wars, Washington, D.C.; 

and Vietnam Veterans of America. In addition, we spoke with several 

scientists involved in research on the health effects associated with 

exposure to herbicides and dioxin. 

We performed our work from January through October 1999 in accordance 

with generally accepted government auditing standards.
Page 28 GAO/NSIAD-00-31 Agent Orange

Appendix II

Statistical Power of the Ranch Hand Study ApendxiI

This appendix describes in more detail the limited statistical power of the 

study caused by the small size of the Ranch Hand population. There were a 

total of about 1,200 Ranch Hands in 1982, and there are currently fewer 

than 1,150 still alive. An estimate of the ideal statistical power of the Ranch 

Hand study from 1982 to the last physical examination, to be performed in 

2002, is summarized for several cancers in tables 1-3. The last column of 

each table shows the expected number of cases among Ranch Hands 

necessary for the study to be likely to detect an increased risk of cancer 

with 95-percent significance. Statistical power increases with time because 

cancers, like most health problems, are more common among older people. 

This calculation is “ideal” because it assumes that all Ranch Hands and 

comparisons participate and that none of them die. It also does not account 

for a possible “healthy-soldier effect,” in which military populations are 

healthier than nonmilitary ones because entrance into military service 

requires passing a medical examination that screens out applicants with 

chronic illnesses and because remaining in the service requires maintaining 

good enough health to perform relatively rigorous duties. Because these 

numbers represent what the study could detect under the most ideal 

conditions, the actual power of the study is probably less than indicated 

here. 

As shown in these tables, the study cannot detect a doubling of cancer 

incidence (relative risk of at least 2) until at least 10 years into the study for 

many types of site-specific cancers. For all sites combined, the study can 

detect a doubling in risk. 
Appendix II

Statistical Power of the Ranch Hand Study

Page 29 GAO/NSIAD-00-31 Agent Orange

Table 1: Minimum Detectable Effect in 1982

Table 2: Minimum Detectable Effect in 1992

Cancer site

Cumulative

expected

percentage

with cancer

Number of

cases

expected

among

comparison

group

Minimum

relative riska

likely to be

detected

Cases expected

among Ranch

Hands

corresponding to

minimum relative

risk likely to be

detected

Liver 0.012 0.14 42 6.0

Prostate 0.088 1.0 7.5 7.9

Non-Hodgkins 

Lymphoma 0.10 1.2 6.7 8.3

Melanoma 0.17 2.0 4.9 9.7

All anatomical sites 2.2 26 1.7 43

Cancer site

Cumulative

expected

percentage

with cancer

Number of

cases

expected

among

comparison

group

Minimum

relative riska

likely to be

detected

Cases expected

among Ranch

Hands

corresponding to

minimum relative

risk likely to be

detected

Liver 0.053 0.63 11 7.1

Prostate 1.1 13 2.0 27

Non-Hodgkins 

Lymphoma 0.36 4.3 3.1 14

Melanoma 0.49 5.9 2.7 16

All anatomical sites 7.8 94 1.3 122
Appendix II

Statistical Power of the Ranch Hand Study

Page 30 GAO/NSIAD-00-31 Agent Orange

Table 3: Minimum Detectable Effect in 2002

a The minimum relative risk is the smallest relative risk that must exist in order for the likelihood that the 

study can detect the effect with 95-percent significance to be greater than 50 percent. The last column 

is the corresponding minimum number of cases expected among Ranch Hands in order for the study 

to detect an adverse effect. Numbers are rounded to two digits unless otherwise shown.

Note: The Ranch Hands used in this analysis were the 1,197 who were identified and alive as of 

December 31, 1982. An identical number of comparisons were assumed. They were stratified by birth 

in 5-year bins, with blacks and non-blacks considered separately. The age-specific cancer incidence 

rates for 1973-96 (from the Surveillance, Epidemiological, and End Results database) were used. 

Years before 1973 used the 1973 rates and years after 1996 used the 1996 rates. The cumulative 

incidence was calculated assuming all Ranch Hands left Vietnam in 1967. 

Sources: GAO analysis of National Cancer Institute and Air Force data.

Cancer site

Cumulative

expected

percentage

with cancer

Number of

cases

expected

among

comparison

group

Minimum

relative riska

likely to be

detected

Cases expected

among Ranch

Hands

corresponding to

minimum relative

risk likely to be

detected

Liver 0.18 2.1 4.6 9.9

Prostate 5.5 65 1.4 90

Non-Hodgkins 

Lymphoma 0.86 10 2.2 22

Melanoma 1.2 15 1.9 28

All anatomical sites 22 258 1.2 299
Page 31 GAO/NSIAD-00-31 Agent Orange

Appendix III

Duties of the Ranch Hand Advisory 

Committee ApendxIi

This appendix discusses the duties of the Advisory Committee in 

overseeing the Ranch Hand study. In general, the Committee conducts 

scientific reviews of the study and provides the Air Force with comments 

and advice. However, this advice, which may be in the form of either formal 

or informal recommendations, is nonbinding.

The Committee’s primary purpose, as described in the study protocol, is to 

review and assess the study’s conduct. The Committee performs this role 

through both site visits and by reviewing study reports before their release. 

Site visits include visits to Brooks Air Force Base, where the Air Force 

scientists are located, and to locations where physical examinations are 

performed. The Committee also reviews all official Air Force reports on the 

study prior to release, including executive summaries. The Committee does 

not review peer-reviewed journal articles but has consistently encouraged 

the Air Force to publish its results in these journals. The Committee 

convenes meetings when the Air Force has drafted reports to be reviewed 

and has not met on a regular basis.

According to the study protocol, the Committee should also review 

suggested changes in the data collected and methods of analysis used by 

the study. To this end, the Committee reviews the content of physical 

examinations, along with the statements of work for those examinations. In 

addition, the Air Force has contracted out the overall management, 

logistics, statistical analyses, and writing of Air Force morbidity reports 

subsequent to the first report in 1984. The Committee reviews plans for 

statistical analyses.
Page 32 GAO/NSIAD-00-31 Agent Orange

Appendix IV

Roles of Air Force, National Academy of 

Sciences, and Department of Veterans Affairs 

in Evaluating Health Effects of Herbicides ApendxIVi

In evaluating the health effects of herbicides, the Air Force, the National 

Academy of Sciences, and the Department of Veterans Affairs have 

different goals, use different criteria, and produce different types of 

products. These differences are illustrated in table 4. 

The goal of the Ranch Hand study is to determine whether health effects 

are attributable to herbicide exposure, while the goals of the Academy and 

the Department of Veterans Affairs emphasize establishing only an 

association. Although statistical methods are the primary tools the Air 

Force uses in the study, the “strength of association” of a result is only one 

of the many criteria Air Force scientists use in evaluating evidence of a 

causal relationship between adverse health and exposure to herbicides. 

The National Academy of Sciences and the Department of Veterans Affairs, 

on the other hand, both look primarily at evidence of a “statistical 

association” between diseases in humans and exposure to herbicides. The 

Academy considers evidence of a causal relationship separately.

The Ranch Hand study and the reviews by the National Academy of 

Sciences apply a relatively high standard in evaluating evidence. The 

Department of Veterans Affairs, on the other hand, is charged with using a 

specified standard of evidence intended to give veterans the “benefit of the 

doubt.” Specifically, a “positive association” for the Department of Veterans 

Affairs is defined by law as when “the credible evidence for the association 

is equal to or outweighs the credible evidence against the association.” 

Although both the Air Force and the National Academy of Sciences report 

findings of lesser significance, their emphasis is on establishing 

relationships in which they are reasonably confident. Air Force scientists 

noted, for instance, that though not all their criteria need to be satisfied for 

a finding to be reported, the more indicators that point to a causal 

relationship, the stronger the finding. The Academy defines “sufficient” 

evidence as findings in which “a positive association has been observed ...

in studies in which chance, bias, and confounding could be ruled out with 

reasonable confidence.” A second Academy category, one of 

“limited/suggestive” evidence, includes situations in which “evidence is 

suggestive of an association ... but is limited because chance, bias, and 

confounding could not be ruled out with confidence.” 
Appendix IV

Roles of Air Force, National Academy of 

Sciences, and Department of Veterans Affairs 

in Evaluating Health Effects of Herbicides

Page 33 GAO/NSIAD-00-31 Agent Orange

Table 4: Summary of Roles and Criteria in Evaluating the Health Effects of Herbicides

a

Criterion required by law.

Sources: Air Force, Public Law, National Academy of Sciences, Department of Veterans Affairs, and 

Federal Register.

Goal(s) Evidence evaluated Standards/criteria used Product(s) Uses of products

Air Force 

Ranch 

Hand study

Existence of health 

effects in Vietnam 

veterans attributable to 

exposure to herbicides.

Data collected by Air 

Force on Ranch Hands 

and comparison group.

Plausibility.

Internal and external 

consistency.

Strength of association.

Statistical power.

Biological mechanism.

Validity or bias.

Statistical significance.

Statistical assumptions.

Ranch Hand 

study official 

reports.

Peer-reviewed 

journal articles.

National Academy 

of Sciences 

reviews. 

Department of 

Veterans Affairs 

compensation 

determinations.

Scientific base of 

knowledge.

National 

Academy of 

Sciences 

Existence of statistical 

association between 

disease in Vietnam 

veterans and 

herbicides.

Scientific literature on 

effects of herbicides in 

humans.

a

Extent to which scientific data 

permit determinations.

a

Strength of evidence.

a

Appropriateness of methods.

Biennial reports: 

Veterans and 

Agent Orange.

Department of 

Veterans Affairs 

compensation 

determinations.

Recommendations 

for future studies.

Existence of an 

increased risk among 

Vietnam veterans.

Scientific literature on 

Vietnam veterans and 

other human 

populations. 

a

Extent to which scientific data 

permit determinations.

Existence of a plausible 

biological mechanism 

or other causal 

relationship.

Scientific literature on 

effects of herbicides in 

humans and animals.

a

Extent to which scientific data 

permit determinations.

Validity of animal models.

Biochemical origin of dose 

response. 

Department 

of Veterans 

Affairs

Existence of a positive 

association between 

disease in humans and 

herbicide exposure. 

National Academy of 

Sciences review of 

scientific literature. 

Other available 

scientific and medical 

information and 

analyses.

a

Whether evidence for an 

association equals or outweighs 

evidence against an 

association, considering 

(a) academy reviews, 

(b) statistical significance, 

(c) capability of replication, and

(d) capability to withstand peer 

review.

Department Task 

Force Reports.

Regulations of 

diseases with 

presumption of 

service 

connection.

Compensation 

determinations.

Recommendations 

for future studies.
Page 34 GAO/NSIAD-00-31 Agent Orange

Appendix V

Comments From the Department of Defense ApendxVi
Appendix V

Comments From the Department of Defense

Page 35 GAO/NSIAD-00-31 Agent Orange

See p. 24.

See p. 24.

See p. 24.
Appendix V

Comments From the Department of Defense

Page 36 GAO/NSIAD-00-31 Agent Orange
Page 37 GAO/NSIAD-00-31 Agent Orange

Appendix VI

Comments From the Food and Drug 

Administration ApendxViI
Appendix VI

Comments From the Food and Drug 

Administration

Page 38 GAO/NSIAD-00-31 Agent Orange

p

See p. 24.

(713035) Letr
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the 

following address, accompanied by a check or money order made 

out to the Superintendent of Documents, when necessary, VISA and 

MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are 

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and 

testimony. To receive facsimile copies of the daily list or any list 

from the past 30 days, please call (202) 512-6000 using a touchtone 

phone. A recorded menu will provide information on how to obtain 

these lists.

For information on how to access GAO reports on the INTERNET, 

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at: 

http://www.gao.gov

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00