Gulf War Veterans: Incidence of Tumors Cannot Be Reliably Determined From Available Data
Published: 1998-03-03T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed data on the incidence of tumors among Gulf War veterans, focusing on: (1) the reliability of data sources available for determining the incidence of tumors among Gulf War veterans; and (2) the Department of Veterans Affairs' (VA) and the Department of Defense's (DOD) use of data sources to monitor tumors and other illnesses among Gulf War veterans.
section_title: What GAO Found
paragraphs: GAO noted that: (1) none of the data sources that provide information on the health characteristics of Gulf War veterans can be used to reliably estimate the incidence of tumors; (2) VA's benefits information system can track the vital status and causes of deaths among Gulf War veterans; (3) however, not all cancers result in death and those that do may take several years to show up; (4) as a result, the system will underreport overall incidence; (5) DOD and VA maintain large hospitalization reporting systems; (6) however, a large majority of Gulf War veterans do not use DOD and VA hospitals and there has been little effort to determine whether this hidden population has health conditions similar to those of the population captured by the reporting systems; (7) DOD's reporting system also does not account for outpatient medical care; (8) VA has recently begun to fill this gap for its outpatient facilities, but it may take several years before consistent and reliable reporting is available; (9) a national cancer registry reports aggregate population rates and trends but cannot be used to track the Gulf War population; (10) DOD and VA health registries report information on the type of health problems Gulf War veterans have experienced at the time of their examination; (11) however, because not all veterans are examined, the information collected cannot be used to estimate the frequency of illnesses among all Gulf War veterans; (12) VA is conducting a national survey to study the general health status of Gulf War veterans; (13) the study uses representative samples of deployed and nondeployed veterans; (14) however, the response rate to the survey has been low and the study's sample size may be too small to assess any elevated incidence of most cancers; (15) DOD and VA have initiated efforts to improve the utility of these data systems but have not developed the capability to specifically address questions about tumors or other illnesses among Gulf War veterans; (16) as a result, it is not known how many Gulf War veterans have tumors or whether they have a higher incidence of them than other veterans; and (17) according to agency officials, no other plans aside from periodic assessments of mortality have been made to monitor tumor cases within this population.

section_title: Letter
paragraphs: Numerous claims of illnesses from Gulf War veterans and inconsistent reports on veterans’ exposure to risk factors have prompted a number of government studies on the nature, extent, and treatment of Gulf War illnesses. In June of last year, we reported to you and other members of Congress that government research efforts may not be able to provide conclusive answers about the causes of veterans’ illnesses and that several risk factors such as chemical and biological agents had been ruled out prematurely as possible explanations for the reported illnesses. We also reported that many veterans were unhappy with the quality of health care treatments they had received from the Department of Veterans Affairs (VA) and that enhanced monitoring is needed to determine if veterans are getting any better or worse over time.
In your letter to us on June 12, 1997, you raised concerns about whether Gulf War veterans have an increased risk of developing certain cancers as a result of their exposures to hazards in the Persian Gulf. You asked us to examine pertinent data on the incidence of tumors among Gulf War veterans and determine whether these data indicate any differences in rates of tumors between Gulf War veterans and appropriate comparison groups. In this report, we assessed (1) the reliability of data sources available for determining the incidence of tumors among Gulf War veterans and (2) VA’s and the Department of Defense’s (DOD) use of data sources to monitor tumors and other illnesses among Gulf War veterans. The five types of data sources we reviewed were mortality data, medical records, cancer registries, Gulf War veterans health registries, and survey results.
Although we sought to examine data sources that pertain directly to tumors, many of the sources can also be used to assess other diseases that may be associated with service in the Gulf War. Because of this, our findings concerning the strengths and limitations of these data should be relevant to the broader issue of monitoring other Gulf War illnesses as well. In addition, it is important to point out that several of the data sources we reviewed were not designed specifically for medical research purposes but rather for other uses such as veteran outreach or the administration of records.
subsections: section_title: Background
paragraphs: Following Iraq’s invasion of Kuwait in August 1990, the United States and other allied nations sent troops to the Persian Gulf region in Operation Desert Shield. In the winter of 1991, the allied forces successfully attacked Iraq in an air campaign and subsequent invasion by ground forces (Operation Desert Storm). Approximately 700,000 American troops participated in these actions. Although casualties were relatively light, thousands of veterans have come forward complaining of various illnesses, including cancer, in the years following the war.
During the Gulf War, American troops may have been exposed to several known and potential health risks. These included chemical and biological warfare agents, depleted uranium from munitions, smoke from oil-well fires, infectious diseases, pesticides, petroleum fuels, and vaccines. Some of these substances have been previously associated with different types of cancer through animal laboratory studies and other epidemiological research investigations. For example, combustion products from petroleum include polyaromatic hydrocarbons, benzene, and carbon disulfide, some of which are known to cause lung cancer when inhaled. Exposure to certain pesticides has also been linked to lymphatic and lung cancers. In addition, exposure to radioactive particles has been tied to higher rates of respiratory and other cancers. Information on exposures that took place during the Gulf War, however, has been either incomplete or nonexistent due to the lack of record keeping and measurement before, during, and after the deployment of troops; loss of key records; poor recall by veterans; and other factors.
The development of cancer is usually characterized by a long latency period of several years from an initial exposure to a harmful agent to a definitive medical diagnosis. Depending on the nature and extent of the exposure, type of cancer, and characteristics of different individuals, the latency period may be as long as 30 years or more. The most common types of cancers have a latency period of 15 years or more, but in certain situations cancer can develop more quickly. For example, when the immune system is compromised, such as in cases of organ transplants, certain types of cancer may appear within 1 year.
Given that there is a lengthy latency period for most tumors, it may be too soon to detect any increase in tumors occurring among Gulf War veterans. Also, since cancer is a relatively rare event, large population groups may need to be observed over several years to assess incidence and determine whether it has changed over time. Furthermore, without credible exposure information, it is hard to form specific hypotheses about what kinds of tumors might occur with what individuals. Although such constraints exist, it is nonetheless important to begin monitoring and assessing whether Gulf War veterans are suffering from an increase in tumors so that appropriate health care and treatment can be provided where needed. With many types of tumors, early detection is important to more effective treatment outcomes.
section_title: Mortality Data
subsections: section_title: VA Benefits Information System
paragraphs: One source for estimating the incidence of cancer among Gulf War veterans utilizes mortality as an indicator for incidence. VA maintains a large administrative database containing records of claims for benefits made by veterans and their dependents. This database, the Beneficiary Identification and Records Locator Subsystem (BIRLS), includes information on more than 40 million individuals. Although not designed as a research database, it does contain information on the vital status of veterans and the location of veterans’ claim forms so that death certificates can be retrieved to ascertain causes of death. VA pays death benefits, including fixed payments for burial and funeral expenses, to eligible survivors of deceased veterans. Dependents are required to submit a copy of the veteran’s death certificate in order to receive these benefits. The BIRLS datafile also contains the veteran’s name, social security number, claim number, current address, and other benefits information. In addition, the file includes a code that indicates whether the veteran was deployed to the Gulf War.
VA and other researchers have estimated that death reporting in the BIRLS database is relatively complete in terms of its coverage of the veteran population. Studies using large samples of known deaths have compared vital status reporting in BIRLS with reporting in other national sources of mortality data, such as the Master Beneficiary Record of the Social Security Administration and the National Death Index, and found that BIRLS covers 80 to 90 percent of the deceased veteran population. Another strength of the BIRLS data is that they provide a relatively cost-effective way to assess causes of death among veterans. Obtaining death certificates directly from state health departments involves paying a fee of several dollars for each certificate.
Using BIRLS or other mortality databases to assess overall cancer incidence has several key limitations. First, mortality is a lagging indicator of incidence. The latency period for most tumors can be more than  10 years and the period of time until mortality is even longer. Second, in general, mortality is an incomplete measure of cancer incidence because cancer is not always fatal. Mortality data provide good estimates of incidence for cancers that have a high mortality rate (such as lung and liver cancers) but they are less useful for cancers with lower rates of mortality (such as prostate and breast cancers). Because of these limitations, mortality data will systematically underreport overall cancer incidence. In addition, by the time cases show up in the data, it may be too late to help Gulf War veterans. Furthermore, while the reporting of external causes of death, such as accidents, or broad disease categories, such as coronary heart disease, is reasonably reliable, some inconsistencies in cause of death reporting on death certificates have occurred. Death certificates are less accurate in tracking difficult to diagnose diseases, deaths involving multiple causes, and where there are underlying causes of death that are not readily discernible.
section_title: Using VA’s Benefits Information System to Assess Tumors Among Gulf War Veterans
paragraphs: VA in one published study used data from BIRLS to assess the mortality risk from a range of diseases (including cancer) for all Persian Gulf veterans compared with a sample of veterans who were not deployed to the Persian Gulf. The study covered deaths occurring in a 2-year period after the war (May 1991 to September 1993). Death certificates for those veterans who had been identified as having passed away were obtained from VA regional offices and other locations and were reviewed for cause of death. The study found that overall there was a small but significant excess of deaths among Gulf War veterans (1,765 deaths) compared with nondeployed veterans (1,729 deaths) and that the excess was due mainly to accidents and not disease. Of the 1,765 Gulf War veterans who died during the study period, 119 died from cancer, and there was no statistically significant difference compared with the cancer death rate among nondeployed veterans.
If a higher death rate from cancer was expected among Gulf War veterans as a result of some exposure occurring during deployment, then they would be unlikely to appear in this study given the short time period that elapsed. VA is currently updating the study, extending the study period through 1995, and the results should be published later this year. The study’s authors identified another limitation (but it is not clear what effect it had on the study’s findings)—specifically, whether the study’s comparison groups were appropriately matched. Military personnel who were ill or recovering from an illness would not have been deployed to the Gulf War area. However, these personnel were included in the comparison group of nondeployed veterans. This meant that the comparison group may have been somewhat less healthy than the deployed veterans group. The extent to which a higher rate of prior illnesses among nondeployed veterans resulted in a different rate of mortality (or cancer mortality in particular) and thus biased the study findings is unknown. Finally, the use of broad comparisons between deployed and nondeployed veterans rather than more targeted comparisons of veterans based on specific types and levels of exposures may also have affected the soundness of the study. A comparison of deployed and nondeployed veterans has merit for identifying potential widespread and severe health consequences. However, defining the exposed population group as “all those who served in the Gulf War” without regard to individual groups’ exposure histories may obscure some service-connected illnesses. Efforts to exclude from such studies portions of the deployed force who were at low risk of exposure to harmful agents could lead to more meaningful results in such comparative studies.
section_title: Medical Records
subsections: section_title: DOD’s and VA’s Automated Hospitalization Reporting Systems
paragraphs: DOD and VA each maintain an automated database containing medical and demographic information on patients discharged from DOD and VA hospitals. DOD’s system collects information from DOD military hospitals. These hospitals are open to active duty personnel and, to a limited extent, retired personnel. VA’s data system, the Patient Treatment File, covers all VA hospitals. VA hospital care is generally available to veterans for service-connected illnesses. Care is also provided on a discretionary basis for nonservice-connected illnesses, depending on the availability of facilities and resources and payment of a required co-payment by the veteran. Veterans who may have been exposed to a toxic substance or environmental hazard while serving in the Gulf War are included in a designated category of veterans who have special eligibility for VA medical care services. Both the DOD and VA data systems include medical discharge diagnoses, which are coded according to standard ICD-9 (International Classification of Diseases, 9th Revision) disease categories. In addition, the data contain relevant information such as social security numbers, date and place of birth, period of military service, length of hospital stay, and surgical and other medical procedures conducted.
DOD’s and VA’s hospitalization data systems are large and contain millions of records, but they do not represent the entire active duty and veteran population. While DOD’s data include most hospitalizations of active duty personnel, in large part because DOD medical care is free and readily available to active duty personnel, there have been reports from some veterans’ groups of Gulf War veterans seeking medical care outside DOD. According to these groups, veterans have done so to obtain specialized care or because of concerns that the acknowledgment of their illnesses within DOD could have a negative effect on their military careers. It is not clear whether this would more often be the case for Gulf War veterans with tumors than for nondeployed veterans.
VA also has an extensive network of medical centers across the country but the overwhelming majority of veterans use other private and public hospitals. A survey conducted by VA in the late 1980s, estimated that only about 20 percent of veterans had ever used a VA hospital. With respect to Gulf War veterans, it is not known whether there is greater or less use of VA medical facilities. Since Gulf War veterans have been authorized special eligibility for medical care, there may be greater use of VA medical centers compared with some other groups of veterans. On the other hand, there have been numerous accounts in the media and by veterans’ groups of dissatisfaction with government efforts to address Gulf War veterans’ health problems. This may contribute to a greater reluctance among some veterans to seek hospital care at VA medical facilities.
Another weakness of the hospitalization data systems has been the lack of coverage of outpatient medical care. DOD currently has no centralized reporting system for its outpatient facilities, although an automated system is under development. Until recently, VA did not have an automated system either. VA established an automated system in October 1996 to begin collecting information on the use of its outpatient facilities and the different types of medical care provided. Information is not available yet on the accuracy and completeness of the reporting. Coverage of outpatient facilities is important because there is a current trend in the health field toward outpatient diagnosis and treatment of many types of tumors.
In addition to limitations in terms of population coverage, there are also issues regarding the accuracy and completeness of hospitalization data reporting. One strength of the reporting process is that standard disease categories are coded so that comparable data can be collected from each hospital and more specific types of diseases can be assessed. Furthermore, the reporting allows for multiple discharge diagnoses to be recorded and not just the principal cause for hospitalization. Miscoding of discharge diagnoses, however, is a potential problem as shown by VA researchers in previous assessments of certain types of cancer among Vietnam veterans.
In a case-control study, for example, of over 400 Vietnam veterans identified in VA’s Patient Treatment File with a malignant tumor of connective and other soft tissue, close to 40 percent of the records were found to be miscoded or misclassified when hospital pathology reports were subsequently collected and independently reviewed by an expert pathologist.
section_title: Using DOD’s and VA’s Automated Hospitalization Reporting Systems to Assess Tumors Among Gulf War Veterans
subsections: section_title: VA Tumors Analysis
paragraphs: In May 1996, VA completed an analysis requested by your Subcommittee that provided some information on the number of tumors among Gulf War veterans compared with the number in a sample of nondeployed veterans. The analysis sought to identify cases of tumors occurring immediately after the war up through the early part of 1996. Existing VA databases, including the Persian Gulf Health Registry, Patient Treatment File, and BIRLS, were used as means to identify tumors. A breakdown by type of tumor, age, gender, race, and branch of service was conducted after merging the health registry and hospitalization data records; however, these records were not subsequently merged with the benefits records data (BIRLS) because the diagnostic coding used in the two systems is different.
As reported by VA, based only on the combined health registry and hospitalization data, the number of individuals with diagnosed tumors was relatively low but the number of Gulf War veterans was substantially higher as compared with nondeployed veterans (1,691 out of 697,000 Gulf War veterans compared with 1,092 out of 1,605,087 nondeployed veterans). Most of the tumors identified, though, were benign and not malignant cancers. Possible reasons for the higher rate according to VA are that (1) priority eligibility status is given to Gulf War veterans for inpatient treatment and (2) a special health registry exists for Gulf War veterans. In contrast to these results, a larger number of diagnosed tumors was reported from the benefits records data (BIRLS) but the number of cases among Gulf War veterans was lower compared with the number among nondeployed veterans (6,397 out of 697,000 Gulf War veterans compared with 21,227 out of 1,605,087 nondeployed veterans). Thus, the different VA data sources present a different result for tumor cases in these population groups.
VA has acknowledged that its analysis is quite limited because of weaknesses in the existing data sources used. A key limitation is the poor coverage the data provide of the veteran population. Veterans who use non-VA medical care facilities are excluded. A potential source for augmenting this is the use of the BIRLS data, which in addition to reporting information on mortality claims reports information on medical disability claims. Veterans, regardless of whether they receive medical treatment from VA or elsewhere, can apply for disability claims. BIRLS tracks denied and approved claims, including those for disabilities associated with tumors. As a condition for qualifying for disability compensation, it must be established that the disability is service-connected and that the condition leading to the disability appeared either while the veteran was on active duty or within a presumptive period after separation from the service. For most cancers, however, the presumptive period is limited to 1 year. In principle, BIRLS includes other cases than those reported in the VA hospital reporting system. The major restriction to using the data to identify illnesses among Gulf War veterans, though, has been the difficulty of merging the claims records with other VA hospitalization records because of the different diagnostic coding systems used. The extent to which a crosswalk could be developed to link the diagnostic codes has not been determined.
section_title: Postwar Hospitalization Experience Study
paragraphs: A large DOD-funded study was published last year examining the hospitalization experience of all active duty Gulf War veterans compared with a sample of other active duty military personnel who were not deployed to the Gulf region. The purpose was to determine whether participation in the Gulf War was associated with the occurrence of serious illnesses requiring hospitalization. Using inpatient hospitalization data records obtained from DOD, the study assessed hospitalizations occurring during the period August 1991 through September 1993. Overall, the authors found there was no excess in hospitalizations among Gulf War veterans compared with other military personnel. Among specific types of hospitalizations, however, Gulf War veterans had higher rates in specific years for tumors (in 1991), mental disorders (both 1992 and 1993), diseases of the blood (in 1992), and diseases of the genitourinary system (in 1991). For tumor cases reported in 1991, most involved benign conditions and, although the rates were higher for Gulf War veterans, the differences were not statistically different. The one exception, where a significant difference (higher rate) was found, was for testicular cancer in 1991 hospitalizations. The investigators conducted a follow-on review of hospitalizations for testicular cancer through March 1996 and found that male Gulf War veterans who remained on active duty after the war were not at increased risk of hospitalization for testicular cancer.
A major strength of this study is its large size and statistical power to detect differences in rates of hospitalizations between deployed and nondeployed military personnel. However, a key limitation of the study, which influences the interpretation of the results, is that hospitalizations of Gulf War veterans who separated from the service as well as any hospitalizations of active duty personnel who used non-DOD hospitals are excluded. The number of hospitalizations excluded from the study is not known, but the number of veterans who separated from the service increased substantially since the end of the war. According to DOD figures, Gulf War veterans who remained on active duty declined to 66 percent in 1993 and to below 50 percent by 1995. Another important limitation of the study is that the timeframe was far too short for detecting any diseases resulting from possible exposures during the war, such as tumors, which have lengthy latency periods. Extending the time period would address the latency issue, but then the problem of missing hospitalizations would increase as the number of Gulf War veterans remaining on active duty gets smaller over time.
A related follow-up study is currently underway by the same researchers, to examine hospitalizations of Gulf War veterans in military and nonmilitary hospitals in the state of California. This study will merge hospitalization data from three sources: DOD, VA, and the state government. California maintains a database of patient hospital discharge information collected from nonfederally licensed hospitals in the state (diagnoses are coded using the ICD-9 classification system). The study will seek to identify all Gulf War veterans who resided in the state at the time of or at least a year prior to deployment. Internal Revenue Service files will be matched with DOD’s roster of 697,000 Gulf War veterans to identify those residing in the state (estimated to be about 12 percent of the Gulf War force), and the resulting resident file will then be matched with the various hospital data files to identify hospitalizations for each veteran. Although the study results will probably not be generalizable to the entire Gulf War population, the study is large and one of the first to systematically combine military and nonmilitary hospitalizations. The study period is longer (1991-95); however, the problem of detecting diseases with a lengthy latency period is still an issue, and outpatient data will be excluded. Currently, the study is in the initial data merging phase and is not expected to be completed until 1998 or later.
section_title: Cancer Registries
subsections: section_title: National and State Registries
paragraphs: Another source for estimating the incidence of cancer is the population-based cancer registries along with other baseline demographic data. Cancer registries are compilations of reports of cancer cases that are filed by medical facilities (typically hospitals) on an ongoing basis according to prescribed data coding procedures. Cancer registries can be effective tools for determining incidence rates and for directing cancer control efforts. Typically they are used to identify and monitor trends, patterns, and variations in cancer incidence and mortality by geographic location, ethnicity, gender, and age.
Cancer registries exist at the national, state, and local level. The national cancer registry (the Surveillance, Epidemiology and End Results [SEER]) was established in 1973 by the National Cancer Institute. SEER collects data from designated cancer registries that operate in various areas of the country. Currently, it covers a group of five states and four metropolitan areas that were selected for “their ability to operate and maintain a population-based cancer reporting system and for their epidemiologically significant population subgroups.” Most states also maintain cancer registries. In 1992, Congress enacted the National Program of Cancer Registries (P.L. 102-515), which authorized the Centers for Disease Control and Prevention to fund states to improve existing cancer registries and develop registries where they do not exist. In 1996, 41 funded states were collecting statewide cancer data. The Centers for Disease Control and Prevention has set standards for reporting accuracy, timeliness, and completeness. However, some differences exist across states in the level and quality of reporting. Voluntary reporting is also encouraged from DOD and VA hospitals. In addition to these registries, DOD administers its own cancer registry (the Automated Central Tumor Registry) for active duty troops and others (i.e., retirees and family members) who use DOD military medical facilities. It was set up as a central registry in 1986 to compile, track, and report cancer patient information from military medical treatment facilities.
SEER is a comprehensive system for tracking cancer incidence for the general population and key subgroups. In addition, provisions are made for quality assurance checks on data reliability. However, in terms of its suitability for assessing cancer rates in Gulf War veterans, SEER only collects and reports aggregate information and does not include the necessary individual-level identifiers that would be needed to distinguish Gulf War veterans. SEER can provide incidence rates for the general population and key population subgroups but not for the Gulf War veteran population.
Many of the state registry systems generally do include individual identifiers such as social security numbers, so a match against a roster of Gulf War veterans, using common identifiers, could be conducted to identify cancer cases within this group. Such a match of course would need to address potential confidentiality issues involving the privacy of cancer patient information. The accuracy, timeliness, and completeness of reporting also varies by state registries. For example, many states only require reporting by hospitals and do not capture cases diagnosed by private physicians, laboratories, and health maintenance organizations. Many of the registries also have different data field structures and are not designed to be readily merged with other registries.
DOD’s central tumor registry contains over 188,000 records of current and past cancer patients. While the reporting system is designed to capture all cancer cases treated at DOD medical facilities, DOD officials have indicated that complete reporting is not occurring. No systematic assessment has been conducted to measure how complete the reporting is, and no quality assurance system is in place to ensure that reporting is being done.
section_title: Using Cancer Registries to Assess Tumors Among Gulf War Veterans
paragraphs: VA has provided initial funding to the Boston Environmental Hazards Center to assess cancer incidence among Gulf War veterans in New England. The Center previously examined cancer incidence among Vietnam veterans and will employ a similar methodology for looking at Gulf War veterans. The approach entails developing a roster of Gulf War veterans and linking it (by identifying information such as names, dates of birth, and social security numbers) with cancer cases that appear in the state registries. The first phase of the study has been funded to create a roster of Gulf War veterans in the New England area and develop a framework for merging data together from the individual state registries. The next phase of the study, to begin by 1999, will involve an assessment of cancer incidence and mortality. The study design notes that it would not be informative to analyze cancer incidence sooner because the time interval between any exposures that may have occurred during the Gulf War and the diagnosis of most cancers is “probably at least 10 years and may extend as long as 20 to 40 years.” It is also intended that these health outcomes be linked to information about potential environmental exposure factors that may exist or become available from DOD military records and other sources, including the location database being compiled by the U.S. Army Center for Health Promotion and Preventive Medicine.
Although this study is several years away from completion, it appears to provide a useful means for obtaining information about cancer incidence in the future. Some of the strengths of the study are that it will use existing data systems, identify and assess a large cohort of Gulf War veterans, and can be readily updated over time. One key limitation of the study, however, is that the results will not be generalizable to the entire Gulf War population since only the New England states will be included. Also, it is not known whether there is complete reporting of cases to the registries, particularly with respect to cases diagnosed outside of the hospital setting and cases from border areas that may get reported in other state registries outside the New England area.
section_title: Gulf War Veteran Health Registries
paragraphs: Both DOD and VA have established separate programs that provide medical examinations and diagnostic services, free of charge, to Gulf War veterans. VA began its Persian Gulf Health Registry Examination Program in 1992, and DOD started its Comprehensive Clinical Evaluation Program in 1994. The programs are open to all active duty, separated, and retired military personnel who were veterans of the Persian Gulf deployment. An existing health problem is not necessary for participation in the programs; any Gulf War veteran with health questions or concerns is eligible to enroll on a voluntary basis. Currently, the programs are designed to follow a standard protocol that requires registry physicians to obtain a detailed medical history, conduct a physical examination, and order basic laboratory tests. Further diagnostic procedures and referral to specialized medical centers are available for veterans with health problems that cannot be satisfactorily diagnosed from the initial evaluation. As of April 1997, approximately 66,000 veterans completed VA’s registry examination, and over 31,000 veterans completed DOD’s examination.
While the registry programs are primarily intended to provide diagnostic services and treatment to Gulf War veterans, the programs also gather and report data on the nature of the veterans’ health problems and the types of risk factors veterans may have been exposed to in the Gulf War. The most common symptoms reported among veterans examined include fatigue, skin rashes, muscle and joint pain, headaches, and memory loss. Approximately 80 percent of the veterans with symptoms have been diagnosed with one or more recognizable diseases; however, the other 20 percent with symptoms remain undiagnosed. Diseases involving musculoskeletal and connective tissue, psychological conditions, and the respiratory system were diagnosed most frequently. The number of registry veterans with a primary diagnosis of a malignant or benign tumor is very small, less than 1 percent.
The suitability of the registries for assessing cancer incidence is extremely limited. As designed, the registries are not intended to be used to determine the frequency and causes of illnesses among the general Gulf War veteran population. A principal reason for this is that the participants volunteered for their examinations and were not selected based on a random sample (selection bias). Therefore, there is no way to know whether the health problems found among the registry participants are similar to those of the general population of Gulf War veterans. In addition, because there is no ready comparison or control group for the registry participants, there is no means to interpret the significance of the data that are reported. A further limitation of the registry data is that they capture information about the health of veterans only at one point in time. Thus, if a veteran develops cancer or another illness later on, the registry data will not reflect this.
Data quality concerns also have been raised in a previous review of the VA registry by the Institute of Medicine. The Institute found, for example, that there was a considerable delay between the collection of the examination data and their entry into the registry database. In other ongoing work we are conducting on the quality of health care Gulf War veterans are receiving, we also found that VA medical facilities have not reported registry examination information consistently. It appears that a large number of case records submitted for input into the registry database have been rejected and sent back to the medical facilities due to coding errors. At the same time, effective quality assurance procedures have not been in place to ensure that rejected records are corrected and reentered into the database. Thus, data coverage even for those who participated in the registries has been incomplete.
section_title: Survey Data
paragraphs: Another data approach involves developing information about incidence by using survey methods, such as administering a questionnaire to a sample of veterans. As opposed to the other approaches, which employ data from databases administered by federal or state agencies, the researcher has more control over the data that are being gathered. Specifically, the researcher could ensure that the data are representative of the overall population of Gulf War veterans.
Significant advantages to using the survey approach include the ability to draw a random sample of Gulf War veterans and a comparison group  (e.g., veterans who had not deployed to the Gulf War region). A survey also permits the researcher to gather other information, such as information about exposures and family history, that might shed light on the etiology of disease.
Limitations with this approach include the possibility of response bias (individuals who complete the survey not being representative of the sample as a whole) and the subjectivity of self-assessments. There are standard ways of dealing with these limitations. The problem of response bias can be dealt with first by sending out multiple questionnaires. The extent to which response bias is a factor can be estimated through a special survey of nonrespondents, which is typically conducted by telephone. The results of the nonrespondent survey are compared against the results of the principal survey to gauge the degree to which respondents are typical of the overall sample. Subjectivity of the assessments of cancer can also be gauged to a degree through an independent medical review of a subsample of respondents. In addition, subjectivity of assessments is somewhat less of a concern in terms of tumors than many other illnesses.
Also, care needs to be taken to ensure that the size of the sample is large enough to characterize with confidence differences between the Gulf War veteran and the comparison groups. Tumors that have a low background incidence would need to be studied with extremely large sample sizes to detect an elevated incidence among Gulf War veterans. Sample sizes required to draw conclusions would need to be determined at the earliest stages of the study.
A further concern in implementing large population surveys is that they tend to be much more costly than the other approaches being presented here. In addition, the type and number of questions must be restricted or people will not respond.
subsections: section_title: VA National Survey
paragraphs: This approach is being employed by VA to study the general health status of Gulf War veterans. The National Health Survey of Persian Gulf War Era Veterans uses a mailed survey to compare self-reported symptoms and illnesses between a random sample of 15,000 Gulf War and 15,000 nondeployed veterans. The questionnaire includes a checklist of illnesses, including skin cancer and “any other cancer” and a checklist of symptoms such as “coughing” and “skin rashes.” In addition to questions about current health status, respondents are also asked to report about their exposure to a list of agents including nerve gas, depleted uranium, and smoke from oil well fires while they were in the Gulf War region.
The overall response rate to the survey has been relatively low (57 percent). VA is conducting a survey of nonrespondents in order to evaluate nonresponse bias. VA is also addressing the limitation imposed by subjective assessments through an independent review of medical records and “comprehensive physical examination” of a subsample of 2,000 respondents (1,000 in each of the Gulf War veteran and nondeployed veterans groups).
The sample size of VA’s survey may also be too small to identify elevated incidence of most cancers. With respect to the issue of statistical power, VA has acknowledged that, “the study may provide inadequate statistical power to detect a small increase in risk for rare adverse health outcomes in a particular subgroup of veterans.”
In 1996, the Institute of Medicine concluded, “This is a well-designed and well-intended study.” According to the Institute, however, “there appeared to be little statistical input in the analysis plan reviewed, and these data will require sophisticated statistical adjustment.”
section_title: Iowa State Survey
paragraphs: A population-based survey to assess the prevalence of self-reported symptoms and illnesses among Gulf War veterans was also conducted in Iowa. The study used a telephone interview approach to survey a random sample of Gulf War and non-Gulf War veterans from Iowa. Approximately 3,700 veterans were interviewed during the period September 1995 through May 1996. Overall, the study found that Gulf War veterans reported a significantly higher prevalence of a wide range of medical and psychiatric conditions compared with military personnel who were not deployed to the Gulf War. The primary conditions where differences were reported include depression, posttraumatic stress syndrome, chronic fatigue, cognitive dysfunction, and respiratory diseases. The rate of cancer reported among these Gulf War veterans was generally low (an estimated rate of about 1 per 100 subjects), but it was slightly higher than that of the comparison group.
section_title: Conclusions and Recommendation
paragraphs: No direct link has been established between potential exposures that occurred during the Gulf War and the development of tumors among veterans. However, concerns have been raised because many of the exposure agents in question have previously been associated with certain cancers. This has led to interest in determining if the cancer incidence rate among Gulf War veterans is higher than the rates within other appropriate comparison groups. If there is a higher incidence that identifies an emerging health problem, then outreach efforts could be conducted to target appropriate diagnosis and treatment to those potentially at risk.
Due to the long latency period of most tumors, it may be too soon to detect whether there is an increased incidence of tumors among Gulf War veterans. Nonetheless, it may be important to collect information now and begin planning for monitoring the future health status of veterans. However, the existing data sources and research applications we reviewed, provide very limited information about the incidence of tumors or other illnesses among Gulf War veterans. Federal research studies that are currently underway to assess tumors should help fill the gap, but these studies are limited in terms of their coverage of the Gulf War veteran population, data quality, and timeliness. Thus, it will also be difficult to determine whether Gulf War veterans have a higher incidence of tumors than other veterans in the future.
In our June report of last year, we found that DOD and VA had no effective means to determine whether ill Gulf War veterans were getting better or worse over time and recommended that DOD and VA develop a plan to monitor their medical progress. This recommendation was effectively incorporated into the recently enacted National Defense Authorization Act for Fiscal Year 1998, as part of a broader initiative requiring the Secretaries of Defense and Veterans Affairs to monitor health care services and treatment to Gulf War veterans. In response to this, DOD and VA have asked the Institute of Medicine to establish a committee of experts to assess the appropriate methodology for monitoring health outcomes. However, the ability to monitor veterans’ health conditions may be seriously handicapped by the data constraints we have noted in this report.
In order to evaluate more effectively the incidence of tumors and other Gulf War illnesses over time, we recommend that the Secretaries of Defense and Veterans Affairs continue to strengthen existing monitoring capabilities. Attention should be directed toward improving the utility of existing data systems and particularly in developing cost-effective ways to make data systems more compatible with one another so that information from different sources can be linked. In addition, steps should be taken to address the data quality concerns we identified in this report. While we believe such improvements can lead to more effective monitoring capabilities, the existing data systems may be insufficient to answer the question about cancer incidence or other illnesses among Gulf War veterans. Therefore, further research efforts will be needed to supplement the available data systems. For example, little is known about the health status of veterans who receive medical care from sources other than DOD and VA facilities. Practical approaches should be developed to determine whether there may be emerging health problems among these veterans.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided written comments on a draft of this report (see app. I), and VA provided oral comments. Generally, DOD and VA concurred with our overall findings regarding the inadequacies of existing data systems for assessing tumors among Gulf War veterans and our recommendations to improve monitoring capabilities. They emphasized that our recommendations support initiatives they currently have underway to strengthen health information reporting systems and the transfer of data between the two agencies. According to DOD, these actions are part of a long-term effort it is working on to develop a comprehensive system for maintaining medical records of all illnesses and injuries that military personnel may suffer, the care they receive, and their exposure to different hazards. DOD noted a key objective of this work is to apply lessons learned from the Gulf War experience to improve necessary medical surveillance and record keeping for future military deployments. While we recognize that DOD’s efforts will likely improve the utility of these data systems to some extent, we are concerned that they will continue to be insufficient to assess Gulf War illnesses such as tumors.
DOD and VA were concerned about references in the report to illnesses other than cancer. They noted the report does a good job in highlighting the strengths and weaknesses of the data sources available for assessing the incidence of tumors among Gulf War veterans, but that it does not include a comparable review of other non-cancer health information systems or research studies that may exist. We modified the report title and some language to clarify the scope of our work, where appropriate. However, we continue to believe that our findings regarding the key data sources discussed in this report, with the exception of the cancer registry reporting systems, are applicable to assessing other illnesses that may be associated with Gulf War veterans. For example, the lack of outpatient data affects the reporting of many types of illnesses, not just tumors. Although there are other research studies underway to investigate different symptoms and illnesses in the Gulf War veteran population, we are unaware of any other government data reporting systems that provide a means to assess illnesses in this population.
VA officials also questioned statements we made in the report highlighting the importance of collecting baseline information on tumor incidence among Gulf War veterans at this time. They believe that available research information, which has shown a lack of any increase in cancer mortality or hospitalization rates among Gulf War veterans, does not support investing in further monitoring, given other Gulf War research priorities. VA contends that the usefulness of such baseline information is negligible for future research efforts because any future rates of tumors would need to be compared with military and general population controls from the same time period. While we agree that such comparisons are important, the existing research information on tumor incidence in the Gulf War population is quite limited and, therefore, cannot be used as a basis to say there is no increased rate in tumor cases. Tracking incidence over time is also important in order to assess whether the number of new cases is occurring at a similar or different rate than a comparison group of veterans.
DOD also made the point that establishing a national cancer registry with standardized reporting of cancers across the nation, would address many of the data problems noted in our report and be of use in assessing military and nonmilitary populations. A recommendation for establishing such a reporting system was beyond the scope of our work.
DOD and VA also provided technical comments, which we incorporated where appropriate.
section_title: Scope and Methodology
paragraphs: The focus of our work was to identify and assess available data sources and federal research initiatives to estimate the incidence of tumors among Gulf War veterans. To address these objectives, we reviewed relevant literature and agency documents and collected information directly from agency officials and selected outside experts. We identified and reviewed available literature on a variety of topics including Gulf War illnesses, cancer epidemiology, and other uses of federal and state medical information systems (e.g., studies of cancer rates among Vietnam veterans potentially exposed to Agent Orange). We also interviewed officials from DOD, VA, the National Cancer Institute, and the Medical Followup Agency of the Institute of Medicine. Through these interviews and materials collected, we learned about the availability of data, their strengths and limitations, and their applications for monitoring the incidence of tumors and other illnesses. We also discussed past and ongoing research efforts to assess the magnitude and frequency Gulf War illnesses with agency officials.
An important limitation of our study is that we did not obtain the databases and independently assess their reliability and validity. We did not, for example, assess data entry procedures to verify the accuracy and completeness of reporting. Furthermore, we did not evaluate the effectiveness of database quality assurance practices.
We conducted our review between July and December 1997 in accordance with generally accepted government auditing standards.
As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies of this report to other interested congressional committees, the Secretaries of Defense and Veterans Affairs, and other interested parties. We will also make copies available to others upon request.
If you or your staff have any questions or would like additional information, please contact me at (202) 512-3092. Major contributors to this report were John Oppenheim, Dan Engelberg, and Lê Xuân Hy.
section_title: Comments From the Department of Defense
paragraphs: The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 37050 Washington, DC  20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (202) 512-6061, or TDD (202) 512-2537.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.