United States General Accounting Office

GAO Report to the Chairman and Ranking

Minority Member, Committee on Armed

Services, U.S. Senate

October 1998 ENVIRONMENTAL

CLEANUP

DOD’s Implementation

of the Relative Risk

Site Evaluation

Process

GAO/NSIAD-99-25

GAO United States

General Accounting Office

Washington, D.C. 20548

National Security and

International Affairs Division

B-279582 

October 7, 1998

The Honorable Strom Thurmond

Chairman

The Honorable Carl Levin

Ranking Minority Member

Committee on Armed Services

United States Senate

The Department of Defense (DOD) uses a relative risk site evaluation

process as part of its decision criteria to allocate about $2 billion annually

to clean up contaminated sites that pose the greatest risk to human health

and the environment. DOD estimates that it will spend about $27 billion to

complete cleanup on contaminated sites from fiscal year 1998 to

completion.

The Senate Report on the National Defense Authorization Act for Fiscal

Year 1998 (Report 105-29, June 17, 1997) requires us to review DOD’s

relative risk site evaluation process. In response to that requirement, this

report addresses (1) the extent to which DOD has issued uniform relative

risk site evaluation guidance and the application of the guidance by the

defense components and (2) whether the relative risk site evaluation

process provided data that enabled the defense components to categorize

sites and prioritize required work. This report complements the

information previously provided to you on DOD’s relative risk site

evaluation process.1

Background DOD adopted the relative risk site evaluation process in 1994 to address

inconsistencies in the evaluation methods it used to prioritize

contaminated sites. The process is intended to provide defense

components with a common methodology for assigning high, medium, and

low relative risk categorizations at each potentially contaminated site on

the basis of evaluations of water, soil, and sediments for their

contamination levels; the likelihood of contaminant migration; and the

presence of potential receptors such as humans, plants, and animals. In

addition, DOD’s relative risk site evaluation guidance requires that sites

lacking sufficient information for a relative risk site evaluation be given a

“not evaluated” designation, and provides that certain other sites do not

1

Environmental Protection: Information Used for Defense Environmental Management

(GAO/NSIAD-97-135, June 11, 1997) and Environmental Cleanup: DOD’s Relative Risk Process

(GAO/NSIAD-98-79R, Feb. 26, 1998).

Page 1 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

require evaluation.2 Not evaluated sites, sites that do not require

evaluation, and sites with risk characterizations are reported in the

Defense Environmental Restoration Program annual reports and budget

justification exhibits provided to Congress.

Environmental remediation includes cleanup and other efforts aimed at

reducing the risk to an acceptable level. The Comprehensive

Environmental Response, Compensation, and Liability Act of 1980, as

amended, serves as the statutory basis for the environmental remediation

of contaminated sites. Under the act, the Environmental Protection

Agency (EPA) ranks sites for inclusion on the National Priorities List on the

basis of public health risks and other factors.3 DOD’s relative risk site

evaluation process provides a tool for categorizing sites and sequencing

priorities for cleanup on the basis of relative risk. In the relative risk site

evaluation process, relative risk site evaluations are used only to screen

and categorize sites and are not substitutes for baseline risk assessments.4

DOD has stated that the relative risk site evaluation process provides a

quantifiable basis for justifying requirements and allocating funds. It has

also stated that relative risk is only one of the priority-setting factors

managers consider for programming and sequencing work at and across

defense installations. Also important, according to DOD, are such things as

program and economic evaluations and baseline risk assessments that

managers may use to determine which sites should be worked on first in

light of available resources.5

Since DOD’s implementation of the process in 1994, Congress has raised

questions about the relative risk site evaluation methodology. In

2

The guidance requires sites designated as not evaluated to be programmed for additional study, a

removal action if warranted, or other appropriate response action, including deferral, before they are

evaluated. DOD does not require relative risk site evaluations for sites that require only the removal of

building demolition/debris or contain abandoned ordnance, have final cleanup remedies in place, or

are designated as “response complete”—require no further cleanup action. A risk-based priority

system is being developed for unexploded ordnance removal.

3

The National Priorities List is the list of the nation’s most heavily contaminated sites.

4

DOD’s 1997 updated Relative Risk Site Evaluation Primer and subsequent March 1998 guidance

amplify this policy. Under the act, DOD performs comprehensive baseline risk assessments to identify

potential exposure on receptors such as humans, and according to DOD, these assessments are used

to justify decisions to clean up.

5

DOD’s March 1998 “Management Guidance for the Defense Environmental Restoration Program”

details other management factors that may be considered in sequencing restoration requirements.

These factors include, among other things, legal requirements; actual and anticipated funding

availability; and acceptability of the action to regulators, tribes, and public stakeholders. DOD’s primer

describes the relative risk site evaluation process, and the Quality Assurance Plan defines objectives

for relative risk site evaluation data as established by the Environmental Security office.

Page 2 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

September 1996, concerned that a high proportion of cleanup sites was

justified as “high relative risk,” the House Committee on National Security

requested that we review the relative risk process.6

 In June 1997, the

Senate Committee on Armed Services questioned the credibility of DOD’s

risk-based approach and the degree to which it facilitates the

establishment of legitimate funding priorities. Because of this concern, the

Committee directed DOD to define the elements of a relative risk site

evaluation, develop uniform guidance for evaluations and ranking, and

ensure consistent application of the guidance.

Results in Brief DOD’s Environmental Security office provided guidance in August 1997 on

its relative risk site evaluation process to defense components by updating

its 1994 Relative Risk Site Evaluation Primer and publishing a Quality

Assurance Plan. The principal change in the updated primer was to

establish new, DOD-wide criteria for defining low-risk sites. Defense

components are to provide site evaluation results to DOD semi-annually.

The Environmental Security office issued the Quality Assurance Plan to

help ensure the integrity of data that is reported to DOD managers and

Congress. The plan requires defense components to ensure that data are

credible, auditable, accountable, and consistent and to follow specific data

verification and reporting procedures. For example, the plan requires

defense components to review the process used to derive relative risk site

evaluations for accuracy and consistency, and report the results to

appropriate organizational elements.

DOD used the relative risk site evaluation process as one of several key

factors for making site funding priority decisions. Our analysis of site

evaluation data being reported by the defense components showed that

over 99 percent of the reported categorizations were consistent with DOD’s

criteria. However, while consistent with the criteria, we did note two

categories that DOD identifies as high relative risk and evaluation not

required, account for two-thirds of the sites and three-fourths of the

estimated $15 billion completion costs at the about 6,000 sites we

analyzed. The high relative risk category contains 1,622 sites with an

estimated $9 billion cost to complete, and the evaluation not required

category contains 2,584 sites, with an estimated $2.2 billion cost to

complete. Also, sites in the evaluation not required category may have high

to low or no known levels of contamination, and costly long-term remedial

actions or no anticipated costs. With so many sites in these categories, the

6

Environmental Protection: Information Used for Defense Environmental Management

(GAO/NSIAD-97-135, June 11, 1997).

Page 3 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

designations are not as helpful as they could be to managers as one of the

key pieces of information they use in making cleanup priority decisions

among competing projects. They are also not as helpful as they could be in

assessing the status of sites not required to be evaluated. We found that

officials at the installations we visited were differentiating further within

the categories to aid in their decision-making processes.

DOD Actions to

Provide Uniform

Guidance and Apply

the Relative Risk Site

Evaluation Process

DOD’s Environmental Security office has taken steps to provide uniform

relative risk site evaluation guidance to defense components. The office

updated, in August 1997, its 1994 Relative Risk Site Evaluation Primer to

establish new, agencywide criteria for categorizing low-risk sites. At that

time, the office also published a new Quality Assurance Plan to help

ensure data integrity in the application of the relative risk site evaluation

process.7

 As a result of the updates to the primer and the establishment of

the Quality Assurance Plan, defense components updated fiscal year 1996

and 1997 relative risk data. The updated data showed changes of pathway

or receptor data, or changes of 20 percent or greater in reported

contamination levels, for about 2,100 of the 6,000 sites considered in our

review. Our analyses of data submitted for DOD’s 1997 annual report

showed that over 99 percent of the defense components’ relative risk

categorizations were consistent with the Environmental Security office’s

most recent criteria. DOD corrected the data for the other 1 percent (92

sites) where we found that categorizations were not consistent with the

reported data. Officials from the Environmental Security office stated that

they included the revised data in the current fiscal year 1997 annual report

to Congress.8

About 1,600 of the 6,000 sites changed relative risk rankings from one year

to the next because of changes in contamination data or site status and/or

because of the establishment of a new threshold below which sites must

be ranked as low relative risk. Table 1 shows the aggregate changes from

fiscal year 1996 to 1997 that affected 1,637 of the 6,015 sites that were

common to both years.

7

DOD’s primer describes the relative risk site evaluation process, and the Quality Assurance Plan

defines objectives for relative risk site evaluation data as established by the Environmental Security

office.

8

In discussions with Environmental Security officials, they stated that the services and Environmental

Security had discovered and corrected many, if not all, of the errors we had identified during DOD’s

quality assurance process.

Page 4 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

Table 1: Reported Changes in Site

Evaluation Categories (fiscal years

1996-97)

Sept. 30,

1996

Sept. 30,

1997a 1996-97

1996-97 1996-97

Category

Number

of sites

Number

of sites

Net

change Decreases Increases

High 1,830 1,594 –236 443 207

Medium 626 674 48 218 266

Low 779 645 –134 369 235

Not evaluated 932 553 –379 506 127

Not required 1,848 2,549 701 101 802

Total 6,015 6,015 0 1,637 1,637

aAt the time of our analysis, Environmental Security officials stated that they considered the data

as draft input to the annual report to Congress on environmental restoration.

Note: This table does not include sites that the Environmental Security office removed from the

fiscal year 1996 data or added to the fiscal year 1997 data. For the 97 high-cost installations in

our review, DOD provided data on 6,088 sites for fiscal year 1996 and 6,279 sites for fiscal 

year 1997; 6,015 sites were common to both years.

New Criteria for Low

Relative Risk

In August 1997 Environmental Security officials revised the Relative Risk

Site Evaluation Primer to provide more specific guidance for categorizing

sites. The only substantive change to the primer was the addition of a

requirement that media (water, soil, and sediment) with a numeric value

called a “contaminant hazard factor” below 0.005 be assigned to the low

relative risk category.9

 The Environmental Security office made this

change to ensure that the defense components assign to the low relative

risk category sites with reliable analytical data that are within

measurement ranges normally found in noncontaminated areas

surrounding a contamination site.10 Although only five sites actually

changed categories from fiscal year 1996 to 1997 as a result of the new

criteria, media ranking changes that occurred from one year to the next

show that the site data appeared to be consistent with DOD’s overall

criteria for assigning relative risk categories.11 For fiscal year 1997, DOD

9

The contaminant hazard factor is a ratio that compares contamination levels with goals that DOD

calls comparison values.

10Prior to the implementation of the revised primer, factors below 0.005 and within these measurement

ranges could be categorized high or medium. Officials believe that assigning such a limit will help them

to more accurately identify appropriate relative risk categories.

11We analyzed data reported to the Environmental Security office by the defense components after

data were screened by Environmental Security officials. We verified relative risk site evaluation

categories and supporting data only at the six locations we visited. In addition, defense components

dropped some sites from their inventory, for example, Anniston Army Depot, one of the site locations

we visited. Based on additional data, managers deleted this previously high risk site with an estimated

cost to complete of $118.5 million.

Page 5 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

reported that 184 of 6,279 sites had contamination levels meeting the new

criteria as low relative risk. Table 2 shows the five sites that changed

categories.

Table 2: Sites Where the Reported

Overall Relative Risk Category

Changed as a Result of New DOD

Criteria in 1997 Site

Fiscal year

1996 relative

risk category

Fiscal year

1997 relative

risk category

Underground Storage Site 00009, Marine

Corps Air Station, Cherry Point, North Carolina

High Low

Site 000005, Marine Corps Air Station, El Toro,

California

Medium Low

Site 000007, Naval Air Station, Moffett Field,

California

High Low

Site 00043, Naval Air Station, Patuxent River,

Maryland

Medium Low

Site 00048, Naval Coastal Station, Stockton,

California

Medium Low

Note: Our review of the reported data showed that the contamination levels for each of the five

sites either did not change significantly from one year to the next or did not change at all.

Relative Risk

Categorizations Consistent

With DOD Criteria

Our review of data submitted for DOD’s 1997 annual report showed that

over 99 percent of the defense components’ updated relative risk

categorizations were consistent with the Environmental Security office’s

most current criteria. Furthermore, before DOD published its most recent

annual report to Congress, DOD corrected data for the 1 percent (92 sites)

where we found that categorizations were not consistent with the reported

data.

Specifically, our review of the reported data showed that 28 sites had

contamination levels below DOD’s threshold for low relative risk but were

categorized as medium or high relative risk. We brought these sites to the

Environmental Security office’s attention during our review. Officials

subsequently provided the final data for the sites in question, which

revised data for 25 of the 28 sites and adjusted the relative risk evaluation

categories for the remaining 3 sites. We also noted that another 67 sites

had overall relative risk categories and/or individual media rankings that

did not appear to be supported by the reported data. Of these, 61 had

incorrect overall relative risk categorizations, and 6 appeared to have

correct overall relative risk categorizations despite errors in supporting

Page 6 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

media rankings.12 Environmental Security officials subsequently provided

us with supplemental spreadsheets showing that the final data had been

corrected by either changing the media or overall relative risk site

evaluation categorizations.

Usefulness of Relative

Risk Evaluation

Process for

Categorizing Sites

Our review of the reported fiscal year 1996 and 1997 relative risk

evaluation data for over 6,000 sites shows that, with some minor

exceptions, the relative risk site evaluation process enabled the defense

components to categorize sites. However, two categories account for

two-thirds of the sites and three-fourths of the estimated completion cost:

the high relative risk category (1,622 sites with $9 billion cost to complete)

and evaluation not required category (2,584 sites with an estimated

$2.2 billion cost to complete). Having such a large number of sites in these

two categories in comparison with the other categories, raises questions

as to how useful the categorizations are to managers in assessing relative

risks. We observed, however, that officials at the installations we visited

were attempting to further differentiate within each of the two categories

to aid in the decision-making process.

Wide Range of

Characteristics in High

Relative Risk Category

DOD’s Relative Risk Site Evaluation Primer states that for all environmental

media, three factors (contaminant hazard factor, migration pathway

factor, and receptor factor) are used in determining a site’s overall relative

risk evaluation categories of high, medium, or low based on 27 possible

different ways these factors can be related to each other. The high relative

risk categorizations can be given based on a wide variety of

characteristics; 8 out of 27 possible different combinations of contaminant

levels, pathways, and receptors can be categorized as high, and the level of

contamination can range from minimal to significant.13 In fiscal year 1997,

about 55 percent (1,622) of the 2,964 high, medium, and low relative risk

sites fell in the high relative risk category.14 These sites have an estimated

cleanup completion cost of over $9 billion of the reported $15 billion total

to complete cleanup at sites in all of the categories.

12Defense components assign a ranking of high, medium or low to each affected media before

assigning an overall relative risk categorization. In some cases, the supporting media rankings were

inaccurate, but the overall relative risk categorization was correct.

13DOD defines contaminant hazard factors greater than 100 as “significant;” 2 to 100 as “moderate;” and

less than 2 as “minimal.” Our analysis showed that for the fiscal year 1997 sites categorized as high

relative risk, the contaminant hazard factors for the four highest groundwater contamination sites

were more than 3 million times greater than values for the four lowest sites.

14The 2,964 fiscal year 1997 relative risk site categorizations were as follows: high = 1,622; 

medium = 686; and low = 656.

Page 7 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

Specific characteristics must be present for a site to be categorized high

relative risk. Table 3 shows the eight combinations of the three factors

that permit categorizing sites as high relative risk.

Table 3: Combinations of

Characteristics Required for the High

Relative Risk Category

Contaminationa Migration Pathway Receptor

1. Significant Evident Identified

2. Significant Evident Potential

3. Significant Potential Identified

4. Significant Potential Potential

5. Moderate Evident Identified

6. Moderate Evident Potential

7. Moderate Potential Identified

8. Minimal Evident Identified

aOur report entitled Environmental Cleanup: DOD’s Relative Risk Process (GAO/NSIAD-98-79R,

Feb. 26, 1998) describes DOD’s process for determining contamination levels.

Groupings of fiscal year 1997 high relative risk sites having similar

reported characteristics of contamination, pathway, and receptor showed

little change from our previously reported analyses. The major difference

between our June 1997 analysis of 266 selected high relative risk sites and

our current analysis of 1,622 fiscal year 1997 high relative risk sites was

that the percentage of sites in the significant category fell from 20 percent

to 15 percent. Sites in this category all had reported significant

contamination, an evident pathway, and an identified receptor. In

commenting on a draft of this report, DOD stated that these percentages

indicate that the combination of relative risk and other management

factors is working very well in sequencing requirements because the

components are focusing their efforts on the high relative risk sites, and,

within the high category, they are focusing on the sites with higher levels

of contamination that also have evident pathways and identified receptors.

Figure 1 shows that there has been little change in site categorizations

between fiscal year 1996 and 1997.

Page 8 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

Figure 1: DOD Sites With High Relative Risk

20%

27%

7%

46%

15%

29%

7%

49% 49% 789 sites with contamination levels below DOD 

comparison value 1.0, or potential pathways or receptors 

15% 246 sites with 

 significant contamination,

 an evident pathway,

 and an identified receptor

29% 466 sites with moderate contamination, 

 an evident pathway, and an identified receptor 

Fiscal year 1996 data as of June 1997 Updated fiscal year 1997 data as of January 1998

7% 121 sites with minimal

 contamination, an evident

 pathway, and an identified receptor

46% 123 sites with contamination levels below DOD 

comparison value 1.0, or potential pathways or 

receptors 

20% 52 sites with significant 

contamination, an evident

pathway, and an identified receptor

27% 71 sites with moderate 

 contamination, an evident pathway,

and an identified receptor

7% 20 sites with

 minimal contamination,

 an evident pathway, and

 an identified receptor 

Source: Our analysis of high relative risk sites based on data provided by the Environmental

Security office.

As shown in figure 1, we also reported in June 1997, that 54 percent of the

high relative risk sites had significant, moderate, or minimal

contamination with an evident pathway and identified receptor. DOD

criteria require the categorization of sites to be evaluated based on

available data. Forty-six percent of the high relative risk sites were first

reported as having contamination levels that fell in EPA’s range of

acceptable risk, or had pathways or receptors that were identified as

Page 9 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

potential although the available data were not reported.15 This latter

category increased to 49 percent for fiscal year 1997.

Finally, the high relative risk category, which is disproportionately large

compared with the medium and low categories, contains contamination

levels ranging from highly contaminated to minimally contaminated. To

make this large category more meaningful, installation managers have

ranked further within the category. They told us they found the additional

differentiation useful in making management and assessment decisions.

Range of Characteristics in

Evaluation Not Required

Category

DOD’s Primer also specifies the circumstances under which site evaluations

are not required and permits a range within the evaluation not required

category as to what can be included. The category can include sites that

range from little or no contamination to sites with high levels of

contamination. The sites may require costly long-term cleanup actions or

less costly monitoring, continued expenditures under other programs, or

no anticipated costs. DOD officials stated the category is one of the ways

DOD defines success in the restoration program. They stated that they

expect this category to grow as final remedies are put in place and the

only funding requirements left, if any, are for remedial action operations or

long-term monitoring. The evaluation not required category accounted for

41 percent of all five site evaluation categories for fiscal year 1997.16

Continued remediation can be costly, involving millions of dollars over

long periods of time. A recent Army study on the effectiveness of soil

remediation programs and groundwater treatment systems noted that

some of the systems “appear to have an indefinite operational lifetime with

no known or estimated target date for completion of remediation.”17

Annual costs for the operation and maintenance of Army remediation

systems are approximately $60 million. One remediation system at the

Twin Cities Army Ammunition Plant, Minnesota, is planned to operate

through the year 2080 at a total estimated cost of over $300 million.

15These include contamination levels with a DOD comparison value below 1.0. In commenting on a

draft of this report, DOD stated some, but not all, of its comparison values are based on EPA’s

Preliminary Remediation Goals. For carcinogens, DOD stated that its comparison values are 100 times

the corresponding remediation goal value.

16The five categories include high, medium, low, not evaluated, and evaluation not required. For fiscal

year 1997, 2,584 sites were categorized as evaluation not required. The evaluation not required

category also accounted for nearly 78 percent of those sites categorized as either not evaluated or

evaluation not required.

17Evaluation of the Effectiveness of Existing Soil Remediation Programs and Groundwater Treatment

Systems in the U.S. Army, U.S. Army Science Board Infrastructure and Environment Issue Group

(Jan. 1998).

Page 10 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

Our analysis of fiscal year 1997 evaluation not required sites showed that

• At Bangor Naval Submarine Base, Washington, one of two sites that fall

under the evaluation not required category has implemented a long-term

“pump and treat” remedial action; the other site contains high levels of

contamination requiring regulators to agree on future actions. The “pump

and treat” remedial action is part of an estimated $26.7 million remediation

effort that is not immediately evident in the reported data. DOD’s annual

report to Congress shows one evaluation not required site, but provides no

data on continued associated estimated costs.

• At Camp LeJeune, North Carolina, a burn area site was found to require no

further action after a remedial investigation was completed in 1996.

Subsequently, the relative risk ranking was changed from high to

evaluation not required. The site had previously been listed as high due to

the presence of previously detected contaminants, which were not present

during the remedial investigation. Additionally, within the underground

storage tank program, two sites had been ranked as low and medium risk,

respectively. After corrective action plans were completed, Camp LeJeune

officials discovered that existing groundwater contamination at both sites

contained chlorinated solvents and should be handled under a separate

program. Therefore, the sites were administratively closed out under the

underground storage tank program (listed as response complete), the cost

to complete reduced to zero, and the relative risk changed to evaluation

not required. These sites were then transferred to the installation

restoration program to address groundwater contamination with the

relative risk listed as high and medium, respectively. The combined cost to

address groundwater contamination is estimated at $13.3 million.

• At Norton Air Force Base, California, one site has begun remedial action

that will include about 10 to 15 years of long-term operations and

monitoring at an estimated cost of $17 million. Another site plans to

excavate about 5,000 tons of soil. At the time of this report, officials had

not yet estimated completion costs.

Reporting of “Evaluation

Not Required” Category

DOD’s annual environmental restoration report to Congress and

attachments to the fiscal year 1998-99 budget submission include status

information on relative risk sites that have been evaluated. But, for

evaluation not required sites, the range of site characteristics is not easily

distinguishable: the documents do not disclose associated ongoing or

completion cost estimates, or which sites have ongoing long-term

Page 11 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

monitoring or operations.18 For example, the latest report to Congress for

fiscal year 1997 separately identifies 17,124 sites considered to be

evaluation not required, of which over 15,000 are response complete.19 But

the report also separately identifies 3,177 sites that are in or planned for

long-term operations without indicating how this information applies to

the evaluation not required sites. DOD reported that such sites cost

$172 million in planned execution for fiscal year 1998. We believe that

such an investment warrants identifying which evaluation not required

sites are affected, and at what cost.

Appendix I illustrates the range of site characteristics on evaluation not

required sites for three installations we visited.

Conclusion DOD has taken actions that, if implemented properly, should improve the

accuracy and consistency of information resulting from relative risk site

evaluations. DOD is updating data in accordance with the Relative Risk Site

Evaluation Primer and the Quality Assurance Plan. However, the current

site categorizations are not as helpful as they could be to managers in

making priority decisions among competing projects, or evident to

Congress in the annual reports. The criteria used for determining whether

a site is high relative risk are overly broad and the evaluation not required

category contains sites with a range of characteristics that isn’t easily

evident in reviewing the annual environmental restoration report.

As the process works now, for example, a site with potential risk to

humans can receive the same priority as a site with actual risk to humans.

Similarly, an evaluation not required site with no contamination or

remediation costs is grouped for reporting with contaminated sites with

years of operational costs to remediate. Given that DOD uses these

categories to define success in its restoration program, it would seem

appropriate to separately identify, for example, the highest relative risk

sites in the annual report, and to identify whether the “evaluation not

required” sites have long-term monitoring or operations, along with their

associated costs.

18Also, the annual report differentiates evaluation not required sites “in progress” from sites with

response complete.

19As reported in “DOD Summary Status” table B-6, for DOD installations and formerly used defense

sites.

Page 12 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

Recommendation To assist managers in making priority decisions in the relative risk site

evaluation process, we recommend the Secretary of Defense direct the

Deputy Under Secretary of Defense for Environmental Security and/or

service components to provide more specific categories that aid in priority

setting and in the accurate reporting of the status of sites.

Agency Comments

and Our Evaluation

In commenting on a draft of this report, DOD stated that it does not concur

with our recommendation. It stated that the relative risk tool was never

intended to differentiate precisely among sites or risk levels and that it is

used in conjunction with a number of other factors to determine the

priority of a site for funding. Consequently, it does not believe our

recommendation would necessarily help managers make better decisions.

DOD stated that flexibility in decision making about project sequencing

may be lost if managers are required to address the top category of high

relative risk sites first. Further, it stated that subdividing the evaluation

not required category would also not improve management or oversight

for sites in this group that have final remedies in place or have completed

response actions. DOD stated that restoration activities at these sites are

limited to remedial action operations and long-term monitoring, which it

treats as non-discretionary cost commitments, not subject to prioritization.

DOD stated that regulatory agencies and the public would be incensed if it

delayed action at remedial sites in the evaluation not required category

based on a presumed priority, rather than fulfill its commitments. Finally,

DOD stated that implementing our recommendation would not benefit DOD’s

program but would actually be detrimental because it would require a

reclassification of existing sites causing restructuring and expansion of

programming and budgeting data.

We continue to believe that implementing our recommendation would be a

useful aid in program management and congressional oversight. As we

pointed out in our report, officials at installations we visited are already

differentiating further within the categories to aid them in their

decision-making process. We added information to our report to make it

clear that relative risk is only one of a number of available factors.

However, the remainder of DOD’s position does not accurately characterize

our recommendation. The intent of our recommendation is to provide

more specific relative risk categories, and would not limit flexibility, since

neither the current system nor our recommendation requires sites to be

funded strictly in relative risk order. Also, our recommendation does not

suggest the interruption or delaying of remedial actions. Such actions, if

taken, would be a natural part of the decision-making process. Finally, we

Page 13 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

recognize that the recommendation could result in sites being reclassified,

but it would not necessitate delays. Reclassification is a normal part of the

relative risk site evaluation process and is provided for in DOD program

guidance. For example, as noted in our report, 1,637 of 6,015 sites changed

categories from 1996 to 1997.

DOD also suggested technical comments which we incorporated where

appropriate. DOD’s comments and our detailed evaluation are included in

appendix II.

Scope and

Methodology

To determine the extent to which DOD has issued uniform relative risk site

evaluation guidance to defense components, we interviewed and reviewed

policy and procedural documents from officials at DOD’s Office of the

Deputy Under Secretary of Defense for Environmental Security; the Army,

Navy, Air Force, and Defense Logistics Agency headquarters; and selected

defense component field offices. To address how defense installations

applied the relative risk site evaluation process, we visited and obtained

information from officials at DOD’s Office of the Deputy Under Secretary of

Defense for Environmental Security and selected defense component field

offices. We also requested and analyzed fiscal year 1996-97 data on over

6,000 sites for 97 high-cost installations identified in our June 1997 report.

The high-cost installations identified in our June 1997 report were

reported in DOD’s fiscal year 1995 annual report to Congress. Each

installation accounted for more than $20 million of planned funding during

fiscal years 1996-98 or more than $100 million of projected costs from 1996

to completion.

To determine whether the relative risk site evaluation process provided

data that enabled the defense components to categorize sites and prioritize

work, we analyzed the fiscal year 1996-97 data on the over 6,000 sites for

the 97 high-cost installations identified in our June 1997 report. DOD

extracted the data, which are input from the defense components, from

the environmental Resource Management Information System database.

The data were provided to us incrementally from November 1997 to

January 1998. We relied on the accuracy of DOD and service data in

conducting our analysis and selectively verified contamination data for

specific projects but did not verify overall database accuracy. We did not

assess installation judgments regarding pathways and receptors of

individual sites. We visited and/or obtained information on the relative risk

site evaluation process at the following military installations, which

Page 14 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

included installations in addition to the 97 high-cost installations just

identified:

Army

Anniston Army Depot, Alabama

Navy

Engineering Field Activity, Northwest, Naval Facilities Engineering

Command, Bangor Submarine Base, Washington

U.S. Marine Corps, Camp LeJeune, North Carolina

Air Force

Kirtland Air Force Base, New Mexico

Norton Air Force Base, California

Tinker Air Force Base, Oklahoma

For this report, we analyzed reported data before (fiscal year 1996) and

after (fiscal year 1997) DOD’s quality control updates for all of the over

6,000 sites at the 97 high-cost installations considered in our June 1997

report. The sites account for about $15 billion of DOD’s estimated total

remaining cleanup costs of $27 billion.20

We conducted our analysis from November 1997 to May 1998 in

accordance with generally accepted government auditing standards.

20Cost estimates in DOD’s database are reported for individual sites at an installation. Although we

could not verify the estimates for each site, we noted that the total estimates for each installation

generally agreed with the totals reported in DOD’s fiscal year 1996 annual report to Congress.

Page 15 GAO/NSIAD-99-25 Relative Risk Implementation
B-279582 

We are sending copies of this report to other appropriate congressional

committees; the Secretaries of Defense, the Army, the Navy, and the Air

Force; the Commandant, U.S. Marine Corps; and the Directors, Office of

Management and Budget and Defense Logistics Agency. We will also make

copies available to others on request.

David R. Warren, Director

Defense Management Issues

Page 16 GAO/NSIAD-99-25 Relative Risk Implementation
Page 17 GAO/NSIAD-99-25 Relative Risk Implementation
Contents

Letter 1

Appendix I 

Characteristics of

Sites Designated

Evaluation Not

Required at Three

Installations

20

Appendix II 

Comments From the

Department of

Defense

22

Appendix III 

Major Contributors to

This Report

34

Tables Table 1: Reported Changes in Site Evaluation Categories 5

Table 2: Sites Where the Reported Overall Relative Risk Category

Changed as a Result of New DOD Criteria in 1997

6

Table 3: Combinations of Characteristics Required for the High

Relative Risk Category

8

Figure Figure 1: DOD Sites With High Relative Risk 9

Abbreviations

DOD Department of Defense

EPA Environmental Protection Agency

Page 18 GAO/NSIAD-99-25 Relative Risk Implementation
Page 19 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix I 

Characteristics of Sites Designated

Evaluation Not Required at Three

Installations

Relative risk category

Site Fiscal year 1996 Fiscal year 1997 Status

Bangor 00204 H R Remedial actions implemented; in long-term operation

Bangor 00025 H R Remedial actions implemented; long-term monitoring

Norton WT007 R R Final decision document issued; no further response action

required

Norton S1011 R R Final decision document issued; no further response action

required

Norton TU015 R R Final decision document issued; no further response action

required

Norton SS018 R R Final decision document issued; no further response action

required

Norton SP009 R R Final closure report issued

Norton WR020 R R Final basewide radionuclide characterization report issued; no

further actions anticipated

Norton CG097 R R Remedial action implemented; 10-15 years long-term

operations and monitoring anticipated

Norton DA096 R R Closure report issued; remedial action complete; long-term

monitoring required

Norton DP003 R R No further response action decision document signed

Norton DP004 R R No further response action decision document signed

Norton ID022 R R Final closure report issued; no removal action required

Norton AOC N R Remedial actions implemented; closure report in regulatory

review

Norton SA017 N R Remedial actions implemented; long-term monitoring

Norton DP012 N R Deed restriction placed on the use of contaminated parcels

Norton LF010 N R Deed restriction placed on the use of contaminated parcels

Norton SS008 H R Remedial actions implemented; site closed

Norton SI001 H R Remedial actions implemented; long-term monitoring

Norton AT005 H R Remedial actions implemented; additional excavation planned

Norton DP014 H R Final closure report issued; removal action complete

Norton WT013 H R Final closure report issued; removal action complete

Norton SA019 L R Remedial actions implemented—deed restriction

Norton TV021 N R Final closure report prepared; additional contamination

detected

Norton S1016 N R Remedial actions implemented; long-term monitoring

LeJeune UST 000047 M R Corrective action plans completed; continuing work under

installation restoration program as site 00091

(continued)

Page 20 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix I 

Characteristics of Sites Designated

Evaluation Not Required at Three

Installations

Relative risk category

Site Fiscal year 1996 Fiscal year 1997 Status

LeJeune UST 000023 L R Corrective action plans completed; CERCLA investigation is

underway under a separate installation restoration program;

site name changed to site 00073a

LeJeune 00016 H R Remedial investigation completed and record of decision

signed; no further action required

Note: Categories are as reported by DOD’s Environmental Security office—H=high relative risk;

L=low relative risk; R=not required; and N=not evaluated.

aLeJeune site 00073, with an estimated completion cost of $11.3 million, is also reported in DOD’s

fiscal year 1997 data as a high relative risk site.

Page 21 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

See p. 13.

See comment 1.

See comment 2.

See comment 3.

See comment 4.

Page 22 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

See comment 5.

See comment 6.

See comment 4.

Page 23 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

See p. 13.

See comment 7.

Page 24 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

See comment 8.

Now on p. 1.

See comment 8.

See comment 8.

Now on p. 2.

See comment 8.

See comment 8.

Now on p. 2.

See comment 8.

See comment 8.

See comment 8.

Now on p. 3.

See comment 8.

Page 25 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

Now on p. 3

See comment 9.

See comment 7.

Now on p. 14.

See comment 10.

Now on p. 7.

See comment 11.

Page 26 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

Now on p. 7.

See comment 8.

See comment 8.

See comment 8.

Now on p. 8.

See comment 12.

Now on p. 9.

See comment 8.

Now on p. 9.

See comment 8.

Page 27 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

Now on p. 10.

See comment 9.

See comment 7.

Now on p. 11.

See comment 13.

See comment 8.

Now on p. 11.

Page 28 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

Now on p. 11.

See comment 8.

See comment 8.

See comment 8.

See comment 8.

See comment 8.

Now on pp. 12-13.

See comment 15.

See comment 7.

Page 29 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

See comment 7.

Page 30 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

The following are GAO’s comments on the Department of Defense’s (DOD)

letter dated September 4, 1998.

GAO Comments 1. DOD’s high relative risk category is overly large in relation to the medium

and low categories, and thus does not provide three broad categories. To

make this more clear, we added an example in the report that DOD’s

highest contaminant hazard factors were more than 3 million times greater

than the lowest ones at sites within the high relative risk category.

2. We agree that relative risk is used in conjunction with other important

factors, but DOD’s conclusion does not follow. For those site rankings

where relative risk affected managers’ decisions, the additional category

data would help the decision. We added to our report an example of a

previously high relative risk site that managers deleted from inventory

based on additional data.

3. DOD’s comment that the two types of sites are non-discretionary cost

commitments is inconsistent with defense components’ ongoing

reconsideration of remedial action operations, such as cited in our report.

We agree such sites are not subject to further prioritization.

4. Our recommendation does not necessitate interruption of remedial

actions. Reclassification is a normal part of the relative risk site evaluation

process and is provided for in DOD’s Relative Risk Site Evaluation Primer.

Also, as stated in our report, defense components are themselves

considering where such actions may be appropriate.

5. Although our recommendation could result in reclassified sites, such

actions are a normal part of DOD’s process, as evidenced by the over 1,600

category changes from 1996 to 1997.

6. Our recommendation does not prohibit managers from using other

management considerations in their decision process. It simply suggests

that more specificity within the two categories could be more helpful to

decisionmakers.

7. Both our draft and final reports recognize that relative risk is one of

several factors considered by decisionmakers. Also, as we stated in

addressing DOD comments on our June 1997 report, we recognize that the

relative risk site evaluation process is an initial screening method that is

only one of the factors considered by decisionmakers. Further, DOD fully

Page 31 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

concurred with the description of the relative risk site evaluation process

in our report entitled Environmental Cleanup: DOD’s Relative Risk Process

(GAO/NSIAD-98-79R, Feb. 26, 1998). The recommendation, as written, is

sufficiently broad as to allow DOD flexibility in how best to differentiate

within the categories. We do not believe that better defining and making

categories more specific would limit flexibility because neither the current

system nor our recommendation requires sites to be funded strictly in

relative risk order. Thus, the recommendation also would not delay

remedial action, require programming and budget data restructuring and

additional data collection, upset the existing system for identifying

requirements, or disrupt the existing program.

8. We revised our report to reflect DOD’s suggested changes.

9. We addressed DOD’s questions. The evaluation not required category

may have high to low or no known levels of contamination, and costly

long-term remedial actions or no anticipated costs. Officials at

installations we visited were differentiating further within both the high

relative risk and evaluation not required categories to aid in their

decision-making processes. We inserted DOD’s statement that it uses the

evaluation not required category to define success in the restoration

program.

10. We revised our report to state that the updated data referred to the

about 6,000 sites considered in our current review. This is simply a

statement of fact describing the number of changes and their extent, and

its purpose is evident, as stated.

11. We revised our report to make it more clear that the $2.2 billion refers

to sites in the evaluation not required category.

12. The statements simply reflect the results of our analysis. We added

language to reflect DOD’s comment that it believes the combination of

relative risk and other management factors is working well in sequencing

requirements.

13. The information in our draft report was based on specific input from

Marine Corps officials. Although the information was accurate as stated,

we included DOD’s suggested language.

14. We modified this section of the report to focus on reporting of the

evaluation not required category.

Page 32 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix II 

Comments From the Department of Defense

15. We revised our report to include specific reference to DOD’ s positive

actions. Although our tests confirmed that components followed

processes for contamination levels, we could not state that 100 percent of

all data met requirements.

Page 33 GAO/NSIAD-99-25 Relative Risk Implementation
Appendix III 

Major Contributors to This Report

National Security and

International Affairs

Division, Washington,

D.C.

Charles I. Patton, Jr.

Uldis Adamsons

Elizabeth Mead

Dallas Field Office David Marks

Kansas City Field

Office

Virgil Schroeder

Ben Douglas

Steve Pruitt

(709320) Page 34 GAO/NSIAD-99-25 Relative Risk Implementation
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100