ENVIRONMENTAL

INFORMATION

EPA Needs Better

Information to Manage

Risks and Measure

Results

Statement of Peter F. Guerrero, Director

Environmental Protection Issues,

Resources, Community, and Economic Development Division

United States General Accounting Office

GAO Testimony

Before the Committee on Environment and Public Works,

U.S. Senate

For Release on Delivery

Expected at 9:30 a.m., EDT

Tuesday

October 3, 2000

GAO-01-97T
Mr. Chairman and Members of the Committee:

We appreciate the opportunity to discuss our observations on the data that the

Environmental Protection Agency (EPA) needs to manage its programs more effectively.

In reports going back to our comprehensive general management review of EPA in 1988,1

we have identified numerous long-standing problems in the agency’s efforts to collect

and use environmental data. Drawing from this work, I will discuss today the limitations

in the data that EPA needs to (1) set risk-based priorities for its programs and (2)

develop outcome-oriented measures of its programs’ results. Our observations are as

follows:

• EPA’s ability to assess risks and establish risk-based priorities has been hampered by

data quality problems, including critical data gaps, databases that do not operate

compatibly with one another, and persistent concerns about the accuracy of the data

in many of EPA’s data systems. While EPA’s priorities should reflect an

understanding of relative risk to the environment and public health, good data often

do not exist to fully characterize risk. In the absence of reliable data, public

perceptions of risk can influence how EPA determines its priorities and allocates

resources. EPA has taken major steps during the past few years to improve its data

and to better inform the scientific community and general public of environmental

and public health risks. To finish this job, the agency will need to expand its data

improvement initiatives to fill key gaps in its data, take advantage of opportunities to

develop and implement data standards to achieve compatibility among environmental

databases, and ensure the accuracy of its data.

• Measuring the results (outcomes) of its programs is critical to determining EPA’s

effectiveness. Nevertheless, the agency historically has relied on activity-based

output measures, such as the number of inspections performed, because of inherent

technical difficulties in establishing sound linkages among program activities,

environmental improvements, and public health. Spurred by the requirements of the

1

Environmental Protection Agency: Protecting Human Health and the Environment Through Improved

Management (GAO/RCED-88-101, Aug. 16, 1988).
2

Government Performance and Results Act of 1993 (Results Act), EPA has made

progress in recent years in measuring the outcomes of its programs. To ensure future

success in developing outcome measures, however, EPA will need to make a longterm management commitment to overcome major challenges to obtaining the data

needed to show the results of environmental programs.

Background

Since EPA’s establishment in 1970, the federal government has developed a complex

system of laws and regulations to address the nation’s environmental problems. Over

the years, as environmental threats were identified, the Congress responded by enacting

laws to address each problem, incrementally adding to the statutory framework that sets

EPA’s agenda. However, these laws were not coordinated or integrated to provide EPA

with an overall system for prioritizing problems so that the most serious problems can be

addressed first.

Impelled by budgetary constraints and a growing list of environmental problems, EPA, in

the late 1980s, began to consider whether its resources were being spent on the problems

that pose the greatest risks to public health and the environment. The agency concluded

that the nation actually was devoting more resources to problems that had captured

public attention than to problems that were less well known but potentially more

serious. Subsequently, EPA began incorporating the concept of relative health and

environmental risk into decisions on environmental priorities and emphasizing the need

to identify the most serious risks and to keep the public informed about the relative

seriousness of various environmental problems. To assess risks and deal with those

likely to do the most harm, EPA has recognized that it needs to have adequate

environmental and scientific data to conduct risk assessments, set standards, and

develop regulations. It also needs such data to identify and develop measures of

environmental quality and to assess the effectiveness of its programs by linking program

activities to changes in environmental conditions.
3

EPA Needs Better Data to Establish

Risk-based Program Priorities

Establishing risk-based priorities for EPA’s program activities requires good data on the

use and disposal of thousands of chemicals. To assess human exposure to a chemical,

EPA needs to know how many workers, consumers, and others are exposed; how the

exposure occurs; and the amount and duration of the exposure. For environmental

exposure, EPA needs to know whether the chemical is being released to the air, water,

or land; how much is being released; and how wide an area is being affected. EPA’s

ability to make such assessments is limited by (1) gaps in environmental and health data,

(2) databases that do not operate compatibly with one another, and (3) the lack of an

effective system for ensuring the accuracy of the agency’s data. Although EPA has

implemented several agencywide initiatives to address these problems, each of the

initiatives has encountered obstacles that must be overcome to substantially improve the

agency’s data.

Extensive Gaps Exist in EPA’s Information About the Environment and Health Risks

Our work over the past few years has shown that very little is known about the risks of

potential exposure to chemicals and environmental conditions for workers, the general

public, and plant and animal life. For example, we reported the following:

• EPA’s Integrated Risk Information System, which is a database of the agency’s

consensus on the potential health effects of chronic exposure to various substances

found in the environment, lacks basic data on the toxicity of about two-thirds of the

known hazardous air pollutants.2

• EPA’s National Water Quality Inventory does not accurately describe water quality

conditions nationwide. Only 19 percent of the nation’s rivers and streams were

assessed for the 1996 Inventory (the latest report available at the time of our review),

2

Major Management Challenges and Program Risks: Environmental Protection Agency (GAO/OCG-99-17,

Jan. 1999).
4

as were 6 percent of ocean and other shoreline waters. Pollution of the latter has

resulted in an increasing number of beach advisories and closures in recent years.3

• Of 1,456 toxic chemicals we recently reviewed, data on human exposure were being

collected for only about 6 percent. For example, of the 476 chemicals that EPA

identified as most in need of testing under the Toxic Substances Control Act, only 10,

or 2 percent, were being measured for human exposure. (See table 1.)

Table 1: Extent to Which Human Exposure Data Are Collected for Potentially Harmful Chemicals Through

Surveys of EPA and the Department of Health and Human Services

Priority chemicals

Chemicals measured or

being measured

Number

Description of list In list Number Percentage

Chemicals found most often at the national Superfund 275 62 23

sites and of most potential threat to human health

EPA's list of toxics of concern in air 168 27 16

Chemicals harmful because of their persistence in the 368 52 14

environment, tendency to bioaccumulate in plant or

animal tissues, and toxicity

Pesticides of potential concern as listed by EPA's 243 32 13

Office of Pesticide Programs and the U.S.

Department of Agriculture's Pesticide Data Program

Chemicals that are reported in the Toxic Release 579 50 9

Inventory; are considered toxic; and are used,

manufactured, treated, transported, or released into

the environment

Chemicals most in need of testing under the Toxic 476 10 2

Substances Control Act (Master Testing List)

Note: Our analysis was based on human exposure data collected through the Department of Health and

Human Services’ National Health and Nutrition Examination Survey or EPA’s National Human Exposure

Assessment Pilot Surveys through 2000.

EPA has recognized that it has numerous and significant gaps in its data and has initiated

several efforts to fill at least some of the gaps. For example, under its Environmental

Monitoring and Assessment Program, EPA is working with other federal agencies to

develop information that the public, scientists, and the Congress can use to evaluate the

overall health of the nation’s ecological resources. EPA also recently launched its High

3

Water Quality: Key EPA and State Decisions Limited by Inconsistent and Incomplete Data (GAO/RCED-00-

54, Mar. 15, 2000).
5

Production Volume Challenge Program, which asked chemical companies to voluntarily

generate data on the effects of the chemicals they manufacture or import. As of

December 1999, over 400 participants had agreed to make public, before the end of 2005,

basic hazard data on over 2,000 of 2,800 high-production-volume chemicals, which are

chemicals manufactured or imported into the United States in amounts equal to or

greater than one million pounds per year. Furthermore, EPA’s new information office

will be responsible for encouraging the agency’s program offices to reach out to other

federal agencies as well as to universities, research institutes, and other sources of

environmental information for data that EPA does not collect but that may exist

elsewhere. To date, however, such efforts have been hampered by technological

limitations imposed by the myriad of incompatible information systems in use across the

government.

Moreover, much of the information needed, such as environmental monitoring data, will

be expensive to obtain. Thus, it will be important for EPA to work with the states and

industry to reduce the reporting burden and to encourage efforts to use data that may

already have been collected by other federal agencies or other entities. Likewise, as we

recommended to EPA in our September 1999 report on its information management

activities, it will be essential for the agency to develop a strategy that prioritizes its

requirements for additional data and identifies milestones and needed resources. EPA

can then use this information to support its budget requests.

Incompatible Data Systems Limit the Usefulness of Environmental Data

Over the years, EPA has developed and maintained “stovepipe” data systems that are not

capable of sharing the enormous amounts of data gathered. EPA now recognizes that

common data definitions and formats, known as data standards, are essential to its

efforts to integrate data from various databases, including those of its state partners.

EPA also considers data standards as key to reducing the reporting burden on industry

and the states because such standards would permit integrated, and thus more efficient,

reporting of information to the agency. In recent years, EPA has undertaken several

efforts to develop standards for some of the data items in its information systems.
6

According to the Office of Environmental Information, EPA recently approved six data

standards and expects that all of these standards will be implemented in the relevant

data systems by fiscal year 2003.

EPA recognizes that its current data improvement efforts are only first steps toward its

goal of full data integration. For example, EPA has focused primarily on the

compatibility of its data with those of state environmental agencies, rather than of other

federal agencies and nongovernmental sources. In a May 2000 report, we stated that

improved collaboration among federal agencies in meeting the needs for human

exposure data is essential because individual agencies have different capacities and

skills and separate attempts have fallen short of supporting the large efforts that are

needed.4

EPA’s Science Advisory Board5has also recommended that EPA do more to

link the agency’s databases with external databases. The Board noted that “answering

many health-related questions frequently requires linking environmental data with

census, cancer or birth registry data, or other data systems (such as water distribution

maps) to determine whether there is a relationship between the environmental measures

and health.”6

EPA officials acknowledge the importance of linking EPA’s databases with

those of other agencies at all levels of government. However, they told us that their

actions to do so have been limited by resource constraints and by the fact that EPA’s

statutes do not give the agency the authority to require that other agencies collect or

report data using formats compatible with those used by EPA.

Concerns Persist About the Accuracy of EPA’s Data

In various reviews, we and others have identified persistent concerns about the accuracy

of the data in many of EPA’s information systems. EPA acknowledges that data errors

exist but believes that, in the aggregate, its data are of sufficient quality to support its

programmatic and regulatory decisions. However, EPA has not assessed the accuracy of

4

Toxic Chemicals: Long-Term Coordinated Strategy Needed to Measure Exposures in Humans

(GAO/HEHS-00-80, May 2, 2000).

5

The EPA Science Advisory Board was created by the Congress to provide advice to EPA from scientists

outside the agency.

6

Science Advisory Board, Review of the Agency-Wide Quality Management Program, EPA-SAB-EEC-LTR98-003 (Washington, D.C.: EPA, July 24, 1998).
7

its information systems agencywide, and preventing errors and correcting them once

they have been identified has proved daunting for the agency. For example, in January

1998, an EPA advisory council on information management issues described the

difficulty of correcting errors in EPA’s databases: “Once an error is stored in one or

more of the agency’s systems, making corrections to all those systems is an exercise in

frustration and futility. There is no simple way to ensure corrections are made to all

possible systems.”

To address such problems, EPA revised its agencywide quality system in 1998 to expand

and clarify requirements for how environmental data are collected and managed.

Although the Science Advisory Board recently commended the agency for its

development of this system, the Board also found that its implementation has been

uneven within the agency. Moreover, the Board reported that more than 75 percent of

the states authorized to implement EPA’s environmental programs lack approved quality

management plans for all or some of these programs and thus are likely to be generating

data of unknown quality. We recently reported that EPA’s National Water Quality

Inventory, which EPA uses as a basis for measuring progress under the Clean Water Act,

does not accurately describe water conditions nationwide. While EPA prepares the

Inventory on the basis of data submitted by the states, the states do not use a statistical

sampling design that provides a comprehensive picture of water quality. The Science

Advisory Board has pointed out that EPA programs that rely on data of unknown quality

are exposing themselves, the reliability of their decisions, and their credibility to

criticisms.

Correcting errors in the agency’s data is an important responsibility for the new

information office. This office recently developed an Internet-based system to identify,

track, and resolve errors found in national environmental databases. The system

currently allows individuals to notify EPA of suspected errors in some of the agency’s

major databases, and EPA intends to implement the data correction system in additional

databases during the next two years.
8

Efforts to Develop Outcome-Oriented Performance

Measures Are Constrained by Data Limitations

Well-chosen environmental measures inform policymakers, the public, and EPA

managers about the condition of the environment and provide for assessing the potential

danger posed by pollution and contamination. They also serve to monitor the extent to

which EPA’s programs contribute to environmental improvement and can be used in

future priority-setting, planning, and budgeting decisions. EPA has been aware of the

need for environmental measures since the mid-1970s. Nevertheless, the agency made

little progress in developing such measures until the Results Act mandated their use by

requiring federal agencies to report annually on their progress in meeting performance

goals. Under the Results Act, EPA has begun to set goals and measures that are intended

to help the agency, as well as the Congress and the public, assess the environmental

results of the agency’s activities. While EPA has made progress in adopting more

measures that reflect the environmental or health outcomes of programs, the

overwhelming number of EPA’s measures reflect outputs, such as the number of

inspections performed or regulations issued, and additional progress is needed.

EPA considers getting the data needed to measure results its biggest challenge in

developing outcome-oriented performance measures. To date, EPA and the states have

made limited progress in developing such measures, as these examples indicate:

• Of the 364 measures of performance that EPA has developed for use during fiscal

year 2000, only 69 (19 percent) are environmental outcomes; the other measures

reflect program activities, such as the number of actions taken to enforce

environmental laws. (See table 2.)

• Given inherent uncertainties about the results of research and development activities,

the problem of developing outcome-oriented measures is particularly difficult for

EPA’s science activities. Of 36 measures related to EPA’s strategic goal of “sound

science,” only 2 reflect outcomes.
9

Table 2: EPA’s Analysis of the Number and Type of Annual Performance Measures for Its Strategic

Goals for Fiscal Year 2000

Number of annual

performance measures

EPA's strategic goal Output Outcome Total

Goal 1: Clean Air 19 14 33

Goal 2: Clean and safe water 65 17 82

Goal 3: Safe food 16 1 17

Goal 4: Preventing pollution and reducing 28 14 42

risk in communities, homes, workplaces,

and ecosystems

Goal 5: Better waste management, 34 8 42

restoration of contaminated sites, and

emergency response

Goal 6: Reduction of global and 27 7 34

cross-border environmental risks

Goal 7: Expansion of Americans' right to 28 3 31

know about their environment

Goal 8: Sound science, improved 34 2 36

understanding of environmental risk and

greater innovation to address environmental

problems

Goal 9: A credible deterrent to pollution and 15 3 18

greater compliance with the law

Goal 10: Effective management 29 0 29

Total 295 69 364

Source: GAO’s analysis of EPA data.

In addition to establishing output-and outcome-oriented performance measures, EPA has

adopted a framework for categorizing its performance measures according to the type of

outputs or outcomes to be achieved. As shown in figure 1, most of the performance

measures are outputs involving either research and development efforts or actions by

EPA, states, tribes, or other governmental bodies, such as establishing standards for

hazardous levels of lead in paint, dust, and soil. The other categories represent

outcomes, including measures that focus on risks to ecology, health, or welfare;

pollutants absorbed by the body; and concentrations of pollutants in the environment.

Over time, EPA plans to increase the number of such measures, as it is able to obtain

better data linking its program activities with changes in environmental and health

conditions.
10

Figure 1: Number and Percentage of Performance Measures for Each Type of Activity

Even with better data, it will be a major challenge for EPA to link its environmental

programs and activities to outcomes. Environmental conditions may change because of

a number of factors, including variables such as the weather or economic activity, many

of which are beyond the control of EPA and its state partners. Likewise, it may be

difficult to show the relationship between EPA’s annual program activities and some

outcomes that may not be apparent until many years later. For example, current EPA

activities to reduce the amount of polluting nutrients from fertilizers in the ground may

not result in improved water quality for a decade or more.

EPA program officials recognize that they need additional measures that show the

outcomes of programs, and they have recently taken actions that should strengthen the

agency’s ability to develop them. For example, EPA is developing processes and long-

<1%

1%

3%

7%

8%

24%

57%

Environmental risks or

impacts to ecology, health, or

welfare (1)

Pollutants absorbed by the

body (9)

Research and development (86)

Discharges and emissions of

pollutants (30)

Change in behavior of

regulated parties (24)

Concentrations of pollutants (5)

Actions by EPA, states, tribes, or

other governmental bodies (209)

Total: 364 performance measures
11

term strategies to improve the quality of performance measures and link the activities of

program offices with environmental results. However, substantial resources are required

to identify and test the potential measures. Once the measures are established, gathering

and analyzing the data can be resource-intensive, and it can take years to show

environmental improvement.

Observations

Our prior work has identified numerous problems in the quality of EPA’s data and the

way that the agency manages its data systems. These problems cut across the various

programs regulated by EPA and have limited the agency’s ability to assess risks and

measure environmental results. To its credit, EPA has initiated actions to improve its

information management activities. While EPA has made progress, its initiatives do not

provide a long-term strategy to ensure the completeness, compatibility, and accuracy of

its data. Furthermore, the initiatives have encountered obstacles that highlight the

difficulties facing EPA as it attempts to improve its information management activities.

As we recommended in our September 1999 report, to substantially improve the quality

of the data used to set risk-based priorities and report on progress toward improving

environmental conditions and human health, EPA needs to develop a strategy that

reflects a long-term commitment to resolving data problems. Such a strategy should

include establishing milestones and identifying the resources necessary to fill major data

gaps, identify and develop a- .-1. d - . ended - . ata - 9 bstadwares - . a and - . iomplement - . tm ipyw

at-basps, - 9. and - . ci the agency’s 

fdertal agenisps, 

provio saurcg res urces - . t o - . m ent - 9 ci t icall - a . a - . rquo . ire m na . es . - 9 Ao -1. a . hoge 11 ho - . Eo -1. PA - -1. ce 11 oncuo . 

a comreehoenivhetrategs. Foer cAo 

thyagy of anfotf 
12

plan for the agency. EPA plans to complete the first stage by December 2000, which will

identify broad options for information management over the next several years.

-----

Mr. Chairman, I would be happy to respond to any questions that you or other Members

of the Committee may have.

Contact and Acknowledgments

For further information, please contact Peter F. Guerrero at (202) 512-6111. Individuals making

key contributions to this testimony included Ed Kratzer and Cecilia Lee.

(360000)
Orders by Internet

For information on how to access GAO reports on the Internet, send an e-mail message

with “info” in the body to

info@www.gao.gov

or visit GAO’s World Wide Web home page at

http://www.gao.gov

To Report Fraud, Waste, and Abuse

in Federal Programs

Web site: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: 1-800-424-5454