GAO/RCED-00-270R EPA Science Issues

United States General Accounting Office

Washington, DC 20548

Resources, Community, and

Economic Development Division

B-286079

August 31, 2000

The Honorable F. James Sensenbrenner, Jr.

Chairman, Committee on Science

House of Representatives

Subject: Government Performance and Results Act: Information on Science

Issues in EPA’s Performance Report for Fiscal Year 1999 and

Performance Plans for Fiscal Years 2000 and 2001

Under the Government Performance and Results Act (GPRA) of 1993, the

Environmental Protection Agency (EPA) published a 5-year strategic plan in

1997. The plan established 10 strategic goals to serve as a framework for EPA’s

planning and resource allocation decisions. Since fiscal year 1999, EPA has also

issued annual performance plans that set annual performance goals to provide a

direct link between the agency’s strategic goals and its day-to-day activities. EPA

issued its performance plan for fiscal year 2001 in February 2000. GPRA further

requires annual performance reports on the degree to which annual goals were

met. In March 2000, EPA issued its first performance report, which described the

progress the agency made in achieving its fiscal year 1999 goals. EPA’s revised

strategic plan for fiscal years 2000-2005 is due September 29, 2000, and the

agency has made a June 2000 draft for the revised plan available to the Congress

and other interested parties for comment and consultation.

To facilitate your review and oversight of EPA’s science programs, you asked us

to review selected science programs and provide you with information on (1)

whether EPA links science issues in its performance report for fiscal year 1999

with performance plans for fiscal years 2000 and 2001 and whether the agency’s

funding request for fiscal year 2001 is linked to specific science activities in its

performance plan for that year, (2) whether EPA’s performance plan for fiscal

year 2001 addresses weaknesses in the agency’s science program that we

previously identified, and (3) the extent to which EPA’s draft strategic plan for

fiscal years 2000-2005 complies with GPRA’s requirements. To address our

objectives, we focused on EPA’s sound science, climate change, and waste

management programs that have major science components.
B-286079

2 GAO/RCED-00-270R EPA Science Issues

Science Activities Are Generally Linked in EPA’s

Performance Report, Plans, and Budget Request

EPA generally links the science activities in its performance report for fiscal year

1999, performance plans for fiscal years 2000 and 2001, and budget request for

fiscal year 2001. In each of these documents, EPA explains the purpose of its

research and how it expects the results to contribute to its strategic objectives.

Where appropriate, EPA also links its science goals across fiscal years. For

example, in fiscal year 1999, EPA finished the first stage of an ecological

monitoring system for the Mid-Atlantic Region that it planned to eventually

expand to other regions. EPA’s performance plan for fiscal year 2000 contains a

goal to report on the monitoring findings in the Mid-Atlantic Region and the costeffectiveness of the monitoring system. EPA’s follow-on goal shown in the

performance plan for fiscal year 2001 is to expand the scope of this research to

establish baseline data for estuaries nationwide. According to EPA, these data

will allow the agency to evaluate the effects of environmental management

policies on a national and regional basis. (Enc. I lists EPA’s performance goals

for science for fiscal years 1999 through 2001.)

EPA’s performance plan for fiscal year 2001 is generally linked to its science

activities in its budget justification for that year. As result of EPA’s decision to

combine the performance plan with its budget justification, the agency now

shows how much it is requesting for each strategic science objective and the

objective’s supporting performance goals. For each objective, EPA has included

tables that detail some of the requested amounts by key programs. For example,

under the objective “Increase Use of Integrated, Holistic, Partnership

Approaches” in the budget justification for fiscal year 2001, EPA indicates that

the $17.1 million requested for the objective is divided between two key

programs: the “Innovative Community Partnership Program” and the “Regional

Geographic Program.” However, EPA’s budget request does not always provide

the total amounts requested for all activities under each of its strategic

objectives. For example, EPA requested $115 million for its “Research for

Ecosystem Assessment and Restoration” strategic objective. However, the

budget request identifies specific activities for only about $52 million, or 45

percent, of the $115 million requested.1

EPA Partially Addresses Weaknesses

That We Previously Identified

We have identified several weaknesses in EPA’s science programs over the years,

including (1) the uneven implementation of peer review procedures for EPA’s

scientific and technical products, (2) gaps in scientific data, and (3) the lack of

performance goals and measures that show the environmental results of EPA’s

1

See Environmental Protection Agency: Comparing Annual Budgets for Science and Technology

(GAO/T-RCED-00-132, Mar. 23, 2000).
B-286079

GAO/RCED-00-270R EPA Science Issues 3

science activities. EPA partially addresses these weaknesses in its performance

plan for fiscal year 2001.

In 1996, we reported on the uneven implementation of EPA’s policy for peer

reviews—the critical evaluations of scientific and technical work products by

independent experts. In addition, we reported on the need for EPA to educate all

staff about the merits of, and procedures for, conducting peer reviews.2

Although EPA’s performance plan does not include goals for improving peer

reviews, the plan does discuss the actions that were taken in response to our

findings. For example, EPA issued a Peer Review Handbook, which provides

guidance on implementing peer reviews agencywide. Nonetheless, the National

Research Council of the American Academy of Sciences and EPA’s Science

Advisory Board believe that further improvements are needed to expand the

scope of peer reviews and make them more independent.3

For example, these

groups have said that project managers for work products should no longer be

allowed to lead peer reviews of these products. However, EPA’s performance

plan for fiscal year 2001 does not address this or other peer review issues.

In September 1999, we reported that EPA lacks fundamental environmental and

scientific data about pollutants and their effects on human health and the

environment. For example, EPA lacks toxicity data for more than one-third of

the chemicals produced in large volumes.4EPA established the Office of

Environmental Information in October 1999 to, among other things, fill significant

gaps in the agency’s data. Although EPA has not established performance goals

or measures for environmental or scientific data improvements, the agency’s

draft strategic plan for fiscal years 2000-2005 includes a strategic goal for its

information activities called “Quality Environmental Information.” According to

EPA, beginning with fiscal year 2002, the agency’s performance plan will contain

specific outcome-oriented goals and measures for its initiatives to improve the

quality of its information.

In an April 2000 review of EPA’s performance plan for fiscal year 2000, we found

that no performance goals, and only one performance measure showing changes

in environmental conditions, had been established for EPA’s research and

development activities.5 Under this measure, a 30-percent cumulative reduction

in a variety of emissions, including toxic chemicals and volatile organic

compounds, was to result from the introduction of various environmental

strategies. Officials of EPA’s Office of Research and Development told us that its

2

See Peer Review: EPA’s Implementation Remains Uneven (GAO/RCED-96-236, Sept. 24, 1996). 3

See Strengthening Science at the U.S. Environmental Protection Agency: Research Management

and Peer Review Practices, National Research Council (draft report, 2000) and An SAB Report:

Review of the Peer Review Program of the Environmental Protection Agency (EPA-SABRSAC-00-002, Nov. 1999).

4

See Environmental Information: EPA Is Taking Steps to Improve Information Management, but

Challenges Remain (GAO/RCED-99-261, Sept. 17, 1999). 5

See Managing for Results: EPA Faces Challenges in Developing Results-Oriented Performance

Goals and Measures (GAO/RCED-00-77, Apr. 28, 2000).
B-286079

4 GAO/RCED-00-270R EPA Science Issues

activities provide scientific and engineering information, models, methods, and

tools. The officials further stated that specific changes in pollutant levels

generally cannot be tied to its activities in a quantifiable manner. These officials

also noted that the models, methods, and tools are inputs to the development of

environmental regulations and policies, and frequently advance the state

environmental science, thereby better enabling EPA to achieve its environmental

outcomes. While EPA officials state that it is inherently difficult to link research

activities to specific environmental outcomes, other measures can be used to

measure the results of these activities. For example, EPA’s Science Advisory

Board has recommended that goals for science activities should concentrate on

outcomes—in terms of improved understanding of the environment—rather than

outputs, such as data collected and reports issued.6

EPA’s Draft Strategic Plan Generally Complies With

the Results Act, but Several Areas Could

Be Improved

EPA’s June 2000 draft strategic plan for fiscal years 2000-2005 generally complies

with the GPRA requirements that such plans include a mission statement, longterm strategic goals, strategies for achieving the strategic goals, linkage between

long-term strategic goals and annual performance goals, and a description of the

program evaluations used in establishing or revising performance goals. In

addition, EPA’s draft plan contains information on the functions that crosscut

those of other agencies, as suggested by the Office of Management and Budget’s

(OMB) guidance. However, EPA could improve several areas of its draft strategic

plan for fiscal years 2000-2005.

• In the 1997 strategic plan, EPA elaborated on its mission statement with a

series of purpose statements. The statements were provided to clarify the

scope of EPA’s mission. By including such purpose statements in the draft

strategic plan, EPA would focus the mission statement more directly on the

agency’s core activities.

• EPA’s discussion of human capital requirements for meeting its strategic

objectives could be more comprehensive. Under its “Effective Management”

strategic goal, EPA states that managing human capital will be a key priority.

EPA notes that it will work hard to “secure, develop, empower, and retain

talented people” through such efforts as workforce planning and training.

The only other substantive discussion of human capital requirements is under

EPA’s strategic goal for “Quality Environmental Information.” EPA states that

staff resources for environmental information activities will be a critical

factor affecting its success under this goal. However, EPA does not discuss

any means or strategies for addressing this goal.

6

See An SAB Report: Review of the FY 2001 Presidential Science and Technology Budget Request

for the Environmental Protection Agency (EPA-SAB-RSAC-00-007, Mar. 17, 2000).
B-286079

GAO/RCED-00-270R EPA Science Issues 5

• Program evaluations are an essential element of performance management.

EPA’s draft plan identifies previous or ongoing evaluations for each of its

strategic goals. However, EPA’s discussion of future evaluations was limited

to its “Clean and Safe Water” strategic goal, for which it listed three potential

program evaluations. Providing additional details on future evaluations of

other strategic goals would bring the discussion more in line with GPRA’s

requirements.

• EPA’s draft strategic plan could be made more informative if it provided

information on how EPA’s goals complement or supplement those of other

agencies, such as the Departments of Health and Human Services,

Agriculture, and the Interior. EPA’s 1997 strategic plan included a matrix that

gave a “bird's eye view” of the extent to which EPA’s goals were interrelated

with those of other agencies. By including such information in its draft plan,

EPA would provide perspective on its contribution to federal efforts to

improve health and environmental conditions.

(Enc. II contains additional observations on the draft strategic plan.)

Agency Comments and Our Evaluation

We provided EPA with a draft of this report for review and comment. EPA

officials, including the Director of the Planning Staff, Office of Planning, Analysis,

and Accountability, informed us that they generally agreed with the information

presented in the draft report. However, the officials wanted to emphasize two

revisions that EPA is making to its draft strategic plan for June 2000. First, they

told us that EPA’s final strategic plan will include an expanded discussion of the

agency’s human capital activities and will provide a matrix showing the extent to

which its strategic goals are interrelated with those of other agencies.

Second, concerning program evaluations, EPA officials said that the agency has

created the Office of Program Evaluation within EPA’s Office of the Inspector

General and the Evaluation Support Division to support and advocate

agencywide evaluation activities. They also said that the agency will revise

its draft strategic plan to identify information on additional ongoing program

evaluations. However, EPA did not clarify how it intends to meet GPRA’s and

OMB’s requirements that it identify future program evaluations for each strategic

goal.

EPA officials said that they will not include statements of purpose in the strategic

plan as we suggested in our draft report because the statements would duplicate

other information. While we believe that providing statements of purpose in a

prominent, central place would be useful to the reader, we recognize that such

statements, which are not required by GPRA or OMB, are at EPA’s discretion.
B-286079

6 GAO/RCED-00-270R EPA Science Issues

The EPA officials also provided technical clarifications, which were incorporated

into the report, as appropriate.

Scope and Methodology

To address our objectives, we focused on EPA’s sound science, climate change,

and waste management programs that have major science components. We

reviewed GPRA and OMB’s Circular A-11, which provide criteria for strategic

plans, annual performance plans, and performance reports. We reviewed EPA’s

performance report for fiscal year 1999, performance plans for fiscal years 2000

and 2001, and June 2000 draft strategic plan for 2000-2005. We also analyzed

EPA’s research strategy documents and reviewed our reports on EPA’s science

programs and performance planning activities. We conducted our review from

July through August 2000 in accordance with generally accepted government

auditing standards.

-----

As arranged with your office, unless you publicly announce its contents earlier,

we plan no further distribution of this report for 30 days from the date of this

report. At that time, we will send a copy to the Honorable Carol M. Browner,

Administrator, EPA, and will make copies available to others on request. If you

or your staff have any questions about this report, please call me at (202) 512-

6111. Key contributors to this report were Ed Kratzer, Ralph Running, Rosemary

Torres-Lerma, and Bernice Dawson.

Sincerely yours,

Peter F. Guerrero

Director, Environmental

Protection Issues

Enclosures - 2
Enclosure I

GAO/RCED-00-2

7

EPA’s Science Performance Goals for Fiscal Years 1999 Through 2001

Strategicgoal Performance plan for FY 1999 Performance report for FY 1999a Performance plan for FY 2000 Pe1 Identify and evaluate at least two

plausible biological mechanisms bywhich particulate matter causesdeath and disease in humans.

In fiscal year 1999, theEnvironmental Protection Agency(EPA) identified and evaluated twoplausible biological mechanisms bywhich particulate matter causesdeath and disease in humans.

Provide new information on theatmospheric concentrations, humanexposure, and health effects ofparticulate matter, includingparticulate matter (PM) 2.5, andincorporate it and other peerreviewed research findings in thesecond External Review Draft of thePM Air Quality Criteria Document forNational Ambient Air QualityStandards review.

ProatmhuanpaNaStaCrco

1 Complete health assessments for

five air toxics to be indicated ashigh priority by EPA and its regionaloffices.

In fiscal year 1999, the researchprogram completed healthassessments for four high-priority airtoxics—one short of the fiveassessments that were planned.

Provide methods to estimate humanexposure and health effects fromhigh-priority urban air toxics andcomplete health assessments for thehighest-priority hazardous pollutants,including fuel/fuel additives.

Promeexhigcohigpoad

1 Noneb Not applicable Develop tropospheric ozone

precursor measurement methods,emissions information to guide StateImplementation Plan’s (SIP)development under the currentnational ambient air quality standards(NAAQS) for ozone.

DepreemobmeinfdeNA

aEPA discussed only the goals that it identified as being the “vital few” for congressional reporting purposes for its performa

year 1999. However, to provide a complete picture, we have listed all of EPA’s performance goals for science in the table.

b“None” indicates that there is no related goal.
Enclosure I

GAO/RCED-00-2

8

Strategicgoal Performance plan for FY 1999 Performance report for FY 1999a Performance plan for FY 2000 Pe

1 Evaluate Models-3/Community

Multi-Scale Air Quality against fileddata to demonstrate reliability inNAAQS for ozone.

Not applicable Provide new information on the

atmospheric concentrations of,human exposure to, and health andenvironmental effects of troposhericozone and incorporate it and otherpeer-reviewed research findings inand External Review Draft of theOzone Air Quality Criteria Documentfor NAAQS review.

DeDonitthrco

1 Complete preliminary evaluations of

Models-3/Community Multi-ScaleAir Quality for PM to demonstratereliability NAAQS attainmentplanning for PM.

Not applicable Develop PM measurements,

methods, emissions-based air qualitymodels, and source emissions andcontrol information to guidedevelopment of SIPs under thecurrent PM NAAQS.

Propameemto the 

1 Develop a preliminary urban scale

Models-3/Toxics Model forcommunity-based human exposureassessment for air toxics havingknown emissions and air chemistry.

Not applicable Develop (1) an air quality model

incorporating air toxics, as their airchemistry and emissions becomeknown and (2) source emissions andcontrol information for both mobileand stationary sources to guide costeffective risk management options.

Deincchkncoaneff

2 EPA will develop critical doseresponse data for disinfectant byproducts, waterborne pathogens,

and arsenic for addressing keyuncertainties in the risk assessmentof municipal water supplies.

In fiscal year 1999, EPA met its goalof developing dose-responseinformation on disinfectantbyproducts, waterborne pathogens,and arsenic for characterizingpotential exposure risks fromconsuming drinking water.

Reduce uncertainties and improvemethods associated with theevaluation and control of risks posedby exposure to disinfection byproducts in drinking water.

No
Enclosure I

GAO/RCED-00-2

9

Strategicgoal Performance plan for FY 1999 Performance report for FY 1999a Performance plan for FY 2000 Pe

2 None Not applicable Reduce uncertainties and improve

methods associated with theevaluation and control of risks posedby exposure to microbialcontaminants in drinking water.

Remeaspocofocthe 

2 By 2003: Deliver support tools,

such as watershed models,enabling resource planners toselect consistent, appropriatewatershed management solutionsand alternative, less costly wetweather flow control techniques.

In fiscal year 1999, EPA continuedefforts to deliver support tools, suchas watershed models, which enableresource planners to selectconsistent and appropriatewatershed management solutionsand alternatives as well as costlywet-weather flow technologies.EPA is making progress toward thisgoal in 2003, which it expects toreach.

Develop modeling, monitoring, andrisk management methods thatenable planners and regulatoryofficials to more accuratelycharacterize receiving andrecreational water quality and toselect appropriate controltechnologies.

No

2 EPA will provide data and

information for use by states andregions in assessing and managingaquatic stressors in the watershedto reduce toxic loadings andimprove ecological riskassessment.

In fiscal year 1999, EPA provideddata and information for use bystates and its regional offices inassessing and managing aquaticstressors in watersheds to reducetoxic loadings and improveecological risk assessment.

Develop the scientific rationale fornumerical criteria for surface waters.Develop a conceptual framework forthe diagnosis and assessment ofwater quality impairment in U.S.watersheds.Identify the primary life supportfunctions of surface waters thatcontribute to the management andsustainability of watersheds.

Dediapo
Enclosure I

GAO/RCED-00-2

10

Strategicgoal Performance plan for FY 1999 Performance report for FY 1999a Performance plan for FY 2000 Pe

3 None Not applicable Provide methodologies to evaluate

the risk to human health posed byfood-use products.

DeeffmeancoFo

4 Improve in vitro screening methods

for one-electron mechanisms oftoxicity among industrial chemicals.

Not applicable Provide methods and models to

evaluate the impact of environmentalstressors on human health andecological end points for use inguidelines, assessments, andstrategies.

Deascochreden

5 Demonstrate and verify the

performance of 18 innovativetechnologies by 2001, emphasizingremediation and characterization ofgroundwater and soils.

In fiscal year 1999, work under theprogram proceeded according toschedule to meet the 2001 targets,as EPA completed demonstrationsof seven innovative technologiesthrough partnerships with the privatesector and other governmentagencies.

Enhance scientifically defensibledecisions for site cleanup byproviding targeted research andtechnical support.

Prosucoup washreden

5 Complete prototype model for

assessing cumulative exposure-riskassessments integrating theenvironmental impact of multiplechemicals through multiple mediaand pathways.

In fiscal year 1999, EPA completeda test version of a cumulativeexposure model that integrates theenvironmental impact of multiplechemicals through multiple mediaand pathways.

Enhance scientifically defensibledecisions for active management ofwastes, including combustion, byproviding targeted research andtechnical support.

ProsuRedeco
Enclosure I

GAO/RCED-00-2

11

Strategicgoal Performance plan for FY 1999 Performance report for FY 1999a Performance plan for FY 2000 Pe

6 Conduct preliminary assessment of

the consequences of climatechange at three geographicallocations: (Mid-Atlantic, Gulf Coast,and upper Great Lakes).

EPA completed the Mid-Atlantic andGreat Lakes regional assessmentsas well as the human healthsectarian assessments on schedulein fiscal year 1999, but the GulfCoast regional assessment isbehind schedule because ofdifficulties in obtaining a high-qualityproject proposal able to passrigorous scientific peer review.

Assess the consequences of globalchange and climate variability at aregional scale.

Aschanhe

7 By 1999, complete five to seven

monitoring pilot projects inenvironmental monitoring for publicaccess and community tracking(EMPACT) cities.

During fiscal year 1999, EPAawarded eight grants to localcommunities for innovativemonitoring research pilot projects,exceeding the agency’s goal of fiveto seven pilot projects for theprogram.

By fiscal year 2000, 75 percent ofEMPACT communities have in place,or have initiated, community-basedstrategies for time-relevantenvironmental monitoring,information management, andcommunication that will result insustained community capacity todeliver timely environmentalinformation.c

No

7 Publish guidelines for Carcinogenic

Risk Assessment, Neurotoxity RiskAssessment, and a technical reporton the health risk assessment ofchemical mixtures.

Not applicable Develop data interpretation tools and

risk communication tools to providethe public and environmentalcommunities with time-relevantinformation.

Proasbama

cThe goal for fiscal year 2000 is no longer considered a research and development activity and has been transferred to the 

Information.
Enclosure I

GAO/RCED-00-2

12

Strategicgoal Performance plan for FY 1999 Performance report for FY 1999a Performance plan for FY 2000 Pe

8 Complete and evaluate a

multitiered ecological monitoringsystem for the Mid-Atlantic regionand provide select land cover andaquatic indicators for measuringstatus and trends.

In fiscal year 1999, EPA completedthe first stage of the EnvironmentalMonitoring and AssessmentProgram (EMAP) effort in support ofthe agency’s fiscal year 2001commitment to complete andevaluate a multitiered ecologicalmonitoring system for the MidAtlantic region and provide selectland cover and aquatic indicators formeasuring status and trends.

Report on monitoring findings in theMid-Atlantic region as a costeffective means of measuring thecondition of these systems.Develop monitoring designs,including indicators, for streams inwestern watershed.Develop monitoring designs fornational coastal monitoring.

Eswhtrethe cowhmaev

8 None Not applicable Publish a conceptual model for

developing watershed assessmenttechniques that would assist local,regional, and national environmentaldecisionmakers in maintaining theecological integrity of the watershed.

No

8 Analyze existing monitoring data for

acid deposition and ultravioletradiation-B (UVB) and implement amultiple site UVB monitoringsystem for measuring status andtrends.

EPA performed several prototypeanalyses addressing wet and drydeposition trends of sulfur andnitrogen to meet the agency’s fiscalyear 1999 commitment to analyzeexisting monitoring data for aciddeposition.

None No

8 Provide ecological risk assessment

case studies for two watersheds,final guidelines for reportingecological risk assessment, andecological risk assessmentguidance and support.

In fiscal year 1999, EPA providedecological risk assessment casestudies for two watersheds, finalguidelines for reporting ecologicalrisk assessment, and ecological riskassessment guidance and support.

None De

coasas 
Enclosure I

GAO/RCED-00-2

13

Strategicgoal Performance plan for FY 1999 Performance report for FY 1999a Performance plan for FY 2000 Pe

8 Develop and verify innovative

methods and models for assessingthe susceptibilities of populations toenvironmental agents aimed atenhancing risk assessment andmanagement strategies andguidelines.

EPA also made significant progressin fiscal year 1999 toward meetingits long-term goal of developing andverifying innovative methods andmodels for assessing thesusceptibilities of populations toenvironmental agents.

None No

8 Complete and submit an external

review draft of the Air QualityCriteria Document for carbonmonoxide.

Not applicable None No

8 Initiate field exposure study of

children to two endocrine-disruptingchemicals.

EPA completed a protocol for a fieldstudy of children exposed to twoendocrine-disrupting chemicals.

Develop tools to identify hazards andformulate strategies to manage risksfrom exposure to endocrinedisrupting chemicals capable ofinducing adverse effects in humansand wildlife.

No

8 Improve computational efficiency of

the fine particulate matter model by25 percent.

EPA finished work on the aircomponent of the MultimediaIntegrated Modeling System andmet its fiscal year 1999 goal ofimproving computational efficiencyin the analysis of particulate matterby 25 percent.

None No
Enclosure I

GAO/RCED-00-2

14

Strategicgoal Performance plan for FY 1999 Performance report for FY 1999a Performance plan for FY 2000 Pe

8 None Not applicable Develop risk assessment guidance

and regional assessmentsconcerning risks to children exposedto environmental contaminants.

ImRisfocextagsuhanoevhe

8 Produce first-generation exposure

models describing residentialexposure to pesticides.Produce a first-generationchlorpyrifos/pesticide exposuremodel and a technical report onchildren’s activities associated withpotentially high exposures.

In fiscal year 1999, EPA produced afirst-generation model that can beused in a prospective context toprovide reliable assessments of thepotential risks to human populationsposed by exposure to pesticidesand other toxic chemicals.

Develop initial measurements,methods, and models to evaluateexposures and effects ofenvironmental contaminants,particularly in children.

Demeexenpa

8 None Not applicable Initiate a research program to

address the most pressing issuesrelated to the prevention, control, andelimination of mercury as a humanhealth and environmental problem.

Proabrefanen
Enclosure I

GAO/RCED-00-2

15

Strategicgoal Performance plan for FY 1999 Performance report for FY 1999d Performance plan for FY 2000 Pe

8 None Not applicable Provide decision-support tools and

methods that can be applied todetermine the value and costs ofsolutions to environmental problems,and develop partnerships to assist inthe application of these tools andmethods to community-basedenvironmental programs, includingtribal partners.Complete the development of one ormore computer-based tools thatsimulate product, process, or systemdesign changes, and complete proofof-process structure for one or moregeneric technologies (apply to morethan one environmental problem) toprevent or reduce pollution inchemicals and industrial processes.

Preprefor ordapadprede

8 Use of a pilot program to verify

environmental technologies.

Not applicable Use of a pilot program to verify

environmental technologies.

DetecelimrisEmpreanmesta

dEPA discussed only the goals that it identified as being the “vital few” for congressional reporting purposes for its performa

year 1999. However, to provide a complete picture, we have listed all of EPA’s performance goals for science in the table.
Enclosure II

16 GAO/RCED-00-270R EPA Science Issues

GAO’s Observations on EPA’s Draft 2000 Strategic Plan

The Government Performance and Results Act (GPRA) required each federal agency to develop,

by the end of fiscal year 1997, a strategic plan covering at least a 5-year period. The act

stipulated that each plan should state the agency’s mission, identify long-term strategic goals, and

describe how the agency intends to achieve these goals through its activities and human capital,

information, and other resources. Agencies are required to update their plans as least every 3

years. The Environmental Protection Agency (EPA) issued its strategic plan in September 1997

and plans to issue its revised plan by September 29, 2000. The agency has made the revised plan

available to the Congress and other interested parties for comment and consultation.

EPA’s 2000 draft strategic plan complies with GPRA’s requirements and/or guidance provided by

the Office of Management and Budget (OMB) for (1) a mission statement, (2) long-term strategic

goals, (3) strategies for achieving the strategic goals, (4) a linkage between the agency’s longterm strategic goals and annual performance plans, (5) the identification of those key external

factors that could significantly affect EPA’s ability to achieve its strategic goals, (6) a description

of program evaluations used in establishing or revising the goals, and (7) references to the roles

and responsibilities of other agencies.

Mission Statement

GPRA mandates a “comprehensive mission statement covering the major functions and

operations of the agency.” OMB’s guidance states that a mission statement should be brief,

defining the agency’s basic purpose, with particular emphasis on its core programs and activities.

EPA’s draft strategic plan states that the agency’s mission is “to protect human health, and to

safeguard the natural environment—air, water, and land—upon which life depends.” The draft

plan’s overarching statement is general but states concisely what EPA is broadly charged with

doing under its statutes. In its 1997 Strategic Plan, EPA elaborated on its mission statement with

a series of purpose statements that provided a clear picture of the scope of the agency’s mission.

Such statements are not included in the draft strategic plan. The mission statement could be

improved by including such statements, which would focus the mission statement more directly

on the agency’s core activities.

Goals and Objectives

Under GPRA, an agency’s strategic plan must describe the general goals and objectives, including

outcome-related goals and objectives, for the major functions and operations of the agency.

These goals should explain what results are expected from the agency’s major functions and

when to expect those results.

EPA’s plan identifies 10 goals, which have multiple objectives. The goals are generally outcomeoriented and measurable and are developed along the same lines as EPA’s statutory requirements

and organizational structure. Likewise, EPA’s strategic goals are linked to objectives that, for the

most part, are outcome-oriented. Such objectives should encourage EPA to focus more attention

on performance goals and measures based on outcomes (results), rather than outputs (activities

and processes).
Enclosure II

17 GAO/RCED-00-270R EPA Science Issues

However, as we stated in our September 1999 report on EPA’s information management,1getting

the data needed to measure progress toward the objectives will remain a challenge for EPA.

Under goal 7 (Quality Environmental Information), EPA discusses the importance of enhancing

the quality of its data by developing integrated data systems. EPA also states that it will set

annual performance goals and measures to gauge its progress toward this goal. This would

demonstrate the agency’s commitment to hold itself accountable for obtaining the data needed to

manage for results.

Achieving Goals and Objectives

Under GPRA, an agency’s strategic plan must address how the goals and objectives are to be

achieved, including a description of the operational processes; skills and technology; and human

capital, information, and other resources required to meet those goals and objectives.

For each goal, the plan has sections on the importance of the goal, the objectives of the goal, the

results they intend to achieve over the next several years, and the means and strategies to

achieve these goals. However, EPA’s discussion of human capital requirements for meeting its

strategic goals and objectives could be more comprehensive. Under goal 10 (Effective

Management), EPA states that managing human capital will be a key priority. The agency notes

that it will work hard to “secure, develop, empower, and retain talented people” through efforts

such as workforce planning and training. The only other substantive discussion of human capital

requirements is under goal 7 (Quality Environmental Information). In the draft strategic plan,

EPA states that staff resources for environmental information will be a critical factor affecting its

success under this goal. EPA also notes that it is important to address the challenge of recruiting

and retaining skilled information and technology personnel. However, EPA does not discuss any

means or strategies for addressing this challenge.

Relationships Between Long-Term Strategic and Annual Performance Goals

GPRA requires that an agency’s strategic plan contain a description of how the performance

goals included in its annual performance plans relate to the general goals and objectives of the

strategic plan. OMB directs that the strategic plan should briefly outline (1) the type, nature, and

scope of the performance goals to be included in a performance plan; (2) the relationship

between the performance goals and the general goals and objectives; and (3) the relevance and

use of performance goals in helping determine the achievement of general goals and objectives.

Each goal in EPA’s draft strategic plan has a section entitled “Relating Annual Performance

Goals to Strategic Objectives.” The information in these sections generally conforms to the

GPRA requirements and OMB Circular A-11. For example, under goal 5 (Better Waste

Management and Restoration of Contaminated Waste Sites, and Emergency Response), EPA

discusses the general type, nature, and scope of the performance goals and their relationship to

the strategic goal and objectives. The plan also gives examples of performance goals to be used

1

Environmental Information: EPA Is Taking Steps to Improve Information Management, but Challenges Remain

(GAO/RCED-99-261, Sept. 17, 1999).
Enclosure II

18 GAO/RCED-00-270R EPA Science Issues

to measure progress, such as the number of contaminated sites cleaned up and the number of

hazardous waste sites with controls in place to prevent dangerous releases.

Key External Factors

GPRA requires that a strategic plan identify factors external to the agency and beyond the

agency’s control that could significantly affect the achievement of goals and objectives. OMB’s

guidance directs the agencies to identify each key external factor, indicate how it is linked with a

particular goal, and describe how each factor may affect the accomplishment of the goal.

With the exception of goal 10 (Effective Management), EPA’s draft strategic plan identifies

external factors that could affect the achievement of goals and measures. For example, under

goal 9 (A Credible Deterrent to Pollution and Greater Compliance With the Law), EPA states that

its projected performance would be affected by natural catastrophes, such as major floods or

significant oil spills. Such disasters would require a redirection of resources to address

immediate environmental threats.

Program Evaluation

GPRA requires that strategic plans describe the program evaluations used in establishing or

revising goals and objectives with a schedule of future program evaluations. The act defines

program evaluations as assessments, through objective measurement and systematic analysis, of

the manner and extent to which federal programs achieve intended objectives. According to

OMB’s guidance, the schedule of future program evaluations should outline the general

methodology to be used.

EPA’s draft strategic plan identified some means of evaluating for each of the agency’s 10 goals.

The discussion included GAO reviews, peer review, cost-benefit analyses, and internal

evaluations of regional, state, and tribal performance. However, the discussion focused on past

or ongoing evaluations. EPA’s discussion of future evaluations was limited to goal 2 (Clean and

Safe Water), for which, EPA listed three potential future program evaluations. Providing

information on future program evaluations would bring the discussion more in line with GPRA’s

requirements.

Crosscutting Functions

Consistent with OMB’s guidance, agencies’ strategic plans should reflect program goals that

crosscut those of other agencies. Likewise, the agencies should coordinate among themselves to

ensure that information is shared and duplication is eliminated.

EPA’s draft strategic plan, in discussing strategies and external factors, includes references to

the roles and responsibilities of other agencies. For example, under goal 6 (Reduction of Global

and Cross-Border Environmental Risks), EPA states that it is working with the Department of

State, the National Oceanic and Atmospheric Administration, and other federal agencies to

negotiate and implement legally binding, multilateral agreements that address significant sources

of marine pollution. The plan also devotes a chapter to six cross-agency programs that promote

partnerships with states and tribes and enlist the efforts of others outside EPA. For example, the
Enclosure II

19 GAO/RCED-00-270R EPA Science Issues

plan discusses its state partnerships with the National Environmental Performance Partnership

System and the Environmental Council of States.

While EPA addresses crosscutting efforts, the discussion does not provide a complete picture of

how its goals complement or supplement those of other agencies that share responsibilities with

EPA. For example, the Department of Health and Human Services has responsibilities for

protecting human health, and the Departments of Agriculture and the Interior have

responsibilities for protecting the environment. In its 1997 Strategic Plan, EPA included a matrix

showing how the programs of these and other agencies intersected with EPA’s programs.

Although it did not provide detailed information, the matrix gave a bird's eye view of the extent

to which other agencies had goals related to EPA’s. Providing such information in its revised

plan would provide a perspective on EPA’s contribution to federal efforts to improve health and

environmental conditions.

(160544)
Orders by Internet

For information on how to access GAO reports on the Internet, send an e-mail message

with “info” in the body to

info@www.gao.gov

or visit GAO’s World Wide Web home page at

http://www.gao.gov

To Report Fraud, Waste, and Abuse

in Federal Programs

Web site: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: 1-800-424-5454