Office of Inspector General 

Special Review 

EPA’s Progress in Using the

Government Performance and Results Act 

to Manage for Results

2001-B-000001 

June 13, 2001 
Table of Contents Table of Contents

Executive Summary ............................... 1

1. Goals - Are EPA’s goals consistent with its statutory 

authority and key partner and stakeholder expectations? ......... 3

2. Priorities - Does EPA have a system for setting priorities 

to best achieve its environmental mission? .................. 6

3. Strategies - Has EPA developed results-oriented strategies 

that clearly link annual plans and budgets to accomplishments? ....... 9

4. Measurement - Does EPA have systems to measure, 

evaluate, and report on environmental results? ................ 14

5. People - Does EPA have the human resource systems 

needed to manage for results? ........................ 17

6. Accountability - Has EPA established an accountability 

system to ensure efficient achievement of its goals? ............. 20

Exhibit 1: Scope and Methodology ....................... 25
Executive Summary Executive Summary

The Government Performance and Results Act of 1993 (GPRA) seeks to shift the focus of 

government performance and accountability away from simply the activities being performed to 

the results and outcomes of those activities. The purpose of GPRA is to systematically hold 

federal agencies accountable for achieving program results. To address GPRA requirements, the 

U.S. Environmental Protection Agency (EPA) has taken a number of steps to implement a 

practical GPRA framework to link EPA’s activities to long-term results. EPA is considered a 

leader in implementing GPRA because of the innovative approach it has taken to align its 

planning, budgeting, analysis and accountability processes. 

To determine what progress EPA has made in using GPRA to manage for environmental results, 

we reviewed EPA’s GPRA implementation activities, focusing on choices made, progress 

achieved, challenges faced, and opportunities for improvement. We used the following six 

questions to assess EPA’s progress in implementing GPRA. We selected these six questions 

because we believe they address areas that are critical in establishing an effective framework for 

maximizing environmental results. 

1. GOALS - Are EPA’s goals consistent with its statutory authority and key partner and 

stakeholder expectations? 

EPA’s goals are consistent with traditional interpretations of its authorizing statutes and 

GPRA, and accurately reflect the Agency’s role in setting and implementing environmental 

standards. However, EPA’s goals have fallen short of key partner and stakeholder 

expectations. 

2. PRIORITIES - Does EPA have a system for setting priorities to best achieve its 

environmental mission? 

While EPA sets multi-year, Agency level planning and budgeting priorities, there is no 

readily visible system or process used to identify them. There is insufficient stakeholder 

input and formal examination of options, costs, and benefits. 

3. STRATEGIES - Has EPA developed results-oriented strategies that clearly link 

annual plans and budgets to accomplishments? 

EPA has taken many actions to comply with GPRA’s strategic planning requirements, 

although it faces significant challenges in developing results-oriented strategies that clearly 

link annual plans and budgets to accomplishment of long term outcomes. EPA needs to 

better describe how, when, and at what cost agency goals are to be achieved; the 

behavioral changes needed to accomplish these goals; and the links between current 

activities and long term outcome goals. 

4. MEASUREMENT - Does EPA have systems to measure, evaluate, and report on 

environmental results? 
EPA has taken steps to put systems in place to measure and report on progress under 

GPRA, but continues to experience difficulty developing and obtaining the information 

necessary for measuring, evaluating, and reporting on long-term environmental outcomes. 

Most of EPA’s environmental quality information is collected by states, and the 

inconsistency and incompatibility of this data makes measurement difficult. 

5. PEOPLE - Does EPA have the human resource systems needed to manage for results? 

EPA recognized the importance of human capital as a key Agency priority in its Strategic 

Plan. However, EPA did not specifically address human capital in connection with any of 

its goals, and needs to better integrate human capital into its overall strategic plan, develop 

competencies in a wide range of skills, and better align individual accountability. 

6. ACCOUNTABILITY - Has EPA established an accountability system to ensure 

efficient achievement of its goals? 

EPA has established an accountability system with the intent of ensuring the efficient 

achievement of its goals, and has been recognized as a leader in developing a goals-based 

budget aligned with its programmatic and operational outputs and outcomes. However, 

this framework tends to perpetuate EPA’s “stove piped” organization, and does not 

address overlapping environmental issues and the different needs and priorities of EPA’s 

various partners (states, local, tribal agencies). 

In general, to improve GPRA implementation, EPA needs to strengthen its partnerships with 

states and other federal agencies. Also, EPA needs to place greater focus on the ultimate results 

and outcomes of its activities rather than actions performed. For example, EPA needs to more 

carefully consider science and cost, in addition to laws and public perceptions, when setting 

Agency priorities. Furthermore, EPA needs to invest in developing performance information that 

is more outcome oriented. 

2
1. Goals 1. Goals Are EPA’s goals consistent with its statutory 

authority and key partner and stakeholder 

expectations? 

EPA’s 10 goals are consistent with traditional interpretations of its authorizing statutes and

GPRA, and accurately reflect the Agency’s role in setting and implementing environmental

standards. However, EPA’s goals have fallen short of key partner and stakeholder expectations in

several ways. Stakeholders have stated that EPA’s mediabased goals restrict cross-media implementation, often the

preferred strategy for addressing local environmental

problems. In addition, stakeholders expressed concern that

EPA’s non-environmental goals do not appear to be

consistent with GPRA’s outcome orientation. Finally, some

of the original stakeholder expectations for EPA goals went

beyond EPA’s authority under its environmental statutes and

GPRA.

Development of EPA Goals 

EPA began developing national environmental goals in 1992, 

a year before enactment of GPRA. The objective of this 

effort, known as the National Environmental Goals Project, 

was to develop national, cross-media goals that reflected the 

priorities of the American public. This emphasis on public 

priorities shaped the process for selecting goal topics. In 

1994, EPA initiated a series of public meetings to obtain 

public views. Public comment reflected various themes, 

including the following: 

EPA’s 10 Goals 

1. Clean Air 

2. Clean and Safe Water 

3. Safe Food 

4. Pollution Prevention and Reducing 

Risk in Communities, Homes, 

Workplaces, and Ecosystems 

5. Better Waste Management, 

Restoration of Contaminated 

Waste Sites, and Emergency 

Response 

6. Reduction of Global and Cross 

Border Environmental Risks 

7. Expansion of Americans’ Right to 

Know About their Environment (a) 

8. Sound Science, Improved 

Understanding of Environmental 

Risk, and Greater Innovation to 

Address Environmental Problems 

9. A Credible Deterrent to Pollution 

and Greater Compliance With the 

Law 

10. Effective Management 

(a) In EPA’s Revised Strategic Plan Goal 7 is titled 

“Quality Environmental Information.” 

• Goals should cut across administrative boundaries of 

EPA offices and other government departments. 

• Goals set at the national level need to be flexible enough to accommodate regional differences 

and local priorities. 

EPA used public input from the Goals Project to draft its goals, and initially developed 

13 environmental goals. By condensing and combining some of the goals, EPA subsequently 

reduced that number to seven environmental goals (Goals 1-7), and added three 

non-environmental goals (Goals 8-10). These 10 goals formed the basis of EPA’s 1997 Strategic 

Plan. 

3
EPA Emphasis on Traditional Role During Goal Development 

EPA used three major criteria in 1992 to begin its goal development. First, the goals should be 

environmental, so as to be meaningful and understandable by the general public. Second, the 

goals should be attributable to the laws, programs, and resources that EPA administers, but broad 

enough to encourage substantial roles by other parties. Third, the goals should be measurable, 

using available data and systems, although data quality and availability should not entirely drive 

the choice of goals. These three criteria led to goals that: 

• Paralleled EPA’s organizational structure in many cases, reflecting a combination of 

environmental media (air and water) and programs (waste management). 

• Emphasized progress toward compliance with health- and ecological-based environmental 

standards established in EPA regulations (for example, attainment of air quality standards). 

• Focused mainly on EPA’s role and statutory responsibilities rather than on a broader scope of 

federal environmental responsibilities. 

Mindful that GPRA directed agencies to develop strategic goals that reflect the agencies’ 

statutory authorities, EPA developed goals that accurately reflect its role in setting federal, healthand ecological-based environmental standards and implementing and enforcing major federal 

environmental laws. However, these goals have fallen short of key partner and stakeholder 

expectations in several ways, as follows. 

Cross-Media Implementation 

In 1996, EPA chose a 10-goal structure composed of 7 goals derived from the National 

Environmental Goals Project and 3 additional non-environmental goals. This goal structure 

reflected existing Agency programs and structure and required less change to linkages among 

existing planning, budget, and accountability systems. In doing so, the Agency chose not to adopt 

a broader, more multi-media oriented goal structure proposed later in the process by EPA Region 

5. According to EPA decision documents, this approach would have better supported placebased and multi-media initiatives. EPA placed this more multi-media orientation in the Agency’s 

mission statement rather than in the specific goals and objectives used to track EPA’s efforts. 

EPA regions and the states have commented that EPA’s 10-goal structure can restrict crossmedia implementation, often the preferred strategy for addressing local environmental problems. 

Non-Environmental Goals 

EPA expanded its goal structure by adding three non-environmental goals (Sound Science, 

Enforcement, and Effective Management) to provide an explicit, comprehensive system of budget 

accountability under GPRA for each of EPA’s major program offices. EPA stakeholders, 

however, have expressed concern that the non-environmental goals were process rather than 

outcome oriented. In particular, states objected to EPA’s enforcement goal as inconsistent with 

4
GPRA’s outcome orientation. States have consistently commented that enforcement is a key tool 

supporting federal and state environmental efforts in achieving environmental goals, but should 

not in itself be considered an environmental goal. 

Expectations Beyond EPA’s Authority 

To accomplish its goals and objectives, EPA works in partnership with many other federal 

agencies. This includes agencies on whom EPA relies as co-implementors of federal 

environmental programs (such as the Army Corp of Engineers) and as members of the regulated 

community (such as the Departments of Energy and Defense). EPA also works with federal 

agencies involved with public health and ecological protection (such as the Public Health Service 

and Department of the Interior). 

EPA’s goals are generally consistent with GPRA’s requirements limiting agency goal 

development to areas of direct statutory responsibility. However, as stakeholders noted in the 

National Environmental Goals Project, the ultimate aims of environmental laws -- i.e., healthy 

people and ecosystems -- and the federal partnerships necessary to achieve those larger goals, are 

beyond EPA’s scope of authority or responsibility under GPRA. If EPA were to attempt to meet 

these larger stakeholder expectations in the future, it would be necessary for EPA to seek the 

support of the Administration and Congress for developing broader goals in partnership with 

other federal agencies. 

Recommendation 

EPA has laid a solid foundation for future improvements with its goal development process. 

There is a substantial record of stakeholder and partner opinion and preference. Stakeholders are 

in substantial agreement over the importance of EPA goals for setting Agency direction and 

strengthening federal and state partnerships. In order for the Agency to assume a greater 

leadership role through the future development of its GPRA goals, we recommend that EPA: 

1-1. Move expeditiously to amend EPA’s goals and the processes for implementing them, as 

necessary, to focus on environmental outcomes and encourage cross-media implementation. 

EPA should also seek the support of the Office of Management and Budget and Congress 

for a federal government process to develop environmental goals that promote the ultimate 

human health and ecological aims of environmental laws, and the federal partnerships 

necessary to achieve them. 

5
2. Priorities 2. Priorities Does EPA have a system for setting 

priorities to best achieve its environmental 

mission? 

While EPA sets multi-year, Agency level planning and budgeting priorities, there is no readily 

visible system or process used to identify them. Also, while EPA does consult informally with 

stakeholders on planning and budgeting priorities, there is not a formal opportunity for comment 

until submission of the budget to Congress.1 The priorities do not undergo any formal 

examination of options, costs, and benefits. As a consequence, EPA’s multi-year priorities have 

been shaped more by law and public perception than by science and cost. 

System for Setting Priorities 

Because resources are limited, EPA must set priorities among competing environmental problems, 

programs, and issues. EPA’s media program offices set and implement priorities among the many 

objectives in EPA’s strategic plan by setting 5-year performance goals and allocating funds to 

them annually. To set priorities, program offices consider information on legal authority, cost, 

environmental risk, and public perceptions. However, we found that program offices do not 

document or explain how these factors are used in decision making. 

In discussions with Agency staff, we 

found that the specific statutory 

requirements of EPA’s laws, as well as 

public perceptions, have shaped EPA 

priorities more than science and cost 

information. EPA’s senior management 

has not established risk based decisionmaking criteria to set priorities for the 

Agency. Although environmental risk 

assessment is used by EPA to set specific 

health-based standards, it is not usually 

used for setting broad Agency and 

programmatic priorities. 

Historically, EPA’s priorities have been shaped by 

a variety of factors: 

Environmental laws Environmental laws

Public perceptions of 

environmental problems environmental problems

Science/risk information Science/risk information

Economic costs Economic costs

EPA Priorities EPA Priorities

Internal EPA Internal EPA

Priority Setting Priority Setting

Also, while EPA estimates the compliance costs of specific rules to assess cost-effectiveness, 

there is no formal process for considering cost or cost effectiveness in Agency priority-setting 

decisions. Thus, there is little assurance that EPA is allocating its limited resources to those 

problems which pose the greatest environmental risks and opportunities for risk reduction. 

1

 OMB Circular A-11 Section 36, Communications with Congress and the Public and Clearance 

Requirements, states, in part, that the nature and amounts of the President's decisions and the underlying materials 

are confidential and are not to be released outside the Agency until the budget is transmitted to Congress. 

6 
Consistency of Priorities 

Each year, EPA identifies the Administrator’s planning and budgeting priorities in budget 

formulation guidance to EPA program offices. We identified 22 distinct Administrator priorities 

in a review of Congressional budget summaries for fiscal years 1996 through 2000, covering 

EPA’s experience prior to and in the first year of GPRA implementation. Of these 22 priorities, 

only 2 appeared in all 5 years, and 11 appeared only once or twice: 

Community-Based Environmental Protection 2

Use Advanced Technologies in Environmental Management 2

Toxic Waste Sites 3

Clean Water 3

Sound Science 3

Protect Human Health by Reducing Environmental Threats 1

Achieving Savings Through Streamlining efforts 1

Establish Working Capital Fund 1

Implement Performance Partnership Agreements 1

Balanced Compliance and Enforcement 1

Implement Resource Conservation and Recovery Act Reinvention 1

Eliminating Barriers to Regulatory and Permitting Process 1

Toxic Pollutants 2

Livable Communities 2

Safe Drinking Water 3

Reinvent Environmental Regulation 3

Protect Children’s Health 3

Achieve Progress on Clean Air Programs 3

Brownfields 3

Improve Environmental Information 4

Strengthen State and Tribal Partnerships 5

Global Climate Change 5

Community-Based Environmental Protection 2

Use Advanced Technologies in Environmental Management 2

Toxic Waste Sites 3

Clean Water 3

Sound Science 3

Protect Human Health by Reducing Environmental Threats 1

Achieving Savings Through Streamlining efforts 1

Establish Working Capital Fund 1

1996

Implement Performance Partnership Agreements 1

Balanced Compliance and Enforcement 1

Implement Resource Conservation and Recovery Act Reinvention 1

Eliminating Barriers to Regulatory and Permitting Process 1

Toxic Pollutants 2

Livable Communities 2

Safe Drinking Water 3

Reinvent Environmental Regulation 3

Protect Children’s Health 3

Achieve Progress on Clean Air Programs 3

Brownfields 3

Improve Environmental Information 4

Strengthen State and Tribal Partnerships 5

Global Climate Change 5

19971998 1999 2000

Number of 

Budgets Budgets

Budget Years in Which Appear Budget Years in Which Appear

Administrator Priorities Administrator Priorities

The multitude of national priorities is a source of concern and confusion to EPA’s regions and the 

states, as noted in a recent National Academy of Public Administration report: 

EPA’s management system is stymied by a wide variety of priorities imposed by Congress and 

the administration, and by itself. EPA staff are bewildered by the number of conflicting 

priorities, statutory deadlines, court-imposed requirements and public participation needs. 

States have challenged EPA to limit its priorities and devise a more flexible and open priority 

setting approach that accounts for state priorities. 

7
Recommendation 

By making EPA’s priority setting clearer, the Agency can increase involvement of both internal 

and external stakeholders. A more deliberate process for considering state priorities may begin to 

address the criticism of EPA’s national strategic planning process. In general, although risk and 

cost information is imperfect, it is already used by some EPA program offices to set priorities. 

Therefore, we recommend that EPA: 

2-1. Define and document a national, multi-year priority setting process, and ensure that the 

process fully and visibly uses available risk and cost information in setting priorities. 

8
3. Strategies 3. Strategies Has EPA developed results-oriented 

strategies that clearly link annual plans 

and budgets to accomplishments? 

EPA has taken many actions to comply with GPRA’s strategic planning requirements, although it 

faces significant challenges in developing results-oriented strategies that clearly link annual plans 

and budgets to accomplishment of long term outcomes. EPA has provided a GPRA framework, 

guidance, and training, among other actions. However, EPA needs to better describe how, when, 

and at what cost agency goals are to be achieved; the behavioral changes needed to accomplish 

these goals; and the links between current activities and long term outcome goals. 

Agency Progress in Strategic Planning 

Since GPRA’s passage, EPA has undertaken many actions

to comply with the Act’s strategic planning requirements

and enhance the Agency’s ability to manage for results. A

critical early step was the establishment of a framework

within the Office of the Chief Financial Officer (OCFO)

designed to align planning, budgeting, analysis, and

accountability in an integrated system. This framework is

designed to link EPA’s annual planning and budgeting

decisions to EPA goals.

In addition, OCFO has: (1) provided guidance on the

planning, budgeting, analysis, and accountability system;

(2) facilitated workgroups to help implement the new

system; (3) issued guidance on how to link annual

performance goals to long term outcomes; and (4) assisted

national program managers in developing and delivering

GPRA training to selected staff.

Annual 

Performance Goals 

Annual Planning 

And Budgeting 

Strategies 

Subobjectives 

Objectives 

Goals 

Framework for GPRA 

Implementation at EPA 

Another important step involved creating categories to aid

program planners in understanding where their environmental program goals and activities fall

within a results-oriented hierarchy. Known as the Hierarchy of Performance Indicators, program

managers are now required to use this environmentally-based categorization tool when developing

their annual performance goals and corresponding performance measures. According to OCFO

guidance, this tool should help the Agency move away from activity-based outputs to resultsbased outcomes. The most recent categorizations from OCFO are shown in the following table.

Modified Hierarchy of Performance Indicators 

9
Level Measure of Progress 

1A Customer Satisfaction 

1B Service Delivery 

1C Any EPA, state/tribal, other governmental, multi-governmental, and/or multinational regulations, 

activities, tools, standards 

2 Actions/Responses by regulated community 

3 Changes in, or prevention of, volume, mass, and/or toxicity of discharges, loadings, or emissions 

4 Ambient conditions 

5 Exposure or body burden/uptake 

6 Risk or impacts to ecology, health, and/or welfare 

R Research and development 

Agency Challenges in Strategic Planning 

Although the actions taken by EPA to comply with GPRA’s strategic planning requirements have 

helped managers improve their focus on results, the Agency still faces a number of challenges. 

These include: (1) describing in sufficient detail for external stakeholders how, when, and at what 

cost goals are to be achieved; (2) ensuring that strategies clearly identify the behavioral changes 

required to achieve the results-oriented outcomes desired; and (3) clearly linking current activities 

to long term outcome goals. 

Describing Plans in Sufficient Detail 

Our analysis of the key components of five key GPRA strategic planning documents2 for three 

EPA goals (air, water, and waste management) indicated that these strategies do not provide 

sufficient detail to allow meaningful evaluation and input by interested stakeholders for the out 

years. In each instance, the strategies were vague in their explanation of the actions that state, 

local, and tribal agencies needed or would be expected to take to achieve the program’s outcome 

goals. Congress anticipated that it may take several planning cycles to refine the strategic 

planning process and that strategic plans would be continually improved as various planning 

cycles occurred. According to the General Accounting Office’s (GAO’s) 1997 GPRA Guide,3 

external reviewers of agency strategies should be able to understand and evaluate the linkages 

between inputs, outputs, and outcomes and reasonably see which activities lead to what 

intermediate and, eventually, long term outcomes. Vagueness in this critical aspect of the 

2 The documents are (1) FY 1999 Annual Performance Report - EPA’s March 2000 GPRA submission to Congress (Version ID 3830 

enacted in President’s Budget, Version 1); (2) EPA’s September 2000 Strategic Plan (EPA/190-R-00-002); (3) EPA’s September 1997 Strategic Plan 

(EPA/190-R-97-002); (4) EPA’s FY 2001 Annual Plan (EPA/205-R-00-002); and (5) EPA’s FY 2001 Congressional Justification (CJ), also known 

as the Budget Justification. 

3

 Agencies’ Strategic Plans Under GPRA: Key Questions to Facilitate Congressional Review, GAO/GGD-10.1.16, May 1997. 

10 
agency’s GPRA planning was previously noted by the House Science Committee in 1997 when it 

examined EPA’s first plan. The committee noted that 

The EPA draft plan acknowledges the States as a significant partner in achieving 

environmental goals. Yet beyond this acknowledgment, the State role (as well as 

that of the Indian tribes) is unaddressed in the draft strategic plan. 

Desire for greater understanding of the details about how EPA plans to achieve its GPRA goals 

continues to be expressed by the public, members of Congress, GAO, and others. EPA officials 

for air, water, and waste management recognize that their respective programs need to improve 

their descriptions of how goals are to be achieved, when, and at what cost, particularly for 

Years 2 through 5 of their strategies. They also pointed out that, in some instances, they have 

developed more detailed strategies which could be cost effectively linked using the World Wide 

Web. For example, air program officials noted that they developed a fairly extensive urban air 

toxics strategy in 1999, setting forth the agency’s strategic thinking 10 years into the future. 

Similarly, water program officials noted that in 1998 they developed a clean water action plan 

describing over 100 actions that EPA, Agriculture, and other federal agencies planned to take to 

improve the Nation’s surface water over a 7-year period. Lastly, EPA officials suggested that the 

existing Hierarchy of Performance Indicators could be improved by establishing a category for 

state, local, and tribal partners separate from EPA programs. 

Ensuring Clear Identification of Behavioral Changes 

A second challenge involves ensuring that strategies clearly identify the actions and behavioral 

changes that the regulated community needs or would be expected to make in order for the 

Agency to achieve its outcome goals. Our analysis of the GPRA strategic planning documents 

indicates that the air and water program strategies do not provide sufficient details about the 

expectations for the regulated community. The strategies were vague in their explanation of the 

actions that the regulated community would need to take to achieve the program’s outcome 

goals. The following table shows how these behavioral changes are a critical step in the logic 

matrix for accomplishing EPA’s outcome goals. 

11
Logic Matrix for Accomplishing Goals 

Note a 

Note b 

Level Level 2 Level 3 Level Level 5 Level 6 

INPUTS OUTPUTS OUTCOMES 

1 4 

Resources 

Goal 

Objectives 

Subobjectives 

Strategies 

Annual Performance Goals 

Annual Planning 

and Budgeting 

Internal 

EPA 

Actions and 

Activities 

Actions by 

State, 

Local, 

Tribal and 

other 

Regulatory 

Agencies 

Actions and 

Behavioral 

Changes 

by the 

Regulated 

Community 

Reduced 

Amounts 

and/or 

Toxicity of 

emissions 

Improved 

ambient 

environ -

mental 

conditions 

Changes in 

ambient 

conditions 

relative to 

health-based 

standards 

Reduced 

exposure 

or body 

burden 

Human and 

ecological 

health 

Progress toward 

long-term outcomes 

Note c 

Note a: Simple logic model categorizations. 

Note b: OCFO Modified Hierarchy of Indicators 

Note c: EPA/OIG expanded hierarchy of indicators 

According to representatives of the national programs for air and water, using logic model 

concepts to perform a matrix analysis similar to that above would be helpful in understanding the 

critical linkages in managing for results. They noted that it is difficult to link behavioral 

expectations in their media programs when the majority of activities and funding for the needed 

actions are contained in a separate Agency goal. Although a critical link in the GPRA process, 

they explained that the existing Agency goal structure, where compliance and enforcement is a 

separate goal (Goal 9) generally funded and carried out separately from their media programs, 

sometimes hinders their efforts to clearly identify the actions and behavioral changes that the 

regulated community needs or would be expected to make in order for the Agency to achieve its 

outcome goals. Representatives of both programs noted that EPA’s decision to separate 

compliance and enforcement goals from media program goals reflected extensive deliberations 

and recognized that there would be tradeoffs; however, they also said it may be appropriate to 

evaluate these tradeoffs again. 

Linking Current Activities to Long Term Outcome Goals 

A third challenge the Agency faces involves clearly linking current activities to long term outcome 

goals. A vague statement of actions beyond the near term makes it difficult for external groups to 

see clear linkages, although the agency staff may be well aware of these linkages. Additionally, 

the Agency’s overall long term goal - - true human and ecological health - - is shared with many 

12
other federal and non-federal entities, difficult and costly to measure directly, and hard to 

explicitly link to some near term agency activities. As such, strategies often rely heavily on the 

fulfillment of largely EPA-controlled surrogate measures of human and ecological health, such as 

a reduction in the emissions of regulated pollutants or a reduction in the number of violations 

reported annually. The implicit assumption is that these reductions or changes will eventually lead 

to true human and ecological health. 

EPA’s strategies recognize the need to rely on others for long term goal achievement. However, 

EPA needs to provide more explicit descriptions of the partnering activities, roles, and 

responsibilities for outcomes by these non-EPA groups. This would better enable external 

reviewers to both understand and evaluate the linkages between inputs, outputs, and outcomes, 

and accomplish GPRA’s purpose of improving the public’s confidence in the ability of federal 

agencies to manage for results. 

Recommendations 

We believe EPA has an opportunity to improve its GPRA strategic plan and simultaneously 

improve its relationship with state, local, and tribal agencies, as well as improve the public’s 

confidence in the Agency. As such, we recommend that EPA: 

3-1. Link other, more detailed, planning documents to its GPRA strategic plan by providing 

World Wide Web citations (links), thus providing external stakeholders with easy access 

to more in-depth information on how EPA plans on achieving long term goals. 

3-2. Revise the existing Hierarchy of Performance Indicators to clearly establish a place for 

state, local, and tribal agencies’ expected roles in achieving environmental goals and 

proactively involve these partners in GPRA strategic planning for the next round of 

strategic plans, due in 2003. 

3-3. Work across agency goals to develop clear expectations for the behavioral changes EPA 

expects industry and the regulated public to make if envisioned goals are to be achieved. 

3-4. Work with other federal agencies that have human health and ecology goals to establish 

long-term outcome-oriented goals that reflect the coordinated efforts of multiple federal 

agencies. 

13
4. Measurement 4. Measurement Does EPA have systems to 

measure, evaluate, and report on 

environmental results? 

EPA has taken steps to put systems in place to measure and report on progress under GPRA, but 

continues to have difficulty developing and obtaining the information necessary for measuring, 

evaluating, and reporting on long-term environmental outcomes. Since GPRA’s enactment, EPA 

has aligned its management systems to help support implementation of GPRA, by establishing: 

• An Office of the Chief Financial Officer, to work with program offices to develop and track 

Agency-wide environmental measures. 

• An Office of Environmental Information, to centralize responsibility for improving interagency 

data sharing, as well as improve the accuracy, reliability, and scientific basis of environmental 

information. 

• An Evaluation Network, to stimulate and promote evaluation as a valuable tool for EPA 

program managers. 

EPA has been recognized for its effort to align its budgeting, planning, and accounting systems in 

an attempt to track and report on the resources used in achieving its GPRA objectives. 

Nonetheless, EPA continues to face challenges in implementing GPRA. EPA’s performance 

information tends to focus on program activities (outputs) rather than results (outcomes), due to 

the difficulty in obtaining and measuring environmental results. Further, the majority of EPA’s 

information is collected by states, and the inconsistencies in that information makes its use for 

measuring, evaluating, and reporting difficult. 

Measuring and Reporting on Results 

EPA faces a number of challenges in attempting to 

measure and report on environmental results. A 

majority of EPA’s annual performance goals and 

measures focus on program activities or outputs 

(number of enforcement actions, number of permits 

issued, etc.). Despite the vast array of data in EPA’s 

information systems, GAO, the states, regulated 

entities, and EPA itself have pointed out that the 

Agency does not have much of the information it 

needs pertaining to environmental conditions and 

trends and the potential human health risks of various 

pollutants. This makes it difficult to report on the 

environmental results of EPA’s activities. Further, 

environmental programs may take several years to 

Activity/Output 

Measures 

Human and 

Ecological Health 

Outcomes 

Data Available 

Data Unavailable at 

present Time 

Availability of Environmental 

Results Data 

14
show results. Thus, measuring and reporting on end results may also take years. In addition, 

even with better measures and data, linking environmental programs and activities to outcomes is 

complicated by a myriad of external factors, including weather, international environmental issues, 

and economic activity, many of which are outside of EPA’s control. 

Further, while EPA does not currently receive all state information, a majority of the information 

that EPA uses for measuring and reporting is collected by states. According to the Environmental 

Council of States, some 94 percent of all environmental quality data in national databases 

originates with the states. In previous reports, the OIG, as well as others, have found data for 

individual states to be inconsistent, and data among various states to be incompatible. States are 

encouraging EPA to allow more flexibility in what data is collected and how it is collected. 

According to the Environmental Council of States, the states would rather concentrate on setting 

broad indicators of environmental improvement and state capacity rather than on traditional 

statistical and financial measures. This concern is echoed by the regulated community, which is 

urging EPA to focus on defining, measuring, and rewarding environmental results and reorienting 

core regulatory functions so they are driven primarily by performance goals and not activity 

measures. While the EPA’s Office of Chief Financial Officer continues to work with program 

offices to develop measures that demonstrate environmental outcomes, it is proving to be a 

difficult and complex task. 

Evaluating and Reporting on Results 

There is an important distinction between performance measurement and program evaluation. 

Measurement is the ongoing monitoring and reporting of program accomplishments and seeks to 

answer the question, “what was achieved?” Program evaluation attempts to determine the 

manner and extent to which programs achieve intended objectives, usually through objective 

measurement and systematic analysis. The intent is to analyze why results were achieved, so that 

further action can be taken based on lessons learned. Thus, evaluation is aimed at providing the 

link between actions taken and results achieved. 

According to GAO, in some programs, outcomes are not quickly achieved or readily observable, 

or their relationship to the program is uncertain. In such cases, program evaluation is needed, 

along with performance measurement, to examine the extent to which a program is achieving its 

objectives.4 EPA has been without a centralized evaluation function since 1995, when the 

Program Evaluation component of the Office of Policy, Planning and Evaluation was disbanded. 

More recently, EPA established a Program Evaluation Network to help stimulate interest in 

evaluation and increase the Agency’s capacity to conduct evaluations. The Office of Policy, 

Economics and Innovation created a division to evaluate EPA’s many reinvention initiatives and 

help support and promote evaluation activities throughout the Agency. OIG has also established 

an Office of Program Evaluation to evaluate EPA programs. 

4

 Program Evaluation: Studies Helped Agencies Measure or Explain Program Performance 

(GAO/GGD-00-204) 

15 
While the Office of Chief Financial Officer’s Office of Planning, Analysis and Accountability has 

an analysis component, the staff in this office do not see their role as conducting evaluations for 

EPA program offices. Rather, they view their role in evaluation as facilitating and helping EPA 

programs develop capacity through mechanisms such as the evaluation network. EPA program 

offices are beginning to focus some attention on conducting evaluations, but the resources they 

are devoting are limited and the distribution of those resources uneven from program to program. 

Additionally, EPA has not established policy or guidance on how program evaluation results are 

to be used and tracked by management. Without such controls, there is little assurance that 

evaluative results will be used. 

Recommendations 

For EPA to demonstrate and report on the outcomes and impact of environmental programs, a 

concerted effort will be needed between EPA, states, and the regulated community to develop 

outcome-oriented performance information and environmental indicators. Where such 

information does not exist or will take time to develop, program evaluation can help bridge the 

gap and examine the extent to which programs are achieving their objectives. Consequently, we 

recommend that EPA: 

4-1. Work with states and the regulated community to develop an information strategy that 

includes: 

• Clearly defined human health and ecological outcomes, as well as output measures. 

• Expanded use of environmental indicators and performance-based management. 

• Clear and consistent data definitions and standards for improving data quality. 

• Identification of the resources necessary to obtain, compile, store, and share data. 

4-2. Encourage the use of evaluation staff as a resource for program managers and develop 

guidance for how evaluative results should be used and tracked. 

16
5. People 5. People Does EPA have the human resource systems 

needed to manage for results? 

EPA recognized the importance of human capital as a key Agency priority in its Strategic Plan, 

and has recently developed and begun implementing a separate Strategy for Human Capital. 

However, in its Strategic Plan, EPA did not specifically address the human capital needs of any of 

its strategic goals. A prerequisite for EPA’s successful implementation of GPRA is the 

application of the right skills and competencies within a results-oriented culture. EPA needs to 

better integrate human capital into strategic plans; more effectively develop competencies in 

management, science, and technical skills; and better align individual accountability with Agency 

goals and results. 

Human Capital Emphasized 

GPRA expresses the importance of planning human capital needs to attain Agency goals. Several 

independent evaluations by GAO, National Academy of Public Administration, Environmental 

Council of States, National Research Council, and EPA’s OIG have focused on aspects of EPA’s 

human capital management. These evaluations addressed the need for developing and aligning 

scientific and management competencies, as well as individual performance expectations and 

accountability, with strategic goals. For example: 

• GAO stated that, “successful human capital management strategies are the key to maintaining a highly 

skilled, energized, and empowered workforce that is focused on results.” 

• GAO also stated, “EPA is not able to identify the size of its workforce and the competencies that need 

to be deployed among its organizational components to effectively carry out its strategic goals and 

objectives.” 

• The National Research Council stated that, “to achieve scientific and technical excellence, EPA must 

attract, retain, and properly support first rate dedicated professional staff.” 

The OIG recently reported in Superfund that the Headquarters program office and several regions 

did not identify the quality assurance training needs, and in some regions training was not always 

provided. Also, audits have repeatedly noted a need to better train managers in their 

administration and oversight of EPA’s assistance agreements. Additionally, we found EPA 

employees in the hazardous waste program needed more rigorous training to calculate proposed 

penalties against violating facilities, and that the National Environmental Performance Partnership 

System (NEPPS) has not been well-integrated into EPA due to lack of, among other things, 

training and guidance. Human capital concerns will intensify in the coming years as 52% of 

Senior Executive Service, 47% of scientific staff, and 25% of EPA’s total workforce will be 

eligible for retirement within 5 years, while competition for scientific, technical, and managerial 

talent will likely increase. 

17
Human Resource Elements Essential 

Effective implementation of performance-based management as envisioned by GPRA is dependent 

on senior management’s willingness and ability to strategically manage all of the Agency’s 

resources, including human capital. Specifically, as noted by GAO, this requires aligning and 

developing day-to-day workforce competencies and expectations with strategic and program 

plans and results at all levels for efficient and effective application of limited human resources, and 

to promote a results-oriented culture. Human resource elements essential for successfully 

implementing GPRA and attaining EPA’s goals are: 

• Integrating Human Capital into Strategic Plans 

• Developing Competencies in Management, Science, and Technical Skills 

• Aligning Individual Accountability with Agency Goals and Results 

Our review of these elements noted the following future challenges EPA faces in attaining its goal 

of having the human resource systems needed to manage for results. 

Integrating Human Capital into Strategic Plans 

The Agency recognizes that one of its biggest challenges over the next several years is the 

development and implementation of a workforce planning system that focuses attention on the 

identification of needed skills and competencies, and the means of addressing them. EPA 

developed a Workforce Development Study to discuss competencies needed to: meet the current 

Agency mission; identify possible future missions of the Agency; and suggest new competencies 

for future scenarios. EPA has broadly recognized the importance of human capital as a key 

Agency priority in its September 2000 Strategic Plan, but has not specifically addressed human 

capital needs in its goals. Although neither the Strategic Plan nor the Workforce Study address 

specific human capital issues or make commitments to specific human capital actions, EPA is 

developing a more specific workforce development plan. 

EPA needs to follow through on its recently developed Strategy for Human Capital by making a 

consistent commitment to evaluate the number of staff and competencies needed, and how those 

employees should be deployed among its program areas and in various locations to meet its 

strategic goals. GAO recently reported that while EPA has implemented several initiatives during 

the past decade to better understand the demands facing its workforce, these initiatives have not 

received the resources and senior management commitment to bring them to fruition. Therefore, 

EPA needs to implement a strong management approach to training and development, and 

organize its efforts around principles agreed upon by senior Agency management. The need for 

developing, integrating, and aligning human capital planning with the strategic and annual plans 

will be magnified as: EPA’s role changes; the need for specific sophisticated scientific, 

management, social, and technical skills increases; the current workforce ages; and competition to 

recruit and retain highly qualified staff intensifies. 

18
Developing Competencies in Management, Science, and Technical Skills 

One of EPA’s long-standing human capital challenges is attaining and developing the needed staff 

competencies in management, scientific, and technical positions. EPA has not assessed the gap 

between (1) critical skills and competencies needed to achieve its goals and objectives and (2) 

existing staff skills and competencies. Furthermore, appropriate training is essential for 

management to lead and promote a results-oriented culture in EPA and for scientific and technical 

staff to achieve and maintain excellence and credibility among their peers. However, National 

Research Council, GAO and OIG reviews found that EPA staff often lacked specific training for 

technical, scientific, and financial program responsibilities. 

Aligning Individual Accountability with Agency Goals and Results 

GPRA requires EPA to clearly establish results-oriented performance goals for which it will be 

held accountable. Since Agency accountability is dependent upon the collective accountability of 

its staff, EPA must align individual employee performance expectations with Agency goals. This 

is necessary so that managers and staff understand the connection between their daily activities 

and the Agency’s progress in accomplishment of its goals. As described by EPA staff at different 

levels and components, EPA has not consistently shifted the orientation of individual performance 

expectations and accountability from process or activity completion to achieving or contributing 

toward attainment of Agency goals. EPA also has not integrated a GPRA results-based 

accountability framework and culture into staff day-to-day activities by regularly incorporating 

results-based expectations into its staffing, performance management, and development 

documents, such as vacancy announcements, position descriptions, performance agreements, and 

individual development plans. 

Recommendations 

For EPA to achieve its strategic goals, it must focus on how to best invest in its people and link 

their responsibilities and performance to Agency results. EPA is taking steps in the right direction 

with its workforce development plans and strategies, but now must make decisive commitments 

and investments to integrate human capital requirements into its Strategic Goals and Annual 

Plans. In order for EPA to effectively implement GPRA, we recommend that EPA: 

5-1. Identify needed scientific, technical, and management competencies to achieve specific 

mission goals, and measure the extent to which employees possess these competencies. 

5-2. Fund and implement training and development to address skill gaps. 

5-3. Apply management, technical, and scientific competency criteria to employee recruitment, 

selection, development, and performance accountability processes. 

5-4. Align staff performance expectations and contributions with Agency goals. 

19
6. Accountability 6. Accountability Has EPA established an 

accountability system to ensure 

efficient achievement of its goals? 

EPA has established an accountability system with the intent of managing resources efficiently, 

with information to ensure the effective achievement of its goals, and has been recognized as a 

leader in developing a goals-based budget aligned with its programmatic and operational outputs 

and outcomes. However, this framework, while well intentioned, tends to perpetuate EPA’s 

“stove-piped” organization. Consequently, it does not address overlapping environmental issues, 

as well as the different needs and priorities of EPA’s partners – the states – who implement the 

majority of EPA’s programs. EPA’s system of accountability for attainment of its goals do not 

sufficiently link or align authority, responsibility, resources, measures, and outcomes. 

Statutes Require Accountability 

Congress’ desire to hold agencies accountable for their performance and results was the motivating 

force behind the Chief Financial Officers Act of 1990 (CFO) and GPRA. While the CFO Act 

established the foundation for improving management and financial accountability, GPRA created 

requirements for agencies to generate additional information Congressional and executive branch 

decision makers need in considering measures to improve government and reduce costs. The 

purpose of GPRA, as stated in the law, is to systematically hold federal agencies “accountable for 

achieving program results.” Additionally, the Reports Consolidation Act of 2000 authorizes 

agencies to combine performance information with cost information, to impose a new, more 

business-like framework for management and accountability for federal agencies. Appropriate 

information on results and costs is necessary for accountability. 

EPA Has Established an Accountability System 

EPA has established an accountability 

system tied to its goals, as shown in 

the accompanying chart. The Agency 

has taken a goals-based budget that is 

aligned with its programmatic and 

operational outputs and outcomes as 

a means of accountability and to 

accomplish its goals. EPA has been 

recognized as a leader in developing 

this innovative approach. 

Memoranda Memoranda of Agreement of Agreement

Grant 

Workplans/Performance 

Partnership Agreements Partnership Agreements

Annual Performance 

Report/ Mid-Year Deputy 

Administrator Meetings Administrator Meetings

EPA’s Accountability System EPA’s Accountability System

Administrator’s Administrator’s

Priorities Priorities

National Program National Program

Goals and Priorities Goals and Priorities

Regions Regions

States and Tribes States and Tribes

Annual Performance 

Plan and Budget Plan and Budget

Accomplishment Reports Accomplishment Reports 

Regional Mid-Year and 

Annual Reporting Annual Reporting

20
Essential elements for an accountability system that will ensure efficient achievement of EPA 

goals are: 

• Alignment of EPA Goals with Stakeholder Multimedia Priorities 

• Agreement on Roles, Responsibilities, and Expectations 

• Comparability and Standardization of Results-Oriented Measures and Data 

• Linkage of Cost Accounting Information with Activities used for Decision Making, and 

Results 

Our review of these elements noted the following challenges EPA faces in attaining its goal of 

having an effective accountability system. 

Alignment of EPA Goals with Stakeholder Multimedia Priorities 

EPA establishes its long-term and annual goals and develops its budget by goal. Accountability 

for progress and the use of resources by EPA goal is not consistent with the flexibility necessary 

for multi-media activity, investment, and priorities that EPA regional offices need in order to 

address the interests of state partners. For example, EPA regional officials told us that EPA’s 

goal structure tends to promote a stove-piped organization and that it is not conducive to crossmedia work, such as Environmental Justice, Brownfields, and Children’s Health. EPA needs to 

develop a process to accurately capture state priorities, and where agreed upon, adjust budgets 

and accountability to reflect those priorities. 

Agreement on Roles, Responsibilities, and Expectations 

EPA depends heavily on states to fund and implement national programs as well as provide most 

of the environmental data. The charts below demonstrate the trend in environmental work done 

by the states in terms of percentage of the programs that have been delegated to the states, and 

the relative percentage of EPA’s investment in environmental protection compared with that of 

the states. Also, according to the Environmental Council of States, at least 80 percent of all 

environmental enforcement actions are taken by the states. 

Delegated to the States 

1993 

1998 

41% 

71% 

Percent of Environmental Programs FY 1996 Environmental Investment 

By States & U.S. EPA 

$ in Billions 

EPA 

$6.5 

34.2% 

States 

$12.5 

65.8% 

Total: $19.0 

Source: Environmental Council of States (ECOS) 

21
EPA and states have not yet agreed on how states should be provided with flexibility, along with 

accountability, for environmental results. Relations between EPA and states have been strained 

due to disagreements over: 

• Respective roles and the extent of federal oversight 

• Priorities and budgets 

• Results-oriented performance measures, milestones, and data 

Comparability and Standardization of Results-Oriented Measures and Data 

There is limited standardization and comparability of data between states. As already noted, the 

Environmental Council of States estimates that about 94 percent of the national data on 

monitoring of environmental quality is collected by the states, and lack of compatibility of data 

makes it difficult to aggregate data for trending, comparing, and evaluating performance and 

progress. 

Linkage of Cost Accounting Information with Activities and Results 

Since Fiscal Year 1999, EPA has been able to link its budget and costs to goals, objectives, and 

sub-objectives, but not often to activities and outcomes. For example, EPA’s FY 2000 Annual 

Report does not provide cost information associated with its annual performance goals and 

measures. According to the Office of the Chief Financial Officer, at least one EPA region is 

piloting a cost accounting and performance reporting tool, and EPA has also adopted cost 

accounting practices in the Superfund program, Oil Spill programs, and in the Working Capital 

Fund. EPA has not consistently decided what level of cost accounting is needed for decision 

making in all programs and has not defined an approach for using cost information. EPA’s cost 

accounting system generally does not track costs below the Sub-Objective (i.e., 1-5 year) level, nor 

does it track across goals as demonstrated in the diagram below. The Agency recognizes the need 

for cost accounting for key activities and is currently developing cost accounting tools to better 

respond to Agency needs. To obtain this information, EPA should identify the key activities for 

which it needs to accumulate costs and develop a means for collecting and using this information 

for results based management, program execution, and evaluation. 

Hierarchy of EPA Resource Allocation 

Goal 

Objective 

Sub-Objective 

Strategy/Activity 

Annual Performance 

Goal 

A cost accounting system linking activity costs with performance, 

would enable EPA to: 

T Identify Return of Investment 

T Compare investment options and trade-offs 

T Account for many regional and state multi-media activities 

linked to multiple sub-objectives 

T Promote and recognize greater operational efficiencies 

T Make optimal budgeting and planning decisions 

Annual Planning & 

Budgeting 

22
Recommendations 

EPA has many federal and state partners and stakeholders that collectively have a variety of needs, 

levels of involvement, and interdependence in contributing to EPA’s national goals. EPA can 

improve its accountability by seriously considering state and stakeholder priorities and 

responsibilities in its planning, budgeting, and reporting systems, and aligning its goals, activities 

and resources with performance measures and results. The Agency has the capacity to link its 

budgeting, planning, and accountability information at a high level and must follow through on the 

actions it has started to improve performance and cost data. To strengthen accountability 

processes, we recommend that EPA: 

6-1. In setting expectations: 

• Define roles and responsibilities for EPA and its partners; 

• Agree on priorities, strategies and results; and 

• Develop and use better environmental performance measures and data. 

6-2. In the short-term, integrate existing cost and performance data, through Agency-wide cost 

accounting, to: 

• Identify return on investment, both at the activity and outcome level; 

• Compare different investment options for future investment choices; and 

• Promote and recognize greater efficiencies and opportunities for savings. 

6-3. In the long-term, identify the key activities for which it needs to accumulate costs and 

develop a means for collecting and using this information for budget formulation and 

planning, program execution, and evaluation. 

23
Scope and Methodology

Exhibit 1 

Scope and Methodology 

Purpose of Review 

To determine what progress EPA has made in using GPRA to manage for environmental results, 

we conducted a review of GPRA’s implementation activities. The intent was to inform EPA’s 

leadership and interested stakeholders about EPA’s decisions that set the stage for GPRA 

implementation; progress in using GPRA; challenges that EPA faces in implementing GPRA; and 

opportunities for near and long-term improvements. 

Questions Posed 

To conduct our review, we posed questions related to six different critical aspects of GPRA: 

1. Goals - Are EPA’s goals consistent with its statutory authority and key partner and 

stakeholder expectations? 

2. Priorities - Does EPA have a system for setting priorities to best achieve its environmental 

mission? 

3. Strategies - Has EPA developed results-oriented strategies that clearly link annual plans 

and budgets to accomplishments? 

4. Measurement - Does EPA have systems to measure, evaluate, and report on 

environmental results? 

5. People - Does EPA have the human resource systems needed to manage for results? 

6. Accountability - Has EPA established an accountability system to ensure efficient 

achievement of its goals? 

Review Steps Taken 

To answer these questions, we interviewed EPA Headquarters and regional officials, and also 

discussed GPRA implementation with environmental officials in five states. In addition, we 

reviewed previous reports, publications, and correspondence related to GPRA implementation 

written by the: 

• General Accounting Office 

• Environmental Council of States 

• Senate Governmental Affairs Committee 

• House Oversight and Reform Committee 

24
• National Academy of Public Administration 

• National Research Council 

• George Mason University’s Mercatus Center 

We also took into account information in other reports published by the EPA OIG. 

We reviewed Agency-wide projects on core competencies and suggestions by states and the 

regulated community for improving EPA’s management of environmental programs. We reviewed 

the GPRA framework for EPA’s Offices of Water, Air and Radiation, and Solid Waste and 

Emergency Response (Goals 1, 2 and 5). We reviewed publicly available documentation, such as 

EPA’s Strategic Plan, Congressional Budget Justification, Annual Performance Plan, and Annual 

Performance Report; as well as comments from the public, states, and the regulated community 

regarding these documents. This review was considered a special project and, as such, was not 

conducted in accordance with Generally Accepted Government Auditing Standards. 

25