OFFICE OF INSPECTOR GENERAL

 

Catalyst for Improving the Environment

 Evaluation Report

EPA Needs a Better Strategy

to Measure Changes in the

Security of the Nation’s Water

Infrastructure

Report No. 2003-M-00016 

September 11, 2003
Report Contributors: Erin Barnes-Weaver

Eric Hanger

Jeffrey Harris

Fredrick Light

Ricardo Martinez 

Erin Mastrangelo

Abbreviations

 

DHS Department of Homeland Security

EPA Environmental Protection Agency

ISAC Information Sharing and Analysis Center

VA Vulnerability Assessment
1

The EPA Office of Inspector General is conducting preliminary research on an evaluation of water system

security activities in support of the Agency’s Strategic Plan for Homeland Security.

 

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

 WASHINGTON, D.C. 20460

THE INSPECTOR GENERAL

MEMORANDUM

SUBJECT: Final Evaluation Report: EPA Needs a Better Strategy to Measure Changes in the

Security of the Nation’s Water Infrastructure

Report No. 2003-M-00016

FROM: Nikki L. Tinsley

TO: G. Tracy Mehan, III

Assistant Administrator for Office of Water

As part of our ongoing evaluation of Environmental Protection Agency (EPA) activities to

enhance the security of the Nation’s water supply, we noted an issue that requires your attention. 

Specifically, we suggest that EPA develop specific measurable performance indicators of water

security activities. We propose this action because, during our preliminary research,1

 we

obtained information that indicates EPA has neither:

• Articulated measurable goals for EPA’s water security efforts; nor

• Obtained or analyzed data to develop a baseline for water security.

It is important for EPA to develop measures to monitor the security of our Nation’s water supply

and to ensure Federal funds are not spent without clear goals or expectations. To effectively

perform its lead agency responsibilities, EPA needs to collect and analyze data that depicts the

changes in security levels at water utilities.

Our observations and suggestions are based on information obtained from our interviews with

water security experts, water utility officials, and EPA headquarters and regional representatives;

attendance at water vulnerability assessment training; and a review of EPA’s Strategic Plan for

Homeland Security, dated September 2002. We are performing our evaluation in accordance

with Government Auditing Standards, issued by the Comptroller General of the United States.

EPA Designated as Lead Agency for Water Security

The Nation’s water supply is one of our most vital natural resources. Potential threats to this

resource include contamination with biological, chemical, or radiological agents, or destruction

of physical infrastructure. The water supply is also dependent on other critical infrastructures,
2

such as energy and transportation. Presidential Decision Directive 63, issued in May 1998,

designated EPA as the lead agency responsible for the water sector and for accomplishing the

following functions:

• Establish and maintain channels of communication with all private and public entities having

an infrastructure assurance interest in the sector;

• Facilitate the selection of a Sector Infrastructure Assurance Coordinator;

• Assist the Sector Coordinator in establishing and operating an effective information-sharing

program;

• Draft new legislation and regulations, as required, and propose the use of Federal incentives

to facilitate private investment in assurance programs, if appropriate;

• Promote infrastructure assurance education and training, to include advocating use of best

practices, within the sector;

• Assist in developing plans for prevention (long-term reduction of vulnerabilities and shortterm defensive actions), mitigation, restoration, and reconstitution; and

• Coordinate, in support of the Federal Response Plan, as amended, management of the

consequences of a successful infrastructure attack and prepare for various contingent attacks

through participation in training and exercise programs.

EPA’s Efforts to Improve Water Security

To better execute its responsibilities after the terrorist attacks on September 11, 2001, EPA

developed a strategy for improving the security of water utilities. EPA’s Strategic Plan focuses

on preparedness and prevention, assisting those responsible for critical infrastructures in

assessing and reducing vulnerabilities and maximizing their response capabilities. Also, EPA

intends to develop technologies to improve the Nation’s critical infrastructure and key

responders’ abilities to detect and monitor environmental threats. Through this work, EPA plans

to “significantly improve the Nation’s overall capacity to protect critical infrastructure from

terrorist attacks.” EPA will rely on relationships with water utilities, water-related governmental

entities, and associations to assist utilities, and the Agency has already taken the following steps:

• Facilitated the development of vulnerability assessment methodology and training;

• Provided threat guidance to utilities to help them conduct vulnerability assessments and

identify possible threats to critical assets;

• Provided financial assistance to large drinking water systems to conduct one-time

vulnerability assessments;

• Funded research on technology development and verification; and

• Facilitated the development of a secure Information Sharing and Analysis Center for the

water utility sector (Water-ISAC) to exchange threat/incident information.

EPA’s Strategic Plan is organized into four mission-critical areas, the first being “Critical

Infrastructure Protection.” This area states that EPA will work with the States, tribes, drinking

water utilities, and other partners to enhance the security of water utilities. The Strategic Plan

articulates both the tactics to execute the Plan as well as the anticipated results. The following

table illustrates selected tactics and results. 
3

Selected Water Infrastructure Protection Tactics Anticipated Results

Technical Assistance and Grants

EPA will provide tools, training, and technical assistance

to assist water utilities in conducting vulnerability

assessments, implementing security improvements, and

effectively responding to terrorist events. 

EPA provided grants to large drinking water utilities for

vulnerability assessments, security enhancement

designs, and/or emergency response plans. 

By the end of fiscal 2003, all water utility

managers will have access to basic

information to understand potential water

threats, and basic tools to identify security

needs. By 2005, unacceptable security

risks at water utilities across the country

will be significantly reduced through

completion of appropriate vulnerability

assessments, design of security

enhancement plans, development of

emergency response plans, and

implementation of security enhancements.

Terrorism Methods and Prevention Techniques

EPA will work with the Department of Homeland Security

(DHS), other Federal agencies, universities, and the

private sector to:

– Solicit and review methods to prevent, detect, and

respond to chemical, biological, and radiological

contaminants that could be intentionally introduced in

drinking water systems;

– Review methods and means by which terrorists could

disrupt the supply of safe drinking water or take other

intentional actions against water collection, pretreatment,

treatment, storage, and distribution facilities; and

– Review methods and means by which alternative

supplies of drinking water could be provided in the event

of a disruption.

Starting in fiscal 2003, water utilities, key

response agencies, and policymakers will

have improved information and knowledge

to make timely and effective analytical and

technological decisions to enhance

security, detect contamination, and

respond to incidents.

Security Practices

EPA will work to implement water security practices in

ongoing water utility operations. EPA will also work to

build security concerns into ongoing review systems

(e.g., sanitary survey, capacity development, operator

certification, and treatment optimization program for

drinking water systems).

Beginning in fiscal 2003, water utilities will

incorporate security measures as a

standard aspect of day-to-day operations

and EPA, States, and tribes will review

security measures at water utilities on a

continuous basis. Through ongoing

practice and review, water utility managers

and employees will optimize security

measures.
Selected Water Infrastructure Protection Tactics Anticipated Results

2

The EPA Office of Inspector General has compiled these program management principles in Assessing

Organizational Systems: A User’s Guide, OA/OPE-5, November 5, 2002. The Guide identifies seven areas

requiring management attention for a successful program. The seven areas consist of Leadership, Strategic Planning,

Customer/Stakeholder and Market Focus, Information and Analysis, Human Capital, Process Management, and

Performance Results.

4

Communication with Utilities

EPA will work with other government agencies, utility

organizations, and water utilities to establish formal

communication mechanisms to facilitate the timely and

effective exchange of information on water utility security

threats and incidents. 

Starting in fiscal 2003, water utilities, law

enforcement agencies, and State and

Federal response and prevention

programs will have timely and accurate

security threat information and incident

analysis to make effective decisions for

water security preparedness and

response.

Coordination with First Responders

EPA will work in coordination with DHS to foster

coordination among Federal, State, tribal, and local

emergency responders, health agencies, environmental

and health laboratories, the medical community, and the

law enforcement community at all levels (Federal, State,

and local) concerning response to potential terrorist

actions against water utilities. This will be achieved

through training and support of simulations and

emergency response exercises.

In the majority of water security incident

responses and exercises, the decisionmaking and communication structures of

response agencies will function smoothly

(without critical errors).

Coordination with Other Critical Infrastructures

EPA will work with other critical infrastructure sectors to

further understand and reduce the impact to water

utilities of terrorist attacks on related infrastructures as

well as the impacts of attacks on water utilities to other

critical infrastructures.

Water sector vulnerabilities and impacts

resulting from attacks on other critical

infrastructure sectors will be reduced and

vice versa.

EPA Has Not Articulated Measurable Goals for Water Security

The Office of Water has not outlined how resources, activities, and outputs will achieve the

water security program’s goals. EPA’s Strategic Plan lacks fundamental components, such as

measurable performance results and information and analysis, to ensure the greatest practicable

reductions in risks to the critical water sector infrastructure. We based our observations on key

program management practices consistent with the President’s Management Agenda and the

Government Performance and Results Act.2

 
3

Floor Action on H.R. 3448, the Public Health Security and Bioterrorism Preparedness and Response Act

of 2002, in the Senate on May 23, 2002, and in the House on May 22, 2002.

5

Because EPA lacks the indicators that define the baseline for water security, EPA cannot monitor

program performance against goals. Without this baseline, EPA cannot determine whether its

strategy resulted in improved water security. Officials we interviewed at the Office of

Management and Budget and DHS also endorsed the need for EPA to establish performance

indicators to determine the effectiveness of its water security activities.

In the absence of specific measurable water security goals, the Agency has focused on

compliance with legislative requirements. For example, EPA states, “By 2005, unacceptable

security risks at water utilities across the country will be significantly reduced through

completion of appropriate vulnerability assessments; design of security enhancement plans;

development of emergency response plans; and implementation of security enhancements.” 

The completion of these documents, however, does not equate to the outcome of reducing

unacceptable security risks.

EPA Needs to Obtain and Analyze Information to Measure Changes in Security 

For EPA to develop performance indicators that measure changes in security, EPA needs to

collect and analyze information from water utilities. However, the Agency believes that it:

(1) lacks the authority to ask for information directly from water utilities or utilize information in

the Water-ISAC that would show changes in security levels, and (2) cannot analyze information

in vulnerability assessments because that would violate the Public Health Security and

Bioterrorism Preparedness and Response Act (Bioterrorism Act). 

However, we believe that EPA does in fact have the authority, as well as the responsibility, to

collect and analyze necessary information from these sources. Information obtained

anonymously, through the Water-ISAC, or through coordination with DHS, would provide EPA

with data to benchmark changes in security levels. EPA needs to review the vulnerability

assessments to identify and prioritize threats to water utilities. Subsection 6(A) of the

Bioterrorism Act appears to contemplate review and analysis of vulnerability assessments by

duly authorized Agency officials. Further, additional support for vulnerability assessment

analysis is found in the legislative history of the Bioterrorism Act. Specifically, congressional

members showed bipartisan support for EPA’s review of the vulnerability assessments to

develop plans to protect drinking water supplies, and noted the Agency’s discretion to review the

assessments and make recommendations to improve water security.3

• Representative Frank Pallone, Jr. (D-NJ) stated that EPA’s review of the findings of the

vulnerability assessments would “help the government understand the threats to our water

systems and develop plans to protect our safe drinking water supply.” 

• Representative Michael Bilirakis (R-FL) stated that EPA could use the assessments

“to address the threat of terrorism and for any other lawful purpose.” 
6

• Senator James Jeffords (I-VT) stated that “there is not a restriction on EPA’s discussing the

content of the assessments with persons who may benefit from information about the security

of our Nation’s water supply, such as state and local officials, nor is there restriction intended

by this bill upon a water system’s voluntarily sharing information with other systems,

emergency responders or communities. Our attempt to provide a safeguard against broad

disclosure of sensitive information does not lead us to conclude that our citizens should not

have the information they need to protect and inform themselves.”

EPA could also utilize information from DHS to develop performance indicators that illustrate

changes to security at critical infrastructures. An official at DHS stated that the Department is

developing performance indicators for other critical infrastructures, such as the chemical

industry. According to The National Strategy for the Physical Protection of Critical

Infrastructure and Key Assets (February 2003), DHS coordinates with Federal agencies to assess

threats to critical infrastructure and evaluate preparedness. The National Strategy places

responsibility with DHS for gathering threat and vulnerability information. However, EPA has

only recently allowed DHS to review drinking water utilities’ vulnerability assessments.

Suggestions

We suggest that EPA:

(1) Develop specific, measurable goals, objectives, and performance indicators for its water

security programs; and

(2) Utilize available sources of information to collect and analyze data to develop a

baseline for water security.

Agency Comments and Office of Inspector General Evaluation

In its response to our draft report, EPA agreed with our assessment that the Agency’s Strategic

Plan for Homeland Security lacks clearly defined performance measures for critical water

infrastructure protection activities. EPA indicated that it will be actively involved in developing

more outcome-focused performance indicators in its revised strategic plan. 

EPA also agreed with our assessment that it needs to develop a baseline for water security. EPA

said it will analyze a sample of vulnerability assessments to develop the baseline. EPA also

plans to use this information to assist in identifying tools that need to be developed, and to

research priorities and appropriate security enhancements for water utilities. EPA indicated it

will share this analysis with Congress and the Department of Homeland Security. 

We commend EPA’s commitment to create outcome-focused performance measures and a

baseline to measure and monitor changes in water security. However, the necessary information

required for these initiatives may not reside solely in the water system vulnerability assessments. 

As the lead agency for water security, we suggest that EPA partner with the utilities and collect

and analyze information from additional sources such as the Water-ISAC and through
7

coordination with DHS so that the appropriate actions could be taken to enhance the security of

the Nation’s water infrastructure.

The full Agency response is provided in Appendix A.

- - -

If you or your staff have any questions regarding this report, please call me at (202) 566-0847 or

Kwai Chan, Assistant Inspector General for Program Evaluation at (202) 566-0828.
8

Appendix A

Agency Response

August 22, 2003

MEMORANDUM

SUBJECT: Response to OIG Concerns regarding “EPA Needs a Better Strategy to Measure

Changes in Utilities Water Security”

DRAFT: Report No. 2003-M-00016

FROM: G. Tracy Mehan, III /signed/

Assistant Administrator

TO: Jeffrey K. Harris

Director for Program Evaluation, Cross-Media Issues

Office of Inspector General

I am responding to the two principal issues and concerns that were stated in your July 23,

2003, memorandum/report to me. Your preliminary research on performance measures of water

security activities indicates that EPA has neither:

• articulated measurable goals for EPA’s water security efforts; nor

• obtained or analyzed data to develop a baseline for water security.

I agree with your assessment that the Agency’s Strategic Plan for Homeland Security,

dated September 2002, lacks clearly defined performance measures for critical water

infrastructure protection activities. I believe this document should be considered a blueprint of

near-term (2002 and 2003) homeland security activities rather than a strategic plan in the strict

sense of that term. Attachment 1 highlights our significant accomplishments under this nearterm plan. I am a strong proponent of EPA’s Office of Homeland Security’s recently-announced

initiative to develop a revised strategic plan. This exercise may serve as the opportunity to

identify longer-term goals, objectives, and measurable performance indicators for all EPA

programs involved in high priority activities to protect public health and the environment from

terrorist and other intentional acts. I assure you that my staff will be very active participants in

this endeavor and will devote significant attention to the challenge of developing outcomefocused performance measures. For the FY 2005-2010 Agency-wide strategic plan, I assigned

my Senior Advisor for Water Policy to work with all water programs to shift from a

predominance of output-oriented performance measures to ones that will demonstrate more direct

results and benefits to public health and environmental protection. We made considerable
9

progress and will continue to make improvements in all water programs including critical water

infrastructure protection. 

In addition to the limited performance measures in the 2002 Strategic Plan for Homeland

Security, the Agency’s FYs 03 and 04 annual performance plans have contained annual

performance goals and measures. (Attachment 2) Because these measures are indeed outputs not

indicators of outcomes, I and my staff would welcome any specific recommendations and direct

assistance from you as we proceed to formulate water security program goals and appropriate

performance measures.

With respect to your second issue/concern on obtaining and analyzing data to develop a

baseline for water security, I want to clarify the Office of Water’s position on examining

vulnerability assessments (VAs) submitted by some 9,000 community water systems as required

by the Bioterrorism Act of 2002. I personally identified to the few members of the Water

Protection Task Force who have been designated by me to have access to the VAs three specific

purposes for reviewing and analyzing them. The first is that all VAs should be reviewed to

determine both compliance with the statutory requirements for submission/ certification of

completion and necessary enforcement action. Second, a representative sample (based on the

total number of systems in each size of systems as stipulated in the statute) of VAs should be

examined for: a) compliance (and subsequent enforcement, if any) with the statutory requirement

that the VA address all applicable parts of a system (e.g., pipes, physical barriers, treatment, etc.)

and b) determination that drinking water systems used a “reasoned process” (e.g., a tool like

RAM-W) to evaluate their vulnerabilities. Third, aggregated data are to be compiled and

analyzed to assist in identifying tools that need to be developed, research priorities and

appropriate security enhancements. In addition, this aggregated information will help us develop

a baseline for water security, which is consistent with your suggestion as well as OW’s emphasis

on improving performance indicators and measures for all its programs. Besides using this

aggregated data to inform and guide our future actions, we intend to share this information

(mindful of the restrictions imposed by the Bioterrorism Act of 2002) with Congress and other

decision makers, e.g., the Department of Homeland Security.

I appreciate the opportunity to respond to your draft report. Should you have any

questions or need additional information, please contact Michael Mason, the Office of Water’s

liaison to OIG, on 564-0572.

Attachments
10

Attachment 1

ACCOMPLISHMENTS OF THE WATER PROTECTION TASK FORCE

UNDER EPA’S HOMELAND SECURITY STRATEGIC PLAN

7/24/03

EPA has and is continuing to support a number of activities to improve security of both drinking

water and wastewater utilities using approximately $90 million and $23 million appropriated in

Fiscal Years ‘02 and ‘03, respectively. Examples of these activities include: 

< Developed vulnerability assessment tools for both drinking water and wastewater utilities and

supported extensive training for thousands of utility operators. 

< Provided more than $50 million in grants to more than 400 of the Nation’s largest drinking

water systems to undertake vulnerability assessments and do related security planning. 

Visited 30 of the largest cities to discuss water security. Received 463 of the 466

vulnerability assessments required to be submitted to EPA in March 2003. 

< Provided more than $24 million to the States and non-profit organizations to provide training

and technical assistance to small and medium water utilities on vulnerability assessments. 

< Developed and distributed Baseline Threat Information for Vulnerability Assessments of

Community Water Systems. 

< Met statutory deadline to implement protocol to protect vulnerability assessments. 

< Supported establishment of a state-of-the-art, secure information sharing system (the

WaterISAC) to share up-to-date threat and incident information between the intelligence

community and the water sector. Provided several water-specific advisories. 

< Developed guidelines on what actions utilities should take under DHS-specified threat levels. 

< Developed and began testing/distribution of Riverspill and Pipeline Net models to determine

fate and transport of contaminants in both source water and drinking water systems. 

< Supported establishment of a water security emphasis for the Environmental Technology

Verification Program and the WATERS test site to evaluate water security technologies. 

< Collaborated with ORD in development of the Water Security Research and Technology

Development Action Plan, currently under Review by the National Academy of Science. 

< Developed initial guidance on emergency response notification protocols. Currently

developing detailed guidance on revising emergency response plans to meet Bioterrorism Act

requirements, as well as a protocol to respond to a drinking water contamination event. 

< More information on EPA’s Water Protection Task Force water security program can be

found at: www.epa.gov/safewater/security
11

Attachment 2

Annual Performance Goals and Measures for Critical Water Infrastructure Protection 

 (Drinking Water and Wastewater Utilities)

Drinking Water 

FY 03:

Annual Performance Goal

Enhance public health protection by securing the Nation’s critical water infrastructure through

support for counter-terrorism preparedness.

Percent of the population and the number of community water systems - - serving 100,000 or

more people - - that have certified the completion of their vulnerability assessment and

submitted a copy to EPA.

Percent of the population and the number of community water systems - - serving 100,000 or

more people - - that have certified the completion of the preparation or revision of their

emergency response plan.. 

FY 04: 

Annual Performance Goal

Enhance public health protection by securing the Nation’s critical water infrastructure through

support for counter-terrorism preparedness.

Annual Performance Measures

Percent of the population and the number of community water systems - - serving more than

50,000 but less than 100,000 people - - that have certified the completion of their vulnerability

assessment and submitted a copy to EPA. 

Percent of the population and the number of community water systems - - serving more than

50,000 but less than 100,000 people - - that have certified the completion of the preparation or

revision of their emergency response plan.

Percent of the population and the number of community water systems - - serving more than

3,300 but less than 50,000 people - - that have certified the completion of their vulnerability

assessment and submitted a copy to EPA. 
12

Annual Performance Goals and Measures for Critical Water Infrastructure Protection 

Wastewater

FY 03:

Annual Performance Goal

Enhance public health and environmental protection by securing the Nation’s critical water

infrastructure through support for counter-terrorism preparedness including system operator

training.

Annual Performance Measure

Percent of the population and the number of large and medium size (10,001 and larger) of

Publicly Owned Treatment Works (POTWs) that have been taken for homeland security

preparedness.

FY 03:

Annual Performance Goal

Enhance public health and environmental protection by securing the Nation’s critical water

infrastructure through support for counter-terrorism preparedness including system operator

training..

Annual Performance Measure

Percent of the population and the number of large and medium size (10,001 and larger) of

Publicly Owned Treatment Works (POTWs) that have been taken for homeland security

preparedness.
13

Appendix B

Distribution

Acting Administrator

Associate Administrator for Congressional and Intergovernmental Relations

Acting Associate Administrator, Office of Public Affairs

Assistant Administrator, Office of Water

Audit Followup Coordinator, Office of Water

Director, Office of Ground Water and Drinking Water

Chair, Water Protection Task Force

General Counsel

Director, Office of Homeland Security