U.S. ENVIRONMENTAL PROTECTION AGENCY

OFFICE OF INSPECTOR GENERAL

The EPA Should Assess the 

Utility of the Watch List as a 

Management Tool 

Report No. 13-P-0435 September 30, 2013 

Scan this mobile 

code to learn more 

about the EPA OIG. 
 

Report Contributors: Dan Engelberg 

 Khadija Walker

 Tiffine Johnson-Davis 

 Genevieve Soule 

Andre von Hoyer 

Abbreviations 

ECHO Enforcement and Compliance History Online 

EPA U.S. Environmental Protection Agency 

OECA Office of Enforcement and Compliance Assurance 

OIG Office of the Inspector General 

Hotline 

To report fraud, waste, or abuse, contact us through one of the following methods: 

email: OIG_Hotline@epa.gov write: EPA Inspector General Hotline 

phone: 

fax: 

1-888-546-8740 

202-566-2599 

1200 Pennsylvania Avenue, NW 

Mailcode 2431T 

online: http://www.epa.gov/oig/hotline.htm Washington, DC 20460 
 

 

U.S. Environmental Protection Agency 13-P-0435 

September 30, 2013 Office of Inspector General 

At a Glance
 

Why We Did This Review 

We sought to determine 

how the U.S. Environmental 

Protection Agency (EPA) uses 

the Watch List as a tool to 

target enforcement actions and 

whether there are opportunities 

for improvement. 

Used by the EPA since 2004, 

the Watch List is designed to 

provide EPA regions and states 

with a list of facilities that are in 

significant violation of 

environmental laws and that 

appear not to have been 

addressed by timely and 

appropriate enforcement. 

The Watch List initially was 

just used as an in-house 

management tool, but in 2011 

the EPA made its data 

available to the public. 

This report addresses 

the following EPA theme: 

 Embracing EPA as a high 

performing organization. 

For further information, 

contact our public affairs office 

at (202) 566-2391. 

The full report is at: 

www.epa.gov/oig/reports/2013/ 

20130930-13-P-0435.pdf 

The EPA Should Assess the Utility of the 

Watch List as a Management Tool 

What We Found 

According to the EPA, the Watch List is a management tool used to identify 

long-standing significant violations. However, through interviews, regional 

questionnaires, and data analysis, we found that: 

 Use of the Watch List differs among headquarters and regional enforcement 

programs. 

 Trends in the number of facilities on the Watch List differ among 

enforcement programs. 

 EPA staff suggest multiple opportunities for improvement in the Watch List. 

 The public version of the Watch List has limited search capabilities and 

information. 

Without a proper assessment of the Watch List, the agency runs the risk of 

maintaining a management tool that does not assist in tracking facilities with longstanding significant violations, has limited transparency and utility to the public, 

and does not meet the needs of EPA users. 

Recommendations and Planned Agency Corrective Actions 

We recommend that the EPA Office of Enforcement and Compliance Assurance 

assess the Watch List’s utility as a management tool. If the agency determines 

that the tool is useful, it should: 

 Ensure that Watch List criteria are consistent with relevant enforcement 

response policies, and reassess criteria to determine relevance. 

 Develop an approach for identifying and/or removing facilities on the list that 

have been referred elsewhere or are under a consent decree. 

 Improve transparency of the publicly available Watch List to allow users to 

search and receive information similar to what is available internally. 

 Identify and implement other improvements to the Watch List identified in 

the EPA’s assessment. 

The agency agreed with the recommendation to assess the Watch List by 

September 30, 2014. If the agency decides to retain the list, it will act on the 

remainder of the recommendations by December 30, 2014. The 

recommendations are resolved with corrective actions pending. 
 Cynthia Giles, Assistant Administrator 

Office of Enforcement and Compliance Assurance 

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY 

WASHINGTON, D.C. 20460 

THE INSPECTOR GENERAL 

September 30, 2013 

MEMORANDUM 

SUBJECT:	 The EPA Should Assess the Utility of the Watch List as a Management Tool 

 Report No. 13-P-0435 

FROM:	 Arthur A. Elkins Jr. 

TO: 

This is a report on the subject evaluation conducted by the Office of Inspector General (OIG) of the 

U.S. Environmental Protection Agency (EPA). This report describes issues the OIG identified and 

makes recommendations to address those issues. This report represents the opinion of the OIG and does 

not necessarily represent the final EPA position. Final determinations on matters in this report will be 

made by EPA managers in accordance with established audit resolution procedures. 

Action Required 

The report recommendations are resolved with corrective actions pending. Therefore, the agency is not 

required to provide a final response to this report. However, if you choose to provide a final response, 

we will post your response on the OIG’s public website, along with our memorandum commenting on 

your response. You should provide your response as an Adobe PDF file that complies with the 

accessibility requirements of Section 508 of the Rehabilitation Act of 1973, as amended. 

We will post this report to our website at http://www.epa.gov/oig. 

If you or your staff have any questions regarding this report, please contact Assistant Inspector General 

for Program Evaluation Carolyn Copper at (202) 566-0829 or copper.carolyn@epa.gov, or Director for 

Water Issues Dan Engelberg at (202) 566-0830 or engelberg.dan@epa.gov. 
 

 

 

Purpose 

The purpose of this review was to determine how the U.S. Environmental 

Protection Agency (EPA) uses the Watch List as a tool to target enforcement 

actions and whether there are opportunities for improvement. 

Background 

In 2003, the assistant administrator for the Office of Enforcement and Compliance 

Assurance (OECA) requested that a tool be developed that would allow senior 

enforcement officials to track facilities that were in noncompliance with regulations 

(specifically, the Clean Air Act, Clean Water Act, and Resource Conservation and 

Recovery Act) without timely and appropriate enforcement actions. This request 

resulted in the implementation of the Watch List in 2004. The Watch List, one of 

several management tools used by the EPA enforcement programs, was designed to 

provide OECA and the EPA’s regions, as well as states, with a list of facilities that 

data indicate are in significant violation of these environmental laws and have not 

been addressed by timely and appropriate enforcement. The system identifies 

facilities as high-priority violators under the Clean Air Act or significant 

noncompliers under the Clean Water Act’s National Pollutant Discharge 

Elimination System and the Resource Conservation and Recovery Act. 

The Watch List contains a subset of violations identified under EPA enforcement 

response policies. The policies define when a violation of environmental laws and 

regulations becomes significant. This automated management tool is generated 

using compliance information from the EPA’s data systems, including the Air 

Facility System, the Integrated Compliance Information System–National 

Pollutant Discharge Elimination System (previously the Permit Compliance 

System), and the Resource Conservation and Recovery Act Information System 

(RCRAInfo). In 2011, in response to a Freedom of Information Act request, the 

EPA made a modified version of the Watch List available to the public on its 

Enforcement and Compliance History Online (ECHO) website; the Watch List 

had previously only been used as an in-house management tool. 

Scope and Methodology 

We conducted this evaluation in accordance with generally accepted government 

auditing standards. Those standards require that we plan and perform our work to 

obtain sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that the 

evidence obtained provides a reasonable basis for our findings and conclusions 

based on our evaluation objectives. We conducted our evaluation from February 

through August 2013. 

To answer our objective, we identified the laws, regulations, policies, procedures 

and guidance used by the EPA for enforcement. We also reviewed documents 

13-P-0435 1 
 

 

related to the EPA’s roles and responsibilities, oversight/management 

requirements, performance tracking and goals for the Watch List. We interviewed 

the EPA program and enforcement staff about data reliability. We interviewed 

agency officials and sent a questionnaire to staff in all 10 EPA regions to 

determine their use of the Watch List and the areas for which it could be 

improved. We also conducted an analysis of Watch List data to identify potential 

trends and anomalies for further discussion with OECA and the regions and 

comparison with agency information. 

Results of Review 

According to OECA, the Watch List, which the agency has used for almost 

10 years, is a management tool to identify long-standing significant 

noncompliance. We found the use of the Watch List differs among headquarters 

and regional enforcement programs. Further, we observed that trends in the number 

of facilities on the Watch List differ among enforcement programs. Additionally, 

staff suggested multiple opportunities for improvement. We also found that the 

public version of the Watch List has restricted capabilities. Without a proper 

assessment of the Watch List, the agency runs the risk of maintaining a 

management tool that does not assist in tracking facilities without timely and 

appropriate enforcement actions. In its current state, the list also has limited 

transparency to the public and may not meet the needs of its users, potentially 

limiting its utility for public users. 

Enforcement Programs Utilize the Watch List Differently 

Information we gathered from interviews with headquarters and questionnaires 

sent to all 10 regions showed that some enforcement staff in regional program 

offices use the Watch List to assist in their daily work (e.g., tracking facilities to 

determine whether they have returned to compliance and/or proceeded to 

follow-up with the state), while others only use it as a part of quarterly meetings 

with states and/or EPA headquarters to update the data on the list. Interviews and 

questionnaires indicated that the following factors might have affected the level of 

utilization of the Watch List: 

	 Seven of 10 regions reported the availability of duplicative/alternate data 

sources. 

	 Four of 10 regions and two headquarters program offices reported 

facilities remaining on the Watch List that have been referred to other 

offices/programs (such as the Department of Justice or the Superfund 

program) or are under a consent decree. 

	 Three of 10 regions and one headquarters program office reported 

inaccurate reporting and timeliness problems with the data. 

	 OECA headquarters and one region reported the Watch List not keeping 

up with technological advancements or changes in criteria definitions 

since it was created nearly a decade ago. 

13-P-0435 2 
 

 

 

Enforcement Programs Have Differing Trends in Watch List Data 

If the Watch List was an effective and key management tool for addressing 

facilities in longstanding noncompliance, we would expect to see some decreasing 

trends in the number of facilities on the list. However, our review of national 

Watch List data, pertaining to three acts (the Clean Air Act, Clean Water Act, and 

Resource Conservation and Recovery Act), from headquarters programs and all 

10 regions, showed different trends in the number of facilities on the Watch List 

from 2004 to 2013. Some regions had decreasing numbers of facilities on the 

Watch List in certain programs, others had increasing numbers, and others 

showed no obvious pattern of increase or decrease. The information gathered 

through interviews with OECA and questionnaires sent to the regions cited 

several reasons for the differences. They included states not reporting in a timely 

manner or having data entry issues, interface issues resulting from the switch 

from the Permit Compliance System to the Integrated Compliance Information 

System–National Pollutant Discharge Elimination System, and states or the 

agency not updating status codes timely and appropriately. Since most of the 

reasons cited were data quality issues, it is possible that, absent these issues, the 

trends would look different. Therefore, we are not able to draw a conclusion from 

the data about whether the Watch List has assisted EPA in addressing long-term 

noncompliance. 

Headquarters and Region Staff Identified Opportunities for 

Improvement 

Through interviews and regional questionnaires, Watch List users identified 

opportunities for improvement, including the following: 

 Incorporate a Watch List status in the ECHO facility search outputs 

(e.g., a “Watch List” column). 

 Create an efficient process to identify and/or remove facilities that are on 

the list in error or that the agency has referred to other programs or have 

been placed under a consent decree. 

 Ensure that criteria on the Watch List match the criteria in the relevant 

enforcement response policy. 

 Finalize Clean Water Act criteria. 

 Add the Safe Drinking Water Act to the Watch List 

 Allow OECA and regions to create region-specific “Watch Lists” based on 

mutually determined criteria while continuing to identify violations that 

meet overall Watch List criteria. 

 Initiate Web-based quarterly reporting to accept status codes and 

explanations from the regions. 

 Make the data on Watch List real time. 

 Enhance the Watch List standard operating procedures. 

 Add additional status codes that better describe the status of the current 

situation of a facility (make codes more specific). 

13-P-0435 3 
 

 

Publicly Available Watch List Lacks Many of the Features and 

Information Available Internally 

The Watch List capabilities on the public website (ECHO) are very different from 

those of the internal Watch List. The public site does not provide users with the 

ability to search the Watch List to return only information that they are interested 

in. In addition, the information provided is in a “PDF” format and therefore 

cannot be easily sorted or analyzed by users, decreasing the utility of the list. 

The public Watch List provides limited information – only facility name, ID and 

location. The internal Watch List tool (available through the Online Tracking 

Information System) provides a substantial amount of additional enforcement 

information that allows the user to put the Watch List designation into context 

with the facility’s enforcement history. Given the agency’s long-stated goal of 

transparency, the information available to the public should contain all 

information that is not deemed enforcement sensitive. 

Conclusion 

Given the initial reasons for the Watch List implementation, its current differing 

use among regions, and differing data trends, the EPA should determine whether 

the Watch List is still a useful management tool. Our review identified 

opportunities for improvement, including updating and reviewing its approach for 

listing facilities to ensure transparency and accuracy of information as well as the 

public utility of the information. OECA is currently modernizing its enforcement 

data system, which is scheduled to be completed by the end of fiscal year 2013. 

We believe this is an opportune time for OECA to make changes to the Watch 

List should that be its decision. 

Recommendation 

We recommend that the assistant administrator, Office of Enforcement and 

Compliance Assurance: 

1.	 Assess the Watch List’s utility as a management tool in assisting the 

agency in monitoring long-standing alleged significant violators. If the 

agency determines that the tool is useful, it should: 

a.	 Ensure that Watch List criteria are consistent with relevant 

enforcement response policies and reassess the criteria to 

determine relevance. 

b.	 Develop an approach for identifying and/or removing facilities on 

the list that have been referred to other offices/programs or are 

under a consent decree. 

13-P-0435 4 
 

 

c.	 Improve transparency of the publicly available Watch List to allow 

users to query and receive information similar to what is available 

through internal data systems. 

d.	 Identify and implement other improvements to the Watch List 

identified in the EPA’s assessment. 

Agency Comments and OIG Evaluation 

The agency agreed with the recommendation to assess the Watch List by 

September 30, 2014. OECA intends to assess the overall appropriateness and 

utility of the Watch List as a multi-media tool as it revises some of the mediaspecific compliance and enforcement policies. In follow-up communications with 

OECA staff, we were told that if OECA decides to retain the Watch List, it will 

implement the remainder of the recommendations by December 30, 2014. The 

recommendations are resolved with corrective actions pending. 

13-P-0435 5 
 

 

 

 

 

 

 

 

 

 

 

Status of Recommendations and 

Potential Monetary Benefits 

POTENTIAL MONETARY 

RECOMMENDATIONS BENEFITS (in $000s) 

Rec. 

No. 

Page 

No. Subject Status1 Action Official 

Planned 

Completion 

Date 

Claimed 

Amount 

Agreed-To 

Amount 

1 4 Assess the Watch List’s utility as a management 

tool in assisting the agency in monitoring longstanding alleged significant violators. If the agency 

determines that the tool is useful, it should: 

a. Ensure that Watch List criteria are consistent 

with relevant enforcement response policies 

and reassess the criteria to determine 

relevance. 

b. Develop an approach for identifying and/or 

removing facilities on the list that have been 

referred to other offices/programs or are 

under a consent decree. 

c. Improve transparency of the publicly 

available Watch List to allow users to query 

and receive information similar to what is 

available through internal data systems. 

d. Identify and implement other improvements 

to the Watch List identified in the EPA’s 

assessment. 

O 

O 

O 

O 

O 

Assistant Administrator, 

Office of Enforcement and 

Compliance Assurance 

9/30/14 

12/30/14 

12/30/14 

12/30/14 

12/30/14 

O = recommendation is open with agreed-to corrective actions pending 

C = recommendation is closed with all agreed-to actions completed 

U = recommendation is unresolved with resolution efforts in progress 

13-P-0435 6 

1 
 

 

 

 

 

 

 

 

 

 

 

Appendix A 

Agency Response 

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY 

WASHINGTON, D.C. 20460

 OFFICE OF

 ENFORCEMENT AND 

COMPLIANCE ASSURANCE 

July 25, 2013 

MEMORANDUM 

SUBJECT:	 Comments on “Quick Reaction Report: The EPA Should Assess the Utility of the 

Watch List as a Management Tool” 

FROM:	 Cynthia Giles, Assistant Administrator 

TO:	 Carolyn Copper, Assistant Inspector General 

Office of Program Evaluation 

Thank you for the opportunity to review the Office of Inspector General (OIG) “Quick Reaction 

Report: The EPA Should Assess the Utility of the Watch List as a Management Tool”. We agree 

that it is an opportune time to discuss the utility of the Watch List as a management tool in 

assisting the Agency in monitoring long-term alleged significant violators. 

We are currently in the process of revising some of the media-specific compliance and enforcement 

policies. As part of this process, we can better assess the overall appropriateness and utility of the 

Watch List as a multi-media management tool, As we are conducting this assessment, we will keep 

in mind the findings and initial recommendations of the OIG and commit to the following corrective 

action: 

Recommendation Lead Office CA Target Date Corrective Action 

Assess the Watch List’s 

Utility as a management 

tool. 

OC/OCE 1 9/30/2014 Evaluate the utility of the 

 the Watch List as a 

 Management tool, and 

identify appropriate next 

steps. 

If you have any questions regarding this response, please contact Gwendolyn Spriggs, the OECA 

Audit Liaison, at 202-564-2439. 

13-P-0435 7 
 

 

 

 

 

cc: Lisa Lund 

 Susan Shinkman

 Mamie Miller

 Lauren Kabler

 Gwendolyn Spriggs 

13-P-0435 8 
 

 

Appendix B 

Distribution 

Office of the Administrator 

Assistant Administrator for Enforcement and Compliance Assurance 

Agency Follow-Up Official (the CFO) 

Agency Follow-Up Coordinator 

General Counsel 

Associate Administrator for Congressional and Intergovernmental Relations 

Associate Administrator for External Affairs and Environmental Education 

Principal Deputy Assistant Administrator for Enforcement and Compliance Assurance 

Audit Follow-Up Coordinator, Office of Enforcement and Compliance Assurance 

Audit Follow-Up Coordinator, Office of Solid Waste and Emergency Response 

Audit Follow-Up Coordinator, Office of Water 

Audit Follow-Up Coordinator, Office of Air and Radiation 

13-P-0435 9