U.S. ENVIRONMENTAL PROTECTION AGENCY

OFFICE OF INSPECTOR GENERAL

Efficiency of EPA’s 

Rule Development Process 

Can Be Better Measured 

Through Improved 

Management and Information 

Report No. 13-P-0167 February 28, 2013 

Scan this code to 

learn more about 

the EPA OIG. 
 

 

Report Contributors: Laurie Adams

 Daniel Carroll 

 Carolyn Copper 

 Jerri Dorsey

 Gabby Fekete 

 Jeffrey Harris 

 Olga Stein 

Abbreviations 

ADP Action Development Process 

EPA U.S. Environmental Protection Agency 

OIG Office of Inspector General 

OMB U.S. Office of Management and Budget 

OP Office of Policy 

RAPIDS Rule and Policy Information and Development System 

Hotline 

To report fraud, waste, or abuse, contact us through one of the following methods: 

e-mail: OIG_Hotline@epa.gov write: EPA Inspector General Hotline 

phone: 

fax: 

1-888-546-8740 

202-566-2599 

1200 Pennsylvania Avenue, NW 

Mailcode 2431T 

online: http://www.epa.gov/oig/hotline.htm Washington, DC 20460 
 

 

U.S. Environmental Protection Agency 13-P-0167 

February 28, 2013 Office of Inspector General 

At a Glance
 

Why We Did This Review 

The purpose of this review was 

to evaluate whether the U.S. 

Environmental Protection 

Agency’s (EPA’s) Action 

Development Process (ADP) 

results in the timeliest, most 

efficient, and most effective 

method for rule development. 

This review was requested by 

the Agency. We focused on the 

key aspects of efficiency with 

which program offices 

implement the ADP’s guidance 

for rule development. EPA’s 

Office of Policy coordinates the 

ADP. 

This report addresses the 

following EPA Goals or 

Cross-Cutting Strategies: 

 Taking action on climate 

change and improving 

air quality 

 Protecting America’s 

waters 

 Cleaning up communities 

and advancing sustainable 

development 

 Ensuring the safety of 

chemicals and preventing 

pollution 

For further information, contact 

our Office of Congressional and 

Public Affairs at (202) 566-2391. 

The full report is at: 

www.epa.gov/oig/reports/2013/ 

20130228-13-P-0167.pdf 

Efficiency of EPA’s Rule Development Process 

Can Be Better Measured Through Improved 

Management and Information 

What We Found 

Rule development is one of the Agency’s principal tasks. EPA develops rules to 

carry out the environmental and public health protection laws passed by 

Congress. Efficient EPA rulemaking may accelerate the progress of protecting 

human health and the environment. However, due to limitations in EPA 

rulemaking documentation and guidance, the Agency is unable to evaluate the 

efficiency of the rulemaking process or identify potential delays in its rulemaking 

activities. For example, EPA has limited information on the time and resources 

used to complete the various stages of the rule development process. 

The development and implementation of management controls to ensure that the 

rulemaking process is progressing efficiently and that resources are accurately 

accounted for will enhance EPA’s ability to assure efficiency during the 

development process and accelerate the progress of protecting human health 

and the environment.

 Recommendations and Planned Agency Corrective Actions 

We recommend that the Associate Administrator for EPA’s Office of Policy 

establish guidance, maintain database documentation, and track resources, to 

enhance the Agency’s ability to determine the efficiency of the rulemaking 

process. The Associate Administrator concurred with the first and second 

recommendations but disagreed with the third recommendation. We consider the 

third recommendation unresolved. As required by EPA Manual 2750, the Agency 

will need to initiate the audit resolution process. 
 

 

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY 

WASHINGTON, D.C. 20460 

THE INSPECTOR GENERAL 

February 28, 2013 

MEMORANDUM 

SUBJECT:	 Efficiency of EPA’s Rule Development Process Can Be Better Measured 

Through Improved Management and Information 

Report No. 13-P-0167 

FROM:	 Arthur A. Elkins Jr. 

TO:	 Michael Goo, Associate Administrator 

Office of Policy 

This is a report on the subject evaluation conducted by the Office of Inspector General (OIG) of 

the U.S. Environmental Protection Agency (EPA). This report contains findings that describe the 

problems the OIG has identified and corrective actions the OIG recommends. This report 

represents the opinion of the OIG and does not necessarily represent the final EPA position. 

Final determinations on matters in this report will be made by EPA managers in accordance with 

established audit resolution procedures. 

Action Required 

In accordance with EPA Manual 2750, you are required to provide a written response to this 

report within 60 calendar days. You are not required to provide a written response to the first 

and second recommendations because you provided agreed-to corrective actions and planned 

completion dates. The OIG may make periodic inquiries on your progress in implementing 

these corrective actions. The OIG and Agency have not agreed on a course of action to address 

the third recommendation. We consider that recommendation unresolved. As required by 

EPA Manual 2750, the Agency will need to initiate the audit resolution process. 

Should you choose to provide a response, we will post your response on the OIG’s public 

website, along with our memorandum commenting on your response. You should provide your 

response as an Adobe PDF file that complies with the accessibility requirements of Section 508 

of the Rehabilitation Act of 1973, as amended. The final response should not contain data that 
you do not want to be released to the public; if your response contains such data, you should 

identify the data for redaction or removal along with corresponding justification. We have no 

objections to the further release of this report to the public. We will post this report to our 

website at http://www.epa.gov/oig. 

If you or your staff have any questions regarding this report, please contact Carolyn Copper, 

Assistant Inspector General for Program Evaluation, at (202) 566-0829 or 

copper.carolyn@epa.gov; or Jerri Dorsey at (919) 541-3601 or dorsey.jerri@epa.gov. 
 

 

 

 

 

Efficiency of EPA’s Rule Development Process Can Be 13-P-0167 

Better Measured Through Improved Management and Information 

Table of Contents 

Chapters 

1 Introduction ........................................................................................................ 1

 Purpose ....................................................................................................... 1 

Background ................................................................................................. 1 

 Scope and Methodology .............................................................................. 3 

2 Additional Controls Needed to Measure Efficiency ........................................ 5
 

Enhanced Tracking and Document Retention Needed ............................... 5 

Rulemaking Costs Are Not Tracked ............................................................ 6 

Conclusion................................................................................................... 8 

 Recommendations ...................................................................................... 8 

Agency Comments and OIG Evaluation ...................................................... 8
 

Status of Recommendations and Potential Monetary Benefits .............................. 10
 

Appendices 

A Completed Rules Sampled and Reviewed by OIG .......................................... 11 

B Agency Response and OIG Comments............................................................ 12 

C Distribution ......................................................................................................... 18 
 

 

 

 

 

Chapter 1

Introduction 

Purpose 

The purpose of this review was to evaluate whether the U.S. Environmental 

Protection Agency’s (EPA’s) Action Development Process (ADP) ensures the 

timeliest, most efficient, and most effective method for rule development. 

Specifically, we focused on key aspects of the efficiency1 with which the program 

offices implement the Agency’s ADP guidance. The initial impetus for this 

review was a request by the Agency to improve the process.2 

Background 

EPA protects public health and the environment in a variety of ways, but one of 

the most important ways is through regulation development. EPA regulations 

cover a range of environmental and public health protection issues, from setting 

standards for clean water and establishing requirements for proper handling and 

reductions of toxic wastes to controlling air pollution from industry and other 

sources. According to EPA, it is one of the most active regulatory agencies in the 

federal government. Each year, the Agency develops a considerable number of 

rules which address highly technical, scientific, and complex environmental 

problems. On average, EPA issues over 122 Administrator-signed regulations 

each year.3

 Between 2005 and 2010, EPA published 735 major or Administratorsigned rules, of which approximately 40 percent were final rules.4 

According to the Agency, much of its environmental success and organizational 

credibility is directly linked to the quality of its work under the ADP. EPA 

designed the ADP to develop rules based on sound scientific, economic, legal, 

and policy analyses. The ADP is designed to be a multi-disciplinary, 

collaborative, cross-office, and cross-media approach to rule development and 

accelerate the progress of protecting human health and the environment. 

1

 For the purposes of this review, “efficiency” is defined as the resources used to achieve program results. This is the 

U.S. Government Accountability Office’s Government Auditing Standards definition. 2

 OIG management received a suggestion to review the Agency’s rule development process in an effort to identify 

areas of redundancy, waste, and inefficiency. 3

 EPA’s Administrator-signed rules are published actions which modify or propose to modify the Code of Federal 

Regulations and, therefore, contain rule text. The Code of Federal Regulations is the official record of all regulations 

created by the federal government. 

4 A final rule is the conclusion of the Agency’s regulatory development. A final rule represents a range of 

documents that are published in the Rules section of the Federal Register. These include final rules that amend the 

Federal Register by adding new text or by revising or removing existing text. These also include other types of final 

actions or documents that have no regulatory text and do not amend the Federal Register but affect the Agency, such 

as general policy statements, final guidance documents, or petitions for reconsideration. 

13-P-0167 1 
 

 

 

 

The ADP serves as a framework to ensure that rules are developed using quality 

information and all scientific, economic, and policy issues are addressed during 

the appropriate rule development stages. The ADP also provides opportunities for 

senior management’s early involvement and provides guidance to rulemaking 

staff at key points of the rule’s development. EPA’s Office of Policy (OP) leads 

the regulatory development process via its Office of Regulatory Policy and 

Management, which provides support and guidance to EPA’s program and 

regional offices as they develop regulations. EPA’s OP coordinates the ADP and 

also serves as liaison to other agencies involved in the rule development, such as 

the Office of Management and Budget (OMB). 

There are five major stages of the ADP: 

1.	 Tiering the action and obtaining commencement approval. 

2.	 Developing the proposed rule or draft action. 

3.	 Requesting OMB review (if needed) for proposed and final actions. 

4.	 Requesting signatures, publishing an action in the Federal Register, and 

soliciting and accepting public comment. 

5.	 Developing the final action and ensuring congressional review. 

Figure 1 illustrates the key steps related to the five stages of the ADP. This figure 

represents the development process for all Tier 1 and 2 actions. 

Figure 1: Flowchart of the Action Development Process in EPA 

Source: EPA’s Action Development Process training presentation. 

Note: Step 7 of the illustration involves the Agency’s necessary analyses and an assessment of all 

applicable statutes and executive orders, of which there are many. 

AA/RA: Assistant Administrator/Regional Administrator 

EO: Executive Order 

13-P-0167 2 
 

 

 

 

 

Scope and Methodology 

We conducted our work from August 2011 through August 2012 in accordance 

with generally accepted government auditing standards. Those standards require 

that we plan and perform our work to obtain sufficient, appropriate evidence to 

provide a reasonable basis for our findings and conclusions based on our 

objectives. We believe that the evidence obtained provides a reasonable basis for 

our findings and conclusions based on our objectives. 

Our review focused on the key aspects of efficiency with which program offices 

implement the ADP’s guidance for rule development. Efficiency can refer to not 

only the timeliness in producing rules but the management of the resources used 

in the rule development process. In an effort to evaluate the efficiency of the 

rulemaking process, we identified rules that had final regulatory actions in 2010 

and 2011. We reviewed these final rules to determine the adequacy with which 

the program offices’ staff complied with preparing documents that evidenced the 

“products of a quality action.”5

 We reviewed information in the Rule and Policy 

Information and Development System (RAPIDS), EPA’s database to manage the 

creating, tracking, approving, and reporting of the Agency’s rulemaking actions. 6 

We also interviewed EPA staff to determine how the Agency tracks the resource 

cost for rule development activities. 

During field work, we took an initial sample of four completed rulemaking 

actions to determine the feasibility of conducting a larger sample of completed 

rules.7 We attempted to document progress for each rule throughout the entire 

ADP process using a data collection instrument developed by the EPA Office of 

Inspector General (OIG). We sought to obtain key documentation and dates both 

from RAPIDS and the workgroup chairs. We were eventually able to obtain some 

documentation for each rule sampled; however, the documentation was not 

always the final signed product. This impeded our ability to track the progress of 

the rules though the ADP process in an effort to identify areas to increase 

efficiency. After completing the limited sample work, we discovered fundamental 

gaps in the ADP tracking process that made further data collection unnecessary. 

These gaps are discussed in detail in chapter 2 of this report. 

5

 “Products of a quality action” are defined by EPA’s ADP guidance document as those actions that include the 

following elements: Tiering Form; Preliminary Analytic Blueprint; Early Guidance Briefing Package; Early 

Guidance Memorandum; Detailed Analytic Blueprint; Options Selection Briefing Package; Draft Action (e.g., rule 

text, report to Congress, policy or guidance); Action Memorandum; and Communications Plan, coordinated with the 

Office of Public Affairs. 

6

 During our field work, RAPIDS was the database used at the time to manage all ADP activities and the one to 

which we were directed for our evaluation’s data collection activities. 

7

 See appendix A for details on these four actions. 

13-P-0167 3 
 

 

 

Prior Reports 

The U.S. Government Accountability Office reviewed federal rulemaking in two 

recent reports.8

 One of the reports concluded that agencies had little data on the 

time and resources used to comply with regulatory requirements, making it 

difficult to evaluate the effects of requirements on rulemaking. In the other report, 

EPA was cited for not following key steps in its guidance before finalizing a 

major rule. There were no previous EPA OIG reviews of this subject matter. 

8

 GAO-08-128: EPA Actions Could Reduce Environmental Information Available to Many Communities, November 

2007; GAO-09-205: Improvements Needed to Monitoring and Evaluation of Rules Development as Well as to the 

Transparency of OMB Regulatory Reviews, April 2009. 

13-P-0167 4 
 

 

 

 

 

Chapter 2

Additional Controls Needed to Measure Efficiency 

EPA established the ADP to ensure the quality of actions developed, but ADP 

guidance does not contain adequate controls to assure that the process is 

completed in the most efficient manner. Inefficiencies in the rule development 

process may result in delays that could postpone new or revised rules and impede 

progress to protect human health and the environment. To determine the 

efficiency of the rulemaking process, the Agency needs adequately maintained 

documentation, defined milestone goals that measure and monitor rule 

development progress, and cost information. Adequate system controls and the 

development of a method to track resource use will provide EPA with the tools 

necessary to measure and assure the efficiency of the development process. 

Enhanced Tracking and Document Retention Needed 

We found that EPA’s database for creating and tracking rules was sparsely 

populated and did not contain the necessary documents or information to allow us 

to complete our review.9 We found program offices were not adequately utilizing 

the Agency’s database due to a lack of standardized procedures from the OP 

describing who was responsible for uploading the developmental documents to 

the databases used to manage the ADP process. The development documents 

called for in the ADP guidance represent tools for both the rulemaking workgroup 

as well as for management’s oversight responsibilities. The availability of these 

documents and associated milestone completion data would provide management 

with the information needed to oversee the rule development process and assure 

the rulemaking process progresses in a timely and efficient manner. 

In February 2012, EPA implemented a new database, ADP Tracker. However, 

there remain some limitations in regards to tracking and documentation which 

challenge the Agency’s ability to monitor, evaluate, and assure the efficiency of 

EPA rulemaking. According to the Agency, the new system will provide 

improved capability to track milestones, manage workgroups, and track work 

flow, as well as provide better security and access.10 According to OP, the Agency 

conducted nearly 20 training sessions for staff throughout the Agency on proper 

use of ADP Tracker. However, there are still no standardized procedures 

describing who is responsible for uploading the key developmental documents to 

the database used to manage the ADP process. OP, the office that manages the 

9 Because Agency staff had not populated RAPIDS with the milestone dates and other data on the ADP, efficiently 

determining any potential sources of delay in the rulemaking process for our sampled rules was not possible. 10 We were unable to access or conduct a full evaluation of the new system because it was not fully implemented 

during our evaluation field work. 

13-P-0167 5 
 

 

 

 

 

ADP, should provide additional instruction to the program offices involved in 

rulemaking regarding the roles and responsibilities for uploading the various 

documents into the new database. Furthermore, OP should assure that the 

program offices comply with the data entry requirements. This will provide OP 

with the ability to ensure data reliability and the rulemaking process progresses in 

a timely and efficient manner. 

Rulemaking Costs Are Not Tracked 

The Agency does not maintain cost data associated with action development and 

rulemaking. Rule development requires input from various offices in the Agency 

with an associated time commitment, and can take several years to complete. The 

lack of cost data means that EPA has limited information on the resources used to 

complete the ADP process. 

Rulemaking Activities Represent Time Commitment Across Agency 

EPA prepares and releases hundreds of actions each year that define the technical 

and operational details of environmental programs. These actions can be complex. 

As illustrated previously in Figure 1, the rulemaking process includes a number of 

steps, including internal and external comment and review. According to the 

Agency, the development time (i.e., time from initiation of a rulemaking to final 

rule stage) for publication of an Administrator-signed rule for the period 2005-

2010 was a total average time of 2 years and 6 months (930 days). Figure 2 

illustrates how rule development time varies by program office and by year.11 

11 The variation in the development time across the five main offices with published Administrator-signed rules is 

greatly influenced by a variety of internal and external factors. These factors include the complexity of the 

rulemaking, whether the rulemaking is under a court-schedule or legislative mandate, the interests of other program 

offices, and the extent of public involvement during the rule development process. 

13-P-0167 6 
 

 

 

Figure 2: Administrator-signed rules, development time 

Source: EPA’s RegStat website. 

OAR: Office of Air and Radiation 

OEI: Office of Environmental Information 

OCSPP: Office of Chemical Safety and Pollution Prevention 

OSWER: Office of Solid Waste and Emergency Response 

OW: Office of Water
 

Rule development requires input and time from offices across the Agency, neither 

of which are tracked for budgetary or management purposes. It should be the 

Agency’s goal to ensure that rulemaking is done in the most cost effective and 

efficient manner. Because EPA does not have input measures that track time spent 

on the development of individual rules, rulemaking cost information is not 

available to EPA management to determine efficiency or return on investment. 

OMB recommends using input measures (e.g., time, employee hours, funding, 

equipment) to determine the resources used to achieve an output (activity) or 

outcome (impact). When compared to the achieved outputs and outcomes, the 

information can be used to determine program efficiency and return on 

investment. 

Resource Data Can Enhance Budgeting and Forecasting of Resources 

EPA has limited data on the time and resources used to complete the various 

stages of the rule development process. By tracking both costs and milestones, 

management can monitor the rulemaking progress and better assess where delays 

are occurring and make the necessary corrections. Tracking and documenting 

costs would also provide management with historical data that could be used for 

13-P-0167 7 
 

 

budgeting and forecasting resource needs in future rulemaking work. For 

budgetary considerations, management can have better control on resource needs 

in the future based on the expected workload of new rules and how much of their 

budget is allocated to rulemaking activities. 

Conclusion 

Developing environmental regulations is one of EPA’s principal tasks. The ADP 

was designed to deliver actions that are based on sound science, promote 

economic efficiency, and accelerate the progress of protecting human health and 

the environment. EPA should take steps to track and document the actions of the 

rule development process. Additionally, the development of a method to track 

resource use in key rulemaking activities will provide the Agency with the tools 

necessary to assess the key aspects of efficiency in the rule development process. 

A commitment to assessing efficiency of the rulemaking process shows the 

Agency values accelerating the protection of human health and the environment. 

Recommendations 

We recommend that the Associate Administrator, Office of Policy: 

1.	 Establish guidance that clarifies roles and responsibilities in ADP 

implementation, including data entry, record keeping, and the status of 

action development. 

2.	 Ensure that ADP Tracker has established clear roles, responsibilities, and 

requirements for the program offices to upload development documents; 

and that entries are updated in a timely manner, are monitored for data 

quality, and all features available are used to evaluate the efficiency of the 

rule development process. 

3.	 Develop and implement a method to track resource use in key rulemaking 

activities. 

Agency Comments and OIG Evaluation 

The Associate Administrator concurred with the first and second 

recommendations. The Agency provided high-level corrective actions in response 

to these recommendations with an estimated milestone completion date of 

June 2013. These recommendations are considered open with agreed-to corrective 

actions pending. 

OP disagreed with the third recommendation because it does not have the expertise 

to develop a method to track resources. The OIG and Agency have not agreed on a 

course of action to address this recommendation. We consider this recommendation 

unresolved with actions underway as required by EPA Manual 2750. 

13-P-0167 8 
 

 

OP believes the report does not present an accurate picture of rulemaking in the 

Agency. Additionally, OP believes that clarification of the purpose of the report is 

critical to assessing what appropriate conclusions can be fairly reached regarding 

the ADP process. OP also questioned the emphasis on the RAPIDS database, 

which was replaced by ADP Tracker in February 2012. We made changes to the 

report as appropriate. The Agency’s complete response, along with the OIG’s 

evaluation, is in appendix B. 

13-P-0167 9 
 

 

 

 

 

 

 

 

 

 

 

Status of Recommendations and 

Potential Monetary Benefits 

POTENTIAL MONETARY 

RECOMMENDATIONS BENEFITS (in $000s) 

Rec. 

No. 

Page 

No. Subject Status1 Action Official 

Planned 

Completion 

Date 

Claimed 

Amount 

Agreed-To 

Amount 

1 

2 

3 

8 

8 

8 

Establish guidance that clarifies roles and 

responsibilities in ADP implementation, including 

data entry, record keeping, and the status of action 

development. 

Ensure that ADP Tracker has established clear 

roles, responsibilities, and requirements for the 

program offices to upload development documents; 

and that entries are updated in a timely manner, 

are monitored for data quality, and all features 

available are used to evaluate the efficiency of the 

rule development process. 

Develop and implement a method to track resource 

use in key rulemaking activities. 

O 

O 

U 

Associate Administrator, 

Office of Policy 

Associate Administrator, 

Office of Policy 

Associate Administrator, 

Office of Policy 

June 2013 

June 2013 

O = Recommendation is open with agreed-to corrective actions pending.
 

C = Recommendation is closed with all agreed-to actions completed.
 

U = Recommendation is unresolved with resolution efforts in progress.
 

13-P-0167 10 

1 
 

 

Appendix A 

Completed Rules Sampled and Reviewed by OIG 

Rule title Tier Program office 

Transport Rule (CAIR Replacement Rule) – Rule No. 2060-AP50 

(SAN: 5336) 

Tier 1 Office of Air and 

Radiation 

Federal Requirements under the Underground Injection Control 

(UIC) Program for Carbon Dioxide (CO2) Geologic Sequestration 

(GS) Wells – Rule No. 2040-AE98 (SAN: 5211) 

Tier 2 Office of Water 

Mercury; Significant New Use Rule for Elemental Mercury in Flow 

Meters, Manometers, and Pyrometers – Rule No. 2070-AJ36 (SAN: 

5238) 

Tier 3 Office of Chemical 

Safety and Pollution 

Prevention 

Protection of Stratospheric Ozone: Listing of Substitutes for OzoneDepleting Substances-Hydrocarbon Refrigerants – Rule No. 2060-

AP54 (SAN: 5339) 

Tier 3 Office of Air and 

Radiation 

Source: OIG analysis. 

13-P-0167 11 
 

 

 

 

 

 

 

Appendix B 

Agency Response and OIG Comments 

October 12, 2012 

MEMORANDUM 

SUBJECT: Response to Draft Report: Efficiency of EPA’s Rule Development Process 

Cannot Be Assessed, Project No. OPE-FY11-0021 

FROM: Michael L. Goo /s/ 

 Associate Administrator 

TO: Carolyn Copper 

Assistant Inspector General for Program Evaluation 

Office of Inspector General 

In accordance with U.S. Environmental Protection Agency (EPA) Manual 2750, I hereby submit 

the Office of Policy’s (OP) response to the findings and recommendations cited in the Office of 

Inspector General’s (OIG) draft evaluation report “Efficiency of EPA’s Rule Development 

Process Cannot Be Assessed, Project No. OPE-FY11-0021.” 

Thank you for providing the Office of Policy with an opportunity to share our feedback on the 

draft evaluation report. We believe the report seeks to examine important features of the EPA 

Action Development Process (ADP) and we are eager to engage with the Office of Inspector 

General in crafting an accurate and relevant report that fairly characterizes the complete ADP 

context. As currently drafted, we believe the report does not present an accurate picture. We 

believe that additional data bearing on this issue will be forthcoming in the near future and that 

such data will be extremely helpful in accurately describing the situation. We also believe that 

clarification of the purpose of the report is critical to assessing what appropriate conclusions can 

fairly be reached regarding the ADP process. 

OIG Response: As communicated to the Agency, the purpose of this review was to 

evaluate whether EPA’s ADP resulted in the timeliest, most efficient, and most effective 

method for rule development. This basis for our review was a suggestion by the Agency to 

look at the process and potential ways to increase efficiency. We focused on the efficiency 

with which program offices implemented the ADP guidance for rule development. 

13-P-0167 12 
 

 

Finally, we believe that the draft findings contemplated in the draft report relate largely to a 

database that is no longer in use by EPA and therefore such conclusions are of, at best, 

diminished relevance to the ongoing work of the Agency. In light of these concerns we would 

appreciate the opportunity to consult further with the Office of the Inspector General regarding 

this Draft Report prior to its finalization. 

OIG Response: The OIG addressed suggested corrections as appropriate. 

Nevertheless, OP agrees with the OIG’s conclusion that opportunities exist to expand the 

usefulness and applicability of the ADP Tracker and concurs with the intent of OIG’s second 

recommendation to “ensure that ADP Tracker has established clear roles, responsibilities, and 

requirements for the program offices to upload development documents; and that the rules are 

updated in a timely manner, entries are monitored for data quality, and all features available are 

used to evaluate the efficiency of the rule development process.” 

The Draft report states, under the heading “Why We Did This Review” that: 

The purpose of this review was to evaluate whether the U.S. Environmental Protection 

Agency’s Action Development Process results in the timeliest, most efficient and most 

effective method for rule development. The initial impetus for this review was a request 

by the Agency to look at the process and potential ways to increase efficiency. We 

focused on the efficiency with which program offices implement the ADP’s guidance for 

rule development. 

Except for the description above, the details of the “request by the Agency” remain largely 

unclear to the Office of Policy at present. Although the Office of Policy welcomes the 

opportunity to explore ways to improve the efficiency of EPA's rulemaking process, the 

language above seems to suggest a concept of “efficiency” in which the speed of ADP is the 

primary focus of the inquiry. Moreover the concept of “efficiency” as “speed” is then 

incorporated into the draft findings which state “the ADP Guidance does not contain controls to 

ensure that the process is completed in the most efficient manner. Inefficiencies can result in 

delays that postpone new or revised rules.” As a result, the report concludes that “The Efficiency 

of EPA’s Rule Development Process Cannot be Assessed” and that “[The] ADP Does Not 

Include Controls to Ensure Efficiency.” 

OIG Response: The OIG had multiple communications with the Agency regarding the 

basis for this evaluation and the details of the request. Our review was requested by 

Agency officials. At no time during our review did anyone within the Agency question or 

raise concerns with our scope. 

As the draft report also acknowledges, “the ADP serves as a framework to ensure that rules are 

developed using quality information and that all scientific, economic and policy issues are 

addressed during the appropriate rule development stages.” Given this purpose for the ADP, it 

should be evident that mere hastening of the rulemaking process, in such a way as to produce 

rulemaking products in the shortest amount of time, and using the fewest amount of Agency 

13-P-0167 13 
 

 

resources, may not ultimately be consistent with a more comprehensive and appropriate 

definition of “efficiency.” 

OIG Response: OIG’s report does not advocate for speed in the rulemaking process over 

effectiveness of the process. OIG’s report recommends that additional controls are needed 

to measure efficiency. 

As OP staff conveyed during our May 30, 2012 meeting and in several conference calls, the 

purpose of the ADP is to guide the process for developing a wide variety of rulemakings, public 

notices, and guidance documents. Each such action type can vary significantly with regard to 

Agency priority, level of input required from other EPA offices, need for inter-agency review, 

and for technical and/or scientific complexity. The goal is to produce quality actions that are 

timely, clear, scientifically sound, technically accurate, effective, and legally defensible. It would 

serve no public purpose to produce a rule quickly, using few resources, if it fails to withstand 

legal challenge and/or produce social benefits. 

OIG Response: The OIG report does not advocate that speed or any other factor is more 

or less important than efficiency. The report presents the aspects of efficiency that we were 

unable to assess due to limitations in the Agency’s data systems. 

Perhaps more importantly, the rapid development of rulemaking products in ways which fail to 

pass either scientific, legal or policy muster cannot credibly be conceived of as “efficient,” since 

the end result is to produce products that must either be substantially reworked or possibly even 

abandoned. Paradoxically, the consequence of moving potentially flawed products through the 

system, with speed as the main goal, may in fact result in greater delay in achieving 

environmental benefits, thereby undermining even the relatively narrow value of increased speed 

at a particular point in the process. 

OIG Response: In this review, we did not assess the effectiveness of the ADP nor the 

potential for delayed environmental benefits due to the process. 

For example, a rule may be “delayed” at a particular ADP stage while important policy and 

analytical issues are resolved. While this may appear to be a “delay,” well crafted changes to the 

rule can speed later stages of review and successful implementation. And of course, rapid action 

can result in an inefficient uses of the agency’s resources negatively, impact transparency, and 

potentially undermine the Agency’s credibility. Thus, the ADP process is designed not only 

with timeliness in mind, but perhaps more importantly, with the paramount goal of quality 

decision making at its heart. We believe the ADP, as currently configured and implemented, 

plays a critical role in achieving that goal that should not be overlooked. 

OIG Response: Based on the information available to us during our review, we concluded 

that the efficiency of the rule development process could not be assessed. We could not 

conduct a detailed review of the Agency’s rulemaking process due to the lack of data and 

documents in support of the process. We believe that the intent of the ADP is to ensure 

effective actions are developed. 

13-P-0167 14 
 

 

Accordingly, any evaluation of the ADP process should begin with an assessment of whether the 

goals of clarity, scientific validity, technical accuracy, effectiveness and legal defensibility are 

systematically being met and should examine the extent to which ADP process requirements 

have contributed to their fulfillment. OP’s review of the draft report leads us to conclude that 

OIG did not fully consider either the complexity or the needed flexibility of the ADP as it 

considered its draft recommendations and conclusions. In the attached redline strikeout of the 

draft report we provide suggested edits to areas that we believe reflect misunderstandings about 

the ADP. 

OIG Response: It is reasonable and appropriate to assess the efficiency of EPA’s 

rulemaking process. Our report does not address effectiveness issues or advocate that those 

factors are more or less important than efficiency concerns. 

We also note that the draft report directs much of its attention toward the RAPIDS system, which 

is no longer in use. A more recent, system, ADP Tracker is in use and we are beginning to have 

data available from that system that could help EPA and the OIG in drafting this report. RAPIDS 

was developed over 15 years ago and was designed as a forward-looking management 

information system to help Agency and program-level leadership track and schedule decision 

meetings. Contrary to the Draft Report’s suggestion, RAPIDS was not intended to be an 

aggregator for all Agency records relating to action development, nor was it intended to be used 

to assess the extent to which the documents supporting EPA’s rules are efficiently produced. 

These activities are managed within individual EPA programs. In any event, RAPIDS has been 

replaced. 

OIG Response: When we initiated our field work, RAPIDS was the database used by the 

Agency to manage the regulatory development activities. The Agency launched ADP 

Tracker in February 2012, approximately a month after we met with the Agency to discuss 

our final evaluation objectives. At that meeting we were advised that since our sample was 

from 2010 and 2011, RAPIDS would have all the information for our selected rules. The 

Agency said that we should not have to get involved with the new database. Consequently, 

we based our review on the data available to us in RAPIDS. The issues we uncovered with 

RAPIDS were not things that the Agency said would be addressed with ADP Tracker, 

particularly issues of responsibility for data entry and attaching documents. Therefore the 

first and second OIG recommendations are directed toward strengthening the controls in 

place governing ADP Tracker. 

The replacement system for RAPIDS is ADP TRACKER. This system does include features 

that can be used to track process milestones. For example, ADP TRACKER allows us to track 

whether ADP milestones were waived by senior management. It also allows us to track the time 

workgroups were given to review milestone material. By early next year, a full year of these data 

will be available for review. Even so, it should be noted again that even when fully utilized, 

ADP TRACKER will not allow us to measure the “efficiency” of the Agency’s rulemaking 

process and whether the Agency is allocating resources optimally. Again, the system was not 

designed for this purpose. A fair notion of “efficiency” would examine whether the ADP 

process has been efficient at enhancing the quality of EPA rulemaking. Such an assessment 

cannot credibly be performed by a tracking or database management system alone. 

13-P-0167 15 
 

 

OIG Response: While we did not conduct a detailed review of ADP Tracker, the issues 

we uncovered with RAPIDS were not things that the Agency said would be addressed with 

ADP Tracker, particularly issues of responsibility for data entry and attaching documents. 

During our review we were advised that a subset of the data was moved from RAPIDS to 

ADP Tracker. If that is the case, the data in ADP Tracker could have the same problems 

that RAPIDS had. The Agency did not disclose what it defines as the “fair notion of 

efficiency” in its response. 

Apart from these concerns, I agree that opportunities exist to expand the usefulness and 

applicability of the ADP TRACKER. Therefore, OP concurs with the draft report’s first 

recommendation to “establish guidance that clarifies roles and responsibilities in ADP 

implementation, including data entry, record keeping, and the status of action development.” OP 

has addressed many of the concerns with regard to clearly delineated roles in recording data and 

key action development documents. Additionally, ADP TRACKER (unlike RAPIDS) utilizes 

controls to ensure that program offices enter key fields before updating later milestones. OP 

plans to issue the updated ADP TRACKER user guides that will clarify these roles and 

responsibilities by June 2013. 

OP also concurs with the intent of OIG’s second recommendation to “ensure that ADP-Tracker 

has established clear roles, responsibilities, and requirements for the program offices to upload 

development documents; and that the rules are updated in a timely manner, entries are monitored 

for data quality, and all features available are used to evaluate the efficiency of the rule 

development process.” Along with the updated ADP TRACKER user guides, OP will issue a 

memo clearly outlining the responsibility of program offices for uploading development 

documents. We will also use this opportunity to ensure that program offices are working to 

achieve to a high degree of data and record-keeping integrity in ADP TRACKER. However, we 

believe the draft report incorrectly suggests that OP is responsible for maintaining records 

associated with each rulemaking either in ADP TRACKER or otherwise, when it is, in fact, the 

Agency policy for the originating office to maintain such records. As OIG states on page 6 of the 

draft report, “program offices are required to maintain electronic records in the enterprise-wide 

electronic content management system to allow for timely access and retrieval.” We believe this 

distinction concerning recordkeeping should be more clearly and consistently communicated in 

OIG’s final report and we suggest edits to this end. 

OIG Response: The OIG acknowledges the Agency’s concurrence with the first and 

second recommendation. While it is the program offices’ responsibility to maintain 

records, the OIG notes that OP still has oversight responsibility over the ADP process and 

program office responsibility does not diminish that oversight role. 

With regard to OIG’s third and final recommendation, OP disagrees with OIG’s suggestion that 

it “develop and implement a method to track resource use in key rulemaking activities.” OP does 

not have the expertise to develop or implement such a method. OP’s resources and expertise are 

best suited to overseeing the regulatory development process to ensure that the Agency produces 

effective regulatory actions to protect human health and the environment. Monitoring the use of 

13-P-0167 16 
 

 

internal Agency costs associated with rule development may fall under the purview of the Office 

of the Chief Financial Officer or the relevant program office. 

OIG Response: The OIG position is that, without the controls outlined in 

recommendation 3, the Agency will continue to be challenged in assessing the efficiency 

of the rulemaking process and therefore the recommendations stands. We consider this 

recommendation unresolved. As required by EPA Manual 2750, the Agency will need to 

initiate the audit resolution process. 

OP welcomes the opportunity to discuss the comments and suggested revisions detailed here and 

in the attached markup. If you or your staffs have questions regarding this response, please feel 

free to contact Nathaniel Jutras at (202) 564-0301 or jutras.nathaniel@epa.gov. 

Attachment 

13-P-0167 17 
 

 

Appendix C 

Distribution 

Office of the Administrator 

Associate Administrator, Office of Policy 

Agency Follow-Up Official (the CFO) 

Agency Follow-Up Coordinator 

General Counsel 

Associate Administrator for Congressional and Intergovernmental Relations 

Associate Administrator for External Affairs and Environmental Education 

Audit Follow-Up Coordinator, Office of the Administrator 

Audit Follow-Up Coordinator, Office of Policy 

13-P-0167 18