U.S. ENVIRONMENTAL PROTECTION AGENCY 

OFFICE OF INSPECTOR GENERAL 

Catalyst for Improving the Environment 

Audit Report 

EPA’s Office of Research and 

Development Could Better Use the 

Federal Managers’ Financial Integrity Act 

to Improve Operations 

 Report No. 09-P-0232 

 September 15, 2009 
Report Contributors: Patrick Gilbride 

 Erin Barnes-Weaver 

Karen L. Hamilton 

Bryan Holtrop 

Alicia Mariscal 

Mary Anne Strasser 

Abbreviations 

BOSC Board of Scientific Counselors 

EPA U.S. Environmental Protection Agency 

FMFIA Federal Managers’ Financial Integrity Act 

FY Fiscal Year 

GAO Government Accountability Office 

GPRA Government Performance and Results Act 

IRIS Integrated Risk Information System 

NHEERL National Health and Environmental Effects Research Laboratory 

OCFO Office of the Chief Financial Officer 

OIG Office of Inspector General 

OMB Office of Management and Budget 

ORD Office of Research and Development 

PART Program Assessment Rating Tool 

Cover photo:	 A photo montage of EPA Office of Research and Development National 

Health and Environmental Effects Research Laboratory facilities 

geographically dispersed across the United States (EPA photos). 
 

U.S. Environmental Protection Agency 09-P-0232 

September 15, 2009 Office of Inspector General 

At a Glance
 

Catalyst for Improving the Environment 

Why We Did This Review 

We conducted this audit to 

determine whether the U.S. 

Environmental Protection 

Agency (EPA) Office of 

Research and Development 

(ORD) fully integrated the 

Federal Managers’ Financial 

Integrity Act (FMFIA) into 

program operations. We 

asked whether ORD has a 

systematic strategy to 

establish, review, and monitor 

internal controls, and what 

ORD’s strategy should contain 

to account for risks in meeting 

program goals. 

Background 

FMFIA requires federal 

managers to improve the 

accountability and 

effectiveness of federal 

programs by establishing, 

assessing, correcting, and 

reporting on internal control. 

FMFIA also requires federal 

managers to annually evaluate 

their agencies' compliance 

with Government 

Accountability Office (GAO) 

internal control standards. 

For further information, 

contact our Office of 

Congressional, Public Affairs 

and Management at 

(202) 566-2391. 

To view the full report, 

click on the following link: 

www.epa.gov/oig/reports/2009/ 

20090915-09-P-0232.pdf 

EPA’s Office of Research and Development Could 

Better Use the Federal Managers’ Financial Integrity 

Act to Improve Operations 

What We Found 

ORD’s management integrity program is inconsistent with Agency FMFIA 

guidance. ORD approaches FMFIA as an administrative reporting activity rather 

than an opportunity to evaluate and report on research program performance. As 

a result, ORD has not: 

•	 Conducted a comprehensive risk assessment, 

•	 Included National Program Directors in the FMFIA process, 

•	 Developed and implemented a strategy to establish and evaluate the 

effectiveness of internal controls over research programs, 

•	 Provided FMFIA training to managers and staff, and 

•	 Included relevant risk and program performance information in assurance 

letters. 

EPA Order 1000.24 requires all organizations to systematically review and assess 

the effectiveness of internal controls consistent with GAO internal control 

standards. The Order gives program managers flexibility in designing review 

strategies. While ORD’s largest lab, the National Health and Environmental 

Effects Research Laboratory (NHEERL), informally identifies program risks, 

neither ORD nor NHEERL conducts internal control risk assessments on which to 

base a program review strategy. Applying FMFIA as intended would help EPA 

achieve its mission and program results through improved accountability. 

ORD’s Administrative Efficiencies Project management integrity workgroup has 

initiated actions that we believe will address our findings, such as developing a 

draft multi-year review strategy. In developing its new strategy, ORD should 

include programmatic elements, a training plan, pertinent results from peer 

reviews, and best practices to ensure more effective FMFIA implementation. 

What We Recommend 

We recommend that ORD (1) conduct a risk assessment using GAO standards and 

develop a comprehensive risk-based program review strategy; (2) develop 

comprehensive, tiered FMFIA training for managers and staff; and (3) revise its 

management integrity program to include programmatic operations. ORD agreed 

with our recommendations and has initiated corrective actions that we believe 

address the intent of our recommendations. 
 

 

 

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY 

WASHINGTON, D.C. 20460 

OFFICE OF 

INSPECTOR GENERAL 

September 15, 2009 

MEMORANDUM 

SUBJECT:	 EPA’s Office of Research and Development Could Better Use the 

 Federal Managers’ Financial Integrity Act to Improve Operations 

 Report No. 09-P-0232 

FROM:	 Melissa M. Heist 

Assistant Inspector General for Audits 

TO: Lek Kadeli 

 Acting Assistant Administrator 

 Office of Research and Development 

The Office of Inspector General (OIG) of the U.S. Environmental Protection Agency (EPA) 

conducted this report on the subject audit. This report contains findings that describe problems 

we identified and corrective actions we recommend. This report represents our opinion and does 

not necessarily represent the final EPA position. EPA managers will make final determinations 

on matters in this report in accordance with established audit resolution procedures. 

The estimated cost of this report – calculated by multiplying the project’s staff days by the 

applicable daily full cost billing rates in effect at the time – is $515,790. 

Action Required 

On September 4, 2009, your office provided comments to our report that included a corrective 

action plan with milestone dates. We believe your planned corrective actions address the intent 

of each of our recommendations. As such, we plan to close this assignment upon issuance of this 

final report. We have no objections to the further release of this report to the public. This report 

will be available at http://www.epa.gov/oig. 

We appreciate the efforts of your staff in working with us during the course of our audit. If you 

or your staff has any questions regarding this report, please contact me at (202) 566-0899 or 

heist.melissa@epa.gov; or Patrick Gilbride, Director, Risk and Program Performance Issues, at 

(303) 312-6969 or gilbride.patrick@epa.gov. 
 

 

 

EPA’s Office of Research and Development Could Better Use the 09-P-0232 

Federal Managers’ Financial Integrity Act to Improve Operations 

Table of Contents 

Chapters 

1	 Introduction ................................................................................................................. 1 

Purpose...................................................................................................................... 1 

Background ................................................................................................................ 1 

Noteworthy Achievements.......................................................................................... 4 

Scope and Methodology ............................................................................................ 5 

2	 Opportunities Exist for ORD to Better Use the
 

FMFIA Process to Improve Programmatic Operations............................................ 6 

Management Integrity Program Inconsistent with FMFIA Guidance.......................... 6 

Management Integrity Strategy Should Include Program Elements........................... 13 

Conclusion ................................................................................................................. 16 

Recommendations ..................................................................................................... 16 

Agency Comments and OIG Evaluation..................................................................... 17 

Status of Recommendations and Potential Monetary Benefits.................................... 18 

Appendices 

A Organization of EPA ORD........................................................................................ 19 

B Details on Scope and Methodology........................................................................ 21 

C Agency Response to Draft Report.......................................................................... 23 

D Distribution ............................................................................................................... 28 
 

 

 

09-P-0232 

Chapter 1

Introduction 

Purpose 

The Office of Inspector General (OIG) reviewed implementation of the Federal 

Managers’ Financial Integrity Act (FMFIA) within the Office of Research and 

Development (ORD), the scientific research arm of the U.S. Environmental 

Protection Agency (EPA). We sought to determine whether ORD fully integrated 

FMFIA into programmatic operations. We examined ORD using its largest lab, 

the National Health and Environmental Effects Research Laboratory (NHEERL), 

as our example. Our objectives were to determine: 

•	 Whether ORD has a systematic strategy to establish, review, and monitor 

internal controls. 

•	 What ORD's internal control strategy should contain to account for risks in 

meeting program goals. 

Background 

EPA’s Office of Research and Development 

ORD is EPA's lead office for the production, review, and integration of scientific 

and technical knowledge into environmental protection policies and regulations. 

ORD has seven laboratories and centers across the country, with ORD 

headquarters in Washington, DC, and main research facilities in Ohio and North 

Carolina. NHEERL is ORD’s largest individual laboratory, accounting for 

21 percent of ORD’s Fiscal Year (FY) 2008 budget and 33 percent of its 

authorized full-time staff. NHEERL has division and field office laboratories in 

eight locations and ecological environments across the country. 

To provide the leadership to accomplish ORD’s strategic goals, ORD created an 

Executive Council, consisting of senior management, to make corporate 

decisions. ORD instituted a strategic multi-year planning process to guide the 

direction of ORD’s research to focus on EPA's highest priority science needs. 

National Program Directors lead development of multi-year plans with 

involvement by staff and managers. There are no direct lines of authority 

between National Program Directors and lab, center, and office directors as both 

positions report to the Assistant Administrator. ORD confirms the relevancy and 

credibility of its science through program reviews by the Board of Scientific 

Counselors (BOSC). ORD aligned BOSC reviews to meet the structure of 

reviews conducted under the Office of Management and Budget (OMB) Program 

Assessment Rating Tool (PART). 

1 
 

 

09-P-0232 

ORD issued a policy in November 2006 on how ORD implements FMFIA. 

ORD’s Assistant Administrator has responsibility for implementing FMFIA. 

Additional responsibilities lie with lab and center directors and deputy directors. 

An ORD Management Integrity Advisor coordinates activities such as the 

assurance letter process. The Advisor works with designated Management 

Integrity Coordinators within ORD’s seven labs and centers. According to ORD’s 

policy, National Program Directors do not have a role in the management integrity 

process. See Appendix A for more details on ORD’s organizational structure. 

Management Integrity Guidance 

FMFIA requires federal managers to improve the accountability and 

effectiveness of federal programs and operations by establishing, assessing, 

correcting, and reporting on internal control. Federal managers must also 

develop and maintain internal control to achieve: (1) effective and efficient 

operations; (2) reliable financial reporting; and (3) compliance with applicable 

laws and regulations per OMB Circular A-123, Management’s Responsibility 

for Internal Control (revised). Effective internal control is a key factor in 

achieving agency missions and program results. 

The Federal Government has implemented several initiatives, such as the 

Government Performance and Results Act of 1993 (GPRA) and PART, to 

improve program management. Activities conducted as part of these initiatives 

support an agency’s overall internal control framework. Figure 1.1 illustrates 

how FMFIA serves as an umbrella under which agencies should coordinate 

internal control efforts. 

Figure 1.1: FMFIA Internal Control Framework 

Source: EPA training, EPA Internal Control and Management Integrity: Make It 

Second Nature, issued (via EPA’s Intranet) on May 28, 2008 (slide 11 of 21). 

2 
09-P-0232 

FMFIA requires federal managers to annually evaluate their agency's compliance 

with Government Accountability Office (GAO) Standards for Internal Control in 

the Federal Government, shown in Table 1.1, and issue a statement indicating full 

compliance or noncompliance. The standards provide the overall framework for 

establishing and maintaining internal controls, and for identifying and addressing 

major performance and management challenges and areas at greatest risk of fraud, 

waste, abuse, and mismanagement. The standards comprise a major part of 

managing an organization. 

Table 1.1: GAO’s Standards for Internal Control in the Federal Government 

Control This standard establishes and maintains an environment throughout 

Environment the organization that sets a positive and supporting attitude toward 

internal control and conscientious management. This includes 

establishing goals, objectives, and performance measures at the entity 

and activity level. 

Risk 

Assessment 

Once the goals, objectives, and measures have been defined, the risks 

that could impede efficiently and effectively achieving those objectives 

are identified. This includes assessing risks the agency faces from 

both internal and external sources. Risk assessment includes 

identifying and analyzing risks associated with achieving objectives 

defined in strategic and annual performance plans developed under 

GPRA, and form a basis for determining how to manage risks. 

Management needs to comprehensively identify risks and should 

consider all significant interactions between the entity and other parties 

as well as internal factors at both the entity-wide and activity levels. 

Control 

Activities 

These are the policies, procedures, techniques, and mechanisms that 

implement management’s direction to achieving goals. Internal control 

activities help ensure that management’s directives are carried out. 

Information and 

Communications 

This standard includes data and information (performance and 

financial) to determine whether the organization meets its goals and 

objectives and maintains accountability over resources. 

Monitoring Internal control monitoring should assess the quality of performance 

over time and ensure that audits and other review findings are 

promptly resolved. 

Source: OIG’s Summary of GAO’s Standards for Internal Control in the Federal Government, 

GAO/AIMD-00-21.3.1 (November 1999). 

To implement FMFIA and OMB Circular A-123, EPA issued Order 1000.24, 

Management’s Responsibility for Internal Control. The Order: 

•	 Prescribes policies, procedures, and standards for internal controls at EPA; 

•	 Outlines Agency senior managers’ roles and responsibilities for developing, 

implementing, assessing, documenting, improving, and reporting on internal 

controls; 

•	 Incorporates specific requirements for assessing internal controls over 

financial reporting; and 

•	 Provides tools to help managers monitor both overall program progress and 

the effectiveness of day-to-day operations. 

3 
 

09-P-0232 

In accordance with the Order, the Office of the Chief Financial Officer (OCFO) 

issues annual guidance to program and regional offices on complying with 

FMFIA. This guidance includes a reporting template with specific instructions 

for completing assurance letters. Assurance letters provide the results of the 

internal control assessment and an overall statement to the Administrator on the 

adequacy of controls for the organization. In 2008 OCFO also developed an 

Intranet training to increase understanding of internal controls, titled EPA 

Awareness Training for Internal Controls and Management Integrity, although 

OCFO did not mandate that all EPA staff complete the course. OCFO annually 

collects all program and regional office assurance letters and compiles a single 

draft assurance letter for the Administrator to review and sign. 

EPA’s Order also requires managers to develop and implement a strategy that 

defines how they use sources of program management information to provide the 

basis for their annual assurance letters. The systematic review strategy should be 

consistent with and integrate Agency-wide processes used to develop and report 

on program performance measures and results required under GPRA. Examples 

of sources of program management information include: OIG and GAO reports, 

internal and external program evaluations, audits and reviews conducted under the 

Chief Financial Officers Act and GPRA, PART, and other reviews. The Order 

recommends that program managers use GAO’s five internal control standards 

when developing a review strategy as the basis for determining the need for and 

design of an internal control and how well it functions. 

EPA also issued a 1996 publication, Management Integrity at EPA - A Manager’s 

“How To” Guide for Program Reviews: Seeing the Forest and the Trees. The 

guide introduced the Agency’s 10 management integrity principles and noted that 

managers often miss the essence of internal controls and FMFIA: 

In complying with FMFIA, many Federal managers historically 

never saw the “Big Picture.” Most focused on filling out 

checklists and performing other routine compliance tests, rather 

than considering management controls in light of broader program 

issues and EPA’s overall mission. In short, they got lost in the 

trees and never saw the forest! 

Noteworthy Achievements 

ORD re-engineered its management integrity function through its Administrative 

Efficiencies Project. This effort focused on improving administrative and 

financial internal controls, including how ORD might conduct a formal risk-based 

assessment of those controls. This effort’s management integrity workgroup is 

also considering a separate “scientific” or programmatic track for assessing 

internal control. 

4 
 

 

09-P-0232 

ORD engaged the National Academy of Sciences to evaluate its research program 

effectiveness in a report, Evaluating Research Efficiency in the U.S. 

Environmental Protection Agency (published in 2008), which significantly altered 

the dialogue and approach to efficiency measurement. 

ORD and NHEERL undergo many external peer reviews to maintain a high level 

of credibility. The Science Advisory Board reviews the quality and relevance of 

scientific and technical information used or proposed as the basis for Agency 

regulations. BOSC evaluates and reviews scientific research programs, plans, and 

laboratories (and related management practices) and recommends improvement 

actions. Since FY 2008, NHEERL has conducted management systems reviews 

in lieu of traditional divisional reviews as a cost saving effort. 

Scope and Methodology 

We conducted our audit from July 2008 through April 2009 in accordance with 

generally accepted government auditing standards. Those standards require that 

we plan and perform the audit to obtain sufficient, appropriate evidence to 

provide a reasonable basis for our findings and conclusions based on our audit 

objectives.1 We believe that the evidence obtained provides a reasonable basis for 

our findings and conclusions based on our audit objectives. 

We focused our evaluation on ORD’s headquarters office in Washington, DC, and 

NHEERL facilities located in Raleigh, North Carolina, and Corvallis and 

Newport, Oregon. We reviewed and analyzed EPA and ORD management 

integrity policies, procedures, and FMFIA guidance; ORD’s budget and 

expenditure data; and FYs 2007 and 2008 FMFIA assurance letters. We 

interviewed ORD and NHEERL personnel at various levels of responsibility. 

We conducted site visits to NHEERL and its Western Ecology Division, including 

tours of several laboratories. We benchmarked risk assessment methods used by 

others in the public sector, as well as the FMFIA process at eight other federal 

agencies. We reviewed NHEERL-related internal/external peer reviews to 

determine the extent to which they addressed internal controls. Appendix B 

provides additional information on our scope and methodology. 

1

 In the course of performing our field work, we identified findings applicable outside of ORD-NHEERL. In 

February and March 2009 we expanded our field work to include reviewing assurance letters and FMFIA processes 

in four regions and two program offices. In August 2009, we issued a report to OCFO on the Agency’s management 

integrity program, summarizing examples from the regions and program offices we reviewed. 

5 
09-P-0232 

Chapter 2

Opportunities Exist for ORD to Better Use the 

FMFIA Process to Improve Programmatic Operations 

ORD’s management integrity program is inconsistent with Agency FMFIA 

guidance. Currently, ORD approaches FMFIA as an administrative reporting 

activity rather than an opportunity to evaluate and report on research program 

performance. As a result, ORD has not: 

•	 Conducted a comprehensive risk assessment, 

•	 Included National Program Directors in the FMFIA process, 

•	 Developed and implemented a strategy to establish and evaluate the 

effectiveness of internal controls over research programs, 

•	 Provided FMFIA training to managers and staff to assess program 

performance, and 

•	 Included relevant risk and program performance information in assurance 

letters. 

EPA Order 1000.24 requires all organizations to systematically review and assess 

the effectiveness of internal controls consistent with GAO internal control 

standards. The Order gives program managers flexibility in designing review 

strategies. While NHEERL, ORD’s largest lab, informally identifies program 

risks, neither ORD nor NHEERL conducts internal control risk assessments on 

which to base a program review strategy. Applying FMFIA as intended would 

help EPA achieve its mission and program results through improved 

accountability. 

Management Integrity Program Inconsistent with FMFIA Guidance 

ORD Has Not Conducted a Comprehensive Internal Control 

Risk Assessment 

ORD has not conducted a formal risk assessment for identifying and analyzing 

risks for possible effects in program operations. OMB Circular A-123 states that 

managers should perform risk assessments to identify significant areas within 

which to place or enhance internal control. The Circular describes risk 

assessment as a critical step in the process to determine the extent of controls. 

While ORD has not assessed risk, NHEERL and its Western Ecology Division 

have informally assessed their program risks, as shown in Table 2.1. 

6 
09-P-0232 

Table 2.1: Program Risks 

Identified Program Risks NHEERL 

Western 

Ecology 

Division 

1. Inability to quickly respond to changing priorities. X X 

2. Imbalance of breadth and depth in research program. X X 

3. Difficulty in building/maintaining research collaborations. X X 

4. Inability to meet commitments in face of declining 

resources. 

X X 

5. Inadequate safeguards to ensure that Agency decisions are 

supported by the highest quality science. 

X X 

6. Unclear priorities. X 

7. Mismatch of skill mix. X 

8. Difficulty in building and maintaining partnerships with 

program offices and regions. 

X 

Source: NHEERL and Western Ecology Division presentations to OIG in November 2008. 

NHEERL conducts quality assurance, peer review, and accountability reviews 

that it believes address three of the five risks identified. However, NHEERL 

identified these risks based on management’s judgment subsequent to initiating 

these reviews and did not assess the effectiveness of internal controls. 

ORD Does Not Include National Program Directors in the FMFIA 

Process 

ORD’s process to evaluate risks and assign priority does not involve National 

Program Directors. These directors lead development of ORD’s multi-year plans 

that tie to the strategic plan and EPA’s mission, so internal control risk 

assessments should focus on impediments to multi-year plans. Some of the 

directors we interviewed said that individual lab research priorities did not 

necessarily align with multi-year plan priorities. Further, even though directors 

play a significant role in directing and ensuring that ORD achieves its mission, 

ORD has not involved them in evaluating internal controls, implementing the 

management integrity program, or preparing FMFIA assurance letters. 

Lab and center directors told us ORD should involve National Program Directors 

in the FMFIA process but were unsure how to do so given ORD’s matrix structure. 

This structure separates program performance aspects such as PART and GPRA 

from FMFIA and provides no clear link between required annual reports. EPA 

Order 1000.24 addresses program managers’ responsibility for internal controls, 

including GPRA performance measures. The Order also specifies that any review 

strategy be consistent with Agency processes for GPRA reporting. 

ORD’s organizational structure sets boundaries for what National Program 

Directors can do in regards to implementing research assigned to lab and center 

directors. While National Program Directors develop research plans, reviews, and 

budgets, they do not oversee day-to-day operations including spending and 

staffing. Without additional involvement, National Program Directors cannot 

evaluate a research program’s internal controls. In our interviews, directors 

7 
09-P-0232 

described difficulties encountered while managing their research programs, such 

as limited access to information on funding spent against the budget and staff time 

charges to research programs. They also indicated they could benefit from 

improved communication and coordination with labs, centers, and offices to 

ensure consensus on prioritizing, implementing, and managing research programs. 

ORD Has Not Developed a Program Review Strategy 

ORD has not developed a strategy to systematically review and assess the 

effectiveness of internal control for program operations. EPA Order 1000.24 

states that program managers should develop a strategy for systematically 

reviewing and assessing the effectiveness of internal controls; detecting 

weaknesses and deficiencies; and providing a sound, documented basis for the 

assurance letter to the Administrator. OCFO’s FY 2008 management integrity 

guidance requires that annual assurance letters describe the organization's review 

strategy for assessing how well internal controls over program operations 

(guidance, procedures, and policies) protect against fraud, waste, abuse, and 

mismanagement. 

ORD managers annually require labs and centers to design a review strategy that 

meets program needs and conduct internal control reviews. However we did not 

find any evidence that these activities took place for research programs. Our 

interviews with ORD and NHEERL staff, as well as our review of ORD’s and 

NHEERL’s FY 2008 assurance letters, confirmed this. ORD states in its FY 2008 

assurance letter that “ORD conducted more than 38 management reviews of the 

following areas: extramural (assistance agreements, interagency agreements, 

contracts, simplified acquisitions), purchase cards, property, funds control and 

flexiplace.” Management reviews focused on administrative and financial 

activities, not program operations. 

ORD managers agreed that their FY 2008 assurance letter did not discuss a 

program review strategy or describe how it reviewed principal research programs. 

ORD stated it believed “Examples exist in the assurance letter of how ORD 

approached the review of some of its programs, for example the approach for 

addressing the Agency’s (and ORD’s) Biofuels Strategy.” However ORD did not 

base this process on a comprehensive risk assessment, did not report on internal 

control effectiveness, and may not have provided a sound basis for the Assistant 

Administrator to assert compliance with FMFIA. 

In August 2008, ORD organized a management integrity workgroup as part of its 

Administrative Efficiencies Project. ORD charged this workgroup with 

developing a plan for conducting a management integrity line of business as an 

ORD-wide function. ORD said the workgroup will coordinate various programs 

that support management integrity into standard ORD operating principles. We 

reviewed ORD’s draft strategy and do not believe that it addresses programmatic 

controls. 

8 
 

09-P-0232 

ORD Relies on Limited OCFO Guidance 

ORD relies on OCFO guidance that does not focus on program operations for 

reporting internal controls. Further, ORD did not change the composition of its 

assurance letter between 2007 and 2008 to reflect changes in OCFO guidance. 

OCFO’s FY 2008 guidance: 

•	 Required a more rigorous review of the Agency’s internal controls against 

GAO’s Standards for Internal Control in the Federal Government. 

•	 Required offices to document their approach to programmatic internal 

control reviews in assurance letters. 

•	 Included an Internal Control Evaluation Checklist as an attachment to 

provide a basis on which to evaluate internal controls and to use the 

checklist to assess the effectiveness of programmatic internal controls. 

OCFO believed its FY 2008 guidance improved reporting on internal control 

effectiveness of program operations. However, the OCFO letter template focused 

on administrative and financial reporting.2 ORD did not follow any strategy or 

report additional information on internal controls beyond what OCFO specified in 

its template. 

ORD staff told us they found OCFO’s 2008 guidance confusing in several areas. 

For example, staff could not discern whether ORD should report the occurrence 

or results of program reviews. Staff also said OCFO’s guidance was not specific 

and did not always tie in to EPA Order 1000.24. 

ORD’s lab and center Management Integrity Coordinators rely on FMFIA 

guidance disseminated by ORD after it receives OCFO’s annual guidance. 

However, ORD did not disseminate all guidance it received from OCFO in 

FY 2008. ORD did not disseminate the checklist until OCFO initiated its 

FY 2009 assurance letter process; 83 percent of Management Integrity 

Coordinators interviewed said they had not seen the checklist before this year. 

We also noted that OCFO’s FY 2008 guidance and ORD’s assurance letter 

contained the subject heading Internal Control Review Strategy while NHEERL’s 

assurance letter did not, because the guidance ORD provided to labs and centers 

did not include that subject heading as a reporting requirement. 

In addition, we found that ORD’s 2006 Management Integrity Policy, a 

supplement to EPA’s guidance, was inconsistent with FMFIA guidance because it 

did not cite GAO’s Standards for Internal Control in the Federal Government. 

The policy also referenced out-of-date information, such as older versions of both 

OMB A-123 and EPA Order 1000.24. ORD staff indicated that its Management 

Integrity Workgroup plans to revise ORD’s policy to include updated guidance. 

2 We issued a report to OCFO in August 2009 describing our concerns on the administrative focus of FMFIA 

guidance. 

9 
09-P-0232 

ORD Managers and Staff Need Additional Training on Internal 

Control Standards 

ORD personnel gain knowledge of FMFIA and internal controls largely through 

on-the-job-training and did not receive sufficient additional training on evaluating 

internal controls. Inadequate understanding of the internal control process 

resulted in ORD relegating FMFIA to a yearly administrative reporting activity. 

ORD managers and staff responsible for FMFIA receive no training on GAO’s 

five internal control standards or how to ensure research programs meet standards. 

GAO’s standards provide the overall framework for establishing and maintaining 

internal control, and for identifying and addressing major performance and 

management challenges. None of ORD’s lab and center directors could say that 

their assurance letters addressed internal control standards. Three of seven 

Management Integrity Coordinators said they addressed all five standards 

throughout their letters while the remaining four acknowledged that their letters 

did not do so. Only four of seven coordinators were aware of GAO’s standards, 

while only two of seven were familiar with GAO’s evaluation tool/checklist. 

Figure 2.1 illustrates coordinators’ awareness of management integrity guidance. 

Figure 2.1: Awareness by Management Integrity Coordinators of FMFIA Guidance FMFIA Guidance 

EPA "How To" Guide 

GAO Tool/Checklist 

GAO Internal Control 

Standards 

EPA Awareness 

Training 

OMB Circular A-123 

EPA Order 1000.24 

0% 20% 40% 60% 80% 100% 

Percentage of MICs Aware of Guidance 

Source: OIG analysis of interviews with Management Integrity Coordinators. 

Managers and staff interviewed said they did not consider results of program 

reviews, such as GPRA performance measures, PART, and peer reviews relevant 

for FMFIA purposes. However, FMFIA guidance, including OMB Circular 

A-123, emphasizes the importance of integrating these reviews into the FMFIA 

process. Several coordinators interviewed said their FMFIA reporting activities 

focused entirely on administrative activities with no linkage between program 

review information and FMFIA. All of ORD’s coordinators – and several 

managers – said that additional training on FMFIA would be helpful. One ORD 

10 
 

09-P-0232 

manager suggested that EPA develop tiered training, with one tier for senior 

managers on understanding controls and FMFIA and another for management 

integrity staff on the "nuts and bolts" of implementing and reporting on controls. 

We believe ORD’s FMFIA training should also include coverage of all key 

guidance documents, such as EPA Order 1000.24. 

ORD managers agree on the need to conduct internal control training at all levels 

within the organization. However, ORD’s Draft Multi-Year Program Review 

Strategy did not elaborate on a specific training plan. ORD states that it will 

periodically train all key personnel involved in the internal control process and 

work with the Agency to identify appropriate training for staff. ORD should 

identify areas of strength and weakness among its staff and, in turn, tailor its 

training around those needs. Coordinators interviewed suggested ORD develop 

training that includes: 

•	 Internal controls, 

•	 An ORD-specific template for reporting, 

•	 Best practices/lessons learned, 

•	 Risk assessment requirements, and 

•	 Training unique to managers on their FMFIA responsibilities and internal 

controls. 

Without comprehensive and up-to-date training, personnel may not be qualified to 

assess performance of programmatic operations. 

Assurance Letters Omitted Program Risk and Performance 

Information 

ORD, NHEERL, and the Office of Science Policy’s FY 2008 assurance letters did 

not document results of relevant program reviews conducted by organizations 

external to ORD. EPA Order 1000.24 provides several examples of program 

management information to incorporate into assurance letters, including 

management reviews, OIG and GAO reports, program evaluations, and other 

audits and reviews such as GPRA. Examples of program risk and performance 

information omitted from assurance letters follow in Table 2.2. 

11 
 

 

 

09-P-0232 

Table 2.2: Examples of Information Omitted from Assurance Letters 

ORD 

• ORD’s FMFIA strategy and FY 2008 assurance letter did not address how ORD 

developed and implemented performance goals and measures to comply with 

GPRA and PART requirements. ORD described this internal control system in its 

“Accountability Handbook for Performance Measurement” (dated August 2007) and 

in Section 5.2 (“ORD Performance Measure Tracking”) of ORD’s Policies and 

Procedures Manual. 

• ORD’s letter did not discuss results of BOSC reviews on four research programs 

and one center in FY 2007 and 2008.3

 Also, ORD’s 2008 assurance letter did not 

mention completed NHEERL-relevant PART and BOSC reviews for two research 

programs.4

 ORD said it incorporates BOSC review results “into ORD management 

decision-making and into the criteria used for budget decisions and related 

documents." Several of these BOSC reviews addressed program management 

issues and could serve to demonstrate ORD's compliance with two GAO internal 

control standards (control activities and monitoring). 

• ORD's letter did not mention GAO’s April and May 2008 testimonial reports where 

GAO found that ORD’s revised Integrated Risk Information System (IRIS) process 

did not respond to GAO’s March 2008 report recommendations and further 

jeopardized IRIS database viability.5

 In recommending that EPA not consider IRIS 

as a management challenge (in an attachment to its 2008 letter), ORD cited its 

revised IRIS process but did not elaborate on GAO’s findings. ORD told us it 

disagreed when GAO first identified IRIS as a management challenge. However, 

ORD now agrees since GAO listed IRIS on its High Risk report. 

• ORD only included performance measures on IRIS and the Human Health Risk 

Assessment program in its FY 2008 letter, excluding all other performance 

measures. ORD said OCFO’s FY 2008 guidance did not require reporting on 

performance measures. However, the first page of the cover memo accompanying 

OCFO’s FY 2008 guidance stated explicitly that FMFIA requires the Administrator 

to report on internal controls over programs, including performance measures. 

OMB Circular A-123 also specifies that agencies consider GPRA and PART 

requirements as part of their internal control structure. Consistent with this 

Circular, EPA Order 1000.24 specifies this same requirement. 

• ORD’s letter did not mention results of a National Academy of Sciences report 

issued in February 2008, Evaluating Research Efficiency in the U.S. Environmental 

Protection Agency, and NHEERL divisional peer reviews. In its FY 2008 

assurance letter, NHEERL described completing the Atlantic Ecology Division peer 

review and responding to the Mid-Continent Ecology Division’s peer review. ORD 

excluded these significant NHEERL items from the FY 2008 ORD assurance letter. 

Our review of the Atlantic Ecology Division peer review report determined that it 

addressed three internal control standards (risk assessment, control activities, and 

monitoring). 

3

 BOSC reviewed the Science and Technology for Sustainability Research Program, Human Health Risk 

Assessment Research Program, Particulate Matter/Ozone Research Program (mid-cycle), Endocrine Disrupting
 

Chemicals Research Program (mid-cycle), and the National Center for Environmental Research. 4

 These included the Ecological and Safe Pesticide/Safe Product Research Programs.
 5

 IRIS provides toxic chemical assessment information to EPA’s stakeholders. 

12 
09-P-0232 

NHEERL and Office of Science Policy 

• NHEERL did not identify results of relevant BOSC reviews (reports issued in July 

and August 2007) in which NHEERL’s Gulf Ecology Division participated. Our 

analysis found that these reviews identified issues relating to four internal control 

standards and all five of NHEERL's self-identified risks. 

• NHEERL did not discuss the contents or results of a detailed Atlantic Ecology 

Division peer review in its FY 2008 letter. NHEERL disclosed that it had completed 

a peer review and that the committee issued a written report “which identifies 

strengths and challenges and offers recommendations for improvement.” 

• ORD’s Office of Science Policy, which manages BOSC efforts, listed in its FY 2008 

assurance letter final reports completed for five research programs but did not 

discuss report contents or results. The office included information on review 

accomplishments, but this information only described the report title, procedural 

activity (e.g., meeting, conference call), and final report. Additionally, its assurance 

letter did not discuss a review strategy to systematically evaluate internal controls. 

Source: OIG analysis. 

ORD managers said the assurance letter “must attest to the soundness of internal 

controls for programs, functions, and financial activities” for labs and centers. 

Completing a risk assessment and developing a review strategy would support 

decisions regarding the relevance of these reviews and, as a result, determining 

whether to include review results in the assurance letter. However, because they 

did not conduct a formal risk assessment nor follow a systematic review strategy, 

ORD, NHEERL, and the Office of Science Policy omitted from assurance letters 

external review results pertinent to management integrity. We found review 

results directly addressed GAO’s five internal control standards. Such omissions 

could impact the accuracy of information ORD reports in its assurance letters, and 

may render invalid any assurance ORD makes as to the integrity of its programs. 

Management Integrity Strategy Should Include Program Elements 

As noted above, ORD has not developed a program review strategy to 

systematically review and assess the effectiveness of internal control as required 

by EPA Order 1000.24. ORD viewed FMFIA as an administrative exercise and 

did not consider external program review results as relevant to its management 

integrity approach. ORD has taken recent steps to develop a draft Multi-Year 

Program Review Strategy – a requirement of OCFO’s 2009 FMFIA guidance. 

We commend ORD for developing a formal strategy and encourage ORD to 

include specific details on how it plans to address strategy recommendations in 

EPA Order 1000.24. In addition, per our second objective, we believe ORD’s 

strategy should also include information on its extensive peer review program as 

well as best practices we identified from other public sources. 

External Peer Program Reviews Conducted by ORD and NHEERL 

ORD’s Strategy should explain how it plans to use external program reviews 

conducted by the Science Advisory Board, BOSC, peers, GAO, and OIG as 

program management elements required by EPA Order 1000.24. ORD initiates 

13 
 

 

09-P-0232 

program reviews at several levels within its complex matrix structure. ORD’s 

strategy should include a schedule for reviews and describe how ORD will use 

and report review results as part of its FMFIA process. Results of these reviews, 

in addition to other program evaluations, should form the basis for any assertions 

ORD makes in its annual assurance letter to the Administrator. ORD should also 

evaluate the scope and frequency of external reviews. The potential impact of any 

risk should include both quantitative and qualitative costs: 

•	 Quantitative costs include the cost of property, equipment, or inventory, 

cash dollar loss, and damage and repair costs. 

•	 Qualitative costs include loss of public trust, loss of future funding, 

increased legislation, violation of laws, not achieving organizational goals, 

and decreased credibility. Such costs, while more difficult to assess, are 

equally important. 

We found that ORD identified completed peer reviews but did not discuss review 

results. Figure 2.2 lists Science Advisory Board, BOSC, and divisional NHEERL 

peer reviews.

 Figure 2.2: Number of ORD Peer Reviews for 2007-2009 

5 

7 

10 

8 

4 

7 

3 

1 

1 

0 2 4 6 8 10 12 

2007 

2008 

2009 

SAB BOSC NHEERL 

* 

* 

* Note: ORD suspended its divisional peer review program in 2008 and, instead, 

initiated a pilot Management Systems Review (first in the Gulf Ecology Division 

in 2008, and planned for the Mid-Continent Ecology Division in September 2009).
 

Source: Data provided by ORD during the course of field work. 

We analyzed select program reviews and identified internal control aspects in 

questions reviewers asked as well as review results and recommendations. While 

each review had different objectives, we found that several reports addressed, to 

varying degrees, GAO’s standards. Some examples include: 

•	 In its mid-cycle review of the Global Change Research Program in September 

2008, BOSC asked, “How responsive has the Global Change Research Program 

14 
 

09-P-0232 

been to the recommendations made in the April 2006 BOSC program review 

report?” This question addresses the “monitoring” internal control standard. 

•	 In its review of the Human Health Risk Assessment Program in April 2008, 

BOSC asked, “How consistent are the Long Term Goals of the Program with 

achieving the Agency’s strategic plan and the Human Health Risk 

Assessment’s MYP (Multi-Year Plan)?” This question is similar to several 

items GAO included in its Internal Control Management and Evaluation Tool 

under the “risk assessment” heading. 

A risked-based strategy that prioritizes systematic reviews can help determine 

whether there are redundancies in a program and the programs at greatest risk are 

being reviewed. ORD managers acknowledged they may have redundancies in 

the peer reviews they conduct, and BOSC came to the same conclusion in a 

report. In addition to the burden of being over-reviewed, ORD does not know if it 

focuses reviews on the highest risk areas that warrant most attention. 

Best Practices ORD Could Implement 

We identified several best practices on management integrity used at public 

organizations that ORD could use (with modifications) in its strategy: 

•	 The State of Minnesota’s risk management plan provides an example of 

steps that any risk assessment methodology should include. This plan, 

shown in Table 2.3, becomes the overall basis for developing, evaluating, 

and maintaining internal control. 

Table 2.3: Elements of Minnesota’s Risk Assessment Methodology 

1. Identify risk. 

2. Categorize risk. 

3. Assess likelihood and effect. 

4. Prioritize risks. 

5. Develop a plan to reduce risks (response). 

6. Document dates and actions taken to reduce risks. 

7. Establish systematic reviews and track responses. 

8. Control risk – use above process, update based on results, and revise. 

Source: State of Minnesota 

•	 The Department of Defense requires its components to: (1) determine 

high risk areas and establish written plans for testing those areas, and 

(2) develop a written strategy for program reviews based on those risks. 

The Defense Acquisition University identifies and describes risks by 

reviewing strategic and other planning documents and communicating 

with stakeholders to assess: (a) deliverables and work processes, 

(b) milestones and schedule dates, (c) resource needs and sources, and 

(d) performance requirements. 

15 
09-P-0232 

In addition, ORD could use GAO’s Internal Control Management and Evaluation 

Tool, which outlines steps for identifying, assessing, and analyzing 

internal/external risks and effects. One step to identify internal risk factors 

includes identifying “any potential risks due to a highly decentralized program 

operation” – a step relevant to ORD given its matrix organization. We believe the 

tool provides a sound starting point that offices can tailor as appropriate, 

particularly since EPA Order 1000.24 affords program managers flexibility in 

designing review strategies. 

ORD could also conduct benchmarking similar to what it did on efficiency 

measures for research organizations (see Chapter 1 “Noteworthy Achievements”). 

ORD finds these organizations more analogous to it and could ask for 

management integrity best practices these organizations apply. Also, four of eight 

federal agencies we reviewed separate FMFIA into two tracks – a program track 

and a financial track. ORD’s Administrative Efficiencies Project workgroup has 

recently considered developing a “scientific,” or programmatic, track, and ORD 

should thoroughly consider this approach. 

Conclusion 

FMFIA requires federal managers to improve the accountability and effectiveness 

of federal programs and operations by establishing, assessing, correcting, and 

reporting on internal controls. Internal controls are key factors in achieving 

agency missions and program results and improving accountability. We 

recognize efforts ORD has made. However, ORD has several opportunities for 

continued improvement. Through its proposed Multi-Year Program Review 

Strategy, ORD could define elements of its training program, consider all 

performance measures and peer review results for FMFIA reporting, and 

incorporate internal control best practices. By doing this, ORD will better 

accomplish FMFIA as intended – the umbrella under which ORD should form its 

internal control framework. 

Recommendations 

We recommend that the Assistant Administrator for Research and Development: 

2-1 Conduct a risk assessment using the GAO internal control standard for 

risk assessment and EPA Order 1000.24 and, based upon the results, 

develop a comprehensive risk-based program review strategy. 

2-2 Train managers and other management integrity staff on FMFIA and 

internal controls. For senior managers, offer training designed to provide 

an overall understanding on internal controls and a manager’s 

responsibilities under EPA Order 1000.24. For Management Integrity 

Coordinators, offer training designed to describe how to implement and 

report on internal controls. 

16 
09-P-0232 

2-3 Revise the Management Integrity Policy to include programmatic 

operations. The policy should include a role for National Program 

Directors, integrate performance measures, reference current FMFIA 

guidance, and include a training plan. The program should incorporate 

public sector best practices and a two-track approach to address 

administrative and programmatic elements. 

Agency Comments and OIG Evaluation 

ORD agreed with our draft report findings and concurred with our 

recommendations. ORD noted, and we agree, that the FMFIA process is not the 

only opportunity to evaluate and report on research program performance, and 

ORD’s comments provided additional information on other activities it conducts. 

ORD included in its report comments a table listing planned corrective actions 

and completion dates to address our recommendations. We believe ORD’s 

planned corrective actions address the intent of each of our recommendations. 

Appendix C includes ORD’s full response. 

17 
 

 

 

 

09-P-0232 

Status of Recommendations and 

Potential Monetary Benefits 

POTENTIAL MONETARY 

RECOMMENDATIONS BENEFITS (in $000s) 

Rec. 

No. 

Page 

No. Subject Status1 Action Official 

Planned 

Completion 

Date 

Claimed 

Amount 

Agreed To 

Amount 

2-1 

2-2 

16 

16 

Conduct a risk assessment using the GAO internal 

control standard for risk assessment and EPA Order 

1000.24 and, based upon the results, develop a 

comprehensive risk-based program review strategy. 

Train managers and other management integrity staff 

on FMFIA and internal controls. For senior managers, 

offer training designed to provide an overall 

understanding on internal controls and a manager’s 

responsibilities under EPA Order 1000.24. For 

Management Integrity Coordinators, offer training 

designed to describe how to implement and report on 

internal controls. 

O 

O 

Assistant Administrator 

for Research and 

Development 

Assistant Administrator 

for Research and 

Development 

September 

2010 

Within 12 

months of 

course 

development 

2-3 17 Revise the Management Integrity Policy to include 

programmatic operations. The policy should include a 

role for National Program Directors, integrate 

performance measures, reference current FMFIA 

guidance, and include a training plan. The program 

should incorporate public sector best practices and a 

two-track approach to address administrative and 

programmatic elements. 

O Assistant Administrator 

for Research and 

Development 

January 

2010 

O = recommendation is open with agreed-to corrective actions pending 

C = recommendation is closed with all agreed-to actions completed 

U = recommendation is undecided with resolution efforts in progress 

18 

1 
 

09-P-0232 

Appendix A 

Organization of EPA ORD 

ORD has facilities geographically located across the country, as shown in Figure A.1, with its 

headquarters in Washington, DC, and main research facilities in Ohio and North Carolina. 

Figure A.1: Location of ORD Labs, Centers, and Offices 

Cincinnati, OH 

Narragansett, RI 

Research Triangle 

Park, NC 

Athens, GA Las Vegas, NV 

Duluth, MN 

Washington, DC 

Ada, OK Gulf Breeze, FL

Corvallis, OR 

Edison, NJ 

Newport, OR 

Grosse lle, MI 

Source: ORD presentation to OIG in October 2008. 

Figure A.2 depicts ORD’s matrix structure. 

Figure A.2: ORD’s Organizational Structure 

Assistant Administrator 

for Research and Development 

Deputy Assistant Administrator for Management 

Deputy Assistant Administrator for Science 

Office of 

Administrative 

and Research 

Support 

National 

Center for 

Computational 

Toxicology 

National 

Homeland 

Security 

Research 

Center 

National 

Center for 

Environmental 

Research 

National Risk 

Management 

Research 

Laboratory 

National 

Center for 

Environmental 

Assessment 

National 

Health and 

Environmental 

Effects 

Research 

Laboratory 

National 

Exposure 

Research 

Laboratory 

Office of 

Resources 

Management 

and 

Administration 

Office of 

Science 

Policy 

Office of 

Science 

Information 

Management 

National 

Program 

Directors 

Chief of Staff EPA Science Advisor 

Office of the Science Advisor 

Source: ORD (organizational chart as of April 2009). 

19 
Descriptions of duties corresponding to ORD organizational components are in Table A.1. 

Table A.1: ORD Organizational Responsibilities 

Assistant • Signs ORD’s annual FMFIA assurance letter. 

• Provides oversight and accountability for ORD’s management integrity program 

and internal controls over program operations and financial reporting. 

• Implements the internal control framework and fosters an organizational 

environment that supports continuous awareness of internal controls at all levels. 

Administrator 

for Research 

and 

Development 

National 

Program 

Directors 

• Responsible for Multi-Year Plans that establish priorities and goals. 

• Serve as primary contacts in PART reviews/GPRA measurement. 

• Coordinate with BOSC regarding Multi-Year Plan program peer reviews. 

Lab, Center, and • Responsible for managing resources allocated to labs, centers, and offices to 

implement research in support of Multi-Year Plans. 

• Sign annual FMFIA assurance letters for their labs, centers, and offices. 

Office Directors 

Office of • Conduit between ORD’s Assistant Administrator, OCFO, and ORD labs, centers, 

and offices for the assurance letter process, including developing and providing 

management integrity guidance for the organization and consolidating annual 

assurance letters for labs, centers, and offices into ORD’s annual letter. 

Resources 

Management 

and 

Administration 

Source: ORD presentation to OIG, and OIG’s February 2009 interviews with National Program Directors. 

09-P-0232 

ORD has developed numerous Multi-Year Plans to administer key research programs and outline 

annual performance goals and associated measures. Multi-Year Plans provide an overview of 

the direction of ORD’s research, present significant research accomplishments, and communicate 

ORD's research program to stakeholders. Key research programs include: 

•	 Clean Air • Ecological Research 

•	 Human Health • Water Quality 

•	 Human Health Risk Assessment • Global Change 

•	 Drinking Water • Land 

•	 Pesticides and Toxic Substances 

(Safe Pesticides/Safe Products)
 

Several years ago, ORD began to focus on the importance of independently confirming that it 

conducts the right science and does it well. Concurrently, OMB indicated, in conjunction with 

PART, the importance of independent expert reviews of federal research programs. ORD 

instituted BOSC reviews of its programs and aligned them to meet the structure of PART. In 

2006, to improve its external review process and better ensure the relevancy and credibility of its 

research programs and science, ORD developed three specific charge questions for use in 

BOSC’s summary assessment of each research program’s long-term goals: 

1.	 Relevance: How appropriate is the research used to achieve each long-term goal? 

Is the program still asking the right questions, or have they been superseded by 

advancements in the field? 

2.	 Quality: How good is the technical quality of the program’s research products? 

3.	 Performance: How much are the program results being used by environmental 

decision-makers to inform decisions and achieve results? 

20 
09-P-0232 

Appendix B 

Details on Scope and Methodology 

We conducted our audit to determine how ORD implements FMFIA. During our audit, we 

identified concerns with ORD’s implementation of internal control standards prescribed by the 

Comptroller General as required by Section 2 of FMFIA. Our findings only address ORD’s 

implementation of Section 2 of FMFIA (internal control over programs), and not Section 4 

(financial accounting systems) or Appendix A of OMB’s Circular A-123 (internal control over 

financial reporting). Our audit focused on ORD’s headquarters office, in Washington, DC, and 

its NHEERL facilities in Raleigh, North Carolina, and Corvallis and Newport, Oregon. 

NHEERL is ORD’s largest laboratory in terms of its budget and number of personnel employed. 

To address our first objective on whether ORD had a systematic strategy to establish, review, 

and monitor internal controls, we did the following: 

•	 Gathered and analyzed FMFIA regulations, policies, and guidance related to GAO’s 

Standards for Internal Control in the Federal Government, and OMB Circular A-123. 

•	 Gathered and analyzed EPA and ORD policies, procedures, guidance documents, and 

budget data related to FMFIA implementation, including EPA Order 1000.24. 

•	 Attended briefings by ORD managers regarding ORD’s organization, resource 

utilization, annual planning, approach to FMFIA implementation, systematic strategy for 

reviewing internal controls, near- and long-term laboratory studies, and the review 

process used by BOSC. 

•	 Conducted site visits to five NHEERL facilities (three collocated in Raleigh, North 

Carolina, and one each in Corvallis and Newport, Oregon) and attended briefings on 

organization, resource utilization, annual planning, and FMFIA implementation. (Site 

visits in Raleigh also included tours of several other ORD laboratories.) 

•	 Reviewed ORD’s, NHEERL’s, and ORD’s Office of Science Policy FMFIA assurance 

letters to determine whether they addressed all five GAO standards as specified in 

OCFO’s FY 2008 guidance. We also reviewed letters to determine whether ORD and 

NHEERL documented and used program review results to establish and assess the 

effectiveness of internal controls. 

•	 Participated in OCFO conference calls and interviewed OCFO staff to understand the 

FMFIA process, particularly concerns regarding programmatic review elements. 

•	 Interviewed ORD’s seven Management Integrity Coordinators and their supervisors, and 

ORD’s eight National Program Directors about roles and responsibilities in implementing 

ORD’s FMFIA process, focusing on FMFIA time and training requirements and needs. 

•	 Developed summary working papers on each set of interviews to obtain quantitative data. 

•	 Identified and analyzed program reviews of ORD research programs for FY 2007 and 

2008 to determine the extent review questions, results, and recommendations addressed 

the five GAO standards. 

•	 Conducted interviews with ORD and NHEERL staff and managers on reasons for including 

and excluding certain information from the assurance letter development process. 

21 
09-P-0232 

To address our second objective on what ORD's internal control strategy should contain to 

account for risks in meeting program goals, we did the following: 

•	 Flowcharted ORD’s calendar of external reviews to determine the number of reviews 

conducted annually. 

•	 Benchmarked FMFIA assurance letters and policies used by other federal agencies to 

determine best practices ORD could use in its own systematic strategy. 

•	 Conducted follow-up interviews with OCFO staff on their understanding of the internal 

control review strategy as required by EPA Order 1000.24. 

•	 Reviewed internal control review strategies from other EPA program offices. 

•	 Benchmarked other sample risk assessment methodologies available on-line and 

reviewed how others established controls based upon the Council of Sponsoring 

Organizations requirements. We also contacted GAO for sample methodologies. 

•	 Obtained and reviewed ORD’s draft strategy to determine any improvement areas and/or 

whether it affected our strategy recommendations. 

•	 Determined the effect of not developing a review strategy by documenting the 

relationship between EPA Order 1000.24 and OCFO assurance letter guidance, 

determining how assurance letters could have referenced prior internal/external reviews 

to demonstrate compliance with internal control standards, and reviewing 

internal/external reviews and how ORD and NHEERL might redirect review resources. 

We did not find any prior audits or evaluations of ORD’s implementation of FMFIA. 

22 
 

09-P-0232 

Appendix C 

Agency Response to Draft Report 

September 4, 2009 

MEMORANDUM 

SUBJECT: ORD Response to OIG Draft Report EPA’s Office of Research and Development 

Could Better Use the Federal Managers’ Financial Integrity Act to Improve 

Operations Project No. OA-FY08-0323 

FROM: Lek G. Kadeli/s/ 

Acting Assistant Administrator (8101R) 

TO: Patrick Gilbride 

Director, Risk and Program Performance Audits (801G) 

This memorandum responds to the Office of Inspector General (OIG) draft audit report, 

EPA’s Office of Research and Development Could Better Use the Federal Managers’ Financial 

Integrity Act (FMFIA) to Improve Operations (Project No. OA-FY08-0323), dated August 6, 

2009. The recommendations provided in the report will help the Office of Research and 

Development (ORD) continue to improve its FMFIA process. 

As the scientific research and assessment arm of EPA, ORD maintains a strong 

management integrity program that systematically reviews and assesses the effectiveness of 

internal controls consistent with GAO Standards and OMB Circular A-123. As required by the 

Federal Managers’ Financial Integrity Act (FMFIA), we annually evaluate our internal controls 

over programs and administrative systems and provide assurance on the integrity of our controls. 

ORD is committed to ensuring that our science is of the highest quality, our programs are 

managed effectively and efficiently, and that we aggressively prevent fraud, waste, and abuse. 

In contrast to the report’s conclusion, the FMFIA process is not the only “opportunity to 

evaluate and report on research program performance.” As you correctly noted in the report, 

ORD has “focused on the importance of independently confirming that it conducts the right 

science and does it well.” ORD instituted a strategic multi-year planning process to guide the 

direction of ORD’s research to focus on EPA's highest priority needs for science and promote 

coordination of research across laboratories, centers and offices to achieve its goals. ORD has 

engaged other agencies and scientific experts in an effort to determine the most effective 

approach(es) to evaluate and measure the efficiency of its research programs through reviews by 

Board of Scientific Counselors, Science Advisory Board, and the National Academy of Sciences; 

quality assurance programs, which include peer reviews and self inspections; and Government 

Accountability Office and OIG audits. Thus far in FY 2009, more than 70 reviews of ORD 

23 
09-P-0232 

programs, functions and operations have been completed. Based on the results of these reviews, 

we are continually improving the science and research we provide to the Agency. 

The OIG provides three recommendations to strengthen ORD’s FMFIA process. In 

general we agree with the recommendations and I am pleased to say that ORD has been actively 

working on revisions to its FMFIA process. ORD will continue to include information in its 

assurance letter that it deems to be of significant importance to the Administrator. ORD remains 

committed to management integrity and maintaining effective internal controls throughout our 

organization. 

Attached please find: (1) our response to each of the three recommendations contained in 

the draft report and 2) a summary table of ORD’s corrective actions and associated projected 

completion dates. If you have any questions, please contact me or Deborah Heckman at (202) 

564-7274. 

Attachment 

cc: Donna Vizian 

Hal Zenick 

 Amy Battaglia 

 Jim Morant 

 Deborah Heckman 

24 
 

09-P-0232 

ORD Response to OIG Recommendations Contained in Draft Report 

“EPA’s Office of Research and Development Could Better Use the 

Federal Managers’ Financial Integrity Act to Improve Operations” 

Project No. OA-FY08-0323 

August 6, 2009 

Recommendation 2-1 - Conduct a risk assessment using the GAO internal control standard for 

risk assessment and EPA Order 1000.24 and, based upon the results, develop a comprehensive 

risk-based program review strategy. 

Response: ORD generally agrees with this recommendation. 

Recognizing the complexity of conducting a comprehensive risk assessment6

 for a research 

organization, ORD is developing an ORD-wide approach to the risk assessment. By December 

2009, ORD senior leaders will be designated to serve on ORD Executive Assessment Team 

(ORDEAT) to: ensure consistency in ORD’s corporate approach to internal controls; review 

internal control information in order to make corporate decisions; concur on the ORD three-year 

program and management review schedule; and make recommendations to the DAA for 

Management and AA regarding the ORD high risk areas. By August 2010, ORD will review its 

processes, test key internal controls related to ORD activities, and assess programmatic and 

administrative risks. By September 2010, after completing the risk assessment activities, ORD 

will revise its multi-year program review strategy as necessary. 

Recommendation 2-2 - Train managers and other management integrity staff on FMFIA and 

internal controls. For senior managers, offer training designed to provide an overall 

understanding on internal controls and a manager’s responsibilities under EPA Order 1000.24. 

For Management Integrity Coordinators, offer training designed to describe how to implement 

and report on internal controls. 

Response: ORD generally agrees with this recommendation. 

ORD is committed to training managers and employees involved with administering ORD’s 

management integrity program. However, OCFO agreed to “complete development of an 

Agency-wide strategy for comprehensive, tiered FMFIA training by the end of fiscal year 2009” 

in its July 16, 2009 response to the OIG draft audit report titled EPA Should Use FMFIA to 

Improve Programmatic Operations (Project No. 08-FY08-0323). In order not to duplicate 

OCFO’s efforts, ORD will collaborate with OCFO on developing and implementing an Agencywide training program which ensures compliance with FMFIA and proper reporting of internal 

controls. ORD will assess the applicability of the newly developed training for senior ORD 

managers and, if necessary, initiate additional course development. ORD will then ensure that 

its managers and integrity staff are trained within 12 months of completion of the course 

development. 

6

 As defined by GAO Standards for Internal Control in The Federal Government 

25 
09-P-0232 

Recommendation 2-3 - Revise the Management Integrity Policy to include programmatic 

operations. The policy should include a role for National Program Directors, integrate 

performance measures, reference current FMFIA guidance, and include a training plan. The 

program should incorporate public sector best practices and a two-track approach to address 

administrative and programmatic elements. 

Response: ORD generally agrees with this recommendation. 

By January 2010, ORD will revise the ORD Management Integrity Policy to include 

programmatic operations, appropriate integration of performance measures and outcomes and 

reference current FMFIA guidance. As recommended, ORD will devise a two-track approach to 

address administrative and programmatic elements as required by GAO and Agency guidance. 

The new policy will define the roles of management and will include National Program Directors 

responsibilities or other matrix managers we may have in the future under ORD’s programmatic 

operations. The Management Integrity Policy will reference Agency training requirements for 

all managers and ORD integrity staff. 

26 
09-P-0232 

ORD Corrective Actions and Projected Completion Dates 

Rec. 

No. 

OIG Recommendation Lead 

Responsibility 

ORD Corrective Action Planned 

Completion 

Date 

2-1 

Conduct a risk assessment using the 

GAO internal control standard for 

risk assessment and EPA Order 

1000.24 and, based upon the results, 

develop a comprehensive risk-based 

program review strategy 

Assistant 

Administrator 

for Research 

and 

Development 

ORD is currently finalizing a strategy that 

examines and reports on internal controls 

covering programmatic and administrative 

operations and financial activities. Once 

finalized, ORD’s multi-year program 

review strategy will help ORD identify 

high-risk areas, detect weaknesses and 

deficiencies, and identify best practices in 

our internal controls. 

September 

2010 

2-2 

Train managers and other 

management integrity staff on 

FMFIA and internal controls. For 

senior managers, offer training 

designed to provide an overall 

understanding on internal controls 

and a manager’s responsibilities 

under EPA Order 1000.24. For 

Management Integrity 

Coordinators, offer training 

designed to describe how to 

implement and report on internal 

controls. 

Assistant 

Administrator 

for Research 

and 

Development 

Collaborate with OCFO 

Within 12 

months of 

Course 

Development 

2-3 

Revise the Management Integrity 

Policy to include programmatic 

operations. The policy should 

include a role for National Program 

Directors, integrate performance 

measures, reference current FMFIA 

guidance, and include a training 

plan. The program should 

incorporate public sector best 

practices and a two-track approach 

to address administrative and 

programmatic elements 

Assistant 

Administrator 

for Research 

and 

Development 

ORD will revise the ORD Management 

Integrity Policy to include programmatic 

operations, a definition of the National 

Program Directors’ role in the process and 

integration of performance measures and 

outcomes. 

January 2010 

27 
09-P-0232 

Appendix D 

Distribution 

Office of the Administrator 

Acting Assistant Administrator, Office of Research and Development 

Agency Follow-up Official (CFO) 

Agency Follow-up Coordinator 

General Counsel 

Associate Administrator for Congressional and Intergovernmental Relations 

Associate Administrator for Public Affairs 

Audit Follow-up Coordinator, Office of Research and Development 

Audit Follow-up Coordinator, Office of the Chief Financial Officer 

Acting Inspector General 

28