Grants Management: EPA Could Improve Certain Monitoring Practices
Released: 2016-08-15T00:00:00

Letter

Background

EPA provides financial assistance to a variety of recipients, including states, tribes, and nongovernmental organizations, through assistance agreements such as grants. EPA awards these grants to recipients to meet local environmental priorities and national objectives established in federal law, regulations, or EPA policy. As we have previously reported, most federal grant-making agencies, including EPA, generally follow a life cycle comprising various stages—preaward (announcement and application), award, implementation, and closeout—for awarding grants, as seen in figure 1.

The federal laws establishing EPA’s grant programs generally specify the types of activities that can be funded, objectives to be accomplished through the funding, and who is eligible to receive the funding. In addition to these statutory requirements, EPA has issued regulations governing its grants, which may impose additional requirements on recipients. EPA either notifies the public of the grant opportunity or notifies eligible state agencies about available grants, and applicants must submit applications to the agency for its review. In the preaward stage, EPA reviews applications to determine or verify which meet eligibility requirements and awards funding. EPA assigns project officers—who manage the technical and program-related aspects of the grants—and grant specialists—who manage the administrative aspects of grants—in program and regional offices to oversee the implementation stage of the grants.

The implementation stage includes development of a grant work plan that outlines EPA and grantee agreed-upon goals, objectives, activities, and time frames for completion under the grant, such as developing certain water quality standards by the end of the year. It also includes payment processing, agency monitoring, and grantee reporting on the results of its individual grant as well as its contribution to program results. For example, results for an individual water quality grant might include the grantee using funds to develop water quality standards, whereas program results might include the grantee’s contribution to the number of water quality permits issued under the program as a whole. Grantees submit information on grant results to EPA through performance reports and progress reports, depending on the grant program. The closeout phase includes preparation of final reports, financial reconciliation, and any required accounting for property.

EPA Awards Different Categories of Grants

EPA generally awards three types of grants:

Formula grants. EPA awards these grants noncompetitively to states in amounts based on formulas prescribed by law to support water infrastructure projects, among other things. For example, grants from the Clean Water and Drinking Water State Revolving Funds can be used to support infrastructure, such as water treatment facility construction, and improvements to drinking water systems, such as pipelines or drinking water filtration plants. According to EPA data, in fiscal year 2015, EPA awarded about $2.25 billion of $3.95 billion (about 57 percent) of grant funds as formula grants, as shown in figure 2.

Categorical grants. EPA generally awards these grants—which EPA also refers to as continuing environmental program grants— noncompetitively, mostly to states and Indian tribes to operate environmental programs that they are authorized by statute to implement. For example, under the Clean Water Act, states and tribes can establish and operate programs for the prevention and control of surface water and groundwater pollution. EPA determines the amount of funding each grantee receives based on agency-developed formulas or program-specific factors. In fiscal year 2015, EPA awarded about $1.09 billion of $3.95 billion (about 28 percent) of grant funds as categorical grants, according to EPA data.

Discretionary grants. EPA awards these grants—competitively or noncompetitively—to eligible applicants for specific projects, with EPA program and regional offices selecting grantees and funding amounts for each grant. EPA primarily awards these grants to states, local governments, Indian tribes, nonprofit organizations, and universities for a variety of activities, such as environmental research, training, and environmental education programs. According to EPA data, in fiscal year 2015, EPA awarded about $0.513 billion of $3.95 billion (about 13 percent) of grant funds as discretionary grants.

EPA also awarded $0.09 billion of $3.95 billion (about 2 percent) of grant funds to special appropriations act projects for specific drinking water and wastewater infrastructure projects in specific communities.

Multiple Federal and EPA Requirements and Guidelines Apply to Monitoring Grant and Grant Program Results

Multiple federal and EPA requirements—established in laws and regulations—and EPA guidelines apply to monitoring the results of individual EPA grants and, more broadly, the results of EPA grant programs.

The following requirements and guidelines form the basis of how EPA aligns individual grants to achieve the agency’s public health and environmental objectives:

Federal laws: Authorizing statutes for certain EPA grant programs, most notably the Clean Water Act, require states—which receive grants from EPA to capitalize state clean water revolving funds—to report annually to EPA on how they have met the goals and objectives identified in their intended use plans for their revolving funds.

EPA regulations: EPA regulations require grantees to submit performance reports to EPA as specified in their grant agreements at least annually and typically no more frequently than quarterly. Under EPA’s regulations, the grantee’s performance should be measured in a way that will help improve grant program outcomes, share lessons learned, and spread the adoption of promising practices. Additionally, under EPA’s regulations, the agency should provide grantees with clear performance goals, indicators, and milestones, and should establish reporting frequency and content that allow EPA to build evidence for program and performance decisions, among other things.

Agency-wide policies and guidance: EPA policies, such as its environmental results directive, call for grant work plans and performance reports to link to the agency’s strategic plan and include outputs and outcomes. The environmental results directive, the Policy on Compliance, Review, and Monitoring, and related guidance also call for EPA program officials to review interim and final performance reports—or for certain programs, use a joint evaluation process—to determine if the grantee achieved the planned outputs and outcomes, and document the results of these reviews in EPA’s grants management databases. Additionally, the environmental results directive calls for EPA program offices to report on significant grant results through reporting processes established by national program managers, such as data submissions to EPA databases.

Program-specific guidance: EPA program offices provide biennial guidance on each program’s priorities and key actions to accomplish health and environmental goals in EPA’s strategic plan. According to EPA officials, this guidance includes annual commitment measures, which guide implementation with EPA regions, states, tribes, and other partners. Many annual commitment measures include regional performance targets, which contribute to meeting EPA annual budget measures, and in turn, long-term strategic measures, according to EPA officials. EPA regional offices use these performance measures and targets to guide their negotiations with grantees on individual grant work plan outputs and outcomes.

Grant-specific requirements: EPA incorporates requirements related to grantee reporting frequency, content, and reporting processes (i.e., written performance report, data submissions to an EPA database, or both) into individual grant terms and conditions as part of the final grant agreement. EPA and grantees also negotiate grant-specific outputs and outcomes, which grantees incorporate into their grant work plans.

EPA Monitors Performance Reports and Data to Ensure Grants Achieve Results, but Certain Practices Hinder EPA’s Ability to Efficiently Monitor Results

EPA monitors performance reports and program-specific data from grantees to ensure that grants achieve environmental and other program results, but certain practices hinder EPA’s ability to efficiently monitor some results. In addition, we identified a variety of monitoring issues that may hinder EPA’s ability to efficiently identify factors affecting grantee results.

EPA Monitors Written Performance Reports and Program-Specific Numeric Data to Ensure Grants Achieve Results

According to EPA policies and officials, after EPA approves grantee work plans that identify agreed-upon environmental and other results for each grant, grantees generally report information on their progress and grant results to EPA in two ways: (1) submitting performance reports— generally written—that describe the grantees’ progress toward the planned grant results in their work plans, such as using grant funds to provide technical assistance to local officials, and (2) electronically submitting program-specific data—generally numeric—on certain program measures, such as the number of hazardous waste violations issued, which EPA tracks in various program databases. According to an EPA official, the information streams from grantees differ in that the performance reports go to EPA project officers for the purpose of managing individual grants, whereas EPA program managers use the electronic data to monitor regional and program progress on EPA’s performance measures.

Performance reports. At least annually, grantees are to submit performance reports to EPA as specified in their grant agreements. EPA policies include general guidelines about what performance reports should include, such as a comparison between planned and actual grant results, but allow the frequency, content, and format of performance reports to vary by program and grant. For more information on the performance reports we reviewed, see appendix II.

According to EPA officials, EPA project officers monitor these reports to review grantee progress toward agreed-upon program results, such as providing outreach to communities about hazardous waste. Project officers conduct two types of routine grants monitoring: (1) baseline monitoring, which is the periodic review of grantee progress and compliance with a specific grant’s scope of work, terms and conditions, and regulatory requirements, and (2) advanced monitoring, which is an in- depth assessment of a grantee or a project’s progress, management, and expectations. EPA assigns a certain number of advanced monitoring reviews to each regional and program office annually. In 2015, OGD assigned program and regional offices to perform advanced monitoring for at least 10 percent of their active grantees, which program and regional offices select based on criteria such as the size of the grant and the experience level of the grantee, among others.

EPA project officers document the results of their monitoring—for example, whether grantees have made sufficient progress and complied with grant terms and conditions—in EPA’s grants management databases at least annually. Based on their baseline monitoring review, EPA project officers may impose more frequent or intensive grant monitoring, such as advanced monitoring, to address any identified concerns. According to EPA data, project officers recommended additional grant monitoring for 78 out of 2,987 reviews (about 3 percent) in 2015. Additionally, program and regional offices summarize any significant grants management- related observations or trends from their advanced monitoring reviews as part of their annual postaward monitoring plans.

Program-specific information. According to program officials, grantees also electronically submit program-specific information—generally numeric data—on certain results, such as the acres of brownfield properties made ready for reuse. According to EPA policy and program officials, program officials monitor these data to track and report program accomplishments, at the regional and agency levels, and, as applicable, to assess the agency’s progress meeting its performance measure targets in support of agency strategic goals. According to EPA officials, generally grantees or EPA program officials—depending on the database—are to enter grant results, such as the number of enforcement actions, into EPA’s program-specific data systems at agreed-upon intervals, such as quarterly. These requirements may be part of a grant’s terms and conditions.

EPA Performance Measures and Data Systems The number of performance measures and data systems that the Environmental Protection Agency (EPA) uses to collect and analyze data on environmental and other program results in 2016—including incorporating performance data from grantees as relevant—varies across the three program offices we reviewed. For example, the Office of Water collects or analyzes grantee data on results for 13 of its 15 grant programs using 20 data systems, and integrates the results as appropriate into its reporting on 111 annual commitment measures. The Office of Land and Emergency Management collects or analyzes grantee data on results for 10 of its 13 grant programs using 4 systems, and integrates the results as appropriate into its reporting on 34 annual commitment measures. The Office of Air and Radiation collects or analyzes grantee data on results for 7 of its 9 grant programs using 3 systems, and integrates the results as appropriate into its reporting on 54 annual commitment measures.

According to EPA officials, there is not always a direct link between individual grantee results and EPA’s annual budget and annual commitment performance measures. However, officials told us that each regional or program office considers information from its program- specific data systems that is relevant to program- or agency-level performance measures, interprets it, and enters the results as appropriate into EPA’s national performance tracking systems. For example, Office of Water officials use data collected from grantees in its Drinking Water National Information Management System database to report annually in EPA’s national performance tracking system the number of Drinking Water State Revolving Fund projects that have started operations.

EPA officials said that reporting grant and program results to EPA has improved over time, as EPA has transitioned from collecting data in hard copy and expanded electronic reporting by grantees. Additionally, officials we spoke with from several states said that electronic reporting had certain benefits. EPA officials told us that collecting certain information electronically from grantees allows EPA to access and analyze grant and program results more efficiently than it can for results collected in a written format, because EPA officials do not have to manually enter information into a data system for analysis. Additionally, in response to information-sharing problems—such as incompatible computer systems, manual data entry, and differing data structures across program offices— EPA and the Environmental Council of States formed the Environmental Information Exchange Network (Exchange Network) in 1998, an information-sharing partnership that uses a common, standardized format so that EPA, states, and other partners can share environmental data across different data systems. As a result, EPA and its partners may access and use environmental data more efficiently, according to Exchange Network documents. For example, officials we interviewed from each of the eight state environmental agencies we reviewed said that they use the information they collect for EPA to either manage their programs or inform the public. Additionally, even with some technical issues with individual databases, officials from six of these eight agencies said that electronic reporting has several benefits, such as improving data timeliness, greater efficiency, and reduced administrative burden.

Furthermore, based on our review of agency policy, analysis, and planning documents, we found that current and past EPA initiatives have taken steps to reduce the reporting burden on grantees and others. For example:

Since 1996 EPA has been authorized to issue performance partnership grants, which allow states, Indian tribes, interstate agencies, and intertribal consortia grantees to combine funds from certain EPA grant programs into a single grant. EPA designed this system to provide grantees with greater flexibility to address their highest environmental priorities and reduce administrative burden and costs, among other objectives. In 2015, EPA issued a policy to increase awareness and encourage the use of these grants.

In 2008, EPA issued a policy to reduce reporting burdens for states awarded grants under 28 grant programs by establishing general frequencies for grant work plan progress reports and specifying that EPA regional offices could only require more frequent progress reports in certain circumstances.

In 2012, EPA’s OGD contracted with external experts to review its grants management processes and identify improvements as part of EPA’s Grants Business Process Reengineering Initiative. This initiative seeks to streamline and standardize the grants management process at EPA and develop an improved business process to be implemented through EPA’s new grants management data systems. The study identified several potential high-level improvements, such as reducing manual activities and expanding standardization in documents to ensure greater consistency and reduce administrative burden.

In 2013, EPA and states established a leadership council for E- Enterprise for the Environment—a joint initiative to streamline and modernize business processes shared between EPA and regulatory partners, such as states, and reduce reporting burden on regulated entities, among other goals. For example, in 2015, EPA and states initiated the Combined Air Emissions Reporting project, which seeks to streamline multiple emissions reporting processes at the federal, state, and local levels, according to EPA’s website. The project will establish a single, authoritative data repository that will reduce the industry and government transaction costs for reporting and managing emissions data through features such as autopopulated forms and data sharing across regulatory agencies.

In 2015, EPA finalized an electronic reporting rule that requires, among other things, states that receive grants to issue National Pollution Discharge Elimination System permits to substitute electronic reporting for paper-based reports, saving time and resources for states, EPA, and permitted facilities. According to an EPA economic analysis, when fully implemented, the new rule will eliminate 900,000 hours of reporting across regulated entities and state agencies. According to EPA’s fiscal year 2017 budget, the agency plans to further reduce the reporting burden by 1 million hours by the end of fiscal year 2017.

In 2016, EPA’s OGD issued its 2016-2020 Grants Management Plan, which includes several streamlining efforts specific to grants. For example, under Goal 2: Streamline Grants Management Procedures, EPA plans to evaluate its grants management processes and assess opportunities to streamline its procedures. Under this goal, EPA also plans to provide a mechanism for staff to submit feedback about existing burdens and new requirements or procedures. Furthermore, under Goal 4: Ensure Transparency and Accountability and Demonstrate Results, EPA plans to improve its process for monitoring grants and will collect input from external stakeholders, such as states and grantees, about how to address burdens.

Certain EPA Monitoring Practices Hinder EPA’s Ability to Efficiently Monitor Some Results and May Increase EPA’s and Grantees’ Administrative Burden

Based on our review of the three program offices that award the majority of EPA grant funding, we found that certain EPA monitoring practices in these offices hinder EPA’s ability to efficiently monitor some results and may increase EPA’s and grantees’ administrative burden. First, EPA collects a variety of information about grant results, but some of the information is not readily accessible. Second, EPA collects certain information from grantees twice, once in a written report and once in an electronic database. Third, one program office transfers data relevant to its annual performance measures from its program-specific databases to EPA’s national database manually rather than electronically. EPA officials and officials from several state environmental agencies who we interviewed said that these practices increase their administrative burden.

EPA Collects a Variety of Information about Grant Results, but Some of the Information Is Not Readily Accessible

EPA collects a variety of information about grant results through grantee performance reports and program-specific databases. However, some of the information was not readily accessible to project officers or grantees. Based on our review of performance reports across 23 grant programs, we found that the types of results that grantees reported, such as data collection and management, covered a variety of topics and were generally similar across programs, as shown in table 2. Additionally, we found that grantees electronically report a variety of information about grant results to program-specific databases, such as enforcement actions and environmental benefits of water infrastructure projects. However, only some of the information reported by grantees was readily accessible, either to the public through user-defined searches on EPA’s website or to grantees through accessing an EPA database directly. This is because the information in grantees’ performance reports is stored as file attachments to database records and EPA’s legacy grants management databases do not have the capability to search data stored in this format. For instance, a program manager that wanted to obtain information on the number and types of training activities funded by a particular grant program—and that are not reported to a program-specific database—would need project officers to open each performance report individually and manually review it for relevant information. OGD officials told us that—depending on the availability of funds—they plan to develop a web-based portal for grantees to submit documents, including their performance reports, centrally as part of their new grants management database.

Under EPA’s regulations, grantee performance should be measured in a way that will help improve grant program outcomes, share lessons learned, and spread the adoption of promising practices. EPA has procedures in place to collect this information through its program-specific databases and performance reports. However, we have previously found that for performance information to be useful, it should meet users’ needs for consistency, relevance, accessibility, and ease of use, among other attributes. EPA’s 2014 internal analysis of its grants management business processes identified improvements that if implemented into EPA’s planned web-based portal, could improve the accessibility and usefulness of information in grantee performance reports for EPA, grantees, and other users. For example, the analysis found that incorporating expanded search capabilities into EPA’s new grants management database, such as keyword searches, could improve users’ access to relevant information. However, it is unclear to what extent, if at all, these features will be applied to the web-based portal because the high-level analysis does not specify how performance reports will be stored and accessed through the web-based portal.

Because EPA, grantees, and other users cannot readily access information in performance reports about grant results and how different grantees achieve them, these reports are less useful for sharing lessons learned and building evidence for demonstrating grant results. Making the information that EPA collects in these reports more accessible by incorporating expanded search capability features, such as keyword searches, into its proposed web-based portal for collecting and accessing performance reports, could improve its usefulness to EPA and grantees in identifying successful approaches to common grantee challenges. Additionally, improved accessibility could facilitate EPA’s ability to assess and report environmental and program results achieved through its grants by reducing the need to manually open and review each performance report to identify relevant information.

EPA Collects Certain Information from Grantees Twice

EPA collects certain information from grantees twice—once in a written report and once in an electronic database—and in some cases, we found varying degrees of overlap between the content of the performance reports and program-specific databases that we reviewed. Specifically, of the performance reports we reviewed across 23 grant programs, we found that one or more grantee performance reports included information that grantees also report to EPA through a program-specific database for 12 programs, as shown in table 3. For 10 of these programs, the content in 15 of the performance reports we reviewed had some overlap with data submitted through relevant program-specific databases, and for 5 of the programs, 12 reports we reviewed had substantial overlap. For more information on the program-specific databases we reviewed, see appendix II.

Additionally, officials we interviewed from five of the eight state environmental agencies we reviewed confirmed that under current reporting requirements, they reported the same information to EPA twice—once electronically and once in a written performance report, which increased their administrative burden. Specifically, these state officials provided the following examples:

Much of grantee reporting for the Clean Water State Revolving Fund—information reported electronically to EPA—is also reported separately in the written state revolving fund annual performance report.

Grantees report the same activities in the Public Water System Supervision program that they report separately to EPA’s state revolving fund databases for the state program set-asides, funded by the Drinking Water State Revolving Fund.

Under the State Hazardous Waste Management Program, EPA calls for grantees to include permitting, compliance, enforcement, and corrective action activities and accomplishments—already reported to EPA electronically—in their performance reports.

Because of different programmatic and reporting needs for water program grants, officials often find themselves reporting the same data multiple times in different formats.

Grantees submit data on actions to address nonpoint source pollution to EPA electronically throughout the year—which grantees also report separately to EPA in the annual performance reports for Nonpoint Source Pollution Grants, as required by the Clean Water Act.

Officials we interviewed from five of the eight state environmental agencies said that EPA could work with states to evaluate how grantees report and further streamline reporting and data collection. Officials we interviewed from one state agency said that with limited resources, they have no capacity for additional reporting requests, without some modification to reporting schedules or simplification of the reporting process.

According to EPA officials, EPA’s reporting process has evolved over time in response to statutory changes, such as amendments to the Government Performance and Results Act of 1993—which generally requires that agencies develop performance goals that are expressed in objective, quantifiable, and measurable form and annually report on their performance in meeting those goals. Additionally, to facilitate grantees’ timely reporting and access to environmental data, EPA and its partners have expanded electronic reporting to program-specific databases through the Exchange Network data-sharing partnership with states and others, according to EPA and Exchange Network documents. Furthermore, EPA officials told us that collecting information in both written performance reports and program-specific databases is beneficial because the information serves different purposes. Specifically, EPA officials said that performance reports are designed to provide project officers with information in the format they need for monitoring grantee progress, for example, narrative information on grantee activities to achieve results. Similarly, program-specific databases are designed to provide program managers with information in the format they need for monitoring program progress, for example, information that will allow them to report national-level results. However, officials from two of the three program offices we reviewed said that project officers either currently used, or could use, data within some program-specific databases to help monitor grantee progress.

Because EPA collects certain information in both performance reports and program-specific databases for 12 of the programs we reviewed, some grantees have an increased administrative burden, which may result in fewer resources dedicated to activities that directly protect human health and the environment. Our prior work and EPA analyses of its business processes have shown that duplication of efforts can increase administrative costs and reduce the funds available for other priorities. By identifying grant programs where existing program-specific data reporting requirements can meet EPA’s performance reporting requirements for grants management purposes, the agency can help reduce duplicative reporting for grantees in a manner consistent with EPA’s ongoing streamlining efforts.

One Program Office Transfers Relevant Program Results Data from Its Databases to EPA’s National Database Manually

Because one program office we reviewed, the Office of Water, transfers certain data relevant to program results from its program-specific databases to EPA’s national database manually, this office does not benefit from greater data quality control, accessibility, and administrative efficiencies reported by another program office that electronically transfers data relevant to program results. Specifically, the Office of Land and Emergency Management transfers data relevant to most of its annual commitment measures from its program-specific databases to EPA’s national database electronically, using EPA’s Performance Assessment Tool business intelligence software. According to Office of Land and Emergency Management officials, the software provides several advantages to manual data transfer, including improved accuracy, efficiency, the ability to trace data between the different data systems, and improved data accessibility for EPA program managers.

In contrast, the Office of Water manually transfers data relevant to its annual commitment measures from its program-specific data systems to EPA’s national performance database—the Budget Automation System— using a spreadsheet. According to Office of Water officials, they are not currently planning to develop the capability to transfer data electronically because EPA is in the process of replacing its Budget Automation System with a new system. Instead, these officials said that the office is using other technology tools—such as collaboration software—to make the data transfer within EPA more efficient and reduce errors. However, an Office of Water official acknowledged that the quality assurance process for data transferred manually is lengthy.

Standards for Internal Control in the Federal Government states that control activities can be implemented in either an automated or a manual manner but that automated control activities tend to be more reliable because they are less susceptible to human error and are typically more efficient. Furthermore, EPA planning documents and analyses demonstrate the potential benefits of improving efficiency in government operations by using automated control activities, such as reduced administrative burden and cost savings. However, by transferring data from its program-specific databases to EPA’s agency-wide system manually, the Office of Water does not benefit from the greater data quality control, accessibility, and administrative efficiencies available from electronic transfer of data. By adopting software tools, as appropriate, to electronically transfer relevant data on program results from program- specific databases to EPA’s new national performance system, the Office of Water could reduce its administrative burden.

Our Review of Performance Reports Identified a Variety of Monitoring Issues That May Hinder EPA’s Ability to Efficiently Identify Factors Affecting Grantee Results

Our review of 49 written performance reports across 23 grant programs identified a variety of monitoring issues related to EPA’s environmental results directive. First, we found that project officers may interpret EPA’s environmental results directive differently because the directive is unclear. Second, in some cases, grantees did not include references to the agreed-upon outputs and outcomes from their work plan to demonstrate progress achieving planned results. Third, because grantees submit performance reports in a written format, there are no built-in quality controls to ensure these reports’ consistency with EPA’s directive. Each of these issues may have contributed to the inconsistencies we found in the reports we reviewed. Inconsistencies in grantee reports may make it more difficult for EPA project officers to efficiently identify or report patterns in factors affecting grantee’s achievement of their agreed-upon results.

Project Officers May Interpret EPA’s Environmental Results Directive Differently Because the Directive Is Unclear

We found that individual project officers may be interpreting EPA’s environmental results directive differently because the directive is unclear. Specifically, we found that reports’ consistency with the directive varied by grantee and across some of the grant programs we reviewed. One reason for these variations may be that project officers have different interpretations of EPA’s directive, as the directive does not provide specific criteria for evaluating performance reports’ consistency.

EPA’s environmental results directive establishes EPA’s policy to ensure that grant outputs and outcomes are appropriately addressed in grantee performance reports, to the maximum extent practicable. Specifically, it calls for program offices to review performance reports and determine whether the grantees achieved the environmental or other outputs and outcomes in their grantee work plans, which includes assessing whether grantee explanations for unmet outputs or outcomes are satisfactory. According to the directive, the results of this review should be included in EPA’s official project file for each grantee.

However, the directive does not specify what factors the project officers who manage grants should consider when determining whether the grantees’ addressing of outputs and outcomes in their performance reports is appropriate. Based on our review of performance reports, we found that the level of detail in grantees’ descriptions of how they addressed grant outputs and outcomes varied across the reports we reviewed. For example, some grantees reported completing or providing training activities without including additional information on the topic, date, or number of attendees. In contrast, other grantees provided specific information on training, such as which employees attended training, the various courses, and dates of classes.

Similarly, the directive does not specify what factors project officers should consider when determining whether a grantee’s explanation for an unmet output or outcome in a performance report is satisfactory. For example, we found that 17 of 49 (about 35 percent) grantee performance reports were consistent with EPA’s directive because they included explanations for each outcome they did not achieve, and 20 of 49 (about 41 percent) grantee performance reports were partially consistent with the directive because they did not include explanations for all missed outcomes. For the remaining 12 grantee performance reports (24 percent), we could not determine whether the reports were consistent with EPA’s environmental results directive because they did not include any references to the agreed-upon outputs and outcomes from the grantee work plan. (See table 4.)

According to federal standards for internal control, management should implement control activities through policies. Additionally, these standards state that each unit within an agency also is to document policies in the appropriate level of detail to allow management to effectively monitor the control activity. With its environmental results directive, EPA has implemented certain control activities through its policy to help ensure that grantee performance reports appropriately address planned results from grantee work plans. However, the inconsistencies we found in our review of performance reports may indicate that the guidelines within EPA’s environmental results directive may not be at a sufficient level of detail for EPA to effectively monitor its implementation. By clarifying its directive or guidance to discuss the factors project officers should consider when determining whether reports appropriately address planned results and include satisfactory explanations for unmet results, EPA would have better assurance that project officers are implementing its environmental results directive consistently. In turn, implementing its directive consistently may help EPA demonstrate the achievement of environmental results from its grants, and also help project officers better identify or report patterns in factors that are affecting grantees’ achievement of planned results.

Some Grantees Did Not Include References to the Planned Results in Their Work Plans to Demonstrate Progress

For 12 of the 49 (24 percent) performance reports we reviewed, grantees did not include references to the agreed-upon outputs and outcomes from their work plan to demonstrate progress in achieving planned grant results. Because some grantees did not include information from their work plans in their performance reports, we could not determine whether these grantees achieved their planned results or provided explanations for any results they did not achieve, in accordance with EPA’s environmental results directive (see table 4). To assess these grantees’ progress, the project officer managing the grant would have to manually compare the information in each grantee’s performance report against the grantee’s work plan to determine if the actual results matched the planned results.

During a 2010 EPA-contracted review of performance reports’ consistency with EPA’s environmental results directive, the contractor identified the same issue with several performance reports. Specifically, although the contractor found that 147 out of 157 (about 94 percent) performance reports were greater than 60 percent consistent with EPA’s directive, for 55 of these performance reports, the contractor determined their consistency by inference because the performance reports did not contain explicit linkages to planned outcomes within the grantee work plans. Consequently, to improve the consistency of performance reports with EPA’s environmental results directive, the contractor recommended that EPA consider encouraging grantees to more clearly label the planned outputs and outcomes from their work plans in their performance reports.

In fiscal year 2013, EPA implemented a policy for certain categorical grant programs that calls for grantee performance reports to include certain elements, including an explicit reference to the planned results in the work plan and projected time frame. However, this policy does not apply to all EPA grants, including formula grants and other categorical grants. Expanding aspects of this policy, specifically, the call for performance reports to include an explicit reference to the planned results in the work plan and projected time frames, could achieve several benefits identified in the 2010 review. By increasing the extent to which grantees clearly label the planned results from their work plans in their performance reports, EPA would facilitate project officers’ review of grantee progress, reduce the subjectivity of the review, and increase transparency between EPA and grantees about planned grant results.

Because Grantees Generally Submit Written Performance Reports, There Are No Built-In Quality Controls

Because grantees generally submit written performance reports, there are no built-in data quality controls, such as those for certain electronic reporting formats, to ensure that these reports are consistent with EPA’s environmental results directive. In contrast, we found that some of EPA’s program-specific databases include built-in quality controls, such as required fields, drop-down menus, or other data entry rules designed to ensure that the information entered is complete, accurate, and consistent. Because there are no built-in quality controls for written performance reports, EPA project officers must manually review each performance report to determine consistency with EPA’s directive.

An OGD official told us that OGD plans to develop a web-based portal for grantees to submit documents, including their performance reports, electronically as part of its new grants management database. However, the business process analysis underlying the web-based portal feature of the new database does not specify whether these reports would continue to be uploaded by grantees as attachments or input directly into an application with built-in data quality controls, such as required fields, to ensure consistency with EPA’s directives. The OGD official said that the office will not explore options for the web-based portal, including a timeline, until it has migrated from the old database to the new system, which it expects to complete in fiscal year 2018.

According to federal standards for internal control, control activities may be manual or automated. EPA has manual control activities for implementing its environmental results directive, which is consistent with these standards. However, a 2014 analysis of EPA’s grants management business processes found that EPA relied heavily on manual processes and could incorporate several improvements into its new grants management database system, including using electronic templates to increase information consistency and reduce the administrative burden of manual activities. By incorporating built-in data quality controls for performance reports into its planned web-based portal, EPA could improve these reports’ consistency with the environmental results directive and potentially reduce project officers’ administrative burden in performing manual reviews. Furthermore, improved consistency in performance reports could help EPA project officers to more efficiently identify or report patterns in factors that are affecting grantees’ achievement of their agreed-upon results.

Conclusions

EPA has adopted a number of good practices for monitoring environmental and other program results from the nearly $4 billion dollars it distributes each year in grants, in part to implement environmental statutes and regulations. Furthermore, EPA continues to pursue opportunities to streamline its processes and reduce the reporting burden for regulated entities and grantees. Yet certain monitoring practices— collecting some grant results in a format that is not accessible, collecting some information from grantees twice, and manually transferring data between databases—increase EPA and grantees’ administrative burden in monitoring and reporting environmental and program results. By incorporating expanded search capability features, such as keyword searches, into its proposed web-based portal, EPA can improve the accessibility of information in grantees’ performance reports and make them more useful for sharing lessons learned and building evidence for demonstrating grant results. In addition, by identifying grant programs where existing program-specific data reporting can meet EPA’s performance reporting requirements for grants management purposes, the agency can eliminate duplicative reporting by grantees in a manner consistent with EPA’s ongoing streamlining efforts. Furthermore, by adopting software tools, as appropriate, to electronically transfer relevant data on program results from program-specific databases to EPA’s new national performance system, the Office of Water could reduce its administrative burden.

EPA has also implemented certain internal controls, such as its environmental results directive, to ensure that grantees achieve the environmental and other planned results in their work plans. However, we identified a variety of monitoring issues related to EPA’s environmental results directive—such as unclear guidance, the omission of references to planned results in performance reports to document progress, and written grantee performance reports that do not have built-in quality controls— that may undermine these efforts. By clarifying its directive or guidance to discuss the factors project officers should consider when determining whether performance reports are consistent with EPA’s environmental results directive, EPA would have better assurance that project officers are implementing its directive consistently. In addition, expanding aspects of EPA’s policy for certain categorical grants, specifically, the call for performance reports to include an explicit reference to the planned results in grantees’ work plans and their projected time frames for completion to all grants, would among other things facilitate project officers’ reviews of grantee progress results. Finally, by incorporating built-in data quality controls for performance reports into its planned web-based portal, EPA could improve these reports’ consistency with the environmental results directive and potentially reduce project officers’ administrative burden in performing manual reviews.

Recommendations for Executive Action

We recommend that the EPA Administrator direct OGD and program and regional offices, as appropriate, as part of EPA’s ongoing streamlining initiatives and the development of a grantee portal, to take the following six actions: Incorporate expanded search capability features, such as keyword searches, into its proposed web-based portal for collecting and accessing performance reports to improve their accessibility.

Identify grant programs where existing program-specific data reporting can meet EPA’s performance reporting requirements for grants management purposes to reduce duplicative reporting by grantees.

Once EPA’s new performance system is in place, ensure that the Office of Water adopts software tools, as appropriate, to electronically transfer relevant data on program results from program-specific databases to EPA’s national performance system.

Clarify the factors project officers should consider when determining whether performance reports are consistent with EPA’s environmental results directive.

Expand aspects of EPA’s policy for certain categorical grants, specifically, the call for an explicit reference to the planned results in grantees’ work plans and their projected time frames for completion, to all grants.

Incorporate built-in data quality controls for performance reports into the planned web-based portal based on EPA’s environmental results directive.

Agency Comments and Our Evaluation

We provided a draft of this report to EPA for its review and comment. In its written comments, reproduced in appendix III, EPA stated that it agreed with our findings and six recommendations. EPA also provided technical comments, which we incorporated into the report as appropriate.

EPA agreed with our recommendation that the agency incorporate expanded search capability features into its proposed web-based portal for performance reports and stated that incorporating such features would enable easier access to performance report information. EPA also noted that the web-based portal is a long-term initiative, subject to the agency’s budget process and replacement of its existing grants management system, which the agency expects to complete in fiscal year 2018.

EPA generally agreed with our recommendation that the agency identify grant programs where existing program-specific data reporting by grantees can also meet EPA’s separate performance reporting requirements, to reduce duplicative reporting by grantees. EPA stated that it will work with recipient partners to identify where duplicative reporting can be reduced and anticipates completing this effort by the end of fiscal year 2017. However, EPA noted that program-specific data cannot be relied upon to meet all of the agency’s grants management needs and that performance reports often contain other information that allows EPA project officers to monitor a recipient’s progress in meeting work plan commitments, which cannot be gleaned from output data entered into the agency’s program-specific tracking systems. Additionally, EPA said that not all project officers have access to program-specific databases which would require the agency to consider expanding project officer access to those databases to enhance grant performance monitoring.

EPA agreed with our recommendation that the agency ensure that the Office of Water adopts software tools to electronically transfer relevant data from program databases to EPA’s national performance system, as appropriate. EPA stated that it will also apply this recommendation to all program-specific databases—not just Office of Water databases—where appropriate and cost-effective. EPA also noted that in some cases, not all data from program-specific databases may be appropriate for direct electronic transfer because some individual grant data may need to be analyzed before being summarized at the national level.

EPA agreed with our recommendation that EPA clarify the factors project officers should consider when determining whether performance reports are consistent with EPA’s environmental results directive. EPA stated it will modify the implementation guidance for the directive in fiscal year 2017.

EPA agreed with our recommendation that EPA expand aspects of EPA’s policy for certain categorical grants, specifically, the call for an explicit reference to the planned results in grantee work plans and their projected time frames for completion, to all grants. EPA stated it will revise the existing policy in fiscal year 2017.

EPA generally agreed with our recommendation that the agency incorporate built-in quality controls for performance reports into the planned web-based portal based on EPA’s environmental results directive. However, EPA noted that identifying and deploying the appropriate data quality controls is a long-term effort subject to budgetary considerations, completion of the agency’s replacement of its existing grants management system, and extensive collaboration with internal and external stakeholders. EPA also stated that full achievement of built-in quality controls, such as electronic templates, as envisioned in the draft report would require standardized work plan and performance report formats subject to clearance from the Office of Management and Budget. Additionally, EPA noted that grant recipients and EPA program offices have considered but generally not supported standardizing work plans and performance reports in the past. As a first step in implementing this recommendation, EPA stated that it would seek feedback from the recipient and program office community and will initiate this process in fiscal year 2017.

We recognize that EPA has considered standardizing work plans and performance report formats in the past, and we reviewed the agency’s 2009 “lessons learned” analysis as part of this report (see footnote 29, page 15). We are not recommending that EPA repeat its previous effort and develop a template with standardized program-specific measures to improve reports’ consistency. Specifically, implementing built-in quality controls for performance reports in EPA’s web-based portal would not necessarily require grantees to measure and report the same information across grants. For example, EPA could design an electronic template that follows the guidelines of its existing policies for work plans and performance reports—such as allowing grantees and EPA to negotiate appropriate outputs and outcomes for each grant. If grantees entered their grant-specific outputs and outcomes directly into EPA’s web-based portal as an electronic version of their work plan, the portal could use the information to prepopulate an electronic performance report and reduce manual data entry. Additionally, the electronic performance report could include required fields, such as an explanation field, if the grantee did not meet a particular output or outcome from its work plan. We continue to believe that such controls would improve the consistency of grantee performance reports with EPA’s environmental results directive, and that both EPA project officers and grantees could benefit from the reduced administrative burden associated with submitting and reviewing performance reports electronically.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the appropriate congressional committees, the Administrator of the Environmental Protection Agency, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-3841or gomezj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

This report examines (1) how the Environmental Protection Agency (EPA) awards grants, (2) the federal and EPA requirements and guidelines for monitoring grant and program results, and (3) how EPA monitors its grants to ensure that environmental and other program results are achieved.

To examine how EPA awards grants and the federal and EPA requirements and guidelines for monitoring grant and program results, we reviewed relevant federal laws, regulations, and EPA’s policies and guidance for awarding and monitoring grants. Additionally, we reviewed our prior work on grants management. We also spoke to officials from EPA’s Office of Grants and Debarment (OGD) about how EPA awards grants and EPA’s policies for monitoring grants, and the three program offices that award the majority of EPA grant dollars—the Office of Water, Office of Land and Emergency Management, and Office of Air and Radiation—about EPA program-level guidance for monitoring grant results.

To examine how EPA monitors its grants to ensure that environmental and other program results are achieved, we reviewed EPA’s monitoring processes for grants in the three program offices that award the majority of EPA grant dollars. We identified 45 grant programs awarded by the three program offices, from the Catalog of Federal Domestic Assistance, a clearinghouse for information on federal grant opportunities. We identified an initial list of program-specific databases for the grant programs using information from EPA and its partners’ Environmental Information Exchange Network and EPA’s Central Data Exchange websites. For each grant program we identified, we requested information from EPA program offices, including any corrections to the list of grant programs and associated program-specific databases, whether EPA or grantees enter data into the databases, and how grantees submit data. For these 45 programs, we searched EPA’s Integrated Grants Management System and State Grant Information Technology Application for relevant performance reports.

Based on our search results, we selected a nongeneralizable sample of 49 performance reports across 23 grant programs using the following criteria: (1) whether a performance report was electronically available, (2) whether different EPA regions were represented, (3) whether the grantee was a state grantee that we had interviewed, and (4) whether other documentation—such as an EPA routine monitoring report—was available. Although the results of our review cannot be projected agency- wide because our sample was nongeneralizable, the performance reports represent a broad array of grant programs and include grantees in each EPA region. For each of the 23 grant programs for which we obtained a report, we also collected information on the program-specific database associated with the program, as applicable. We collected information on the content of EPA’s program-specific databases from the Environmental Information Exchange Network and the Central Data Exchange websites, EPA documents collected by a prior GAO team, and EPA’s internal and external websites. Two analysts reviewed the reports and coded them in the following ways: (1) type of content and format of the report, (2) degree of consistency with EPA’s environmental results directive, and (3) degree of overlap between the content of the performance reports and information collected from grantees in EPA’s program-specific databases. To ensure consistency in our review, each analyst reviewed the other’s work and resolved any differences.

To describe the grant results reported in performance reports, we reviewed the content of the performance reports we collected and developed nine mutually exclusive categories of information that grantees typically provide to EPA in these reports. To determine performance reports’ consistency with EPA’s environmental results directive, we reviewed each report against the directive’s call for EPA to review performance reports to (1) determine whether the grantees achieved the planned outputs and outcomes in their work plans and (2) explain any unmet outputs and outcomes. From this review, we developed four categories: 1. Consistent—the report describes progress against outputs or outcomes from the grantee’s work plan and explains all missed targets, if any. 2. Partially consistent—the report includes progress against some, but not all, outputs or outcomes from the grantee’s work plan or explains some, but not all missed targets, if any. 3. Not consistent—the report does not describe progress against outputs or outcomes from the work plan. 4. Could not determine—the report describes grantee activities without an explicit reference to outputs or outcomes from the work plan to demonstrate progress or to allow a reviewer to identify missed outputs or outcomes requiring explanations.

We did not review any other documentation from EPA’s official project file or grants management databases, which is consistent with the methodology described in a 2010 EPA-contracted study examining performance reports’ consistency with EPA’s environmental results directive.

To determine whether grantees reported the same information to EPA twice, we reviewed the content of the performance reports and compared the report content against the information we collected describing data elements in EPA’s program-specific databases for that grant, as applicable. Based on this review, we created four categories of overlap between the report content and the data fields in EPA’s databases: 1. No overlap—no matches between content. 2. Minimal overlap—one to two matches between content. 3. Some overlap—three to five matches in content. 4. Substantial overlap—six matches or more between content.

We interviewed officials from EPA’s OGD, Office of Water, Office of Air and Radiation, Office of the Chief Financial Officer, Office of Land and Emergency Management, and lead regional offices for certain programs to discuss EPA’s processes for monitoring environmental and other program results from grants. We also provided program offices with a standard set of follow-up questions about how they collect and monitor environmental and other program results from grantees. Additionally, we interviewed representatives from the Environmental Council of States— an association of state environmental agency leaders—and a nongeneralizable sample of officials from environmental agencies in eight states—California, Hawaii, Maryland, Michigan, New York, North Carolina, Pennsylvania, and West Virginia—to obtain their perspectives on EPA’s monitoring processes for grants. We selected these eight states because they received the greatest amount of funding from the federal government, according to an Environmental Council of States’ analysis of state environmental budgets data in 2012, the most recent publicly available data. The results of our interviews with officials from these agencies cannot be generalized to those of states not included in our review.

We conducted this performance audit from August 2015 through July 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Additional Information on GAO’s Review of 23 EPA Grant Programs

Table 5 summarizes the scope of our review of grantee performance reports.

Table 6 provides information on which program-specific databases we reviewed.

Appendix III: Comments from the Environmental Protection Agency

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Michael Hix (Assistant Director), Amy Bowser, Cindy Gilbert, Taylor Hadfield, Thomas James, Benjamin Licht, Kimberly McGatlin, Alison O’Neill, Danny Royer, Jeanette Soares, Sara Sullivan, Kiki Theodoropoulos, and Lisa Van Arsdale made key contributions to this report.