Grants Management: EPA Could Improve Certain Monitoring Practices
Published: 2016-07-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Grants comprised about half of EPA's budget in 2015, or about $4 billion. Through several grant programs, EPA headquarters and 10 regional offices award these grants to a variety of recipients, including state and local governments. EPA provides guidance through directives that seek to ensure the appropriate use of funds and achievement of environmental results or public health protection, among other purposes. GAO was asked to review how EPA monitors environmental and other grant results. This report examines (1) how EPA awards grants, (2) the federal and EPA requirements for monitoring grant and program results, and (3) how EPA monitors its grants to ensure that environmental and other program results are achieved. GAO analyzed relevant federal laws, regulations, and EPA guidance; reviewed processes for ensuring that environmental results are achieved for the three EPA program offices that award the majority of EPA grant dollars; and interviewed EPA officials and officials from eight state environmental agencies—selected based on the amount of environmental funding they receive from EPA.
section_title: What GAO Found
paragraphs: The Environmental Protection Agency (EPA) generally awards three different categories of grants: formula, categorical, and discretionary. According to EPA data, in fiscal year 2015, EPA awarded the majority of its grant funds— $2.25 billion of $3.95 billion (57 percent)—as formula grants, primarily to states to support water infrastructure based on funding formulas prescribed by law. EPA awarded $1.09 billion (about 28 percent) of its grant funds as categorical grants. These grants were generally awarded noncompetitively, mostly to states and Indian tribes to operate environmental programs. EPA determines the amount of funding each grantee receives based on agency formula or program factors. EPA awarded $0.513 billion (about 13 percent) in discretionary grants for specific activities, such as research. EPA also awarded $0.09 billion (2 percent) in grant funds to special appropriations act projects for specific drinking water and wastewater infrastructure projects in specific communities.
Multiple federal and agency requirements and guidelines apply to monitoring grant and grant program results. For example, under EPA regulations, grantees must submit performance reports to EPA at least annually. EPA policies and guidance, such as its environmental results directive, call for EPA program officials to review performance reports to determine if the grantee achieved the planned results and for program offices to report on significant grant results through other processes, such as submissions to EPA databases. EPA incorporates requirements related to grantee reporting frequency, content, and reporting processes into grant terms and conditions.
EPA monitors performance reports and program-specific data from grantees to ensure that grants achieve environmental and other program results. However, GAO found that certain practices may hinder EPA's ability to efficiently monitor some results and increase administrative burden. For example, EPA collects some information from grantees twice—once in a performance report and once in a database—because EPA uses the information for different purposes. GAO's prior work and EPA analyses have shown that duplication of efforts can increase administrative costs and reduce the funds available for other priorities. By identifying grant programs where existing data reporting can meet EPA's performance reporting requirements, the agency can help reduce duplicative reporting for grantees. Also, GAO's review of grantee performance reports found issues that may hinder EPA's ability to efficiently identify factors affecting grantee results. For example, because grantees submit performance reports in a written format, there are no built-in quality controls to ensure these reports' consistency with EPA's environmental results directive. Rather, EPA officials must perform a manual review. A 2014 analysis of EPA's grants management processes found that EPA relied heavily on manual processes and could incorporate improvements into its new grants management database system. EPA officials said they plan to develop a web-based portal for grantees to submit documents, such as performance reports. By incorporating built-in data quality controls, such as required fields, for performance reports into its planned web-based portal, EPA could improve these reports' consistency with the environmental results directive and reduce the administrative burden of performing manual reviews.
section_title: What GAO Recommends
paragraphs: GAO is making six recommendations, including that EPA (1) reduce duplicative reporting by identifying grant programs where existing data reporting can meet EPA's performance reporting requirements and (2) incorporate data quality controls for performance reports into its planned web-based portal. In response, EPA agreed with GAO's findings, conclusions, and recommendations.

section_title: Letter
paragraphs: The Environmental Protection Agency (EPA) awarded roughly $3.95  billion in 2015—about 49 percent of its budget—in grants to states, local  governments, tribes, and other recipients, in part to implement  environmental statutes and regulations. The agency awards nearly 60  percent of its total grant dollars to states for two grant programs: the  Clean Water State Revolving Fund and Drinking Water State Revolving  Fund, the largest sources of federal funds for repairing and replacing the  nation’s aging water infrastructure, among other things. Other grants are  used for various purposes, such as hazardous waste cleanup, ensuring  air quality, and pollution prevention.
EPA awards and manages its grants at multiple levels across the agency,  including its headquarters and 10 regional offices. EPA’s Office of Grants  and Debarment (OGD) in its headquarters office develops national grant  policies and guidance, awards some grants, and oversees EPA’s grants  management agency-wide. Headquarters program offices implement  national policies for their grant programs, and both headquarters program  offices and regional offices award grants and monitor environmental  results. Program offices also oversee the technical and program-specific  aspects of headquarters grants. Three program offices—the Office of  Water, the Office of Air and Radiation, and the Office of Land and  Emergency Management awarded the majority of EPA grant funding in  2015. EPA’s 10 regional offices provide administrative, technical, and  program-specific oversight of the grants the regions award.
OGD provides guidance to grant management officials through several  management directives that reflect federal statutes, regulations, Office of  Management and Budget guidance, and EPA policies and procedures.  These directives seek to ensure the timely awarding of grants,  appropriate use of funds, and achievement of the desired results of  protecting human health and the environment. Each management  directive contains multiple requirements, such as the frequency and type  of management reviews EPA officials perform, the steps necessary to  perform them, and the documentation of grantees meeting applicable  requirements.
Prior EPA Office of Inspector General and GAO reports have identified  problems with EPA’s grants management. For example, in 2014, EPA’s  Office of Inspector General found that EPA did not enforce reporting  requirements for grantees to provide data that would demonstrate the  effects the Drinking Water State Revolving Fund has on public health.  Additionally, in an August 2015 report, we found that EPA had made  progress monitoring agency-wide compliance with certain grants  management directives, such as EPA policies, by electronically tracking  unspent grant funds and grantee submission of required reports in a  timely manner. However, we found that two key challenges hampered  EPA’s efforts to monitor agency-wide compliance with such directives.  First, several EPA regional offices use paper files to document  compliance, so monitoring these offices’ compliance with directives  requires resource-intensive manual file reviews. Second, the limited  reporting and analysis capabilities of its information technology (IT)  systems leaves EPA without agency-wide information for most of the  directive requirements we reviewed. EPA officials planned to fully  implement an updated IT system by 2017. We recommended that EPA  could better monitor agency-wide compliance with grants management  directives by developing ways to more effectively use existing web-based  tools until it implements its new IT system. EPA agreed with our findings  and recommendations.
You asked us to examine how EPA monitors environmental and other  results from its grants. This report examines (1) how EPA awards grants,  (2) the federal and EPA requirements and guidelines for monitoring grant  and program results, and (3) how EPA monitors its grants to ensure that  environmental and other program results are achieved.
To examine how EPA awards grants and what federal and EPA  requirements and guidelines apply for monitoring grant and program  results, we reviewed relevant federal laws, regulations, and EPA’s  policies and guidance for awarding and monitoring grants. To examine  how EPA monitors its grants to ensure that environmental and other  program results are achieved, we reviewed EPA’s monitoring processes  for grants in the three program offices that award the majority of EPA  grant dollars—Office of Air and Radiation, Office of Water, and Office of  Land and Emergency Management. We identified 45 programs that  award grants within the three program offices we reviewed, from the  Catalog of Federal Domestic Assistance. For each of the 45 grant  programs, we searched EPA’s grants management databases for  relevant performance reports. Based on our search results, we selected a  nonprobability sample of 49 performance reports across 23 grant  programs based on criteria including whether the report was electronically  available and whether different EPA regions were represented, as shown  in table 1.
Two analysts reviewed the content of the performance reports and the  performance reports’ consistency with EPA’s environmental results  directive, and compared the content to information collected from  grantees in EPA’s program-specific databases. To ensure consistency in  our review, each analyst reviewed the other’s work and resolved any  differences. Additionally, we interviewed representatives from the  Environmental Council of States—an association of state environmental  agency leaders—and a nongeneralizable sample of officials from  environmental agencies in eight states—California, Hawaii, Maryland,  Michigan, New York, North Carolina, Pennsylvania, and West Virginia—to  obtain their perspectives on EPA’s monitoring processes for grants. We  selected these eight states because they received the greatest amount of  funding from the federal government, according to the Environmental  Council of States’ data. The results of our interviews with officials from  these agencies cannot be generalized to those of states not included in  our review. For all three objectives, we interviewed officials from EPA’s  OGD and the three program offices we reviewed. For more information on  our scope and methodology, see appendix I.
We conducted this performance audit from August 2015 to July 2016 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: EPA provides financial assistance to a variety of recipients, including  states, tribes, and nongovernmental organizations, through assistance  agreements such as grants. EPA awards these grants to recipients to  meet local environmental priorities and national objectives established in  federal law, regulations, or EPA policy. As we have previously reported,  most federal grant-making agencies, including EPA, generally follow a life  cycle comprising various stages—preaward (announcement and  application), award, implementation, and closeout—for awarding grants,  as seen in figure 1.
The federal laws establishing EPA’s grant programs generally specify the  types of activities that can be funded, objectives to be accomplished  through the funding, and who is eligible to receive the funding. In addition  to these statutory requirements, EPA has issued regulations governing its  grants, which may impose additional requirements on recipients. EPA  either notifies the public of the grant opportunity or notifies eligible state  agencies about available grants, and applicants must submit applications  to the agency for its review. In the preaward stage, EPA reviews  applications to determine or verify which meet eligibility requirements and  awards funding. EPA assigns project officers—who manage the technical  and program-related aspects of the grants—and grant specialists—who  manage the administrative aspects of grants—in program and regional  offices to oversee the implementation stage of the grants.
The implementation stage includes development of a grant work plan that  outlines EPA and grantee agreed-upon goals, objectives, activities, and  time frames for completion under the grant, such as developing certain  water quality standards by the end of the year. It also includes payment  processing, agency monitoring, and grantee reporting on the results of its  individual grant as well as its contribution to program results. For  example, results for an individual water quality grant might include the  grantee using funds to develop water quality standards, whereas program  results might include the grantee’s contribution to the number of water  quality permits issued under the program as a whole. Grantees submit  information on grant results to EPA through performance reports and  progress reports, depending on the grant program. The closeout phase  includes preparation of final reports, financial reconciliation, and any  required accounting for property.
section_title: EPA Awards Different Categories of Grants
paragraphs: EPA generally awards three types of grants:
Formula grants. EPA awards these grants noncompetitively to states  in amounts based on formulas prescribed by law to support water  infrastructure projects, among other things. For example, grants from  the Clean Water and Drinking Water State Revolving Funds can be  used to support infrastructure, such as water treatment facility  construction, and improvements to drinking water systems, such as  pipelines or drinking water filtration plants. According to EPA data, in  fiscal year 2015, EPA awarded about $2.25 billion of $3.95 billion  (about 57 percent) of grant funds as formula grants, as shown in  figure 2.
Categorical grants. EPA generally awards these grants—which EPA  also refers to as continuing environmental program grants— noncompetitively, mostly to states and Indian tribes to operate  environmental programs that they are authorized by statute to  implement. For example, under the Clean Water Act, states and tribes  can establish and operate programs for the prevention and control of  surface water and groundwater pollution. EPA determines the amount  of funding each grantee receives based on agency-developed  formulas or program-specific factors. In fiscal year 2015, EPA  awarded about $1.09 billion of $3.95 billion (about 28 percent) of grant  funds as categorical grants, according to EPA data.
Discretionary grants. EPA awards these grants—competitively or  noncompetitively—to eligible applicants for specific projects, with EPA  program and regional offices selecting grantees and funding amounts  for each grant. EPA primarily awards these grants to states, local  governments, Indian tribes, nonprofit organizations, and universities  for a variety of activities, such as environmental research, training,  and environmental education programs. According to EPA data, in  fiscal year 2015, EPA awarded about $0.513 billion of $3.95 billion  (about 13 percent) of grant funds as discretionary grants.
EPA also awarded $0.09 billion of $3.95 billion (about 2 percent) of grant  funds to special appropriations act projects for specific drinking water and  wastewater infrastructure projects in specific communities.
section_title: Multiple Federal and EPA Requirements and Guidelines Apply to Monitoring Grant and Grant Program Results
paragraphs: Multiple federal and EPA requirements—established in laws and  regulations—and EPA guidelines apply to monitoring the results of  individual EPA grants and, more broadly, the results of EPA grant  programs.
The following requirements and guidelines form the basis of how EPA  aligns individual grants to achieve the agency’s public health and  environmental objectives:
Federal laws: Authorizing statutes for certain EPA grant programs,  most notably the Clean Water Act, require states—which receive  grants from EPA to capitalize state clean water revolving funds—to  report annually to EPA on how they have met the goals and objectives  identified in their intended use plans for their revolving funds.
EPA regulations: EPA regulations require grantees to submit  performance reports to EPA as specified in their grant agreements at  least annually and typically no more frequently than quarterly. Under  EPA’s regulations, the grantee’s performance should be measured in  a way that will help improve grant program outcomes, share lessons  learned, and spread the adoption of promising practices. Additionally,  under EPA’s regulations, the agency should provide grantees with  clear performance goals, indicators, and milestones, and should  establish reporting frequency and content that allow EPA to build  evidence for program and performance decisions, among other  things.
Agency-wide policies and guidance: EPA policies, such as its  environmental results directive, call for grant work plans and  performance reports to link to the agency’s strategic plan and include  outputs and outcomes. The environmental results directive, the  Policy on Compliance, Review, and Monitoring, and related guidance  also call for EPA program officials to review interim and final  performance reports—or for certain programs, use a joint evaluation  process—to determine if the grantee achieved the planned outputs  and outcomes, and document the results of these reviews in EPA’s  grants management databases. Additionally, the environmental  results directive calls for EPA program offices to report on significant  grant results through reporting processes established by national  program managers, such as data submissions to EPA databases.
Program-specific guidance: EPA program offices provide biennial  guidance on each program’s priorities and key actions to accomplish  health and environmental goals in EPA’s strategic plan. According to  EPA officials, this guidance includes annual commitment measures,  which guide implementation with EPA regions, states, tribes, and  other partners. Many annual commitment measures include regional  performance targets, which contribute to meeting EPA annual budget  measures, and in turn, long-term strategic measures, according to  EPA officials. EPA regional offices use these performance measures  and targets to guide their negotiations with grantees on individual  grant work plan outputs and outcomes.
Grant-specific requirements: EPA incorporates requirements  related to grantee reporting frequency, content, and reporting  processes (i.e., written performance report, data submissions to an  EPA database, or both) into individual grant terms and conditions as  part of the final grant agreement. EPA and grantees also negotiate  grant-specific outputs and outcomes, which grantees incorporate into  their grant work plans.
section_title: EPA Monitors Performance Reports and Data to Ensure Grants Achieve Results, but Certain Practices Hinder EPA’s Ability to Efficiently Monitor Results
paragraphs: EPA monitors performance reports and program-specific data from  grantees to ensure that grants achieve environmental and other program  results, but certain practices hinder EPA’s ability to efficiently monitor  some results. In addition, we identified a variety of monitoring issues that  may hinder EPA’s ability to efficiently identify factors affecting grantee  results.
subsections: section_title: EPA Monitors Written Performance Reports and Program-Specific Numeric Data to Ensure Grants Achieve Results
paragraphs: According to EPA policies and officials, after EPA approves grantee work  plans that identify agreed-upon environmental and other results for each  grant, grantees generally report information on their progress and grant  results to EPA in two ways: (1) submitting performance reports— generally written—that describe the grantees’ progress toward the  planned grant results in their work plans, such as using grant funds to  provide technical assistance to local officials, and (2) electronically  submitting program-specific data—generally numeric—on certain  program measures, such as the number of hazardous waste violations  issued, which EPA tracks in various program databases. According to  an EPA official, the information streams from grantees differ in that the  performance reports go to EPA project officers for the purpose of  managing individual grants, whereas EPA program managers use the  electronic data to monitor regional and program progress on EPA’s  performance measures.
Performance reports. At least annually, grantees are to submit  performance reports to EPA as specified in their grant agreements. EPA  policies include general guidelines about what performance reports  should include, such as a comparison between planned and actual grant  results, but allow the frequency, content, and format of performance  reports to vary by program and grant. For more information on the  performance reports we reviewed, see appendix II.
According to EPA officials, EPA project officers monitor these reports to  review grantee progress toward agreed-upon program results, such as  providing outreach to communities about hazardous waste. Project  officers conduct two types of routine grants monitoring: (1) baseline  monitoring, which is the periodic review of grantee progress and  compliance with a specific grant’s scope of work, terms and conditions,  and regulatory requirements, and (2) advanced monitoring, which is an in- depth assessment of a grantee or a project’s progress, management, and  expectations. EPA assigns a certain number of advanced monitoring  reviews to each regional and program office annually. In 2015, OGD  assigned program and regional offices to perform advanced monitoring  for at least 10 percent of their active grantees, which program and  regional offices select based on criteria such as the size of the grant and  the experience level of the grantee, among others.
EPA project officers document the results of their monitoring—for  example, whether grantees have made sufficient progress and complied  with grant terms and conditions—in EPA’s grants management databases  at least annually. Based on their baseline monitoring review, EPA project  officers may impose more frequent or intensive grant monitoring, such as  advanced monitoring, to address any identified concerns. According to  EPA data, project officers recommended additional grant monitoring for  78 out of 2,987 reviews (about 3 percent) in 2015. Additionally, program  and regional offices summarize any significant grants management- related observations or trends from their advanced monitoring reviews as  part of their annual postaward monitoring plans.
Program-specific information. According to program officials, grantees  also electronically submit program-specific information—generally  numeric data—on certain results, such as the acres of brownfield  properties made ready for reuse. According to EPA policy and program  officials, program officials monitor these data to track and report program  accomplishments, at the regional and agency levels, and, as applicable,  to assess the agency’s progress meeting its performance measure  targets in support of agency strategic goals. According to EPA officials,  generally grantees or EPA program officials—depending on the  database—are to enter grant results, such as the number of enforcement  actions, into EPA’s program-specific data systems at agreed-upon  intervals, such as quarterly. These requirements may be part of a grant’s  terms and conditions.
EPA Performance Measures and Data  Systems   The number of performance measures and  data systems that the Environmental  Protection Agency (EPA) uses to collect and  analyze data on environmental and other  program results in 2016—including  incorporating performance data from grantees  as relevant—varies across the three program  offices we reviewed. For example, the Office  of Water collects or analyzes grantee data on  results for 13 of its 15 grant programs using  20 data systems, and integrates the results as  appropriate into its reporting on 111 annual  commitment measures. The Office of Land  and Emergency Management collects or  analyzes grantee data on results for 10 of its  13 grant programs using 4 systems, and  integrates the results as appropriate into its  reporting on 34 annual commitment  measures. The Office of Air and Radiation  collects or analyzes grantee data on results  for 7 of its 9 grant programs using 3 systems,  and integrates the results as appropriate into  its reporting on 54 annual commitment  measures.
According to EPA officials, there is not always a direct link between  individual grantee results and EPA’s annual budget and annual  commitment performance measures. However, officials told us that  each regional or program office considers information from its program- specific data systems that is relevant to program- or agency-level  performance measures, interprets it, and enters the results as appropriate  into EPA’s national performance tracking systems. For example, Office of  Water officials use data collected from grantees in its Drinking Water  National Information Management System database to report annually in  EPA’s national performance tracking system the number of Drinking  Water State Revolving Fund projects that have started operations.
EPA officials said that reporting grant and program results to EPA has  improved over time, as EPA has transitioned from collecting data in hard  copy and expanded electronic reporting by grantees. Additionally, officials  we spoke with from several states said that electronic reporting had  certain benefits. EPA officials told us that collecting certain information  electronically from grantees allows EPA to access and analyze grant and  program results more efficiently than it can for results collected in a  written format, because EPA officials do not have to manually enter  information into a data system for analysis. Additionally, in response to  information-sharing problems—such as incompatible computer systems,  manual data entry, and differing data structures across program offices— EPA and the Environmental Council of States formed the Environmental  Information Exchange Network (Exchange Network) in 1998, an  information-sharing partnership that uses a common, standardized format  so that EPA, states, and other partners can share environmental data  across different data systems. As a result, EPA and its partners may  access and use environmental data more efficiently, according to  Exchange Network documents. For example, officials we interviewed  from each of the eight state environmental agencies we reviewed said  that they use the information they collect for EPA to either manage their  programs or inform the public. Additionally, even with some technical  issues with individual databases, officials from six of these eight agencies  said that electronic reporting has several benefits, such as improving data  timeliness, greater efficiency, and reduced administrative burden.
Furthermore, based on our review of agency policy, analysis, and  planning documents, we found that current and past EPA initiatives have  taken steps to reduce the reporting burden on grantees and others. For  example:
Since 1996 EPA has been authorized to issue performance  partnership grants, which allow states, Indian tribes, interstate  agencies, and intertribal consortia grantees to combine funds from  certain EPA grant programs into a single grant. EPA designed this  system to provide grantees with greater flexibility to address their  highest environmental priorities and reduce administrative burden and  costs, among other objectives. In 2015, EPA issued a policy to  increase awareness and encourage the use of these grants.
In 2008, EPA issued a policy to reduce reporting burdens for states  awarded grants under 28 grant programs by establishing general  frequencies for grant work plan progress reports and specifying that  EPA regional offices could only require more frequent progress  reports in certain circumstances.
In 2012, EPA’s OGD contracted with external experts to review its  grants management processes and identify improvements as part of  EPA’s Grants Business Process Reengineering Initiative. This  initiative seeks to streamline and standardize the grants management  process at EPA and develop an improved business process to be  implemented through EPA’s new grants management data systems.  The study identified several potential high-level improvements, such  as reducing manual activities and expanding standardization in  documents to ensure greater consistency and reduce administrative  burden.
In 2013, EPA and states established a leadership council for E- Enterprise for the Environment—a joint initiative to streamline and  modernize business processes shared between EPA and regulatory  partners, such as states, and reduce reporting burden on regulated  entities, among other goals. For example, in 2015, EPA and states  initiated the Combined Air Emissions Reporting project, which seeks  to streamline multiple emissions reporting processes at the federal,  state, and local levels, according to EPA’s website. The project will  establish a single, authoritative data repository that will reduce the  industry and government transaction costs for reporting and managing  emissions data through features such as autopopulated forms and  data sharing across regulatory agencies.
In 2015, EPA finalized an electronic reporting rule that requires,  among other things, states that receive grants to issue National  Pollution Discharge Elimination System permits to substitute  electronic reporting for paper-based reports, saving time and  resources for states, EPA, and permitted facilities. According to an  EPA economic analysis, when fully implemented, the new rule will  eliminate 900,000 hours of reporting across regulated entities and  state agencies. According to EPA’s fiscal year 2017 budget, the  agency plans to further reduce the reporting burden by 1 million hours  by the end of fiscal year 2017.
In 2016, EPA’s OGD issued its 2016-2020 Grants Management Plan,  which includes several streamlining efforts specific to grants. For  example, under Goal 2: Streamline Grants Management Procedures,  EPA plans to evaluate its grants management processes and assess  opportunities to streamline its procedures. Under this goal, EPA also  plans to provide a mechanism for staff to submit feedback about  existing burdens and new requirements or procedures. Furthermore,  under Goal 4: Ensure Transparency and Accountability and  Demonstrate Results, EPA plans to improve its process for monitoring  grants and will collect input from external stakeholders, such as states  and grantees, about how to address burdens.
section_title: Certain EPA Monitoring Practices Hinder EPA’s Ability to Efficiently Monitor Some Results and May Increase EPA’s and Grantees’ Administrative Burden
paragraphs: Based on our review of the three program offices that award the majority  of EPA grant funding, we found that certain EPA monitoring practices in  these offices hinder EPA’s ability to efficiently monitor some results and  may increase EPA’s and grantees’ administrative burden. First, EPA  collects a variety of information about grant results, but some of the  information is not readily accessible. Second, EPA collects certain  information from grantees twice, once in a written report and once in an  electronic database. Third, one program office transfers data relevant to  its annual performance measures from its program-specific databases to  EPA’s national database manually rather than electronically. EPA officials  and officials from several state environmental agencies who we  interviewed said that these practices increase their administrative burden.
subsections: section_title: EPA Collects a Variety of Information about Grant Results, but Some of the Information Is Not Readily Accessible
paragraphs: EPA collects a variety of information about grant results through grantee  performance reports and program-specific databases. However, some of  the information was not readily accessible to project officers or grantees.  Based on our review of performance reports across 23 grant programs,  we found that the types of results that grantees reported, such as data  collection and management, covered a variety of topics and were  generally similar across programs, as shown in table 2. Additionally, we  found that grantees electronically report a variety of information about  grant results to program-specific databases, such as enforcement actions  and environmental benefits of water infrastructure projects. However, only  some of the information reported by grantees was readily accessible,  either to the public through user-defined searches on EPA’s website or to  grantees through accessing an EPA database directly. This is because  the information in grantees’ performance reports is stored as file  attachments to database records and EPA’s legacy grants management  databases do not have the capability to search data stored in this  format. For instance, a program manager that wanted to obtain  information on the number and types of training activities funded by a  particular grant program—and that are not reported to a program-specific  database—would need project officers to open each performance report  individually and manually review it for relevant information. OGD officials  told us that—depending on the availability of funds—they plan to develop  a web-based portal for grantees to submit documents, including their  performance reports, centrally as part of their new grants management  database.
Under EPA’s regulations, grantee performance should be measured in a  way that will help improve grant program outcomes, share lessons  learned, and spread the adoption of promising practices. EPA has  procedures in place to collect this information through its program-specific  databases and performance reports. However, we have previously  found that for performance information to be useful, it should meet users’  needs for consistency, relevance, accessibility, and ease of use, among  other attributes. EPA’s 2014 internal analysis of its grants management  business processes identified improvements that if implemented into  EPA’s planned web-based portal, could improve the accessibility and  usefulness of information in grantee performance reports for EPA,  grantees, and other users. For example, the analysis found that  incorporating expanded search capabilities into EPA’s new grants  management database, such as keyword searches, could improve users’  access to relevant information. However, it is unclear to what extent, if at  all, these features will be applied to the web-based portal because the  high-level analysis does not specify how performance reports will be  stored and accessed through the web-based portal.
Because EPA, grantees, and other users cannot readily access  information in performance reports about grant results and how different  grantees achieve them, these reports are less useful for sharing lessons  learned and building evidence for demonstrating grant results. Making the  information that EPA collects in these reports more accessible by  incorporating expanded search capability features, such as keyword  searches, into its proposed web-based portal for collecting and accessing  performance reports, could improve its usefulness to EPA and grantees in  identifying successful approaches to common grantee challenges.  Additionally, improved accessibility could facilitate EPA’s ability to assess  and report environmental and program results achieved through its grants  by reducing the need to manually open and review each performance  report to identify relevant information.
section_title: EPA Collects Certain Information from Grantees Twice
paragraphs: EPA collects certain information from grantees twice—once in a written  report and once in an electronic database—and in some cases, we found  varying degrees of overlap between the content of the performance  reports and program-specific databases that we reviewed. Specifically, of  the performance reports we reviewed across 23 grant programs, we  found that one or more grantee performance reports included information  that grantees also report to EPA through a program-specific database for  12 programs, as shown in table 3. For 10 of these programs, the content  in 15 of the performance reports we reviewed had some overlap with data  submitted through relevant program-specific databases, and for 5 of the  programs, 12 reports we reviewed had substantial overlap. For more  information on the program-specific databases we reviewed, see  appendix II.
Additionally, officials we interviewed from five of the eight state  environmental agencies we reviewed confirmed that under current  reporting requirements, they reported the same information to EPA  twice—once electronically and once in a written performance report,  which increased their administrative burden. Specifically, these state  officials provided the following examples:
Much of grantee reporting for the Clean Water State Revolving  Fund—information reported electronically to EPA—is also reported  separately in the written state revolving fund annual performance  report.
Grantees report the same activities in the Public Water System  Supervision program that they report separately to EPA’s state  revolving fund databases for the state program set-asides, funded by  the Drinking Water State Revolving Fund.
Under the State Hazardous Waste Management Program, EPA calls  for grantees to include permitting, compliance, enforcement, and  corrective action activities and accomplishments—already reported to  EPA electronically—in their performance reports.
Because of different programmatic and reporting needs for water  program grants, officials often find themselves reporting the same  data multiple times in different formats.
Grantees submit data on actions to address nonpoint source pollution  to EPA electronically throughout the year—which grantees also report  separately to EPA in the annual performance reports for Nonpoint  Source Pollution Grants, as required by the Clean Water Act.
Officials we interviewed from five of the eight state environmental  agencies said that EPA could work with states to evaluate how grantees  report and further streamline reporting and data collection. Officials we  interviewed from one state agency said that with limited resources, they  have no capacity for additional reporting requests, without some  modification to reporting schedules or simplification of the reporting  process.
According to EPA officials, EPA’s reporting process has evolved over  time in response to statutory changes, such as amendments to the  Government Performance and Results Act of 1993—which generally  requires that agencies develop performance goals that are expressed in  objective, quantifiable, and measurable form and annually report on their  performance in meeting those goals. Additionally, to facilitate grantees’  timely reporting and access to environmental data, EPA and its partners  have expanded electronic reporting to program-specific databases  through the Exchange Network data-sharing partnership with states and  others, according to EPA and Exchange Network documents.  Furthermore, EPA officials told us that collecting information in both  written performance reports and program-specific databases is beneficial  because the information serves different purposes. Specifically, EPA  officials said that performance reports are designed to provide project  officers with information in the format they need for monitoring grantee  progress, for example, narrative information on grantee activities to  achieve results. Similarly, program-specific databases are designed to  provide program managers with information in the format they need for  monitoring program progress, for example, information that will allow  them to report national-level results. However, officials from two of the  three program offices we reviewed said that project officers either  currently used, or could use, data within some program-specific  databases to help monitor grantee progress.
Because EPA collects certain information in both performance reports  and program-specific databases for 12 of the programs we reviewed,  some grantees have an increased administrative burden, which may  result in fewer resources dedicated to activities that directly protect  human health and the environment. Our prior work and EPA analyses of  its business processes have shown that duplication of efforts can  increase administrative costs and reduce the funds available for other  priorities. By identifying grant programs where existing program-specific  data reporting requirements can meet EPA’s performance reporting  requirements for grants management purposes, the agency can help  reduce duplicative reporting for grantees in a manner consistent with  EPA’s ongoing streamlining efforts.
section_title: One Program Office Transfers Relevant Program Results Data from Its Databases to EPA’s National Database Manually
paragraphs: Because one program office we reviewed, the Office of Water, transfers  certain data relevant to program results from its program-specific  databases to EPA’s national database manually, this office does not  benefit from greater data quality control, accessibility, and administrative  efficiencies reported by another program office that electronically  transfers data relevant to program results. Specifically, the Office of  Land and Emergency Management transfers data relevant to most of its  annual commitment measures from its program-specific databases to  EPA’s national database electronically, using EPA’s Performance  Assessment Tool business intelligence software. According to Office of  Land and Emergency Management officials, the software provides  several advantages to manual data transfer, including improved accuracy,  efficiency, the ability to trace data between the different data systems,  and improved data accessibility for EPA program managers.
In contrast, the Office of Water manually transfers data relevant to its  annual commitment measures from its program-specific data systems to  EPA’s national performance database—the Budget Automation System— using a spreadsheet. According to Office of Water officials, they are not  currently planning to develop the capability to transfer data electronically  because EPA is in the process of replacing its Budget Automation System  with a new system. Instead, these officials said that the office is using  other technology tools—such as collaboration software—to make the data  transfer within EPA more efficient and reduce errors. However, an Office  of Water official acknowledged that the quality assurance process for data  transferred manually is lengthy.
Standards for Internal Control in the Federal Government states that  control activities can be implemented in either an automated or a manual  manner but that automated control activities tend to be more reliable  because they are less susceptible to human error and are typically more  efficient. Furthermore, EPA planning documents and analyses  demonstrate the potential benefits of improving efficiency in government  operations by using automated control activities, such as reduced  administrative burden and cost savings. However, by transferring data  from its program-specific databases to EPA’s agency-wide system  manually, the Office of Water does not benefit from the greater data  quality control, accessibility, and administrative efficiencies available from  electronic transfer of data. By adopting software tools, as appropriate, to  electronically transfer relevant data on program results from program- specific databases to EPA’s new national performance system, the Office  of Water could reduce its administrative burden.
section_title: Our Review of Performance Reports Identified a Variety of Monitoring Issues That May Hinder EPA’s Ability to Efficiently Identify Factors Affecting Grantee Results
paragraphs: Our review of 49 written performance reports across 23 grant programs  identified a variety of monitoring issues related to EPA’s environmental  results directive. First, we found that project officers may interpret EPA’s  environmental results directive differently because the directive is unclear.  Second, in some cases, grantees did not include references to the  agreed-upon outputs and outcomes from their work plan to demonstrate  progress achieving planned results. Third, because grantees submit  performance reports in a written format, there are no built-in quality  controls to ensure these reports’ consistency with EPA’s directive. Each  of these issues may have contributed to the inconsistencies we found in  the reports we reviewed. Inconsistencies in grantee reports may make it  more difficult for EPA project officers to efficiently identify or report  patterns in factors affecting grantee’s achievement of their agreed-upon  results.
subsections: section_title: Project Officers May Interpret EPA’s Environmental Results Directive Differently Because the Directive Is Unclear
paragraphs: We found that individual project officers may be interpreting EPA’s  environmental results directive differently because the directive is unclear.  Specifically, we found that reports’ consistency with the directive varied  by grantee and across some of the grant programs we reviewed. One  reason for these variations may be that project officers have different  interpretations of EPA’s directive, as the directive does not provide  specific criteria for evaluating performance reports’ consistency.
EPA’s environmental results directive establishes EPA’s policy to ensure  that grant outputs and outcomes are appropriately addressed in grantee  performance reports, to the maximum extent practicable. Specifically, it  calls for program offices to review performance reports and determine  whether the grantees achieved the environmental or other outputs and  outcomes in their grantee work plans, which includes assessing whether  grantee explanations for unmet outputs or outcomes are satisfactory.  According to the directive, the results of this review should be included in  EPA’s official project file for each grantee.
However, the directive does not specify what factors the project officers  who manage grants should consider when determining whether the  grantees’ addressing of outputs and outcomes in their performance  reports is appropriate. Based on our review of performance reports, we  found that the level of detail in grantees’ descriptions of how they  addressed grant outputs and outcomes varied across the reports we  reviewed. For example, some grantees reported completing or providing  training activities without including additional information on the topic,  date, or number of attendees. In contrast, other grantees provided  specific information on training, such as which employees attended  training, the various courses, and dates of classes.
Similarly, the directive does not specify what factors project officers  should consider when determining whether a grantee’s explanation for an  unmet output or outcome in a performance report is satisfactory. For  example, we found that 17 of 49 (about 35 percent) grantee performance  reports were consistent with EPA’s directive because they included  explanations for each outcome they did not achieve, and 20 of 49 (about  41 percent) grantee performance reports were partially consistent with the  directive because they did not include explanations for all missed  outcomes. For the remaining 12 grantee performance reports (24  percent), we could not determine whether the reports were consistent  with EPA’s environmental results directive because they did not include  any references to the agreed-upon outputs and outcomes from the  grantee work plan. (See table 4.)
According to federal standards for internal control, management should  implement control activities through policies. Additionally, these  standards state that each unit within an agency also is to document  policies in the appropriate level of detail to allow management to  effectively monitor the control activity. With its environmental results  directive, EPA has implemented certain control activities through its policy  to help ensure that grantee performance reports appropriately address  planned results from grantee work plans. However, the inconsistencies  we found in our review of performance reports may indicate that the  guidelines within EPA’s environmental results directive may not be at a  sufficient level of detail for EPA to effectively monitor its implementation.  By clarifying its directive or guidance to discuss the factors project officers  should consider when determining whether reports appropriately address  planned results and include satisfactory explanations for unmet results,  EPA would have better assurance that project officers are implementing  its environmental results directive consistently. In turn, implementing its  directive consistently may help EPA demonstrate the achievement of  environmental results from its grants, and also help project officers better  identify or report patterns in factors that are affecting grantees’  achievement of planned results.
section_title: Some Grantees Did Not Include References to the Planned Results in Their Work Plans to Demonstrate Progress
paragraphs: For 12 of the 49 (24 percent) performance reports we reviewed, grantees  did not include references to the agreed-upon outputs and outcomes from  their work plan to demonstrate progress in achieving planned grant  results. Because some grantees did not include information from their  work plans in their performance reports, we could not determine whether  these grantees achieved their planned results or provided explanations  for any results they did not achieve, in accordance with EPA’s  environmental results directive (see table 4). To assess these grantees’  progress, the project officer managing the grant would have to manually  compare the information in each grantee’s performance report against the  grantee’s work plan to determine if the actual results matched the  planned results.
During a 2010 EPA-contracted review of performance reports’  consistency with EPA’s environmental results directive, the contractor  identified the same issue with several performance reports. Specifically,  although the contractor found that 147 out of 157 (about 94 percent)  performance reports were greater than 60 percent consistent with EPA’s  directive, for 55 of these performance reports, the contractor determined  their consistency by inference because the performance reports did not  contain explicit linkages to planned outcomes within the grantee work  plans. Consequently, to improve the consistency of performance reports  with EPA’s environmental results directive, the contractor recommended  that EPA consider encouraging grantees to more clearly label the planned  outputs and outcomes from their work plans in their performance reports.
In fiscal year 2013, EPA implemented a policy for certain categorical  grant programs that calls for grantee performance reports to include  certain elements, including an explicit reference to the planned results in  the work plan and projected time frame. However, this policy does not  apply to all EPA grants, including formula grants and other categorical  grants. Expanding aspects of this policy, specifically, the call for  performance reports to include an explicit reference to the planned results  in the work plan and projected time frames, could achieve several  benefits identified in the 2010 review. By increasing the extent to which  grantees clearly label the planned results from their work plans in their  performance reports, EPA would facilitate project officers’ review of  grantee progress, reduce the subjectivity of the review, and increase  transparency between EPA and grantees about planned grant results.
section_title: Because Grantees Generally Submit Written Performance Reports, There Are No Built-In Quality Controls
paragraphs: Because grantees generally submit written performance reports, there are  no built-in data quality controls, such as those for certain electronic  reporting formats, to ensure that these reports are consistent with EPA’s  environmental results directive. In contrast, we found that some of EPA’s  program-specific databases include built-in quality controls, such as  required fields, drop-down menus, or other data entry rules designed to  ensure that the information entered is complete, accurate, and consistent.  Because there are no built-in quality controls for written performance  reports, EPA project officers must manually review each performance  report to determine consistency with EPA’s directive.
An OGD official told us that OGD plans to develop a web-based portal for  grantees to submit documents, including their performance reports,  electronically as part of its new grants management database. However,  the business process analysis underlying the web-based portal feature of  the new database does not specify whether these reports would continue  to be uploaded by grantees as attachments or input directly into an  application with built-in data quality controls, such as required fields, to  ensure consistency with EPA’s directives. The OGD official said that the  office will not explore options for the web-based portal, including a  timeline, until it has migrated from the old database to the new system,  which it expects to complete in fiscal year 2018.
According to federal standards for internal control, control activities may  be manual or automated. EPA has manual control activities for  implementing its environmental results directive, which is consistent with  these standards. However, a 2014 analysis of EPA’s grants management  business processes found that EPA relied heavily on manual processes  and could incorporate several improvements into its new grants  management database system, including using electronic templates to  increase information consistency and reduce the administrative burden of  manual activities. By incorporating built-in data quality controls for  performance reports into its planned web-based portal, EPA could  improve these reports’ consistency with the environmental results  directive and potentially reduce project officers’ administrative burden in  performing manual reviews. Furthermore, improved consistency in  performance reports could help EPA project officers to more efficiently  identify or report patterns in factors that are affecting grantees’  achievement of their agreed-upon results.
section_title: Conclusions
paragraphs: EPA has adopted a number of good practices for monitoring  environmental and other program results from the nearly $4 billion dollars  it distributes each year in grants, in part to implement environmental  statutes and regulations. Furthermore, EPA continues to pursue  opportunities to streamline its processes and reduce the reporting burden  for regulated entities and grantees. Yet certain monitoring practices— collecting some grant results in a format that is not accessible, collecting  some information from grantees twice, and manually transferring data  between databases—increase EPA and grantees’ administrative burden  in monitoring and reporting environmental and program results. By  incorporating expanded search capability features, such as keyword  searches, into its proposed web-based portal, EPA can improve the  accessibility of information in grantees’ performance reports and make  them more useful for sharing lessons learned and building evidence for  demonstrating grant results. In addition, by identifying grant programs  where existing program-specific data reporting can meet EPA’s  performance reporting requirements for grants management purposes,  the agency can eliminate duplicative reporting by grantees in a manner  consistent with EPA’s ongoing streamlining efforts. Furthermore, by  adopting software tools, as appropriate, to electronically transfer relevant  data on program results from program-specific databases to EPA’s new  national performance system, the Office of Water could reduce its  administrative burden.
EPA has also implemented certain internal controls, such as its  environmental results directive, to ensure that grantees achieve the  environmental and other planned results in their work plans. However, we  identified a variety of monitoring issues related to EPA’s environmental  results directive—such as unclear guidance, the omission of references to  planned results in performance reports to document progress, and written  grantee performance reports that do not have built-in quality controls— that may undermine these efforts. By clarifying its directive or guidance to  discuss the factors project officers should consider when determining  whether performance reports are consistent with EPA’s environmental  results directive, EPA would have better assurance that project officers  are implementing its directive consistently. In addition, expanding aspects  of EPA’s policy for certain categorical grants, specifically, the call for  performance reports to include an explicit reference to the planned results  in grantees’ work plans and their projected time frames for completion to  all grants, would among other things facilitate project officers’ reviews of  grantee progress results. Finally, by incorporating built-in data quality  controls for performance reports into its planned web-based portal, EPA  could improve these reports’ consistency with the environmental results  directive and potentially reduce project officers’ administrative burden in  performing manual reviews.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the EPA Administrator direct OGD and program and  regional offices, as appropriate, as part of EPA’s ongoing streamlining  initiatives and the development of a grantee portal, to take the following  six actions:  Incorporate expanded search capability features, such as keyword  searches, into its proposed web-based portal for collecting and  accessing performance reports to improve their accessibility.
Identify grant programs where existing program-specific data reporting  can meet EPA’s performance reporting requirements for grants  management purposes to reduce duplicative reporting by grantees.
Once EPA’s new performance system is in place, ensure that the  Office of Water adopts software tools, as appropriate, to electronically  transfer relevant data on program results from program-specific  databases to EPA’s national performance system.
Clarify the factors project officers should consider when determining  whether performance reports are consistent with EPA’s environmental  results directive.
Expand aspects of EPA’s policy for certain categorical grants,  specifically, the call for an explicit reference to the planned results in  grantees’ work plans and their projected time frames for completion,  to all grants.
Incorporate built-in data quality controls for performance reports into  the planned web-based portal based on EPA’s environmental results  directive.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to EPA for its review and comment. In  its written comments, reproduced in appendix III, EPA stated that it  agreed with our findings and six recommendations. EPA also provided  technical comments, which we incorporated into the report as  appropriate.
EPA agreed with our recommendation that the agency incorporate  expanded search capability features into its proposed web-based portal  for performance reports and stated that incorporating such features would  enable easier access to performance report information. EPA also noted  that the web-based portal is a long-term initiative, subject to the agency’s  budget process and replacement of its existing grants management  system, which the agency expects to complete in fiscal year 2018.
EPA generally agreed with our recommendation that the agency identify  grant programs where existing program-specific data reporting by  grantees can also meet EPA’s separate performance reporting  requirements, to reduce duplicative reporting by grantees. EPA stated  that it will work with recipient partners to identify where duplicative  reporting can be reduced and anticipates completing this effort by the end  of fiscal year 2017. However, EPA noted that program-specific data  cannot be relied upon to meet all of the agency’s grants management  needs and that performance reports often contain other information that  allows EPA project officers to monitor a recipient’s progress in meeting  work plan commitments, which cannot be gleaned from output data  entered into the agency’s program-specific tracking systems. Additionally,  EPA said that not all project officers have access to program-specific  databases which would require the agency to consider expanding project  officer access to those databases to enhance grant performance  monitoring.
EPA agreed with our recommendation that the agency ensure that the  Office of Water adopts software tools to electronically transfer relevant  data from program databases to EPA’s national performance system, as  appropriate. EPA stated that it will also apply this recommendation to all  program-specific databases—not just Office of Water databases—where  appropriate and cost-effective. EPA also noted that in some cases, not all  data from program-specific databases may be appropriate for direct  electronic transfer because some individual grant data may need to be  analyzed before being summarized at the national level.
EPA agreed with our recommendation that EPA clarify the factors project  officers should consider when determining whether performance reports  are consistent with EPA’s environmental results directive. EPA stated it  will modify the implementation guidance for the directive in fiscal year  2017.
EPA agreed with our recommendation that EPA expand aspects of EPA’s  policy for certain categorical grants, specifically, the call for an explicit  reference to the planned results in grantee work plans and their projected  time frames for completion, to all grants. EPA stated it will revise the  existing policy in fiscal year 2017.
EPA generally agreed with our recommendation that the agency  incorporate built-in quality controls for performance reports into the  planned web-based portal based on EPA’s environmental results  directive. However, EPA noted that identifying and deploying the  appropriate data quality controls is a long-term effort subject to budgetary  considerations, completion of the agency’s replacement of its existing  grants management system, and extensive collaboration with internal and  external stakeholders. EPA also stated that full achievement of built-in  quality controls, such as electronic templates, as envisioned in the draft  report would require standardized work plan and performance report  formats subject to clearance from the Office of Management and Budget.  Additionally, EPA noted that grant recipients and EPA program offices  have considered but generally not supported standardizing work plans  and performance reports in the past. As a first step in implementing this  recommendation, EPA stated that it would seek feedback from the  recipient and program office community and will initiate this process in  fiscal year 2017.
We recognize that EPA has considered standardizing work plans and  performance report formats in the past, and we reviewed the agency’s  2009 “lessons learned” analysis as part of this report (see footnote 29,  page 15). We are not recommending that EPA repeat its previous effort  and develop a template with standardized program-specific measures to  improve reports’ consistency. Specifically, implementing built-in quality  controls for performance reports in EPA’s web-based portal would not  necessarily require grantees to measure and report the same information  across grants. For example, EPA could design an electronic template that  follows the guidelines of its existing policies for work plans and  performance reports—such as allowing grantees and EPA to negotiate  appropriate outputs and outcomes for each grant. If grantees entered  their grant-specific outputs and outcomes directly into EPA’s web-based  portal as an electronic version of their work plan, the portal could use the  information to prepopulate an electronic performance report and reduce  manual data entry. Additionally, the electronic performance report could  include required fields, such as an explanation field, if the grantee did not  meet a particular output or outcome from its work plan. We continue to  believe that such controls would improve the consistency of grantee  performance reports with EPA’s environmental results directive, and that  both EPA project officers and grantees could benefit from the reduced  administrative burden associated with submitting and reviewing  performance reports electronically.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the  appropriate congressional committees, the Administrator of the  Environmental Protection Agency, and other interested parties. In  addition, the report will be available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-3841or gomezj@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: This report examines (1) how the Environmental Protection Agency (EPA)  awards grants, (2) the federal and EPA requirements and guidelines for  monitoring grant and program results, and (3) how EPA monitors its  grants to ensure that environmental and other program results are  achieved.
To examine how EPA awards grants and the federal and EPA  requirements and guidelines for monitoring grant and program results, we  reviewed relevant federal laws, regulations, and EPA’s policies and  guidance for awarding and monitoring grants. Additionally, we reviewed  our prior work on grants management. We also spoke to officials from  EPA’s Office of Grants and Debarment (OGD) about how EPA awards  grants and EPA’s policies for monitoring grants, and the three program  offices that award the majority of EPA grant dollars—the Office of Water,  Office of Land and Emergency Management, and Office of Air and  Radiation—about EPA program-level guidance for monitoring grant  results.
To examine how EPA monitors its grants to ensure that environmental  and other program results are achieved, we reviewed EPA’s monitoring  processes for grants in the three program offices that award the majority  of EPA grant dollars. We identified 45 grant programs awarded by the  three program offices, from the Catalog of Federal Domestic Assistance,  a clearinghouse for information on federal grant opportunities. We  identified an initial list of program-specific databases for the grant  programs using information from EPA and its partners’ Environmental  Information Exchange Network and EPA’s Central Data Exchange  websites. For each grant program we identified, we requested  information from EPA program offices, including any corrections to the list  of grant programs and associated program-specific databases, whether  EPA or grantees enter data into the databases, and how grantees submit  data. For these 45 programs, we searched EPA’s Integrated Grants  Management System and State Grant Information Technology Application  for relevant performance reports.
Based on our search results, we selected a nongeneralizable sample of  49 performance reports across 23 grant programs using the following  criteria: (1) whether a performance report was electronically available, (2)  whether different EPA regions were represented, (3) whether the grantee  was a state grantee that we had interviewed, and (4) whether other  documentation—such as an EPA routine monitoring report—was  available. Although the results of our review cannot be projected agency- wide because our sample was nongeneralizable, the performance reports  represent a broad array of grant programs and include grantees in each  EPA region. For each of the 23 grant programs for which we obtained a  report, we also collected information on the program-specific database  associated with the program, as applicable. We collected information on  the content of EPA’s program-specific databases from the Environmental  Information Exchange Network and the Central Data Exchange websites,  EPA documents collected by a prior GAO team, and EPA’s internal and  external websites. Two analysts reviewed the reports and coded them in  the following ways: (1) type of content and format of the report, (2) degree  of consistency with EPA’s environmental results directive, and (3) degree  of overlap between the content of the performance reports and  information collected from grantees in EPA’s program-specific databases.  To ensure consistency in our review, each analyst reviewed the other’s  work and resolved any differences.
To describe the grant results reported in performance reports, we  reviewed the content of the performance reports we collected and  developed nine mutually exclusive categories of information that grantees  typically provide to EPA in these reports. To determine performance  reports’ consistency with EPA’s environmental results directive, we  reviewed each report against the directive’s call for EPA to review  performance reports to (1) determine whether the grantees achieved the  planned outputs and outcomes in their work plans and (2) explain any  unmet outputs and outcomes. From this review, we developed four  categories:  1.  Consistent—the report describes progress against outputs or  outcomes from the grantee’s work plan and explains all missed  targets, if any.  2.  Partially consistent—the report includes progress against some, but  not all, outputs or outcomes from the grantee’s work plan or explains  some, but not all missed targets, if any.  3.  Not consistent—the report does not describe progress against outputs  or outcomes from the work plan.  4.  Could not determine—the report describes grantee activities without  an explicit reference to outputs or outcomes from the work plan to  demonstrate progress or to allow a reviewer to identify missed outputs  or outcomes requiring explanations.
We did not review any other documentation from EPA’s official project file  or grants management databases, which is consistent with the  methodology described in a 2010 EPA-contracted study examining  performance reports’ consistency with EPA’s environmental results  directive.
To determine whether grantees reported the same information to EPA  twice, we reviewed the content of the performance reports and compared  the report content against the information we collected describing data  elements in EPA’s program-specific databases for that grant, as  applicable. Based on this review, we created four categories of overlap  between the report content and the data fields in EPA’s databases:  1.  No overlap—no matches between content.  2.  Minimal overlap—one to two matches between content.  3.  Some overlap—three to five matches in content.  4.  Substantial overlap—six matches or more between content.
We interviewed officials from EPA’s OGD, Office of Water, Office of Air  and Radiation, Office of the Chief Financial Officer, Office of Land and  Emergency Management, and lead regional offices for certain programs  to discuss EPA’s processes for monitoring environmental and other  program results from grants. We also provided program offices with a  standard set of follow-up questions about how they collect and monitor  environmental and other program results from grantees. Additionally, we  interviewed representatives from the Environmental Council of States— an association of state environmental agency leaders—and a  nongeneralizable sample of officials from environmental agencies in eight  states—California, Hawaii, Maryland, Michigan, New York, North  Carolina, Pennsylvania, and West Virginia—to obtain their perspectives  on EPA’s monitoring processes for grants. We selected these eight  states because they received the greatest amount of funding from the  federal government, according to an Environmental Council of States’  analysis of state environmental budgets data in 2012, the most recent  publicly available data. The results of our interviews with officials from  these agencies cannot be generalized to those of states not included in  our review.
We conducted this performance audit from August 2015 through July  2016 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Additional Information on GAO’s Review of 23 EPA Grant Programs
paragraphs: Table 5 summarizes the scope of our review of grantee performance  reports.
Table 6 provides information on which program-specific databases we  reviewed.
section_title: Appendix III: Comments from the Environmental Protection Agency
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Michael Hix (Assistant Director),  Amy Bowser, Cindy Gilbert, Taylor Hadfield, Thomas James, Benjamin  Licht, Kimberly McGatlin, Alison O’Neill, Danny Royer, Jeanette Soares,  Sara Sullivan, Kiki Theodoropoulos, and Lisa Van Arsdale made key  contributions to this report.