Grants Management: EPA Has Made Progress in Grant Reforms but Needs to Address Weaknesses in Implementation and Accountability
Published: 2006-05-12T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Environmental Protection Agency (EPA) has faced challenges for many years in managing its grants, which constitute over one-half of the agency's budget, or about $4 billion annually. EPA awards grants through 93 programs to such recipients as state and local governments, tribes, universities, and nonprofit organizations. In response to concerns about its ability to manage grants effectively, EPA issued its 5-year Grants Management Plan in 2003, with performance measures and targets. GAO was asked to assess EPA's progress in implementing its grant reforms in four key areas: (1) awarding grants, (2) monitoring grantees, (3) obtaining results from grants, and (4) managing grant staff and resources. To conduct this work, GAO, among other things, examined the implementation of the reforms at the regional level for two Clean Water Act programs in 3 of EPA's 10 regional offices.
section_title: What GAO Found
paragraphs: EPA has made important strides in achieving the grant reforms laid out in its 2003 Grants Management Plan, but weaknesses in implementation and accountability continue to hamper effective grants management in four areas. First, EPA has strengthened its award process by, among other things, (1) expanding the use of competition to select the most qualified applicants and (2) issuing new policies and guidance to improve the awarding of grants. Despite this progress, EPA's reviews found that staff do not always fully document their assessments of grantees' cost proposals; GAO also identified this problem in one region. Lack of documentation may hinder EPA's ability to be accountable for the reasonableness of the grantee's proposed costs. EPA is reexamining its cost review policy to address this problem. Second, EPA has made progress in reviewing its in-depth monitoring results to identify systemic problems, but long-standing issues remain in documenting ongoing monitoring and closing out grants. EPA and GAO found that staff do not always document ongoing monitoring, which is critical for determining if a grantee is on track in meeting its agreement. Without documentation, questions arise about the adequacy of EPA's monitoring of grantee performance. This lack of documentation occurred, in part, because managers have not fulfilled their commitment to improve monitoring documentation. In addition, grant closeouts are needed to ensure that grantees have met all financial requirements, provided their final reports, and returned any unexpended balances. For fiscal year 2005, EPA closed out only 37 percent of grants within 180 days after the grant project ended, as required by its policy. EPA also did not always close out grants properly in the regional files GAO reviewed. Third, EPA has initiated actions to obtain environmental results from its grants, but these efforts are not complete. For example, EPA's 2005 environmental results policy establishes criteria grants should meet to obtain results. However, EPA has not established a performance measure that addresses these criteria. Furthermore, EPA has not yet identified better ways to integrate its grant reporting systems. Finally, the Office of Management and Budget's 2006 assessment indicates that EPA needs to continue its concerted efforts to achieve results from grants. Finally, EPA has taken steps to manage grant staff and resources more effectively by analyzing workload, providing training, assessing the reliability of its grants management computer database, and holding managers and staff accountable for successfully fulfilling their grant responsibilities. Management attention is still needed because, among other things, EPA has just begun to implement its performance appraisal system for holding managers and staff accountable for grants management.

section_title: Letter
paragraphs: The Environmental Protection Agency (EPA) has faced challenges for  many years in managing its grants, which constitute over one-half of the  agency’s budget, or about $4 billion annually. To support its mission of  protecting human health and the environment, EPA awards grants through  93 programs to a variety of recipients, including state and local  governments, tribes, universities, and nonprofit organizations. As of  September 30, 2005, EPA was administering grants through headquarters  and regional offices to 4,075 grant recipients. Given the size and diversity  of EPA’s grant programs, its ability to efficiently and effectively  accomplish its mission depends to a large extent on how well it manages  its grant resources and builds accountability for results into its efforts.
In response to concerns about its ability to manage grants effectively, EPA  issued its Grants Management Plan. In this plan, EPA for the first time  presented goals, objectives, milestones, and performance measures with  targets for tackling the agency’s long-standing grants management  problems. The 5-year plan was comprehensive in that it set forth five  major goals that the agency sought to achieve. These goals addressed  major concerns we had identified in our 2003 report on grants  management. EPA has also issued a series of policies to implement the  plan and other grant reforms. The plan’s goals are to (1) strengthen the  award of grants by using competition to select grantees for certain awards  to ensure that the best applicants are chosen; (2) monitor grants to ensure  that grantees are making progress toward their objectives and, at the end  of the project period, to ensure that recipients have provided all financial  and technical reports before closing out the grants; (3) obtain results from  grants by identifying and measuring their environmental and public health  outcomes; (4) enhance the skills of EPA personnel involved in grants  management; and (5) leverage technology to improve program  performance. Each of these goals is challenging, especially measuring the  outcomes of environmental activities, but EPA needs to demonstrate the  results achieved through its $4 billion annual investment in grant  programs. EPA is currently revising this plan to reflect accomplishments  achieved and to address remaining issues.
You asked us to assess EPA’s progress in implementing its grant reforms  for (1) awarding grants, (2) monitoring grantees, (3) obtaining results from  grants, and (4) managing grant staff and resources.
To address these issues, we conducted our work at EPA’s headquarters  and regional offices. EPA conducts its grant activities through its Office of  Grants and Debarment—the office primarily responsible for grant  policies—and program offices, located in both headquarters and 10  regional offices—offices primarily responsible for implementing the grant  policies. At EPA headquarters, we reviewed EPA documents, including the  Grants Management Plan, policies and guidance, internal reviews of  management operations, and annual post-award monitoring plans. We  also interviewed Office of Grants and Debarment officials. In addition, we  interviewed officials and obtained information from EPA’s Office of Water,  one of the program offices involved with grants. We reviewed EPA’s Office  of Inspector General reports as well as the Office of Management and  Budget’s (OMB) Program Assessment Rating Tool (PART). We also  reviewed prior GAO reports.
In conducting our work, you asked us to address the implementation of  EPA’s grant reforms at the regional level for Clean Water Act programs.  We selected Wetland Program Development Grants (wetland grants)  because it is a discretionary grant program—that is, EPA decides who  receives the award and its amount, and the program is subject to  competition. We selected Nonpoint Source Management Program Grants  (nonpoint source grants) because it is a type of formula-based grant  program—grants that are often awarded on the basis of formulas  prescribed by law or agency regulation. We reviewed EPA’s progress at  the regional level by selecting grants in 3 of EPA’s 10 regional offices:  Region 1 (Boston), Region 5 (Chicago), and Region 9 (San Francisco). We  selected these regions, in part, because, collectively, they represent a  significant share of regional grant funding for the two programs we  reviewed and geographic dispersion. At the regional offices, for these two  grant programs, we reviewed grant files, and interviewed EPA grant  specialists—who are responsible for overseeing the grantees’  administrative and financial activities—and EPA project officers—who are  responsible for overseeing the grantees’ programmatic and technical  activities. Our regional work provides insights into regional grant activities  in the two Clean Water Act programs in the three regions we visited, but it  is not generalizable to all grants in all regions because we (1) selected only  two of the programs conducted in these offices and (2) incorporated  nonprobability sampling into our grant selection process. In conducting  our review, we did not assess the implementation of some of EPA’s  policies because they had been issued too recently. Finally, we obtained  EPA data used in this report from the agency’s grants computer databases  (Grants Information and Control System and the Integrated Grants  Management System), which have not had a complete data reliability  review. However, we determined that the data elements we used in this  report were sufficiently reliable for our purposes in this report. (App. I  provides a more detailed description of our scope and methodology.) We  performed our work between February 2005 and April 2006, in accordance  with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: EPA administers and oversees grants primarily through the Office of  Grants and Debarment in the Office of Administration and Resources  Management, 10 program offices in headquarters, and program offices and  grants management offices in EPA’s 10 regional offices. Figure 1 shows  the key EPA offices involved in grants activities for headquarters and  regions, and figure 2 shows the states covered by the 10 regional offices.
The Office of Grants and Debarment develops national grant policy and  guidance. This office also carries out certain types of administrative and  financial functions for the grants approved by program offices, such as  awarding headquarter grants and overseeing the financial management of  program office and regional grants. On the programmatic side, national  program managers are responsible for establishing and implementing  national policies for their grant programs, for setting funding priorities,  and for identifying specific environmental results from grant programs.  They are also responsible for technical and programmatic oversight of  headquarter grants. Regional grants management offices provide  administrative management for regional grants, while regional program  offices provide technical and programmatic oversight. Both headquarters  and regional program offices conduct grant competitions. EPA has  designated officials—referred to as senior resource officials—who are  typically deputy assistant administrators in program offices and assistant  regional administrators. These senior resource officials are in charge of  strengthening agencywide fiscal resource management while also ensuring  compliance with laws and regulations and are responsible for effective  grants management within their units.
As of September 30, 2005, 119 grant specialists in the Office of Grants and  Debarment, and the regional grants management offices, were largely  responsible for administrative and financial grant functions. Furthermore,  2,064 project officers were actively managing grants in headquarters and  regional program offices. These project officers are responsible for the  technical and programmatic management of grants. Unlike grant  specialists, however, project officers also have nongrant responsibilities,  such as using the scientific and technical expertise for which they were  hired.
In fiscal year 2005, EPA took 6,728 grant actions involving funds totaling  about $4 billion. These awards were made to six main categories of  recipients, as shown in figure 3.
EPA offers three types of grants—discretionary, nondiscretionary, and  continuing environmental grants:    Discretionary grants fund a variety of activities, such as environmental  research and training. EPA has the discretion to independently determine  the recipients and funding levels for these grants. EPA has awarded these  grants primarily to state and local governments, nonprofit organizations,  universities, and Native American tribes. In fiscal year 2005, EPA awarded  about $644 million in discretionary grants.
Nondiscretionary grants are awarded primarily to state and local  governments and support water infrastructure projects, such as the  drinking water and clean water state revolving fund programs. For these  grants, Congress directs awards to one or more classes of prospective  recipients who meet specific eligibility criteria, or the grants are often  awarded on the basis of formulas prescribed by law or agency regulation.  In fiscal year 2005, EPA awarded about $2.4 billion in nondiscretionary  grants.
Continuing environmental program grants contain both nondiscretionary  (formula) and discretionary features. These grants are nondiscretionary in  the sense that (1) they are awarded noncompetitively to the same  government units to support ongoing state, tribal, and local programs that  do not change substantially over time, and (2) allotments of funds are  initially made on the basis of factors contained in statute, regulation, or  agency guidance. These grants are also discretionary in the sense that  allotments are not entitlements, and EPA exercises judgment in  determining what the final award amount should be. In fiscal year 2005,  EPA awarded about $1 billion in grants for continuing environmental  programs.
In this report, we focused on two EPA programs under the Clean Water  Act:    Wetland Program Development Grants (Wetland grants) Wetlands are  areas where water covers the soil, or is present either at or near the  surface of the soil throughout the year or for various portions of the year,  including during the growing season. Wetlands, such as bogs, swamps, and  marshes, support a number of valuable functions—controlling floods,  improving water quality, and providing wildlife habitat, among other  things. The wetland grants provide applicants with an opportunity to carry  out projects to develop and refine comprehensive wetland programs. The  authority for the program is under section 104(b) (3) of the Clean Water  Act. Grant funding must be used to improve the wetlands program by  conducting or promoting the acceleration of research and studies relating  to the causes, effects, and other aspects of water pollution. Wetland grants  provide states, tribes, local governments, interstate agencies, intertribal  consortia, nonprofits, and nongovernmental organizations an opportunity  to carry out wetland projects and programs. Wetland grants are  discretionary grants.
Nonpoint Source Management Program (Nonpoint source grants).
Nonpoint source pollution is pollution that does not have a well-defined  source but instead originates from a number of sources, such as acid mine  drainage, agricultural runoff, and roads and highways. Under section  319(h) of the Clean Water Act, EPA makes grants to states, territories, and  Indian tribes to support a wide variety of activities, including technical and  financial assistance, education, training, technology transfer,  demonstration projects, and monitoring. Nonpoint source grants are  continuing environmental program grants.
Grants from these two programs can be incorporated into the National  Environmental Performance Partnership System, which was established in  response to state needs for greater flexibility in using and managing their  continuing grant funds. Under this system, states may enter Performance  Partnership Agreements with EPA and into Performance Partnership  Grants. The agreements set out jointly developed priorities and protection  strategies, including innovative solutions for addressing water, air, and  waste problems. The partnership grants allow states to combine  continuing environmental program grant funds to implement those  solutions. States can also enter into Performance Partnership Grants  without a Performance Partnership Agreement. Under traditional  continuing environmental program grants, states received funds to  implement particular waste, air, water or other program—such funding  can only be spent on activities that fall within the statutory and regulatory  parameters of that program. Under Performance Partnership Grants,  states can combine up to 21 separate grant programs into one award, and  move funds from one media, such as air, to another.
Both the wetland and nonpoint source grants are under the auspices of  EPA’s Office of Water in headquarters, but grants under these programs  are carried out by water program staff in regional offices. Moreover, in its  national guidance, the Office of Water states that it is committed to  accomplishing the goals that the Office of Grants and Debarment has  identified in its Grants Management Plan. To do this, for example, the  Office of Water provided regions with information on revised competition  and environmental results policies.
section_title: EPA Has Strengthened the Award Process, but Lack of Key Documentation Raises Accountability Concerns
paragraphs: EPA has strengthened its award process by, among other things, (1)  expanding the use of competition to select the most qualified applicants  and (2) issuing new policies and guidance to improve the awarding of  grants. However, EPA’s internal reviews of program and regional offices  have found weaknesses in documenting the review of grantees’ cost  proposals. We also found this weakness in one of the three regions we  visited. This documentation weakness may hinder EPA’s ability to ensure  the reasonableness of its grantees’ expenditure of federal funds. Because  of the continuing problems with documenting cost reviews, EPA is  reexamining its cost review policy for grants.
subsections: section_title: EPA Has Expanded the Use of Competition to Select the Most Qualified Applicants
paragraphs: To promote widespread competition for grants, in September 2002, EPA  issued a policy that for the first time required competition for many  discretionary grants. Before 2002, even though EPA had a competition  policy, it did not compete grants extensively or provide widespread  notification of upcoming grant opportunities.
EPA’s 2002 policy was designed to promote competition in awarding  grants to the “maximum extent practicable” and to ensure that the  competitive process was “fair and open.” This policy represented a major  cultural shift for EPA managers and staff, requiring EPA staff to take a  more planned, rigorous approach to awarding grants. Specifically, the  policy    was binding on managers and staff throughout the agency;  required EPA staff to determine evaluation criteria for grant solicitations  and publish a grant announcement at least 60 days before the application  deadline; and   created the position of a senior-level competition advocate for grants. The  advocate oversees the policy’s implementation and compliance and  evaluates its effectiveness.
According to EPA’s Inspector General, the 2002 policy was a “positive  step” toward promoting competition, and competitions under the policy  were generally fair and open.
Specifically, for the 38 grants that the Inspector General reviewed, EPA  had (1) published an announcement soliciting proposals, (2) written  procedures to ensure an objective and unbiased process for reviewing and  evaluating applications, and (3) selected recipients according to reviewers’  recommendations.
In 2004, EPA’s grants competition advocate reviewed the policy, as  required, and reported, among other things, that steps should be taken to  improve justifications for not competing certain grants by (1) increasing  review and approval requirements for exceptions to competition and (2)  clarifying the language in the policy to ensure appropriate use of  exceptions. The advocate also found that the threshold for requiring  competition for grants of $75,000 or more in the 2002 policy was too high.
In response, EPA issued a revised competition policy, effective January  2005. It enhanced competition by, among other things,  increasing review and approval requirements for justifying exceptions and  clarifying the language to ensure appropriate use of these justifications; reducing the threshold for competition from $75,000 to $15,000; and   strengthening requirements for documenting the competition process and  results.
In addition, EPA added (1) conflict-of-interest provisions to increase  awareness of situations that could arise for applicants, reviewers, and  others involved in competition matters; and (2) dispute procedures.
In commenting on both the 2002 and 2005 policies, the Inspector General  stated that the policies did not fully promote competition and  recommended that EPA could further expand competition in the 2005  policy by eliminating certain remaining exemptions and exceptions for  which the Inspector General believed competition is practicable. EPA  responded, however, that further expansion was not practicable for  reasons of congressional intent, regulatory limitations, and program  effectiveness.
EPA’s Grants Management Plan lays out goals, objectives, milestones,  and performance measures with targets for promoting competition. For  one of these objectives, EPA planned to improve the accuracy and  specificity of information available to the public on the agency’s grant  opportunities in the federal government’s Catalog of Federal Domestic  Assistance—a listing of available grants and other federal funding  opportunities (available at www.CFDA.gov). However, as we reported in  2005, EPA was not consistently providing this information. Without such  information, potential applicants might not apply, and EPA would not have  the broadest applicant pool from which to select grantees. EPA officials  were unaware of continuing problems with funding priorities and funding  levels in the Catalog of Federal Domestic Assistance until we brought  them to their attention during our review. In response to our  recommendations, in April 2005, EPA implemented revised guidance for  providing complete and accurate information in the Catalog of Federal  Domestic Assistance. For example, EPA now strongly encourages its  offices to provide information on the funding priorities on an ongoing  basis, instead of annually, so that the public has up-to-date information in  the Catalog of Federal Domestic Assistance.
For the competition goal, the agency developed a performance measure  for increasing the percentage of new grants subject to the competition  policy that are actually competed and set increasing targets for achieving  this measure. According to EPA, about $249 million of the approximately  $3.1 billion it awarded in new grants in fiscal year 2005 were eligible for  competition. EPA exempts certain grant categories from competition,  including all nondiscretionary grants, certain discretionary grants, and  most continuing environmental programs grants. EPA also established a  separate measure for nonprofit grantees. The first performance measure is  for all new eligible grants, including new grants to nonprofit recipients,  and the second is only for new eligible grants to nonprofit recipients, as  table 1 shows. EPA established a separate measure for competing grants  to nonprofit organizations because it believes that selecting the most  qualified nonprofit applicants through a competitive process could  address concerns about the effectiveness of nonprofit grantees in  managing grants.
As the table shows, EPA reports it now competes a higher percentage of  eligible grants, up from 27 percent in fiscal year 2002 to 93 percent in fiscal  year 2005, exceeding its targets for fiscal years 2003 through 2005. The 7  percent of new grants that EPA reported it did not compete—which  totaled about $10 million of the $249 million eligible for competition in  fiscal year 2005—resulted from exceptions to the policy. EPA’s  competition policy provides for exceptions that meet criteria specified in  the policy, if supported by a written justification and approved by an  appropriate official. Even after taking the exceptions into account, EPA  exceeded the 85 percent target it set for new grants in 2005. It has also  exceeded its target for new grants to nonprofit recipients in 2005.
section_title: EPA Has Issued New Policies and Guidance to Improve Grant Awards
paragraphs: To improve the award of grants, EPA issued additional policies and  guidance. Specifically:  In January 2005, EPA issued a policy to improve the description of the  grant in its grants database so that the description would be  understandable to the public. EPA now presents this information on the  Office of Grants and Debarment’s Web site—www.epa.gov/ogd—so that  the public has improved access to grant information.
In March 2005, EPA issued a policy establishing additional internal  controls for awarding grants to nonprofit organizations. The policy  addresses both the programmatic capability of a nonprofit applicant to  carry out a project and its administrative capability to properly manage  EPA grant funds—problems EPA and the Inspector General have  identified. Under the policy, EPA assesses programmatic capability for  both competitive and noncompetitive grants. The policy also requires the  agency to conduct different types of administrative capability reviews  based on the amount of the grant to the nonprofit organization. For grants  of $200,000 or more, applicants must complete a questionnaire and  provide documents to show that they have administrative and financial  systems to manage grants. For grants below the $200,000 threshold, EPA  staff must query the agency’s grants database for any findings of problems  in the applicant’s administrative capability. If problems are identified in  any of these reviews, the applicant must take corrective actions before  receiving the grant. In 2005, EPA approved 75 of the 87 nonprofit  organizations it reviewed; the remaining 12 nonprofit organizations are  taking steps to address problems identified.
Also in March 2005, EPA issued a memorandum clarifying the criteria that  must be documented to justify the use of a grant or a contract as the  award mechanism. EPA issued this guidance in response to a  recommendation in our 2004 report to better document the justification  for using grants rather than contracts.
In April 2005, EPA issued a policy memorandum and interim guidance  establishing a certification process that applies to certain discretionary  grant programs (currently 58). The new policy and guidance instruct  senior EPA officials—assistant administrators and regional  administrators—to certify, among other things, that (1) certain grant  awards and amendments identify environmental outcomes that further the  goals and objectives in the agency’s strategic plan and (2) there is no  questionable pattern of repeat awards to the same grantee. For  competitive announcements, these officials must certify that the (1)  expected outcomes from the awards under the proposed competitive  announcement are appropriate and in support of program goals and (2)  proposed competitive announcement is written in a manner to promote  competition to the maximum extent practicable. The Office of Grants and  Debarment has assigned a grant specialist to conduct random spot checks  of these certifications and provide assistance to program offices in  implementing this new policy.
section_title: Lack of Cost Review Documentation Could Compromise Accountability
paragraphs: While EPA has improved its award process, both EPA and we found  weaknesses in the agency’s documentation of its cost reviews before  awarding grants. EPA policy requires both the grants management and  program offices to conduct a cost review for every grant before awarding  it to ensure that the grantee’s proposed costs are necessary, reasonable,  allowable, allocable, and adequately supported. These reviews are central  to ensuring that EPA carries out its fiduciary responsibilities. However, in  2004 and 2005, in six of the seven program and regional offices it reviewed,  the Office of Grants and Debarment either found no documentation of cost  reviews or found that documentation was not sufficient. As a result of  these continuing documentation problems, EPA is reexamining its cost  review policy for grants.
We also found problems with cost review documentation in one of the  three regions we visited—Region 5. This region has a checklist to ensure  that staff members who are responsible for each aspect of the cost review  had completed and documented their review before awarding a grant. The  checklist requires approval from both the grant specialist and the project  officer on certain items and requires supervisors to review the checklist to  ensure that any concerns raised by the project officer or grant specialist  were addressed. While a project officer and grant specialist could initially  disagree on some aspects of the checklist, the regional office expects them  to resolve their differences and document the final resolution on the  checklist. However, for most of the 12 approved award files we reviewed,  we found instances in which the resolution of the issues was not  documented. Specifically,  the grant specialist and the project officer had both neglected to answer  the same two questions on the cost review checklist, or the grant specialist and the project officer did not agree on the answers to  multiple questions on the checklist and did not document any resolution of  their disagreements.
According to regional staff, these problems occurred because of workload  and errors. Nevertheless, the lack of documentation for awarded grants  raises concerns about the appropriateness of the award. More effective  supervisory review might have resulted in a documented resolution of  these differences.
section_title: EPA Has Improved In- depth Monitoring to Identify Agencywide Problems, but Weaknesses Remain in Ongoing Monitoring and in Closing Out Grants
paragraphs: EPA has improved some aspects of monitoring, but long-standing  problems in documentation and grant closeouts continue. EPA has made  progress in using in-depth monitoring to identify grantee problems  agencywide, but it does not always document whether corrective actions  have been taken. Furthermore, for ongoing monitoring, the agency found,  as we did in the regional offices, that in some cases agency staff do not  consistently document their monitoring of grantees, which hinders  accountability for effective grants management. Finally, we found that  grant closeouts were often delayed and sometimes improperly carried out,  which diminishes EPA’s ability to ensure that grantees met the terms and  conditions of their award and that grant funds were spent appropriately.  EPA has formed a work group to review its monitoring and closeout  polices and plans to revise these policies in 2006.
subsections: section_title: In-depth Monitoring Results Can Be Analyzed Nationwide to Identify Problems, but Staff Do Not Always Document Whether Corrective Actions Have Been Taken
paragraphs: EPA has made progress in conducting in-depth monitoring since it issued a  new monitoring policy in December 2002, which it revised in 2004 and  2005. Under its monitoring policy, grants management offices and  program offices in headquarters and the regions conduct in-depth  monitoring either (1) at the grantee’s location (on-site) or (2) at an EPA  office or at another location (off-site)—referred to as desk reviews. EPA’s  policy for these reviews requires the following, among other things:    Grants management offices must conduct in-depth administrative reviews,  on a minimum of 10 percent of grantees annually, to evaluate the grantee’s  administrative and financial capacity. For on-site administrative reviews,  EPA conducts “transaction testing”—that is, reviewing a grantee’s  accounting ledgers and underlying documentation for unallowable costs,  such as lobbying and entertainment expenditures.
Program offices must conduct programmatic reviews on a minimum of 10  percent of grantees annually to assess the grantees’ activities in key areas,  such as the progress the grantees are making in conducting the work and  in meeting the grant’s terms and conditions.
In 2003, we reported that although the in-depth review is a useful tool for  monitoring a grantee’s progress, the agency lacked a way to systematically  identify grantee problems agencywide because the information from its in- depth monitoring was gathered in a form that could not be readily  analyzed. We also found that the policy did not incorporate a statistical  approach to selecting grantees for review. Without a statistical approach,  EPA could not evaluate whether 10 percent was appropriate, nor could it  project the results of the reviews to all EPA grantees.
We recommended that EPA take action to address these issues. EPA has  since incorporated the data from its in-depth monitoring into a database,  analyzed the information to identify key problems, and taken corrective  actions to address systemic problems. By taking these actions, EPA has  found, among other things, that grantees have not had documented  policies and procedures for managing grants. Without these policies and  procedures, grantees may not be able to operate their financial and  administrative systems appropriately. As a result of this finding, EPA is  conducting the preaward reviews discussed earlier to ensure that  nonprofit grantees have required financial and administrative systems in  place. EPA has also increased training to grantees.
Since issuing its most recent revision to the monitoring policy in 2005,  EPA has initiated several practices that should further strengthen in-depth  monitoring. In 2006, it began incorporating a statistical approach for  selecting grantees for administrative in-depth reviews. In 2007, EPA plans  to use a statistical approach to select grants for programmatic in-depth  reviews. When the statistical approach is fully implemented, it should  significantly reduce the percent of grantees reviewed, according to an  agency official. Furthermore, the statistical approach will enable the  agency to project results among various types of grantees. EPA also began  incorporating transaction testing into administrative desk reviews in 2006  because it found that administrative desk reviews were not otherwise  yielding adequate financial information about grantees.
While EPA has improved its in-depth monitoring, the Office of Grants and  Debarment has found that staff do not always take corrective actions, or  document actions taken, to address findings identified during this  monitoring. The office found that corrective actions were documented for  only 55 percent of the 269 problems identified through administrative and  programmatic reviews. We reported similar results in August 2003.  According to an Office of Grants and Debarment official, while some EPA  staff took corrective actions, they did not document those actions in EPA’s  grantee computer database. Until this problem is addressed, the Office of  Grants and Debarment will not be able to fully assess the extent to which  corrective actions have or have not been taken to address identified  grantee problems. Without these assessments, EPA cannot be assured that  grantees are in full compliance with the terms and conditions of their  grants.
section_title: Inadequate Documentation of Ongoing Monitoring Hinders Accountability
paragraphs: Ongoing monitoring is critical because, in contrast to in-depth monitoring,  it is conducted on every grant at least once a year throughout the life of  the grant, and the results are used to determine whether the grantee is on  track to meeting the terms and conditions of its grant agreement. EPA’s  grant specialist and project officer manuals—used as training tools for  EPA staff involved in grants—emphasize that staff should properly  document grant monitoring activities to maintain an official agency  record. Agency officials state that proper documentation of monitoring is  necessary to ensure that third parties—such as other EPA staff who  assume responsibility for the grant or a supervisor—can fully understand  and review the actions that have occurred during the project period.  Moreover, a lack of documentation raises questions about the adequacy of  project officers’ and grant specialists’ ongoing monitoring of grantee  performance. Despite the importance of documenting ongoing monitoring,  the absence of documentation in grant files has been a long-standing  problem that we reported on in 2003.
To conduct ongoing monitoring, EPA policy requires the following:    Grant specialists should ensure that administrative terms and conditions  of the grants are met and review the financial status of the project. The  grant specialist is to speak with the project officer and the grantee at least  annually during the life of the grant.
Project officers should ensure that programmatic award terms and  conditions are being met, including ensuring that they have received  progress reports from the grantee. Project officers are also to speak with  the grant specialists and the grantee at least annually during the life of the  grant.
According to the monitoring policy, the grant specialists and project  officers must document the results of their ongoing monitoring in their  grant files.
Despite this policy, the lack of documented ongoing monitoring remains a  problem. EPA’s recent internal reviews in program and regional offices  demonstrate—as did our review in three regional offices—that EPA grant  specialists and project officers still do not consistently document ongoing  monitoring. In 2004 and 2005, the Office of Grants and Debarment found  limited or incomplete documentation of ongoing monitoring in internal  reviews it conducted in seven program and regional offices. In addition,  self-assessments completed by 11 program and regional offices during this  period identified the same lack of documentation. Our analysis of these  reviews indicates that several offices experienced recurring problems in  2004 and 2005. For example, an August 2004 Office of Grants and  Debarment internal review cited one regional office as having “very  limited” documentation of ongoing monitoring; and in the following year,  the regional office’s self-assessment found the same documentation  problem with project officer files.
Because of these documentation problems, two of the three regional  offices we visited have committed to using checklists to document their  ongoing monitoring. Regions 1 and 9 had implemented such checklists at  the time of our review. As table 2 shows, however, of the 40 project officer  and grant specialist files we reviewed in Regions 1 and 9, more than half of  the checklists were either missing, blank, or incomplete.
The water program office in Region 5 also developed a checklist for  documenting ongoing monitoring but had not yet implemented it at the  time of our review. Consequently, in Region 5, we examined other  documentation of ongoing monitoring in the grant files and found similar  omissions. None of the six files requiring annual contact with the  grantee—three grant specialist files and three project officer files—had  documentation showing that this contact had occurred.
In the three regions, we also found that project officers’ files did not  always contain grantees’ progress reports, which can be required  quarterly, semiannually, or annually, as defined by an individual grant’s  terms and conditions. Thirteen of the 32 project officer grant files we  reviewed in these regions were missing at least one or more progress  reports required by the grant’s terms and conditions. According to EPA’s  project officer manual, progress reports are the project officer’s primary  mechanism for determining if the grantee is fulfilling its grant agreement  obligations. In general, progress reports should contain information that  compares grantee progress with the stated grant objectives, identify  problems with meeting these objectives, and state the reasons for those  problems. While the submission of progress reports is clearly the grantee’s  responsibility, it is also the project officer’s responsibility to work with the  grantees to ensure that they provide their progress reports in accordance  with the terms of the grant. When EPA staff do not obtain progress  reports, they cannot monitor effectively, which may hinder accountability.
In the three regions we visited, the lack of documentation for ongoing  monitoring occurs because of weaknesses at the staff, supervisory, and  management level. First, grant specialists and project officers do not  consistently document key monitoring efforts. For example, several staff  stated that they had not printed out their e-mail correspondence with  grantees or recorded those contacts in the official grant files. Other staff  cited their workload as a reason for not documenting monitoring. Lack of  documentation also occurs because grant specialists and project officers  rely on other staff with technical expertise, known as “technical contacts,”  to assist with ongoing monitoring, and these technical contacts may not  provide the documented results of their monitoring for inclusion in the  grant file. We found this situation had occurred in two of the three regions  we visited. For example, one administrative project officer—a project  officer who maintains files but is not necessarily knowledgeable about the  technical aspects of the project—had asked for key monitoring  documentation from a technical contact, who did not provide it. The  technical contact had the monitoring documents in his work area and said  he would routinely provide them to the project officer in the future.
Second, the lack of ongoing monitoring documentation may occur, in part,  because supervisors do not always effectively review grant files for  compliance with grant policies. According to staff we interviewed in the  three regions, to their knowledge, their supervisors had not reviewed their  files to assess compliance with the agency’s monitoring policies, which  could contribute to the lack of documentation. A regional project officer  told us that he would have completed the ongoing monitoring checklist if  his regional program supervisor had made it a priority. In another region,  officials told us that some supervisors do review some files, but they do  not have enough time to review every file.
In contrast, supervisory review can contribute to complete documentation  of ongoing monitoring. For example, Region 5 was cited as having  “excellent” documentation for ongoing monitoring in an Office of Grants  and Debarment 2003 internal review. According to the EPA supervisor in  Region 5 at the time of the 2003 review, she had notified staff that she  would review their grant files to assess compliance with EPA policy for  ongoing monitoring, among other things. She believes that her review  contributed to the region’s excellent rating.
Third, senior EPA managers in the regions do not always ensure that their  commitments to improve monitoring documentation are being met. For  example, in the post-award monitoring plans submitted to the Office of  Grants and Debarment for two of the EPA regions we visited, the plans  stated that the regions would place a checklist in the grant specialist and  project officer files documenting ongoing monitoring activities. Although  the two regions developed the checklists, more than half of the checklists  we reviewed were missing, blank, or incomplete. This occurred, in part,  because senior managers did not ensure that the commitments they made  were met in their post-award monitoring plans.
Despite the importance of ongoing monitoring, EPA has not created a  performance measure for documenting ongoing monitoring that would  underscore its importance to managers and staff. Furthermore, EPA’s  Integrated Grants Management System has a field for recording  information about ongoing monitoring that could enable the agency to  systematically identify whether this monitoring is documented  agencywide, but recording this information is optional. Establishing a  performance measure and/or requiring the entry of information could  enhance accountability for implementing the monitoring policy.
section_title: EPA Has Reduced Its Closeout Backlog, but Grant Closures Are Often Delayed and Sometimes Improperly Carried Out
paragraphs: As part of its grant reforms, EPA incorporated grant closeout into its  monitoring policy and its Grants Management Plan. During closeout,  EPA ensures that the grant recipient has met all financial requirements  and provided final technical reports, and ensures that any unexpended  balances are “deobligated” and returned to the agency. Delays in closing  out the grant can unnecessarily tie up obligated but unexpended funds that  could be used for other purposes. Furthermore, according to EPA’s  closeout policy, closeout becomes more difficult with the passage of time  because persons responsible for managing various aspects of the project  may resign, retire, or transfer; and memories of events are less clear.
The monitoring policy states that the agency is committed to closing out  grants within 180 days after the end of the grant’s project period. Under its  monitoring policy, EPA provides 180 days for closeout because (1)  grantees—by regulation and policy—have up to 90 days after the grant  project period to provide all financial and technical reports; and (2) by  policy, agency staff—grant specialists and project officers—have 90 days  to review grantee information and certify that financial and technical  requirements have been met. Following certification, the grant specialist  closes out the grant with a letter to the grantee stating that the agency  closed out the grant. EPA’s Grants Management Plan identified measures  with targets that were developed to assess EPA’s closeout performance.
In reviewing EPA’s management of grant closeouts, we found that EPA (1)  has effectively reduced its historic backlog of grants due for closeout; (2)  does not always close out grants in a timely way—within 180 days after  the project period ends, as required by agency policy; and (3) does not  always close out grants properly based on the regional files we reviewed.
subsections: section_title: EPA Has Effectively Reduced Its Historic Closeout Backlog
paragraphs: In the past, EPA had a substantial backlog of grants that it had not closed  out. EPA reported that by 1995, the agency had amassed a backlog of over  18,000 completed grants that had not been closed out from the past 2  decades. In fact, EPA had identified closeout, among other things, as a  material weakness—an accounting and internal control weakness that the  EPA Administrator must report to the President and Congress. As we  reported in 2003, however, EPA improved its closeout of backlogged  grants, eliminating backlog as a material weakness. Specifically, for fiscal  year 2005, using its historic closeout performance measure, EPA reported  that it had closed 97.8 percent of the 23,162 grants with project end dates  between the beginning of fiscal year 1999 and the end of fiscal year 2003.  EPA came close to its 99-percent target of closing out this backlog.
section_title: EPA Does Not Always Close Out Grants in a Timely Manner
paragraphs: EPA developed a second closeout performance measure—which we call  the current closeout performance measure—to calculate the percent of  grants with project end dates in the prior fiscal year that were closed out  by the end of the current fiscal year (September 30). For example, as table  3 shows, EPA closed out 79 percent of the grants with project end dates in  fiscal year 2004 by the end of reporting fiscal year 2005 (September 30,  2005) but did not meet its performance target of 90 percent.
EPA’s current closeout performance measure does not calculate whether  EPA closed the grant within 180 days. Rather, this measure only reports  whether EPA closed the grant by the end of the following fiscal year (the  fiscal year in which it reports on closeouts—the reporting year). The  measure, in fact, can allow for a much more generous closeout time, from  183 days beyond the 180 days to as much as 547 days (18 months) beyond  the 180 days—because EPA does not report the performance measure  until September 30, the end of the current fiscal year, as shown by  hypothetical examples in table 4.
EPA’s current performance measure for closing out grants is a valuable  tool for determining if grants were ultimately closed out. However, we  believe that this performance measure—taken alone—is not a sufficient  way to measure closeout because it does not reflect the 180-day standard  specified in EPA policy.
To determine the percentage of grants that were closed within 180 days,  we examined EPA’s analysis of closeout time frames for regional offices,  headquarter offices, and agencywide. As table 5 shows, EPA is having  significant difficulty in meeting the 180-day standard.
As the table shows, agencywide, only 37 percent of grants with project end  dates in fiscal year 2004 were closed out within 180 days, 25 percent were  significantly late—at least 3 months beyond the 180-day standard, and 19  percent were not closed.
Table 6 shows that EPA’s current performance measure is masking the  fact that the agency is having significant difficulty in closing out grants  within 180 days.
In guidance on preparing the annual post-award monitoring plans, the  Office of Grants and Debarment has indicated that agency offices should  use the agency’s current closeout performance measure—90 percent of  the grants with project end dates in the prior fiscal year—as the closeout  goal. In effect, as a regional grants management office manager stated,  the performance measure, not the 180-day standard, is the target EPA is  working toward for closing out grants.
At the regional level, our analysis of closeout data for the wetland and  nonpoint source grant programs indicates that grants were closed out late  because of (1) grantee delays and/or (2) internal delays within the agency.  We reviewed 34 closed grants in three regions. First, as table 7 shows,  grantees often submit their final financial and technical reports after the  90 days that they are allowed.
According to regional staff, different types of grantees may be submitting  their reports late for different reasons. Specifically:    States do not always provide their final technical and financial closeout  reports on time. The states may not be on time because, for example, they  (1) are understaffed; (2) are awaiting the completion of work conducted  by sub-grantees or subcontractors—which can be legitimately delayed  because of weather conditions that affect the project’s progress; or (3)  consider closeout a lower priority than applying for new grants.  Furthermore, states do not believe there will be any consequences if they  submit final reports late because their grants are for continuing  environmental programs.
Tribes may submit their final reports late because of high turnover among  tribal staff and limited organizational capacity, and because tribal  councils, which meet intermittently, must approve the reports.
While grantees are responsible for providing final reports within 90 days,  EPA staff are responsible for working with the grantees to ensure that the  reports are received on time. Under EPA’s 1992 closeout policy, grant  specialists must notify grantees 90 days before the project end date that  final reports will be due 90 days thereafter. However, we found that  regional staff do not always send out these letters. For example, Region 5  adopted the practice of reminding grantees 45 days before the project end  date because the grants management office believes that the 90 days is too  long in advance to be effective. However, several Region 5 grant  specialists stated that their workload is also preventing them from sending  out the 45-day letter to grantees.
According to EPA’s closeout policy, if a grantee is late with the final  financial or technical report, the region should send reminder letters that  escalate in tone as time progresses. In Region 1, for example, if the grantee  does not submit its materials within 90 days, the grants management office  sends a letter asking that the grantee contact its grant specialist to discuss  the reasons for the overdue reports; if the region does not receive the  reports within 120 days after the project has ended, the grants  management office sends a certified letter that more strongly calls for the  submission of these required reports.
When these letters do not result in grantee compliance, regional staff and  managers told us that they have no realistic option for taking strong action  against states that are late—such as withholding money—because these  grantees have continuing grants for environmental programs.
Second, late closeouts result from a variety of internal agency delays. As  shown in table 8, of those files that had information that we could use to  determine the dates the reports were submitted, regional staff closed out  about half within the 90 days provided for in EPA guidance. For 9 of the 30  files that had this report information, it took the project officers or the  grant specialists over 180 days to close out the grant after receiving the  final reports from the grantees.
For grant closeouts, generally, regional staff often cited workload as a  factor contributing to delays in agency closeout. Delays also occurred  because of peak workload periods during the year, such as the fourth  quarter of the fiscal year, when regions generally give priority to awarding  new grants. Officials in the three regions we visited also told us that they  have transferred or will transfer the administrative and financial functions  of grants closeout to EPA’s Las Vegas Finance Center, which should  reduce the grant specialists’ workload, allowing them to focus on other  aspects of grants management.
Regional practices also may have contributed to delays in two of the three  regions we reviewed. Region 5 had two practices that contributed to  delays in closing out grants. First, the region uses technical contacts to  assist with monitoring the wetland and nonpoint source grant programs,  including closeouts. The project officer first reviews the grantee’s final  reports to ensure they are complete and then asks the technical contact to  comment on specific points and certify in writing that technical  requirements have been met. The project officer then certifies in writing to  the grant specialist that the grantee has met programmatic terms and  conditions and, from that perspective, the grant can be closed out.  Regional staff stated that in certain cases the added step of getting signoff  by the technical contact resulted in closeout delays because the technical  contact did not always review the grantee’s final reports in a timely way.
Second, to address its closeout problem, the region’s grants management  office attempted an administrative change to expedite closeout—having a  single grant specialist manage closeout. When this approach did not prove  effective, the region returned to its practice of having the original grant  specialists responsible for closing out grants. According to regional staff,  the transition to and from this process exacerbated delays in grant  closeouts. The original grant specialists had other grant work and waited  until that work was completed before closing out the grants that were  returned to them. Region 5 had the lowest percentage of grants closing out  within 180 days for all its programs among EPA’s 10 regions (16 percent  for fiscal year 2005 as shown in table 5).
Region 9 had delayed closures for continuing nondiscretionary grants, in  part, because of a practice, discontinued in November 2004, of routinely  carrying over unspent funds from these grants. That is, the region would  not close out a grant until it had awarded a new grant. The unspent funds  from the old grant would then be processed as an amendment to the new  grant, in order to allow grantees to keep their unspent funds. For example,  one state nonpoint source grant was closed 278 days beyond the 180 days  because the project officer had asked the grant specialist to carry over  $426,000 in unspent funds to the following year’s grant.
Overall, a combination of grantee lateness and internal inefficiencies  contributed to late closeouts. For example:  In Region 5, it took 795 days—615 days beyond the 180-day standard—to  close out a 2-year wetland grant for $56,778. The grantee submitted the  final financial status report 114 days late because a key grant contact had  died. However, it took the region an additional 591 days after the grantee  provided the final reports to close out the grant. According to the grant  specialist, closeout was delayed, in part, because of internal administrative  delays and because the grant was “lost” under a stack of other closeout  files.
In Region 1, closure of a nonpoint source grant that provided $796,532  over 10 years was delayed primarily because of a lack of documentation.  According to the project officer who inherited the file from a retiring  employee, the file had unusually poor documentation, with no assurance  that the grant’s terms and conditions had been met. Moreover, the state  employee who assumed responsibility for the grant could not locate all the  reports detailing how the grant money had been used. Consequently, it  took the project officer nearly 5 months beyond the allotted 180 days to  review available information, ascertain that grant activities had been  completed, and close out the grant.
According to some of the project officers and grant specialists staff with  whom we spoke, the 180 days allowed for closeout in EPA’s policy is a  reasonable amount of time. Moreover, some staff said that if more days  were allowed, EPA might take longer. As noted in the closeout policy, as  more time passes and the original grant specialists and project officers  move on, it becomes more difficult to close out a grant. Finally, one  regional official pointed out that if the deadline for closeout were  extended, then unexpended funds would go unused for longer periods of  time, which would tie up funds that could have been used for other  purposes.
We note, however, that EPA still has a 1992 closeout policy that is not  consistent with its current monitoring policy. Specifically, although both  the 1992 closeout policy and the monitoring policy state that closeout  should occur within 180 days after the end of the project period, the 1992  policy also states that closeout should occur within 180 days after receipt  of all required reports and other deliverables. This aspect of the 1992  policy could be construed to mean that EPA has up to 270 days to close  out grants since grantees have up to 90 days to submit their reports. Office  of Grants and Debarment officials stated that EPA has formed a work  group to review its monitoring and closeout polices. As part of its review,  the office plans to examine this inconsistency and the reasonableness of  the 180-day closeout requirement. It expects to revise these policies in  2006.
subsections: section_title: Grants Were Not Always Closed Out Properly
paragraphs: Adding to the agency’s closeout problems, 8 of the 34 closed grants we  reviewed in the regions were not closed out properly. Specifically:    Region 1 grant specialists had not adequately reviewed the indirect cost  rate grantees submitted as part of their final financial status report which,  in turn, led to improper closeout in 5 of the 10 files we reviewed.  Reviewing the files’ final financial report checklist, we found instances in  which the question on the checklist that addresses indirect cost rates had  been left blank or had been answered incorrectly. This problem occurred,  in part, because the grant specialists did not adequately review the work of  student interns who initially reviewed the financial status reports and  completed the checklists. These “noncore” employees were used to help  reduce the grant specialists’ workload and the grant specialists were  expected to review their work before they signed off on the checklist.
In Region 5, one grant specialist’s file was missing the final financial status  report, which is a key report that describes how the grantee spent the  grant funds and whether any unspent funds remain that need to be  deobligated.
In Region 9, Lobbying and Litigation Certification Forms—whose purpose  is to ensure that federal dollars are not spent for lobbying or litigation  activities—were missing from two grant files. After waiting some time, the  grant specialist decided to close out the grants without the forms. The  grant specialist manual states that grant specialists are responsible for  notifying the grants management office if the grantee has not complied  with this certification requirement.
EPA’s guidance states that inadequate file documentation, among other  things, (1) violates the file management requirement that all significant  actions must be documented, (2) provides an incomplete historical record  of a grant project, (3) prevents staff from substantiating facts if a dispute  arises, and (4) creates the appearance of poor grant administration and  oversight. Furthermore, the guidance specifically states that the file  should include evidence of closeout, including the final report or product.
In Region 1, we also identified an accountability concern when grants  were closed out by administrative project officers. An administrative  project officer for a Performance Partnership Grant had not always  received written approval from the technical contacts, who evaluated  grantee documents before the administrative project officer certified that  the grantee had met all the terms and conditions of the grant. According to  a regional official, technical contacts at times tell the project officer that  they have reviewed technical documents but do not provide written  approval. Although the administrative project officer certified that  grantees met their programmatic obligations, the administrative project  officer was “uncomfortable” doing so without written approval from  technical contacts that they had reviewed final documents.
As with monitoring, without effective supervisory review of the grant and  project officer files, grants may be improperly closed out. With more  effective supervision, grants would be more likely to be properly closed  out.
section_title: EPA Has Initiated Actions to Obtain Results from Grants, but Its Efforts Are Not Complete
paragraphs: EPA has taken steps to obtain environmental results from its grants, but  its efforts are not complete. First, EPA included a performance measure in  its Grants Management Plan for identifying expected environmental  results in grant workplans. In 2004, EPA was far from meeting its  performance target. Although EPA does not yet have final data for 2005,  EPA officials told us that their preliminary data indicate they are closer to  meeting this performance target. Second, EPA issued an environmental  results policy, effective in January 2005 that for the first time requires EPA  staff to ensure that grants specify well-defined environmental outcomes.  However, EPA’s current performance measure does not take into account  the new criteria for identifying and measuring results from grants  established by the policy. EPA acknowledges that it has not yet fully  identified better ways to integrate the agency systems for reporting on the  results of grants. While EPA has taken these positive steps, OMB’s  evaluations of EPA grant programs in 2006 indicate that EPA must  continue its concerted efforts to achieve results from its grants.
subsections: section_title: EPA Has Developed a Performance Measure and Issued a New Policy for Specifying Environmental Results
paragraphs: The Grants Management Plan established a performance measure for  identifying environmental outcomes from grants: the percent of grant  workplans that discusses how grantees plan to measure and report on  environmental outcomes. In 2004, EPA was far from meeting its  performance target. Although EPA does not yet have final data for 2005, an  EPA official told us that their preliminary data indicate that the agency is  closer to meeting this performance target of 80 percent for 2005.
EPA also issued an environmental results policy in 2004, which was  effective in January 2005, or about 2 years later than proposed in the  Grants Management Plan. The policy is promising in that—for the first  time—it requires EPA staff to ensure that grant workplans specify well- defined environmental outputs (activities) and environmental outcomes  (results), which enables EPA to hold grantees accountable for achieving  them. However, planning for grants to achieve environmental results, and  measuring results, is a difficult, complex challenge. As we have reported,  while it is important to measure the results of environmental activities  rather than just the activities themselves, agencies face difficulties in  doing this. Environmental outputs are inherently easier to develop and  report on than environmental outcomes.
The policy is also promising because, among other things, it (1) is binding  on managers and staff throughout the agency; (2) emphasizes  environmental results throughout the grant life cycle—awards,  monitoring, and reporting; and (3) requires that grants be aligned with the  agency’s strategic goals and linked to environmental results.
To align grants with the agency’s strategic goals and link the grants to  results, the policy requires for the first time that EPA program offices  ensure that (1) each grant funding package includes a description of the  EPA strategic goals and objectives the grant is intended to address and (2)  the offices provide assurance that the grant workplan contains well- defined outputs and, to the “maximum extent practicable,” well-defined  outcome measures. Outcomes may be environmental, behavioral, health- related, or programmatic in nature, and must be quantitative. EPA  included the provision to “the maximum extent practicable” in the policy  because it recognized that some types of grants do not directly result in  environmental outcomes. For example, EPA might fund a research grant  to improve the science of pollution control, but the grant would not  directly result in an environmental or public health benefit. In June 2005,  the EPA Inspector General found that the agency’s results policy was  generally consistent with leading nongovernmental organizations that fund  environmental projects and that emphasize grants performance  measurements.
section_title: Efforts to Obtain Results from Grants Are Not Yet Complete
paragraphs: EPA’s performance measure and the new results policy are positive steps,  but the agency’s efforts to address results are not yet complete. Although  EPA has issued an environmental results policy, its current performance  measure does not take into account the new criteria for identifying and  measuring results. EPA has identified the following seven criteria that  grant agreements should meet and is using these seven criteria as the basis  for assessing the implementation of the policy. That is, the agreements  should  include a description of how the grant is linked to EPA’s Strategic Plan,   specify at least one EPA goal and its related objective that the project  identify the appropriate program results code—a code applied to new  grant awards that aligns the grant with EPA’s strategic goals and  objectives, include an assurance that the program office has reviewed the workplan  and that the workplan includes well-defined outputs and outcomes,  include a requirement for performance reports from recipients, include well-defined outputs in the workplans, and  include well-defined outcomes in the workplans.
According to an Office of Grants and Debarment official, the results policy  calls for outcomes that are not only well defined but that also include  quantitative measures. However, recognizing the difficulty and complexity  posed by applying such measures, the official said, for the purposes of  assessment, EPA modified its criteria to include well-defined outcomes  with or without quantitative measures.
Since EPA has adopted new criteria for assessing environmental results  from grants based on its environmental results policy, its current  performance measure—the percentage of grant workplans that discuss  how grantees plan to measure and report on environmental outcomes— may not be sufficient to assess the implementation of the policy. EPA’s  current performance measure does not take into account the new criteria  for identifying and measuring results from grants established by the policy.  Establishing a new performance measure and target to reflect the new  policy would enhance EPA’s ability to assess the agency’s effectiveness in  implementing the policy.
In addition, EPA continues to face difficulties in ensuring that its grants  are achieving public health and environmental results. Specifically, EPA  acknowledges that it has not yet fully identified better ways to integrate  the agency systems for reporting on the results of grants. EPA does not  have a systematic way of collecting information about the results of its  grants agencywide. As stated in the results policy, the Office of Grants and  Debarment convened a workgroup to (1) examine existing EPA systems  for collecting results from grant programs, (2) identify better ways to  integrate these systems, and (3) potentially amend the policy to reflect its  findings. The workgroup has begun an inventory of existing EPA systems.  Until recently, EPA recognized—but had not addressed in its results  policy—the known complexities of measuring environmental outcomes:  (1) demonstrating outcomes when there is a long lag time before results  become apparent and (2) linking program activities with environmental  results because of multiple conditions that influence environmental  results. In April 2006, the Office of Grants and Debarment provided an  online training course for project officers on environmental results with  guidance on how to address these measurement complexities.
Furthermore, OMB has found that EPA has problems in demonstrating  results from its grants. Using its Program Assessment Rating Tool (PART),  OMB annually evaluates federal programs in four critical areas of  performance: program purpose and design, planning, management, and  results, each scored from 0 to 100. OMB combines these scores to create  an overall rating: effective, moderately effective, adequate, and ineffective.  In addition, programs that do not have acceptable performance measures  or have not yet collected performance data generally receive a rating of  “results not demonstrated.” As table 10 shows, the PART ratings have  found that some of EPA’s programs are “ineffective” or “results not  demonstrated,” although there has been some improvement from 2004  through 2006.
Despite this progress, a closer examination of the ratings for 2006  indicated that, with one exception, the scores for the results component  were lower than the scores given to other components. (See table 11).
While EPA has taken positive steps, OMB’s 2006 assessment indicates that  EPA must continue its concerted efforts to achieve results from its grants.
section_title: EPA Has Taken Steps to Manage Grants Staff and Resources More Effectively but Still Faces Major Management Problems
paragraphs: EPA has taken steps to manage grants staff and resources more effectively  in four key areas: (1) analyzing workload; (2) providing training on grant  policies; (3) assessing the reliability of the agency’s grants management  computer database—the Integrated Grants Management System; and (4)  holding managers and staff accountable for successfully fulfilling their  grant responsibilities. Because much remains to be accomplished,  management attention to these issues is still needed.
subsections: section_title: EPA Is Examining Grants Staff Workload
paragraphs: As we reported in 2003 and found again in this review, regional grants  managers and staff are concerned that staff do not have sufficient time to  devote to effective grants management. They pointed out that the new  policies increased the time needed to implement each step of the grants  process, such as the more planned, rigorous approach now required for  competing grants. However, one regional official pointed out that this  increased workload has not been offset with an increase in resources or  the elimination of other activities.
Fulfilling an objective identified in the Grants Management Plan, in April  2005, an EPA contractor completed a workload analysis of project officers  and grant specialists. The analysis showed that EPA had an overall  shortage of project officers and grant specialists, expressed in full-time  equivalents. However, the contractor recommended that before EPA adds  staff, it take steps to improve the effectiveness and efficiency of its grants  management operations. For example, the contractor recommended that  EPA review its grant activities and assign “noncore” activities where  possible to auxiliary federal or nonfederal staff to improve operations,  freeing EPA staff to conduct their core work. It defined noncore activities  to typically include those that are related to grant closeouts.
The Office of Grants and Debarment asked the grant offices to prepare  project officer workforce plans—due in 2006—that incorporate the  workload analysis to promote “accountable” grants management.
section_title: EPA Has Provided Some Training on Grant Policies
paragraphs: As outlined in the Grants Management Plan, EPA has developed a long- term grants management training plan. Under the plan, EPA continues to  certify project officers for grant activities by requiring them to take a 3-day  project officer course before they are allowed to conduct grant  management activities, and thereafter take a refresher course to maintain  their certification.
To address the grant reforms, the agency provided additional training. For  example, EPA held a grants management conference in 2004, attended by  465 EPA staff, which included workshops on new policies. In 2005, the  Office of Grants and Debarment conducted agencywide training on the  new competition policy. It also conducted training on the environmental  results policy.
However, according to EPA staff, the amount of training has not been  sufficient to keep pace with the issuance of new grant policies. For  example:    A 2006 self-assessment conducted by one program office found that  project officers and managers expressed frustration that both the pace and  complexity of new policy requirements left project officers vulnerable  because they were not properly trained in the policies.
A 2005 Region 9 self-assessment found that the region’s project officers  did not believe that they had received sufficient guidance from their  programs in headquarters.
A Region 1 official stated that the rapid pace of new policies and brief lead  time between issuance and the effective date made it too difficult for the  regions to adequately train staff on all the new policies related to grants  management. Nevertheless, Region 1 developed a training course for its  project officers on the award process to address new grant reform policies  issued in 2005. However, only about 25 of the region’s 200 project officers  attended the optional 90-minute course, although there were three  opportunities to do so.
Regional officials also noted that the grant reforms are changing the skill  mix required of both project officers and grant specialists. According to a  Region 5 official, the grant specialist was once a clerical position, but  additional responsibilities required under the new grants policies indicates  that a business degree or financial background would be helpful. Region 9  officials told us that traditionally project officers had technical and  scientific skills. However, the grant reforms had increased the need for  interaction with grantees, which required more skills in oral  communications, organization, and analysis. An Office of Grants and  Debarment official explained the agency is weighing what should be  considered as the right skill mix for agency staff involved in grant  activities.
section_title: EPA Has Developed a Grants Database but Has Not Completed a Data Quality Review
paragraphs: In 1997, EPA began developing the Integrated Grants Management System  to better manage its grants, and EPA now also uses this database to inform  the public and the Congress about its $4 billion investment in grants. Data  quality problems in this database could impair the agency’s ability to  effectively manage grants and provide accurate information. In 2005, we  recommended that EPA conduct a comprehensive data quality review of  its Integrated Grants Management System. EPA undertook a review,  which it expects to be completed in 2006.
section_title: EPA Has Taken Steps to Enhance Staff and Management Accountability, but Concerns Remain
paragraphs: EPA’s Grants Management Plan included an objective of establishing  clear lines of accountability for grants management, including  performance standards that address grants management responsibilities  for project officers. As we reported in 2003, project officers did not have  uniform performance standards; instead, each supervisor set standards for  each project officer, and these standards may or may not have included  grants management responsibilities.
Later in 2003, EPA’s Assistant Administrator for the Office of  Administration and Resources Management asked all senior resource  officials to review the current performance standards of all employees  below the senior executive service who had grants management  responsibilities. This review was to ensure that the complexity and extent  of these employees’ grants management duties were reflected in their  performance standards and position descriptions. The Assistant  Administrator asked senior resource officials to ensure that such  standards were in place. The Office of Grants and Debarment is assessing  the extent to which the guidance was implemented; the assessment is to  be completed in May 2006.
As we reported in 2003, the Office of Grants and Debarment faces some  difficulties in holding managers and staff accountable for effective grants  management. The office does not directly oversee many of the managers  and staff who perform grants management duties, particularly the  approximately 2,100 project officers in headquarter and regional program  offices. This division of responsibilities makes it more difficult to hold  these staff accountable for grants management.
In 2005, EPA’s Inspector General reported that EPA was not holding  supervisors and project officers accountable for grants management.  Specifically:    EPA does not have a process to measure an individual project officer’s  performance in carrying out grants management duties. In practice,  supervisors relied on project officers to inform them of grants  management weaknesses.
EPA managers and supervisors are not discussing project officer grants  management responsibilities during end-of-year evaluations. Managers  were not discussing project officers’ grants management responsibilities  during year-end evaluations; and, if grant issues were addressed, the  discussion focused on the grant recipient’s performance, rather than on  the project officer’s performance. Supervisors provided various reasons  for rating project officers without discussing grants management  responsibilities, stating, for example, that the year-end evaluation should  focus on problems or issues with grantee performance, and project  officers’ responsibilities should be discussed at staff meetings or at other  times through the year.
EPA managers had not conveyed weaknesses from the agency’s internal  reviews and self-assessments to project officers. EPA managers did not  communicate weaknesses identified in internal reviews, such as a lack of  documentation of cost reviews and ongoing monitoring, and supervisors  were not aware of these identified weaknesses.
Our review is consistent with the Inspector General’s findings. As  previously discussed, EPA grants staff told us that their supervisors were  not reviewing their grant files to determine compliance with grant  monitoring policies. It is possible that the awarding, monitoring, and  closeout problems we found would have been mitigated by effective  supervisory review.
In response to the Inspector General’s concerns, EPA issued a plan in  January 2006 to ensure that the agency’s new performance appraisal  system—Performance Appraisal and Recognition System—addresses  grants management responsibilities. The new system requires that (1)  appraisals of project officers and supervisors/managers include a  discussion of grants management performance; (2) performance  agreements and associated mid-year and end-of-year performance  discussions focus on key areas of preaward reviews of nonprofit grantees,  competition, post-award monitoring, and environmental results; and (3)  performance discussions take into account the results of internal reviews,  such as those conducted by the Office of Grants and Debarment, self- assessments, and performance measure reviews. For the 2007  performance appraisal process, EPA plans to establish a workgroup to  develop final performance measures to assess the grants management  performance of project officers and supervisors and plans to incorporate  these measures into 2007 performance agreements.
More broadly, to address the growing demands of the grant reforms and  enhance accountability, the Office of Grants and Debarment formed a  senior-level grants management council that cuts across the organization  by including representatives from program offices, such as the Office of  Water, and regional offices. The council is to help develop and implement  new policies agencywide. Similarly, the regional offices we visited have  formed grants management councils to coordinate and implement grant  reforms with the region’s grants management office and various program  offices.
Despite the efforts of these various national- and regional-level offices,  managers, and councils to identify problems and undertake corrective  actions, some grants management problems still persist. For example,  although some of the regions we visited had implemented checklists as  internal controls to ensure the documentation of ongoing monitoring, the  regions did not ensure they were actually completed. Closeout problems  that were identified by EPA’s current performance measure have not been  effectively addressed.
section_title: Conclusions
paragraphs: About 3 years into its Grants Management Plan, 2003-2008, EPA has  made important strides in achieving its grant reforms, particularly in  competing a higher percentage of grants and trying to identify results from  its grants. However, EPA has not resolved its long-standing problems in  documenting ongoing monitoring and closing out grants. As it revises its  management plan, EPA has an opportunity to tackle these continuing  problems.
Without adequate documentation of ongoing monitoring, EPA cannot be  fully assured that grantees are on track to fulfilling the terms and  conditions of their grants. Furthermore, the agency’s lack of  documentation indicates weaknesses at all levels: staff do not always  document their monitoring; supervisors do not always effectively review  grant files; and managers are not always meeting their commitments to  address known problems with lack of documentation. Despite the  importance of ongoing monitoring, EPA has not created a performance  measure and target for documenting monitoring, which should elevate the  importance of ongoing monitoring to the agency. EPA is also not taking  full advantage of its grants management database, which has a data field  for documenting ongoing monitoring. That is, EPA does not require  project officers and grant specialists to enter monitoring documentation  information into its database. With the information in the database, EPA  would be better able to determine that staff are meeting documentation  requirements. Use of performance measures and targets for ongoing  monitoring as well as the database could enhance accountability.
EPA’s current performance measure and target for closing out grants is a  valuable tool for determining if grants were ultimately closed out, but it is  not a tool for determining whether grants were closed out within the 180- days now specified in EPA’s monitoring policy. EPA needs an  accompanying measure that accurately reports whether grants are closed  out within the 180 days or other standard EPA may establish. EPA’s  current measure and target mask the fact that, agencywide, almost two- thirds of grants are not closed out within 180 days. A specific performance  measure and target will enable EPA to oversee and manage the timeliness  of grant closeout. We recognize that grantees’ late submission of required  reports is a common problem that contributes to the lack of timeliness in  closing out grants. Because fixing this problem on a case-by-case basis is  difficult, the agency needs an overarching strategy to address it.
Furthermore, in the three regions we visited, we found instances in which  appropriate documentation was missing from the closeout files. While we  do not know the extent of this problem agencywide, these problems could  indicate a major weakness that EPA may need to address. In responding to  a draft of this report, EPA acknowledged that it does have a problem in  closing out grants properly. Finally, we note that inconsistencies in EPA’s  monitoring and closeout policies may hinder EPA’s ability to close out  grants in a timely fashion.
While EPA has made strides in trying to identify and obtain results from its  grants by issuing an environmental results policy, it has not yet established  a performance measure and target that reflect the policy’s direction.
Finally, the lack of effective supervision may have contributed to the  problems we identified. EPA has issued a plan in January 2006 to ensure  that the agency’s new performance system addresses grants management  responsibilities. It is too early to tell whether this plan will effectively hold  managers and staff accountable for grants management.
section_title: Recommendations
paragraphs: As EPA revises its Grants Management Plan, the agency has an  opportunity to strengthen the management of its grants. We recommend  that the Administrator of EPA direct the Office of Grants and Debarment  to take action in the following three areas:  Ongoing monitoring.
Develop a performance measure and a performance target for ongoing    Consider requiring project officers and grant specialists to document  ongoing monitoring in the agency’s grants database so that the managers  can monitor compliance agencywide.
Grant closeout.
Establish a standard for the timely closeout of grants and ensure that  EPA’s monitoring and other policies are consistent with that standard.
Develop a performance measure and target for the grant closeout  standard.
Develop a strategy for addressing grantees’ late submission of required  final documentation.
Issue revised policies and procedures to ensure proper closeout of grants.
Environmental results.
Develop a performance measure and target that better reflects the new  environmental results policy.
section_title: Agency Comments and Our Response
paragraphs: We provided a draft of this report to EPA for review and comment. We  received oral comments from EPA officials, including the Director of the  Office of Grants and Debarment. Overall, EPA officials agreed with our  recommendations, and they stated that the agency has begun to take steps  to implement them and will incorporate them into the agency’s and Grants  Management Plan, policies, and procedures. In addition, EPA officials  provided some clarifying language for our recommendations, which we  incorporated as appropriate. Furthermore, EPA officials acknowledged  that proper closeout of grants is an agencywide problem that needs to be  addressed. Based on this acknowledgement, we strengthened our  recommendation to state that EPA needs to issue revised policies and  procedures to better ensure the proper closeout of grants, rather than  determine the extent of improper closeouts at the agency. EPA agreed.  Finally, EPA provided additional information about the agency’s efforts to  address the complexities of measuring environmental results and other  clarifying comments, which we incorporated into this report, as  appropriate.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 6 days from the  report date. At that time, we will send copies of this report to the  congressional committees with jurisdiction over EPA and its activities; the  Administrator, EPA; and the Director, Office of Management and Budget.   We will also make copies available to others upon request.  In addition, the  report will be available at no charge on the GAO Web site at  http://www.gao.gov.
If you have any questions about this report, please contact me at (202) 512- 3841.  Key contributors to this report are listed in appendix II. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report.
section_title: Appendix I: Scope and Methodology
paragraphs: This appendix details the methods we used to assess the progress the  Environmental Protection Agency (EPA) has made in implementing its  grant reforms.
To assess EPA’s progress, we reviewed information from both  headquarters and the regions. At headquarters, we reviewed EPA’s Grants  Management Plan, 2003-2008 and EPA policies that address awarding,  monitoring, and obtaining results from grants. We also reviewed reports  on EPA’s grants management, including prior GAO reports; EPA’s  Inspector General reports; the Office of Management and Budget’s  Program Assessment Rating Tool; EPA’s internal management reviews,  including comprehensive grants management reviews, post-award  monitoring plans, grants management self-assessments from 2003 to 2005;  an April 2005 workload analysis conducted by LMI, a government  consultant; and a closeout analysis prepared for GAO by the Office of  Grants and Debarment. We also interviewed officials in the Office of  Grants and Debarment and at the Office of Water.
In addition, we reviewed EPA performance metric information. These  metrics are based on data from the agency’s Integrated Grants  Management System, which is currently undergoing a data quality review,  and the Grant Information and Control System, which has not undergone a  data quality review. Given EPA’s ongoing data quality review of the  Integrated Grants Management System—and because we present EPA’s  performance metric data as documentary evidence and do not use it as the  sole support for findings, conclusions, or recommendations—we did a  limited reliability review of the two systems. Our assessment included (1)  information from GAO’s prior data reliability assessment work on the two  systems and (2) interviews with an Office of Grants and Debarment  official about the data systems and data elements. We determined that the  performance information we used is sufficiently reliable for our purposes.
To assess the progress and problems EPA has experienced from a regional  perspective, we selected EPA Office of Water programs under the Clean  Water Act, at the Subcommittee’s request. We selected Wetland Program  Development Grants (wetland grants) because it is a discretionary grant  program—that is, EPA decides who receives the award and its amount,  and the program is subject to competition. We selected Nonpoint Source  Management Program grants (nonpoint source grants) because it is type of  formula-based grant program—grants that are often awarded on the basis  of formulas prescribed by law or agency regulation. We reviewed EPA’s  progress at the regional level by selecting grants in 3 of EPA’s 10 regional  offices: Region 1 (Boston), Region 5 (Chicago), and Region 9 (San  Francisco). We selected these regions because collectively they represent  a significant share of grant funding for the two programs we reviewed,  geographic dispersion, and a significant share of Performance Partnership  Grants among the regional offices.
To ensure coverage of the grant life cycle—from awarding to closing out  grants, we conducted a case study review of a nonprobability sample of  these two programs. Specifically, we asked EPA Regions 1, 5, and 9 to  provide lists of wetlands and nonpoint source grants awarded between  January 1, 2004, and June 30, 2005, and those grants officially closed  during this period. We targeted recently awarded and recently closed  grants because the grant reforms began in 2002. To complete the case  study, we reviewed two files per grant—the project officer file and the  grant specialist file—using a detailed data collection instrument. All data  entered in the data collection instrument was verified by a second party to  ensure the accuracy and validity of each entry. Additionally, we conducted  semistructured interviews with project officers and the grant specialists in  order to understand the files. Overall, we reviewed the files for 32 active  grants and 34 closed grants; and we interviewed administrative project  officers, project officers, and technical contacts, and grant specialists  about those files. We also interviewed senior resource officials and grants  management office managers in the three regions we visited.
We were limited in the number of grants we could review because the case  study approach required multiple, detailed file reviews and interviews for  each grant. Consequently, we selected a nonprobability sample of active  and closed grants in the wetland and nonpoint source programs to review  in each of the three regions. For active grants in our nonprobability  sample, we sorted the grants by recipient type, project officer, and grant  specialist to provide a distribution, and then randomly selected grants for  review. For closed grants, we sorted the grants similarly, but we also  selected grants based on the length of time it took to close out the grant.  Because the case study design is nonprobabilistic, the findings are not  generalizable to all grants in all regions. However, the case study design  provides insights into regional grant activities for the two Clean Water Act  programs in three regions, and it offers an in-depth perspective on some of  the successes and continuing problems EPA faces in implementing its  grants management reforms. Table 12 shows the population of wetland  and nonpoint program grants in Regions 1, 5, and 9, and the number of  those grants we reviewed.
We experienced some limitations in conducting our review. For example,  we were not able to assess the implementation of some EPA policies at the  regional level because they had been issued too recently to assess during  our file review time frame. We also found evidence in two of the three  regions we visited that staff had added materials to their files after we had  requested the files and before our review, despite the fact that we had  taken precautions to avoid this situation. That is, we had asked the regions  to inform staff not to add documents to files once they were requested,  and we limited the time frame between our request for specific files and  our review of the files. When we determined that these additions had  occurred, we took mitigating steps to “restore” the grant files to their  original state. Specifically, we checked the dates of documents to detect  any widespread updating of files, asked all project officers and grant  specialists we interviewed who were assigned to the grants in our sample  whether they added anything to the file in preparation for the GAO visit,  asked managers to tell their staff to point out materials added to the file,  and, in one region, shortened the time between file request and our visit.  To adjust for the alterations, we used a special code in our data collection  instrument to denote “additions,” and later subtracted the information in  our analysis. The file alteration mitigation strategies and the analysis  adjustments afford us confidence in the accuracy and validity of our file  review results.
We conducted our work between February 2005 and April 2006, in  accordance with generally accepted government auditing standards.
section_title: Appendix II: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Andrea Wamstad Brown,  Assistant Director; Bruce Skud, Analyst-in-charge; Rebecca Shea; Lisa  Vojta; Carol Herrnstadt Shulman; Omari Norman; David Bobruff; Matthew  J. Saradjian; and Jessica Nierenberg made key contributions to this report.
section_title: Related GAO Products
paragraphs: Grants Management: EPA Needs to Strengthen Efforts to Provide the  Public with Complete and Accurate Information on Grant  Opportunities. GAO-05-149R. Washington, D.C.: February 3, 2005.
Grants Management: EPA Continues to Have Problems Linking Grants  to Environmental Results. GAO-04-983T. Washington, D.C.: July 20, 2004.
Grants Management: EPA Needs to Better Document Its Decisions for  Choosing between Grants and Contracts. GAO-04-459. Washington, D.C.:  March 31, 2004.
Grants Management: EPA Needs to Strengthen Efforts to Address  Management Challenges. GAO-04-510T. Washington, D.C.: March 3, 2004.
Grants Management: EPA Actions Taken against Nonprofit Grant  Recipients in 2002. GAO-04-383R. Washington, D.C.: January 30, 2004.
Grants Management: EPA Needs to Strengthen Oversight and Enhance  Accountability to Address Persistent Challenges. GAO-04-122T.  Washington, D.C.: October 1, 2003.
Grants Management: EPA Needs to Strengthen Efforts to Address  Persistent Challenges. GAO-03-846. Washington, D.C.: August, 29, 2003.
Environmental Protection Agency: Problems Persist in Effectively  Managing Grants. GAO-03-628T. Washington, D.C.: June 11, 2003.
Federal Assistance: Grant System Continues to Be Highly Fragmented.  GAO-03-718T. Washington, D.C.: April 29, 2003.
Results-Oriented Cultures: Creating a Clear Linkage between Individual  Performance and Organizational Success. GAO-03-488. Washington, D.C.:  March 14, 2003.
Major Management Challenges and Risks: Environmental Protection  Agency. GAO-03-112. Washington, D.C.: January 2003.
Results-Oriented Cultures: Using Balanced Expectations to Manage  Senior Executive Performance. GAO-02-966. Washington, D.C.: September  27, 2002.
Environmental Protection: Grants Awarded for Continuing  Environmental Programs and Projects. GAO-01-860R. Washington, D.C.:  June 29, 2001.
Environmental Protection: EPA’s Oversight of Nonprofit Grantees’ Costs  Is Limited. GAO-01-366. Washington, D.C.: April 6, 2001.
Environmental Protection: Information on EPA Project Grants and Use  of Waiver Authority. GAO-01-359. Washington, D.C.: March 9, 2001.
Environmental Research: STAR Grants Focus on Agency Priorities, but  Management Enhancements Are Possible. GAO/RCED-00-170.  Washington, D.C.: September 11, 2000.
Environmental Protection: Grants for International Activities and  Smart Growth. GAO/RCED-00-145R. Washington, D.C.: May 31, 2000.
Environmental Protection: Collaborative EPA-State Effort Needed to  Improve Performance Partnership System. GAO/T-RCED-00-163.  Washington, D.C.: May 2, 2000.
Managing for Results: EPA Faces Challenges in Developing Results- Oriented Performance Goals and Measures. GAO/RCED-00-77.  Washington, D.C.: April 28, 2000.
Environmental Protection: Factors Contributing to Lengthy Award  Times for EPA Grants. GAO/RCED-99-204. Washington, D.C.: July 14,  1999.
Environmental Protection: Collaborative EPA-State Effort Needed to  Improve New Performance Partnership System. GAO/RCED-99-171.  Washington, D.C.: June 21, 1999.
Environmental Protection: EPA’s Progress in Closing Completed Grants  and Contracts. GAO/RCED-99-27. Washington, D.C.: November 20, 1998.
Dollar Amounts of EPA’s Grants and Agreements. GAO/RCED-96-178R.  Washington, D.C.: May 29, 1996.