Grants Management: EPA Needs to Strengthen Efforts to Address Persistent Challenges
Published: 2003-08-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Environmental Protection Agency (EPA) has long faced problems managing its grants, which constitute over one-half of the agency's annual budget, or about $4 billion. EPA uses grants to implement its programs to protect human health and the environment and awards grants to thousands of recipients, including state and local governments, tribes, universities, and nonprofit organizations. EPA's ability to efficiently and effectively accomplish its mission largely depends on how well it manages its grant resources. As requested, GAO determined (1) major challenges EPA faces in managing its grants and how it has addressed these challenges in the past, (2) extent to which EPA's recently issued policies and grants management plan address these challenges, and (3) promising practices, if any, that could assist EPA in addressing these challenges.
section_title: What GAO Found
paragraphs: EPA continues to face four key management challenges, despite past efforts to address them. These challenges are (1) selecting the most qualified applicants, (2) effectively overseeing grantees, (3) measuring the results of grants, and (4) effectively managing grants staff and resources. In recent years, EPA has taken a series of actions to address these challenges by, among other things, issuing policies, conducting training, and developing a new data system for grants management. However, these past actions had mixed results because of the complexity of the problems, weaknesses in design and implementation, and insufficient management emphasis. EPA's 2002 competition and oversight policies and 2003 grants management plan focus on the major challenges GAO identified, but will require strengthening, enhanced accountability, and a sustained commitment to succeed. For example, EPA's new oversight policy mandates more in-depth monitoring of grantees but it does not build in a process for analyzing the results of the in-depth monitoring to address systemic grantee problems. Such analysis could better target EPA's oversight efforts. In addition, its 5-year grants management plan does offer, for the first time, a comprehensive road map with objectives, goals, and milestones for addressing grants management challenges. However, the plan does not completely address how EPA will hold all managers and staff accountable for successfully fulfilling their management responsibilities. Therefore, EPA cannot ensure the sustained commitment required for the plan's success. Although information on promising grants management practices is limited, the federal agencies and other organizations GAO contacted identified some practices for each of EPA's four major challenges that may further assist EPA in improving its grants management. For example, one federal agency takes into account applicants' potential to achieve results when selecting grantees. A private foundation conducts preaward reviews of some applicants to learn about the organization and assess its fiscal health. In addition, GAO has developed a guide for federal agencies to use to hold managers and staff accountable for achieving desired agency results. This guide could be useful in helping EPA ensure accountability for grants management performance.

section_title: Letter
paragraphs: The Environmental Protection Agency (EPA) has faced persistent  challenges in managing its grants, which constitute over one-half of the  agency’s budget, or about $4 billion annually. To support its mission of  protecting human health and the environment, in fiscal year 2002, EPA  awarded grants to a variety of recipients, including state and local  governments, tribes, universities, and nonprofit organizations. As of  September 30, 2002, there were 4,100 grant recipients. Given the size and  diversity of EPA’s programs, its ability to efficiently and effectively  accomplish its mission largely depends on how well it manages its grant  resources and builds accountability into its efforts.
Congressional hearings in 1996, 1999, and 2003 have focused on EPA’s  problems in effectively managing its grants. We and EPA’s Inspector  General have reported on a number of weaknesses throughout the grants  management process—from awarding grants to measuring grant results.  EPA’s efforts to address its grants management problems have not fully  resolved them. Because EPA has faced persistent problems in managing  grants, we designated its grants management as a major management  challenge in January 2003.
Late in 2002, EPA launched new efforts to address some of its grants  management problems. It issued two new policies—one to promote  competition in awarding grants and one to improve its oversight of grants.  Furthermore, in April 2003, EPA issued a 5-year grants management plan to  address its long-standing grants management problems.
You asked us to determine the (1) major challenges EPA faces in managing  its grants and how it has addressed these challenges in the past, (2) extent  to which EPA’s recently issued policies and grants management plan  address these challenges, and (3) promising practices, if any, that could  assist EPA in addressing these challenges.
To identify the challenges EPA faces in managing its grants and to examine  how it has addressed these challenges in the past, we (1) analyzed 93  reports on EPA’s grants management, including our reports, EPA’s  Inspector General reports, and EPA’s internal management reviews  conducted from 1996 through 2003, (2) systematically reviewed and  recorded information from the 1,232 records of calendar year 2002 in-depth  reviews of grantee performance—from financial management to progress  in achieving grant objectives, and (3) interviewed EPA officials and  reviewed documents obtained from them. To determine the extent to  which EPA’s recently issued policies and grants management plan address  these challenges, we (1) reviewed the new policies and plan and  interviewed EPA officials responsible for key aspects of the plan, (2)  attended EPA’s grants management training courses, and (3) observed five  EPA in-depth reviews of grantees. To identify what promising practices, if  any, could assist EPA in addressing these challenges, we interviewed  officials in six federal agencies, including the five largest grants-awarding  agencies, and in private sector organizations and reviewed documentation  from them. We also reviewed two GAO reports that describe effective  training and performance management systems for federal agencies. Our  detailed scope and methodology are discussed in appendix I.
subsections: section_title: Background
paragraphs: EPA administers and oversees grants primarily through the Office of Grants  and Debarment, 10 program offices in headquarters, and program offices  and grants management offices in EPA’s 10 regional offices. Figure 1 shows  EPA’s key offices involved in grants activities for headquarters and the  regions. (See app. II for a map of EPA’s regional office locations.)
The management of EPA’s grants program is a cooperative effort involving  the Office of Administration and Resources Management’s Office of Grants  and Debarment, program offices in headquarters, and grants management  and program offices in the regions. The Office of Grants and Debarment  develops grant policy and guidance. It also carries out certain types of  administrative and financial functions for the grants approved by the  headquarters program offices, such as awarding grants and overseeing the  financial management of these grants. On the programmatic side,  headquarters program offices establish and implement national policies for  their grant programs, and set funding priorities. They are also responsible  for the technical and programmatic oversight of their grants. In the regions,  grants management offices carry out certain administrative and financial  functions for the grants, such as awarding grants approved by the regional  program offices, while the regional program staff provide technical and  programmatic oversight of their grantees.
As of June 2003, 109 grants specialists in the Office of Grants and  Debarment and the regional grants management offices were largely  responsible for administrative and financial grant functions. Furthermore,  1,835 project officers were actively managing grants in headquarters and  regional program offices. These project officers are responsible for the  technical and programmatic management of grants. Unlike grant  specialists, however, project officers generally have other primary  responsibilities, such as using the scientific and technical expertise for  which they were hired.
In fiscal year 2002, EPA took 8,070 grant actions totaling about $4.2  billion. These awards were made to six main categories of recipients as  shown in figure 2.
EPA offers two types of grants—nondiscretionary and discretionary:   Nondiscretionary grants support water infrastructure projects, such as  the drinking water and clean water state revolving fund programs, and  continuing environmental programs, such as the Clean Air Program for  monitoring and enforcing Clean Air Act regulations.   For these grants,  Congress directs awards to one or more classes of prospective  recipients who meet specific eligibility criteria; the grants are often  awarded on the basis of formulas prescribed by law or agency  regulation. In fiscal year 2002, EPA awarded about $3.5 billion in  nondiscretionary grants. EPA awarded these grants primarily to states  or other governmental entities.
Discretionary grants fund a variety of activities, such as environmental  research and training. EPA has the discretion to independently  determine the recipients and funding levels for these grants. In fiscal  year 2002, EPA awarded about $719 million in discretionary grants. EPA  awarded discretionary grants primarily to nonprofit organizations,  universities, and government entities.
The grant process has the following four phases:   Preaward. EPA reviews the application paperwork and makes an award  decision.
Award. EPA prepares the grant documents and instructs the grantee on  technical requirements, and the grantee signs an agreement to comply  with all requirements.
Postaward. After awarding the grant, EPA provides technical assistance,  oversees the work, and provides payments to the grantee; the grantee  completes the work, and the project ends.
Closeout of the award. EPA ensures that all technical work and  administrative requirements have been completed; EPA prepares  closeout documents and notifies the grantee that the grant is completed.
EPA’s grantees are subject to the same type of financial management  oversight as the recipients of other federal assistance. Specifically, the  Single Audit Act requires grantees to have an audit of their financial  statements and federal awards or a program specific audit if they spend  $300,000 or more in federal awards in a fiscal year. Grantees submit these  audits to a central clearinghouse operated by the Bureau of the Census,  which then forwards the audit findings to the appropriate agency for any  necessary action. However, the act does not cover all grants and all aspects  of grants management and, therefore, agencies must take additional steps  to ensure that federal funds are spent appropriately.
In addition, EPA conducts in-depth reviews to analyze grantees’  compliance with grant regulations and specific grant requirements.  Furthermore, to determine how well offices and regions oversee grantees,  EPA conducts three types of internal management reviews that address  grants management: (1) Management Oversight Reviews, which assess  grants management activities in regional offices; (2) Management  Effectiveness Reviews, which are self-assessments of grants management  activities performed by EPA’s headquarters and regional offices; and (3)
Post-Award Validation Reviews, which assess the effectiveness of  initiatives EPA has taken to improve oversight efforts.
EPA has faced persistent problems in managing its grants. EPA’s Inspector  General testified before Congress in 1996 and again in 1999 that EPA did  not fulfill its obligation to properly monitor grants. Acknowledging these  problems, EPA identified oversight, including grant closeouts, as a material  weakness—an accounting and internal control system weakness that the  EPA Administrator must report to the President and Congress. EPA’s  fiscal year 1999 Federal Managers’ Financial Integrity Act report indicated  that this oversight material weakness had been corrected, but the Inspector  General testified that the weakness continued. In 2002, the Inspector  General again recommended that EPA designate grants management as a  material weakness. The Office of Management and Budget (OMB) also  recommended in 2002 that EPA designate grants management as a material  weakness. In its fiscal year 2002 Annual Report, EPA ultimately decided to  maintain this issue as an agency-level weakness, which is a lower level of  risk than a material weakness. EPA reached this decision because it  believes its ongoing corrective action efforts will help to resolve  outstanding grants management challenges. However, in adding EPA’s  grants management to our list of EPA’s major management challenges in  January 2003, we signaled our concern that EPA has not yet taken  sufficient action to ensure that it can manage its grants effectively.
section_title: Four Key Grants Management Challenges Persist despite EPA’s Past Efforts to Address Them
paragraphs: We identified four key challenges that EPA continues to face in managing  its grants despite efforts to address them. These challenges are (1)  selecting the most qualified grant applicants, (2) effectively overseeing  grantees, (3) measuring the results of grants, and (4) effectively managing  grant staff and resources. In recent years, EPA took a series of actions to  address these challenges by, among other things, issuing policies on  competition and oversight, conducting training for project officers and  nonprofit organizations, and developing a new data system for grants  management. However, these actions had mixed results because of the  complexity of the problems, weaknesses in design and implementation,  and insufficient management attention.
subsections: section_title: EPA Has Not Always Selected the Most Qualified Applicants despite Issuing a Competition Policy
paragraphs: EPA has not always ensured that the most qualified applicants receive  grant awards. For discretionary grants, the agency requires project officers  to evaluate grant applications and recommend funding those that will most  effectively contribute to EPA’s program objectives and priorities. But  project officers have not always performed all of the steps required to  review grant applications. According to a 2003 Inspector General report,  project officers did not always, among other things, (1) ensure a link  between the project funded and EPA’s mission, (2) assess the probability of  success prior to the award, and (3) determine the reasonableness of  proposed project costs. As a result, EPA may have lost the opportunity to  fund other projects that could have better achieved its mission.
The Federal Grant and Cooperative Agreement Act of 1977 encourages  agencies to use competition in awarding grants. To encourage competition,  EPA issued a grants competition policy in 1995. However, EPA’s policy did  not result in meaningful competition throughout the agency, according to  EPA officials. Furthermore, EPA’s own internal management reviews and  a 2001 Inspector General report found that EPA has not always encouraged  competition.
Finally, EPA has not always engaged in widespread solicitation when it  could be beneficial to do so. Widespread solicitation would provide greater  assurance that EPA receives proposals from a variety of eligible and highly  qualified applicants who otherwise may not have known about grant  opportunities. According to a 2001 EPA Inspector General report,  program officials indicated that widespread solicitation was not necessary  because “word gets out” to eligible applicants. Applicants often sent their  proposals directly to these program officials, who funded them using  “uniquely qualified” as the justification for a noncompetitive award. This  procedure created the appearance of preferential treatment by not offering  the same opportunities to all potential applicants. In addition, the agency  provided incomplete or inconsistent public information on its grant  programs in the Catalog of Federal Domestic Assistance. Therefore,  potential applicants may not have been adequately informed of funding  opportunities.
section_title: EPA Has Not Always Effectively Overseen Grant Recipients despite Past Actions to Improve Oversight
paragraphs: EPA has not always effectively overseen grant recipients despite efforts to  improve monitoring. To address oversight problems, EPA issued a series of  policies starting in 1998. However, these oversight policies have had mixed  results in addressing this challenge.
subsections: section_title: EPA Has Not Always Effectively Overseen Grant Recipients
paragraphs: EPA’s oversight efforts have not always been proactive or well  documented, or ensured compliance with grant requirements and  objectives. According to EPA’s 2001 internal management reviews, project  officers had minimal contact with grantees, limiting their oversight to  reviewing grantees’ progress reports. However, project officers did not  proactively ensure the recipient submitted the reports on time or that the  reports included the required financial information. According to our  analysis of EPA’s 2002 in-depth reviews, 15 percent of the reviews identified  a problem with grantees’ submitting complete progress reports on time.
Furthermore, EPA’s guidance emphasizes the importance of documenting  oversight efforts, in part, because a lack of documentation raises questions  about the extent of the monitoring conducted. However, project officers  and grants specialists have not always documented their monitoring  activities. According to a 2002 EPA internal management review, for  example, one grants management office developed a form to make it easier  to ensure monitoring activities were completed, but the form was missing  from 50 percent of the grant files reviewed. Even when staff used the  monitoring form, they did not always complete it. Furthermore, project  officers did not always document that they had monitored the required five  key areas under the policy, such as ensuring compliance with the terms and  conditions of the grant award.
Finally, EPA has not effectively ensured that grantees comply with the  following grant requirements and objectives:  Adequate financial and internal controls. In 2001, we reported that  EPA’s oversight of nonprofit grantees’ costs did not ensure that grantees  used funds for costs allowed under OMB’s guidance. For example, EPA  did not include transaction testing, which can identify unallowable  expenditures, such as lobbying.
Compliance with grant regulations. In 2002, according to the Inspector  General, EPA did not monitor grantees’ procurements to determine if  the grantees were using a competitive process to obtain the best  products, at the best price, from the most qualified firms. In addition,  in 1999 and again in 2002, the Inspector General reported conflict-of- interest problems because grant recipients had awarded contracts to  parties who had helped them prepare their grants and, therefore, had  advance knowledge about grantees’ plans to award contracts. Our  review of EPA’s 2002 in-depth reviews also found that a number of  compliance problems persist. (See table 7 in app. III.) For example, EPA  found accounting problems in 37 of 189 administrative reviews and  procurement problems in 70 of 189 of these reviews.
Meeting grant objectives. EPA has not fully ensured that recipients are  meeting grant objectives. For example, in 2000, we reported that EPA  had not adequately tracked its Science To Achieve Results (STAR)  research grants to ensure that the grantees had submitted the results of  their scientific investigations—the objective of the grant—in a timely  manner. We found that 144 of the nearly 200 grants we reviewed had  missed their deadline for submitting final reports, even after some  extensions had been given.
section_title: EPA’s Past Actions to Improve Oversight Have Had Mixed Results
paragraphs: To address oversight problems, EPA issued three oversight policies—in  1998, 1999, and February 2002—designed to promote proactive and well- documented oversight of grantees to ensure compliance with grant  regulations and objectives. For the first time, these policies required the  following:     Baseline monitoring of all grantees at least once during the life of the  grant. EPA staff are responsible for contacting the grantee, reviewing  the grantee’s reports, assessing compliance with terms and conditions,  and monitoring payments to the grantee.
In-depth reviews for at least 5 to 10 percent of grant recipients. Both  grants management offices and program offices conduct these reviews  either at the grantee’s location (on-site) or at EPA’s office or another  location (off-site). The grant management offices conduct  administrative reviews, which are designed to evaluate grantees’  financial and administrative capacity. The program offices conduct  programmatic reviews, which are designed to assess the grantees’  activities in five key areas: assessing progress of work, reviewing  financial expenditures, meeting the grant’s terms and conditions,  meeting all programmatic and statutory and regulatory requirements,  and verifying that equipment purchased under the award is managed  and accounted for. The policy required that programmatic staff use a  suggested protocol in conducting their in-depth reviews. Furthermore,  the policy included suggested criteria for selecting grantees to be  reviewed.
Annual monitoring plans submitted by headquarters and regional offices  that outline the offices’ proposed oversight activities. The policy also  suggested activities to be included in these plans, such as monitoring  grantees’ progress of work, documenting their efforts, and closing out  grants in a timely manner.
Grantee compliance database—developed by the Office of Grants and  Debarment—that the grants management offices would use to store the  in-depth reviews.
We found that EPA’s design and implementation of its actions to improve  grantee oversight have had mixed results. In particular, we found both  improvements and weaknesses in EPA’s in-depth reviews and its ability to  track grantee compliance.
On the positive side, EPA’s policies required headquarters and regional  offices to conduct a minimum number of in-depth reviews. According to  our analysis of EPA’s in-depth reviews, the agency conducted 1,232 reviews  in 2002. Table 1 shows that these reviews were conducted almost equally  on-site and off-site, and for a variety of different types of grantees.
EPA also more evenly distributed, and increased the number of, on-site  reviews among its program and regional offices between 2001 and 2002.  According to the Inspector General, EPA conducted 466 on-site, in-depth  reviews in 2001, but 265 of these, or 57 percent, were conducted by only  two of EPA’s offices. However, according to our analysis of the 2002 in- depth reviews, on-site reviews were more evenly distributed: 588 on-site  reviews, with the two EPA offices that conducted the most reviews  representing approximately 28 percent of the total. (See table 6 in app. III.)
Furthermore, about half of EPA’s in-depth reviews found problems with  grantees. (See table 11 in app. III.) According to our analysis, in 2002, EPA  grants specialists and project officers identified 1,250 problems in 21 areas.  (See table 7 in app. III.)   Tables 2 and 3 show the most frequently identified  problems for the 189 administrative and 1,017 programmatic reviews.   For  example, 73 of 189 administrative reviews found problems with grantees’  written procedures, while 308 of the 1,017 programmatic reviews identified  technical issues.
The differences in types of problems frequently identified, as shown in  tables 2 and 3, reflect differences in the focus of administrative and  programmatic reviews. Table 4 describes the nature of these problems.  (See table 12 in app. III for a description of all the problems.)
In response to these problems, EPA required or recommended that  grantees take corrective actions for about half, or 640, of the 1,250  problems identified. (See table 13 in app. III.) For example, EPA found that  one grantee had not adequately separated duties to provide checks and  balances with the grantee’s organization. That is, the executive director  was also the project manager, payment authorizer, contract executor, and  payroll officer. EPA recommended that the organization separate these  responsibilities to ensure proper handling of federal funds.
EPA also took more significant actions against some grantees. These  significant actions include the following:  Suspension order—EPA requires a grantee to stop work and take action  to minimize its grant-related expenditures until it resolves EPA’s areas of  concern.
Payment holds—EPA decides it will not make any more payments to a  grantee until the grantee has resolved an area of concern.
Disallowance of costs—EPA disallows a grantee’s expenditures when it  determines that the grantee did not spend its grant funds in accordance  with federal cost principles or its particular grant agreement.
Debarment—EPA declares that a particular organization or individual is  not eligible to receive grants for a specific period of time.
Termination—EPA cancels the grant.
In our analysis of EPA’s 2002 in-depth reviews, we identified 47 significant  actions against its grantees. The most frequent action taken was  withholding payment: 15 of the 47 significant actions. In no cases did EPA  debar a grantee. Nonprofit organizations and tribes each received 21 of  these actions, states received 5, and local governments and universities  received none. (See table 16 in app. III.) EPA’s significant actions included  the following, for example:   Region 8 issued a payment hold against one of its tribal grantees after  questioning costs because the recipient appeared to be performing work  outside the scope of the grant agreement. The region determined that  the remaining $27,459 in the grantee’s account should be withheld until  the tribe submitted documentation to demonstrate that the questioned  costs were allowable.
Region 3 disallowed costs for one of its nonprofit grantees. The region  disallowed $51,085 based on a conflict-of-interest, lack of competition in  the subagreement, and lack of supporting documentation in accounting  records.
The Grants Administration Division terminated a grant for a nonprofit  grantee. EPA took this action because the grantee actually had no paid  employees of its own but instead used the employees of a for-profit  company to conduct the work. This arrangement created an apparent  conflict of interest.
Region 4 withheld $40,000 of funding from a state because it found that  the state had not agreed to fully implement its responsibilities as defined  by the grant.
EPA data also showed that the agency took other significant actions in 2002  that did not result from in-depth reviews. According to our analysis, EPA  took 24 other significant actions because of problems identified during an  investigation by EPA’s Inspector General or during oversight of grantees in  actions other than in-depth reviews.
On another positive note, EPA improved its closeout of grants after it  developed specific procedures for closing out nonconstruction grants and  identified a strategy for closing out construction grants. This strategy  assessed impediments to closing out grants and actions to address these  impediments. EPA had successfully resolved its backlog by 2002. As a  result, EPA has eliminated this backlog as a material weakness, and thus it  better ensures that grant commitments have been met.
Despite these positive steps, EPA’s actions did not address the need to  develop information on systemic problems with grantees to better target its  oversight efforts—a weakness in policy design. EPA could not develop  systemic information for three reasons. First, EPA did not use a statistical  approach to selecting grantees for in-depth reviews. In particular, EPA  allowed each of its offices to determine how many in-depth reviews the  office would conduct—from at least 5 to 10 percent of grantees—and what  criteria the office would use to select grantees for in-depth reviews. The  policy’s flexibility in selecting grantees was premised on the belief that  those closest to grantees, such as a program or regional office, could best  decide on which grantees to select for oversight. However, if EPA used a  statistical approach to grantee selection, it could then project the results of  the reviews to all EPA grantees. Since EPA did not use such a sampling  plan in conducting its 2002 reviews, neither EPA nor we could use data  from these reviews to determine the overall compliance of grantees.  Furthermore, without a statistical approach, EPA cannot determine  whether 5, 10, or any other percentage, adequately provides it with a  comprehensive assessment of its grantees.
Second, EPA does not have a standard reporting format to ensure  consistency and clarity in reporting review results. Review results include  such items as problems identified and corrective actions required to  address these problems. As we found in our analysis of the 2002 in-depth  reviews, EPA officials across the agency report in various formats that do  not always clearly present the results of the review.   For example, some  EPA officials provided a narrative report on the results of their reviews,  while others completed a protocol that they used in conducting their  review. We found 349 instances where the project officer or grants  management specialist did not clearly explain whether he or she had  discovered a problem or not.
Finally, EPA did not develop data elements for its grantee compliance  database or a plan for using that data to identify and act on systemic  problems. We found that valuable information could be collected from the  2002 in-depth reviews for assessing such issues as the (1) types of grantees  having problems, (2) types of problem areas needing further attention, (3)  types of reviews—on-site or off-site—that provide the best insights into  certain problems areas, and (4) corrective actions required or  recommended to resolve problems.
With such information, EPA could, over time, identify problem areas and  develop trends to assess the effectiveness of corrective actions in order to  better target its oversight efforts. In particular, according to our analysis of  EPA’s 2002 in-depth reviews, administrative reviews identify more  problems when conducted on site, while the number of problems identified  by programmatic reviews does not differ by on-site or off-site reviews. (See  table 10 in app. III.) However, nearly half of the programmatic reviews,  which constituted more than 80 percent of the 2002 reviews, were  conducted on site. (See table 5 in app. III.) Since on-site reviews are  resource intensive because of travel costs and staff used, a systematic  analysis could enable EPA to better target its resources. Similarly, EPA  could incorporate other information into its grantee compliance database,  such as Inspector General reports, to identify problem areas, and target  oversight resources. In addition, EPA could use the database to track the  resolution of problems.
The weaknesses in the policies are compounded by the fact that EPA did  not always effectively implement its oversight policies. Specifically, EPA  used its database to collect and store in-depth reviews, but not all EPA  offices were required under the policies to enter the results of their reviews  into the database, and not all of the offices required to enter information  into the database did so. Moreover, EPA officials acknowledged the agency  did not have an effective method to verify the number of in-depth reviews it  reported to Congress. We found that our total number of in-depth  reviews—1,232—was about the same as EPA’s total—1,208—but we found  discrepancies, on an office-by-office basis, in the number of reviews  conducted. (See table 6 in app. III.) For example, in one office we identified  176 more reviews than EPA did, and in another office we identified 76  fewer reviews than EPA did. Finally, we found that EPA’s in-depth reviews  did not always indicate whether these reviews had used a protocol. For  example, in 294 of the 893 nonmandated reviews, it was unclear whether  the grants specialist or project officer used a protocol. (See tables 14 and  15 in app. III.)
The Inspector General has also reported on problems with EPA’s  implementation of its actions to address oversight problems. In 2002, the  Inspector General found, among other things, inconsistent performance of  monitoring responsibilities, inadequate preparation of monitoring plans,  and inadequate internal management reviews. For example, the Inspector  General found that EPA did not use the internal management reviews to  draw overall conclusions on the effectiveness of oversight and did not  quantify its review results. Furthermore, some senior resource officials did  not emphasize the importance of grants oversight. According to these  officials, the minimal control they have over the regional program offices’  priorities contributes to problems in effective grants management  oversight. The Inspector General found that EPA lacked an official who is  clearly responsible for monitoring the regional program offices’ priorities  for overseeing grants.
section_title: EPA’s Efforts to Address the Complex Problem of Measuring Environmental Results Have Not Consistently Ensured That Grantees Achieve Them
paragraphs: Planning for grants to achieve environmental results—and measuring  results—is a difficult, complex challenge. As pointed out in an earlier  report, while it is important to measure outcomes of environmental  activities rather than just the activities (outputs) themselves, agencies face  challenges in doing this. Output measures are inherently easier to develop  and report on than environmental results measures. In contrast, defensible  outcome measures are substantially more difficult to develop because of  the (1) absence of baseline data against which environmental  improvements can be measured, (2) inherent problems in quantifying  certain results, (3) difficulty of linking program activities with  environmental results because multiple conditions influence  environmental results, and (4) resources needed for measurement.
Nonetheless, EPA has focused on grants’ environmental results in its  partnership with the states under the National Environmental Performance  Partnership System. Under this system, states may enter into “Performance  Partnership Agreements” with their EPA regional offices. The partners use  a common set of national environmental indicators (“core performance  measures”). In developing these performance measures, EPA and state  officials have sought to move beyond counting the number of actions  conducted and toward environmental outcomes.
However, EPA has awarded some discretionary grants before considering  how the results of the grantees’ work would contribute to achieving  environmental results. In 2001, we reported that EPA program officials  treated EPA’s strategic goals and objectives as a clerical tool for  categorizing grants after the funds were already awarded rather than as a  tool to guide grant selection, although the grants generally were aligned  with EPA’s strategic goals and objectives. By assessing the relevance of  these grants to EPA’s strategic plan after awarding the grant, EPA cannot  ensure that it is awarding grants that will best help it accomplish its  mission.
EPA has also not developed environmental measures and outcomes for all  of its grant programs. In 2000, we reported that EPA did not have program  criteria to measure the effectiveness of its STAR program. Instead, EPA  focused on the procedures and processes for awarding grants. As a result,  EPA was uncertain about what the program was achieving. Similarly, OMB  recently evaluated four EPA grant programs to assess their effectiveness in  achieving and measuring results. According to this evaluation, these four  programs lacked outcome-based measures—measures that demonstrated  the impact of the programs on improving human health and the  environment. It concluded that one of EPA’s major challenges was  demonstrating program effectiveness in achieving public health and  environmental results. Furthermore, as we reported in 1999, the National  Environmental Performance Partnership System was hampered by the  technical problems we noted earlier, such as the absence of baseline data  against which environmental improvements could be measured.
Finally, EPA often does not require grantees to submit work plans that  explain how a project will achieve measurable environmental results. The  grantee work plan should describe the project, its objectives, the method  the grantee will use to accomplish the objectives, and expected outcomes.  The project officer uses the work plan to evaluate performance under the  grant agreement. In 2002, EPA’s Inspector General reported that EPA  approved some grantees’ work plans without determining the projects’  long-term human health and environmental outcomes. In fact, for almost  half of the 42 discretionary grants the Inspector General reviewed, EPA did  not even attempt to measure the projects’ outcomes. Instead, EPA funded  grants on the basis of work plans that focused on short-term procedural  results, such as meetings or conferences. In some cases, it was unclear  what the grant had accomplished. In 2003, the Inspector General again  found the project officers had not negotiated environmental outcomes in  work plans. The Inspector General found that 42 percent of the grant work  plans reviewed—both discretionary and nondiscretionary grants—lacked  negotiated environmental outcomes.
section_title: EPA Has Not Always Effectively Managed Its Grants Staff and Resources despite Some Past Efforts
paragraphs: EPA has not always appropriately allocated the workload for staff  managing grants, provided them with adequate training, or held them  accountable. Additionally, EPA has not always provided staff with the  resources, support, and information necessary to manage the agency’s  grants. To address these problems, EPA has taken a number of actions,  such as conducting additional training and developing a new electronic  grants management system. However, implementation weaknesses have  meant that EPA has not completely resolved its resource management  problems.
subsections: section_title: EPA Has Not Always Effectively Managed Its Grant Resources
paragraphs: In terms of staff, EPA faces workload problems, according to its internal  management reviews. EPA has not had enough staff in some locations to  adequately manage the number of grants it awards. Furthermore, workload  was sometimes unevenly distributed. For example, in one program office,  29 project officers had more than 4 grants to manage, while 11 had more  than 20. Additionally, in one regional office, project officers and grant  specialists did not promptly provide required approvals for certain grants  management tasks, in part because their competing workloads prevented  them from processing approvals on time. Finally, in 2002, the Inspector  General reported that there was considerable uncertainty over the size of  the workload that grants specialists and project officers could manage  effectively. These workload problems could, in some instances, result in  poor grants management.
EPA requires all project officers to certify their grants management skills  by attending a 3-day training course; this course is based on a training  manual that EPA periodically updates. Even though EPA has this  requirement, it has not always provided grants staff with the training  necessary to properly manage all aspects of grants. According to EPA’s  internal management reviews from 1997 through 2002, some staff managing  grants had not completed the basic training required to manage grants.  Other staff may have completed the basic training but need additional  training to refresh their skills or to become familiar with all of their grants  management responsibilities and requirements. For example, in some  instances, project officers were not familiar with the five key areas they  were to review when monitoring grantees, such as the financial aspects of a  grantee’s performance. According to a senior EPA official, with such a  small percentage of some project officers spending time on grants  management duties, it is difficult for EPA to ensure that its 1,835 project  officers who are actively managing grants have the level of training they  need.
Furthermore, the agency has not always held its staff—such as project  officers—accountable for fulfilling grants management responsibilities.  According to the Inspector General and internal management reviews, EPA  has not clearly defined project officers’ grants management responsibilities  in their position descriptions and performance agreements. Without  specific standards for grants management in performance agreements, it is  difficult for EPA to hold staff accountable. It is, therefore, not surprising  that, according to the Inspector General, project officers faced no  consequences for failing to effectively perform grants management duties.  Compounding the problem, agency leadership has not always emphasized  the importance of project officers’ grants management duties.
It may be difficult to hold project officers accountable because many  project officers spend only a small proportion of their time managing  grants. For example, a recent EPA workload assessment noted that  approximately 35 percent of the agency’s project officers manage only one  grant, instead devoting the bulk of their time to their scientific and  technical responsibilities.
EPA has also not adequately managed its resources to support grant staff in  performing their duties. Some EPA internal management reviews noted a  lack of resource commitment—time and money—to conduct grant  management activities and develop staff. For example, in some cases,  grantee oversight, particularly the on-site reviews, was limited by the  scarcity of such resources as travel funds.
Finally, staff did not always have the information they needed to effectively  manage grants. According to several EPA internal management reviews,  staff lacked accessible or useable reference material—such as policy and  guidance documents, and other information resources, such as reports of  grantee expenditures. Additionally, we and others have reported that EPA  did not use information from performance evaluations or information  systems to better manage its grants. For example, one EPA region did not  analyze the results of its own internal surveys, which were designed to  assess the effectiveness of its grants management operations.
section_title: Efforts to Improve the Management of Grant Resources Have Had Mixed Results
paragraphs: EPA has taken actions to address concerns that all of its grants  management staff receive training. EPA developed a database to document  which project officers had received the required 3-day basic training and  began to require that all project officers take an on-line refresher course  every 3 years. While these steps were primarily designed to improve the  training of project officers, grants specialists also have received some  training, and EPA has recently pilot-tested a standard training course  specifically designed for them. EPA managers and staff have also received  other types of training to enhance grants management skills. For example,  EPA has piloted a 1-day course on grants management for managers and  supervisors. The Office of Grants and Debarment also provides training on  specific aspects of grants management—such as postaward monitoring— as requested by program or regional offices. Finally, in response to  concerns that some grantees did not have the skills required to manage  grants, EPA developed a 1-day grants management training course for  nonprofit grantees.
However, these actions have not been fully successful. In 2002, EPA’s  Inspector General reported that the agency did not have adequate internal  controls in place to ensure that project officers complied with the training  requirements. Specifically, one region did not track the names and dates  of project officers who received training, the agencywide database on  training for project officers was inaccurate and had limited functionality,  and the on-line refresher course did not have the controls necessary to  prevent staff from obtaining false certifications.
EPA’s training efforts have also needed strengthening to better ensure that  staff receive and retain the skills necessary to manage grants. In particular,  while offices can request specific training from the Office of Grants and  Debarment, EPA does not have a process to ensure that all staff receive the  training needed to implement new policies and regulatory requirements in  a timely manner. For instance, when EPA issued new post-award  monitoring policies to improve its grants oversight, project officers did not  receive training on these new requirements unless their program or  regional office specifically requested that training or it was time for the  officers to take the refresher course. EPA also does not measure the impact  of its training programs by determining the baseline level of skills that  employees possess before taking training and determining what, if any,  skills have improved as a result of the training. Nor does EPA link its  training activities with improved performance on the job, especially in  targeted areas such as monitoring contacts with grantees and  documentation of files. As a result, EPA cannot hold either the trainers or  the trainees accountable for improving grants management skills and  performance.
EPA has also taken steps to improve another critical resource—its primary  data system for managing grants. In 1997, it began developing the  Integrated Grants Management System (IGMS), which, according to an  EPA official, will facilitate oversight by allowing electronic management  throughout the life of a grant. EPA is developing the system through a  series of modules (databases). EPA estimates that once IGMS becomes  fully operational, it will save roughly $1.6 million annually. EPA believes  IGMS could help resolve some of the long-standing problems in grants  management by implementing controls to prevent the submission of  certain documents without required elements and to provide electronic  reminders of when certain activities or documents are due. Additionally,  EPA designed the system to reduce the potential for data entry errors. In  2001, EPA began implementing the module for the application and award  phases of a grant. Using IGMS, EPA expects to electronically review the  grantee’s application, prepare and review EPA’s documents, and approve  the award. In April 2003, EPA began using IGMS’ postaward module. This  module allows project officers to electronically enter project milestones  into the system, communicate with other staff involved in overseeing  grants, receive reports from grantees, and initiate closeout activities. EPA  expects that grants specialists and regional program staff will use IGMS to  electronically manage grants by September 2004 and headquarters program  staff by the end of 2006.
Despite this promising development, the implementation period for IGMS  has been lengthy. This delay was due in part to Y2K concerns, the  complexity of the system for grants management, and the addition of other  major business processes into the system. Consequently, EPA has had to  plan a lengthy implementation schedule in order to ensure that it can  properly roll out the system to all of the programs and regions with its  available training and user-support resources. Therefore, while those  offices that have received IGMS are using it to manage grants, EPA cannot  yet determine whether IGMS has solved some of the agency’s resource  management problems.
section_title: New Policies and Plan Show Promise but Require Strengthening, Enhanced Accountability, and Sustained Commitment to Succeed
paragraphs: EPA’s recently issued policies on competition and oversight and a 5-year  plan to address its long-standing grants management problems are  promising and focus on the major management challenges, but these  policies and plan require strengthening, enhanced accountability, and  sustained commitment to succeed. In particular, EPA’s policy generally  requiring competition for discretionary grants should encourage selection  of the most qualified applicants. However, successful implementation of  this policy requires a more planned, rigorous approach to awarding some  grants than in the past, which is a major cultural shift for EPA managers  and staff. Similarly, EPA’s new oversight policy should increase the agency’s  monitoring of individual grantees, but it does not enable the agency to  identify and address systemic problems. Finally, EPA’s new grants  management plan takes significant steps toward improving grant  management, but the plan will be difficult to implement if EPA does not  demonstrate that it is committed to carrying out the plan over the 5-year  period and if it does not hold managers and staff accountable for their  grants management performance.
subsections: section_title: Competition Policy Should Encourage the Selection of the Most Qualified Applicants if Successfully Implemented
paragraphs: In September 2002, EPA issued a policy to promote competition in grant  awards by requiring that most discretionary grants be competed.  Discretionary grants totaled about $719 million of the $4.2 billion in grants  awarded in fiscal year 2002.   The policy applies to most discretionary grant  programs or individual grants of more than $75,000. It exempts individual  grants only if they meet certain criteria, such as national security interests.  Exemptions require detailed, written justification and approval. The policy  also promotes widespread solicitation for competed grants by establishing  specific requirements for announcing funding opportunities in, for  example, the Federal Register and on Web sites.
Because the competition policy represents a major change in the way EPA  awards discretionary grants, the agency created the position of a senior- level competition advocate for grants. This advocate oversees the policy’s  implementation and compliance and evaluates its effectiveness. Among  other things, the competition advocate is responsible for developing  implementation guidance and coordinating training to ensure successful  implementation. The competition advocate ensures compliance, in part, by  reviewing any requests for programs to be exempted from competition.  The advocate will also have to work with headquarters and regional offices  as well as staff and management officials at all levels throughout EPA to  successfully implement the policy.
The competition policy faces implementation barriers because it  represents a major cultural shift for EPA staff and managers, who have had  limited experience with competition, according to the Office of Grants and  Debarment. The policy requires EPA officials to take a more planned,  rigorous approach to awarding grants. That is, EPA staff must determine  the evaluation criteria and ranking of these criteria for a grant, develop the  grant announcement, and generally publish it at least 60 days before the  application deadline. Staff must also evaluate applications—potentially  from a larger number of applicants than in the past—and notify applicants  of their decisions. These activities will require significant planning and take  more time than awarding grants noncompetitively. Office of Grants and  Debarment officials anticipate a learning curve as staff implement the  policy and will evaluate the policy’s effectiveness, including the threshold  dollar level in 2005. While the policy has been in effect for a number of  months, it is too early to tell if the policy has resulted in increased  competition over the entire fiscal year. EPA officials believe that  preliminary results indicate that the policy is increasing the use of  competition.
section_title: Oversight Policy Makes Important Improvements but Requires Strengthening to Identify Systemic Problems
paragraphs: EPA’s December 2002 oversight policy makes important improvements, but  it does not fully address the weaknesses in previous oversight policies that  we noted earlier. The new policy, among other things, requires the  following:  Offices to increase baseline monitoring from at least once during the life  of the grant to annually throughout the life of the grant;  Offices to increase the percentage of in-depth monitoring to at least 10  percent of grant recipients;  Grant management specialists conducting on-site, in-depth reviews to  include “transaction testing” for unallowable costs, such as lobbying and  entertainment expenditures. Transaction testing consists of reviewing  grantees’ accounting ledgers and underlying documentation;  Offices to identify their criteria for selecting recipients for in-depth  reviews and submit these criteria as part of their annual plans;  Offices to enter in-depth reviews into the grantee compliance database;   The Office of Grants and Debarment’s Grants Administration Division to  have a work group recommend database enhancements to make the  database more useful for program offices. This work group plans to  report its findings in August 2003.
While the new policy makes these refinements in oversight, it still does not  enable EPA to identify systemic problems in grants management that we  noted earlier. Specifically, the policy does not do the following:  Incorporate a statistical approach to selecting grantees for review.
EPA’s new policy annually selects 10 percent of grant recipients for in- depth review. EPA officials believe that a single, higher percentage,  rather than a range, will eliminate variation among its offices and  provide better coverage. However, without a statistical approach, EPA  cannot evaluate whether this 10 percent is appropriate. With such an  approach, EPA could adjust the number and allocation of in-depth  reviews to match the level of risk associated with each type of grant  recipient and project the results of the reviews to all EPA grantees.
Require a standard reporting format for in-depth reviews. EPA needs  the clear and precise information from its in-depth reviews that a  standard reporting format would provide—and which was absent from  the 2002 in-depth reviews we analyzed. Standard information is  important so that EPA can use the information to guide its grants  oversight efforts agencywide.
Maximize use of information in the grantee compliance database.
While EPA has recognized the importance of the information in the  database by forming a database work group and collecting a limited  amount of data from in-depth reviews, it has not yet developed a plan  for using data from all its oversight efforts—in-depth reviews, corrective  actions, and other compliance efforts—to fully identify systemic  problems and then inform grants management officials about oversight  areas that need to be addressed.
section_title: New Plan Focuses on the Four Major Management Challenges but Will Require Strengthening, Sustained Commitment, and Enhanced Accountability
paragraphs: According to EPA’s Assistant Administrator for Administration and  Resources Management, the agency’s April 2003 5-year grants management  plan is the most critical component of EPA’s efforts to improve its grants  management. The plan has five goals and accompanying objectives, as  follows:  Promote competition in awarding grants. Identify annual funding  priorities, encourage a large and diverse group of applicants, promote  the importance of competition within the agency, and provide adequate  support for the grant competition advocate.
Strengthen EPA’s grants oversight. Improve internal management  reviews of EPA’s offices, improve and expand reviews of grant  recipients, develop approaches to prevent or limit grants management  weaknesses, establish clear lines of accountability for grants oversight,  and provide high-level coordination, planning, and priority setting.
Support the identification and achievement of environmental  outcomes. Ensure that grantees include expected environmental  outcomes and performance measures in grant work plans and improve  reporting on progress made in achieving environmental outcomes.
Enhance the skills of EPA personnel involved in grants management.  Update training materials and courses and improve delivery of training  to project officers and grants specialists.
Leverage technology to improve program performance. Enhance and  expand information systems that support grants management and  oversight, among other things.
In addition to the goals and objectives, the plan establishes performance  measures, targets, and action steps with completion dates for 2003 through  2006. For example, to strengthen EPA’s grants oversight, the plan calls for  conducting consolidated management reviews of headquarters and  regional offices to review the operations of grants management activities  and for completing reviews for three regional offices and four headquarters  offices in 2003. These reviews will examine EPA offices’ oversight of  grantees throughout the life of the grant. (See app. IV.)
We believe the grants management plan is comprehensive in that it focuses  on the four major management challenges—grantee selection, oversight,  environmental results, and resources—that we identified in our work. For  the first time, EPA plans a coordinated, integrated approach to improving  grants management. The plan is also a positive step because it (1) identifies  objectives, milestones, and resources to achieve the plan’s goals; (2)  provides an accompanying annual tactical plan that outlines specific tasks  for each goal and objective, identifies the person accountable for  completing the task, and sets an expected completion date; (3) attempts to  build accountability into grants management by establishing performance  measures for each of the plan’s five goals; (4) recognizes the need for  greater involvement of senior officials in coordinating grants management  throughout the agency by establishing a senior-level grants management  council to coordinate, plan, and set priorities for grants management; and  (5) establishes best practices for grants management offices.
EPA has already begun implementing several of the actions in the plan or  meant to support the plan; these actions address previously identified  problems. For example, EPA now ensures that grants are adequately  solicited by posting its available grants on the federal grants Web site  http://www.fedgrants.gov. To enhance oversight, it initiated the  consolidated grants management reviews of headquarters and regional  offices. To better manage for results, EPA formed an Environmental  Outcomes Work Group to address concerns about focusing on  environmental outcomes. Finally, for training, EPA (1) instituted  verification procedures to ensure that only certified project officers  manage grants; (2) drafted its long-term plan for grants management  training; (3) updated the project officer training manual; and (4) launched a  new training course for grants managers. In addition, EPA is implementing  an office award program to promote effective grants management.
Although EPA’s plan lays out a road map for improving grants management,  it has two weaknesses. First, in terms of measuring results, while the plan  supports identifying and achieving environmental outcomes, the suggested  protocol for in-depth reviews does not call for project officers to ask  grantees about their progress in using measures to achieve environmental  outcomes. Second, EPA’s plan may not fully address long-standing training  problems. Specifically, it is too early to tell whether EPA’s effort to develop  a long-term training plan will provide (1) a process to ensure that all staff  members receive the training necessary to implement new policies and  regulations in a timely manner or (2) a mechanism to evaluate the impact of  its training efforts on improving staff’s grants management knowledge and  skills, the extent to which staff’s performance improves over time, and  demonstrated results in improving grants management performance. The  impact of training could be measured by indicators such as the proportion  of financial or other problems identified early on by ongoing monitoring,  thereby avoiding potentially greater losses in the long run. Furthermore,  the plan’s performance measures for evaluating training are output- oriented—how many staff are trained—rather than outcome-oriented— how much skills have improved.
Successful implementation of the new plan requires more than addressing  these two weaknesses, however. It requires all staff—senior management,  project officers, and grants specialists—to be fully committed to, and  accountable for, grants management. Recognizing the importance of  commitment and accountability, EPA’s 5-year grants management plan has  as one of its objectives the establishment of clear lines of accountability for  grants oversight. The plan, among other things, calls for (1) ensuring that  performance standards established for grants specialists and project  officers adequately address grants management responsibilities in 2004; (2)  clarifying and defining the roles and responsibilities of senior resource  officials, grant specialists, project officers, and others in 2003; and (3)  analyzing project officers’ and grants specialists’ workload in 2004.
In implementing this plan, however, EPA faces challenges to enhancing  accountability. Although the plan calls for ensuring that project officers’  performance standards adequately address their grants management  responsibilities, agencywide implementation may be difficult. Currently,  project officers do not have uniform performance standards, according to  officials in EPA’s Office of Human Resources and Organizational Services.  Instead, each supervisor sets standards for each project officer, and these  standards may not include grants management responsibilities. It could  take up to a year to establish and implement a uniform performance  standard, according to these officials. Instead, the Assistant Administrator  for the Office of Administration and Resources Management is planning to  issue guidance in October 2003 on including grants management  responsibilities in individual performance agreements for the next  performance cycle beginning in January 2004. Once individual project  officers’ performance standards are established for the approximately  1,800 project officers, strong support by managers at all levels, as well as  regular communication on performance expectations and feedback, will be  key to ensuring that staff with grants management duties successfully meet  their responsibilities.
Although EPA’s current performance management system can  accommodate development of performance standards tailored to each  project officer’s specific grants management responsibilities, the current  system provides only two choices for measuring performance— satisfactory or unsatisfactory, which may make it difficult to make  meaningful distinctions in performance. Such an approach may not provide  enough meaningful information and dispersion in ratings to recognize and  reward top performers, help everyone attain their maximum potential, and  deal with poor performers.
Furthermore, it is difficult to implement performance standards that will  hold project officers accountable for grants management because these  officers have a variety of responsibilities and some project officers manage  few grants, and because grants management responsibilities often fall into  the category of “other duties as assigned.” To address this issue, EPA  officials are considering, among other options, whether the agency needs  to develop a smaller cadre of well-trained project officers to oversee  grantees, rather than rely on approximately 1,800 project officers with  different levels of grants management responsibilities and skills. Some EPA  officials believe that having a cadre may help the agency more effectively  implement revised grants management performance standards because  fewer officers with greater expertise would oversee a larger percentage of  the grants.
EPA will also have difficulty achieving the plan’s goals if all managers and  staff are not held accountable for grants management. The plan does not  call for including grants management standards in managers’ and  supervisors’ agreements. In contrast, senior grants managers in the Office  of Grants and Debarment as well as other Senior Executive Service  managers have performance standards that address grants management  responsibilities. However, middle-level managers and supervisors also  need to be held accountable for grants management because they oversee  many of the staff that have important grants management responsibilities.  According to Office of Grants and Debarment officials, they are working on  developing performance standards for all managers and supervisors with  grants responsibilities.
Further complicating the establishment of clear lines of accountability, the  Office of Grants and Debarment does not have direct control over many of  the managers and staff who perform grants management duties— particularly the approximately 1,800 project officers in headquarters and  regional program offices. The division of responsibilities between the  Office of Grants and Debarment and program and regional offices will  continue to present a challenge to holding staff accountable and improving  grants management, and will require the sustained commitment of EPA’s  senior managers.
section_title: Promising Practices Could Assist EPA in Addressing Its Grants Management Challenges
paragraphs: Although information on successful grants management practices is  limited, we identified promising practices in the five federal agencies that  award the largest amount of grant dollars, as well as in other organizations,  that may help EPA address its grants management challenges. These  practices include (1) screening grantees and assessing some grantees’  financial and internal controls before awarding a grant; (2) using an  information system to track and manage in-depth grantee reviews, identify  problems, and analyze trends; and (3) incorporating a results orientation  throughout the grants process, including the preaward phase and the final  reporting of results achieved. In addition, we have developed a guide to  assist federal agencies in developing effective training programs and  performance management systems.
subsections: section_title: Screen and Select the Best Applicants
paragraphs: According to some grants management officials we spoke with, screening  and reviewing the adequacy of grantees’ financial and internal controls  before awarding a grant helps to avoid problem grantees. For example,  The Robert Wood Johnson Foundation conducts preaward desk reviews  of some applicants to learn about the organization and assess its fiscal  health. Specifically, for grants under $200,000, the foundation may check  a Web site that contains financial information on nonprofit  organizations. For grants larger than $200,000, especially when the  grantee is newly formed or has never received money from the  foundation, staff may request more detailed information—such as an  organizational chart, copies of audit reports, accounting policies,  investment policies, and a conflict-of-interest statement. In rare cases,  the foundation’s financial staff may visit the applicant to assess its  systems.
The Department of Commerce (Commerce) Office of Inspector General  conducts a three-step screening process to identify deficiencies that  could hinder the ability of nonprofit applicants to properly control  federal funds: (1) a name check, through the Federal Bureau of  Investigation, to determine whether the individuals connected with a  proposed award have been convicted of embezzlement or other crimes;  (2) a review of the single audit clearinghouse database to determine if  the applicant had a single audit performed and subsequent follow-up to  determine if any findings were resolved; and (3) a review of a credit  report to determine whether the applicant is financially viable. For the  last two reporting periods—April 2002 to March 2003—Commerce’s  Inspector General screened 1,657 proposed awards; delayed 67 until  concerns were satisfactorily resolved; and established special  conditions for 42 awards to adequately safeguard federal funds.
The Department of Health and Human Services (HHS) uses an internal  “alert list,” accessible to grants management staff throughout the  department, to help identify potential grantees who (1) previously had  not complied with grant regulations or the terms and conditions of a  grant, (2) failed to have had a single audit performed, or (3) have  significant single audit findings that need correction. For these potential  grantees, HHS may incorporate special terms and conditions into their  agreements and more closely monitor them throughout the grant.  Applicants may be put on the alert list by agencies within the  department or at the recommendation of the Office of Inspector  General. According to HHS officials, the alert list should be used for  problem grantees so that the department has the necessary controls in  place to safeguard federal funds rather than be used to prohibit grantees  from receiving grant funding.
HHS’s Substance Abuse and Mental Health Services Administration  (SAMHSA) uses the alert list and also conducts pre-award reviews to  assess applicants’ ability to manage grant funds. Like the Department of  Commerce, SAMHSA screens some nonprofit grantees to ensure they  have the financial systems needed to manage grant funds. SAMHSA  examines the database for the single audit clearinghouse and its own  audit database to determine if the grant applicant has received a single  audit and whether the audit had significant findings. If the applicant has  no significant findings, SAMHSA assumes that the applicant has  adequate financial systems in place. However, if an audit found  significant problems that the applicant did not adequately address,  SAMHSA may designate the applicant as high risk and add it to the HHS  alert list. SAMHSA may still award grant funds but set restrictions to  safeguard the funds. For example, the grantee may be required to draw  funds on a reimbursement basis, that is, after it submits documentation  on its expenses. Grantees who have not received a single audit must  provide copies of their policies and procedures. SAMHSA assesses this  information to determine if it is sufficient for managing federal funds.
section_title: Use an Information System to Manage In-Depth Reviews, Identify Problems, and Analyze Trends
paragraphs: The Federal Transit Administration, within the Department of  Transportation, uses an information system to track and manage its  reviews of grantees. This system contains information on problems  identified in the reviews and a schedule of corrective actions that the  grantee needs to take. Using this system, officials can track  correspondence between the agency and grantee and document the  resolution of problems. In addition, officials can use this system to analyze  trends, such as the compliance of individual grantees over time, the  compliance of all grantees or a group of grantees (such as those in a  region), compliance issues found in different types of reviews, and areas  where grantees may require additional technical assistance. This  information system could serve as a model for enhancing EPA’s grantee  compliance database.
section_title: Incorporate a Results Orientation throughout the Grants Process
paragraphs: Several agencies consider grant results in awarding and reviewing  grantees, including the following:  Rural Development, within the U.S. Department of Agriculture, takes  outcomes into account by requiring applicants to provide information  on the results they expect to achieve and by selecting grantees, in part,  on the basis of this information. For instance, when job creation is a  goal of the grant, the Rural Business Enterprise Grant Program gives  applicants points for jobs they expect to create or save with a grant and  requires them to provide written commitments from businesses that  plan on creating jobs as a result of the grant. Rural Development then  gives points to applicants based on the grant dollars per job they expect  to create.
HHS’s Health Resources and Services Administration incorporates  information on the results its grantees have achieved into its annual  performance report. The administration collects data directly from its  grantees and surveys a sample of the people its grantees serve to  determine the results the grantees have achieved. Because of the  agency’s success in managing its health center grants to achieve  results, OMB rated the program “effective” in its governmentwide  performance and assessment-rating tool. Out of 79 grant programs  evaluated, this was one of only two rated “effective,” OMB’s highest  rating category. According to OMB, the health center program uses  performance information to improve annual administrative and clinical  outcomes. The agency also conducts its own evaluations and has a  contractor regularly evaluate the program; these evaluations indicate  the program is effective at achieving its goal of extending high-quality  health care to underserved populations.
section_title: Train and Hold Staff Accountable for Grants Management
paragraphs: We have developed a guide that can assist federal agencies in planning,  designing, implementing, and evaluating effective training and  development programs. This guide may help EPA address a training  weakness in the plan that we identified—EPA’s lack of a mechanism to  determine the extent to which training improves staff’s grant management  skills. For example, the guide suggests evaluating the impact of training  based on such measures as changes in employee skills, knowledge, or  abilities; changes in on-the-job behaviors; the impact of the training on  program or organizational results, including, in some cases, assessing the  return on investment by comparing training costs with benefits.
In terms of accountability, we developed key practices that federal  agencies can use to establish effective performance management systems.  These systems can be used to drive internal change and achieve desired  results. The key practices have helped public sector organizations in the  United States and abroad create a clear linkage between individual  performance and organizational success. They are also critical for ensuring  individual accountability for results throughout the organization. Key  practices of an effective performance management system include, for  example:   Align individual performance expectations with organizational goals.
An explicit alignment helps individuals see the connection between  their daily activities and organizational goals.
Use competencies to provide a fuller assessment of performance.
Competencies define the skills and supporting behaviors staff need to  effectively contribute to organizational results.
Make meaningful distinctions in performance. Effective performance  management systems strive to provide candid and constructive  feedback and the necessary objective information and documentation to  award top performers and deal with poor performers.
section_title: Conclusions
paragraphs: If EPA is to better achieve its environmental mission, it must more  effectively manage its grants programs—which account for more than half  of its annual budget. EPA’s new policies and 5-year grants management plan  show promise, but they are missing several critical elements necessary for  the agency to address past grants management weaknesses. Specifically,  EPA’s new oversight policy lacks (1) statistical methods to identify grantees  for review, (2) a standard reporting format to ensure consistency and  clarity in report results, and (3) a plan for integrating and analyzing  compliance information from multiple sources. These actions would help  EPA identify systemic problems with its grantees and better target its  oversight resources. EPA’s plan does not adequately address the need to  assess grantees’ progress towards achieving environmental outcomes. If  project officers tracked grantees’ progress during in-depth reviews, they  would better help the agency manage grants for results. Finally, in our view,  EPA’s plan does not go far enough to ensure that all managers and staff with  grant management responsibilities are held accountable for fulfilling their  grants management responsibilities. Until EPA holds all managers and staff  accountable for these responsibilities, it cannot be assured that it will  achieve the objectives of its grants management plan.
EPA recognizes the importance of continuously improving its grants  management practices. To this end, we believe EPA could benefit from  considering the practical approaches successfully implemented by other  federal agencies and organizations. It could also benefit from using our  guide to establish effective training programs to develop relevant staff  skills and a performance management system to hold staff accountable for  using these skills to carry out their grants management responsibilities.
Given EPA’s uneven performance in improving its grants management, it is  too early to tell if its new policies and comprehensive 5-year grants  management plan will succeed more than past initiatives. While the plan  shows promise, we believe that congressional oversight is important to  ensure that EPA’s Administrator, managers, and staff implement the plan in  a sustained, coordinated fashion to meet the plan’s ambitious targets and  time frames. To help facilitate this oversight, EPA’s annual report to  Congress could serve as a vehicle for EPA to report on its achievements in  improving grants management.
section_title: Recommendations for Executive Action
paragraphs: To ensure that EPA’s recent efforts to address its grants management  challenges are successful, we recommend that the Administrator of EPA  provide sufficient resources and commitment to meeting the agency’s  grants management plan’s goals, objectives, and performance targets  within the specified time frames. Furthermore, to strengthen EPA’s efforts  we recommend   incorporating appropriate statistical techniques in selecting grantees for   requiring EPA staff to use a standard reporting format for in-depth  reviews so that the results can be entered into the grant databases and  analyzed agencywide;   developing a plan, including modifications to the grantee compliance  database, to use data from its various oversight efforts—in-depth  reviews, significant actions, corrective actions taken, and other  compliance information—to fully identify systemic problems, inform  grants management officials of areas that need to be addressed, and  take corrective action as needed;      modifying its in-depth review protocols to include questions on the  status of grantees’ progress in measuring and achieving environmental  outcomes;   incorporating accountability for grants management responsibilities  through performance standards that address grants management for all  managers and staff in headquarters and the regions responsible for  grants management and holding managers and staff accountable for  meeting these standards; and  evaluating the promising practices identified in this report and  implementing those that could potentially improve EPA grants  management.
To better inform Congress about EPA’s achievements in improving grants  management, we recommend that the Administrator of EPA report on the  agency’s accomplishments in meeting the goals and objectives developed in  the grants management plan and other actions to improve grants  management, beginning with its 2003 annual report to Congress.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to EPA for its review and comment. In  response, EPA stated that it agreed with our recommendations and that it  will implement them as part of its 5-year grants management plan. EPA  further pointed out that it has demonstrated commitment to grants  management as evidenced by its (1) issuance of directives from the Deputy  Administrator/Acting Administrator requiring senior managers to hold  employees accountable and to include compliance with grants  management policies as part of midyear performance discussions; (2)  ongoing review of performance standards, which will ensure that  performance agreements of all employees involved in grants management  adequately reflect their grants management responsibilities; (3)  requirement for assistant administrators and regional administrators to  address in the assurance letters under the Federal Managers’ Financial  Integrity Act the steps they are taking to address grants management  weaknesses; and (4) development of a tactical action plan, which outlines  commitments and milestone dates under the grants management plan and  identifies who is responsible for completing those commitments. EPA also  provided the Office of Grants and Debarment’s Fiscal Year 2003 Federal  Managers’ Financial Integrity Act assurance letter, which describes in  detail the actions the office has taken.
In technical comments, EPA added that it had redeployed resources to the  Office of Grants and Debarment and anticipates receiving new resources in  fiscal year 2004 to conduct grants management oversight. Other technical  comments have been incorporated into the report, as appropriate.
EPA’s written comments are presented in appendix V.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to the  congressional committees with jurisdiction over EPA and its activities; the  Honorable Marianne Lamont Horinko, Acting EPA Administrator; and the  Honorable Joshua B. Bolten Director, Office of Management and Budget.  We will also make copies available to others upon request. In addition, the  report will be available at no charge on the GAO Web site at  htpp://www.gao.gov.
If you have any questions about this report, please contact me at (202) 512- 3841. Key contributors to this report are listed in appendix VI.
section_title: Objectives, Scope, and Methodology
paragraphs: This appendix details the methods we used to determine the (1) major  challenges the Environmental Protection Agency (EPA) faces in managing  its grants and how it has addressed these challenges in the past, (2) extent  to which EPA’s recently issued policies and grants management plan  address these challenges, and (3) promising practices, if any, that could  assist EPA in addressing these challenges.
To identify the challenges EPA faces in managing its grants and how it has  addressed these challenges in the past, we identified and analyzed 93  reports written on EPA’s grants, including our reports, EPA’s Inspector  General reports, and EPA’s internal management reviews conducted from  1996 through 2003. To analyze the findings from these reviews, we  summarized the findings from each report and then grouped the findings  using the long-table method of analysis. From those groups, we identified  four broad grants management challenges. We next interviewed and  obtained documents from EPA officials to determine the actions they have  taken in the past to address these challenges. We also obtained the records  of 1,232 in-depth grantee reviews conducted in 2002 and analyzed each one  using a data collection instrument. We conducted inter-rater reliability tests  to ensure that information from in-depth reviews was entered consistently  by each reviewer. In the event a document item was unclear, reviewers  flagged the case and consulted to determine the appropriate response. We  requested that EPA send documents for all evaluative reviews conducted in  calendar year 2002. Since EPA does not have information on the exact  number of reviews it conducts each year, we were not able to verify  whether we received information on the total population of evaluative  reviews conducted in 2002. Because we could not verify the total number  of reviews conducted and because some regions and program offices  conduct a small number of reviews, we requested documents for all in- depth evaluative reviews in 2002 (rather than a sample) to ensure coverage  of all 10 regions and 10 program offices.
To determine the extent to which EPA’s recently issued policies and grants  management plan address these challenges, we reviewed the new policies  and plan and interviewed EPA officials responsible for implementing key  aspects of the plan. We also attended EPA’s grants management training  course for project officers both before and after they revised their project  officer training manual and EPA’s training course for nonprofit  organizations. We also observed several of EPA’s in-depth reviews of  grantees: one administrative on-site review conducted by EPA  headquarters, two on-site joint reviews conducted by EPA’s Region 1 office,  and two joint desk reviews conducted by EPA’s Region 1 office. In addition,  we interviewed officials in EPA’s Office of Human Resources and  Organizational Services to discuss the agency’s performance management  system.
To identify what promising practices, if any, could assist EPA in addressing  its grants management challenges, we interviewed and obtained  documentation from grants management officials, and other officials, in the  federal government and from private sector organizations. For the federal  government, we met with officials from the five largest granting agencies:  the Departments of Agriculture, Education, Health and Human Services,  Housing, and Transportation. At the recommendation of these and other  officials, we interviewed other agency officials, including those from  Agriculture’s Rural Development, the Department of Commerce, Health  and Human Service’s Health Resources and Services Administration and  Substance Abuse and Mental Health Services Administration, and  Transportation’s Federal Transit Administration. We also interviewed  officials from the Council on Foundations, a membership organization, and  two foundations: the Robert Wood Johnson Foundation and the Ford  Foundation. We asked these officials for examples of their grant practices  that might address the types of challenges EPA faces in managing grants.  We also reviewed two GAO reports that describe effective training and  performance management systems for federal agencies.
We conducted our work from June 2002 through June 2003 in accordance  with generally accepted government auditing standards.
section_title: EPA’S 10 Regions and Regional Office Locations
section_title: In-depth Reviews
paragraphs: This appendix presents tables that provide detail on the in-depth reviews  that the Environmental Protection Agency (EPA) conducted in 2002. In- depth reviews include both on-site reviews conducted at the grantee’s  location and off-site reviews conducted at an EPA office or another  location. Tables 5 and 6 show the number of different types of reviews and  how many reviews each EPA office conducted. Tables 7 to 12 show the  number and types of problems EPA found and how many problems were  found for each type of grantee and size of grant. Table 13 shows the number  of times EPA required or recommended action to correct a problem. Tables  14 and 15 show how many reviews EPA conducted with or without a  protocol. Tables 16 and 17 show how many and what types of significant  actions EPA took against grantees.
section_title: Summary of EPA’s Grants Management Plan, 2003-2008
subsections: section_title: Goal 1: Enhance the Skills of EPA Personnel Involved in Grants Management
paragraphs: 1.1 Update and enhance training materials and course curricula to   emphasize high priority areas. 1.2 Improve delivery and availability of training programs. 1.3 Provide training to managers and supervisors.
Table 18 shows the performance measures for goal 1.
Issue Project Officer Training Manual (fifth edition), which will focus  on core competencies needed to manage grants.
Conduct project officer training with special emphasis on the core  competency areas that were enhanced in the fifth edition of the training  manual.
Issue first edition of the Grants Specialist Training Manual.
Conduct grants specialist training focusing on core competency areas.
Develop long-term Grants Management Training Plan.
Issue guidance on the proper use of amendments.
Ensure that project officers receive basic and refresher grants  management training on a timely basis (ongoing).
Develop a pilot grants management training program for managers and  supervisors.
Enhance and update on-line reference materials for grant specialists in  core competencies.
Implement on-line training for grants specialists in core competency  areas.
Enhance on-line training for refresher project officers course to include  additional materials on preapplication review, competition, post-award  monitoring, environmental outcomes, and other new areas covered in  the Project Officer Training Manual, as well as verification of  completion at each step.
Conduct grants management classroom training for managers and  supervisors.
Implement on-line basic project officers course, including verification of  completion at each step.
Conduct grants management on-line training for managers and  supervisors.
Establish certification program for grants specialists based on an  examination of skills and core competencies required to manage grants.
section_title: Goal 2: Promote Competition in the Award of Grants
paragraphs: 2.1 Improve the agency’s process for identifying annual funding priorities     and planning for competition. 2.2 Encourage a large and diverse group of grant applicants. 2.3 Promote agencywide understanding of the importance of competition. 2.4 Provide adequate support to the Grants Competition Advocate.
Table 19 shows the performance measures for goal 2.
Provide guidance to national program managers on Catalog of Federal  Domestic Assistance (CFDA) descriptions that publicize funding  priorities and opportunities.
Include national program managers funding priorities in the CFDA on an  annual basis to ensure they are well advertised and linked to the  agency’s Government Performance and Results Act goals.
Develop guidelines and make postcompetition award information  available to the public through the Internet.
Develop a central grants competition Web page linked to program office  Web sites to make grant solicitations easily available to the public.
Post all grant solicitations on the Fed Biz Opps Web site.
Develop standard agencywide grants competition training materials and  incorporate them into project officer and grant specialist training.
Provide full staffing for the Grants Competition Advocate.
Develop a structured annual planning process for grant competitions.
Expand public awareness of funding opportunities by improving the  accuracy and specificity of program descriptions in the CFDA.
Research innovative grants competition strategies performed by federal,  state, or local governments and determine suitability for adapting them  for the agency’s use.
Implement a structured annual planning process for grant competitions.
section_title: Goal 3: Leverage Technology to Improve Program Performance
paragraphs: 3.1 Continue deployment of the Integrated Grant Management System  (IGMS). 3.2 Integrate IGMS with federal-wide e-grant initiatives. 3.3 Enhance and expand information systems that support grants oversight.
Table 20 shows performance measures for goal 3.
Complete deployment of IGMS in Regions 2, 4, 5, and 8 and of pilot in  headquarters component.
Develop an interface between IGMS and the IFMS so that commitment  notice information need be entered only once.
Complete analysis of changes required to IGMS and Grants Information  Control System to enable these systems to interface with federal   e-grants portal.
Develop and implement an IGMS postaward module.
Correct inconsistencies and inaccuracies in the naming of grantees,  grant specialists, and project officers so that accurate reports of  workload and recipients counts are available.
Enhance the IGMS electronic grant file reporting views for the reports  needed to monitor grant specialist and project officer workloads.
Expand the project officer database to track certification and  recertification activities.
Establish a tracking system to identify competition status of grants.
Enable electronic transmission of award data from IGMS to the IFMS.
Develop an interface to federal e-grants portal to enable IGMS to accept  applications.
Develop a project officer interface to simplify use of IGMS.
Ensure that agency information technology systems make grants  information readily available to EPA personnel.
Create an agency Web site to allow grant specialists and project officers  to access best practices and other tools for postaward management.
Expand e-grants portal to enable IGMS to accept reporting.
Modify IGMS data elements to e-grants standards for reporting.
Develop an administrative review checklist in IGMS to ensure that grant  packages are complete, comprehensive and in compliance with EPA  orders and policies.
Complete deployment of IGMS in headquarters.
section_title: Goal 4: Strengthen EPA Oversight of Grants
paragraphs: 4.1 Improve grants management reviews of EPA offices. 4.2 Improve and expand external reviews of grant recipients. 4.3 Develop approaches to prevent or limit grants management    weaknesses. 4.4 Establish clear lines of accountability for grants oversight. 4.5 Provide high-level coordination, planning, and priority setting for grants    management.
Table 21 shows the performance measures for goal 4.
Combine the management oversight and postaward validation grants  management review protocols of offices.
Conduct cradle-to-grave management reviews of headquarters and  regional offices starting in 2003.
Issue revised guidance on grants management self-assessments.
Require all offices to conduct baseline monitoring of all active grantees  (ongoing).
Increase the level of advanced monitoring of grantees by offices,  including desk reviews and on-site reviews, to a minimum of 10 percent  annually of active recipients.
Ensure that all offices record their activities in the agency’s compliance  database including documentation of the results of compliance reviews.
Ensure that all offices submit timely annual postaward monitoring  plans.
Ensure that all offices, on an annual basis, define and identify categories  of at-risk grantees requiring technical and/or compliance assistance.
Complete development, with the EPA’s Office of Inspector General, of  an instructional video for nonprofit recipients on how to manage their  grants.
Provide training courses for nonprofit recipients.
Develop grants management tribal training manual and provide training  to tribes.
Develop guidance on required procurement processes under grants,  including competition, cost and price analyses, and avoidance of  conflicts of interest, and make available to grantees.
Clarify roles and responsibilities of senior resource officials.
Issue grants policy document defining roles and responsibilities of  grants management officers, program office officials, and project  officers.
Establish senior-level Grants Management Council.
Update Grants Management Office business plans in 2003 and annually  thereafter.
Establish a preaward financial systems review program for at-risk  grantees.
Establish grants clearinghouse/hotline for grant recipients.
Review and update EPA’s indirect cost rate policies.
Ensure that performance standards established for grant specialists and  project officers adequately address grants management responsibilities.
Develop workload analysis of project officers and grant specialists.
Establish standard operation procedures and best practices for grants  management offices.
Provide guidance to grants management offices and program offices on  how resources should be allocated to ensure effective and efficient  grants management oversight.
Develop grants management resource tool to replace the Assistance  Administration Manual.
section_title: Goal 5: Support Identifying and Achieving Environmental Outcomes
paragraphs: 5.1 Include expected environmental outcomes and performance measures     in grant work plans. 5.2 Improve reporting on progress made in achieving environmental    outcomes.
Table 22 shows the performance measures for goal 5.
Develop a tutorial for grantees on how to develop performance  measures for work plans.
Issue grants policy guidance to ensure that all grant work plans,  decision memoranda, and/or terms of condition include environmental  outcomes and how to measure them.
Require a discussion of expected environmental outcomes and  performance measures in grant solicitations.
Establish reporting on environmental outcomes as a criterion for  approval of interim and final reports.
Incorporate success in reporting on outcomes into the criteria for  awarding new grants.
section_title: Comments from the Environmental Protection Agency
section_title: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to those named above, Carl Barden, Christopher Murray, Paul  Schearf, Rebecca Shea, Carol Herrnstadt Shulman, Kelli Ann Walther, and  Amy Webbink made key contributions to this report.
section_title: Related GAO Products
paragraphs: Environmental Protection Agency: Problems Persist in Effectively  Managing Grants. GAO-03-628T. Washington, D.C.: June 11, 2003.
Federal Assistance: Grant System Continues to Be Highly Fragmented.  GAO-03-718T. Washington, D.C.: April 29, 2003.
Results-Oriented Cultures: Creating a Clear Linkage between Individual  Performance and Organizational Success. GAO-03-488. Washington, D.C.:  March 14, 2003.
Major Management Challenges and Risks: Environmental Protection  Agency. GAO-03-112. Washington D. C.: January 2003.
Results-Oriented Cultures: Using Balanced Expectations to Manage  Senior Executive Performance. GAO-02-966. Washington, D.C.: September  27, 2002.
Environmental Protection: Grants Awarded for Continuing  Environmental Programs and Projects. GAO-01-860R. Washington, D.C.:  June 29, 2001.
Environmental Protection: EPA’s Oversight of Nonprofit Grantees’ Costs  Is Limited. GAO-01-366. Washington, D.C.: April 6, 2001.
Environmental Protection: Information on EPA Project Grants and Use  of Waiver Authority. GAO-01-359. Washington, D.C.: March 9, 2001.
Environmental Research: STAR Grants Focus on Agency Priorities, but  Management Enhancements Are Possible. GAO/RCED-00-170.  Washington, D.C.: September 11, 2000.
Environmental Protection: Grants for International Activities and Smart  Growth. GAO/RCED-00-145R. Washington, D.C.: May 31, 2000.
Environmental Protection: Collaborative EPA-State Effort Needed to  Improve Performance Partnership System. GAO/T-RCED-00-163.  Washington, D.C.: May 2, 2000.
Managing for Results: EPA Faces Challenges in Developing Results- Oriented Performance Goals and Measures. GAO/RCED-00-77.  Washington, D.C.: April 28, 2000.
Environmental Protection: Collaborative EPA-State Effort Needed to  Improve New Performance Partnership System. GAO/RCED-99-171.  Washington, D.C.: June 21, 1999.
Environmental Protection: Factors Contributing to Lengthy Award  Times for EPA Grants. GAO/RCED-99-204. Washington, D.C.: July 14, 1999.
Environmental Protection: EPA’s Progress in Closing Completed Grants  and Contracts. GAO/RCED-99-27. Washington, D.C.: November 20, 1998.
Dollar Amounts of EPA’s Grants and Agreements. GAO/RCED-96-178R.  Washington D.C.: May 29, 1996.
subsections: section_title: GAO’s Mission
paragraphs: The General Accounting Office, the audit, evaluation and investigative arm of  Congress, exists to support Congress in meeting its constitutional responsibilities  and to help improve the performance and accountability of the federal government  for the American people. GAO examines the use of public funds; evaluates federal  programs and policies; and provides analyses, recommendations, and other  assistance to help Congress make informed oversight, policy, and funding  decisions. GAO’s commitment to good government is reflected in its core values of  accountability, integrity, and reliability.
section_title: Obtaining Copies of GAO Reports and Testimony
paragraphs: The fastest and easiest way to obtain copies of GAO documents at no cost is  through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older  products. The Web site features a search engine to help you locate documents  using key words and phrases. You can print these documents in their entirety,  including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and  correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site  daily. The list contains links to the full-text document files. To have GAO e-mail this  list to you every afternoon, go to www.gao.gov and select “Subscribe to   e-mail alerts” under the “Order GAO Products” heading.
subsections: section_title: Order by Mail or Phone
section_title: To Report Fraud, Waste, and Abuse in Federal Programs
section_title: Public Affairs