Grants Management: Despite Efforts to Improve Weed and Seed Program Management, Challenges Remain
Published: 2004-03-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Weed and Seed program, within the Department of Justice's Office of Justice Programs (OJP), aims to prevent and reduce violent crime in targeted neighborhoods, but it cannot optimize its effectiveness without sound management practices. In 1999, GAO made four recommendations to the Executive Office for Weed and Seed (EOWS) to improve the program's management, including (1) developing adequate internal controls to fully document decisions, (2) improving program monitoring, (3) developing criteria for determining when sites have become self-sustaining and when to reduce or withdraw program funding, and (4) developing additional performance measures. GAO did this study to assess progress in implementing these recommendations.
section_title: What GAO Found
paragraphs: Despite some progress toward addressing GAO's recommendations aimed at improving program management, GAO's review shows that EOWS has not fully implemented the management improvement recommendations GAO made in 1999. First, although EOWS has revised its internal controls to require that significant qualification and funding decisions be documented and readily available in the central grant files for review, EOWS has not always ensured that its policies and procedures were followed, for the grant files GAO reviewed. Second, EOWS reported taking a number of actions intended to improve program monitoring, such as mandating the timely submission of progress reports and adequate recording of site visits as GAO recommended. Nonetheless, GAO found that while EOWS was able to provide such documentation before its review ended, documentation was not available in some of the central grant files GAO reviewed. Thus, the documentation was not readily available for external reviewers, as required by OJP policies and GAO's internal control standards. Third, GAO found that EOWS still lacks fully developed criteria to determine when sites become self-sustaining and when to reduce or withdraw Weed and Seed funds because of the level of sustainability, even though sustainability is a central goal of the program. At the time of GAO's review, no site's funding had been reduced or withdrawn because of sustainability during the 13 years of the program's existence. Fourth, EOWS has not developed outcome performance measures that can be used to adequately track progress toward program outcomes of the Weed and Seed program. While EOWS has initiated studies on how to develop performance measures, at the time of GAO's review, none of these studies had been completed. Without requirements to monitor improvements and assign accountability, progress will be difficult to achieve.

section_title: Letter
paragraphs: The Department of Justice’s (Justice) Weed and Seed program proposes to  “weed” out crime from targeted neighborhoods and “seed” them with a  variety of programs and resources to prevent crime from recurring. Weed  and Seed sites have sponsored activities such as police bike patrols,  community cleanups, youth and recreational activities, and computer  training. According to Justice, a central tenet of the Weed and Seed  program is for local Weed and Seed sites to develop partnerships with  other federal, state, and local governments and private sector agencies to  leverage federal Weed and Seed grant funds with additional resources  from these partners to promote weeding and seeding activities. These  additional resources are intended to help the sites achieve the goal of  becoming self-sustaining after Weed and Seed funding ends and to sustain  crime reduction and community revitalization activities to ensure stable  communities.
This report responds to your request to assess the Executive Office for  Weed and Seed’s (EOWS) efforts to implement the management  improvement recommendations we made in 1999. It is not intended to  evaluate the overall management or results of the program. In our  previous report on the Weed and Seed program, we recommended that  EOWS improve its management of the Weed and Seed program.  Specifically, we recommended that EOWS (1) develop adequate internal  controls to ensure that the basis and the rationale for new and existing site  qualification and funding decisions are always fully documented,   (2) improve program monitoring to ensure that sites meet the grant  requirement of submitting progress reports and that EOWS site visits are  documented, (3) develop criteria for determining when sites are self- sustaining and when to reduce or withdraw program funding, and   (4) develop additional performance measures that track program  outcomes. This report summarizes our assessment of EOWS’s efforts to  address our recommendations.
To obtain information on EOWS’s efforts, we reviewed relevant  documents, including EOWS’s policies and procedures, monitoring  documentation, and agency staffing and budget data. We reviewed GAO,  Congressional Research Service, and Justice’s Office of Inspector General  reports. In addition, we interviewed officials from EOWS, the Office of  Justice Programs, other entities in the Justice Department, and  researchers in performance measurement and evaluation at the Justice  Research and Statistics Association and the Urban Institute in Washington,  D.C. We visited and interviewed program staff at three Weed and Seed  sites. We reviewed 30 randomly selected Weed and Seed grant files to  identify the actions taken by EOWS for those grants to document its  qualification and funding decisions and the steps EOWS took to ensure  that grant documentation requirements were met. Since the files we  reviewed were not representative of all EOWS grant files, we cannot  project the results to the larger population. However, the information  helps to identify the level of implementation of our recommendations for  those grants. Additional information about the report’s scope and  methodology is presented in appendix I. We conducted this engagement in  accordance with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: The Weed and Seed program is a Justice discretionary grant program  within OJP. The Assistant Attorney General for OJP is responsible for the  overall management and oversight of offices within OJP, including EOWS.  The Assistant Attorney General for OJP sets policies, promotes  coordination among OJP bureaus and offices, and ensures that EOWS  follows its policies and procedures. EOWS provides funding to grantees to  help prevent and control crime and improve the quality of life in targeted  high-crime neighborhoods across the country. It is a joint federal, state,  and local program for coordinated law enforcement and neighborhood  reinvestment. Federal program funding is to support Weed and Seed sites  and to provide training and technical assistance.
The Weed and Seed program has grown since it began in fiscal year   1991 with three pilot sites in Kansas City, Missouri; Omaha, Nebraska; and  Trenton, New Jersey, and a relatively small investment of federal money.   From fiscal year 1999 through 2003, the number of Weed and Seed sites  increased from 163 to 221, while the total annual program budget generally  increased from about $45 million to $70 million. See table 1 for fiscal years  1999-2003 data on the Weed and Seed program, including EOWS’s funding  history. In fiscal year 2003, with a budget of about $70 million, EOWS  awarded grants to 221 Weed and Seed sites.
EOWS is responsible for the national management and administration of  the Weed and Seed program, including developing policy and providing  guidance and oversight. EOWS currently administers the Weed and Seed  program with a staff of 2 management officials, 10 grant monitors, 3  support staff, 3 detailees, and 6 contractors.
Before a community can apply for Weed and Seed grant funding and  become an eligible site, it must first apply for and gain official recognition  from EOWS. In order to obtain official recognition, a potential Weed and  Seed site must show in its application that it has a strategy for weeding  and preventing crime. Once the application has been received, EOWS  creates and maintains the official recognition files that are intended to  include documentation such as the rationale for decisions to grant or deny  official recognition to an applicant. Official recognition requires the U.S.  Attorney in the area where the Weed and Seed site is to be located to  organize a local steering committee made up of various federal, state, and  local representatives, including residents, to be responsible for local  administration of the program. For official recognition, a site is also  required to develop a management plan, engage residents and other  partners in its activities, and develop a comprehensive program to weed  out crime and gang activity and seed the area with social services,  economic services, and economic revitalization.
Weed and Seed program guidance requires that its sites show plans for  addressing four required elements: (1) law enforcement; (2) community  policing; (3) crime prevention and intervention, and substance abuse  prevention, intervention, and treatment; and (4) neighborhood restoration.  According to EOWS, law enforcement should attempt to eliminate the  most violent offenders by coordinating and integrating the efforts of  federal, state, and local law enforcement agencies in targeted high-crime  neighborhoods. The objective of community policing is to raise the level of  citizen and community involvement in crime prevention and intervention  activities. Crime and substance abuse prevention, intervention, and  treatment should include youth services, school programs, community and  social programs, and support groups. Finally, neighborhood restoration  should focus on assistance to distressed neighborhoods through economic  and housing development. Weed and Seed sites fund a variety of law  enforcement and community activities. For example, law enforcement- funded activities range from participation in a multijurisdictional task  force to conducting bike patrols in the community. See appendix II for  additional information on Weed and Seed activities at the sites GAO  visited.
Applicants that are officially recognized are eligible to apply for funding  for up to 5 years. OJP’s Office of the Comptroller creates and maintains  official grant files for sites that are awarded funding by EOWS. Applicants  are rated based on the strategy they developed in response to the four  required elements, as stated above.
In our 1999 report, we made four recommendations to help EOWS  improve program monitoring and management. The first was to develop  adequate internal controls to fully document significant qualification and  funding decisions. We found that EOWS lacked internal controls requiring  significant program management decisions be documented. Therefore,  EOWS was not able to ensure that it was making the best decisions about  allocating available funds. Our second recommendation was to improve  program monitoring to ensure that sites met the grant requirement of  submitting progress reports and that EOWS site visits were documented.  We found that EOWS did not always ensure that local sites submitted  progress reports and that grant monitors documented the results of site  visits. Such documentation would help EOWS management and grant  monitors determine how sites are meeting program objectives and how  well sites are complying with grant requirements, and assist them in  making future grant qualification decisions. Our third recommendation  was to develop criteria for determining when sites have become self- sustaining and when to reduce or withdraw program funding. We found  that although self-sustainability is central to the program, no site’s funding  had been reduced or withdrawn as a result of its efforts to become self- sustaining in the 9 years of the program’s existence. Establishing such  criteria is important because the Weed Seed program was founded on the  premise that federal funding would continue for a finite period, after  which a Weed and Seed site would be self-sustaining. Our fourth  recommendation was to develop additional performance measures to  track program outcomes. We found that EOWS’s performance indicators  generally tracked activities rather than program results; therefore, EOWS  was not able to measure the success of the program.
EOWS generally agreed with three of the four recommendations presented  in the report and discussed future actions it planned to take. EOWS  officials disagreed with our recommendation on self-sustainability. They  stated that developing criteria to ascertain self-sustainment is redundant  since EOWS adopted a 5-year rule under which it could discontinue  awarding funding to qualifying sites unless the sites expanded to an  additional neighborhood site.
section_title: Internal Controls Have Been Developed, but Challenges Remain
paragraphs: Although EOWS has developed internal controls intended to require that  significant qualification and funding decisions be documented and readily  available for review as we recommended in 1999, these policies and  procedures are generally not being followed in the files we reviewed. In  response to our 1999 recommendation on internal controls, EOWS  established policies and procedures intended to ensure that significant  qualification decisions were documented. In addition, OJP requires that  qualification and funding decisions are to be documented in official  recognition and official grant files. Our review of 20 official recognition  and 10 official grant files showed that some of EOWS’s official recognition  and official grant files were missing full documentation regarding the  qualification and funding decisions. However, before our file review  ended, EOWS officials produced further documentation, which they  acknowledged was not in the official recognition and official grant files as  required, but rather in the personal working files of grant monitors and  thus not readily available to EOWS management and external reviewers.  This lack of ready availability is not in keeping with EOWS’s and OJP’s  policies and procedures, or with the Comptroller General’s standards for  internal controls. Without having official recognition and official grant  files complete and readily available, it may delay and complicate EOWS  officials’ oversight of the documentation of qualification and funding  decisions.
subsections: section_title: Policies and Procedures Have Been Developed to Document Significant Decisions
paragraphs: Both EOWS and OJP policies and procedures have been developed to help  ensure the documentation of significant decisions. In 2000, EOWS  developed a policies and procedures guide in response to the  recommendation we made in our 1999 report intended to ensure that  significant qualification and funding decisions for new and existing sites  were always fully documented, and further revised the guide in 2003. This  guide requires that all documentation pertaining to official recognition  decisions be kept in the official recognition files. EOWS is also required to  follow the policies in the OJP Grants Management Policies and  Procedures Manual, which requires that EOWS fully document program  management decisions in both official recognition files and official grant  files. The OJP manual specifically requires that the official recognition  files and official grant files contain key documents such as ratings  information, rejection letters, and applications.
section_title: Official Recognition and Official Grant Files Were Incomplete
paragraphs: Despite EOWS’s and OJP’s efforts to require the documentation of  qualification and funding decisions in official recognition and official grant  files, some of EOWS’s official recognition and official grant files we  reviewed were incomplete at the time of our file review. In reviewing the  official recognition files, we randomly selected 10 files to review from the  94 files submitted for official recognition in fiscal year 2002. Additional  information about how we selected the files is in appendix I. We found  sufficient documentation in 7 of the 10 files to determine the basis and  rationale for decisions to award official recognition. However, in the  remaining 3 files we were unable to determine the basis and rationale for  such decisions because documentation was insufficient. For example, key  documents such as the ratings information that EOWS grant monitors use  to record their assessment of the official recognition applications did not  contain the basis and rationale for the award decision. Additionally,  OJP’s grant policies and procedures require EOWS’s grant monitors to  prepare a rejection letter, informing applicants of reasons for rejection  when funding is denied and place a copy of the rejection letter in the  official recognition file. We reviewed all 10 official recognition files for the  applicants that were eligible to apply for funding in fiscal year 2002 but  were rejected. We found a rejection letter was missing in 7 of the 10 files.
We also reviewed documents in EOWS’s official grant files to assess  EOWS’s efforts to comply with EOWS and OJP policies for fully  documenting funding decisions. We randomly selected 10 official grant  files to review from the 31 sites that were funded in fiscal year 2002. We  found that none of the 10 official grant files fully documented funding  decisions for Weed and Seed sites. For example, the application, a basic  component of the official grant file, was missing in 8 of the 10 files.
section_title: Challenges Persist in Making Documentation Readily Available
paragraphs: Several directives require that grant documentation be readily available.  OJP’s policies and procedures manual requires that documentation be  kept readily available in the official recognition and official grant files, so  that OJP and EOWS management can identify and resolve any problems or  deficiencies in grantees’ compliance with relevant policies and  procedures. OJP’s policies and procedures manual requires that all  pertinent information that should be in the official recognition and the  official grant files be kept in a centralized location to facilitate reviewing  for completeness. In addition, the Comptroller General’s guidance on  internal controls in the federal government, Standards for Internal  Control in the Federal Government, requires that the documentation for  all transactions and significant events be readily available for examination.  The Assistant Attorney General for OJP is responsible for holding EOWS  accountable for following its policies and procedures. In reference to the  ratings information and rejection letters missing from the official  recognition files, EOWS produced the documents after our initial review,  informing us that the documents were kept in the personal files of the  grant monitors--rather than the official recognition files--a practice not in  keeping with EOWS’s and OJP’s policies and procedures. In reference to  the missing applications, EOWS officials also produced them after our  initial review, but did not specify where they found them. Not having  complete and readily available official recognition and official grant files  may delay and make it more difficult for OJP and EOWS officials to  perform management functions such as overseeing and assessing the  documentation and oversight of qualification and funding decisions. While  we eventually were able to obtain documentation showing the basis and  rationale for EOWS decision making, the incomplete official recognition  and official grant files made the review difficult and time-consuming.
In responding to a draft of this report, Justice stated that electronic  documents maintained in its grant management system (GMS) are  considered a part of the official recognition and official grant files and are  readily available to EOWS, and that they should have been readily  available to, and easily accessible by, GAO.
section_title: EOWS Has Taken Steps to Improve Program-Monitoring Documentation, but Challenges Remain
paragraphs: In response to our 1999 recommendation, EOWS has taken steps to  improve program monitoring, as with the documentation of qualification  and funding decisions, but challenges remain in making the monitoring  documentation readily available. While OJP requires progress reports and  site visit reports to be included in the official grant files, our review in   2003 of 10 official grant files showed that some of EOWS’s files were  missing full documentation. However, before our review ended, EOWS  provided further documentation that fully documented progress reports  and site visits, which it acknowledges was not in the official grant files,  and thus not readily available. This lack of readily available documentation  is not in keeping with OJP’s policies and procedures. Not having complete  and readily available official grant files may delay and complicate EOWS  officials’ and external reviewers’ assessment of whether EOWS’s  monitoring requirements are being followed.
subsections: section_title: Official Grant Files Were Incomplete
paragraphs: Despite EOWS’s efforts, some of the official grant files we reviewed were  incomplete at the time of our file review. We randomly selected 10 files  from the 31 sites that were first funded in fiscal year 2002 to determine  whether the sites had submitted the required progress reports from  January 1, 2003, through June 30, 2003. Of the 10 files we reviewed, 6 did  not contain any of the required progress reports for the period. The  remaining 4 files included the required progress reports. OJP’s policies and  procedures require EOWS to ensure that progress reports are included in  the official grant files. However, when we asked EOWS about the missing  progress reports, EOWS provided us with the requested documents. We  asked, but EOWS did not specify where it found the missing  documentation.
In addition, EOWS’s grant monitors are required to conduct site visits a  minimum of every 18 months and document their visits. This  documentation is meant to convey to EOWS management officials how  well sites are complying with grant requirements and is to be used by  EOWS in making funding decisions. Grant monitors are to prepare a report  of the visits and forward a copy to the Office of the Comptroller for  inclusion in the official grant file. We reviewed the same 10 grant files  discussed above to determine whether site visits were fully documented.  We found that a site visit report was not yet due for 7 of the 10 files. In the  remaining 3 files where a site visit was due, none of the 3 files contained  documentation that the site visit had been conducted. We requested to  speak to the responsible grant monitors to determine whether site visits  had been conducted. EOWS officials told us that the monitors were no  longer involved with the program but provided additional documentation  for the 3 files we reviewed. The documentation showed that of the 3 sites,  only 1 visit had been conducted; the other 2 were scheduled but had been  delayed. While we only reviewed 3 files where a site visit was due, the lack  of documentation in all 3 files, as well as the fact that only 1 of 3 site visits  had been conducted, added to our concerns about the completeness of  official grant files for program oversight.
section_title: Challenges Persist in Making Monitoring Documentation Readily Available
paragraphs: As we have previously mentioned, OJP’s policies and procedures manual  and the Comptroller General’s guidance on internal controls require that  documentation should be kept readily available for examination.  Subsequent to our finding missing progress reports and site visit reports,  EOWS provided us with additional documentation that demonstrated that  the progress reports and one of the site visit reports had been completed.  When asked about where they found the missing progress reports and site  visit reports, EOWS officials did not specify where they found the missing  documentation.
section_title: Despite Some Steps, Challenges Remain in Developing Criteria for Self-Sustainability
paragraphs: EOWS has established a rule and set some activities for sites to complete  to encourage them to become self-sustaining. However, it has still not fully  developed criteria to determine when sites have become self-sustaining  and when to reduce or withdraw Weed and Seed grant funds, as we  recommended in 1999. To become self-sustaining, sites must leverage  additional resources from sources other than EOWS to sustain the Weed  and Seed program without EOWS funds. We found that EOWS has not  reduced or withdrawn funds from any Weed and Seed sites for reasons  related to becoming self-sustaining. This is important because without  overall program funding increases, new sites cannot be funded unless  funding can be reduced or discontinued from sites that have achieved self- sustainability.
subsections: section_title: EOWS Encourages Self- Sustainability, yet Needs Criteria to Make Funding Decisions
paragraphs: In 1999, EOWS established a rule to encourage self-sustainability that was  generally intended to limit Weed and Seed funding to a site to 5 calendar  years and require grantees to shift the majority of EOWS funds to a  different site after 5 years. According to EOWS officials, EOWS also  encouraged sites to become self-sustaining by establishing activities to be  completed during each year of the 5-year grant. Program monitors are to  assess sites’ completion of these activities by recording results on a  checklist.
We found that while EOWS undertook efforts to encourage self- sustainability, it has not fully developed criteria to make funding decisions  based on whether sites had achieved self-sustainability. In fiscal year 2002,  73 sites, which had completed 5 years of EOWS funding, reapplied for  funding. EOWS approved an additional round of 5-year funding to 67 of  these sites. EOWS denied funding to 6 sites, but not for reasons related to  self-sustainability. At the time of our review, no site’s funding had been  reduced or withdrawn because the site had achieved self-sustainability,  even though EOWS has funded some sites since the early 1990s. In relation  to the activities EOWS set to encourage sites to achieve self-sustainability,  because these activities were established in 2003, we could not assess the  sites’ progress toward completing them. As we reported in 1999, without  criteria, EOWS does not have a basis for determining when sites are self- sustaining and when to reduce or withdraw Weed and Seed funds.
EOWS officials told us that OJP is currently developing criteria for self- sustainability for EOWS and other OJP programs and that a report  detailing the criteria may be completed in 2004. According to Justice,  EOWS developed a new criterion for self-sustainability in the FY2003  Competitive Application Kit regarding whether or not a full-time  coordinator is funded by a reallocation of resources other than the Weed  and Seed grant.
For information about proposed legislation that could affect self- sustainability for Weed and Seed sites, similar to other Justice programs,  see appendix III.
section_title: Performance Measures Generally Did Not Track Program Outcomes
paragraphs: In 1999, we reported that while EOWS had developed various  performance measures in an attempt to respond to the Government  Performance and Results Act (GPRA) of 1993, EOWS’s measures  generally did not track program outcomes. GPRA seeks to shift the focus  of federal management and decision making away from activities  performed to outcomes, or the results of activities undertaken. Since our  last report, EOWS has continued to refine and develop its performance  measures, but we found that EOWS still generally collects site activity data  rather than measuring outcomes. EOWS also generally does not use  intermediate measures, which represent conditions believed to precede or  contribute to achieving the ultimate outcomes that may be considered  constructive steps toward measuring outcomes. While assessing success  using outcomes and/or intermediate measures can be difficult, doing so is  important to program management and to policy makers for  understanding whether the program is achieving the intended results, and  for identifying opportunities for improvement.
subsections: section_title: EOWS Generally Collects Activity Data Rather Than Outcome Performance Measures
paragraphs: Our review showed that EOWS generally collects activity data, but has not  developed outcome or intermediate measures that enable EOWS  management to track the success of the Weed and Seed program in  meeting its goals. As established in prior work by GAO, outcome  measures help officials track the success of their programs in meeting  program goals. EOWS mostly collects data about site activities, such as  whether sites have foot patrols, safe havens, and provide job training. An  intermediate measure, for example in relation to job training programs,  might be the extent to which those who attend job training obtain  employment. This intermediate measure rests on the assumption that  individuals who are employed are less likely to commit crimes. See  appendix V for activity data collected by EOWS.
EOWS officials told us that to measure its success in reducing violent  crime, they collect data on the number of homicides and consider  homicides the significant indicator for measuring performance outcomes.  However, using the number of homicides as an outcome measure  indicative of program success is problematic for several reasons. First,  because homicides are relatively rare, even in high crime areas, homicide  trends may be too unstable (fluctuate too much from year to year) to  assess the success of the Weed and Seed program. Second, outcome  measures, such as homicides, can present some methodological challenges  because it is difficult to draw a direct causal link between the homicides  and a program’s work. Economic trends and other law enforcement  initiatives could also be responsible for the observed outcomes. If  homicides are used as a performance measure, any analysis should  attempt to control for other factors influencing the outcomes. Third, given  the broad nature of this program goal -- to reduce violent crime, measuring  outcomes in relation to only one type of violent crime also seems  problematic because it is too narrowly focused.  According to EOWS  officials, homicide data was selected because of its reliability, and because  gang-related homicides are a significant indicator of the success of a Weed  and Seed strategy.
In February 2004,the Office of Management and Budget (OMB) issued a  detailed assessment of the Weed and Seed program. According to OMB,  Weed and Seed has not demonstrated results. OMB’s report states that  Weed and Seed lacks clear targets or timelines for reducing violent crime.  Additionally, the report states that Weed and Seed has difficulty collecting  data and developing aggregate measures for assessing program  performance.
section_title: GAO Has Previously Reported on How Other Federal Programs Have Developed Effective Performance Measures, Despite Challenges
paragraphs: Although the Weed and Seed program faces many challenges in developing  outcome measures for its various activities, we have previously reported  that other federal programs have developed effective performance  measures even under difficult circumstances. These reports identified a  variety of strategies that other federal programs have used to develop  performance measures. For example, we reported that some federal  programs utilized a mix of outcome and intermediate measures. This  combination of measures allowed them to minimize the risk of not  showing outcomes because of their limited control over external factors  such as economic trends, which may prevent programs from achieving  intended outcomes. Intermediate measures are also helpful to show  progress when it is expected to take many years before the desired  outcome is likely to be achieved. In such instances, progress toward  program outcomes may be demonstrated through intermediate outcomes.
We also reported that where measuring outcomes is, after careful  consideration, deemed infeasible, intermediate measures can be used to  track progress toward outcomes for programs such as Weed and Seed.
In addition, we have reported other ways  federal agencies have used  intermediate outcomes. For example, the National Highway Traffic Safety  Administration uses the rate of safety belt use as an intermediate measure  of its goal to reduce motor vehicle crashes and the resulting fatalities and  injuries. As we reported, an increase in the use of safety belts is  considered an intermediate outcome--not an end outcome--because it is  desirable not in itself but because it has been demonstrated to contribute  to the ultimate goal--reducing highway-related fatalities and injuries.
section_title: Additional Performance Measures Are Being Developed for EOWS
paragraphs: The Justice Research and Statistics Association and the Urban Institute  are currently developing additional outcome performance measures under  contracts awarded by EOWS in 2002 and 2003. According to EOWS  officials, these research organizations have the expertise needed to  develop outcome performance measures for the Weed and Seed program.  We interviewed researchers from these organizations and spoke to EOWS  officials about their research plans. However, none of the studies on  developing additional performance measures had been completed at the  time of our review, and it is too soon to tell whether the studies will  produce outcome measures needed to adequately assess the Weed and  Seed program. The Justice Research Statistics Association and other  researchers were conducting studies on crime data and evaluation during  our review; one of those studies was submitted to EOWS in November  2003. See appendix VI for a description of EOWS’s ongoing and completed  studies.
section_title: Conclusions
paragraphs: We believe that 5 years has been ample time for OJP and EOWS to fully  implement the recommendations we made in our 1999 report. EOWS has  developed and partially implemented policies requiring the completion of  documentation to support major decisions, such as qualification and  funding decisions, and recorded monitoring information. But by failing to  ensure that these documents are appropriately maintained in official  recognition and grant files, the documents are not readily available to  meet their purpose. That is, the intent of this type of internal control is to  ensure that both management and external reviewers such as auditors can  adequately perform their responsibilities, for example, reviewing work and  making management decisions based on complete and accurate  information. While EOWS was eventually able to produce the  documentation we requested, failure to appropriately file the  documentation made the process of using it very inefficient and, in effect,  may have defeated the purpose of having it completed.
While EOWS may have moved forward in addressing its self-sustainability  goals by recently developing an activity checklist for grantees to use to  document actions that are intended to achieve self-sustainability, the  activities do not constitute criteria for determining when sites should be  considered self-sustaining and consequently have federal funds reduced or  discontinued. One of the benefits of such criteria is to enable EOWS to  determine when current projects are likely to be able to self-sustain so that  available funding can be used to help reduce crime and achieve other  benefits in other deserving communities. Because no sites have had their  funding withdrawn because they were deemed self-sustaining during the  13-year life of this program, EOWS may be foregoing the opportunity to  use the funds in another location where the need for federal funding is  greater or EOWS may need less overall funding. Without criteria to  determine when federal funds are no longer needed, EOWS also runs the  risk of providing funds beyond what is needed to sustain some sites.
EOWS also needs performance measures that focus on program outcomes,  and/or achievement of intermediate goals, so that it and those that provide  oversight, such as the Congress and OMB, will be able to adequately assess  the extent to which the program is achieving its goals. Management  depends on this type of assessment to make the strategic and operations  decisions needed to achieve the program’s missions and goals. Congress  and oversight agencies need this type of assessment so they can make  funding decisions and help ensure that EOWS is in the best position and  has the best tools to accomplish its mission. While EOWS recognizes the  need for outcome performance measures and has funded studies to help  develop them, it is unclear at this time whether these studies will be able  to suggest the needed measures.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Attorney General of the United States require the  Assistant Attorney General for OJP to ensure that the Executive Office for  Weed and Seed fully implement the intent of our previous  recommendations by taking the following four steps:    maintain the documentation of the basis and rationale for qualification  and funding decisions in appropriate grant files;   retain progress reports and site visit reports in official grant files;     clearly define criteria to assess when sites are self-sustaining and apply  the criteria to sites when making further funding decisions; and     develop outcome performance measures--or, where measuring  outcome is, after careful consideration, deemed infeasible,  intermediate measures--that can be used to adequately track progress  toward program outcomes of the Weed and Seed program.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Attorney General of the United  States for review and comment. In a March 11, 2004, letter, the Assistant  Attorney General (AAG) for OJP commented on the draft.  Her written  comments are summarized below and presented in their entirety in  appendix VII. Based on Justice’s comments, we modified the text of our  report where appropriate and also incorporated Justice’s technical  comments into this report where appropriate.
In the AAG’s response, the Justice Department generally agreed with our  recommendations on maintaining the documentation of the basis and  rationale for qualification and funding decisions in appropriate grant files  and retaining progress reports and site visit reports in official grant files.  Justice agreed in part with our recommendation on self-sustainability  criteria and fully agreed with our recommendation on outcome  performance measures, and offered additional information to show it had  moved forward in responding to our past recommendation.
Justice agreed that controls should be strengthened to ensure that the  basis and rationale for qualification and funding decisions are documented  in the centrally maintained official recognition (OR) files. In Justice’s  comments, the AAG explained that electronic documents maintained in its  grant management system (GMS) are considered a part of the official  recognition file and are readily available to EOWS, and that they should  have been, but were not, readily available to, and easily accessible by,  GAO. As of the current fiscal year, OJP will not only post all solicitations  and receive all grant applications via GMS, but also receive and maintain  all grant progress reports on GMS. In this way, there will be no  opportunity to misplace official grant documents. Justice stated that OJP’s  Office of the Comptroller will implement procedures to ensure that  external reviewers have ready access to GMS information.
Justice agreed with our second recommendation on retaining progress  reports and site visit reports in official grant files. Justice noted that EOWS  recently implemented a progress reporting module in GMS that now  enables recipients of grants awarded through GMS to submit semi annual  progress reports electronically. OJP is developing a monitoring module  that will enable program monitors to record and report on site visits  directly in GMS. In the interim, EOWS and the Office of the Comptroller  will take steps to strengthen controls to ensure that copies of completed  site visit reports are included in the official grant file.
Justice agreed in part with our third recommendation on self-sustainability  and noted that EOWS will further define the criteria to assess when sites  are self-sustaining and apply the criteria when making funding decisions.  Further, Justice said it is promoting a “graduation” process to bring sites  to sustainability and cease providing Weed and Seed funding to those  sites, thus making funding available to assist newer developing sites.    However, Justice does not agree that EOWS has never set criteria for self- sustainability. Justice said that EOWS set a criterion for self-sustainability  in the FY2003 Weed and Seed Competitive Application Kit, which asked if  the application provided for a full-time coordinator funded by reallocation  of existing resources other than the Weed and Seed grant. According to  Justice, applications were ranked against this criterion. According to  EOWS officials, in March 2004, EOWS will announce an additional  criterion specific to measuring self-sustainability. The new criterion will  require grant applicants to identify other funding sources at a level five  times the EOWS contribution. This criterion will be considered when  making funding decisions.
We applaud EOWS’s commitment to further develop criteria to assess  when sites are self-sustaining and apply the criteria to sites when making  further funding decisions.  However, Justice’s comments did not provide  specific information about how the Application Kit criterion has been used  to determine self-sustainability. We added Justice’s belief that this  criterion relates to self-sustainability to the text, but also note that over  the 13-year history of the program, funding has never been reduced or  withdrawn from a site because a site was deemed to be self-sustaining.
Justice agrees that developing outcome performance measures is  important and states that EOWS has been working to develop measures  that track progress toward program outcomes in the Weed and Seed  Program. In a letter commenting on this draft report, Justice provided a  summary of results from JRSA’s study to support EOWS’s use of the  change in homicides as a measure of program success. According to  Justice’s summary, this study concluded that measuring reductions of  homicides is an adequate performance measure of the Weed and Seed  program because of its reliability and explicit selection as a goal by over  30 percent of the Weed and Seed sites participating in the Crime Pattern  Study. In addition, Justice includes a 5 percent reduction in homicides in  Weed and Seed sites as part of its Strategic Plan. With its comments,  Justice included summaries of two recent research reports. One of these,  Analysis of Homicide in Weed and Seed Sites, showed positive results in  homicide trends in Weed and Seed sites when compared to host  jurisdictions. Another report, Crime Pattern Analysis (Three-Top Crime  Study), showed that after the third year of Weed and Seed program  implementation, sites encountered a significant decrease in reported  crimes.
Because the Crime Pattern Analysis study was not completed at the time  of our review, we could not fully assess its methodological rigor and  whether it would result in effective outcome performance measures.   Although we believe that EOWS may be moving in the right direction  regarding developing additional performance measures and we mention  the studies in the report text, we also believe that homicides have  shortcomings as described in the text when used as the only measure of  the program’s performance. In addition, our review of the Analysis of  Homicide in Local Weed and Seed Sites study generated several concerns.  First, the study used changes in the raw volume of homicides, instead of  changes in homicide rates, as its indicator of success or failure of program  sites. This method does not account for changes in the population as  would be accounted for were a homicide rate measured. Second, the study  used the larger host jurisdictions for specific Weed and Seed sites as the  comparison locations to which the Weed and Seed sites are compared. In  doing so, Weed and Seed sites’ homicide data are included with host  jurisdictions’ homicide data, thus making the differences between them  extremely difficult to interpret. It is also worth noting that this study  evaluates Weed and Seed sites only on the basis of decreases in homicides,  while Weed and Seed was intended to decrease crime more generally.
We are sending copies of this report to the Attorney General, appropriate  congressional committees, and other interested parties. We will also make  copies available to others upon request. In addition, the report will be  available at no charge on GAO’s Web site at http://www.gao.gov. If you or  your staff has any questions on this report, please call Weldon McPhail on  (202) 512-8644 or me on (202) 512-8777.
section_title: Appendix I: Scope and Methodology
paragraphs: To determine what actions the Executive Office for Weed and Seed  (EOWS) has taken to develop an adequate internal control to ensure full  documentation of the basis and rationale for qualifying new and existing  sites for funding, we reviewed (1) the criteria used to determine which  new and existing sites should be qualified for funding, and (2) EOWS’s  policies and guidance. To gather this information, we interviewed officials  from Justice and EOWS and reviewed pertinent documents, including  EOWS policies and procedures, official recognition and grant applications,  management oversight and monitoring documentation, and budget  reports. In addition, we randomly selected 10 of 94 fiscal year 2002 official  recognition files submitted to EOWS for review. These 10 files included 5  files from new applicants and 5 files from existing sites that applied in  2002 and were funded in fiscal year 2002. We reviewed 10 of the 14 official  recognition files that EOWS decided not to fund in fiscal year 2002. We  also reviewed 10 randomly selected official grant files from the 31 new and  existing sites that were approved for funding in fiscal year 2002. Because  of limited resources, we did not attempt to review all the files. While the  small sample size prevents us from making reliable generalizations, the   30 files we reviewed represented various types of files that EOWS  maintains. We selected the files from a list that was provided to us by  EOWS. To ensure the list of files was sufficient for the purpose of our  review, we spoke to knowledgeable EOWS officials about the  completeness and accuracy of the list.
To assess what steps EOWS has taken to improve program monitoring to  ensure that sites meet the grant requirement of submitting progress  reports, and that EOWS site visits are documented, we reviewed EOWS  program grant guidance, including training offered, the EOWS monitoring  guidance used by grant monitors when conducting site visits, and the grant  files for the Weed and Seed sites that we visited: Pittsburgh, Pennsylvania;  Charleston, South Carolina; and North Charleston, South Carolina. We  selected these sites from the 221 sites funded by EOWS in fiscal year   2003. Based on our discussions with EOWS officials about the sites, we  selected three sites, which although not representative of all sites, had  received Weed and Seed grant funds since the early 1990s, which enabled  us to learn about their efforts to implement the Weed and Seed strategy  over time. We also reviewed selected monitoring visit reports prepared by  grant monitors for these sites and biannual progress reports submitted in  fiscal years 2002 and 2003. We interviewed EOWS officials and grant  coordinators at these 3 sites regarding procedures used for monitoring  Weed and Seed sites. We also reviewed the same 10 randomly selected  official grant files discussed above from the 31 new and existing sites that  were funded in fiscal year 2002 to determine whether the progress reports  and site visit reports were in the official grant files and readily available.  The sites were funded under EOWS’s competitive application and did not  include sites funded under EOWS’s continuation application. While the   10 files we reviewed represented various types of files that EOWS  maintains, the small sample size prevents us from making reliable  generalizations about all official grant files. To ensure the list of files was  sufficient for the purpose of our review, we spoke to a knowledgeable  official about the completeness and accuracy of the list.
To assess what criteria, if any, EOWS has developed when sites are self- sustaining and when to reduce or withdraw program funding, we asked  EOWS officials for any relevant information. Following repeated requests,  EOWS officials did not provide any documentation on criteria that they  had developed. They did, however, provide information on the five-year  rule for self-sustainability and the sites’ activities, which we reviewed.
To determine what additional performance measures EOWS has  developed that track program outcomes, we interviewed officials from  EOWS and the sites we visited. We reviewed pertinent documents,  including EOWS policies and procedures, grant applications, and data  collected pursuant to the Government Performance and Results Act of  1993 (GPRA). In relation to the studies EOWS commissioned to develop  program-wide performance measures, we interviewed the authors of the  studies under way to discuss their research plans. At the time of our  review, none of the studies on developing additional performance  measures had been completed.
We conducted our audit work between June and December 2003 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: GAO Site Visit Summary
subsections: section_title: Pittsburgh, Pa.
paragraphs: Pittsburgh established its first Weed and Seed site, the Hill District, in  1992. In 1995, Hazelwood became the second Pittsburgh site. In 1997 and  1999, East Liberty and Homewood became the third and fourth Weed and  Seed sites in Pittsburgh. Pittsburgh currently receives funds for three sites:  East Liberty, Hazelwood, and Homewood. The sites are located between  the Allegheny and Monongahela rivers on the eastern side of the city. The  population of the target area is 10,897 in East Liberty, 15,849 in  Hazelwood, and 9,283 in Homewood. In fiscal year 2003, the Pittsburgh  sites in total received $435,000 in Weed and Seed funding.  See table 2 for  the Weed and Seed funding history for Pittsburgh’s three sites.
Pittsburgh’s weeding activities include funding a drug violence initiative  headed by the Federal Bureau of Investigation (FBI) and a community- based crime prevention activity coordinated with the police. Pittsburgh’s  seeding activities include funding a drug education youth camp that  provides children with leadership training and fitness tests, and a  downtown revitalization program. See table 3 for examples of the types of  activities funded by the Pittsburgh Weed and Seed program.
An important goal of the Weed and Seed program is for sites to leverage  resources from sources other than EOWS in order to become self- sustaining. During our site visit and through documentation obtained from  Pittsburgh Weed and Seed program staff, we identified several  partnerships established by the Pittsburgh Weed and Seed program to  leverage resources. These cooperative arrangements involved partners  such as the University of Pittsburgh Medical Center and the Pittsburgh  Harlequin Rugby Association. Table 4 provides specific examples of  leveraging efforts that we identified. Pittsburgh Weed and Seed program  staff estimate that they coordinated and leveraged approximately   $1.1 million in 2002.
section_title: Charleston, S.C.
paragraphs: Charleston became a Weed and Seed site in the spring of 1992. The  Charleston Weed and Seed target area includes approximately 8 square  miles on the Charleston Peninsula and in the city of Charleston. The entire  Weed and Seed site is located in Charleston’s Renewal Community. The  population of the target area, encompassing 13 neighborhoods, is roughly  17,000. In fiscal year 2003, the site received $275,000 in Weed and Seed  funding. See table 2 for Charleston’s Weed and Seed funding history.
Charleston’s weeding activities include supporting a youth court with a  Weed and Seed juvenile investigator and funding community policing foot  patrols. Charleston’s seeding activities include supporting a police-youth  after-school boxing program, and boarding up abandoned and vacant  buildings to improve the appearance of the area. See table 3 for examples  of the types of activities funded by the Charleston Weed and Seed  program.
An important goal of the Weed and Seed program is for sites to leverage  resources from sources other than EOWS in order to become self- sustaining. During our site visit and through documentation obtained from  Charleston, we identified several partnerships established by the  Charleston Weed and Seed program to leverage additional resources.  These cooperative arrangements involved partners such as the FBI and the  College of Charleston’s School of Education. Table 4 provides specific  examples of leveraging efforts that we identified. Charleston Weed and  Seed program staff estimate that they were able to coordinate and  leverage approximately $1 million in 2002.
section_title: North Charleston, S.C.
paragraphs: North Charleston was originally funded as a Weed and Seed site in August  1993. The city of North Charleston is located about 7 miles north of the  city of Charleston. Today, North Charleston has four target  neighborhoods: Union Heights, Chicora/Cherokee, Accabee, and Liberty  Hill. The population of the target area is 13,606. In fiscal year 2003, the site  received $225,000 in Weed and Seed funding. See table 2 for North  Charleston’s Weed and Seed funding history.
North Charleston’s weeding activities include participating in Operation  Cease Fire with the U.S. Attorney’s Office, to reduce gun crime in target  neighborhoods, and partnering with local law enforcement to improve the  bike patrol program. North Charleston’s seeding activities include  developing the Drug Education for Youth program and supporting an  annual neighborhood cleanup. See table 3 for examples of the types of  activities funded by the North Charleston Weed and Seed program.
An important goal of the Weed and Seed program is for sites to leverage  resources from sources other than EOWS in order to become self- sustaining. During our site visit and through documentation obtained from  North Charleston, we identified several partnerships established by the  North Charleston Weed and Seed program to leverage resources. These  cooperative arrangements involved partners such as the United States Air  Force and the Medical University of South Carolina. Table 4 provides  specific examples of leveraging efforts that we identified. North  Charleston Weed and Seed program staff estimate that they coordinated  and leveraged approximately $3.3 million in 2002.
section_title: Appendix III: Proposed Legislation Could Affect Self-Sustainability
paragraphs: Proposed legislation could affect self-sustainability for Weed and Seed  sites by mandating that EOWS impose more rigorous requirements on its  grantees, similar to some other Justice programs. Unlike some other  Justice discretionary grant programs, the law establishing the Weed and  Seed program currently does not have a matching funding requirement,  nor does it limit the maximum number of years a grantee can receive  funding. H.R. 3036, a bill to reauthorize the Department of Justice for  fiscal years 2004 through 2006, was introduced on September 9, 2003. This  bill includes a provision that would limit the federal contribution to   75 percent of the total Weed and Seed program costs. The bill would also  impose limitations on the duration of grants funded under the Weed and  Seed program, limiting to 10 the total number of fiscal years a grantee may  receive grants. Additionally, the bill requires that in order for a grantee to  be eligible for a grant, it must agree to formulate a timely and effective  plan to independently sustain the Weed and Seed program when federal  funding ends.
Currently there are other federal grant programs that require a funding  match and a limit on the duration of grant awards. For example, the Drug- Free Communities Support Program (DFCSP) and Community Oriented  Policing Services (COPS) both have matching requirements and clearly  specify the number of years a grantee may be funded. DFCSP is a  discretionary grant program with a budget of about $60 million in fiscal  year 2003. DFCSP requires a dollar-for-dollar match, limits grantees to no  more than 5 years of funding, and reduces awards by 25 percent in the  fourth and fifth years. The COPS program, with a budget of about   $929 million in fiscal year 2003, generally requires grantees to contribute  25 percent of the costs of the program. The federal share of a COPS grant  that covers more than 1 year must decline year to year.
section_title: Appendix IV: Executive Office for Weed and Seed Selected Activities
subsections: section_title: The steering committee continually educates, guides, and reevaluates site management.
paragraphs: Steering committee meets at least  monthly.
Steering committee  and organizational  structure continue to  function. Steering  committee meets at  least quarterly.
The grantee and/or local U.S.  Attorney’s Office evaluates the site  coordinator’s job performance,  with input from the steering  committee.
The site coordinator and the  steering committee work together  to ensure timely expenditure of  grant funds.
The site’s strategy to achieve  measurable outcomes in the four  required program areas (especially  law enforcement/community  policing) is on track. The site  continues to collect data related to  goals and objectives.
The site continues to  implement strategy  for achieving goals  and objectives in all  four required program  elements.
The site makes plans to  continue community  development efforts and  strategic planning  through various funding  sources.
The site continues program  publicity and outreach to recruit  neighborhood leaders and  mobilize community.
The site continues  program publicity and  outreach efforts to  recruit neighborhood  leaders and to  mobilize the  community while  identifying and  building sustainable  leadership among  community members.
The site continues  program publicity and  outreach efforts to  mobilize community  and recruit new  community leaders.
The site’s community  leaders are working to  sustain efforts and  continue community  mobilization.
The site continues to coordinate its  efforts with local initiatives,  particularly programs with local  parks and recreation departments.
The Weed and Seed  effort begins to be  included in  city/community plans.
Community leaders  have established  working relationships  with local officials.
The Weed and Seed  community is included in  city/community planning.
The steering committee develops  and implements the special  emphasis program based on  community needs.
The special emphasis  program is fully  operational.
The site submits a final  progress report.
The site communicates regularly  with all partners, including U.S.  Attorney’s Office, Executive Office  for Weed and Seed, and  community members, using  newsletters, faxes, neighborhood  visits, e-mail, community meetings,  and so forth.
The steering  committee and site  coordinator review  previous years’ grant  applications to  determine plans for  final year of official  recognition.
The steering committee  determines if there is a  need to continue the  program in other  neighborhoods based on  the program.
The site establishes a  subcommittee to look  for additional funding  or in-kind resources  to sustain the  program.
Subcommittee  continues to search for  additional resources to  sustain the program.
The site begins to develop plans  for evaluating its programs.
The site identifies an  evaluation tool and  begins developing an  evaluation based on  Executive Office for  Weed and Seed site  evaluation literature.
The site undergoes  program evaluation to  assess overall  effectiveness of official  recognition strategy  implementation.
By the end of the year,  the site releases its full  evaluation report,  assessing the overall  program  accomplishments and its  effectiveness.
A site is a geographically defined area ranging in size from several neighborhood blocks to several  square miles. With the input of the U.S. Attorney’s Office and the steering committee, each site  develops and implements its own Weed and Seed strategy.
Official recognition is a designation given to a grantee by EOWS that signifies that EOWS has  approved the grantee’s Weed and Seed strategy in a specific site. A grantee must receive official  recognition from EOWS before a grantee can apply for funding. Official recognition requires, among  other things, that the site develop a management plan and a comprehensive strategy for  implementing the Weed and Seed strategy.
The four required program elements that sites are required to address are (1) law enforcement;   (2) community policing; (3) crime and substance prevention, intervention, and treatment; and   (4) neighborhood restoration.
section_title: Appendix V: Activity Data Collected by EOWS in Fiscal Year 2003
section_title: Appendix VI: Description of EOWS’s Ongoing and Completed Studies
subsections: section_title: Compare homicides in Weed and Seed sites with homicides in the jurisdictions that sites are located in, to determine whether homicides will decline faster in Weed and Seed sites than in the remainder of the jurisdiction.
paragraphs: Develop additional performance indicators that will be used by EOWS and Weed and  Seed sites to evaluate program performance with respect to crime control.
Summarize the types of crime selected as “target” crimes by Weed and Seed sites  (sites choose three crimes) to determine crime reduction strategies. Assess the  degree of success of Weed and Seed sites by analyzing the percentage of reduction  in crime compared with the percentage in the rest of the jurisdiction that they are  located in.
Summarize the completed evaluations of individual Weed and Seed sites. The  analysis will include more than 80 studies. JRSA’s preliminary analysis (March 3,  2003) includes 36 evaluations. Of the 36 evaluations, JRSA determined that 5 were  process, 19 were impact, and 9 were both process and impact evaluations. The  remaining 3 evaluations were not identified as process or impact.
Conduct local evaluations of Weed and Seed sites. In fiscal year 2002, EOWS funded  13 local evaluations. Each site contracts with its own researcher.
section_title: Appendix VII: Comments from the Department of Justice
section_title: Appendix VIII: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to those named above, the following individuals made key  contributions to this report: Lisa G. Shibata, Kirstin B. L. Nelson, Cady L.  Summers, David Alexander, Sidney Schwartz, Kevin L. Jackson, Elizabeth  Curda, Tom James, Denise Fantone, Jan Montgomery, Ann H. Finley,  Katherine M. Davis, and Leo Barbour.
section_title: Related GAO Products
paragraphs: Law Enforcement: Better Performance Measures Needed to Assess  Results of Justice’s Office of Science and Technology, GAO-04-198.  Washington, D.C.: November 14, 2003.
Standards for Internal Control in the Federal Government.   GAO/AIMD-00-21.3.1. Washington, D.C.: November 1999.
Federal Grants: More Can Be Done to Improve Weed and Seed Program  Management. GAO/GGD-99-110. Washington, D.C.: July 16, 1999.
Small Business Administration: Enhancements Needed for Loan  Monitoring System Benchmark Study. GAO/AIMD-99-165. Washington,  D.C.: May 14, 1999.
Managing for Results: Measuring Program Results That Are Under  Limited Federal Control. GAO/GGD-99-16. Washington, D.C.: December  11, 1998.
Managing for Results: Analytic Challenges in Measuring Performance.  GAO/HEHS/GGD-97-138. Washington, D.C.: May 30, 1997.
Federal Grants: Design Improvements Could Help Federal Resources Go  Further. GAO/AIMD-97-7. Washington, D.C.: December 18, 1996.
Highway Safety: Safety Belts Use Laws Save Lives and Reduce Costs to  Society. GAO/RCED-92-106. Washington, D.C.: May 1992.