Justice Discretionary Grants: Byrne Program and Violence Against Women Office Grant Monitoring Should Be Better Documented
Released: 2001-11-28T00:00:00

Letter

Background

OJP, the grant-making arm of DOJ, provides grants to various organizations, including state and local governments, universities, and private foundations, that are intended to develop the nation’s capacity to prevent and control crime, administer justice, and assist crime victims. OJP’s Assistant Attorney General is responsible for overall management and oversight of OJP through setting policy and for ensuring that OJP policies and programs reflect the priorities of the President, the Attorney General, and the Congress. The Assistant Attorney General promotes coordination among the various bureaus and offices within OJP, including BJA, one of the five bureaus within OJP, and VAWO, one of OJP’s seven program offices. In fulfilling its mission, BJA provides grants for site- based programs and for training and technical assistance to combat violent and drug-related crime and help improve the criminal justice system. VAWO administers grants to help prevent, detect, and stop violence against women, including domestic violence, sexual assault, and stalking.

Since 1996, OJP’s budget has increased substantially, following the passage of the Violent Crime Control and Law Enforcement Act of 1994.Figure 1 shows changes to OJP’s budget from fiscal year 1990 through fiscal year 2000 and compares those changes in relation to BJA’s budget over the same period and VAWO’s budget since its inception in 1995.

One of BJA’s major grant programs is the Edward Byrne Memorial State and Local Law Enforcement Assistance Program. Under the Byrne Discretionary Grants Program, BJA provides federal financial assistance to grantees for educational and training programs for criminal justice personnel; technical assistance to state and local units of government; and projects that are replicable in more than one jurisdiction nationwide. In fiscal year 2000, BJA awarded 99 Byrne discretionary grants worth about $69 million.

VAWO was created in 1995 to carry out certain programs created under the Violence Against Women Act of 1994. The Victims of Trafficking and Violence Prevention Act of 2000 reauthorized most of the exisiting VAWO programs and added new programs as well. VAWO’s mission is to lead the national effort to end violence against women, including domestic violence, sexual assault, and stalking. VAWO programs seek to improve criminal justice system responses to these crimes by providing support for law enforcement, prosecution, courts, and victim advocacy programs across the country. In addition, programs are to enhance direct services for victims, including victim advocacy, emergency shelter, and legal services. VAWO also addresses violence against women issues internationally, including working to prevent trafficking in persons. In fiscal year 2000, VAWO awarded 425 discretionary grants worth about $125 million.

Appendix I discusses the growth in OJP, BJA, and VAWO budgets and provides information on the number and amount of BJA and VAWO discretionary grants awarded from fiscal year 1990 to fiscal year 2000.

Scope and Methodology

To meet our objectives, we conducted our work at OJP, BJA, and VAWO headquarters in Washington, D.C. We reviewed applicable laws and regulations and OJP, BJA, and VAWO policies and procedures for awarding and managing grants, and we interviewed responsible OJP, BJA, and VAWO officials, including grant managers. As agreed with your offices, we focused on monitoring activities associated with the Byrne and VAWO discretionary grant programs. In particular, we focused on grant monitoring for grants that were active during fiscal years 1999 and 2000 and supported a program or theme, rather than technical assistance or training efforts.

To address our first objective, concerning OJP’s process and requirements for discretionary grant monitoring, we reviewed applicable laws and regulations and OJP policies and procedures for grant administration and grant monitoring. We also interviewed OJP, Comptroller, BJA, and VAWO staff. We obtained information about the Comptroller’s Control Desk, which maintains the official grant files and is responsible for receiving, distributing, and tracking grant documents, including financial and progress reports.

To address our second objective regarding the extent to which BJA and VAWO documented their monitoring activities for discretionary grants, we reviewed representative samples of official grant files and grant manager files using a data collection instrument to record whether evidence of the required monitoring activity—progress reports, financial reports, and other required documents—were included in the files. For each grant, we also reviewed the most recent award covering 12 months to examine the documentation of specific monitoring requirements and activities, such as telephone calls and site visits. Specifically, we reviewed a random sample of 46 of 110 Byrne and 84 of 635 VAWO discretionary grants that had a program theme and were active throughout fiscal year 1999 or 2000. The results of the samples are representative of the populations from which they were drawn. We express our confidence in the precision of our sample results as a 95-percent confidence interval. Unless otherwise noted, all confidence intervals are less than or equal to plus or minus 10 percentage points.

In regard to grantee financial and progress reports, we reviewed all available reports for the BJA and VAWO discretionary grants included in our sample from the initiation of the grant through December 31, 2000. We determined timeliness using dates recorded on reports compared with the specified intervals when they were supposed to be received. Also, to determine whether BJA and VAWO closeout procedures were implemented in accordance with OJP policy, we reviewed those grant files in our sample when the grant end date was between September 30, 1999, and August 31, 2000. For our review, we focused on required closeout documentation, such as precloseout contacts, closeout checklists, and final financial and progress reports.

To address our third objective, regarding how BJA and VAWO determine compliance with OJP monitoring requirements, we requested information on any existing oversight and review processes relating to grant monitoring at BJA and VAWO and gathered and reviewed documentation that BJA and VAWO officials provided concerning their oversight processes. We also met with BJA and VAWO officials to obtain information on any new initiatives they had to address the oversight and management of their grants.

To meet our fourth objective, on OJP’s efforts to identify and address grant management problems, we met with OJP officials to discuss their efforts, and we reviewed reports and documents they had prepared about the new Grant Management System (GMS), revisions to the OJP Handbook and associated decision documents, and other initiatives. In addition, we reviewed the DOJ Fiscal Year 2000 Performance Report and Fiscal Year 2002 Performance Plan and information on grant management developed by the DOJ’s Office of the Inspector General.

Finally, to obtain information on the size and growth of BJA and VAWO grant programs within the context of OJP, we obtained and analyzed budget and resource data from OJP on grant funds and programs from fiscal years 1990 through 2000.

We conducted our work between October 2000 and October 2001 in accordance with generally accepted government auditing standards.

We requested comments on a draft of this report from DOJ on October 24, 2001. Its comments are discussed near the end of this letter and are reprinted as appendix III. DOJ also provided technical comments that were incorporated in the report.

Discretionary Grant Monitoring and Requirements

After BJA and VAWO award discretionary grants, OJP policies require them, in coordination with Office of the Comptroller, to monitor grants and related activities and document the monitoring results. Monitoring is done to ensure compliance with relevant statutes, regulations, policies, and guidelines; responsible oversight of awarded funds; implementation of approved programs, goals, objectives, tasks, products, time lines, and schedules; identification of issues and problems that may impede grant implementation; and implementation of adjustments by the grantee as approved by BJA or VAWO.

The OJP Handbook is the basic reference for OJP policies and procedures for the administration of OJP grants, including discretionary grant monitoring. According to the Handbook, each grant manager is to prepare a monitoring plan as part of a grant manager’s memorandum recommending initial or continuation funding. The level of monitoring is to be based on the stated monitoring plan in the grant manager’s memorandum. The plan is to contain information on who will conduct the monitoring, how it will be done, and when and what type of monitoring activities are planned. Monitoring information is to be collected using such techniques as on-site visits, telephone calls, and desk reviews, which are reviews to ensure that the grant files are complete and the grantee is in compliance with the program guidelines.

In addition, grant managers are to review grantee program and financial progress reports. According to OJP’s Handbook, grantees are required to submit periodic progress reports that summarize project activities to aid program and grant managers in carrying out their responsibilities for grant-supported activities. Likewise, grantees are required to submit periodic financial status reports to update OJP on how grant funds are being spent. In addition, OJP requires that grant managers close out the grant when the project period ends to ensure that the agency has received all required financial, programmatic, and audit reports and that all federal funds have been accounted for.

OJP bureaus and program offices, such as BJA and VAWO, are to carry out the program (nonfiscal) monitoring aspects of the grants they award. During fiscal year 2000, BJA had approximately 20 program managers responsible for the monitoring activities for most nonformula BJA grants. Generally, each BJA program manager had responsibility for monitoring 25 to 40 discretionary grants, including those under the Byrne program. VAWO had 13 program managers, each of whom had responsibility for the program monitoring aspects for about 60 VAWO grants, including VAWO discretionary grants.

The Office of the Comptroller has primary responsibility for monitoring the fiscal aspects of all OJP-awarded grants, including those awarded by BJA and VAWO. To assess grantee financial records, the Comptroller’s Monitoring Division is to perform risk-based, on-site financial reviews for a sample of grantee organizations to monitor administrative and financial capability. The Monitoring Division is to review various program and financial documents contained in the official grant files to ensure that they are complete and the documents are properly executed. For discretionary grants, the Monitoring Division conducted 36 desk reviews for Byrne and 38 for VAWO discretionary grants in fiscal year 1999 and 6 desk reviews for Byrne and 16 for VAWO discretionary grants in fiscal year 2000. In addition, the Control Desk is to maintain the official grant files and is responsible for tracking the receipt of all grantee documents. The Control Desk is to receive grantee progress and financial reports, log the date of receipt into a tracking system, file the original in the official grant file, and forward a copy to the cognizant program office. The Control Desk also is to generate a monthly report on delinquent progress and financial reports, which is distributed to responsible officials within bureaus and program offices. BJA and VAWO grant managers are responsible for ensuring that grantees submit timely progress and financial reports and are to contact the grantee if reports are delinquent.

Appendix II discusses grant monitoring within the context of the grant award process throughout the life of a grant.

BJA and VAWO Grant Files Did Not Always Contain Adequate Documentation That Monitoring Occurred

OJP requires that grant managers document their monitoring of the grants. This documentation is to include both the grant manager’s written plan for monitoring the project and grantee progress and financial reports. In addition, OJP requires certain documentation at the time of grant closeouts from both the grantee and the grant manager. Byrne and VAWO grant files did not always contain monitoring plans, and grant managers were not consistently documenting their monitoring activities, according to the monitoring plans that we reviewed. In fact, for those award files representing the most recent 12-month period of activity for each grant, few contained records to show that such activities as telephone contacts and site visits occurred. Furthermore, the progress and financial reports did not always cover the entire period covered by the award, a few grants were missing all progress reports, and progress and financial reports were often late. For those closed grants that we reviewed, key documents, which are to ensure a final accounting of federal funds and show whether the grantee met the programmatic goals of the grant, were sometimes missing.

Missing Monitoring Plans and Inadequately Documented Monitoring Activities

Our review of grant files showed that monitoring plans were not always in the files. Each grant may contain one or more individual grant awards,and for each award, OJP requires that grant managers prepare a monitoring plan containing information on, among other things, who will conduct the monitoring, how it will be done, and when and what type of monitoring activities and reports are planned. Our review of documentation on the awards in 46 Byrne grant files and 84 VAWO grant files showed that an estimated 29 percent of the Byrne awards and about 11 percent of the VAWO awards did not contain a grant manager’s monitoring plan.

We also compared the planned monitoring activities in the monitoring plans for the most recent 12-month period of grant activity for each of the 46 Byrne and 84 VAWO grant files to actual monitoring documentation. Of those files that contained a monitoring plan, some had specific plans for monitoring, while others did not. When the file contained a plan that outlined the type of monitoring and its frequency, our review showed that the documentation in the files was inadequate to demonstrate whether or not BJA and VAWO grant managers were consistently following the monitoring plans. For Byrne files, there were not enough monitoring plans with specific planned activities to ensure that our assessment was representative of all the awards. However, our limited comparisons of planned to actual documentation regarding phone contacts, site visits, and desk reviews revealed the following: Phone Contacts. 13 of 46 Byrne files contained monitoring plans that specified the planned number and frequency (e.g., monthly or quarterly) of telephone contacts to be made. Of those 13 files, none contained documentation to show that all of the planned number of telephone contacts occurred. Furthermore, 4 of the 13 files had documentation that showed that some, but not all, of the planned telephone contacts had been made, while the remaining 9 files contained no documentation that any telephone contacts occurred.

Site Visits. 25 of 46 Byrne files contained monitoring plans that specified the number of grantee site visits to be conducted, but only 4 of 25 files contained documentation that showed that the planned number of site visits occurred. Only 1 of the 25 files had documentation that some, but not all, of the site visits had been made, and the remaining 20 of the 25 files contained no documentation that any site visits occurred.

Desk Reviews. 15 of 46 Byrne files contained monitoring plans that specified the frequency of desk reviews to be conducted. However, none of the files showed evidence that any desk reviews were conducted.

Our assessment of the VAWO files provided enough cases to develop a representative sample of all available VAWO monitoring plans. Specifically, our review of planned to actual monitoring activities—phone contacts, site visits, and desk reviews—showed the following: Phone Contacts. 59 of 84 VAWO files contained monitoring plans that specified the planned number and frequency (e.g., monthly or quarterly) of telephone contacts to be made. However, none contained documentation to show that the planned number of telephone contacts occurred. Furthermore, we estimate that 56 percent of VAWO files had documentation that some, but not all, of the planned telephone contacts had been made, and 44 percent of VAWO files with monitoring plans containing criteria for a specific number of telephone contacts had no documentation that any telephone contacts occurred.

Site Visits. 53 of 84 VAWO files contained monitoring plans that specified the number of grantee site visits to be conducted. We estimate that 10 percent of VAWO files contained documentation that the planned number of site visits occurred. Furthermore, only 2 percent of VAWO files had documentation that even some, but not all, of the planned site visits had been made, while the remaining 88 percent of VAWO files contained no documentation that any site visits occurred.

Desk Reviews. 47 of 84 VAWO files contained monitoring plans that established specific criteria for the frequency of desk reviews. However, none of the VAWO files showed evidence that any desk reviews were conducted.

Progress and Financial Reports Did Not Cover the Entire Grant Period and Were Often Late

OJP requires that grantees file semiannual progress reports and quarterly financial reports throughout the life of a grant. Progress reports are to supply information on the activities and accomplishments of the grantee during the previous reporting period. Financial reports are to show the actual expenditures and unliquidated obligations for the reporting period (calendar quarter) and cumulative for the award. We found Byrne and VAWO grant files in which progress reports and financial reports did not cover the entire period of the grant and a few files with no evidence of progress reports. We also found that progress and financial reports were often late. Combined, these factors resulted in unaccounted periods of time, where OJP had either no information or no up-to-date information about grantee progress or financial activities.

For example, we compared the time periods progress and financial reports were supposed to cover with the time periods they actually covered for each of the Byrne and VAWO grant files we reviewed. Based on our analysis of the grant files, we estimate that 70 percent of the Byrne grantshad periods of time during the grant period not covered by progress reports, and 41 percent of the Byrne grants had periods of time not covered by financial reports. Likewise, we estimate that 66 percent of the VAWO grants had periods of time during the grant period not covered by progress reports and 36 percent of the VAWO grants had periods of time not covered by financial reports. These gaps, cumulative over the life of the grant, ranged from as little as 2 weeks to over 3 years for progress reports and from 1 month to 1½ years for financial reports. With regard to progress reports, we estimate that 30 percent of Byrne grants and 18 percent of VAWO grants had more than 12 months that were not covered. With regard to financial reports, we estimate that only 4 percent of grants at both BJA and VAWO had more than 12 months that were not covered.

Our analysis also showed that a few grant files did not contain any progress reports. For the files we reviewed, seven—three Byrne and four VAWO—contained no progress reports. We noted that OJP awarded one VAWO grantee two supplemental awards over the course of a 3-year period, even though there were no progress reports in the file. In total, these seven grant files without progress reports represent over $2 million in grantee funds awarded over a 5-year period. Finally, we noted examples of progress reports covering more than the required 6-month period and noted that one VAWO progress report that was submitted to cover a 2½- year period was a half-page long. Given OJP’s requirement that progress reports are to be submitted for each 6-month period of the grant, the fact that reports covered periods well beyond the required 6 months raises questions about whether a grant manager has sufficient information to monitor progress and identify any potential grantee problems.

We asked BJA and VAWO officials why these seven grant files did not contain any progress reports. BJA’s Acting Director at the time of our review said that reorganizations in BJA over the last 2 years have contributed to difficulty in ensuring complete and accurate grant files. VAWO officials explained that one of the four grant files involved a grantee that had some grant implementation problems that have since been corrected and said that problems aside, the grantee had been unaware that progress reports were required for periods of grant inactivity. They said that another of the four grantees had been accustomed to the reporting requirements for formula grants, but unaware of different reporting requirements for discretionary grants. According to VAWO officials, this situation has since been corrected and the files have been made current. VAWO officials were unclear about why another grantee had not submitted progress reports. However, they said that the one grantee that had been awarded two supplemental grants is now scheduled for closeout because of the extended period of time the grantee has been delinquent in submitting progress reports. In addition, VAWO officials said that it was unacceptable for a grantee to submit a half-page progress report covering 2½ years.

Our review of grant files also showed that late reports were common.OJP has specific guidance that spells out when progress and financial reports are to be filed by grantees. Progress reports are due 30 days after the close of each previous 6-month period, and financial reports are to be turned in 45 days after the end of the calendar quarter. We compared the dates progress and financial reports were supposed to be filed by grantees with the dates they were actually received by OJP’s Office of the Comptroller. We estimate that 68 percent of Byrne and 85 percent of VAWO progress reports were late. However, for both Byrne and VAWO, we estimate that about 40 percent of all reports were late by only about a month. Table 1 shows our estimates of the timeliness of Byrne and VAWO progress reports.

Similarly, in the case of financial reports, our review showed an estimated 53 percent of Byrne and 54 percent of VAWO financial reports were submitted late. Similar to progress reports, about one-third of all reports were late by 30 days or less. Table 2 shows our estimates of timeliness of Byrne and VAWO financial reports.

OJP’s Office of the Comptroller also found problems similar to those we identified regarding financial reports. As mentioned earlier, the Office of the Comptroller does periodic financial reviews of the official grant file. We examined Comptroller records for 22 grants also covered in our review and found they identified 8 Byrne and 9 VAWO files that were missing some financial reports and 5 Byrne and 8 VAWO files in which financial reports were late. We did not determine what was done to follow up on the late or missing financial reports found during these financial reviews. However, Comptroller procedures call for contacting the grant manager or grantee to request copies of documents that may be missing from the file and to ensure that all documentation related to the financial review is included in the official grant file before the review is closed.

Byrne and VAWO Closeout Documents Missing

Our review also showed that BJA and VAWO grant managers did not always document key closeout activities for those files we examined. OJP requires that grants be closed in a timely manner and considers the process to be one of the most important aspects of grant administration. Closing out grants is the final step in a process by which OJP ensures that all required financial and progress reports and final accounting of federal funds have been received. The timeframe for completion of closeout is no more than 180 days after the end of the grant project. According to OJP guidance, as part of the closeout process, the grant manager is to review the grant file and contact the grantee about the upcoming grant end date and final report submissions. The grant manager is to use the closeout checklist as a means of ensuring that all closeout requirements—the grantee’s submission of final progress and financial reports—are met.

We identified 19 closed Byrne grants and 3 closed VAWO grants that ended between September 30, 1999, and August 31, 2000. There were not enough closed cases in our sample to ensure that our assessment was representative of all grant closeouts. However, our limited review showed that some grant files did not contain required closeout materials. For example, for the Byrne grants: 15 files did not contain documentation of the precloseout contact with the 9 files did not contain closeout checklists; 10 files did not contain the final narrative progress report; and 7 files did not contain the final financial report.

For the VAWO grants: 3 files did not contain documentation of the precloseout contact with the 3 files did not contain closeout checklists; 2 files did not contain the final narrative progress report; and 1 file did not contain the final financial report.

BJA and VAWO Officials Acknowledge Lack of Consistent Compliance With OJP Requirements

BJA and VAWO officials acknowledged that their file maintenance and documentation may not have always been in compliance with OJP monitoring requirements. Officials at both agencies stated that, in some instances, lack of documentation was because of an increased workload among grant managers. In addition, VAWO officials said that since VAWO’s inception, VAWO grant managers have had the responsibility to not only monitor an increasing volume of grants, but also to develop and implement several new grant programs. In commenting on a draft of this report, the Assistant Attorney General stated BJA grant managers have had similar responsibilities. The Assistant Attorney General also commented that one reason for VAWO’s lack of documentation regarding site visits might be a past practice by which all VAWO grant managers memorandums included a standard monitoring plan that was developed in fiscal year 1995, when VAWO was responsible for monitoring one formula and one small discretionary grant program.

Regardless, officials from both BJA and VAWO stated that the monitoring activities may still be taking place, even though they are not documented consistently. When asked how one would know whether a desk review had been done, one BJA official told us that desk reviews were not a specific process, rather they were the type of activities that grant managers did on a day-to-day basis. He added that since BJA had no formalized process for conducting desk reviews, no documentation was required of the grant managers upon completion. Nonetheless, OJP’s guidance for the period covered by our review stated that a desk review form is to be prepared periodically to note, among other things, all contacts, reports, and product reviews. The form is also to include issues, accomplishments, and problems, noting recommended solutions.

BJA’s Acting Director at the time of our review told us that oversight of grants can and has suffered through changes at BJA, including reorganizations within BJA and the increased number of grants and greater workload for BJA grant managers. He said that there is no question that, as mentioned earlier, reorganizations contributed to the difficulty in ensuring complete and accurate grant files and cited transfers of grant files among grant managers as one reason why files were inaccurate or incomplete. He pointed out that BJA has made some changes, including the drafting of new policies and procedures, that are designed to assist grant managers in their grant responsibilities. For example, BJA’s draft procedures called for desk reviews to be performed by grant managers every 6 months or when files were transferred among grant managers. These reviews were to require that each grant manager fill out a checklist—covering such things as the completeness of grant paperwork and the timeliness of progress and financial reports—that upon completion, was to be reviewed by a branch chief. According to the Acting Director at the time of our review, BJA’s guidance, which was undated, had been drafted sometime before July 2000, but had since been subsumed into OJP’s January 2001 update to its grant monitoring procedures. At the time of our review, it was unclear whether BJA had put any of these procedures into practice. In commenting on a draft of this report, the Assistant Attorney General noted that BJA has indicated that it expects to put these procedures into practice in fiscal year 2002.

In those same comments, the Assistant Attorney General also stated that like BJA, VAWO has been working on developing policies and procedures for monitoring grantees more effectively. For example, she stated that a VAWO Monitoring Working Group was formed in spring 2001. The group is working to develop a risk-based assessment tool to develop more realistic monitoring plans; pre-, post-, and on-site protocols for site visits; a standard site visit report form; and a comprehensive training program on monitoring for new employees. Also, according to the Assistant Attorney General, VAWO has developed a desk review checklist that will begin to be used in fiscal year 2002.

OJP requires documentation of grant monitoring activities to provide assurance to OJP grant managers and supervisors that appropriate oversight of grant activities is taking place. It is possible that grant managers are conducting grant monitoring activities even if no documentation exists. However, without documentation, neither OJP, BJA, VAWO, nor we are positioned to tell with any certainty whether such monitoring occurred. The Comptroller General’s internal control standards require that all transactions and other significant events be clearly documented and that the documentation be readily available for examination. Appropriate documentation is an internal control activity to help ensure that management’s directives are carried out. Without such documentation, OJP, BJA, and VAWO have no assurance that grants are meeting their goals and funds are being used properly. In addition, such documentation is essential to systematically address grant performance problems.

BJA and VAWO Are Not Positioned to Systematically Determine Staff Compliance With Monitoring Requirements and Assess Overall Performance

BJA and VAWO are not positioned to systematically determine grant managers’ compliance with monitoring requirements because documentation about monitoring activities is not readily available. BJA officials told us that they do not have a management information system to collect and analyze data that would help them oversee the monitoring process. Instead, they rely on staff meetings and informal discussions with staff to oversee grant monitoring activities and identify potential grantee problems. As discussed earlier, BJA’s Acting Director at the time of our review acknowledged that BJA had experienced some documentation problems and told us that, in addition to drafting the aforementioned BJA guidelines, BJA had begun to modify an officewide management information system to capture data on monitoring activities. According to BJA officials, the enhanced system is to enable grant managers to enter data on such activities as site visits and phone contacts so that they can be tracked. However, at the time of our field work, the system was still in developmental stages. In commenting on a draft of this report, the Assistant Attorney General stated that the monitoring portion of this system was deployed in October 2001.

Like BJA, VAWO does not have an overall management information system to track monitoring activities. VAWO officials said that they hold weekly staff meetings during which they rely on their grant managers to proactively identify and discuss any grant problems or monitoring issues.They added that VAWO has developed a computerized site visit tracking sheet that provides information about the details of grant managers’ on- site visits. A VAWO official said that information reported in site visit reports is shared at staff meetings and is accessible to all staff on VAWO’s internal computer system. VAWO officials also indicated that they are in the process of developing a management information system that will track, in addition to site visits, other monitoring activities such as the submission of progress and financial reports. However, like BJA, VAWO’s system was still in the developmental stages at the time of our review.

BJA and VAWO also do not appear to be routinely using available OJP- wide data on late progress and financial reports that could help them identify potential grantee documentation problems. As mentioned earlier, the Office of the Comptroller has primary responsibility for carrying out the monitoring of the fiscal aspects of grants awarded by OJP bureaus and program offices, and the Control Desk issues monthly reports on whether grantee progress and financial reports are late. These reports are to be forwarded to the administrative officers in the bureaus and program offices. According to OJP’s Chief of Staff at the time of our review, once the monthly report is distributed, bureaus and program offices are responsible for determining what action to take regarding delinquent reports. BJA and VAWO program officials told us that they were aware of the monthly report, but the Acting Director of VAWO’s Program Management Division told us that VAWO probably uses the report every other month. A BJA supervisor indicated that he did not receive the reports, but he could get it, as needed, from the Office of the Comptroller.

The lack of systematic data associated with program monitoring activities and the documentation problems we observed raise questions about whether BJA and VAWO are positioned to measure their performance consistent with the Government Performance and Results Act of 1993. For example, in its Fiscal Year 2000 Performance Report and Fiscal Year 2002 Performance Plan, DOJ articulated a strategic goal to “prevent and reduce crime and violence by assisting state, tribal, local and community-based programs.” DOJ’s performance report and plan list various annual goals and performance measures along with performance data needed to gauge performance, including three program targets and performance measures for VAWO formula and discretionary grant programs. To gauge annual performance for the three VAWO targets, the plan and report state that needed performance data will be obtained from grantee progress reports, on-site monitoring, and VAWO program office files; will be verified and validated through a review of grantee progress reports, telephone contacts with grantees, and on-site monitoring of grantee performance by grant program managers; and contain no known limitations.

Although the DOJ performance report and plan state that there are no known data limitations, inconsistent documentation and the lack of systematic data could be a serious limitation that hinders VAWO’s ability to measure whether it is achieving its goals. VAWO officials told us that they are not satisfied with the current performance measures because they do not believe they are meaningful for measuring program outcomes. They said that they have begun to work with OJP’s Office of Budget and Management Services and an outside contractor to develop new measures. They added that their goal is to have these new measures available for the fiscal year 2003 GPRA performance plan.

OJP’s Efforts to Resolve Continuing Grant Monitoring Problems

Over the last few years, we and others, including OJP, have identified various grant monitoring problems among OJP’s bureaus and offices. OJP has begun to work with its bureaus and offices, such as BJA and VAWO, to address these problems, but it is too early to tell whether its efforts will be enough to resolve many of the issues that we and others, including OJP, have identified.

Problems With OJP Grant Monitoring Are Not New

Since 1996, we have testified and issued reports that document grant monitoring problems among some of OJP bureaus and offices. In 1996, we testified on the operations of the Office of Juvenile Justice and Delinquency Prevention. We found that almost all of the official discretionary grant files we reviewed contained monitoring plans, but there was little evidence that monitoring had occurred. More recently, in an October 2001 report, we observed many of the same issues concerning OJJDP’s lack of documentation of its monitoring activities. Also, in 1999, we issued a report that, among other things, addressed how OJP’s Executive Office for Weed and Seed monitors local Weed and Seed sites to ensure that grant requirements are met. We found that, during fiscal year 1998, grantees had not submitted all of the required progress reports and grant managers had not always documented the results of their on-site monitoring visits.

OJP has also identified problems with grant monitoring. In 1996, an OJP- wide working group, established at the request of the Assistant Attorney General, issued a report on various aspects of the grant process, including grant administration and monitoring. Among other things, the working group found that the administration of grants, including monitoring, was not standardized within OJP; given the monitoring resources available, monitoring plans were overly ambitious, the usual result being failure to attain the level of monitoring indicated in the plans; and an OJP-wide monitoring tracking system was needed to document all monitoring activities on an individual grant and facilitate control of the monitoring process.

The working group recommended that OJP establish another working group to develop detailed operating procedures, giving special attention to the issue of grant monitoring.

Almost 4 years later, in February 2000, Dougherty and Associates, under contract, delivered a report to OJP containing similar findings. The report stated that that OJP lacks consistent procedures and practices for performing grant management functions, including grant monitoring, across the agency. For example, according to the report, no formal guidance had been provided to grant managers on how stringent or flexible they should be with grantees in enforcing deadlines, due dates, and other grant requirements. Also, the report stated that the official grant files were often not complete or reliable. To improve grant monitoring, the contractor recommended, among other things, that OJP develop an agencywide, coordinated and integrated monitoring strategy; standardize guidelines and procedures for conducting site-visits, product reviews, and other monitoring activities; and mandate the timeliness and filing of monitoring reports.

The DOJ’s Office of Inspector General has also identified and reported on OJP-wide grant management and monitoring problems. For example, in December 2000, the Inspector General identified grant management as one of the 10 major management challenges facing DOJ. The Inspector General stated that DOJ’s multibillion-dollar grant programs are a high risk for fraud, given the amount of money involved and the tens of thousands of grantees. Among other things, the Inspector General said that past reviews determined that many grantees did not submit the required progress and financial reports and that program officials’ on-site monitoring reviews did not consistently address all grant conditions.

Too Early to Gauge Effectiveness of OJP’s Efforts to Resolve Grant Monitoring Problems

OJP has begun to work with bureaus and offices to resolve some of the problems it and others have identified, but it is much too early to tell how effective these efforts will be in resolving these issues. In its Fiscal Year 2000 Performance Report and Fiscal Year 2002 Performance Plan developed under GPRA, OJP established a goal to achieve the effective management of grants. Among other things, DOJ plans to achieve this goal by continued progress toward full implementation of a new Grant Management System as a way of standardizing and streamlining the grant process. According to the performance report and performance plan, GMS will assist OJP in setting priorities for program monitoring and facilitate timely program and financial reports from grantees.

OJP’s Chief of Staff at the time of our review told us that, in his view, the only way to ensure that staff consistently document their monitoring activities is to require grant managers to enter information about their monitoring activities, when they occur, into an automated system, like GMS. He said that currently, OJP runs the risk of losing or misplacing key documentation, especially since documents are kept in two files physically maintained by two organizations in different locations—one, the official file maintained by the Office of the Comptroller, and the other, the grant manager’s file maintained by the program office. He said that the new system currently covers grants for some OJP organizations up to the award stage, but that OJP’s goal is to include tracking information about all stages of the grant process from preaward through closeout.

Although GMS may ultimately help OJP better manage the grant administration process, DOJ’s Fiscal Year 2000 Performance Report and Fiscal Year 2002 Performance Plan does not state when GMS will be expanded—either to all of the OJP components or to include the full range and scope of monitoring activities. Regarding the latter, OJP’s Director of Office of Budget and Management Services indicated that it is unlikely that GMS will cover the full range of monitoring activities; instead, OJP would be more likely to develop a monitoring management information system to capture monitoring data that would link to GMS. The Director said that OJP has formed an OJP-wide working group to further study data issues related to monitoring activities, but the group is in its preliminary stages and has yet to develop a charter to define its activities.

OJP has also been working on two key efforts to enhance its ability to better control grant administration. One of these initiatives, “Operation Closeout,” was a pilot project announced in February 2000 by OJP’s Working Group on Grant Administration that was to, among other things, accelerate the grant closeout process through revised closeout guidelines and elevate the importance of the closeout function as a required procedure in the administration of grants. In November 2000, the working group announced that it had realized several of the Operation’s objectives and, working with the Office of the Comptroller, was able to reduce the backlog of grants, including some managed by BJA and VAWO, that were eligible to be closed but had not been closed. According to the Chairman of the working group, this operation closed out 4,136 outstanding grants over a 6-month period, resulting in over $30 million in deobligated funds. In September 2001, the Chairman said that OJP was going to initiate another closeout operation based on the success of the pilot.

Another OJP initiative involved the development and issuance of new OJP- wide guidance for grant administration, including grant monitoring. As mentioned earlier, in January 2001, OJP released Grant Management Policies and Procedures Manual to update and codify OJP’s current policies and procedures regarding its business practices. According to OJP officials, the new guidance was developed at the direction of the former Assistant Attorney General to address overall concerns about weaknesses in the 1992 version. The guidance was developed over a period of about two years, with the goal of reengineering the grant management process based on the best practices of bureaus and offices throughout OJP. For example, the changes include some provisions pertaining to some of the aforementioned closeout activities--grantees are now given 120 days to submit final financial reports (instead of 90 days). Also, grant managers are given greater latitude to close out a grant if they have been unsuccessful in obtaining the final financial report from the grantee.

OJP trained over 300 grant managers during summer 2001 and, according to the Chairman of the working group, intends to train supervisors about the new guidance in fall 2001. OJP also has drafted and plans to send a questionnaire to recently trained grant managers to identify any issues or problems with using the online manual and to identify potential training interest and topics. OJP plans to develop and send a similar questionnaire to supervisors once they are trained. However, the Chairman indicated that there are no plans to test or systematically monitor compliance with the new guidelines to ensure that grant managers are fulfilling their responsibilities. He said that OJP had not contemplated testing or systematic monitoring because of other initiatives currently under way.

Conclusions

Because BJA and VAWO discretionary grant files were insufficiently documented neither OJP, BJA, VAWO, or we can determine the level of monitoring being performed by grant managers as required by OJP and the Comptroller General’s internal control standards. BJA and VAWO supervisors rely on staff meetings and discussions with staff to alert them to grantee problems or grant monitoring issues, but these activities are not sufficient to ensure that required monitoring is taking place or that the proper documentation is occurring. Furthermore, BJA and VAWO do not have readily available data on most monitoring activities that would help them determine grant managers’ compliance with OJP guidelines, and even when data are available, it is not clear that supervisors use the data to ensure that monitoring activities occurred. The lack of systematic data, combined with poor documentation, limits BJA and VAWO’s ability to manage the grant monitoring process so that they can determine whether grant managers are monitoring grantees, and if not, why not, or if so, why are they not documenting their activities. It also hinders BJA and VAWO’s ability to measure their performance consistent with GPRA, especially given that DOJ is relying on data collected through grant monitoring to measure the Department’s performance for many of its grant programs. Furthermore, it places BJA and VAWO at risk in ensuring that the millions of dollars in discretionary grant funds that they distribute are effectively and responsibly managed.

Grant monitoring problems have been long-standing at DOJ, and although OJP has taken steps intended to resolve some of them, it is too early to tell whether these steps will effectively solve the types of documentation problems that we and others have identified. Automation of the grant management process, particularly in regard to grant monitoring, holds some promise if OJP takes steps to ensure that all monitoring activities are consistently recorded and maintained in a timely manner. However, current and future efforts will be futile unless OJP and its bureaus and offices, such as BJA and VAWO, periodically test grant manager compliance with OJP requirements and take corrective actions to enforce those requirements.

Recommendations for Executive Action

To facilitate and improve the management of program monitoring, we recommend that the Attorney General direct BJA and VAWO to review whether the documentation problems we identified were an indication of grant monitoring requirements not being met or of a failure to document activities that did, in fact, take place. If monitoring requirements are not being met, we recommend that the Attorney General direct BJA and VAWO to determine why this is so and to take into account those reasons as they consider solutions for improving compliance with the requirements. If it is determined that required monitoring is taking place but is not being documented, we recommend that the Attorney General take steps to direct BJA and VAWO to periodically articulate and enforce clear expectations regarding documentation of monitoring activities.

We also recommend that the Attorney General direct OJP to study and recommend ways to establish an OJP-wide approach for systematically testing or reviewing official and program files to ensure that the grant managers in its various bureaus and offices are consistently documenting their monitoring activities in accordance with OJP requirements and the Comptroller General’s internal control standards. Furthermore, we recommend that the Attorney General direct OJP to explore ways to electronically compile and maintain documentation of monitoring activities to facilitate (1) more consistent documentation among grant managers; (2) more accessible oversight by bureau and program office managers; and (3) sound performance measurement, consistent with GPRA.

Agency Comments and Our Evaluation

We provided a copy of a draft of this report to the Attorney General for review and comment. In a November 9, 2001, letter, the Assistant Attorney General commented on the draft. Her comments are summarized below and presented in their entirety in appendix III.

The Assistant Attorney General said that overall, the report provides useful information in highlighting management and monitoring activities in need of improvement. She noted that BJA and VAWO have already taken steps to address the recommendations for follow-up action included in the draft report. For example, the Assistant Attorney General said that BJA has taken steps to, among other things, expand its grants tracking system to include tracking of staff and grantee contacts and instituted a policy that a desk review be conducted twice per year for all grants. With regard to VAWO, the Assistant Attorney General said that, among other things, VAWO had established a monitoring working group tasked with developing monitoring policies and procedures for monitoring grantees more effectively, including more realistic monitoring plans and a standardized site visit reporting format.

We acknowledge that BJA and VAWO appear to be taking steps in the right direction toward resolving some of the issues we identified. However, until these actions become operational, BJA and VAWO will not be able to determine whether the problems we identified constitute either a failure to carry out required monitoring activities or a failure to document monitoring activities. Once BJA and VAWO make this determination, they will be better positioned to consider what additional steps need to be taken, such as articulating and enforcing clear expectations regarding the documentation of monitoring activities.

In her letter, the Assistant Attorney General did not address our recommendations that OJP (1) study and recommend ways to establish an OJP-wide approach for systematically testing or reviewing official and program grant files or (2) explore ways to electronically compile and maintain documentation of monitoring activities. Although the steps BJA and VAWO are taking may help them better understand and act upon problems associated with the documentation of monitoring activities, the steps discussed in the Assistant Attorney General’s letter appear to be actions specific to those organizations. Thus, it is unclear whether and to what extent those actions can be applied throughout OJP. Without a more focused and concerted effort to implement an OJP-wide approach for systematically testing or reviewing program grant files and an automated approach to compile and document monitoring data, OJP could continue to face the grant monitoring problems we and others, including OJP, have identified.

In addition to the above comments, the Assistant Attorney General made a number of suggestions related to topics in this report. We have included the Assistant Attorney General's suggestions in the report, where appropriate. Also, the Assistant Attorney General provided other comments for which we did not make changes. See appendix III for a more detailed discussion of the Assistant Attorney General's comments.

We are sending copies of this report to the Chairmen of the Senate Judiciary Committee and its Subcommittee on Youth Violence; Chairmen and Ranking Minority Members of the House Committee on Education and the Workforce; Attorney General; OJP Assistant Attorney General; BJA Administrator; VAWO Administrator; and Director, Office of Management and Budget. We will also make copies available to others on request.

If you or your staff have any questions about this report, please contact John F. Mortin or me at (202) 512-8777. Key contributors to this report are acknowledged in appendix IV.

Appendix I: Size and Growth of OJP, BJA, and VAWO Budgets and BJA and VAWO Discretionary Grant Programs

The Office of Justice Programs (OJP) and its bureaus and offices, including the Bureau of Justice Assistance (BJA) and the Violence Against Women Office (VAWO), experienced budget growth in the latter half of the 1990s, following the passage of the 1994 Crime Act. According to data we obtained from OJP, in the 1990s, the yearly number of BJA discretionary awards and total dollar amount of those awards fluctuated somewhat, but generally increased. The number of Byrne discretionary grant awards have decreased since a high point in fiscal year 1994; however, the total yearly dollar amount has increased overall. Furthermore, while the yearly number of Byrne awards and the dollar amount of those awards generally increased, there were some yearly decreases. Following the creation of VAWO in 1995, the yearly number of discretionary awards and total dollar amount of those awards increased overall.

Budget Increases at OJP, BJA, and VAWO in the 1990s

According to OJP data, from 1990 through 2000, OJP’s budget grew, in constant fiscal year 2000 dollars, by 323 percent, from about $916 million in fiscal year 1990 to nearly $3.9 billion in fiscal year 2000. Our analysis of the OJP data also showed that BJA’s budget grew during the 1990s, but to a lesser extent than OJP’s. BJA’s budget increased by 173 percent, from about $618 million in fiscal year 1990 to nearly $1.7 billion in fiscal year 2000. In fiscal year 1996, OJP and BJA’s budget increased sharply after enactment of the 1994 Crime Act. BJA’s budget as a percentage of OJP’s budget decreased from about 67 percent in fiscal year 1990 to 57 percent in fiscal year 1996. Following its creation in 1995, VAWO’s budget increased by 42 percent, from $176 million in fiscal year 1996, its first full year of funding, to about $250 million in fiscal year 2000. Figure 2 shows the growth in overall OJP, BJA, and VAWO budgets, illustrating how BJA and VAWO fit into the overall OJP budget from fiscal years 1990 through 2000.

Discretionary Grant Award Increases at BJA and VAWO in the 1990s

For this report, we analyzed the growth in the number and dollar amount of discretionary awards each year by BJA and VAWO from 1990 through 2000, based on data provided by OJP. From 1990 to 2000, the number of BJA Byrne discretionary awards increased by about 83 percent—from 54 in fiscal year 1990 to 99 in fiscal year 2000. Overall, the total number of BJA discretionary awards, including Byrne discretionary awards, increased by about 320 percent—from 65 in fiscal year 1990 to 273 in fiscal year 2000. As shown in figure 3, the overall increase in BJA discretionary awards for the 11-year period we analyzed was moderate, with some yearly decreases.

Along with the increased number of discretionary awards by BJA, the yearly total dollar amount of those awards also increased. As illustrated in figure 4 below, the total dollar amount of Byrne discretionary awards increased, in constant fiscal year 2000 dollars, by 256 percent—from about $19 million in fiscal year 1990 to almost $69 million in fiscal year 2000. The total dollar amount of all BJA discretionary awards, including Byrne discretionary awards, increased even more—by about 422 percent—from just over $36 million in fiscal year 1990 to nearly $189 million in fiscal year 2000. The increase in the yearly total dollar amount of Byrne discretionary awards was moderate, with some yearly decreases. For all BJA discretionary awards, particularly non-Byrne discretionary awards, the total dollar amount increases were substantial in fiscal years 1998 through 2000. Figure 4 shows the dollar amount of BJA discretionary awards (Byrne and non-Byrne) from fiscal year 1990 to 2000.

Data provided by OJP showed that at VAWO, since its inception in 1995, the yearly total number and dollar amount of its discretionary awards also increased. As shown in the figures below, the yearly number of VAWO discretionary awards increased by about 362 percent—from 92 in fiscal year 1996, the first full year of funding to 425 in fiscal year 2000. In addition, the yearly dollar amount of VAWO discretionary awards increased by about 940 percent—from just over $12 million in fiscal year 1996, the first full year of funding, to about $125 million in fiscal year 2000.

Figure 5 shows the number of VAWO discretionary awards for each fiscal year from 1995 to 2000. Figure 6 shows the dollar amount of VAWO discretionary awards for the same period.

Appendix II: Stages of the Discretionary Grant Process

OJP awards two types of grants: formula and discretionary. OJP formula grants are awarded directly to state governments, which then make subawards to state and local unites of government. Discretionary grants can be awarded to states, local units of government, Indian tribes and tribal organizations, individuals, educational institutions, private nonprofit organizations, and private commercial organizations. With some discretionary grant programs, OJP has some flexibility in selecting topics as well as grantees. Some discretionary awards are competitive, while others are non-competitive, owing to limited amount of funds available to a limited number of potential recipients. Figure 7 summarizes the life of a discretionary grant from application to closeout.

Once the application for the discretionary grant is accepted, OJP guidance requires the grant manager to prepare a grant manager’s memorandum, which OJP reviews before the award is made. The memorandum is to consist of the following, among other things: an overview of the project; a detailed description of what type of activities the grantee plans to a discussion of the financial justification of the grant funds and of the cost- effectiveness evaluation of the application; and a discussion of past assessments, where applicable.

As part of a grant manager’s memorandum, each grant manager is to prepare a monitoring plan. The plan is to contain information on who will conduct the monitoring, how it will be done, and when and what type of monitoring activities are planned. Monitoring information is to be collected using such techniques as on-site visits, telephone calls, and desk reviews, which are reviews to ensure that the grant files are complete and that the grantee is in compliance with the program guidelines. Also, OJP guidance requires the grantee to file specific reports with the Office of the Comptroller: semiannual progress reports that summarize project activities and quarterly financial reports that provide an accounting of grant expenditures. The Office of the Comptroller is to forward the reports to grant managers.

OJP is to apply the same process to supplemental awards as it does to the original award of a grant. When the grantee requests an extension, thus requiring supplemental funding, the grantee must repeat the award application process, and more time and money is expended.

At the end of the grant period, the grant manager is required to close out the grant according to OJP guidance. Closing out grants is the final step in a process by which OJP ensures that all required financial and progress reports and a final accounting of federal funds have been received. Figure 8 illustrates the grant process, including supplemental or extended funding.

Appendix III: Comments From the Department of Justice

The following are GAO’s comments on the Department of Justice’s November 9, 2001, letter.

GAO Comments

1. We report on the growth in the number and dollar amount of VAWO discretionary grants since fiscal year 1996 in appendix I (see pp. 34-35). 2. We have amended the Background section of the report to add this information (see p. 6). 3. We disagree. As we stated in the report (see p. 2), financial monitoring was not within the scope of our work. It is important to note that the scope of our work was based on agreements with our requesters and was not influenced by whether or not financial monitoring information is included in OJP’s annual financial statement audit. 4. According to the OJP Handbook: Policies and Procedures for the Administration of OJP Grants (Feb. 1992), official grant files kept by the Office of Comptroller Control Desk are to contain documents relating to each grantee, including progress and financial reports and site visit reports. In addition, for documentation to be readily available for examination, as required by the Comptroller General’s internal control standards, keeping them in the official grant files seems appropriate. 5. As we reported, in our review of closeout procedures, we waited more than the required 180 days before reviewing grant files to allow sufficient time for BJA and VAWO to complete the grant closeout process (see footnote 24, p. 18). However, the files we reviewed did not contain the required closeout documents. 6. As we reported, BJA and VAWO officials told us that supervisors discuss monitoring activities with staff through informal discussions or meetings, which could include one-on-one meetings with staff. As we stated, it is possible that grant managers are conducting grant monitoring activities even if no documentation exists. However, without documentation, neither OJP, BJA, VAWO, nor we are positioned to tell with any certainty whether such monitoring occurred (see pp. 20-21). 7. We have amended the report to add some of this information (see p. 7).

As discussed in comment 1, we report on the growth in the number and dollar amount of VAWO discretionary grants in appendix I (see pp. 34-35). 8. We disagree. Financial monitoring was not part of our review as clearly stated in our introduction. Therefore, we do not believe that the title of this section, as stated, implies that financial monitoring was part of our review. 9. We agree that the grant files did not always contain documentation and acknowledge that the lack of documentation does not necessarily indicate whether monitoring did or did not occur. As we stated in comment 6, without required progress reports and other documentation, neither OJP, BJA, VAWO, nor we are positioned to tell with any certainty whether such monitoring occurred. 10. We have amended the report to add most of this information (see p. 20). A discussion of the development of VAWO’s management information system can be found on page 21 of the report. 11. As discussed in comment 6 and as we reported, it is possible that grant managers are conducting grant monitoring activities even if no documentation exists. However, without documentation, neither OJP, BJA, VAWO, nor we are positioned to tell with any certainty whether such monitoring occurred. 12. We disagree. As we reported, the Comptroller General’s internal control standards require that all transactions and other significant events be clearly documented and that the documentation be readily available for examination. Appropriate documentation is an internal control activity to help ensure that management’s directives are carried out. Without such documentation, OJP, BJA, and VAWO have no assurance that grants are meeting their goals and funds are being used properly. 13. We have amended the report to incorporate the first of these two changes (see p. 31). However, as illustrated in figures 3 and 4, the annual number of Byrne awards and the dollar amount of those awards have generally increased, although there were some yearly decreases (see pp. 33-34). 14. We reported that, according to VAWO officials, VAWO grant managers have sometimes been responsible for additional duties beyond grant monitoring over the last few years (see p. 19).

Appendix IV: GAO Contacts and Staff Acknowledgments

GAO Contacts

Staff Acknowledgments

In addition to the above, Kristeen G. McLain, Samuel A. Caldrone, Dennise R. Stickley, Keith R. Wandtke, Jerome T. Sandau, Michele C. Fejfar, and Ann H. Finley made key contributions to this report.