Juvenile Justice: Better Documentation of Discretionary Grant Monitoring Is Needed
Published: 2001-10-10T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Office of Juvenile Justice and Delinquency Prevention (OJJDP) provides block grants and discretionary funding to help states and communities prevent juvenile delinquency and improve their juvenile justice systems. OJJDP has specific program monitoring and documentation requirements for its discretionary grants. These monitoring requirements include having the grant manager make quarterly telephone calls, undertake on- and off-site grant monitoring visits, and review interim and final products.
section_title: What GAO Found
paragraphs: In a review of OJJDP's most recent award of grants active in fiscal years 1999 and 2000, GAO found that OJJDP's grant monitoring activities were not consistently documented. These findings are similar to those GAO reported in May 1996.

section_title: Letter
paragraphs: The Office of Juvenile Justice and Delinquency Prevention (OJJDP) was established to help states and communities prevent and control juvenile delinquency and improve their juvenile justice systems. OJJDP provides funding to states, territories, localities, and private organizations through block grants and discretionary funding. Between September 1996 and September 2000, the number of active OJJDP discretionary grants has more than tripled, from 240 to 790, and the related funding for active grants almost doubled from $286 million to $555 million.
Monitoring these grant activities is a key management tool to help ensure that funds are being properly spent. In May 1996, we testified that the official grant files for discretionary grants generally contained plans for agency monitoring, but little evidence that it occurred.
Your original request asked about various aspects of OJJDP’s grant management. We will be responding to that request with two reports. This, the first, addresses how well OJJDP monitors its grants to ensure that grantees properly implement the programs that the grants support. The second report will discuss OJJDP grantee reporting requirements, the number of juveniles OJJDP programs have directly served, and our analysis of the program impact evaluations OJJDP has funded of its programs.
Specifically, we reviewed and are reporting on the (1) OJJDP grant monitoring process, (2) extent to which OJJDP documented its discretionary grant monitoring activities, and (3) process it uses to oversee those activities. In appendix I, we describe OJJDP’s award process for discretionary grants.
To address these areas, we met with officials from OJJDP and OJP concerning discretionary grant monitoring activities, including OJJDP’s activities to oversee its monitoring. We reviewed OJP and OJJDP’s grant monitoring requirements and selected and reviewed a representative sample of OJJDP demonstration discretionary grants that were active in all of fiscal years 1999 and/or 2000. We did this to determine whether OJJDP officials were adhering to the agency’s discretionary grant monitoring requirements and how the results of its monitoring efforts were documented in its grant files. We also made a similar review of a representative sample of OJJDP training and technical assistance grants.To determine OJJDP’s oversight role of its discretionary grant monitoring activities, we reviewed information such as performance work plans. We also discussed with OJJDP officials the steps it has taken to improve previously reported monitoring problems.
Our analysis did not address the quality of how OJJDP monitored its grantees. For example, we reviewed documentation to determine whether OJJDP made required site visits, but we could not review the quality of the site visit contact.
subsections: section_title: Background
paragraphs: OJJDP, within the U.S. Department of Justice, is the primary federal agency responsible for the prevention and control of juvenile delinquency in the United States. This includes preventing, treating and controlling youth gang activity and violence through sponsored research, evaluation, and demonstrating new approaches for communities across the country. The initiatives are carried out by seven components within OJJDP: Research and Program Development Division, Training and Technical Assistance Division, Special Emphasis Division, State Relations and Assistance Division, Information Dissemination Unit, Concentration of Federal Efforts Program, and Child Protection Division. OJJDP annual appropriations have more than tripled from $162 million in fiscal year 1996, with 71 authorized staff positions, to $568 million in fiscal year 2000, with 87 authorized staff positions.
In 1996, we reviewed the operations of OJJDP. We found that official grant files for discretionary grants generally contained monitoring plans, but little evidence that monitoring occurred. We reported that none of the grant files had documentation of telephone contacts, site visits, or product reviews. Also, no quarterly program and financial reports were included in 11 of the 78 files for which projects had been ongoing for at least 2 quarters at the time of our review, and one or more reports were missing from another 61 files. In addition, only 6 of the 78 files had program and financial reports for all quarters of work completed.
In commenting on our testimony, the OJJDP Deputy Administrator said that the heavy workload of OJJDP staff may have resulted in a lack of monitoring records and that, as a result of the our 1996 review, OJJDP would take the steps necessary to improve records.
section_title: Scope and Methodology
paragraphs: To address the first objective, to report on OJJDP’s grant monitoring process, we reviewed applicable federal laws and regulations as well as OJP and OJJDP policies and procedures. We also interviewed staff in OJJDP and OJP’s Office of the Comptroller.
To accomplish the second objective, to report on OJJDP’s documentation of its discretionary grant monitoring activities, we reviewed probability samples of official grant files and grant managers’ files using a data collection instrument to record whether required monitoring activity occurred—that is, that the progress reports, financial reports, and other required documents were included in the files. For each grant, we also reviewed the most recent award covering 12 months to examine specific monitoring requirements and activities, such as telephone calls and site visits. Specifically, we reviewed a random sample of 89 of 545 OJJDP demonstration discretionary grants, and 45 of 83 training and technical assistance discretionary grants that were active in all of fiscal years 1999 and/or 2000. The results of these samples are representative of the populations from which they were drawn. We express our confidence in the precision of our sample results as a 95-percent confidence interval. Unless otherwise noted, all percentage estimates have confidence intervals plus or minus 10 percentage points or less.
To determine the timeliness of grantee progress reports and financial status reports, we reviewed all such reports for OJJDP demonstration and training and technical assistance discretionary grants from the initiation of the grant through the date of our review.
For our review, we provided a list of the selected grants to OJJDP and OJP’s Comptroller. Because we did not always receive the grant files in the order requested, we reviewed them out of order. When files were missing or lost, we substituted replacement files. After our review was complete, some of the missing files were located. We did not review the missing files but used the substitute files that were a part of our random sample. Four grant files were not found in time to be included in our review. In addition, when only one of the two files (the official grant file or the grant managers’ file) was available, we used the available file to complete the data collection instrument. We have no reason to believe that the substitution of files affected the validity of our sample because they were a part of our random sample.
To determine whether OJJDP grant closeout procedures were implemented in accordance with OJP policy, we reviewed grant files in our sample whose project periods expired before August 31, 2000. For our review, we focused on required closeout documentation, such as final progress or performance reports, final financial status reports, grant checklist, and closeout notification and grant adjustment notice. Our results should not be projected to the universe of cases in our sample.
To address the third audit objective, to report on the processes OJJDP uses to oversee monitoring activities, we examined OJJDP’s monitoring and oversight of its grant managers’ compliance with applicable laws, regulations, and procedures for monitoring. We obtained information from OJJDP officials at headquarters and obtained data related to these activities, such as performance work plans. To determine the requirements in the performance work plans, OJJDP officials gave us a plan that they considered to be representative.
Our work was done between March 2001 and September 2001 in accordance with generally accepted government auditing standards.
section_title: Discretionary Grant Monitoring Process
paragraphs: After OJJDP awards discretionary grants, OJP policies require OJJDP to monitor the grants and related activities and document the monitoring results in the grant managers’ program files and OJP’s Office of the Comptroller official grant files. The monitoring is to ensure   compliance with relevant statutes, regulations, policies, and guidelines; responsible oversight of awarded funds; implementation of required programs, goals, objectives, tasks, products, time lines, and schedules; identification of issues and problems that may impede grant implementation; and   adjustments required by the grantee as approved by OJJDP.
One of the OJJDP grant managers’ main responsibilities is project monitoring. Each grant manager prepares a monitoring plan as part of a grant manager’s memorandum recommending initial or continuation funding. The level of monitoring required is based upon the stated monitoring plan in the grant manager’s memorandum. The memorandum includes   an overview of the project;   a detailed description of what type of activities the grantee plans on implementing;   a discussion of past monitoring activities and assessments, if the grant is a multiyear grant that is awarded yearly;   an identification of monitoring activities to be performed for the current   a discussion of the financial justification for the grant funds and of the cost-effectiveness evaluation of the application.
Another part of the monitoring plan may list specific activities or milestones to be monitored by the grant manager.
The 1991 OJJDP Monitoring Desk Book contains specific monitoring procedures that grant managers used for monitoring OJJDP’s discretionary grants. Monitoring activities documented included site visit reports and quarterly monitoring telephone calls as well as other significant telephone calls.
Also, according to the OJP 1992 Handbook, all discretionary grantees are required to submit categorical assistance progress reports that summarize project activities and quarterly financial status reports for review by grant managers. The OJP Handbook also stated that when all of the grant funds are spent, all grant activities are completed, and all grantee and grant managers’ administrative requirements, including all paperwork and review of grant files, are completed, the grant managers have to close out the grant. (See app. I for a detailed discussion of the grant award processes.)
The OJP Comptroller’s Control Desk is to maintain the official grant files and is responsible for tracking the receipt of all grant documents. The Control Desk receives the progress reports, files the original in the official grant file, and forwards a copy to the cognizant program office. The grant manager is responsible for the timely acceptance, review, and analysis of progress reports.
The Comptroller’s Monitoring Division performs an in-house review of various program and financial documents contained in the official grant files. The Office of the Comptroller also performs risk-based on-site financial reviews of grantee organizations to monitor administrative and financial capability. The Comptroller applies risk-based criteria to a universe of grants to develop a sample for each fiscal year monitoring plan. The risk-based criteria include the dollar amount of the grant, new grantees, new grant programs, and programs with known problems. The Comptroller excludes from its sample any grantee organizations that were subject to its other on-site financial monitoring or to audit by the Office of the Inspector General during the last 2 years.  According to OJP officials, the purpose of the reviews is to provide as much of a comprehensive financial review as possible without going on site. Our review of the results of the Comptroller’s in-house financial reviews of the grants in our sample disclosed that they identified a number of problems similar to our own observations. These included missing program and financial documents and program and financial monitoring not always being documented in a timely manner.
In commenting on a draft of this report, the Acting Assistant Attorney General stated that there was no mention of delinquent documentation of financial monitoring during our review, nor was there any evidence cited in our draft report to support the contention that financial monitoring was not documented in a timely manner. However, contrary to the Acting Assistant Attorney General comment, our review found that there was no documentation that financial reports were generally reviewed and that many of the reports were not submitted in a timely manner. Specifically, we found one case where no progress or financial reports had been submitted during the life of a grant, and it was not discovered until almost a full year after the award date (see p. 15).
subsections: section_title: Outdated Monitoring Policies and Procedures Were Being Used
paragraphs: In 1991, OJJDP issued a Monitoring Desk Book, which incorporated material from OJP’s 1987 Handbook, including a formula for determining the monitoring level requirements for the grant. According to OJJDP officials, this formula is applied to determine how often a grantee will be visited and officially contacted for monitoring purposes. This formula applies a weight to issues such as program priority (i.e., how important is the grant within the context of the OJJDP’s goals and objectives), size of grant, grantee’s need for assistance, and anticipated difficulties in implementing the approach. The grant manager assigns a weight or “score” to each of these criteria, which are added together for a total score. This score assigns the grant to a category that defines the specific number of on-site and off-site visits and the frequency of monitoring calls.
In February 1992, OJP revised its 1987 Handbook and eliminated the score requirement to determine the monitoring level of a project. However, OJJDP grant managers still use the instructions to determine the monitoring level of a project. We found in our file reviews that 72 of the 89 demonstration grant files and 14 of the 45 training and technical assistance grant files had a score or level indicating monitoring requirements. Using the obsolete monitoring requirement to establish grant manager’s grant monitoring responsibilities may not result in the most efficient use of their time or in the needed monitoring to ensure grantee compliance, as pointed out by the Deputy Administrator in May 1996. He said that site visits did not occur because they are expensive and time-consuming. He added that OJJDP might need to revise some procedures, noting that a site visit once every 2 years and some interim telephone monitoring may be more appropriate than annual visits. Current OJP policy provides flexibility to determine monitoring needs (e.g., site visits as appropriate) consistent with resource availability as compared with a monitoring level that is associated with a specific monitoring requirement that may not be achievable with existing resources.
section_title: OJJDP Current Activities
paragraphs: Since April 2001, OJJDP has been engaged in a review of its monitoring practices. According to OJJDP officials, this includes a thorough review of its monitoring practices, including improvement of its grant records. OJJDP indicated that a Working Group on Monitoring was created and charged with reviewing its revised OJP Handbook: Policies and Procedures for the Administration of OJP Grants to identify new and modified policies and assess the need for training; to review OJJDP’s own policies and procedures to ensure compliance with those in the revised OJP Handbook; and to determine whether there is a continued need for the OJJDP Monitoring Desk Book (1991), and if so, how it should be updated. In addition, OJJDP recently embarked on an overhaul of its grant closeout practices to improve its monitoring records. In spring 2000, it launched a grant closeout effort jointly with the Office of the Comptroller that is intended to significantly reduce the backlog of grants requiring closeout. As part of this activity, grant files were reviewed, and relevant documentation was collected.
section_title: Grant Files Contained Limited Documentation That Monitoring Occurred
paragraphs: OJJDP requires its grant managers to document their monitoring of discretionary grants. This documentation is to include the development and retention of a grant manager’s memorandum for each grant containing a plan to follow for monitoring the project. In addition, progress reports documenting progress achieved in relation to project milestones are required from all grantees. Required documentation also is to include quarterly financial reports from grantees. In addition, OJJDP requires certain documentation at the time of grant closeouts.
While the grant files generally contained monitoring plans, OJJDP grant managers were not consistently documenting their monitoring efforts according to the monitoring plans. The grant files generally did not contain documentation showing required telephone contacts and site visits.Further, progress and financial reports did not cover the entire grant period over half of the time. For those grants in our sample that were closed, most did not contain documentation showing compliance with closeout procedures. We identified some of these same documentation- related problems in May 1996.
We compared the proposed monitoring activities in the grant manager’s monitoring plan for 89 of 545 OJJDP demonstration discretionary grant files and 45 of 83 OJJDP training and technical assistance discretionary grant files to actual monitoring documentation. Our comparisons of the proposed to actual documented monitoring revealed that OJJDP grant managers were not consistently following the monitoring plan.
Telephone Contacts. The grant managers’ monitoring plans for the most recent awards for 79 demonstration grants and 34 training and technical assistance grants had specific requirements for telephone contacts (e.g., monthly or quarterly). There was no documentation showing that the requirement for telephone contacts was met in 96 percent of the demonstration grants and in any of the training and technical assistance grants. Documentation was present that showed that some, but not all, of the required telephone contacts had been made for 49 percent of the demonstration grants and 41 percent of the training and technical assistance grants.
Site Visits. The grant managers’ monitoring plans for the most recent awards for 66 demonstration grants and 20 training and technical assistance grants had a specified number of on-site visits to be made. There was no documentation showing that the requirement for site visits was met in 88 percent of the demonstration grants and 90 percent of the training and technical assistance grants. Documentation was present that showed that some, but not all, of the required on-site visits had been made for 6 percent of the demonstration grants and 10 percent of the training and technical assistance grants. When site visits are made to grantees, a written site visit report is to be prepared. Of the awards that had any evidence of site visits, such as memos or e-mails referring to the visit, 7 out of the 12 demonstration grants and 2 out of the 4 training and technical assistance grants had any of the OJP-required, written site visit reports.
In commenting on a draft of this report, the Acting Assistant Attorney General informed us that most grant managers prepare site visit reports after an on-site review, as their travel vouchers are not forwarded for payment unless the reports are attached. She noted that the grant managers do not routinely forward copies of the reports to the official grant and program files. She said that a copy of the reports is filed in the grant manager’s travel folders that we did not review. However, according to the OJP Handbook, the original site visit report is to be placed in the official grant file. While the grant managers’ travel folders may contain site visit reports, the official grant files kept by the OJP Comptroller’s Control Desk are the official record of the history and activity of the grant. As such, we reviewed the official grant files as well as the grant managers’ files for required grant documents and not the travel voucher folders.
In another comment to a draft of this report, the Acting Assistant Attorney General told us that monitoring plans that call for on-site visits by grant managers are contingent upon staffing capacity and administrative resources and not requirements. Our review indicated that, contrary to the Acting Assistant Attorney General’s comment, certain specific program monitoring requirements are included in the OJP Handbook, such as creating monitoring plans and writing site visit reports within 10 days of a site visit. To be useful, monitoring plans should be feasible. If they are not feasible, then they will not be helpful guidance for monitoring. While it is entirely reasonable to expect that some planned site visits will not take place (for a variety of reasons), the proportion of site visits specified to occur that were not documented seems to indicate that either they occurred but were not documented or monitoring plans for site visits were infeasible.
Progress and Financial Reports. According to the OJP Handbook,recipients of discretionary grants are required to submit initial and then semiannual categorical assistance progress reports documenting progress achieved in relation to project milestones. Before December 31, 1995, progress reports were due every quarter. Since March 1, 1996, progress reports have been due every 6 months. This report must be submitted within 30 days after the end of the reporting period for the life of the award. A final report, which provides a summary of progress toward achieving the goals and objectives of the award, significant results, and any products developed under the award, is due 90 days after the end date of the award.
We found that in 56 percent of the 89 demonstration grant files and 80 percent of the 45 training and technical assistance grant files reviewed, the progress reports did not cover the entire grant period. In addition, 7 percent of the demonstration grants files and 4 percent of the training and technical assistance grants had no progress reports.
We also found that 56 percent of the progress reports submitted for demonstration grants were filed on time; for the other 44 percent, the number of days late ranged from 1 to 545, with 62 percent of the reports for demonstration grants being late up to 30 days and 24 percent between 31 and 90 days. We found that 42 percent of the training and technical assistance grant reports were filed on time; for the other 58 percent, the number of days late ranged from 1 to 297, with 76 percent being late up to 30 days and 15 percent between 31 and 90 days.
According to the OJP Handbook, all discretionary grantees are required to submit quarterly financial status reports to OJP’s Office of the Comptroller Control Desk, which maintains the official grant files. Copies are disseminated to the appropriate financial analysts and cognizant grant managers. These quarterly reports are required for all active grants, even if there has been no financial activity during the reporting period. According to OJP officials, reports are considered delinquent, for payment purposes, if not received 45 days after the end of each calendar quarter. The final financial status report is due 90 days after the end of the grant period or extension. Future awards and fund drawdowns may be withheld if the progress and financial status reports are delinquent.
We found that in 65 percent of the 89 demonstration grant files and 60 percent of the 45 training and technical assistance grant files reviewed, the financial status reports did not cover the entire grant period. We also found that 67 percent of the financial status reports submitted for demonstration grants were filed on time; for the other 33 percent, the number of days late ranged from 1 to 367, with 54 percent of the reports for demonstration grants being late up to 30 days and 34 percent between 31 and 90 days. We found that 70 percent of the training and technical assistance grant reports were filed on time; for the other 30 percent, the number of days late ranged from 1 to 353, with 72 percent being late up to 30 days, and 19 percent between 31 and 90 days.
Further, our review found no documentation in the files that OJJDP grant managers regularly reviewed these reports, as required by OJP. Without such reviews, OJJDP does not have assurances that the grants are being properly administered. For example, one grant we reviewed had no documentation of required semiannual categorical assistance progress reports or quarterly financial reports in the file for the entire life of the award. The project and budget periods ran from October 1, 1999, to September 30, 2000. A $300,000 grant was awarded on March 7, 2000 (6 months after the start of the budget period). The grantee was required to submit an initial and then semiannual categorical assistance progress reports documenting progress achieved in relation to project milestones, starting June 30, 2000, and every 6 months thereafter. The grantee was also required to submit a quarterly financial status report within 45 days after the quarter ended, starting May 15, 2000, and every quarter thereafter.
Although no progress or financial reports had been submitted during the life of the grant period, OJJDP did not discover this until almost a full year from the award date. Nevertheless, OJJDP extended the grant period for an additional year to allow the grantee to spend the funds for the project, which it had not drawn down during the grant period. Missing progress reports and quarterly financial reports increase the risk that OJJDP might not be able to identify and rectify grantee’s program and financial problems in a timely manner.
In commenting on a draft of this report, the Acting Assistant Attorney General acknowledged this lack of documentation, but did not agree that it meant that OJJDP had not been monitoring grants to ensure they were being properly implemented. From our perspective, this lack of documentation suggests to us that OJJDP does not know the extent to which grant managers were carrying out their monitoring responsibilities to ensure that grants were being properly implemented.
Despite the lack of evidence that grant managers were adhering to the established monitoring plans, we did find that some grant monitoring was occurring. For example, the files contained evidence that communication occurred between grantees and the grant managers in the form of reports, faxes, letters, and oral and e-mail communication on specific issues, problems, or requests for information. However, much of the communication between grantees and grant managers was not documented according to OJJDP requirements.
Closeout Procedures. The OJP grant closeout process calls for an accounting of programmatic accomplishments in relation to planned activities; an inventory of all required financial, programmatic, and audit reports; and an accounting of all federal funds. According to the OJP Handbook, the grant manager is to complete an assessment report and other closure requirements within 180 days after the end date of the award or any approved extension. These requirements include a closeout checklist, overall assessment of grant performance, final progress reports, final financial status reports, and closeout notification and grant adjustment notice.
Our assessment of the closeout process did not include enough cases to ensure that it is representative of all grant closeouts. However, our limited review showed that some grant files did not contain required closeout materials.
At the time of our review, the following documents were missing from the files:   The required case checklists were not prepared for two of the eight demonstration grants that expired and required closeout. Similarly, neither of the two training and technical assistance grants had the required case checklists.
An overall assessment of grant performance was not prepared for two of the eight demonstration grants. Neither of the two training and technical assistance grants had this assessment.
Final progress reports were missing from three of the eight demonstration grant files. Neither of the two training and technical assistance grants had final progress reports in the files.
The final financial status report was missing from one of the eight demonstration grant files. Neither of the two training and technical assistance grants had a final financial status report in the files. And there was no documentation that grant managers had reviewed any of the reports or other official documents that were filed.
A closeout notification and grant adjustment notice to deobligate funds or advise the grantee to retain records for 3 years was not prepared for three of the eight expired demonstration grants. Similarly, neither of the two training and technical assistance grants had a closeout notification and grant adjustment notice.
In commenting on a draft of this report, the Acting Assistant Attorney General noted the programmatic and fiscal closure process might not have been completed for those grants until after our review. In our review of closeout procedures, we waited at least the required 180 days before reviewing the grant files to allow sufficient time for OJJDP to complete the grant closeout process. However, the files we reviewed did not contain the required closeout documents.
The Comptroller General’s internal control standards require that all transactions and other significant events be clearly documented and that the documentation be readily available for examination. Appropriate documentation is an internal control activity to help ensure that management’s directives are carried out. Without such documentation, OJJDP has no assurance that grants are meeting their goals and funds are being used properly.
In commenting on a draft of this report, the Acting Assistant Attorney General also said that almost 50 percent of the 89 Special Emphasis Division grants that we reviewed were awarded in fiscal years 1998 and 1999 under the Drug-Free Communities Support Program. Because of the time needed to recruit, hire, and train new program staff, OJJDP handled 217 Drug-Free Communities Support Program grants in those 2 years with less than adequate (i.e., two) staff. According to the Acting Assistant Attorney General, caseloads of this size make proper documentation a difficult goal to achieve.
section_title: OJJDP Does Not Systematically Determine Staff Compliance With Its Monitoring Requirements
paragraphs: OJJDP does not systematically monitor grant managers’ compliance with its monitoring requirements or guidance nor does it assess the effectiveness of OJJDP grant monitoring practices. When we asked OJJDP officials about any oversight of OJJDP grant managers to determine the adequacy of their monitoring of OJJDP discretionary grants, they told us oversight takes several forms. They pointed out that the OJP Handbook requires staff performance to be carefully scrutinized against the performance work plans (i.e., the critical performance elements and standards pertaining to monitoring responsibilities). They said that OJJDP supervisory managers are to address the following questions to determine how well each grant manager with monitoring responsibilities is performing: (1) how informed is the grant manager about his or her project; (2) what specific oversight and review activities are being conducted; and (3) how are grantee performance issues, if any, being resolved?
Our review of the performance work plan OJJDP considered to be representative found that it tended to contain information that made it difficult to precisely understand the nature of the monitoring to be conducted. The performance work plan, for example, referred to monitoring grants to ensure that projects meet their goals and milestones, with no clear expectations regarding monitoring requirements, such as site visits or telephone monitoring, and documentation of these contacts and reviews of grantees’ progress and financial reports.
In addition, according to OJJDP officials, they do not have an officewide management information system that tracks monitoring activities specific to each grant and links these to grant monitoring staff. However, OJJDP officials said they take several actions to ensure that staff are adequately performing their monitoring responsibilities. They said the grant manager’s memorandum is carefully scrutinized to determine that the proper information regarding grant goals, objectives, tasks, program activities, and products are included. Division directors are to review requests for site visits on a quarterly or semiannual basis as part of their development of travel budgets. Site visit reports are reviewed at the time or within a reasonable period of submission of travel reimbursement requests. Finally, division directors are routinely consulted regarding corrective action on grants.
section_title: Conclusions
paragraphs: In our May 1996 work, we reported that OJJDP’s files for discretionary grants generally contained monitoring plans but little documentation that monitoring occurred. OJJDP said that a heavy workload might have resulted in a lack of monitoring records, but that it would take the steps necessary to improve records. OJJDP has made limited progress in addressing the monitoring problems we identified in 1996.
Our review showed that monitoring activities continue to be insufficiently documented, and because of this, neither OJJDP nor we can determine the level of monitoring being performed by grant managers as required by OJP, OJJDP, and the Comptroller General’s internal control standards.
OJJDP also cannot rely on supervisory oversight of its grant managers to ensure that monitoring is being performed as required. The use of staff performance work plans seems problematic as a means of reviewing monitoring activities because the requirements are written in such general terms. Grant managers’ performance work plans do not include clear expectations and accountability for monitoring called for in grant managers’ monitoring plans as well as follow-up requirements to notify and assist grantees who consistently submit late or no progress or financial reports. Further, the approval of grant managers’ memorandums or preapproval of site visits does not ensure that required monitoring is taking place, and none of these oversight activities is designed to ensure that proper documentation of monitoring activity is taking place.
Current activities to review monitoring policies and practices and to complete documentation for closed grants could result in positive changes. However, these activities are too new to be evaluated in relation to their potential for improving grant monitoring or its documentation.
section_title: Recommendations for Executive Action
paragraphs: The current review of monitoring policies and practices affords an opportunity for OJJDP to assess monitoring in light of our recent findings. To facilitate and improve the management of program grant monitoring, we recommend that the Attorney General direct OJJDP, as part of its ongoing review of monitoring policies and practices, to determine whether the monitoring documentation problems that we identified were an indication of grant monitoring requirements not being met or of a failure to document activities in the official grant files that did, in fact, take place.
If monitoring requirements are not being met, we recommend that the Attorney General direct OJJDP to determine why this is so and to consider those reasons as it develops solutions for improving compliance with the requirements.
If it is determined that required monitoring is taking place but is not being documented, we recommend that the Attorney General direct OJJDP to develop and enforce clear expectations regarding monitoring requirements. This policy should, among other things, require supervisory review to ensure that monitoring activities are being carried out and documented as prescribed. Further, grant managers’ performance work plans should include clear expectations and accountability for monitoring called for in grant managers’ monitoring plans as well as follow-up requirements to notify and assist grantees who consistently submit late or no progress or financial reports.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a copy of this report to the Attorney General for review and comment. In a September 19, 2001, letter, the Acting Assistant Attorney General commented on a draft of this report. Her comments are summarized below and are presented in their entirety in appendix II.
The Acting Assistant Attorney General said that overall, the report provides useful information in highlighting management and monitoring activities in need of improvement. Senior staff within OJJDP have been directed to address each of the issues identified in the draft report to focus on adjusting the nature and types of monitoring activities pursuant to these types of grants, thus ensuring that every active grant receives a programmatic assessment. OJJDP will also be considering ways to improve and streamline grant management monitoring activities to ensure that program monitoring efforts are documented and useful, including the development of a management information system and the use of private contractors to assist in some of these review functions.
The Acting Assistant Attorney General also said that the draft report could have gone further in acknowledging the impact of OJJDP’s enormous growth of the past 5 years on its ability to monitor recipients. We recognized that the number of active OJJDP discretionary grants has more than tripled over the past 5 years. However, it is OJJDP’s responsibility to initiate action, such as to request additional funding to meet its monitoring responsibilities or revise its monitoring efforts to comport with its available resources. In commenting on our May 1996 testimony, the OJJDP Deputy Administrator recognized that OJJDP’s heavy workload might have resulted in a lack of monitoring records, but that he would take steps necessary to improve records. However, our current review disclosed that similar monitoring problems persist.
Further, the Acting Assistant Attorney General said that we did not acknowledge OJJDP’s efforts in a major OJP-wide effort to improve grant monitoring practices and establish uniform policies. We recognized some of the efforts to review its monitoring practices, such as revising the OJP Handbook: Policies and Procedures for the Administration of OJP Grants and its recent embarking on an overhaul of its grant closeout practices to improve its monitoring records. Such efforts, we believe, should help address the monitoring problems we identified.
The Acting Assistant Attorney General concurred with our recommendation that grant managers’ performance work plans include clear expectations and accountability for monitoring. She said performance work plans put into effect after February 2001 for grant managers in OJJDP’s Special Emphasis Division more clearly reflect monitoring responsibilities and provide for better supervisory controls. Further, she noted that OJJDP’s Special Emphasis Division has put into place a new process designating one staff member with responsibility for file “audits” to help ensure that files are in order and that documentation of monitoring is in place.
As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution of it until 30 days from the date of this letter. At that time, we will send copies to the Chairmen and Ranking Members of the Senate Judiciary Committee; Senate Subcommittee on Youth Violence; House Committee on Education and the Workforce; House Subcommittee on Early Childhood, Youth and Families; Attorney General; and Director, Office of Management and Budget. We will also make copies available to others upon request.
If you or your staff have any questions about this report, please contact me or James M. Blume at (202) 512-8777. Key contributors to this report are acknowledged in appendix III.
section_title: Appendix I: OJJDP’s Discretionary Grant Award Process
paragraphs: Each year, OJJDP publishes a proposed program plan seeking public comment about proposed discretionary funding opportunities for activities covered under parts C and D (all national programs and gang-related activities) of the Juvenile Justice and Delinquency Prevention Act. The proposed plan is published in the Federal Register. Program announcements and solicitations for discretionary funding opportunities are published as they become available. The announcements and solicitations provide details about specific funding opportunities, including eligibility requirements and deadlines. These include funding opportunities for tribal youth and drug-free communities as well as funding opportunities associated with parts C and D. Once discretionary grant applications are submitted, OJJDP said that its contractor conducts an initial review to eliminate any applications that do not comply with the specified grant instructions. After ineligible grant applications are eliminated, OJJDP has a peer review panel, which consists of three or more outside experts who evaluate and rank OJJDP’s grant applications. After the peer review panel has ranked the applications, OJJDP’s grant managers do an in-depth review of applications and make formal recommendations regarding the funding of individual grants to the OJJDP Administrator.
As shown in figure 1, the key parts of OJJDP’s process for planning and announcing discretionary grants are issuance of a program plan, issuance of discretionary program announcements, and distribution of application kits.
As shown in figure 2, according to OJJDP officials, the key parts of the grant application and review process include initial screening of an application by the OJJDP contractor, peer review of applications, and the final selection of applications for funding by the OJJDP Administrator or Assistant Attorney General.
subsections: section_title: Program Plan Is to Describe Goals and Priorities
paragraphs: OJJDP’s proposed program plan describes OJJDP’s goals and priorities and the particular program activities it plans to carry out during the fiscal year under parts C and D of Title II of the OJJDP Act. OJJDP publicly announces its proposed program plan in the Federal Register. After a 45- day comment period, OJJDP is to publish a final plan that is to take into consideration comments received during the comment period.
According to an official, the OJJDP program plan is generally drafted by the Administrator and senior managers and based on (1) congressional and departmental priorities, (2) their knowledge and expertise, and (3) input during the course of the year from a variety of sources. These sources include program reviews by OJJDP staff and comments from practitioners in the field, officials from Justice and OJP bureaus and offices, and other federal agencies.
section_title: Program Announcements and Application Kits Are to Solicit Applicants for Programs
paragraphs: Following publication of the final program plan, OJJDP is to publish in the Federal Register a notice of discretionary grant programs and to announce the availability of an application kit for grants. The notice is to include the availability of funds, criteria for selection of grant applicants, procedures applicable to the submission and review of application for submission and review of applications for assistance, and information on how to obtain an application kit.
section_title: Peer Review Panels Are to Help Determine Which Grant Applications Are Funded
paragraphs: Upon receipt of discretionary grant applications, the OJJDP contractor does an initial review of each application, using an application review checklist. The objective of this initial review is to determine whether applications are complete and eligible for federal funding.
OJJDP contracts out the administrative arrangements (e.g., arranging panels and notifying reviewers) for the peer review of its discretionary grant applications. The contractor maintains a list of qualified consultants from which peer reviewers were selected by OJJDP. An OJJDP official estimated that there were 73 peer review panels in fiscal year 2000 for its 22 grant programs. The grant managers identify the particular expertise they want the peer reviewer to have. The contractor generates a list of potential reviewers with their resumes, background information, and other helpful information. The grant managers recommend peer reviewers on the basis of the contractor’s list and give the list to the Deputy Administrator for approval. After the Deputy Administrator approves the list, it is sent to the Administrator. Once the Administrator selects the peer reviewers, the contractor makes arrangements with the individuals selected and manages the peer review process.
According to an OJJDP official, consultants performing the peer reviews are reimbursed at a flat rate of $150 a day. OJJDP guidance requires that the grant managers use the following criteria to help achieve balance on the peer review panel.
Each reviewer should have expertise in or complementary to the subject area under review.
Where possible, the peer review panel should be composed of a mix of researchers, practitioners, and academicians.
Special attention should be paid to obtaining qualified representation from women and minority groups.
OJJDP’s peer review process is advisory and is to supplement and assist OJJDP’s consideration of applications. However, an OJJDP official noted that, in practice, results of the peer review panels’ consideration of application were almost always accepted.
section_title: Final Decision to Award Discretionary Grants
paragraphs: After completion of the peer review panels, OJJDP’s grant managers do an in-depth review of the peer review panels’ ranked list. The OJJDP grant managers make recommendations concerning applications to receive funding in a memorandum to the Administrator, and the Administrator is to make tentative grant selections. The OJJDP Administrator makes final awards for research and evaluation grants, while final awards for all training and technical assistance and demonstration grants are made by the Assistant Attorney General. After the applications are approved, OJP’s Office of the Comptroller does a financial review of these applications to determine whether the applicant has the necessary resources and integrity to account for and administer federal funds properly and whether budget and cost data in the application were allowable, effective, and reasonable.
section_title: Appendix II: Comments From the Department of Justice
paragraphs: The following are GAO’s comments on the Department of Justice’s September 19, 2001, letter.
subsections: section_title: GAO Comment
paragraphs: 1.  According to the OJP Handbook: Policies and Procedures for the Administration of OJP Grants (Feb. 1992), official grant files kept by the Office of the Comptroller Control Desk are to contain documents relating to each grantee, including progress and financial reports and site visit reports. In addition, for documentation to be readily available for examination, as required by the Comptroller General’s internal control standards, keeping them in the official grant file seems appropriate.
section_title: Appendix III: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Acknowledgments
paragraphs: In addition to those named above, R. Rochelle Burns, Jill Roth, Michele Fejfar, Sidney H. Schwartz, Michele J. Tong, Jerome T. Sandau, and Ann H. Finley made key contributions to this report.