Recovery Act: Department of Justice Could Better Assess Justice Assistance Grant Program Impact
Published: 2010-10-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Under the American Recovery and Reinvestment Act of 2009 (Recovery Act), the U.S. Department of Justice's (DOJ) Bureau of Justice Assistance (BJA) awarded nearly $2 billion in 4-year Edward Byrne Memorial Justice Assistance Grant (JAG) funds to state and local governments for criminal justice activities. As requested, GAO examined: (1) how Recovery Act JAG funds are awarded and how recipients in selected states and localities used their awards; (2) challenges, if any, selected recipients reported in complying with Recovery Act reporting requirements; (3) the extent to which states shared promising practices related to use and management of funds, and how, if at all, DOJ encouraged information sharing; and (4) the extent to which DOJ's JAG Recovery Act performance measures were consistent with promising practices. GAO analyzed recipient spending and performance data submitted as of June 30, 2010; interviewed officials in a nonprobability sample of 14 states and 62 localities selected based on the amount of their awards, planned activities, and their reported project status; assessed 19 JAG performance measures against a set of key attributes; and interviewed agency officials.
section_title: What GAO Found
paragraphs: Recipients of Recovery Act JAG funding in the 14 states GAO reviewed received more than $1 billion either through direct allocations from DOJ or through an indirect "pass-through" of funds that states originally received from the department. These recipients reported using their funds for a variety of purposes, though predominantly for law enforcement and corrections, which included equipment purchases or the hiring or retaining of personnel. More than half of the funding that state administering agencies (SAA) passed-through to localities was reported to be specifically for law enforcement and corrections activities, while localities receiving direct awards more often reported planning to use their funds for multiple types of criminal justice activities. Officials in all 14 states and 19 percent of localities in GAO's sample (12 of 62) said that without Recovery Act JAG funding, support for certain ongoing local law enforcement programs or activities would have been eliminated or cut. Overall, about $270 million or 26 percent of Recovery Act JAG funds had been reported as expended as of June 30, 2010, but the expenditure rates of funds awarded through SAAs showed considerable variation, ranging from 5 to 41 percent of SAA's total awards. State officials cited challenges in meeting quarterly Recovery Act reporting time frames. Officials from the majority of states in GAO's sample said that workload demands and personnel shortages made meeting Recovery Act deadlines within the prescribed reporting period difficult; however, all states reported that they were able to do so. States reported sharing information and promising practices related to JAG activities in a variety of ways and DOJ encouraged this sharing through a number of programs. More than half of state agencies in GAO's sample generally reported sharing promising practices or lessons learned on topics, such as grant management and administration, with other states and localities through participating in law enforcement and government association conferences, DOJ training, and Web postings, among other methods. DOJ established new performance measures to assess the Recovery Act JAG program and is working to refine them; however, these measures lack key attributes of successful performance assessment systems that GAO has previously identified, such as clarity, reliability, a linkage to strategic or programmatic goals, and objectivity and measurability of targets. Including such attributes could facilitate accountability and management's ability to meaningfully assess and monitor Recovery Act JAG's results. DOJ officials acknowledge that weaknesses exist and they plan to improve their performance measures. For example, the department already took initial steps to incorporate feedback from some states with regard to clarifying the definitions of some performance measures; however, its assessment tool lacks a process to verify the accuracy of the data that recipients self-report to gauge their progress. By including attributes consistent with promising practices in its performance measures, DOJ could be better positioned to determine whether Recovery Act JAG recipients' programs are meeting DOJ and Recovery Act goals. In addition, by establishing a mechanism to verify the accuracy of recipient reports, DOJ can better ensure the reliability of the information that recipients provide.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOJ (1) continue to revise Recovery Act JAG performance measures and consider, as appropriate, including key attributes of successful performance measurement systems, and (2) develop a mechanism to validate the integrity of self-reported performance data. DOJ concurred with these recommendations.

section_title: Letter
paragraphs: The recession that began in December 2007 caused states and localities  significant immediate fiscal pressures in the form of reduced tax revenues  and increased demand for certain programs, including criminal justice  programs. Under the American Recovery and Reinvestment Act of 2009  (Recovery Act), the existing Edward Byrne Memorial Justice Assistance  Grant (JAG) Program, which the Department of Justice’s (DOJ) Bureau of  Justice Assistance (BJA) administers, provided an additional $2 billion to  state and local governments through 4-year, formula-based grants. JAG  Program funds support local efforts to prevent and control crime and  improve the criminal justice system through activities such as drug  reduction and domestic violence prevention. The Recovery Act JAG  Program also attempts to meet the overall purposes of the Recovery Act  which include promoting economic recovery, making investments to  provide long-term economic benefits, and stabilizing state and local  government budgets to minimize and avoid reductions in essential  services.
The Recovery Act emphasizes the need for accountability and  transparency in the expenditure of Recovery Act funds and makes it a  central principle of the act’s implementation. Importantly, the  transparency that is envisioned for tracking Recovery Act spending and  results is an extensive undertaking for the federal government and  tracking billions of dollars that are being disbursed to thousands of  recipients is an enormous effort. The administration expects that  achieving this degree of visibility will be iterative, whereby both the  reporting process and the information recipients provide improve over  time and, if successful, could be a model for transparency and oversight  beyond the Recovery Act. Thus, Recovery Act JAG funding recipients are  required to meet federal reporting requirements that are in addition to the  requirements DOJ established for non-Recovery Act JAG program  recipients. Specifically, Recovery Act JAG recipients are required to  provide quarterly status reports on the amount and use of such funds and  information concerning jobs created or retained by the use of these funds.  Other than the additional reporting requirements, however, the Recovery  Act JAG program did not alter the structure, purpose, or funding allocation  methods of the preexisting JAG program.
Consistent with the preexisting program, states and localities can use their  Recovery Act JAG grant funds over a period of 4 years to support a range  of activities in seven broad statutorily established program areas: (1) law  enforcement; (2) prosecution and courts; (3) crime prevention and  education; (4) corrections; (5) drug treatment and enforcement; (6)  program planning, evaluation, and technology improvement; and (7) crime  victim and witness programs. Across the seven areas, recipients can use  JAG funds for state and local initiatives—which are generally designed to  improve a program, service, or system, or support training, personnel, or  equipment.
You requested that we examine the Recovery Act JAG Program. This  report addresses the following questions:    How are Recovery Act JAG funds awarded and how have recipients in  selected states and localities used their awards?
What challenges, if any, have selected Recovery Act JAG recipients  reported in complying with Recovery Act reporting requirements?
To what extent do states share promising practices related to the use  and management of Recovery Act JAG funds, and how, if at all, does  DOJ encourage information sharing?
To what extent are DOJ’s Recovery Act JAG performance measures  consistent with promising practices?
This report expands upon our May 2010 Recovery Act report, which  described selected states’ uses of JAG funding and accountability  provisions related to Recovery Act JAG, as well as our July 2009 Recovery  Act report, which discussed observations of Recovery Act JAG fund  obligations and planned uses of the funds. In July 2009, we reported that  the 16 states and the District of Columbia in our review had not obligated  their total Recovery Act JAG awards, in part because they were  determining how the funds would be used and passed through to local  entities. In our May 2010 report, we visited 7 of the states from our July  2009 sample and found that all 7 had obligated their Recovery Act JAG  awards and reported planned uses consistent with their states’ priorities  and BJA’s allowable uses of JAG funds.
To conduct our work for this review, we evaluated Recovery Act JAG  awards in a nonprobability sample of 14 states. The states we selected for  our review of Recovery Act JAG spending are a subset of a 16-state (plus  the District of Columbia) sample that we used for our earlier Recovery Act  work, but we did not include Florida, New Jersey, or the District of  Columbia since the DOJ Office of the Inspector General was already  engaged in audit work on the JAG program in these states. The awards to  the 14 states in this review accounted for approximately 50 percent of all  of the Recovery Act JAG funds provided. Where statements are attributed  to state and local officials, we did not analyze state and locality data  sources but relied on state and local officials and other state sources for  relevant state data and materials. We also tabulated and analyzed some  recipient-reported data submitted to Recovery.gov for the quarterly  reports that had been due as of June 30, 2010. We used these data because  they are the official source of Recovery Act spending data and determined  that they were sufficiently reliable for the purposes of this report. We  reviewed the relevant guidance DOJ provides to Recovery Act JAG  recipients on financial and program reporting as well as Recovery Act  guidance related to federal recipient reporting to understand federal  reporting requirements and associated time frames and interviewed DOJ  officials who administer the Recovery Act JAG program.
We also conducted interviews with officials in the state agencies that  administer Recovery Act JAG funds—known as State Administering  Agencies (SAA)—in the 14 states we selected for review. In addition, we  selected a nonprobability sample of 62 local law enforcement agencies and  other recipients receiving Recovery Act JAG funds within these 14 states  and conducted interviews with cognizant officials from those jurisdictions  that received the awards. These jurisdictions were selected based on  award amount, degree of project completion, planned use of funds, and  how they received their funds (either as passed-through funding from their  SAA or localities who received awards directly from DOJ—and in some  cases as part of disparate jurisdictions). Our interviews addressed the use  and perceived impact of Recovery Act JAG funds, program performance  measurement and reporting challenges, and the sharing of promising  practices. Findings from our nonprobability samples cannot be generalized  to all states and localities that were recipients of Recovery Act JAG funds;  however, our samples provided us with illustrative examples of uses of  funds, oversight processes, and reporting issues. Finally, we discussed  DOJ’s performance measurement efforts with DOJ staff and conducted an  assessment of the performance measures applicable to the Recovery Act  JAG activities commonly undertaken by the grant recipients in our sample  to assess the extent to which they contained elements consistent with  promising practices. Specifically, from DOJ’s 86 Recovery Act JAG  performance measures, we selected a nonprobability sample of 19 that  were (1) related to the largest share of reported Recovery Act JAG  expenditures across certain activity types and (2) most often reported by  the recipients in our sample. We then analyzed this sample against a set  of key characteristics that we have previously reported as being associated  with individual measures in successful performance measurement  systems. See appendix I for a more complete description of our  methodology and appendix II for a list and definition of the 19  performance measures we assessed.
We conducted this performance audit from January 2010 through October  2010 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: JAG Purpose Areas
paragraphs: According to DOJ officials, the JAG program provides states and localities  with federal funds to support all components of the criminal justice  system while providing a great deal of flexibility in how they do so.  Recovery Act JAG-funded projects may provide services directly to  communities or improve the effectiveness and efficiency of criminal  justice systems, processes, or procedures. Like non-Recovery Act JAG  funds, Recovery Act JAG awards are to be used within the context of  seven statutorily established areas. The seven statutorily established  areas and examples of how JAG funds may be used within these areas are  outlined in table 1 below.
Financial Requirements and Internal Controls  DOJ requires that all Recovery Act JAG award recipients establish and  maintain adequate accounting systems, financial records, and internal  controls to accurately account for funds awarded to them and their  subrecipients. Award recipients must also ensure that Recovery Act JAG  funds are accounted for separately and not commingled with funds from  other sources or federal agencies. If a recipient or subrecipient’s  accounting system cannot comply with the requirement to account for the  funds separately, then the recipient/subrecipient is to establish a system to  provide adequate fund accountability for each project that has been  awarded.
Recipient Reporting and Performance Measurement Requirements  All state and local Recovery Act JAG recipients are required to meet both  Recovery Act and BJA quarterly reporting requirements. The Recovery Act  requires that nonfederal recipients of Recovery Act funds (including  recipients of grants, contracts, and loans) submit quarterly reports, which  include a description of each project or activity for which Recovery Act  funds were expended or obligated, and an estimate of the number of jobs  created and the number of jobs retained by these projects and activities.  In particular, the Recovery Act requires recipients to report on quarterly  activities within 10 days of the end of each quarter. For Recovery Act JAG  grants, BJA has added language in the grant awards that requires that  grantees meet the federal reporting requirements and provides sanctions if  they do not. Because the Recovery Act JAG program includes a pass- through element, SAAs must gather the required data elements for all pass- through recipients during the same 10-day time frame in order to meet  their own reporting requirements.
Separately, BJA requires that states and those localities receiving their  funds directly through DOJ report on their progress in meeting established  performance measures related to funded activities. BJA also requires all  Recovery Act JAG recipients to submit an annual programmatic report  with narrative information on accomplishments, barriers, and planned  activities, as well as a quarterly financial status report as required by the  Office of Management and Budget (OMB). In early 2010, after a year-long  development and initial refinement period, BJA officially launched a new,  online Performance Measurement Tool (PMT) to improve upon its  previous grants management system and allow online performance  measurement data submission. BJA plans to use the PMT to help  evaluate performance outcomes in at least 13 grant programs, including  Recovery Act JAG. According to the Standards for Internal Control in the  Federal Government, activities need to be established to monitor  performance measures and indicators. Such controls should be aimed at  validating the integrity of performance measures and indicators—in other  words, ensuring they are reliably designed to collect consistent  information from respondents. BJA is also planning on using the PMT to  assess performance measurement data and direct improvement efforts in 5  additional programs by the end of 2010. However, given that grantees  were not required to submit their PMT reports until the second quarter of  fiscal year 2010, some grantees did not begin submitting their first  completed PMT reports until March 2010.
BJA requires Recovery Act JAG recipients to use the PMT for quarterly  reporting on their status in meeting the Recovery Act JAG program’s 86  individual performance measures, such as percent of staff who reported  an increase in skills and percent of Recovery Act JAG-funded programs  that have implemented recommendations based on program evaluation.
section_title: Recovery Act JAG Funds Are Awarded in Different Ways and Recipients Report Using Their Awards to Support Law Enforcement and Corrections Activities among Other Things
paragraphs: Recipients of Recovery Act JAG funding receive their money in one of two  ways—either as a direct payment from BJA or as a pass-through from an  SAA—and they reported using their funds primarily for law enforcement  and corrections. According to state officials from our sample states, more  than half of the funding that localities received as pass-through awards  from their SAAs was obligated specifically for law enforcement and  corrections support, while about a quarter of the funds that recipients of  direct awards received was dedicated exclusively to law enforcement.  Regardless of the source, officials in states and localities reported using  Recovery Act JAG funds to preserve jobs and activities that without  Recovery Act JAG funds would have been cut or eliminated; however,  expenditure rates across states in our sample showed considerable  variation.
subsections: section_title: Localities Receive Funding either Directly from BJA or as a Pass-Through from an SAA
paragraphs: BJA allocates Recovery Act JAG funds the same way it allocated non- Recovery Act JAG funds by combining a statutory formula determined by  states’ populations and violent crime statistics with a statutory minimum  allocation to ensure that each state and eligible territory receives some  funding. Under this statutory JAG formula, the total award allocated to a  state is derived from two sources, each given equal value: half of the  allocation is based on a state’s respective share of the U.S. population, and  the other half is based on the state’s respective share of violent crimes, as  reported in the Federal Bureau of Investigation’s (FBI) Uniform Crime  Report (UCR) Part I for the 3 most recent years for which data are  available. Of such amounts awarded to states, 60 percent of a state’s  allocation is awarded directly to a SAA in each of the states, and each SAA  must in turn allocate a formula-based share of these funds to local entities,  which is known as the “pass-through portion.”
BJA awards the remaining 40 percent of the state’s allocation directly to  eligible units of local government within the state. The eligible units of  local governments that receive direct awards from DOJ either get them  individually or as part of awards to “disparate” jurisdictions which jointly  use correctional facilities or prosecutorial services. In the cases of the  disparate jurisdiction awards, to qualify for funds, the units of local  government involved must submit a joint application to DOJ and sign a  memorandum of understanding (MOU) outlining how they will share  funds. They also are to determine amongst themselves which local  government will serve as the fiscal agent, and thereby be responsible for  reporting to DOJ on behalf of the others and ensuring that all members of  the disparate jurisdiction follow applicable federal financial guidance and  meet reporting requirements. The following figure illustrates the  participation of localities in a disparate jurisdiction award. In the example,  High Point city is the fiscal agent and Greensboro city and Guilford County  are both subrecipients.
The total awards that DOJ allocates directly to units of local government— the 40 percent share—are to be based solely on the local jurisdiction’s  proportion of the state’s total violent crime 3-year average based on  reports from the FBI’s UCR Part I. Units of local government that could  receive $10,000 or more after the Bureau of Justice Statistics (BJS)  analyzes the UCR data are eligible for a direct award from DOJ. Funds that  could have been distributed to localities through awards of less than  $10,000 are grouped together and then provided to the SAA. Under the  JAG program, SAAs and direct grant recipient agencies may draw down  funds from the Treasury immediately rather than requiring up-front  expenditure and documentation for reimbursement. Such funds are  required to be deposited into an interest-bearing trust fund and, in general,  any interest income that states and localities earn from the funds drawn  down is to be accounted for and used for program purposes.
Table 2 shows the total allocation of Recovery Act JAG funding across our  sample states, including the grant amounts BJA made directly to the SAAs  (the 60 percent share); the number of pass-through grants the SAAs made  in turn; and the grant amounts and number of grants BJA made directly to  localities (the 40 percent share). The 14 states in our sample received  $1,033,271,865 in JAG Recovery Act funds, which was more than half of  the funds awarded nationwide for the program.
Of the total of 1,338 direct awards that DOJ made to localities in the 14  states in our sample, approximately one-third of these direct awards, or  436, went to disparate jurisdictions and are split by agreement among the  designated jurisdictions. Under these arrangements, one jurisdiction  functions as the prime recipient and fiscal agent who is supposed to be  responsible for submitting all programmatic or financial reports on behalf  of the disparate group as well as monitoring other neighboring localities’  use of funds on activities covered by the grants. In our sample states,  while one-third of the total number of direct grant awards were made to  disparate jurisdictions, these arrangements accounted for 72 percent of  the funds DOJ awarded directly to local recipients. For example, in  Illinois, 100 percent of direct awards were provided to disparate  jurisdictions, and in 8 of the other 13 states DOJ awarded more than 70  percent of funds in this manner. Officials we met with in localities that  received funds under this type of arrangement reported that they provided  varying amounts of oversight in there role as fiscal agent. The DOJ  Inspector General has raised the oversight of subgrantee awards as an  issue for DOJ’s attention and has recommended that DOJ develop further  training for recipients; DOJ concurred with the recommendation. Table 3  summarizes the distribution of direct award funds to disparate  jurisdictions in our sample states.
The 14 SAAs in our sample received more than $630 million collectively as  their share of the Recovery Act JAG funds. JAG statutory provisions  require that each state pass-through no less than a specific designated  minimum percentage of the funds that they receive as subgrants to  localities, municipal governments, and nonprofit organizations. Among our  sample states, this mandatory pass-through percentage varied from a high  of 67.3 percent in California to a low of 35.5 percent in Massachusetts.  SAAs are also allowed to retain up to 10 percent of the funds that they  receive for administrative purposes. The completion of these pass-through  award processes occurred at different rates across the 14 states that we  sampled and resulted in some states expending their Recovery Act JAG  funds faster than others. As of June 30, 2010, the SAAs we reviewed had  made nearly all of their pass-through awards, with the exception of  Mississippi and Pennsylvania. In addition, many local pass-through  recipients reported that there was a time lag in being reimbursed by their  SAAs for funds that they had spent. Additional information on amounts  drawn down and expended is included in appendix IV.
subsections: section_title: SAAs and Localities Expended Their Awards at Varying Rates
paragraphs: According to Recovery.gov, the SAAs and localities that received grant  funds directly from DOJ in our sample of 14 states were awarded  approximately $1.028 billion in Recovery Act JAG funds. This amount  represents about 52 percent of the nearly $2 billion awarded to SAAs and  directly funded localities across the nation. As of June 30, 2010, the SAAs  and the directly funded localities in our sample expended over $270.7  million or about 26.4 percent of the total amount awarded. Recovery Act  JAG fund recipients may spend their respective awards over a 4-year  period.
As depicted in figure 2 below, in the 14 states in our sample, the  expenditure of Recovery Act JAG funds generally lags behind the amount  of funds awarded by the SAAs and drawn down. For example, as of June  30, 2010, California—whose SAA received the largest direct award in our  sample—had expended only about $6.6 million of the $135 million, or  nearly 5 percent, of JAG grant funds the state received. Texas reported  expending the most—more than $37 million—after combining  expenditures the SAA made independently with the expenditures made by  the more than 400 pass-through recipients.
California SAA officials stated they delayed in awarding JAG funds  because of the design of two new programs focused on probation and drug  offender treatment services that accounted for $90 million of the $135  million in grant funds the SAA received. As of June 30, 2010, 100 percent of  California’s subrecipients were finalized through grant award agreements,  but many projects have recently become fully operational resulting in the  slow expenditure of funds which are handled on a reimbursement basis.  In Pennsylvania, SAA officials said the state faced two challenges in  expending Recovery Act JAG funds quickly: (1) a state budget impasse,  which delayed the allocation of Recovery Act JAG awards; and (2)  Recovery Act JAG funding for state projects focused on technology costs,  which require lengthy procurement times. Further, they noted that state  pass-through funding to localities is recorded on a quarterly basis after  expenses are incurred, so the pace of expenditure could be somewhat  misleading.
Other SAA officials we contacted cited additional reasons for more slowly  expending Recovery Act JAG funds. For example, all of the SAAs we  contacted have procedures in place that require subrecipients to make  their purchases up-front with local funds and request reimbursement from  the SAA after documentation is received. Two states we contacted have  policies that restricted Recovery Act JAG funding to shorter time limits  with an option for renewal rather than providing localities authority to use  grants during the 4-year grant period applicable to the initial recipient of  the grant. In addition, 1 of the 14 SAAs had a preference to retain Recovery  Act JAG funds and expend funds gradually in longer-term projects, such as  technology improvements, as allowed during the 4-year grant period.
section_title: SAAs and Localities Reported Using Recovery Act JAG Funds to Preserve Jobs and Programs, and a Relatively Large Percentage of Both Pass- Through and Direct Funds Were Used to Support law Enforcement Activities
paragraphs: Using funds received through direct and pass-through awards, all states  reported using Recovery Act JAG funds to prevent staff, programs, or  essential services from being cut. In addition, local officials reported that  without Recovery Act JAG funding law enforcement personnel, equipment  purchases, and key local law enforcement programs would have been  eliminated or cut. SAAs reported that they passed through about 50  percent of their funds and collectively they planned to use the largest  share—about 30 percent, or almost $168 million—for law enforcement  purposes. Direct recipients reported that funds were most often to be used  for multiple purposes.
subsections: section_title: States and Localities Used Recovery Act JAG Funds to Help Preserve Jobs and Services
paragraphs: Officials from all states in our sample reported using Recovery Act JAG  funds to prevent staff, programs, or essential services from being cut. Also,  19 percent of localities in GAO’s sample, or officials in 12 of 62 localities,  provided specific examples of ongoing local law enforcement programs or  activities, such as juvenile recidivism reduction programs, prisoner re- entry initiatives, and local foot or bicycle patrols in high-crime  neighborhoods that would not have continued without the addition of  these funds. Table 4 provides some examples that state and local  recipients reported regarding how they used Recovery Act JAG funds to  help them preserve jobs and essential services.
section_title: SAAs Report That More than Half of Funding They Passed-Through Was Designated for Law Enforcement and Corrections, but Funded Activities Varied
paragraphs: SAAs reported that they awarded the largest share—about 30 percent, or  almost $168 million—for law enforcement purposes, such as hiring or  retaining staff who might otherwise have been laid off, or purchasing  equipment in direct support of law enforcement activities, as shown in  figure 3. In addition, SAAs reported awarding approximately 24 percent, or  more than $137 million, to support corrections programs or activities.  SAAs reported allocating the smallest share for crime victim and witness  programs, 2.1 percent or approximately $11.8 million.
Within the category of law enforcement, equipment expenditures spanned  a wide range of law enforcement gear, but vehicles and weapons  purchases were often reported. Frequent types of purchases included:    police cruisers;    weapons, such as TASERs, and ammunition;    communications devices, such as hand-held two-way radios, and  mobile laptops in police cruisers; and    safety equipment, such as protective vests and shields.
See appendix V for examples of selected equipment purchased with JAG  funds.
Overall, localities in 13 out of the 14 states we contacted reported using  Recovery Act JAG funds to maintain positions or pay officer overtime for  activities related to law enforcement. Individual SAAs, however, reported  obligating their Recovery Act JAG funds in a variety of ways as shown in  table 5. The percentages do not include the funds that the SAAs retained  for administrative purposes or funds not yet awarded.
Nearly all SAAs in our sample states, except for Iowa, which reported  using most of its funds to support drug enforcement activities, reported  using Recovery Act JAG funds to support law enforcement activities. With  the exception of Iowa, at the state level the share of Recovery Act JAG  funds used to support direct equipment purchases and personnel expenses  ranges from a high of 65.8 percent in Texas to a low of 1.7 percent in New  York.
Localities in more than a third of the states in our sample (5 of 14)  reported that uncertainties about the availability of future JAG funding  steered them toward one-time equipment purchases, such as the  procurement of license plate readers and in-car laptop computers, rather  than investments, such as hiring new personnel, that would require an  ongoing commitment of funds and whose sustainability could be  threatened when Recovery Act JAG funds expire.
In addition, officials in about a quarter of the localities in our sample (15)  discussed how they coordinate the use of their Recovery Act JAG funds  with resources that they received from other federal funding streams. For  example, the cities of Austin, Texas and Greensboro, North Carolina were  each waiting to receive a separate federal grant specifically for the  purpose of hiring police officers so that they could determine whether to  spend Recovery Act JAG funds to equip the officers once hired. See  figure 4 for an interactive map with additional information on Recovery  Act JAG funds purchases and activities in our sample states.
As shown in figure 5, data reported by direct recipient localities in the 14  states that we sampled indicate that they obligated the largest share— more than 63 percent, or over $256 million—for multiple purposes and  21.5 percent, or about $86.8 million, to directly support law enforcement  programs or activities. Program planning, evaluation, and technology  improvement funds, which accounted for approximately 8 percent of  spending, were primarily used to enhance communications equipment or  purchase computer hardware and software for all types of criminal justice  agencies and programs. Based on the information grantees reported to  Recovery.gov, the number of the projects reported has dropped slightly  over the last three reporting periods since projects that are completed  discontinue reporting. This was the case most often when funds were used  for discrete equipment purchases, such as law enforcement vehicles,  laptop computers in police cars, or weapons.
section_title: State Administering Agencies Cited Challenges Meeting Quarterly Recovery Act Reporting Time Frames
paragraphs: A majority of the SAA officials we interviewed said that workload demand  and personnel shortages made meeting Recovery Act mandated deadlines  within the prescribed reporting period difficult. Section 1512(c) of the  Recovery Act requires that each Recovery Act award recipient submit a  report no later than 10 days after the end of each quarter to the federal  awarding agency. In the case of Recovery Act JAG, the federal awarding  agency is DOJ. The Section 1512(c) report that Recovery Act recipients,  such as Recovery Act JAG recipients, are required to submit must contain  the following data: (1) the total amount of recovery funds received from  the federal awarding agency; (2) the amount of recovery funds received  that were expended or obligated to projects or activities; and (3) a detailed  list of all projects or activities for which recovery funds were expended or  obligated. All 14 SAAs we contacted said that they had the necessary  systems in place to account for Recovery Act JAG funds received and that  subrecipients were generally in compliance with their financial reporting  requirements.
Officials in 10 out of 14 SAAs in our sample specifically cited the Recovery  Act’s window of reporting no later than 10 days after the end of each  quarter as challenging. Officials in 8 out of 14 SAAs in our sample said that  meeting federal Recovery Act reporting requirements increased staff  workload and about one-third of the SAAs told us that personnel shortages  have created challenges in their abilities to specifically meet Recovery Act  reporting deadlines. For example, officials for one county in Colorado  noted that increased reporting responsibilities associated with Recovery  Act JAG grants resulted in one full-time staff member spending nearly 2  full work weeks on federal oversight and reporting requirements over a 5  ½-month time frame. Officials noted that the same individual spent 16  hours on reporting requirements for a non-Recovery Act JAG award and a  state pass-through award during the same time period. Furthermore,  officials in Texas, New York, and Mississippi said they required additional  personnel to manage Recovery Act awards and meet reporting  requirements. In addition, an official in one SAA also told us that because  of short data collection time frames they initially submitted incomplete  quarterly data and likely underreported the impact of the Recovery Act  JAG program in the first two quarterly 1512(c) reports.
While state and local officials we interviewed said that meeting the 1512(c)  report’s 10-day time frame remains challenging, none of the states in our  sample said that they were unable to meet the 1512(c) reporting deadline.  In addition, the number of direct award recipients that completed the  report has generally remained constant (around 800) over the three  reporting quarters from October 1, 2009, to June 30, 2010.
DOJ awarded over 70 percent, or more than $289 million of direct award  funds, to 436 disparate jurisdictions. DOJ guidance states that the recipient  (i.e., fiscal agent) in each disparate jurisdiction is responsible for  monitoring “subawards” and for “oversight of subrecipient spending and  monitoring of specific outcomes and benefits attributable to the use of  Recovery Act funds by its subrecipients.” DOJ guidance provides detailed  information on financial and accounting requirements for direct recipients  and subrecipients of DOJ grant programs. The guidance also states that  fiscal agents must implement and communicate a policy for reviewing  subrecipient data. DOJ guidance, however, does not provide instruction on  what a subrecipient monitoring or data policy should include; nor does it  state how outcomes and benefits tied to the Recovery Act should be  monitored. The DOJ Office of the Inspector General issued a report in  August 2010 which included the results of grant audits it performed across  12 state and local recipients of both Recovery Act and non-Recovery Act  JAG program funds. The Inspector General found that 7 of the 12 grant  recipients had deficiencies in the area of monitoring of subrecipients and  contractors. The Inspector General recommended that DOJ’s Office of  Justice Programs provide additional training and oversight of JAG  recipients to ensure that they establish policies and procedures for  monitoring subrecipients’ activities to provide reasonable assurance that  subrecipients administer JAG funds in accordance with program  guidelines. DOJ concurred with the recommendation that it provide  additional training and oversight over the monitoring of subrecipient  activities, and plans to review financial training course content to ensure  that proper internal control guidance on subrecipient monitoring is  included. DOJ anticipates developing a training module specific to  subrecipient monitoring by March 31, 2011.
section_title: States Reported Sharing Information and Promising Practices in a Variety of Ways and DOJ Encouraged This through a Number of Programs
paragraphs: All of the SAAs we contacted (14 of 14) reported that they generally shared  Recovery Act JAG information, promising practices, or lessons learned  with other states and localities using a variety of techniques. Furthermore,  DOJ had developed a number of programs that encourage the sharing of  information and promising practices.
State SAA officials told us that efforts to share information with one  another or amongst the localities in their jurisdictions include in-person  meetings, telephone calls, e-mail, Web postings, and/or hosting  conferences. In addition, the SAA officials told us they find value in  sharing information by attending DOJ training sessions and conferences  and participating in programs and events sponsored by associations, such  as the National Governors Association (NGA), the National Criminal  Justice Association (NCJA), and the Council of State Governments (CSG).  For example:    Texas officials developed an electronic state government grant  management and tracking system that they stated is helpful and  efficient in managing Recovery Act JAG funds. Texas officials told us  they shared the design of this online system with several states. In  addition, during BJA conferences and other national training  conferences, Texas officials noted that they took the opportunity to  discuss with other states the promising practices and lessons learned  related to grant management and the administration of JAG funds using  their system.
Colorado officials said that SAA staff made presentations at national  and regional conferences regarding the following: (1) grant  management and monitoring of state uses for effective grant  administration, (2) various programs the state has funded, and (3)  outcomes the state has achieved. SAA officials said that the state  encourages subgrantees that have demonstrated successful programs  to respond to requests for presenters at state and national conferences.  Officials told us that staff from three Colorado Recovery Act JAG  subgrantee projects made presentations at the NCJA Western Regional  Conference in April 2010. For example, Colorado officials told us that  one presentation involved the retraining of probation and parole  officers to reduce recidivism by working with other agencies in taking  an overall supportive approach to working with ex-offenders that  included assistance in such areas as housing, health, and finding work.
Ohio officials told us they take the initiative to contact other SAAs to  discuss and share experiences, lessons learned, and promising  practices regarding problems encountered in administering Recovery  Act JAG grants. They also said that NCJA provides SAAs with a forum  to share information and challenges associated with administering  recovery funds, which Ohio has leveraged. For example, they stated  that at the 2010 NCJA Mid-Western Regional Conference that Ohio  officials attended, there were sessions where SAAs shared experiences  about the administration of Recovery Act funds, as well as were  workshops on model projects funded through the Recovery Act.  According to Ohio officials, the information was helpful both in terms  of planning their own initiatives and in reaffirming decisions they had  made regarding Recovery Act and Recovery Act JAG programs.
Illinois officials told us that they hosted a 2-day criminal justice  planning summit in September 2010 for all state actors in the criminal  justice system including Recovery Act JAG practitioners, policymakers,  academics, and legislators. According to SAA officials, the focus of the  summit was on how to fight crime more effectively in a time of  diminishing resources by using the promising evidence-based practices.  State summit planners told us that both presentations by state and  national experts and workshops focused on implementing promising  practices, while the emphasis in follow-up work groups was on  producing a long-range criminal justice plan for the state of Illinois. In  addition, SAA officials told us that they share promising practices and  lessons learned by participating in regional training conferences, Web- based seminars, and/or informational conferences provided by OMB,  DOJ, as well as Illinois state agencies.
DOJ encourages information sharing through regional training  conferences, Web-sites, and Web-based clearinghouses. For example,  training meetings and Webinars provide a forum which states find valuable  for sharing information and promising practices, according to a majority of  (9 of the 14) states we interviewed. In addition, BJA has developed a Web  site that illustrates examples of successful and/or innovative Recovery Act  JAG programs. The Web site highlights JAG subgrantees and/or statewide  projects that BJA believes show promise in meeting the objectives and  goals of Recovery Act JAG. In particular, the site describes the planned  Illinois criminal justice information strategic planning initiative and  summit discussed above. Further, DOJ’s Office of Justice Programs is in  the process of developing an informational Web-based clearinghouse of  promising practice information for the criminal justice community through  a public Web site where researchers, grant applicants, and others may find  a list of model programs proven to be effective. According to DOJ officials,  it will also be a site that SAAs can use to help find best practices and  model programs, thereby funding discretionary programs that show  promise based upon evidence. While the focus of the DOJ information- sharing programs is broader than Recovery Act JAG, they offer methods  and mechanisms to share information related to program priorities, such  as law enforcement, corrections, and technology improvement. SAA  officials, in a majority of the states we interviewed, indicated that they  were supportive of these efforts.
In addition, national associations such as NGA, CSG, and NCJA encourage  states to share information and promising practices. The focus of these  programs is generally broader than Recovery Act JAG, but some  exclusively focus on Recovery Act JAG priorities such as law enforcement,  corrections, and technology improvement. For example, BJA has funded  NCJA to provide on-site training and technical assistance, Webinars, and  regional conferences, and creates and disseminates publications to assist  SAAs in developing their statewide criminal justice plans and ensure  effective use of Recovery Act JAG funds. NCJA also serves as an  information clearinghouse on innovative programming from across the  nation, and coordinates information sharing for the justice assistance  community.
section_title: DOJ’s Performance Measures Could Better Assess Progress Consistent with Characteristics of Successful Performance Measurement Systems
paragraphs: DOJ developed and implemented 86 new performance measures for the  Recovery Act JAG program in 2009 and continues to make efforts to  improve them, but the current set of performance measures varies in the  degree to which it includes key characteristics of successful performance  measurement systems. According to DOJ officials, these performance  measures are currently being refined in consultation with stakeholders,  such as SAAs and the external contractor hired to maintain the PMT. We  acknowledge that creating such measures is difficult, given that the  performance measurement system is under development, but until these  measures are refined, they could hinder the department’s ability to assess  and communicate whether the goals of the Recovery Act JAG program are  being achieved. In addition, states conveyed mixed perspectives about the  utility of DOJ’s performance measurement tool which enables recipients  to self-identify activities associated with their grant and then self-report on  the relevant set of performance measures under each activity. DOJ has not  yet completed development of a mechanism to verify the accuracy of this  recipient-reported information in the PMT.
subsections: section_title: DOJ’s Performance Measures Lack Some Key Characteristics of Successful Assessment Systems
paragraphs: From the more than 80 Recovery Act JAG performance measures, we  analyzed a nonprobability sample of 19 (see app. II) and found several  areas where the measures could better reflect the characteristics that our  prior work has shown to support successful assessment systems (see app.  III) For example, the 19 Recovery Act JAG performance measures we  reviewed generally lacked, in varying degrees, several key attributes of  successful performance measurement systems, such as clarity, reliability,  linkages with strategic or programmatic goals, objectivity, and the  measurability of targets. DOJ officials acknowledge the limitations of the  current system and are undertaking efforts to refine Recovery Act JAG  performance measures. As we have previously reported, performance  measures that evaluate program results can help decision makers make  more informed policy decisions regarding program achievements and  performance. By including key attributes of successful performance  measurement systems into its performance measure revisions, DOJ could  facilitate accountability, be better positioned to monitor and assess  results, and subsequently improve its grants management.
Table 6 describes 5 of 9 key characteristics of successful assessment  systems and the potentially adverse consequences agencies face when  omitting these attributes from their measurement design. These 5  characteristics—clarity, reliability, linkage to strategic goals, objectivity,  and measurable targets—are attributes that may be most effectively used  when reviewing performance measures individually. There are 4 others— governmentwide priorities, core program activities, limited overlap, or  balance—that are best used when reviewing a complete set of measures.  Since we selected a nonprobability sample of 19 measures that were most  closely associated with the majority of expenditures, we focused our  analysis on the 5 that could be applied to individual measures and did not  assess the sample for the other 4 attributes that are associated with an  evaluation of a full set of measures. Nevertheless, these 4 attributes also  can provide useful guidance when establishing or revising a set of  performance measures as a whole.
In conducting our analysis, we applied the 5 characteristics most  applicable to assessment of individual performance to the 19 measures in  our nonprobability sample. Our analysis found that 5 of the 19 measures  were clearly defined but the remaining 14 were not, which is inconsistent  with DOJ’s guidance to grant recipients for assessing program  performance. In particular, DOJ advises that states’ grant programs should  have performance measures with “clearly specified goals and objectives.”  In addition, 14 of the 19 measures were not linked to DOJ’s strategic or  programmatic goals. We also found that while 9 out of the 19 measures  were objective, 13 out of 19 were not reliable, and 17 out of the 19  measures did not have measurable targets.
In addition to our analysis, we provided a standard set of questions to  officials across our sample states seeking their perspectives on how  effectively the Recovery Act JAG performance measures evaluate program  results. These officials provided their comments about the PMT and raised  concerns about how the performance measures lack clarity, reliability, and  linkage to strategic goals.
subsections: section_title: Clarity
paragraphs: From our analysis we determined that 14 out of the 19 measures we  analyzed lacked sufficient descriptive detail to facilitate precise  measurement. For example, our analysis found that 1 of DOJ’s measures  associated with evaluating personnel activities is the “percent of  departments that report desired efficiency.” However, for this measure,  DOJ’s guidance based on the definition provided in the performance  measure lacks key data elements that would make the measure more  clear—namely, which departments should be included in the measure or  how states and localities should interpret “desired efficiency.”
In addition, officials we interviewed from 9 of the 14 SAAs in our sample  stated that DOJ’s Recovery Act JAG performance measures were unclear.  Some examples of states’ perspectives follow:  In particular, an official from the Texas SAA told us that Texas refined  its state data collection tool to clarify performance measure guidance  and eliminate instances where DOJ rejected data entries because the  measure was not clear. As another example, according to Texas  officials, one of the DOJ performance measures related to training is  “Other forms of training conducted during the reporting period.”  However, Texas state officials noted that BJA did not clarify whether  this measure would include non-Recovery Act training. As a result, the  Texas state data collection tool revised the performance measure for  better context and asked for the “the number of other forms of training  conducted during the reporting period and paid with ARRA JAG funds.”
Other state officials from Michigan and Georgia cited challenges in  understanding what is being asked by the 13 measures listed under the  activity type, “state and local initiatives.” In particular, one of these  states noted confusion and lack of clarity related to the measure,  “number of defined groups receiving services,” since in many instances  their initiatives were associated with equipment purchases, and it  would be difficult to determine who and how many benefited from a  new computer system or the acquisition of new ammunition, for  example.
Ohio and Pennsylvania state officials noted that DOJ uses terminology  such as “efficiency” and “quality” that is not clearly defined.
Officials we interviewed from another five states stated that they could not  understand whether the term “personnel” should include the entire agency  or department that was awarded the Recovery Act JAG grant or if it should  include only the portion of staff within a department that is directly  affected by the funding. When we discussed with DOJ officials our  concerns that the performance measure definitions at times lacked clarity,  they stated that each was defined, but that further work was being done to  solicit feedback from grantees on the measures and their definitions.  However, as we discussed above, our analysis determined that 14 out of  the 19 measures do not have clear definitions. DOJ officials noted that the  department hosts several training opportunities designed to provide  grantees opportunities for clarification, including two Webinars every  quarter and ongoing field training. DOJ officials also explained that they  hired an external contractor to operate the PMT Help Desk to provide  grantees guidance from 8:30-5:00 EST. However, officials from three states  we contacted noted that while the PMT Help Desk provided useful  technical assistance, the Help Desk provided limited guidance to clarify  the definition of performance measures. Therefore, officials from these  states reported being confused about what to report. In July 2010, we  reported that a measure not clearly stated can confuse users and cause  managers or other stakeholders to think that performance was better or  worse than it actually was.
section_title: Reliability
paragraphs: Our analysis showed that 13 out of 19 measures could lead to unreliable  findings because respondents could interpret and report on the measures  inconsistently. A performance measure is considered reliable when it is  designed to collect data or calculate results such that each time the  measure is applied—in the same situation—a similar result is likely to be  reported. Respondents’ inconsistent interpretation of the measures could  preclude using many of the measures as indicators of performance. For  example, we found that one measure: “the percent of departments that  report desired efficiency,” was measured and reported on differently by  different recipients. According to SAA officials in one state, different  police department units in a single large metropolitan area counted  themselves as separate departments, while according to SAA officials in  another state, all police department units were counted collectively as  one. In another state, SAA staff stated that BJA’s guidance document for  the Recovery Act JAG performance measures did not provide enough  instruction to ensure that agencies reported the correct data. For example,  the staff said they could not determine whether the PMT measure for “the  number of personnel retained with Recovery Act JAG funds during the  reporting period” was to include any personnel position paid for with   Recovery Act JAG funds during the reporting period, or to represent an  unduplicated number of personnel positions retained with Recovery Act  JAG funds during the reporting period. Given the confusion, the officials  sought and received guidance from the Help Desk on how to interpret and  report the measure. Further, officials from 4 of the 14 SAAs in our sample  expressed concern about possible inconsistent data entry among the  subrecipients of their pass-through grants. For example, officials from  Ohio noted that since subrecipients had their own interpretation of how to  report on the measures, they believed that there would be a lack of  consistency and reliability within the state as well as across all states once  BJA attempted to aggregate the responses.
In addition, a related issue is how DOJ validates the information states and  localities submit in order to ensure that the results the department reports  are accurate and reliable. We have previously reported that weaknesses in  monitoring processes for verifying performance data can raise concerns  about the accuracy of the self-reported data received from grantees. We  also reported that if errors occur in the collection of data or the  calculation of their results, it may affect conclusions about the extent to  which performance goals have been achieved. For example, self-reported  performance information that is not reported accurately could provide  data that are less reliable for decision making.
DOJ officials acknowledged that they have not verified the accuracy of  states’ and localities’ self-reported performance data. However, they told  us they have been meeting with their contractor to review a draft  verification and validation plan, but have not yet implemented a system to  verify and validate grantees’ performance data or implement data  reliability checks on the performance measures in the PMT. DOJ officials  also attributed their challenges to ensuring data integrity to limited  resources, stating that they lack adequate full-time staff to improve,  develop, and implement performance measures at this time. Specifically,  DOJ officials told us that they rely on a contractor because they have only  one staff person overseeing states’ and locals’ completion of the measures,  and improving and developing the tool.
Until a data verification process is in place, DOJ could experience  difficulty in ensuring performance results are reported reliably across  state and local grantee recipients.
section_title: Linkage to Programmatic or Strategic Goals
paragraphs: DOJ communicated specific Recovery Act goals, such as jobs created or  retained, to recipients; but did not provide information on how its  Recovery Act JAG performance measures aligned with programmatic or  strategic goals. Our analysis showed that 5 of the 19 measures were linked  to Recovery Act goals. For example, DOJ recently included a  performance measure for Recovery Act jobs reporting, which is the  “number of personnel retained with Recovery Act JAG funds.” The  remaining 14 measures lacked a clear linkage to any of DOJ’s goals. For  example, 1 of the measures related to the activity type “information  systems” is the “percent of departments that completed improvements in  information systems for criminal justice.” However, DOJ does not explain  how the performance measure for “improvements to information systems  for criminal justice” relates or links to agencywide goals. When we asked  DOJ officials to describe how the Recovery Act JAG performance  measures align with broader departmental goals, they explained that the  JAG authorizing legislation guides the states’ use of the funds within the  seven general purpose areas for JAG and that they do not link these  purpose areas to current year DOJ goals. However, DOJ officials explained  that Recovery Act JAG performance measures are linked to the  department’s strategic goal 2, “Prevent Crime, Enforce Federal Laws, and  Represent the Rights and Interests of the American People,” and strategic  goal 3, “Ensure the Fair and Efficient Administration of Justice.” DOJ  officials did not provide written documentation or guidance to Recovery  Act JAG recipients that explained this linkage to facilitate understanding  of how performance measures were being used consistently with DOJ’s  strategic and programmatic goals. Further, with the exception of Recovery  Act goals, officials from all 14 of the SAAs noted that they did not see a  direct linkage between the Recovery Act JAG performance measures and  DOJ’s overall agencywide goals.
As we have previously reported, successful organizations try to link  specific performance goals and measures to the organization’s overall  strategic goals and, to the extent possible, have performance goals that  will show annual progress toward achieving their long-term strategic  goals. In addition, we have previously reported that, without  performance measures linked to goals on the results that an organization  expects the program to achieve, several consequences can occur: (1)  managers may be held accountable for performance that is not mission  critical or at odds with the mission, and (2) staff will not have a road map  to understand how the measures support overall strategic and operating  goals.
section_title: Objectivity
paragraphs: In our assessment, we determined that 9 out of the 19 measures were objective. We previously reported that to be objective, performance  measures should (1) be reasonably free of significant bias; and (2) indicate  specifically what is to be observed, in which population or conditions, an d  in what time frame. An example of a BJA performance measure that w determined is objective is the measure “amount of Recovery Act JAG  funds used to purchase equipment and/or supplies during the re period.” This measure provides a specific time frame in which  expenditures for equipment and/or supplies must have occurred and  clearly explains that the amount of funds used for purchasing equi and/or supplies is what should be reported. An example of a BJA  performance measure that we determined lacks objectivity is the measure  the “percent of staff that directly benefit from equipment or supplies  purchased by Recovery Act JAG funds, who report a desired change in  their job performance.” We determined that this measure lacks objectivity because it does not indicate specifically what is to be observed, in wh d  population, and in what time frame, and is not free from opinion an judgment. For example, it requires those reporting to subjectively  determine which staff members directly benefit from an equipment or supplies purchase and which staff members do not. It also requires a  subjective determination of how the purchase of equipment or suppli affected a desired change in the performance of staff members who  directly benefited from the purchase. When we discussed the issue of  es  objectivity with DOJ they stated that BJA instructs grantees to only rep on BJA funded activities which occurred during the reporting period.  However, they conceded that the measures were open to interpretation  and that was a weakness, but suggested that that was the best option give the need to have universal measures that apply to a broad range of uses.  We do not agree that all the measures we reviewed were defined  sufficiently to prevent subjective interpretation.
In addition Texas officials expressed concern that DOJ will not be able to  obtain useful data from the PMT because of the subjective interpretation involved in responding to certain of the Recovery Act JAG performance  measures. For example, Texas officials identified responses to questions,  such as the “percent of departments that report desired program quality”  or “percent of staff who reported an increase in skills” as illustrative of the  kinds of questions that are open to wide interpretation based on the siz e of  the law enforcement organization and the classification of individuals  within the organization.
section_title: Measurable Targets
paragraphs: In our assessment, we determined that 17 out of the 19 measures lacke measurable targets. Among the 17, the absence of measurable targets  meant that outside of their original application the award recipients did  not have the opportunity to establish in advance what their target level o performance would be to allow for comparisons to actual performance  achieved for the reporting period covered. For example, in the measure  “Number of overtime hours paid with Recovery Act JAG funds,” BJA did not design the measure to allow award recipients  to specify their target  number of hours paid prior to receiving funding.
DOJ did recognize that the “project objectives,” i.e. the funded activities,  should be linked to meaningful and measurable outcomes associated the Recovery Act and the likelihood of achieving such outcomes be  assessed. For example, language in the Recovery Act JAG application  instructions requires that, where possible and appropriate, an estimate o the number of jobs created and retained be developed. In addition, the  Recovery Act JAG application for funds also requires that the narrative  include performance measures established by the organization to assess  whether grant objectives are being met and a timeline or plan to iden tify  when the goals and objectives are completed. However, measurable  targets against which to benchm the narrative.  ark results are not explicitly required in  As noted, two measures did include measurable targets, and as such will  facilitate future assessments of whether overall goals and objectives ar e  achieved because comparisons can be easily made between projected  performance and actual results. For example in these two measures—”the  change in the number of individuals arrested in a targeted group by crime  type” and “the change in reported crime rates in a community by crime  type”—DOJ provides a list of expectations, such as “we expected number  of individuals arrested to increase as a result of our efforts” or “we  expected number of individuals arrested to decrease as a result of our  efforts,” from which the department expects respondents to choose, to  facilitate comparison between the actual and expected number of arrests  and reported crimes during a particular quarter.
section_title: State Officials Had Vary Views of the PMT and Recovery A Measures
paragraphs: State officials had mixed perspectives on the PMT and Recovery Act  performance measures, with some critiquing it even as they acknowledged its utility in principle. For example, five SAAs noted that DOJ’s measures  were in development and acknowledged the difficulty for DOJ in  developing a tool that could be used nationwide for assessing outputs and  outcomes across multiple programs. They also were hopeful that the tool  would increase uniform program data collection and allow for meaningful  comparisons of data and outcomes across states and different  jurisdictions. State officials also had positive comments about DOJ’s Help  Desk and the staff who provided technical support for the use of the tool.
In addition, while eight states were silent on the issue, state officials from our remaining seven states stressed that reporting on the JAG Recovery  Act performance measures is time-consuming and duplicative of other  existing state performance measurement reporting systems. For examp officials from Colorado, Pennsylvania, and Illinois had concerns about  limited staff availability to monitor the workload associated with meeting  both Recovery Act and the PMT reporting requirements. Specifically,  officials stated that they have to monitor subrecipient activities and  provide monthly and quarterly information—as well as validate jobs  le,  reporting through payroll, expenses, and timesheets—to ensure job cou are calculated accurately and consistently. In other examples, officials  from Colorado and Iowa expressed concern that the PMT duplicates their  existing state performance measurement systems with similar measures  and results in duplication of effort.
In addition, the burden of complying with both BJA and state requirement led some states, such as Michigan, Ohio, and Texas, to eliminate some of  their state performance systems even though officials told us that they  believed that these systems measured performance outcomes better than  the PMT performance measures. For example, Michigan state officials  explained that their preexisting state quarterly performance reports  provided specific data on grant outcomes that were of interest to state  legislators and policymakers, and which are not included in the PMT  performance measures. In particular, Michigan’s state performance system  included measures related to drug courts, such as the number of drug-free  babies that are born to participants.
section_title: Conclusions
paragraphs: Under the Recovery Act, the JAG program made available nearly $2 billion  in additional funds for states and local governments, which states and  localities reported using primarily for law enforcement activities while  also maintaining some programs that would have been eliminated or cut.  Although reporting challenges remain with regard to the Recovery Act  itself, states and localities took steps to share information about promising  practices funded through JAG, and DOJ has measures in place to facilitate  such information sharing. In addition, the new performance measures that  DOJ has developed capture information on the use of Recovery Act JAG  funds.
However, while DOJ’s performance measures include attributes of  successful measures, further improvements are possible. Because the  Recovery Act JAG program supports a wide array of activities, as well as  the personnel to implement them, having clear performance measures that  allow grant recipients to demonstrate results would provide useful  information to DOJ regarding how Recovery Act JAG funds are being  used. Our previous work has identified key attributes of successful  performance measurement systems that would help assess progress and  make performance information useful for key management decisions.
According to the sample we reviewed, DOJ’s performance measures do  not consistently exhibit key attributes of successful performance  measurement systems, such as clarity, reliability, linkage, objectivity, and  measurable targets. Measures that are not clearly stated can confuse users  and cause managers or other stakeholders to think that performance was  better or worse than it actually was. The lack of data reliability can create  challenges in ensuring accurate information is recorded for performance  purposes. Further, the lack of measurable targets also limits the ability to  assess program performance and provides limited information to Congress  about the success of the program. Moreover, successful organizations try  to link performance goals and measures to the organization’s strategic  goals and should have performance goals that will show annual progress  toward achieving long-term strategic goals. In addition, by establishing a  mechanism to verify accuracy of self-reported data, DOJ can better ensure  reliability of information that is reported. By addressing attributes  consistent with promising performance measurement practices as it works  to revise its performance measures, DOJ could be better positioned to  determine whether Recovery Act JAG recipients’ programs are used to  support all seven JAG program purposes and are meeting DOJ and  Recovery Act program goals.
section_title: Recommendations
paragraphs: Recognizing that DOJ is already engaged in efforts to refine its Recovery  Act JAG performance measures in the PMT, we recommend that the  Acting Director of the Bureau of Justice Assistance take the following two  actions to better monitor Recovery Act JAG program performance and  demonstrate results through use of this instrument:  in revising the department’s Recovery Act JAG performance measures  consider, as appropriate, key attributes of successful performance  measurement systems, such as clarity, reliability, linkage, objectivity,  and measurable targets; and    develop a mechanism to validate the integrity of Recovery Act JAG  recipients’ self-reported performance data.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOJ for review and comments.  DOJ  provided written comments on the draft report, which are reproduced in  full in Appendix VII. DOJ concurred with the recommendations in the  report and stated that BJA plans to take actions that will address both of  our recommendations by October 1, 2011.  Specifically, in response to our  first recommendation that DOJ revise the Recovery Act JAG performance  measures to consider, as appropriate, key attributes of successful  performance measurement systems, DOJ stated that BJA is taking steps to  revise the Recovery Act JAG performance measures—in conjunction with  State Administering Agencies—and that it specifically will consider clarity,  reliability, linkage, objectivity, and measurable targets in redesigning its  performance measures. In response to our second recommendation  relating to data quality, DOJ stated that BJA will develop and implement a  mechanism to validate the integrity of Recovery Act JAG recipients’ self- reported performance data. DOJ also provided technical comments on a  draft of this report, which we incorporated as appropriate.
We are sending copies of this report to the Attorney General, selected  congressional committees, and other interested parties. In addition, the  report will be available at no charge on the GAO Web site at  http://www.gao.gov. Please contact David Maurer at (202) 512-9627 if you  or your staff has any questions concerning this report. Contact points for  our Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. Key contributors to this report are listed in  appendix VIII.
section_title: Appendix I: Scope and Methodology
paragraphs: This report addresses the following four questions: (1) How are Recovery  Act Justice Assistance Grant (JAG) funds awarded and how have  recipients in selected states and localities used their awards? (2) What  challenges, if any, have Recovery Act JAG recipients reported in  complying with Recovery Act reporting requirements? (3) To what extent  do states share promising practices related to the use and management of  Recovery Act JAG funds, and how, if at all, does the Department of Justice  (DOJ) encourage information sharing? (4) To what extent are DOJ’s  Recovery Act JAG performance measures consistent with promising  practices?
As agreed with your office, we focused our review on Recovery Act JAG  grants in a nonprobability sample of 14 states. The grants made to these  states included both direct awards that DOJ made to State Administering  Agencies (SAAs) and localities, as well as pass-through awards SAAs made  to localities. A portion of this work was done in conjunction with our  other Recovery Act reviews that focused on those 16 states, as well as the  District of Columbia that represent the majority of Recovery Act  spending. The 16 states included Arizona, California, Colorado, Florida,  Georgia, Illinois, Iowa, Massachusetts, Michigan, Mississippi, New Jersey,  New York, North Carolina, Ohio, Pennsylvania, and Texas. We selected  these states and the District of Columbia on the basis of federal outlay  projections, percentage of the U.S. population represented, unemployment  rates and changes, and a mix of states’ poverty levels, geographic  coverage, and representation of both urban and rural areas. Collectively,  these states contain about 65 percent of the U.S. population and are  estimated to receive about two-thirds of the intergovernmental assistance  available through the Recovery Act. However, for the purposes of this  report, we limited our scope to a subset of 14 of these states so as not to  duplicate ongoing work in the other 3 (Florida, New Jersey, and the  District of Columbia) that the DOJ Office of Inspector General was  conducting. The awards to these 14 states accounted for approximately 50  percent of all Recovery Act JAG funds provided.
To identify how recipients of direct and pass-through funds received and  used their Recovery Act JAG awards in selected states and localities, we  conducted in-person and telephone interviews with officials from SAAs in  all 14 states as well as officials from a nonprobability sample of 62  localities in these states. Where statements are attributed to state and local  officials, we did not analyze state and locality data sources but relied on  state and local officials and other state sources for relevant state data and  materials. We selected these localities based on the amount of their grant  awards, the activities that they were undertaking with grant funds,  whether they reported that they had completed 50 percent or more of their  grant activities according to their responses provided in Recovery Act  reporting, and how they received their funds (either as passed-through  funding from their SAA or received awards directly from DOJ—and in  some cases as part of disparate jurisdictions.) Our interviews addressed  the use and perceived impact of Recovery Act JAG funds, program  performance measurement and reporting challenges, and sharing of  promising practices. Also, we reviewed DOJ direct award data and SAA  pass-through awards in 14 SAAs. We also reviewed Recovery Act quarterly  reports from Recovery.gov (4th quarter 2009, 1st quarter 2010, and 2nd  quarter 2010) to identify additional information on the use of JAG funds.  Based on this information, we assigned the grants to one of the seven JAG  general purpose areas. For those where multiple purposes were indicated,  they were so identified. In cases where a purpose could not be identified  we placed it in the category of “not enough information.” We collected and  used these funding data because they are the official source of Recovery  Act spending. Based on our limited examination of the data thus far we  consider them to be sufficiently reliable for our purposes. Findings from  our nonprobability samples cannot be generalized to all states and  localities that were recipients of Recovery Act JAG funds; however, our  samples provided us with illustrative examples of uses of funds, oversight  processes, and reporting issues.
To determine the extent to which Recovery Act JAG recipients faced  challenges in complying with Recovery Act requirements, we interviewed  representatives from the 14 SAAs and 62 localities and asked them about  their experience with 1512(c) reporting requirements and Office of  Management and Budget (OMB) guidance. In addition, we reviewed our  previous reports that discuss Recovery Act recipient reporting issues.
To identify how states share promising practice information, and the  extent to which DOJ encourages information sharing, we conducted in- person and telephone interviews with representatives from all 14 of the  SAAs. We also reviewed DOJ information, interviewed DOJ officials, and  consulted reports from the National Criminal Justice Association, the  National Governors’ Association, and others that describe their  information-sharing activities.
To identify the extent to which DOJ’s performance measurement approach  is consistent with promising practices to assess progress, we interviewed  representatives from the 14 SAAs and 62 localities and asked them about  their experience with the Performance Measurement Tool (PMT). We also  discussed the PMT’s design and Recovery Act JAG performance measure  improvement efforts with DOJ staff. Further, we conducted a review of the  performance measures that were required for use under the Recovery Act  JAG activities commonly reported to have been undertaken by the grant  recipients in our sample. From the 86 Recovery Act JAG performance  measures under 10 activity types, we analyzed a nonprobability sample of  the 19 performance measures required under 4 of the activity areas  (Personnel, Equipment and Supplies, Information Systems for Criminal  Justice, and the category Outcomes for all Activity Types). We selected  these activity types and measures because they were the ones associated  with the largest share of reported Recovery Act JAG expenditures and  therefore most often encountered by the grant recipients. We then  assessed these measures against a set of key characteristics that we have  previously reported as being associated with promising practices and  successful performance measures we have identified in our previous  work. Some of the 9 key characteristics of successful performance  measures are attributes that may be most effectively used when reviewing  performance measures individually and some are best used when  reviewing a complete set of measures. Since we selected a nonprobability  sample of measures that was most closely associated with the majority of  expenditures, we focused our analysis most heavily on those attributes  that could be applied to individual measures—clarity, reliability, linkage to  strategic goals, objectivity, and measurable targets. We did not assess the  subset of 19 performance measures for the attributes of governmentwide  priorities, core program activities, limited overlap, or balance that are  associated with an evaluation of a full set of measures. To evaluate the  sample, four analysts independently assessed each of the performance  measures against attributes of successful performance measures  previously identified by GAO. Those analysts then met to discuss and  resolve any differences in the results of their analysis. In conducting this  analysis, we analyzed program performance measure information  contained in DOJ’s Performance Measurement Tool for American  Recovery and Reinvestment Act (Recovery Act - ARRA) and fiscal year  2009 Justice Assistance Grant Programs. We did not do a detailed  assessment of DOJ’s methodology for developing the measures, but looked  at the issues necessary to assess whether a particular measure met the  overall characteristics of a successful performance measure. We also  reviewed our previous reports that discuss the importance of performance  measurement system attributes and obtained information on the extent to  which such systems may impact agencies’ planning. The activity types  and number of measures selected are listed in table 7.
We conducted this performance audit from January 2010 through October  2010 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Recovery Act JAG Performance Measures
paragraphs: The following table contains the 19 Performance Measurement Tool  (PMT) performance measures that were required for use under the  Recovery Act JAG activity types commonly undertaken by the grant  recipients in our sample.
section_title: Appendix III: GAO Assessment of Whether DOJ’s Recovery Act JAG Performance Measures Possessed Certain Key Attributes
section_title: Appendix IV: Recovery Act JAG Award Drawdowns and Expenditures
paragraphs: Department of Justice (DOJ) records indicate that all 14 of the states in  our sample have drawn down the vast majority of their Recovery Act  Justice Assistance Grant (JAG) awards as of May 2010. Specifically, the  amounts drawn down range from less than 53 percent to almost 98  percent. Table 10 shows the amount and percentage of these funds that  have been drawn down and expended by State Administering Agencies  (SAAs), their subrecipients, and localities.
section_title: Appendix V: Examples of Use of Recovery Act JAG Funds for Equipment Purchases
paragraphs: The following table illustrates the types of equipment purchases recipients  within our 14 sample states have made using Recovery Act Justice  Assistance Grant (JAG) funds.
section_title: Appendix VI: Full Text for Figure 4 Map of SAAs and Planned Uses of Recovery Act JAG Awards by the Seven Allowable Program Categories across 14 Sample States
paragraphs: This appendix provides the full printed text of the interactive content in  figure 4 on page 22 in the body of the report.  Specifically, the following  figures describe planned uses of Recovery Act Justice Assistance Grant  (JAG) funds by each State Administering Agency (SAA) across our 14  sample states, which are listed in alphabetical order by state name.
According to state officials, without Recovery Act funds, the state faced budget cuts and  would have had to severely cut or discontinue at least half of the projects previously  funded with JAG money.  In particular, about $20.8 million in Recovery Act JAG funds  supported drug task forces and these drug task forces helped account for seizures of  847,665 grams of cocaine; 49,586 grams of heroin; 206,713 grams of methamphetamine;  and 305,082 pounds of marijuana in 2008.
Prosecution and courts $11,074,062 Program planning, evaluation and  technology improvement Crime victim and witness programs $1,265,348 According to state and local officials, Recovery Act JAG supported local gang and drug  reduction efforts, helped prevent human trafficking, facilitated a regional approach to  reducing methamphetamine production and distribution, and helped develop communications infrastructure.
Prosecution and courts $11,981,362  Crime prevention and education $835,678  Drug treatment and enforcement $44,254,215 Program planning, evaluation, and  technology improvement $131,213 Crime victim and witness programs $1,858,242 State officials noted that Recovery Act JAG helped maintain services in corrections, such  as support for problem youth and adult offenders and prison treatment programs, that  faced cuts given the state’s revenue shortfalls and budget reductions. In addition, local  officials stated that Recovery Act JAG helped support jobs and purchase equipment that  otherwise would have been eliminated or gone unfunded.
Prosecution and courts $1,972,990 Crime prevention and education $1,557,764 Drug treatment and enforcement $2,252,813  Program planning, evaluation, and  technology improvement $2,173,632  Crime victim and witness programs $381,322  According to state and local officials, Recovery Act JAG funds helped  support jobs, including retaining public safety personnel, and continue  delivery of services, such as drug court services, drug prevention, and  victims’ assistance. In addition, Savannah Police Department officials noted  that Recovery Act JAG funds were used to purchase a fully “patrol-certified”  Belgian Malinois breed canine to assist with recovery of stolen items,  searching for suspects and missing persons, and tracking narcotics.
Prosecution and courts $8,570,732  Crime prevention and education $185,797  Drug treatment and enforcement $233,962 Program planning, evaluation, and  technology improvement $1,468,394  Crime victim and witness programs $2,138,127  According to state and local officials, Recovery Act JAG funds helped purchase law  enforcement equipment, such as in-car video systems, that would have gone unfunded.  Support for other programs and services include, for example, support for overtime  wages of law enforcement agents, mentoring programs and drug treatment programs,  domestic violence programs, and specialty courts for nonviolent, repeat offenders.
Prosecution and courts $8,142,570 Crime prevention and education $5,671,274 Drug treatment and enforcement $452,965 Program planning, evaluation, and  technology improvement $4,122,386 Crime victim and witness programs Officials in Boone City, Iowa have used a portion of their Recovery Act JAG award to  institute cross-training of some employees in the city’s police and fire department. Under  the city’s public safety umbrella philosophy, some employees in the city’s police and fire  departments receive training in firefighting, emergency response, and law enforcement.   Those who receive this “cross-training” are known as public safety employees and can  respond to any type of incident where a police officer or firefighter is needed.  Officials  said that this type of cross-training has allowed the city to be able to do more with limited  resources.
Crime prevention and education $464,214  Drug treatment and enforcement $7,540,845  Program planning, evaluation, and  technology improvement $36,296  Crime victim and witness programs According to local officials, Recovery Act JAG funds helped supplement current state  public safety programs, retain jobs, and support core services, including supporting local  police departments through funding officer and crime analyst salaries in localities  adversely affected by local budget conditions.
Crime prevention and education $3,100,000 Program planning, evaluation, and  technology improvement $599,672 Crime victim and witness programs The Ottawa County Police Department used their Recovery Act JAG funds to purchase  equipment for law enforcement purposes. The department purchased a 20-foot patrol  boat, a fingerprint and jail mug-shot system, and global positioning satellite (GPS) tracker  devices. The patrol boat replaces a nearly 20-year-old boat in need of major maintenance.  The fingerprint and jail mug-shot system improves efficiency by enabling the department  to identify potential suspects with the state’s criminal databases. The GPS tracker devices  have helped the department in retrieving numerous stolen items and have provided  evidence useful in the prosecution of defendants.
Prosecution and courts $14,270,111 Crime prevention and education $1,067,558  Program planning, evaluation and  technology improvement $1,511,762  Crime victim and witness programs According to state and local officials, Recovery Act JAG funds helped support jobs to  manage the state JAG program, and supported local police departments by filling  positions, retaining other positions, and funding overtime to provide increased patrols and  surveillance. JAG funds will support a variety of programs including multijurisdictional task  forces, victim witness assistance, juvenile justice, drug courts, family violence, and  increased law enforcement training. Recovery Act JAG funds were also used to purchase  law enforcement equipment including crime lab equipment, computers, police cruisers,  and integrated software for patrol car laptops.
Prosecution and courts $825,000 Crime prevention and education $200,000  Drug treatment and enforcement $2,625,320 Program planning, evaluation, and  technology improvement $2,619,462  Crime victim and witness programs According to state and local officials, Recovery Act JAG funds supported the implementa- tion of recent drug law reform, including helping assistant district attorneys in reducing the  number of prison commitments, and continue recidivism pilot programs. New York City  officials estimate that JAG funds enabled New York City to retain 158 jobs that would  otherwise have been eliminated due to budget cuts, and helped create 51 new jobs.
Prosecution and courts $9,586,534  Drug treatment and enforcement $16,740,000 Program planning, evaluation, and  technology improvement $2,100,000 Crime victim and witness programs The Rutherford County Sheriff’s Department used its share of Recovery Act JAG funds to  purchase a tactical vehicle for their officers when responding to volatile situations. The  vehicle replaces an old 1986 Ford van that subjected officers to unnecessary risk and can  accommodate a team of up to 16 officers as well as store equipment, such as weapons  and bullet-resistant vests. The department also purchased portable surveillance equipment that can be thrown or rolled into a room and can provide a 360-degree view to  enable officers to identify any potential threats before entering a risky environment.
Prosecution and courts $577,951 Crime prevention and education $4,035,331  Program planning, evaluation, and  technology improvement $22,242,265 Crime victim and witness programs According to state and local officials, without Recovery Act JAG funds, law enforcement  agencies would have faced massive layoffs.  Additional funds were also used to support  the purchase of law enforcement equipment such as a license plate reader.
Prosecution and courts $2,805,401 Crime prevention and education $4,593,430 Drug treatment and enforcement $934,406  Program planning, evaluation, and  technology improvement $3,590,904  Crime victim and witness programs $2,676,585  State and local officials noted that Recovery Act JAG funds supported regional antidrug  task forces, juvenile programs, and initiatives such as records management improvement,  prisoner re-entry programs, and at-risk youth employment programs.
Prosecution and courts $3,626,239  Crime prevention and education $5,522,163 Program planning, evaluation and  technology improvement $4,838,141 Crime victim and witness programs $3,930,520  According to state and local officials, Recovery Act JAG funds largely helped support  equipment purchases and technology improvements, as well as support law enforcement  personnel, especially police officer overtime.
section_title: Appendix VII: Comments from the Department of Justice
section_title: Appendix VIII: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Joy Gambino, Assistant Director,  managed this assignment. Dorian Dunbar, George Erhart, Richard Winsor,  and Yee Wong made significant contributions to the work. Geoffrey  Hamilton provided significant legal support and analysis. Elizabeth Curda  and Cindy Gilbert provided significant assistance with design and  methodology. Adam Vogt and Linda Miller provided assistance in report  preparation, and Tina Cheng made contributions to the graphics presented  in the report.