Adult Drug Courts: Studies Show Courts Reduce Recidivism, but DOJ Could Enhance Future Performance Measure Revision Efforts
Published: 2011-12-09T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: A drug court is a specialized court that targets criminal offenders who have drug addiction and dependency problems. These programs provide offenders with intensive court supervision, mandatory drug testing, substance-abuse treatment, and other social services as an alternative to adjudication or incarceration. As of June 2010, there were over 2,500 drug courts operating nationwide, of which about 1,400 target adult offenders. The Department of Justice's (DOJ) Bureau of Justice Assistance (BJA) administers the Adult Drug Court Discretionary Grant Program, which provides financial and technical assistance to develop and implement adult drug-court programs. DOJ requires grantees that receive funding to provide data that measure their performance. In response to the Fair Sentencing Act of 2010, this report assesses (1) data DOJ collected on the performance of federally funded adult drug courts and to what extent DOJ used these data in making grant- related decisions, and (2) what is known about the effectiveness of drug courts. GAO assessed performance data DOJ collected in fiscal year 2010 and reviewed evaluations of 32 drug- court programs and 11 cost-benefit studies issued from February 2004 through March 2011.
section_title: What GAO Found
paragraphs: BJA collects an array of data on adult drug-court grantees, such as drug-court completion rates, and during the course of GAO's review, began expanding its use of this performance data to inform grant-related decisions, such as allocating resources and setting program priorities. For example, during September 2011, BJA assessed a sample of adult drug-court grantees' performance across a range of variables, using a new process it calls GrantStat. BJA developed recommendations following this assessment and is determining their feasibility. In addition, in October 2011, BJA finalized revisions to the performance measures on which grantees report. BJA's process of revising its performance measures generally adhered to key practices, such as obtaining stakeholder involvement; however, BJA could improve upon two practices as it continues to assess and revise measures in the future. First, while BJA plans to assess the reliability of the new measures after the first quarter of grantees' reporting, officials have not documented, as suggested by best practices, how it will determine if the measures were successful or whether changes would be needed. Second, should future changes to the measures be warranted, BJA could improve the way it documents its decisions and incorporates feedback from stakeholders, including grantees, by recording key methods and assumptions used to guide its revision efforts. By better adhering to best practices identified by GAO and academic literature, BJA could better ensure that its future revision efforts result in successful and reliable metrics--and that the revision steps it has taken are transparent. In the evaluations that GAO reviewed, drug-court program participation was generally associated with lower recidivism. GAO's analysis of evaluations reporting recidivism data for 32 programs showed that drug-court program participants were generally less likely to be re-arrested than comparison group members drawn from criminal court, with differences in likelihood reported to be statistically significant for 18 of the programs. Cost-benefit analyses showed mixed results. For example: (1) Across studies showing re-arrest differences, the percentages of drug- court program participants re-arrested were lower than for comparison group members by 6 to 26 percentage points. Drug court participants who completed their program had re-arrest rates 12 to 58 percentage points below those of the comparison group. (2) GAO's analysis of evaluations reporting relapse data for eight programs showed that drug-court program participants were less likely than comparison group members to use drugs, based on drug tests or self- reported drug use, although the difference was not always significant. (3) Of the studies assessing drug-court costs and benefits, the net benefit ranged from positive $47,852 to negative $7,108 per participant.
section_title: What GAO Recommends
paragraphs: GAO recommends that BJA document key methods used to guide future revisions of its performance measures for the adult drug-court program. DOJ concurred with GAO's recommendation.

section_title: Letter
paragraphs: Drug court programs were established beginning in the late 1980s as a  local response to increasing numbers of drug-related cases and  expanding jail and prison populations nationwide. A drug court is a  specialized court-based program that targets criminal offenders who have  alcohol and other drug addiction and dependency problems. Drug courts  have implemented deferred prosecution or post-adjudication case- processing approaches, or have blended both in their organizational  structures. In drug courts using deferred prosecution, defendants waive  rights to a trial and enter a treatment program shortly after being charged;  those who subsequently fail to complete the treatment program have their  charges adjudicated, while those who complete the program are not  prosecuted further, or have their charges dismissed. In post-adjudication  case processing, defendants are tried and convicted, but either have  deferred sentences or suspensions of incarceration until they complete or  withdraw from the treatment program. The first approach offers  individuals the opportunity to obtain treatment and avoid the possibility of  a felony conviction, while the second provides a rehabilitation incentive  because treatment progress is factored into the sentencing determination.  As of June 2010, there were over 2,500 drug courts operating throughout  the United States, of which about 1,400 of these target adult offenders.  Drug courts are generally based on a comprehensive model involving    monitoring (e.g., drug testing) and supervision;    graduated sanctions and incentives; and  treatment services.
The Department of Justice (DOJ), through its Office of Justice Programs’  (OJP) Bureau of Justice Assistance (BJA), administers the Adult Drug  Court Discretionary Grant Program, which provides financial and  technical assistance to states, state courts, local courts, units of local  government, and Indian tribal governments to develop and implement  drug treatment courts. The total amount BJA has awarded in grants  through the program increased from about $2 million in fiscal year 2006 to  $29 million in fiscal year 2010, and the number of grants it has awarded  during the same period increased 588 percent. Pursuant to the  Government Performance and Results Act (GPRA), DOJ requires  applicants that receive funding through the program to provide data that  measure the results of their work.
In April 2002, we reported that DOJ had not sufficiently managed its  efforts to collect performance measurement and outcome data from  federally funded drug courts. We recommended that DOJ take actions to  address these concerns, and DOJ agreed with our recommendations and  took actions in response. Appendix I provides information on the status of  these recommendations. In February 2005, we studied drug courts again  and reported that in most of the 27 drug-court program evaluations we  reviewed, adult drug-court programs led to recidivism reductions—that is,  reductions in new criminal offenses—during periods of time that generally  corresponded to the length of the drug court program. We also reported  that the evidence about the effectiveness of drug court programs in  reducing participants’ substance-use relapse was limited and mixed.
This report responds to the Fair Sentencing Act of 2010, which directed  GAO to report on drug court programs. We briefed your offices on our  preliminary results on July 18, 2011. This report includes our final results  related to the following questions: (1) What data does DOJ collect on the  performance of federally funded adult drug courts, and to what extent has  it used these data in making grant related decisions? And (2) What is  known about the effectiveness of adult drug courts in reducing recidivism  and substance-abuse relapse rates, and what are the costs and benefits  of adult drug courts? In addition, appendix I of this report provides  information on the extent to which DOJ has addressed the  recommendations that we made in 2002 regarding drug court programs.
To address the first question, we analyzed: the reporting guidance and  requirements that BJA provided in fiscal years 2007 through 2011 to  grantees applying for Adult Drug Court Discretionary Grant Program  funds; BJA-generated grantee performance data reports from October to  December 2010; and BJA’s guides for managing grants and enforcing  grantee compliance that were issued in fiscal year 2011. We selected  2007 as the starting point for our review because BJA implemented its  Performance Measurement Tool (PMT)—an online reporting tool that  supports BJA grantees’ ability to collect, identify, and report performance  measurement data activities funded by the award—in fiscal year 2007.  We also reviewed our prior reports and internal control standards as well  as other academic literature regarding effective performance  management practices. Further, we interviewed cognizant BJA officials  about the extent to which they use grantees’ performance data when  engaging in these management activities, any challenges faced with  ensuring grantee compliance, ongoing efforts to revise program  performance metrics, and the extent to which BJA’s revisions incorporate  best practices we previously identified.
To address the second question, we conducted a systematic review of  evaluations of drug court program effectiveness issued from February  2004 through March 2011 to identify what is known about the effect of  drug court programs on the recidivism of and relapse of drug involved  individuals as well as the costs and benefits of drug courts. We also  reviewed DOJ’s National Institute of Justice (NIJ)-funded Multi-Site Adult  Drug Court Evaluation (MADCE), a 5-year longitudinal process, impact,  and cost evaluation of adult drug courts that was issued in June 2011, a  summary of which we provide in appendix II. We identified the universe  of evaluations to include in our review using a three-stage process. First,  we identified evaluations by searching databases and Web sites. Second,  we selected evaluations of adult drug court programs in the United States  that report recidivism, substance use relapse, and/or costs and benefits.  Third, we screened the selected studies to determine whether each met  criteria for methodological soundness based on generally accepted social  science principles or cost-benefit analysis criteria. From more than 260  studies in our initial group, we assessed the findings of 44 studies that  met our criteria and reported on the effectiveness of 32 drug court  programs or sets of programs. See appendix III for additional details on  our scope and methodology.
We conducted this performance audit from November 2010 through  December 2011 in accordance with generally accepted government- auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our objectives.
subsections: section_title: Background
paragraphs: Drug court programs are designed to address the underlying cause of an  offender’s behavior—alcohol, drug addiction, and dependency problems.  Drug court programs share several general characteristics but vary in  their specific policies and procedures because of, among other things,  differences in local jurisdictions and criminal justice system practices. In  general, judges preside over drug court proceedings, which are called  status hearings; monitor offenders’ progress with mandatory drug testing;  and prescribe sanctions and incentives as appropriate in collaboration  with prosecutors, defense attorneys, treatment providers, and others.  Drug court programs vary in terms of the substance-abuse treatment  required. However, most programs offer a range of treatment options and  generally require a minimum of 1 year of participation before an offender  completes the program.
Practices for determining defendants’ eligibility for drug court participation  vary across drug court programs, but typically involve screening  defendants for their criminal history, current case information, whether  they are on probation, and their substance use, which can include the  frequency and type of use, prior treatment experiences, and motivation to  seek treatment. In 2005, we reported that based on literature reviewed,  eligible drug-court program participants ranged from nonviolent offenders  charged with drug-related offenses who had substance addictions, to  relatively medium risk defendants with fairly extensive criminal histories  and who had failed prior substance-abuse-treatment experiences.  Appendix IV presents additional information about the general  characteristics of drug court programs. As shown in appendix V, BJA, in  collaboration with the National Association of Drug Court Professionals  (NADCP), identified The Key Components, which describes the basic  elements that define drug courts and offers performance benchmarks to  guide implementation.
BJA administers the Adult Drug Court Discretionary Grant Program to  provide financial and technical assistance to states, state courts, local  courts, units of local government, and Indian tribal governments to  develop and implement drug treatment courts. Through the Adult Drug  Court Discretionary Grant Program, BJA offers funding in four broad drug- court grant categories. See appendix VI for a more detailed discussion on  each of the following grant categories.
Implementation grants: Available to jurisdictions that have completed  a substantial amount of planning and are ready to implement an adult  drug court.    Enhancement grants: Available to jurisdictions with a fully operational  (at least 1-year) adult drug court.    Statewide grants: Available for two purposes: (1) To improve,  enhance, or expand drug court services statewide through activities  such as training and/or technical assistance programs for drug court  teams and (2) To financially support drug courts in local or regional  jurisdictions that do not currently operate with BJA Adult Drug Court  Discretionary Grant Program funding.
Joint grants: In fiscal year 2010, BJA, in collaboration with the  Department of Health and Human Services, Substance Abuse and  Mental Health Services Administration (SAMHSA), offered a joint  grant program for the enhancement of adult drug court services,  coordination, and substance-abuse treatment capacity.
From fiscal years 2006 through 2010, Congress appropriated about $120  million for DOJ’s administration of all drug court programs. Of this  amount, $76 million was used for the Adult Drug Court Discretionary  Grant Program, which includes funding provided to grantees through the  previously mentioned grant categories. The grant award totals for the  Adult Drug Court Discretionary Grant Program increased from $2 million  in fiscal year 2006 to $29 million in fiscal year 2010. Correspondingly,  the number of Adult Drug Court Discretionary Grant Program awards  increased from 16 in fiscal year 2006 to 110 in fiscal year 2010—an  increase of 588 percent, as shown in figure 1.
With regard to drug courts’ effectiveness, however, drug courts have  been difficult to evaluate because they are so varied, and the resources  required to conduct a study that would allow conclusions about the  effectiveness of drug courts can be substantial. In particular, while drug  courts generally adhere to certain key program components, drug courts  can differ in factors including admission criteria, type and duration of drug  treatment, degree of judicial monitoring and intervention, and application  of sanctions for noncompliance. In February 2005, we studied drug courts  and reported that in most of the 27 drug-court program evaluations we  reviewed, adult drug court programs led to recidivism reductions during  periods of time that generally corresponded to the length of the drug court  program. Several syntheses of multiple drug court program evaluations,  conducted in 2005 and 2006, also concluded that drug courts are  associated with reduced recidivism rates, compared to traditional  correctional options. However, the studies included in these syntheses  often had methodological limitations, such as the lack of equivalent  comparison groups and the lack of appropriate statistical controls.
section_title: BJA Is Expanding Use of Grantee Performance Data but Could Enhance Processes as It Continues to Refine Performance Measures
subsections: section_title: BJA’s Ongoing Data Collection Efforts
paragraphs: BJA collects an array of performance data from its adult drug court  grantees through its Performance Measurement Tool (PMT) and OJP’s  Grants Management System (GMS). Since fiscal year 2008, BJA has  required grantees to submit quantitative performance data on a quarterly  basis and qualitative performance information on a semi-annual basis.  The quantitative information grantees submit to BJA varies depending on  the type of grant awarded. For example, information that BJA can  calculate based on what Implementation grantees have been required to  submit quarterly includes “the percent of drug court participants who  exhibit a reduction in substance use during the reporting period,” “the  percent of program participants who re-offended while in the drug court  program,” and “the number and percent of drug court graduates.”  Information that BJA can calculate based on what Enhancement grantees  have been required to submit includes “the increase in units of substance- abuse treatment services” and “the percent increase in services provided  to participants.” In addition to the quarterly reporting of quantitative  performance data, all adult drug court grantees must submit progress  reports semi-annually. As part of these progress reports, grantees provide  qualitative or narrative responses to seven questions. Table 1 shows the  seven questions to which grantees must submit narrative responses  when completing their semi-annual reports.
BJA officials told us that grant managers regularly review individual  grantees’ quarterly performance data and semi-annual progress reports  and use this information to determine whether additional training or  technical assistance could improve their performance. However,  according to BJA officials, resource constraints in the past had prevented  staff from fully analyzing the performance data BJA collects from all adult  drug court grantees—specifically the analysis of grantees’ answers to the  seven narrative questions—to identify more effective program  approaches and processes to share with the drug court community. In  early fiscal year 2011, BJA officials initiated a new process called  GrantStat to maximize the use of performance information by leveraging  the resources of other BJA divisions, BJA’s training and technical  assistance partners, its contractor, and other key stakeholders.  GrantStat provides an analytical framework to assess grantee  performance data and other relevant information on a semi-annual basis  to determine the effectiveness of the grant programs in BJA’s portfolio.
In September 2011, BJA officials applied GrantStat to a review of the  Adult Drug Court Discretionary Grant Program. As part of the process,  they collected, reviewed, and analyzed performance data and other  relevant information from a cohort of Implementation grantees to  determine the overall effectiveness of the adult drug court program and to  identify grantees that might need additional technical assistance to  improve their outcomes. BJA officials told us that as part of the GrantStat  review, they and their technical-assistance provider’s staff reviewed  selected Implementation grantees’ responses to the seven narrative  questions and discussed common issues they each identified. For  example, BJA identified that a number of grantees had lower-than- expected capacity because drug court stakeholders (e.g., district  attorneys) were referring fewer drug-involved defendants to these drug  courts. BJA also reported reviewing and discussing other qualitative  information, such as the training and technical assistance provider’s site- visit reports, to determine grantees’ fidelity to the 10 key components.  BJA officials acknowledged that prior to GrantStat, they had not  leveraged the summary data that its technical assistance providers had  previously compiled from grantees’ narrative responses to these seven  questions and indicated that future iterations of GrantStat would continue  to include both qualitative and quantitative performance data reviews.
Our prior work has emphasized the importance of using performance data  to inform key decisions and underscored that performance measures  can be used to demonstrate the benefits of a program or identify ways to  improve it. In addition, we also have reported that effective performance  measurement systems include steps to use performance information to  make decisions. In doing so, program managers can improve their  programs and results. Recognizing that BJA is working through  GrantStat to improve its use of performance data in managing the drug  court program, we identified six management activities for which  performance information can be most useful to decision makers and  benchmarked BJA’s practices against them. The six activities are:   (1) setting program priorities, (2) allocating resources, (3) adopting new  program approaches, (4) identifying and sharing with stakeholders more  effective program processes and approaches, (5) setting expectations for  grantees, and (6) monitoring grantee performance. See appendix VII for  the definition of the six management activities. As illustrated in table 2,  BJA has current and planned efforts underway across all six activities.
According to BJA officials, after the GrantStat review, they identified  trends and developed several potential findings and action items for  program design changes. However, BJA officials added that since the  action items originated from GrantStat’s first review, they are not  implementing them immediately. Instead, BJA plans to evaluate the  action items over the next 6 months to ensure they are feasible and  effective alternatives for improving grantee outcomes. We are  encouraged by BJA’s recent efforts to regularly analyze grantee  performance data to determine whether the program is meeting its goals.  We also are encouraged that BJA is using this information to better inform  its grant-related management activities, such as setting program  priorities, identifying and sharing effective processes and approaches,  and setting expectations for grantees.
section_title: BJA Recently Revised Its Drug Court Performance Measures
paragraphs: During the course of our review, BJA revised its adult drug court program  performance measures to improve their reliability and usefulness. BJA  provided us with the revised measures on October 28, 2011. According to  BJA officials, unclear definitions of some of the previous measures  confused grantees about what data elements they were expected to  collect. For example, officials told us that grantees may have been  confused with how to measure “the number of participants admitted” and  “the number of drug court participants.” Specifically, BJA officials added  that their analysis of several years of data shows that some grantees  reported the same number for these two measures, some grantees  reported a higher number than were admitted, a few grantees reported a  lesser number for the number of participants than the number admitted,  and some grantees reported these two measures in each of these three  ways over multiple reporting periods. According to BJA officials, such a  wide degree of variability made these measures unreliable, and BJA was  thus hindered from comparing grantee performance data across grantee  cohorts.
BJA’s performance measure revisions resulted in the following:    All grantees are required to report on “participant level” measures.  Examples of these measures include the demographic make-up of  their drug court participant populations, the amount of service  provided to their participants, and the geographic location of their drug  courts;    Enhancement, Joint, and Statewide grantees are required to report on  participant level outcomes, such as graduation rates, to ensure  consistency with measures BJA collects from Implementation  grantees;    Measures previously excluded from the PMT, such as retention rates  and outcomes of participants once they complete the drug court  program, are now included;    BJA has established two sets of benchmarks as points of reference  against which to gauge grantees’ performance. The first set of  benchmarks requires a comparison of grantees’ performance against  averages of drug court performance derived from research. The  second set of benchmarks requires a comparison of grantees’  performance to historical performance data reported to BJA by adult  drug court grantees; and    BJA revised the descriptions and the definitions of the measures to  help ensure their clarity.
To revise the performance measures, BJA officials consulted with  technical assistance providers and a drug court researcher to discuss  possible improvements to the performance measures, reviewed drug  court literature, and reviewed and analyzed BJA grantees’ clarification  and information requests to identify the most common problems adult  drug court grantees historically experienced submitting performance  information to BJA. In addition, BJA obtained comments on the  proposed measures from BJA staff and other DOJ stakeholders, as well  as Enhancement, Implementation, Joint, and Statewide grantees. BJA  officials also invited all current grantees to participate in four  teleconferences to obtain their feedback on the feasibility of collecting and  reporting the new measures and their suggestions to improve the clarity  of the measures’ definitions and descriptions. BJA officials finalized the  new measures in October 2011 and plan to closely monitor grantees’  performance data submissions to ensure the reliability and usefulness of  the measures and then revise as necessary after the first reporting  period. BJA officials also stated that they expected to review the  measures’ overall reliability and validity after the first reporting period— October 1, 2011, through December 30, 2011.
BJA officials reported that the revised measures will strengthen the  reliability and improve the usefulness of grantee performance data in  making grant-related decisions. For example, BJA officials stated that  reliable and useful data would help them to identify the most effective  grantees and common characteristics these courts share to inform the  types of drug courts the officials choose to fund in future grant  solicitations. BJA officials also reported that as a result of the revision,  they expect to be able to conduct more sophisticated analyses using  GrantStat that are needed to inform grant-related decisions. For example,  BJA officials told us that implementing benchmarks and participant level  measures will enable the agency to compare similar drug courts (e.g.,  large-urban jurisdictions of similar size, demographic make-up, and  geographic context) to one another and across jurisdictions, thereby  improving BJA’s understanding of grantees’ impact on the populations  they serve.
section_title: BJA Could Enhance Two Key Practices as It Continues to Review and Revise Its Adult Drug Court Performance Measures
paragraphs: BJA’s process to revise its performance measures generally adhered to  some of the key practices that we have identified as important to ensuring  that measures are relevant and useful to decision-making. These key  practices included obtaining stakeholder involvement and ensuring that  the measures have certain key attributes, such as clarity. The key  practices also describe the value of testing the measures to ensure that  they are credible, reliable and valid and documenting key steps  throughout the revision process. However, BJA could take actions to  improve its efforts in these two areas. For instance, BJA officials told us  that after the grantees’ first reporting period concludes, they plan to assess  the data that grantees submitted to ensure that the measures produce  reliable and useful data over at least the first quarter of fiscal year 2012.  They stated that if necessary, at that point they will then further revise the  measures. Nevertheless, BJA officials have not documented how they will  determine if the measures were successful or whether changes would be  needed. In addition, BJA officials did not record key methods and  assumptions used to guide their revision efforts, such as the feedback  stakeholders provided and BJA’s disposition of these comments. For  example, BJA officials provided a document generally showing the original  performance measure; whether it was removed, revised or replaced; and  BJA’s justification for the action, but this document did not demonstrate  how BJA had incorporated the stakeholder feedback it considered when  making its decisions. The document also did not include a link to a new  performance measure in instances where an older one was being replaced.  Further, BJA’s justification did not include the rationale for the changes it  made to 22 of the 51 performance measures. According to BJA officials,  they did not document their decisions in this way because of the rapid  nature of the revision process and limited staff resources. They also told us  that maintaining such documentation and providing it to stakeholders held  little value.
Our previous work has shown the importance of documentation to the  successful development of effective performance measures. In the past,  we have reported that revising performance measures involves a number of  aspects needing to be carefully planned and carried out and that by  documenting the steps undertaken in developing and implementing the  revised measures, agencies can be better assured their revisions result in  effective performance measures. In addition, academic literature on the  best practices for developing effective performance measures states that  agencies should develop products to document and guide their revision  efforts. These products, among other things, can include plans for ensuring  the quality and integrity of the data for full-scale implementation of the  measures. Further, Standards for Internal Control in the Federal  Government call for clear documentation of significant events, which can  include assumptions and methods surrounding key decisions, and this  documentation should be readily available for examination. As BJA moves  forward in assessing the revised measures and implementing additional  changes, if it deems necessary, BJA could better ensure that its efforts result  in successful and reliable metrics and are transparent by documenting key  methods used to guide revision efforts and an assessment of its measures.  This would also help bolster the integrity of its decisions.
section_title: Drug Courts Were Associated with Lower Recidivism and Relapse Rates for Program Participants Than Criminal Courts
paragraphs: In the evaluations we reviewed, adult drug-court program participation was  generally associated with lower recidivism. Our analysis of evaluations  reporting recidivism data for 32 programs showed that drug court program  participants were generally less likely to be re-arrested than comparison  group members drawn from the criminal court system, although the  differences in likelihood were reported to be statistically significant in 18  programs. Across studies showing re-arrest differences, the percentages  of drug court program participants rearrested were lower than for  comparison group members by 6 to 26 percentage points. One program  did not show a lower re-arrest rate for all drug-court program participants  relative to the comparison group within 3 years of entry into the program,  although that study did show a lower re-arrest rate for drug court  participants who had completed the program than for members of the  comparison group. In general, the evaluations we reviewed found larger  differences in re-arrest rates between drug-court program completers and  members of the comparison group than between all drug-court program  participants and the comparison group members. The rearrest rates for  program completers ranged from 12 to 58 percentage points below those  of the comparison group. The completion rates reported in the  evaluations we reviewed ranged from 15 percent to 89 percent.
Included among the evaluations we reviewed was the MADCE, a 5-year  longitudinal process, impact, and cost evaluation of adult drug courts. The  MADCE reported a re-arrest rate for drug court participants that was 10  percentage points below that of the comparison group; specifically, 52  percent of drug court participants were re-arrested after the initiation of  the drug court program, while 62 percent of the comparison group  members were re-arrested. However, the 10 percentage point difference  between these rearrest rates for the samples of drug court participants  and comparison group members was not statistically significant. The  MADCE study also reported that drug court participants were significantly  less likely than the comparison group to self-report having committed  crimes when they were interviewed 18 months after the baseline (40  percent vs. 53 percent), and drug court participants who did report  committing crimes committed fewer than comparison group members.
We assigned a numerical rating to each evaluation to reflect the quality of  its design and the rigor of the analyses conducted. Our methodology for  rating the evaluation studies is detailed in appendix III. After assigning the  rating, we grouped the studies into two tiers. Tier 1 studies were the most  carefully designed and incorporated substantial statistical rigor in their  analyses. Tier 2 studies, while still meeting our basic criteria for  methodological soundness, were relatively less rigorous in their design and  analyses. Both tier 1 and tier 2 studies reported differences between drug  court participants and comparison group members and both sets of studies  found that some but not all differences were statistically significant.
Table 3 shows whether a difference in recidivism rates was reported for  each program—expressed as the difference in the rate of re-arrest  between all drug court program participants and the comparison group. In  some cases the difference in recidivism was reported as something other  than a difference in the re-arrest rate, such as a difference in the number  of arrests or the relative odds of an arrest. In those cases, table 3 notes  that a difference was reported, but does not include the difference in re- arrest rates. For example, the evaluation of the Queens Misdemeanor  Treatment Court reported that the re-arrest rate for program participants  was 14 percentage points lower than the re-arrest rate of comparison  group members up to 2 years after participants entered into the program,  and 10 percentage points lower at 3 or more years after entry. Similarly,  the evaluation of the Hillsborough County Adult Drug Court reported a  statistically significant difference in the relative odds of an arrest after  drug court program enrollment but did not report the difference in rearrest  rates, therefore table 3 indicates a statistically significant reduction in  rearrest rates but does not show the difference in rates.
The evaluations we reviewed showed that adult drug-court program  participation was also associated with reduced drug use. Our analysis of  evaluations reporting relapse data for eight programs showed that drug  court program participants were less likely than comparison group  members to use drugs, based on drug tests or self-reported drug use,  although the difference was not always significant. This was true for  both within-program and post-program measures, and whether drug use  was reported as the difference in the frequency of drug use or the  proportion of the treatment and comparison groups who used drugs.
The MADCE concluded drug courts produce significant reductions in drug  relapse. Specifically, MADCE reported that “drug court participants were  significantly less likely than the comparison group to report using all drugs  (56 vs. 76 percent) and also less likely to report using ‘serious’ drugs (41  vs. 58 percent), which omit marijuana and ‘light’ alcohol use (fewer than  four drinks per day for women or less than five drinks per day for men).  On the 18-month oral fluids drug test, significantly fewer drug court  participants tested positive for illegal drugs (29 vs. 46 percent). Further,  among those who tested positive or self-reported using drugs, drug court  participants used drugs less frequently than the comparison group.”  Regarding post-drug court program relapses, the MADCE concluded that  participation in drug court—along with less frequent drug use among  offenders prior to arrest, and the absence of mental health problems— were the strongest predictors of success against relapses. Table 4  summarizes the results of drug-use relapse reported in the evaluations  we reviewed.
Of the studies we reviewed, 11 included sufficient information to report a  net benefit figure. Of these studies, the net benefit ranged from positive  $47,852 to negative $7,108 per participant. The net benefit is the  monetary benefit of reduced recidivism accrued to society from the drug  court program through reduced future victimization and justice system  expenditures, less the net costs of the drug court program—that is, the  cost of the program less the cost of processing a case in criminal court. A  negative net benefit value indicates that the costs of the drug court  program outweigh its estimated benefits and that the program was not  found to be cost beneficial. Eight of the studies reported positive net  benefits—the benefits estimated to accrue from the drug court program  exceeded the program’s net costs. Three of the 11 studies reported  negative net benefits. We did not attempt to determine whether the  differences in the reported values were because of differences in study  methodology or the attributes of the drug courts themselves. The  environment in which the drug court operates may also be important. For  example, the largest net benefit reported was for Kings County, in which  members of the comparison group were incarcerated, in contrast to other  programs in which members of the comparison group were given  probation, which is less costly. The more costly the alternative, such as  incarceration, the more likely a drug court will have positive net benefits.  In this case, the study reported that society would accrue $47,852 in  benefits relative to conventional court processing.
Table 5 below shows whether, based on the available information, the  study was shown to be cost beneficial. It also shows the net benefits per  participant of the drug court study. For example, MADCE found that the  drug court participants led to a net benefit of $6,208 per participant— within the range of the other studies. The MADCE analysis of costs and  benefits is discussed further in appendix II.
section_title: Conclusions
paragraphs: During the course of our review, BJA made strides in managing its adult  drug court program, including implementation of the GrantStat process  and recent revisions to the grantee performance measures. Given that  BJA has committed to testing its new measures during this first grantees’  reporting period, enhancements could be made to facilitate this  assessment. By documenting how it plans to assess the measures and  determine any changes that may be needed and providing the rationale  for future revisions, BJA could bolster the transparency and integrity of its  decisions. Doing so could also improve the reliability of the data it  collects, its usefulness to managers in guiding the program, and the  success of its measures.
section_title: Recommendation for Execution Action
paragraphs: Recognizing that BJA has recently revised the adult drug-court  performance measures and has plans to assess their utility, we  recommend that BJA’s Director take the following action to ensure that its  revision process is transparent and results in quality and successful  metrics to inform management’s key decisions on program operations:    Document key methods used to guide future revisions of its adult  drug-court program performance measures. This documentation  should include both a plan for how BJA will assess the measures after  conclusion of the grantees’ first reporting period and a rationale for  why each measure was refined, including a discussion of the scope  and nature of any relevant stakeholder comments.
section_title: Agency Comments
paragraphs: We provided a draft of this report to DOJ for review and comment. On  December 1, 2011, we received written comments on the draft report  from DOJ, which are reproduced in full in appendix VIII. DOJ concurred  with our recommendation and described actions under way or planned to  address the recommendation. DOJ also provided technical comments,  which we incorporated as appropriate.
DOJ stated that BJA will continue to document grantee feedback and will  ensure that revisions to the measures are documented in accordance  with GAO’s best practices standards. In particular, DOJ stated that BJA  will document (1) whether the name and definition of the measure is  consistent with the methodology used to calculate it; (2) whether the  measure is reasonably free from bias; (3) whether the measure meets the  expectation of the program; and (4) its rationale for why each  performance measure was refined, including the scope and nature of any  relevant stakeholder comments. We believe that such actions would  improve the reliability of the information collected, its usefulness to  managers in making key decisions on program operations, and the  success of its measures.
We are sending copies of this report to the Attorney General and  interested congressional committees. In addition, this report will be  available at no charge on the GAO Web site at http://www.gao.gov.
Should you or your staff have any questions concerning this report,  please contact me at (202) 512-9627 or by e-mail at maurerd@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. Key contributors to  this report are listed in appendix IX.
section_title: Appendix I: DOJ Has Fully Implemented Most of Our 2002 Recommendations and Plans to Address the Remaining One
paragraphs: The following provides the current status of the seven recommendations  we made in 2002—which have since been closed—on DOJ’s collection of  performance data. Specifically, DOJ has fully implemented six of them  and partially implemented one. DOJ has plans to fully address the  remaining recommendation related to analyzing performance and  outcome data collected from grantees and reporting annually on the  results. Table 6 reflects this status.
section_title: Appendix II: MADCE Is the Most Comprehensive Study of Drug Courts to Date, but Generalizability of Findings May Be Limited
paragraphs: NIJ’s MADCE was conducted by the Urban Institute, Center for Court  Innovation, and Research Triangle Institute. Data were collected from  1156 drug court participants in 23 different drug courts in 7 geographic  clusters and from a comparison group of 625 drug-involved offenders in  six different sites in four geographic clusters. Data collected included:  three waves of interviews; drug tests; administrative records on treatment,  arrests, and incarceration; court observation and interviews with staff and  other stakeholders; and budget and other cost information. The  evaluation was designed to address the following four questions:  (1)  Do drug courts reduce drug use, criminal behavior, and other  associated offender problems?  (2)  Do drug courts generate cost savings for the criminal justice system  and other public institutions?  (3)  Are drug courts especially effective or less effective for certain  categories of offenders or program characteristics?  (4)  Which drug court policies and offender perceptions explain their  overall impact?
subsections: section_title: MADCE Found Reductions in Recidivism and Relapse, but Generalizability May Be Limited
paragraphs: The MADCE’s major findings can be summarized as follows:    Drug courts produce statistically significant reductions in self-reported  crime. While both the drug court participants and comparison group  participants reported large numbers of crimes in the year preceding  the 18-month follow-up, drug court participants reported statistically  significantly fewer than the comparison group members. Drug court  participants were less likely than members of the comparison group to  report committing any crimes (40 percent vs. 53 percent) and drug  court participants reported committing fewer crimes in the preceding  12 months than comparison group members (43 criminal acts vs. 88  criminal acts). The difference between the two groups in the  probability of an official re-arrest over 24 months was not statistically  significant, though the percentage of individuals rearrested was lower  for the drug court group than the comparison group (52 percent vs. 62  percent), as was the average number of re-arrests (1.24 vs. 1.64).    Drug courts produce statistically significant reductions in drug use.
Drug court participants were less likely than members of the  comparison group to report using any drugs (56 percent vs. 76  percent) and any serious drugs (41 percent vs. 58 percent), and less  likely to test positively for drugs at the 18-month follow-up (29 percent  vs. 46 percent). Furthermore, the large difference in self-reported  relapse rates is evident at 6 months (40 percent vs. 59 percent), so  the impact of drug courts on alcohol and other drug use is sustained.  The interview data also indicate that among the drug court  participants and comparison group members that were using drugs,  the drug court participants, on average, were using them less  frequently.    Drug court participants reported some benefits, relative to comparison  group members, in other areas of their lives. At 18 months, drug court  participants were statistically significantly less likely than comparison  group members to report a need for employment, educational, and  financial services, and reported statistically significantly less family  conflict. However, there were modest, non-significant differences in  employment rates, income, and family emotional support, and no  differences found in experiencing homelessness or depression.    Regardless of background, most offenders who participated in drug  courts had better outcomes than offenders who were in the  comparison programs. However, the impact of drug courts was  greater for participants with more serious prior drug use and criminal  histories, and the impact was smaller for participants who were  younger, male, African-American, or who had mental health problems.    While the treatment and service costs were higher for drug court  participants than treatment and service costs associated with the  alternative “business-as-usual” comparison programs, drug courts  save money through improved outcomes, according to the  researchers, primarily through savings to victims resulting from fewer  crimes and savings resulting from fewer re-arrests and incarcerations.
The authors of the study assert that their findings have strong internal  validity—that is, that the findings were actually produced by the drug court  programs—and external validity—that is, that the findings can be  generalized to the population of all drug court participants and potential  comparison group members. The claim to strong internal validity is not  without merit, given the high response rates, low attrition, propensity  score adjustments, and conservative estimates produced by the  hierarchical models used. The claim of high internal validity is also  supported by the sensitivity analyses undertaken for several outcomes  using other models and methods of adjustments that produced little or no  change in conclusions. The claim to strong external validity, which relates  to the generalizability of the results beyond the sample of courts and  comparison sites and specific offenders considered, may be somewhat  overstated. The authors note that the 23 drug courts included in the study  represent “a broad mix of urban, suburban, and rural courts from 7  geographic clusters nationwide,” but that doesn’t assure that, collectively,  the drug courts that were included resemble the hundreds of drug courts  that were not included, especially since they were not chosen at random.  It also seems unlikely that the six comparison sites from four states are  representative of all potential controls, or all alternative programs in all  states, and it is potentially problematic that all of the selected sites,  including drug court and comparison sites, were alike in their willingness  and interest in participating. Those concerns notwithstanding, this is the  broadest and most ambitious study of drug courts to date; it is well done  analytically; and the results, as they relate to the impact of drug courts,  are transparent and well described.
section_title: MADCE’s Cost Benefit Analysis Focused on Individuals
paragraphs: The MADCE cost benefit analysis approach differed from most of the  other studies we reviewed. In most of the other studies, the average cost  and benefit of a drug court participant was compared to the average cost  and benefit of normal court processing. In contrast, the MADCE obtained  a separate net benefit figure for each individual. The net benefit was  obtained by tracking each individual’s use of resources, such as hearings  or meetings with case managers, and program outcomes like use of  public assistance. The MADCE also tracked each individual’s rates of re- arrest, number of crimes, and time of incarceration. The crimes are  multiplied by cost to victims per crime to obtain the cost to society. The  difference between the net benefits of the drug court participants and the  comparison group were obtained using a hierarchical model similar to the  one used for program outcomes. After applying the method, the MADCE  found that the drug court participants led to a net benefit of $6,208 to  society per participant, as compared to the comparison group. However,  due to the variability in the estimate, the study did not find that the net  benefits were statistically significant.
The lack of a statistically significant difference may be because of greater  variability in the MADCE approach than the approach used in other  studies. Specifically, the MADCE did not assume identical costs for each  participant. As a result, costs may be higher for individuals who have  lower rates of re-arrest, perhaps because those individuals received more  treatment. According to the study’s authors, by assuming identical costs  for each participant, the standard approach understates the variance in  the computed net benefit figure by not including the variability in cost.  However, the MADCE authors assumed that the prices of services were  consistent across sites by using a weighted average. In contrast, some  studies generate site-specific cost figures. In this way, the MADCE  approach did exclude one source of variation that is present in some  other studies.
In addition to tracking costs and benefits at the individual level, the  MADCE also included some effects of drug court participation that some  other studies omit. This is consistent with OMB guidance that states that  studies should be comprehensive in the benefits and costs to society  considered. One of the benefits considered by the MADCE, sometimes  omitted elsewhere, is the estimated earnings of the drug court participant.  However, it is unclear that the full value of earnings should have been  considered a net benefit to society. For example, to be comprehensive, a  study should also consider the cost to society of providing that benefit.  The net benefit would account for the value of production from this  employment less the wages paid. Although in this case, it is unlikely that  this would affect the result of the analysis, as the earnings are similar for  drug court participants and the comparison group.
section_title: Appendix III: Objectives, Scope, and Methodology
paragraphs: To determine what data DOJ collects on the performance of federally  funded adult drug courts and to what extent DOJ has used this data in  making grant-related decisions, we analyzed the reporting guidance and  requirements that BJA provided in fiscal years 2007 through 2011 to  grantees applying for Adult Drug Court Discretionary Grant Program  funds; BJA-generated grantee performance data reports from October to  December 2010; and BJA’s guides for managing grants and enforcing  grantee compliance that were issued in fiscal year 2011. We selected  2007 as the starting point for our review because BJA implemented its  Performance Measurement Tool (PMT)—an online reporting tool that  supports BJA grantees’ ability to collect, identify, and report performance- measurement data activities funded by the grantees’ awards—in fiscal  year 2007. We also reviewed our prior reports and internal control  standards as well as other academic literature regarding effective  performance-management practices. We then used this information and  BJA officials’ statements to identify and define six management activities  for which performance information can be most useful in making grant- related decisions. Further, we interviewed cognizant BJA officials about  the extent to which they use grantees’ performance data when engaging  in these management activities, any challenges faced with ensuring  grantee compliance, ongoing efforts to revise program performance  metrics, and the extent to which BJA’s revisions incorporate best  practices we previously identified.
To determine what is known about the effectiveness of adult drug courts  in reducing recidivism and substance-abuse relapse rates and what the  costs and benefits of adult drug courts are, we conducted a systematic  review of evaluations of drug-court program effectiveness issued from  February 2004 through March 2011 to identify what is known about the  effect of drug court programs on the recidivism of and relapse of drug- involved individuals as well as the costs and benefits of drug courts. We  also reviewed DOJ’s NIJ-funded MADCE, a 5-year longitudinal process,  impact, and cost evaluation of adult drug courts that was issued in June  2011. We identified the universe of evaluations to include in our review  using a three-stage process. First, we (1) conducted key-word searches  of criminal justice and social science research databases; (2) searched  drug court program-related Web sites, such as those of BJA and NADCP;  (3) reviewed bibliographies, meta-analyses of drug court evaluations, and  our prior reports on drug court programs; and (4) asked drug court  researchers and DOJ officials to identify evaluations. Our literature search  identified 260 documents, which consisted of published and unpublished  outcome evaluations, process evaluations, commentary about drug court  programs, and summaries of multiple program evaluations. Second, we  reviewed the 260 documents our search yielded and identified 44  evaluations that reported recidivism or substance use relapse rates using  either an experimental or quasi-experimental design, or analyzed  program costs and benefits. Third, we used generally accepted social  science and cost benefit criteria to review the 44 evaluations.
To assess the methodological quality of evaluations that reported on  recidivism or relapse rates, we placed each evaluation into one of five  categories, with category 1 evaluations being the most rigorous and  category 5 the least, as outlined in table 7.
We excluded studies that were placed in category 5 in the table above or  studies in which the comparison group was not drawn from a criminal  court. We were left with 33 studies, plus the MADCE, that reported on the  effectiveness of 32 drug court programs or sets of programs. As noted  in our report, we then grouped the 34 studies, including the MADCE, into  two tiers according to their quality category, Tier 1 studies were those that  fell into categories 1 or 2, Tier 2 studies were those that fell into  categories 3 or 4.
Observed differences in recidivism could arise from measured and  unmeasured sources of variation between drug court participants and  comparison group members. If comparison group members differed  systematically from drug court participants on variables that are also  associated with recidivism, such as the degree of their substance-abuse  addiction problem and these variables were not accounted for by the  design or analysis used in the evaluation, then the study could suffer from  selection bias wherein observed differences in recidivism could be  because of these sources of variation rather than participation in the drug  court program. As indicated in table 7, our evaluation of the methods  used to deal with selection bias was reflected in the quality categorization  of each study.
To assess the methodological quality of evaluations that reported on drug  court program costs and benefits, we assessed them according to the five  criteria we developed and outlined in table 8 below.
We determined that an essential criterion for reporting a net benefit of  drug courts was that the costs of the drug court were assessed against a  baseline (i.e., “business-as-usual” or traditional court processing). Eleven  studies met this essential standard and were used to report on program  costs and benefits. We excluded other studies not meeting this standard  even though they may have met others.
To obtain information on our outcomes of interest—that is, recidivism,  substance use relapse, and costs and benefits—we used data collection  instruments to systematically collect information about the methodological  characteristics of each evaluation, the drug court participants and  comparison group members studied, and the outcomes of the participants  and other comparable groups reported. Each evaluation was read and  coded by a senior social scientist, statistician, or economist with training  and experience in evaluation research methods. A second senior social  scientist, statistician, or economist then reviewed each completed data  collection instrument to verify the accuracy of the information included.  Part of our assessment also focused on the quality of the data used in the  evaluations as reported by the researchers and our observations of any  problems with missing data, any limitations of data sources for the  purposes for which they were used, and inconsistencies in reporting data.  We incorporated any data problems that we noted in our quality  assessments.
We selected the evaluations in our review based on their methodological  strength; therefore, our results cannot be generalized to all drug court  programs or their evaluations. Although the findings of the evaluations we  reviewed are not representative of the findings of all evaluations of drug  court programs, the evaluations consist of those evaluations we could  identify that used the strongest designs to assess drug-court program  effectiveness.
To identify the extent to which DOJ has addressed the recommendations  that we made in 2002 regarding drug court programs, we interviewed  cognizant DOJ officials and obtained and reviewed documentation (e.g.,  drug-court program grant solicitations and grantee-performance reporting  guidance) on the actions taken to address and implement each of our  prior recommendations. We conducted this performance audit from  November 2010 through December 2011 in accordance with generally  accepted government-auditing standards. Those standards require that  we plan and perform the audit to obtain sufficient, appropriate evidence to  provide a reasonable basis for our findings and conclusions based on our  audit objectives. We believe that the evidence obtained provides a  reasonable basis for our findings and conclusions based on our  objectives.
section_title: Appendix IV: Overview of Drug Court Program Characteristics
paragraphs: This appendix provides a general description of drug court program  components (see table 9). Drug court programs rely on a combination of  judicial supervision and substance-abuse treatment to motivate  defendants’ recovery. Judges preside over drug court proceedings,  which are called status hearings; monitor defendants’ progress with  mandatory drug testing; and prescribe sanctions and incentives, as  appropriate in collaboration with prosecutors, defense attorneys,  treatment providers, and others. Drug court programs can vary in terms of  the substance-abuse treatment required. However, most programs offer a  range of treatment options and generally require a minimum of about 1  year of participation before a defendant completes the program.
section_title: Appendix V: Ten Key Components of a Drug Court— Developed by BJA in Collaboration with the National Association of Drug Court Professionals
paragraphs: Appendix V: Ten Key Components of a Drug  Court— Developed by BJA in Collaboration  with the National Association of Drug Court  Professionals 1.  Integration of substance-abuse treatment with justice system case processing.  2.  Use of a non-adversarial approach, in which prosecution and defense promote public  safety while protecting the right of the participant to due process.  3.  Early identification and prompt placement of eligible participants.  4.  Access to continuum of treatment, rehabilitation, and related services.  5.  Frequent testing for alcohol and illicit drugs.  6.  A coordinated strategy governs drug court responses to participants’ compliance.  7.  Ongoing judicial interaction with each participant.  8.  Monitoring and evaluation to measure achievement of program goals and gauge  effectiveness.  9.  Continuing interdisciplinary education to promote effective planning, implementation,  and operation.  10. Forging partnerships among drug courts, public agencies, and community-based  organizations generates local support and enhances drug court program  effectiveness.
section_title: Appendix VI: BJA Offers Solicitations in Four Broad Drug-Court Grant Categories— Implementation, Enhancement, Statewide, and Joint
paragraphs: As mentioned, the Adult Drug Court Discretionary Grant Program  provides financial and technical assistance to states, state courts, local  courts, units of local government, and Indian tribal governments to  develop and implement drug treatment courts. There are four different  types of awards that BJA makes to adult drug-court grantees through the  program. Table 11 provides a description of the grant types.
section_title: Appendix VII: Key Management Activities Identified for Which Performance Information Can Be Most Useful
paragraphs: How performance information may be used to support  the activity  Performance information is used to set priorities in  budgeting and to target resources. Agencies can also use  this information to identify priorities on which to focus their  efforts. For example, targeting grants to address  “underserved” client groups.
Performance information is used to compare results of  agencies’ programs with goals and to identify where  program resources should be targeted to improve  performance and achieve goals. When faced with reduced  resources, such analyses can assist agencies’ efforts to  minimize the impact on program results.
Performance information is used to assess the way a  program is conducted and the extent to which a program’s  practices and policies have or have not led to  improvements in outcomes. Such information is used to  identify problems and consider alternative approaches and  processes in areas where goals are not being met and to  enhance the use of program approaches and processes  that are working well.
Performance information is used to identify and increase  the use of program approaches that are working well and  share these effective processes and approaches with  stakeholders.
Performance information is used to establish the targets  and goals that grantees are expected to achieve. These  targets and goals can be used as the basis for corrective  action (e.g., technical assistance, freezing of funds) or to  reward high performing grantees.
Performance information is used to compare grantees’  performance results with established targets and goals to  determine the extent to which grantees have met them  and, if necessary, target program resources (e.g.,  technical assistance) to improve grantees’ performance.
section_title: Appendix VIII: Comments from the Department of Justice, Bureau of Justice Assistance
section_title: Appendix IX: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Joy Booth, Assistant Director and  Frederick Lyles, Jr., Analyst-in-Charge, managed this assignment.  Christoph Hoashi-Erhardt, Michael Lenington, and Jerry Seigler, Jr.,  made significant contributions to the work. David Alexander, Benjamin  Bolitzer, Michele Fejfar, and Doug Sloane assisted with design and  methodology. Pedro Almoguera, Carl Barden, Harold Brumm, Jr., Jean  McSween, Cynthia Saunders, Jeff Tessin, Susan B. Wallace, and  Monique Williams assisted with evaluation review. Janet Temko provided  legal support, and Katherine Davis provided assistance in report  preparation.
section_title: Bibliography
paragraphs: Bouffard, Jeffrey A., and Katie A. Richardson. “The Effectiveness of Drug  Court Programming for Specific Kinds of Offenders: Methamphetamine  and DWI Offenders Versus Other Drug-Involved Offenders.” Criminal  Justice Policy Review, 18(3) (September 2007): 274-293.
Carey, Shannon M., and Michael W. Finigan. Indiana Drug Courts: St.  Joseph County Drug Court Program Process, Outcome and Cost  Evaluation-Final Report. Portland, OR: NPC Research, 2007.
Carey, Shannon M., and Michael W. Finigan. Indiana Drug Courts:  Vanderburgh County Day Reporting Drug Court Process, Outcome and  Cost Evaluation-Final Report. Portland, OR: NPC Research, 2007.
Carey, Shannon M., and Michael W. Finigan. “A Detailed Cost Analysis in  a Mature Drug Court Setting: A Cost-Benefit Evaluation of the Multnomah  County Drug Court.” Journal of Contemporary Criminal Justice, 20(3)  (August 2004): 315-338.
Carey, Shannon M., Michael W. Finigan, et. al. Indiana Drug Courts:  Monroe County Drug Treatment Court Process, Outcome and Cost  Evaluation-Final Report. Portland, OR: NPC Research, 2007.
Carey, Shannon M., Michael Finigan, Dave Crumpton, and Mark S.  Waller. “California Drug Courts: Outcomes, Costs and Promising  Practices: An Overview of Phase II in a Statewide Study.” Journal of  Psychoactive Drugs (November 2006).
Carey, Shannon M., Lisa M. Lucas, Mark S. Waller, Callie H. Lambarth,  Robert Linhares, Judy M. Weller, and Michael W. Finigan. Vermont Drug  Courts: Rutland County Adult Drug Court Process, Outcome, and Cost  Evaluation-Final Report. Portland, OR: NPC Research, 2009.
Carey, Shannon, and Gwen Marchand. Marion County Adult Drug Court  Outcome Evaluation-Final Report. Portland, OR: NPC Research, 2005.
Carey, Shannon M. and Mark S. Waller. Oregon Drug Court Cost Study:  Statewide Costs and Promising Practices-Final Report. Portland, OR:  NPC Research, 2010.
Carey, Shannon M., and Mark Waller. California Drug Courts: Costs and  Benefits-Phase III. Portland, OR: NPC Research, 2008.
Carey, Shannon M., and Mark S. Waller. Guam Adult Drug Court  Outcome Evaluation-Final Report. Portland, OR: NPC Research, 2007.
Dandan, Doria Nour. Sex, Drug Courts, and Recidivism. University of  Nevada, Las Vegas: 2010.
Ferguson, Andrew, Birch McCole, and Jody Raio. A Process and Site- Specific Outcome Evaluation of Maine’s Adult Drug Treatment Court  Programs. Augusta, ME: University of Southern Maine, 2006.
Finigan, Michael W., Shannon M. Carey, and Anton Cox. Impact of a  Mature Drug Court Over 10 Years of Operation: Recidivism and Costs  (Final Report). Portland, OR: NPC Research, 2007.
Gottfredson, Denice C., Brook W. Kearley, Stacy S. Najaka, and Carlos  M. Rocha. “How Drug Treatment Courts Work: An Analysis of Mediators.”  Journal of Research in Crime and Delinquency, 44(1) (February 2007): 3- 35.
Gottfredson, Denice C., Brook W. Kearley, Stacy S. Najaka, and Carlos  M. Rocha. “Long-term effects of participation in the Baltimore City drug  treatment court: Results from an experimental study.” Journal of  Experimental Criminology, 2(1) (January 2006): 67-98.
Gottfredson, Denice C., Brook W. Kearley, Stacy S. Najaka, and Carlos  M. Rocha. “The Baltimore City Drug Treatment Court: 3-Year Self-Report  Outcome Study.” Evaluation Review, 29(1) (February 2005): 42-64.
Krebs, C.P., C.H. Lindquist, W. Koetse, and P.K. Lattimore. “Assessing  the long-term impact of drug court participation on recidivism with  generalized estimating equations.” Drug and Alcohol Dependence, 91(1)  (November 2007): 57-68.
Labriola, Melissa M. The Drug Court Model and Chronic Misdemeanants:  Impact Evaluation of the Queens Misdemeanor Treatment Court. New  York, NY: Center for Court Innovation, 2009.
Latimer, Jeff, Kelly Morton-Bourgon, and Jo-Anne Chrétien. A Meta- Analytic Examination of Drug Treatment Courts: Do They Reduce  Recidivism? Ottawa, Ontario: Department of Justice Canada, 2006.
Listwan, Shelley Johnson, James Borowiak, and Edward J. Latessa. An  Examination of Idaho’s Felony Drug Courts: Findings and  Recommendations-Final Report. Kent State University and University of  Cincinnati: 2008.
Logan, T. K., William H. Hoyt, Kathryn E. McCollister, Michael T. French,  Carl Leukefeld, and Lisa Minton. “Economic evaluation of drug court:  methodology, results, and policy Implications.” Evaluation and Program  Planning, 27 (2004):381–396.
Loman, Anthony L. A Cost-Benefit Analysis of the St. Louis City Adult  Felony Drug Court. Institute of Applied Research. St. Louis, MO: 2004.
Lowenkamp, Christopher T., Alexander M. Holsinger, Edward J. Latessa.  “Are Drug Courts Effective: A Meta-Analytic Review.” Journal of  Community Corrections. (Fall 2005): 5-28.
Mackin, Juliette R., Shannon M. Carey, and Michael W. Finigan. Harford  County District Court Adult Drug Court: Outcome and Cost Evaluation.  Portland, OR: NPC Research, 2008.
Mackin, Juliette R., Shannon M. Carey, and Michael W. Finigan. Prince  George’s County Circuit Court Adult Drug Court: Outcome and Cost  Evaluation. Portland, OR: NPC Research, 2008.
Mackin, Juliette R., Lisa M. Lucas, Callie H. Lambarth, Mark S. Waller,  Shannon M. Carey, and Michael W. Finigan. Baltimore City Circuit Court  Adult Drug Treatment Court and Felony Diversion Initiative: Outcome and  Cost Evaluation-Final Report. Portland, OR: NPC Research, 2009.
Mackin, Juliette R., Lisa M. Lucas, Callie H. Lambarth, Mark S. Waller,  Theresa Allen Herrera, Shannon M. Carey, and Michael W. Finigan.  Howard County District Court Drug Treatment Court Program Outcome  and Cost Evaluation. Portland, OR: NPC Research, 2010.
Mackin, Juliette R., Lisa M. Lucas, Callie H. Lambarth, Mark S. Waller,  Theresa Allen Herrera, Shannon M. Carey, and Michael W. Finigan.  Montgomery County Adult Drug Court Program Outcome and Cost  Evaluation. Portland, OR: NPC Research, 2010.
Mackin, Juliette R., Lisa M. Lucas, Callie H. Lambarth, Mark S. Waller,  Theresa Allen Herrera, Shannon M. Carey, and Michael W. Finigan.  Wicomico County Circuit Court Adult Drug Treatment Court Program  Outcome and Cost Evaluation. Portland, OR: NPC Research, 2009.
Mackin, Juliette R., Lisa M. Lucas, Callie H. Lambarth, Mark S. Waller,  Judy M. Weller, Jennifer A. Aborn, Robert Linhares, Theresa L. Allen,  Shannon M. Carey, and Michael W. Finigan. Baltimore City District Court  Adult Drug Treatment Court: 10-Year Outcome and Cost Evaluation.  Portland, OR: NPC Research, 2009.
Marchand, Gwen, Mark Waller, and Shannon M. Carey. Barry County  Adult Drug Court Outcome and Cost Evaluation-Final Report. Portland,  OR: NPC Research, 2006.
Marchand, Gwen, Mark Waller, and Shannon M. Carey. Kalamazoo  County Adult Drug Treatment Court Outcome and Cost Evaluation-Final  Report. Portland, OR: NPC Research, 2006.
Marinelli-Casey, Patricia, Rachel Gonzales, Maureen Hillhouse, Alfonso  Ang, Joan Zweben, Judith Cohen, Peggy Fulton Hora, and Richard A.  Rawson. “Drug court treatment for methamphetamine dependence:  Treatment response and posttreatment outcomes.” Journal of Substance  Abuse Treatment, 34(2) (March 2008): 242-248.
Mitchell, Ojmarrh, and Adele Harrell. “Evaluation of the Breaking the  Cycle Demonstration Project: Jacksonville, FL and Tacoma, WA.” Journal  of Drug Issues, 36(1) (Winter 2006): 97-118.
Piper, R. K., and Cassia Spohn. Cost/Benefit Analysis of the Douglas  County Drug Court. Omaha, NE: University of Nebraska at Omaha, 2004.
Rhodes, William, Ryan Kling, and Michael Shively. Suffolk County Drug  Court Evaluation. Abt. Associates, Inc., 2006.
Rhyne, Charlene. Clean Court Outcome Study. Portland, OR: Multnomah  County Department of Community Justice, 2004.
Rossman, S., M. Rempel, J. Roman, et.al. The Multi-Site Adult Drug  Court Evaluation: The Impact of Drug Courts. Washington, D.C.: Urban  Institute, 2011.
Shaffer, Deborah K., Kristin Bechtel, and Edward J. Latessa. Evaluation  of Ohio’s Drug Courts: A Cost Benefit Analysis. Cincinnati, OH: Center for  Criminal Justice Research, University of Cincinnati, 2005.
Wilson, David B., Ojmarrh Mitchell, and Doris L. Mackenzie. “A  systematic review of drug court effects on recidivism.” Journal of  Experimental Criminology, 2(4) (2006): 459-487.
Zarkin, Gary A., Lara J. Dunlap, Steven Belenko, and Paul A. Dynia. “A  Benefit-Cost Analysis of the Kings County District Attorney’s Office Drug  Treatment Alternative to Prison (DTAP) Program.” Justice Research and  Policy, 7(1) (2005).