Local jurisdictions (i.e., counties and municipalities) administer criminal courts and permit pretrial detention of defendants accused of serious offenses and deemed to be dangerous to prevent them from committing crimes prior to trial, according to the Bureau of Justice Statistics. Many local jurisdictions have begun adopting alternatives to incarceration, which are intended to improve public health and safety while reducing costs. Such alternatives generally represent a shift in emphasis away from prosecuting those associated with lower-level crimes toward providing treatment for underlying mental health or substance abuse disorders and include programs for individuals in jail or court who screen positive for mental illness. Veterans treatment courts are one such alternative. These courts are typically local courts dedicated to handling criminal cases involving veterans with mental health or substance abuse problems. According to VA, veterans treatment courts share several general characteristics but vary in their specific policies and procedures because of, among other things, differences in local jurisdictions and criminal justice system practices. Veterans treatment courts are modeled after adult drug courts, which are specialized courts that target criminal offenders who have drug addiction and dependency problems. As in the case of adult drug courts, judges preside over veterans treatment court proceedings and monitor veterans’ progress with treatment in collaboration with a team that usually includes a court coordinator, prosecutor, public defender, and probation officer. Additionally, the team includes a VJO specialist. Veterans treatment courts vary in terms of criteria for taking a case, such as the types or levels of criminal offenses. As was the case with past generations of veterans, the transition from military life to civilian life can be challenging for Post-9/11 veterans. Most veterans are not involved with local criminal justice systems, but some veterans—particularly if their mental health, family readjustment, or other needs remain unmet—may become justice-involved. The Bureau of Justice Statistics reported that about 7 percent (about 50,000) of the total population of inmates in jail between February 2011 and May 2012 were veterans. According to the Bureau of Justice Statistics report, this estimate represents a 25 percent decrease from the number of veterans in jail in 2004. Moreover, veterans were incarcerated in jails at lower rates than nonveterans between February 2011 and May 2012, according to the Bureau of Justice Statistics. While many veterans who served in the military have successfully readjusted to civilian life with minimal difficulties, researchers and policymakers have identified concerns about how the experience of Post- 9/11 servicemembers may affect the incarceration rates among these veterans. According to VA, military experience (particularly combat) has been an underlying factor in behavior that prompts a response from law enforcement, such as domestic conflicts. During the last 14 years of U.S. military operations, many servicemembers have experienced numerous deployments, which can increase the risk of developing post-traumatic stress disorder (PTSD) and traumatic brain injury. According to VA, a strong relationship exists between PTSD and substance abuse. Research has demonstrated that justice-involved veterans have high rates of mental illness, substance abuse, homelessness, and other health issues. At the same time, some veterans may be unwilling or unable to access the supports and services they need. For example, we have previously reported that some veterans do not seek mental health treatment due to concerns about negative career outcomes, lack of understanding or awareness of treatment, and logistical challenges to accessing care. Left unaddressed, a combination of homelessness, unemployment, mental-health, or substance abuse issues can place veterans experiencing a difficult transition at higher risk of committing a crime. VA initiated the VJO Program in 2009. The mission of the program is to reduce and prevent criminal justice recidivism and homelessness among veterans by linking justice-involved veterans with appropriate supports and services. According to VA, incarceration is a strong predictor of veteran homelessness, and recidivism can limit VA’s ability to provide continuous care for mental health and other issues. The VJO Program’s fiscal year 2012-2016 strategic plan contains five broad strategic goals for the program as well as a number of related objectives (see table 1). Evaluative information helps the executive branch and congressional committees make decisions about the programs they oversee; that is, evaluative information tells them whether and why a program is working well or not. We have previously reported that an important system to obtain such information is through program performance assessment. (See fig. 1.) A program performance assessment system is an important component of effective program management and contains three key elements: 1. Program goals communicate what the agency proposes to accomplish and allow agencies to assess or demonstrate the degree to which those desired results were achieved. Strategic goals and related objectives are long-term goals that set a general direction for a program’s efforts. Performance goals are the specific results an agency expects its program to achieve in the near term. 2. Performance measures are concrete, objective, observable conditions that permit the assessment of progress made toward the agency’s goals. Performance measures show the progress the agency is making in achieving performance goals. 3. Program evaluations are individual systematic studies using performance measures and other information to answer specific questions about how well a program is meeting its objectives. The VJO Program operates through VA medical centers. VA’s central office provides each VA medical center the flexibility to determine how to best respond to the needs of justice-involved veterans within the local community. More specifically, VA established broad program parameters that allow VA medical center officials to set VJO specialists’ activities to meet the needs of veterans in local criminal justice systems. VA issued guidelines on the operations of the VJO Program through a series of memorandums to program officials. According to VA, the program is to serve veterans as they interact with the local criminal justice system at multiple points and settings—from initial contact with local law enforcement to release from jail. VA guidelines define a “justice-involved veteran” as a veteran who is (1) arrested by local law enforcement who can be appropriately diverted from arrest into treatment; (2) incarcerated in a local jail, and who either has a pending trial or is serving a sentence after a conviction; or (3) involved in adjudication or monitoring by a court. (See fig. 2.) The Veterans Health Administration (VHA), which operates VA’s health care system, administers the VJO Program. VA policy requires each of the 167 VA medical centers around the country to provide outreach to justice-involved veterans. VA prioritizes veterans charged with nonviolent crimes, but VA must consider a veteran’s current legal circumstances— and not legal history alone—to determine whether the program can meet the individual veteran’s needs while maintaining safety, according to VA policy. The program does not provide legal representation, nor does it accept legal custody of a veteran. Specifically, VHA staff in Veterans Integrated Service Networks (VISN) help oversee the VJO Program and provide technical assistance to VJO specialists. Further, VA’s central office, its VISNs, and VA medical centers manage different aspects of program operations. VA central office officials directed each medical center to have at least one full-time VJO specialist position, most of which are funded by central office funds, according to VA. Some VA medical centers employ one VJO specialist position while others have multiple positions. VA central office officials also train new VJO specialists and manage national data collection efforts. VJO specialists report directly to VA medical center officials, typically in the homelessness prevention or mental health services. According to VA central office officials, as of September 2015, VA employed 261 full-time VJO specialists, who are mostly social workers, and seven other VJO specialists who play a hybrid role between the VJO and the Health Care for Re-entry Veterans programs. To accomplish the program mission of serving justice-involved veterans, VA tasks VJO specialists to perform three core functions: identify, assess, and link justice-involved veterans to appropriate supports and services. Experience of Veterans Who Have Participated in the VJO Program All of the veterans participating in our group discussions were appreciative of the VJO Program. At one location, veterans said that the VJO specialists understood their needs, educated them about available VA supports and services, assisted them in determining their eligibility, and helped them obtain VA supports and services. An example from a VJO Specialist To illustrate the effect of the program on one veteran’s life, VJO specialists we interviewed in one of our selected areas provided the following anecdote. A homeless veteran—who had served in Operation Enduring Freedom— was arrested for trespassing for sleeping behind a VA building. The veteran had substance abuse issues. When he appeared before a veterans treatment court, he met a VJO specialist who assessed his needs and assisted him in enrolling in VA housing and substance abuse treatment. When the veteran was stable, the VJO specialist was able to link him to permanent housing and VA vocational training. After several years of treatment and assistance, the veteran was hired by VA and purchased a home with a VA loan. helps jail administrators identify incarcerated veterans by comparing the names of inmates with VA’s list of veterans. After they identify justice-involved veterans, VJO specialists determine the veterans’ treatment needs by assessing their mental health, social well-being, appearance, and attitude. VJO specialists also collect information on employment history, current housing situation, military service, and discharge status. According to VA staff, upon completing the assessment, VJO specialists develop a treatment plan to meet the veteran’s needs. A treatment plan typically includes recommendations for medical or mental health services, housing, or other services, according to VJO specialists in five of the nine selected areas. After they identify and assess justice-involved veterans, VJO specialists link veterans to VA or community supports and services. VJO specialists do not directly provide treatment to justice-involved veterans. VA staff said once VJO specialists refer and link veterans to the appropriate supports and services, they perform follow-up visits with veterans to ensure they are receiving them. For example, VJO specialists may assist veterans to find adequate housing or with any transportation issues. Some veterans in jails may receive one follow-up visit while others may receive more, depending on their needs according to a VJO specialist. In contrast, VJO specialists work with veterans who participate in veterans treatment courts for 1 to 2 years, depending on the local criminal justice system and on the amount of time a veteran is required to participate in the court program. VJO specialists refer and link veterans to the VHA for health care, mental health, substance abuse treatment, or housing services where the overseeing staff determine the type of treatment and services provided. VJO specialists also link veterans to the Veterans Benefits Administration for disability compensation, pension benefits, or vocational rehabilitation to be determined by overseeing staff based on a veteran’s eligibility, according to VA (see fig. 3). In addition, VJO specialists link veterans to community service providers when the VA-provided treatment is too far away for a veteran to participate, or when a veteran is not eligible for VA health care services or benefits according to VA. Within these broad program parameters, a VA medical center determines the type and amount of investment it makes in serving justice-involved veterans. VJO specialists’ activities vary based on whether they are working in courts, jails, or both. For example: Specialty courts: VJO specialists provide services to veterans in a variety of specialty courts, such as veterans treatment courts, drug courts, and mental health courts. However, VJO specialists work most often with veterans treatment courts compared to other specialty courts, according to VA central office officials. Local criminal justice systems: In local criminal justice systems without established veteran-focused programs, VJO specialists spend significant time attempting to develop or plan these types of programs, according to VISN officials. These activities involve but are not limited to planning veterans treatment court (or other alternative court) programs and negotiating the terms of access to conduct outreach in jail facilities. In other areas, VJO specialists work with local criminal justice officials who may be unaware of veteran-focused programs. In such cases, VJO specialists focus their work on educating local law enforcement about VA resources available to veterans in crisis and working with veterans in jails, according to two VISN officials with whom we spoke. Jail administrators: VJO specialists in two of the nine areas we selected for interviews worked with local jail administrators to set up a program that provides veteran-specific housing units. These housing units are a designated block of jail cells only for veterans. Within these units, local community service organizations provide incarcerated veterans with treatment and services while they are in jail, and VJO specialists assess and develop community re-entry plans for housing, VA treatment, or other necessary services. In another area we visited, VJO specialists developed a program to help “high-risk” veterans released from jail. VJO specialists used information from the veterans’ clinical assessment to determine which veterans were high-risk for re- offending, dropping out of treatment, or becoming homeless. One VJO specialist at that VA medical center said the program serves these veterans by providing weekly follow-up services within the first month of their release from jail, and offers regular follow-up services and support after the first month, based on the individual veteran’s needs. In fiscal year 2015, the VJO Program served about 46,500 justice- involved veterans, and the program has experienced steady growth in the number of veterans served, according to data from VA. During fiscal years 2012 through 2015, the number of justice-involved veterans annually served by the VJO Program increased from about 27,000 to 46,500, a 72 percent increase (see fig. 4). Justice-involved veterans who were served by the VJO Program in fiscal year 2015 had the following characteristics: Most were young men. Nearly 95 percent of the veterans served by the program were male and 52 percent of justice-involved veterans were between the ages of 18 and 44. Many reported not working full-time during the past 3 years. Approximately 40 percent of the veterans served by the program reported working part time, working irregularly, or having been unemployed during the past 3 years. Another one-third (33 percent) reported being retired or disabled. Almost three-fourths reported serving during a U.S. military intervention. Of veterans who received services through the VJO Program in fiscal year 2015, 73 percent had served during a military intervention (see fig.5). Two-thirds reported mental health problems and the vast majority had substance abuse problems. During the clinical assessment performed by VJO specialists, about two-thirds (68 percent) of the veterans reported one or more mental health problems, and 69 percent had substance abuse problems. Public order offenses were the most common type of reported criminal charges. The types of criminal offenses that veterans were charged with during fiscal year 2015 included, but were not limited to, public order offenses (33 percent), drug offenses (22 percent), property offenses (16 percent), and probation violations (12 percent). More than 40 percent reported either being homeless, losing a home, or living in an unstable housing environment (see fig. 6). VJO specialists we interviewed in two of the nine areas we selected said that some veterans become homeless once they leave jail because they are unable to pay their rent while incarcerated. Additional information about the characteristics of veterans in the VJO Program is provided in appendix II. VA has taken some steps toward evaluating the VJO Program via a longitudinal program evaluation that examines the extent to which veterans are being linked to services for which they are referred. VA planned to obtain information through interviews with 1,500 veterans. However, after the contractor submitted an interim report that detailed problems it encountered recruiting veterans to interview as part of the study, VA central office officials terminated the contract. As an alternative, VA plans to use administrative data to complete this evaluation. However, at the time of our review, VA central office officials were in the process of determining how to complete such a re-tooled evaluation. VA also developed a research agenda to evaluate the outcomes for veterans served by the VJO Program. For example, one research topic is to identify outcomes for veterans who participate in veterans treatment courts. In addition, officials said they plan to evaluate the extent to which veterans served by the VJO Program avoided incarceration and homelessness. Although VA has developed five broad strategic goals for the VJO Program and taken steps to evaluate it, VA has not fully measured progress toward any of the strategic goals because it has developed neither performance goals nor performance measures, contrary to leading practices for managing programs. Performance measures focus on whether a program has achieved measurable standards. They allow agencies to monitor and report program accomplishments on an ongoing basis. VA collects ongoing information about program operations and veterans served by the program, but this information does not help VA measure its progress toward accomplishing any of the five goals outlined in its strategic plan. According to VA central office officials, VA collects some information, such as (1) the number of VJO specialists and vacancies at each VA medical center; (2) VJO specialists’ workload, including the number of veterans that VJO specialists serve and their nonclinical activities, such as the number of trainings they conduct; and (3) information on veterans served in the program, such as demographic characteristics and medical histories. This information is useful for some aspects of program management. For example, VA central office officials told us that supervisors need information about the clinical and administrative workloads of VJO specialists to assess the need for additional staff and to monitor productivity. However, VA does not have a way to use the information it currently collects to compare actual program performance against expected results, or to analyze significant differences, contrary to federal standards for internal control. We have previously reported that performance measurement gives managers crucial information to identify gaps in program performance and plan any needed improvements. The information that VA currently collects does not allow VA to fully answer key questions, such as the reasons for observed performance, what are effective approaches to program implementation, and how to improve program performance. VA central office officials cited the need to be responsive to local conditions as the main reason why they have not set or used performance goals and measures for the VJO Program. Specifically, the areas served by each VA medical center have their own unique circumstances, they said, such as having or not having a veterans treatment court, being located in an urban or rural area, or being located in an area with a large or small population of veterans. Due to these unique local conditions, the officials stated, the program needs to be flexible. VA medical centers have discretion in determining the activities of VJO specialists. VA central office officials added that a broad measure, such as the number of veterans served by each VJO specialist per month, would not be appropriate because it could be misleading to compare VJO specialists since their work circumstances vary. For example, officials said that given VJO specialists work in different criminal justice systems, it would not be reasonable to set a national goal for the number of veterans they reach out to each month. While some VJO specialists conduct outreach to a large number of veterans in jails, others may focus their attention on a smaller number of veterans in veterans treatment courts. Similarly, they pointed out that while some VJO specialists in VA medical centers in large urban areas may be able to serve a large number of veterans, others in rural areas may need to drive several hours to see a veteran and can only see a few in any given day. We recognize that measuring progress toward goals within the VJO Program’s decentralized service delivery model poses challenges in designing performance goals and measures. Our past work has also acknowledged the challenges in developing national performance measures when flexible programs vary in their activities so as to meet local needs. Nonetheless, we have found one approach agencies used to successfully overcome this challenge was to develop common national measures. For example, our past work found that to assess the performance of the Expanded Food and Nutrition Education Program—a program that assists low-income families in acquiring skills to improve their family diet—the U.S. Department of Agriculture assessed local offices on common activities used by all offices. Another approach used by agencies with flexible programs was to encourage local projects to evaluate progress toward their own performance goals. For example, the National Tobacco Control Program—a program operated by the Centers for Disease Control and Prevention—has four goals connected to its mission to reduce tobacco-related diseases and death, including reducing youth’s tobacco use. To accomplish its goals, the agency allows states to direct their own activities. The agency provided states with guidance about identifying short term, intermediate, and long term outcomes, and encouraged states to assess their own individual efforts. VA central office officials we interviewed also said that they have not established performance goals or measures because some of the outcomes that veterans experience are influenced by factors outside the program’s control. For example, VJO Program-related outcomes—such as mental health recovery or criminal recidivism—can often depend on factors such as whether a community has a veterans treatment court or if a veteran is willing to adhere to his treatment plan, according to VA central office officials. We have highlighted strategies in our past work that agencies can use when faced with the challenge of having limited control over external factors that can affect a program’s outcomes. These strategies include: selecting a mix of outcome goals over which the agency has varying levels of control; using data on external factors to statistically adjust for their effect on the desired outcome; and disaggregating goals for distinct target populations for which the agency has different expectations. For example, our past work found that to measure progress toward its strategic goal to eliminate transportation related deaths, injuries, and property damage, the National Highway Traffic Safety Administration measured an intermediate outcome—the rate of front-seat safety belt use—and an end outcome—the rate of transportation related injuries. The agency also statistically adjusted the results of its performance measures by using the ratio of fatalities per vehicle mile driven to control for the simple fact that if more miles are driven, then more crashes are likely to result. Our past work also found that the Natural Resources Conservation Service created separate performance goals for each type of habitat (e.g., croplands, watersheds, wetlands, and grazing lands) so it could measure progress toward its strategic goal of maintaining healthy and productive land. Additionally, to help interpret the results of performance measures, we have also emphasized in our past work the importance of communicating adequate contextual information, such as factors inside or outside the agency’s control that might affect performance. VA central office officials said they plan to update the VJO Program’s strategic plan, which contains the program’s broad strategic goals, by the end of fiscal year 2016, but they do not intend to include performance measures. Leading practices demonstrate that developing ways to measure VJO Program efforts could help VJO specialists, VA central office officials, and Congress measure and monitor the extent to which program efforts are achieving their intended results, make needed improvements, and make funding decisions. Further, a fundamental element in an organization’s efforts to manage for results is its ability to set performance goals with specific targets and time frames that reflect strategic goals and to measure progress toward them as part of its strategic-planning efforts. Our previous work has highlighted characteristics of successful performance measures that could be helpful. Without a robust performance assessment system—which includes both performance measures and evaluations—officials managing the VJO Program lack a full picture of its success and of potentially underperforming areas for improvements. VA identified several key challenges related to the demand for services outpacing the VJO Program’s resources, which could limit the program’s capacity to serve all justice-involved veterans. These demand-resource imbalances were identified through VA’s strategic planning process and in consultation with VJO specialists and VISN officials. In developing its list of five challenges, which it reported out in 2012, VA assumed that demands on the VJO Program would continue to increase. Specifically, two of the five challenges VA identified relate to demand for services: (1) demand may increase due to greater use of VA’s Veterans Re-entry Search Service system and result in the program not being able to properly serve all eligible veterans; and, (2) demand from homeless veterans may increase if economic conditions worsen. The remaining three challenges VA identified relate to resources; (3) the program may not be able to link veterans to treatment due to shortages in VA clinical programs; (4) the program lacks direct control over information technology on which it is highly dependent; and, (5) the program may face funding cuts. The range of stakeholders we interviewed—VISN officials, VJO specialists, justice-involved veterans, and local criminal justice system officials—affirmed many of the challenges identified in the VJO Program strategic plan. In general, many stakeholders said that demand for VJO specialist assistance is outpacing the program’s ability to serve all potentially eligible veterans, and the gap may worsen over time. For example, VJO specialists and VISN officials we spoke with in five of the nine areas we selected for interviews told us workload challenges have intensified in recent years. In addition to the high demand for the program cited by program stakeholders, the increase in veterans treatment courts is further increasing demand for program services, according to stakeholders and VA internal documents. In particular: The number of veterans treatment courts is growing. While these courts can help improve veterans’ mental health and sobriety, the increase in the number of these courts is a major reason for the VJO Program’s workload challenges, according to VJO specialists, VISN officials, VA central office officials, and justice system partners we interviewed. According to these stakeholders, VJO specialists had already been working in many local jails and traditional courts across the country, and the expansion of veterans treatment courts added to their existing workload. Specifically, the number of veterans treatment courts nationwide grew from 65 in fiscal year 2010 to 360 in fiscal year 2015, according to VA data. Moreover, hundreds of additional veterans treatment courts are in the planning stages, according to an organization that advocates on behalf of veterans treatment courts. During fiscal years 2012 through 2015, justice-involved veterans served by the VJO Program and who participated in veterans treatment courts increased from about 1,900 to about 3,900. Working with veterans in veterans treatment courts is more time consuming than working with veterans in jails. According to VA central office officials, working with individual veterans participating in veterans treatment courts typically requires more of VJO specialists’ time than working with veterans in jail. VJO specialists we interviewed in seven of our nine selected areas also said working with veterans treatment courts is more time consuming than working in jails. For example, VJO specialists meet regularly with veterans and their treatment providers and apprise other members of the court team of a veteran’s adherence to court-ordered treatment. A veteran in a veterans treatment court generally participates in the VJO Program for 12 to 24 months. In contrast, VJO specialists we interviewed in seven of the nine areas said they generally work with veterans in jails for much shorter durations. According to VA central office officials, VA attempts to ease workload demands by hiring additional VJO specialists, as funds allow, but this has not fully addressed demand for program services. VA increased staffing for the VJO Program from 43 specialists in fiscal year 2010 to 261 in fiscal year 2015, according to VA data. VA central office officials said they have used an annual hiring process to allocate new VJO specialist positions. Through this process they added 13 VJO specialists in 13 locations in fiscal year 2015. VA central office officials said that the process for hiring additional VJO specialists consists of collecting information from VA medical centers about the current clinical and administrative activities of VJO specialists, and about any imminent workload demands, including those attributable to a new veterans treatment court, to determine which VA medical centers receive additional specialist positions. In fiscal year 2015, 54 of the 167 VA medical centers requested 1 or more of the 13 new VJO specialist positions. However, VA central office officials, VJO specialists, and VISN officials acknowledged that despite additional positions, VJO specialists are at capacity and are not able to fully address the demand for VJO Program services. VA central office officials also told us that they try to address the challenge of limited existing workload capacity by advising VJO specialists and VA medical center officials to avoid overcommitting program resources. For example, VA central office officials told us they advised VJO specialists that they should consider their existing workload and commitments before deciding to work with new veterans treatment courts or visit additional jails. Some VJO specialists we spoke with chose to focus on serving veterans in jails while others decided that serving those in veterans treatment courts was more effective. For example, VJO specialists and two criminal justice system officials in one area expressed concerns about the workload and resources associated with serving veterans treatment courts. In their view, a more cost-effective approach is to focus the program’s efforts on providing case management to justice- involved veterans on probation. However, VA central office officials acknowledge that without VA involvement, these courts likely would not function and proliferate. VJO specialists in five areas with veterans treatment courts told us they decided to prioritize working with veterans in treatment courts over incarcerated veterans given their workloads and because veterans participating in these courts are mandated to seek treatment. Specifically, veterans in these courts are generally held accountable for remaining in treatment for longer periods of time than those released from jails, which tends to produce more stable behavioral changes, according to these VJO specialists. A consequence, however, is that the specialists may see fewer veterans in veterans treatment courts or have less time to spend serving those in jails and other criminal justice settings, according to these VJO specialists we interviewed. In addition, almost every justice-involved veteran (13 of 14) we interviewed said that although they have benefitted from the program, VJO specialists do not frequent the jails enough to fully meet veterans’ needs. VA central office officials we interviewed also said they recognize that demand for the VJO Program could increase beyond existing capacity if its Veterans Re-entry Search Service becomes more widely used by jail administrators. VA developed the Veterans Re-entry Search Service in 2013 to meet a key strategic goal to identify justice-involved veterans more effectively. This online system improves the jails’ identification process compared with the self-identifying process currently used by most jail administrators. According to VA central office officials, jail administrators who are using the self-identification process are reporting a much smaller percentage of veterans. For example, an administrator in one local jail reported that 23 inmates self-identified as veterans, but the Veterans Re-entry Search Service revealed that there were 64 incarcerated veterans in the jail during the same time-period. In another jail system, the administrator reported that 220 inmates self-identified as veterans, but the online search revealed there were 400 incarcerated veterans. VA central office officials we interviewed recognize the risk the Veterans Re-entry Search Service poses to existing capacity. VA central office officials told us they plan to promote the new system to jail officials in small phases because they recognize that this system may overtax existing program capacity. VA central office officials told us that they decided to initially promote VA’s new system to 180 of the largest jails rather than roll it out to jail systems (over 3,000 in the United States) so as to avoid overwhelming VJO specialists with an unmanageable influx of new veterans. Once VA completes its outreach to these larger jail systems, it plans to promote the service to other jail systems, and VA central office officials expect about 1,000 jails to eventually use the service. In addition to challenges related to capacity, VJO specialists and VISN officials in several of our selected areas told us that shortages in clinical programs—especially residential drug treatment programs—can be a challenge because it limits treatment options for veterans. They also reported challenges associated with shortages in public housing and public transportation, the latter of which can impede veterans’ ability to readily attend court and medical appointments for treatment. These capacity challenges include: Limited capacity in residential substance abuse programs. VJO specialists and VISN officials in several of our selected areas reported that the high demand for residential substance abuse treatment— which provide appropriate housing to facilitate sobriety—limits the extent to which these programs are a viable option for justice-involved veterans. When openings are unavailable, VJO specialists said they refer veterans to the next best available treatment option. However, officials from one of the veterans treatment courts we interviewed said the next best available treatment option may not always fully meet a veteran’s treatment needs. For example, officials we interviewed at one veterans treatment court said that a veteran in the program has a heroin addiction and has tested positive for drug use. Thus, they would expect this veteran to be in a residential substance abuse treatment program, which provides greater monitoring. In this case, however, VA has recommended out-patient substance abuse treatment due to limited availability of residential slots. Limited housing options for sex offenders. VJO specialists and VISN officials in several of our selected areas also reported that housing options available for individuals who are required to register as sex offenders are limited. VA central office officials said that they are aware of this issue, which cuts across many of VA’s homeless programs. Limited public transportation options. VJO specialists and VISN officials in several of our selected areas also reported that many justice-involved veterans—some of whom have lost their driving licenses due to their offenses—rely on public transportation to take them to court and VA- or community-provided treatment. VJO specialists told us that the public transportation can be slow, limited, or nonexistent. For example, VJO specialists in one area we visited said it can take a full day for a veteran to travel to and from treatment. VJO specialists in another rural location said they have tried to address this challenge by hiring veterans who have received VJO Program assistance to help drive other veterans to their medical appointments. VA has not performed a comprehensive assessment of risks posed by the challenges the VJO Program faces, which is inconsistent with federal standards for internal control and one component of effective program management. These standards call for agencies to both identify all relevant risks and analyze risks that may prevent them from achieving their goals. Further, this assessment should include an estimation of a risk’s significance, an examination of the likelihood of any risk’s occurrence, and a decision as to what actions should be taken to manage the risk. Risk assessment is important because it also informs an entity’s policies, planning, and priorities. Without understanding risks, an agency may not be able to set appropriate policies, plans, and priorities because it would not be able to account for events that can adversely affect its ability to implement its objectives. In 2014, VHA leadership requested that each homelessness prevention program, including the VJO Program, identify risks that may affect veterans’ access to VA supports and services and identify possible actions that could be taken to mitigate those risks. To identify risks, VJO Program central office officials examined their broad strategic goals. In an October 2014 assessment report, VJO Program officials identified six risks. Identified risks included, for example, lack of an automated interface between the Veterans Re-entry Search Service and correctional facilities’ information technology systems to identify incarcerated veterans; reassignment of VJO specialists to other programs by VA medical centers; and prioritizing clinical services for veterans already receiving treatment over new justice-involved veterans by VA medical centers, thus limiting timely access to appropriate services. VA central office officials, as part of this 2014 effort, also estimated the significance and likelihood of the risks, consistent with federal standards for internal control. For example, regarding the significance if VA medical centers were to de-prioritize outreach to veterans, VA found that the program’s relationship with its criminal justice system partners could be weakened and justice-involved veterans may suffer. In addition, the assessment indicated actions VA would take to address each of the six identified risks, such as collecting additional information to monitor identified risks. VA did not, however, identify or analyze the risks posed by each one of the challenges identified in its strategic plan. Notably absent were the workload challenges facing the VJO Program. VA central office officials we interviewed acknowledged that the challenges identified in the strategic plan still exist and could affect the operations of the program. VA expects the VJO Program’s workloads to increase and its funding and workforce to remain level, raising questions about how VA can best deploy its resources and align its policies to best meet increasing demand. Not identifying all relevant risks limits VA’s ability to effectively compare and prioritize risks faced by the program. In addition, VA’s lack of performance goals, as previously discussed, negatively affects its ability to effectively identify and assess risk. As demonstrated in our previous work, federal standards for internal control state that a precondition to risk assessment is the establishment of clear performance goals (see fig. 7). Translating its broad strategic goals into measurable performance goals would allow VA to target the risks that may impede achievement of the program’s objectives. VA plans to conduct another risk assessment as part of its upcoming strategic planning efforts, using the same methodology it used in its risk assessment in 2014. As VA completes future assessments, it is important that the agency incorporates all of the elements of risk assessment detailed in federal standards for internal control, including identifying all relevant risks posed to achieving performance goals. Lacking comprehensive risk assessments may limit VA’s ability to target areas posing the greatest risks and, in turn, develop appropriate mitigation strategies. Perhaps some of the most challenging veterans to serve in this country are veterans who have committed a crime or other offense and who face significant long-term consequences for their actions. If VA intervention is timely and targeted, then these justice-involved veterans and their families have a better chance to avoid a detrimental future. A relatively new VA program, the VJO Program is designed to help justice-involved veterans—who often have high rates of mental illness, substance abuse, and other issues that may stem from their military service—avoid re- incarceration and homelessness. In designing the program, VA intended for support to respond to local conditions. While this flexibility can help the program cater to veterans’ needs in local communities, VA lacks a complete national perspective necessary to strategically manage the program. We are encouraged that VA has taken some initial steps toward assessing program performance and risks. However, as VA moves forward, it is especially important that the agency fully incorporate key practices for assessing performance and risks to strategically guide program decisions. Incorporating these key practices will allow VA to compare the program’s actual performance against expected results and comprehensively assess risks to develop appropriate mitigation strategies. More specifically, without developing a national perspective that comprehensively considers program performance and the greatest risks to the program’s goals, VA cannot fully target its resources toward efforts that achieve the intended results. As a result, justice-involved veterans may not receive the proper supports and services needed to re- establish healthy lives and avoid re-incarceration. If this lack of a national perspective persists, it will be difficult for VA to know whether its resources are being used to serve justice-involved veterans in the best manner possible. To improve management of the Veterans Justice Outreach Program, we recommend that the Secretary of Veterans Affairs direct the Undersecretary for Health to take the following two actions: 1. Establish performance goals with specific targets, time frames, and related performance measures that are linked to strategic goals to provide a basis for comparing actual program performance against expected results; and 2. Conduct a comprehensive assessment of the risks that challenges pose to achieving the program’s strategic and performance goals, and develop, as necessary, applicable mitigation strategies. We provided a draft of this report to the Department of Veterans Affairs (VA) for review and comment. In written comments, which are reproduced in appendix III, VA agreed with our recommendations and noted steps it plans to take to address them. Specifically, VA agreed to develop a plan linking VJO Program strategic goals with performance goals that sets specific targets at the VA medical center level. With regard to risk assessment, VA agreed to conduct a comprehensive assessment of risks using federal standards for internal control and begin collecting information from VA medical centers about workload challenges to fully inform any efforts to redefine how it delivers services. VA did not provide technical comments. We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, and the Under Secretary for Health. In addition, the report is available at no charge on GAO’s website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202)-512-7215 or bertonid@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV. The objectives of this review were to examine: (1) how the Veterans Justice Outreach (VJO) Program delivers services, and the number and characteristics of veterans served by the program; (2) the extent the Department of Veterans Affairs (VA) has used performance assessment to help manage the Veterans Justice Outreach Program; and (3) what key challenges, if any, VA has identified and to what extent the agency has developed mitigation strategies, as necessary. To address all three objectives, we conducted semi-structured interviews and reviewed documentation from nine areas served by VA medical centers. These areas are: (1) Baltimore, Maryland; (2) Bedford, Massachusetts; (3) Chicago, Illinois; (4) Fargo, North Dakota; (5) Houston, Texas; (6) Orlando, Florida; (7) Salt Lake City, Utah; (8) San Diego, California; and (9) Seattle, Washington. For all nine areas we conducted interviews with VJO specialists and Veterans Integrated Service Network (VISN) officials responsible for overseeing the VA medical centers included in our review. In addition, for three of these areas—Baltimore, Maryland; Orlando, Florida; and San Diego, California—we also interviewed VA medical center officials; local criminal justice system stakeholders, including court coordinators and judges, and jail administrators; and held discussion groups with a small nonprobability sample of veterans participating in the VJO Program. The interviews we conducted with officials and stakeholder at the nine areas are nongeneralizable but provided insights on the challenges facing the program and its operations. We also interviewed relevant VA central office officials. We selected the nine areas in our review based on the size of the population of veterans in the area, geographic diversity, VA officials’ recommendations, and proximity to veterans treatment courts. (See table 2.) Specifically: To identify areas with veterans treatment courts, we obtained a list of veterans treatment courts from Justice for Vets, an organization that advocates on behalf of veterans treatment courts. We considered a VA medical center to be located near a veterans treatment court if the court is located in the same state and is within 40 miles of the center. Our selection process included some VA medical centers located near a court and some which were not. We obtained information about the population of veterans from VA’s Veteran Population Projection Model 2014, an actuarial model developed by VA. This model projects the veteran population from fiscal years 2014 through 2043 by using data through fiscal year 2013. We used this model to select VA medical centers in counties with varying numbers of veterans, ranging from approximately 11,000 (in Cass County, North Dakota, which includes Fargo) to 232,000 (in San Diego, California). In addition we used these cross-question methodologies: To better understand how the VJO Program delivers services to veterans, we reviewed relevant federal laws and regulations, VA policies, procedures, guidance, program fact sheets, the VJO Program’s fiscal year 2012-2016 strategic plan, documents from VJO specialists, and other types of documentation that describe the types of program activities and services used to serve justice-involved veterans. We did not independently verify the actions described in such documents. To describe the number and characteristics of the veterans served by the VJO Program, we obtained data from VA. This data included summary information on the number of veterans in the program from fiscal year 2012 through fiscal year 2015 (the most recent data available) as well as sex, age, race, and types of criminal offense. We also obtained fiscal year 2015 data from VA about veterans’ education level, marital status, the number of years since the veterans separated from the military, and mental health status. We determined that VA’s compilation of data about veterans served by the VJO Program was sufficiently reliable to include in our report by reviewing related documentation and interviewing knowledgeable agency officials. Specifically, we obtained and assessed official documentation such as users’ guides, frequently asked questions, and disclaimers, and we discussed our planned use of the data and any limitations with VA officials. We assessed the degree to which VA uses program performance assessment—setting program goals, evaluating programs and using performance measures—to manage the VJO Program by reviewing VA reports and documents, including the program’s fiscal year 2012- 2016 strategic plan, which is the most recent plan; program evaluation plans and preliminary results; and reports on veteran receipt of program services. We also interviewed VA central office officials and other knowledgeable individuals about VA’s current efforts to evaluate the VJO Program and about the program’s goals and efforts to measure progress toward those goals. We compared VA’s use of performance assessment against best practices for assessing program performance and federal standards for internal control. We obtained information on the challenges and VA’s respective mitigation strategies by reviewing VA documents, including a 2014 report assessing program risks. We also interviewed VA central office officials about the challenges and associated risks, how they are addressing the challenges, and what strategies, if any, they have developed to mitigate risks to achieving the program’s goals. We compared VA’s approach for assessing risks with criteria established in the federal standards for internal control. Specifically, we chose to use risk assessment—one of five key components in standards for internal control—because analyzing risk provides the basis for developing appropriate mitigation strategies. We also obtained data from VA on the number of veterans treatment courts and other veteran-focused courts the VJO Program serves, and on the number of full-time VJO specialists from fiscal year 2012 through fiscal year 2015. We compared this list of veterans treatment courts with other authoritative sources and discussed any limitations with VA. We found this number to be sufficiently reliable for our purposes. Table 3 provides data from VA’s Homeless Operations Management and Evaluation System. The selected demographics of veterans include characteristics, military history, criminal justice system involvement, living situation, employment and income, and clinical impression of veterans served by the VJO program during fiscal year 2015. Table 4 provides data on the number of justice-involved veterans by states and the District of Columbia, who were served by the VJO Program during fiscal year 2015. The highest concentrations of justice-involved veterans were in Florida (1,433), California (1,202), Ohio (1,062), Texas (991), and New York (886). In addition to the contact mentioned above, the following staff members made significant contributions to this report: Brett Fallavollita (Assistant Director), James Whitcomb (Analyst in Charge), Hedieh Fusfield, and John Lack. In addition, key support was provided by James Bennett, Joy Booth, Valarie Caracelli, David Chrisinger, Alexander Galuten, Benjamin Licht, Amy Moran Lowe, Paul Schearf, Stephanie Shipman, Almeta Spencer, and Andrew Stavisky.

Summary: Most veterans transition to civilian life trouble-free. For those who struggle with their transition to the point that they are arrested and jailed, VA created the VJO Program, which connects veterans with supports and services to help avoid re-incarceration. The program relies on VJO specialists to link veterans to treatment. GAO was asked to review the management of the VJO Program. This report examines 1) how the program delivers services and the number and characteristics of veterans in the program, 2) the extent to which VA uses performance assessment of the program, and 3) the key challenges VA has identified and the extent to which VA has developed mitigation strategies. GAO obtained VA data on program participants for fiscal years 2012 through 2015; reviewed documents; interviewed VA officials and staff from nine areas served by a VA medical center and selected for their geographic diversity and differences in the structures of local criminal justice systems; and in three of the areas interviewed criminal justice system stakeholders and veterans. While information from these interviews cannot be generalized, they provide insights on program challenges and operations. The Veterans Justice Outreach (VJO) Program—created by the Department of Veterans Affairs (VA)—operates through VA medical centers to provide services to veterans involved in local criminal justice systems, and in fiscal year 2015 served about 46,500 veterans, mostly men and many diagnosed with mental health or substance abuse problems. Officials from VA medical centers manage more than 260 VJO Program specialists who identify veterans in jails and local courts, assess their health and social needs, and link them to supports and services. VJO specialists monitor veterans' services and treatment in courts dedicated to veteran offenders. According to VA data, the number of veterans served by the program increased 72 percent from fiscal years 2012 - 2015. In addition, many veterans involved in the program were Post-9/11 veterans; about two-thirds were diagnosed with one or more mental health problems. VA has taken some steps to incorporate a performance assessment system into the VJO Program, one component of effective program management (see figure). Specifically, VA developed strategic goals and plans to conduct evaluations. However, VA has not established performance goals with related targets, timeframes, and performance measures for any of the program's five broad strategic goals. VA officials told GAO they have not taken this step, in part, because VA medical centers have flexibility in determining the activities of VJO specialists. GAO's past work has highlighted strategies that agencies can use in this situation, such as developing measures based on common activities. Best practices call for agencies to establish performance goals and associated performance measures. Until VA incorporates performance goals and measures, it will lack a systematic way to obtain ongoing information to identify possible underperforming areas for improvements. VA identified several key challenges—most of which were related to the demand for services outpacing the program's resources—but has not fully developed appropriate mitigation strategies. One key challenge, for example, is addressing increased program demand as jail administrators more widely use VA's online system that better identifies incarcerated veterans. In addition, a major reason for the demand-resource imbalance is the heavier workload of VJO specialists serving veterans in an expanding number of courts dedicated to veterans, according to VA officials and stakeholders that GAO interviewed. However, GAO found that VA did not comprehensively identify and assess risks posed by each of the key challenges it identified, contrary to federal internal control standards. Absent a comprehensive risk assessment, VA is not well-positioned to develop appropriate strategies to mitigate the greatest risks, which may limit its ability to help justice-involved veterans receive assistance and avoid re-incarceration. To improve program management, VA should establish performance goals and measures and conduct a comprehensive risk assessment. In commenting on a draft of this report, VA agreed with the recommendations and discussed actions it plans to take to implement them.