VA Vocational Rehabilitation and Employment: Better Incentives, Workforce Planning, and Performance Reporting Could Improve Program
Published: 2009-01-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In 2004, the Veterans Affairs' Vocational Rehabilitation and Employment (VR&E) program was reviewed by a VR&E Task Force. It recommended numerous changes, in particular focusing on employment through a new Five-Track service delivery model and increasing program capacity. Since then, VR&E has worked to implement these recommendations. To help Congress understand whether VR&E is now better prepared to meet the needs of veterans with disabilities, GAO was asked to determine (1) how the implementation of the Five-Track Employment Process has affected VR&E's focus on employment, (2) the extent to which VR&E has taken steps to improve its capacity, and (3) how program outcomes are reported. GAO interviewed officials from VR&E, the 2004 Task Force, and veteran organizations; visited four VR&E offices; surveyed all VR&E officers; and analyzed agency data and reports.
section_title: What GAO Found
paragraphs: By launching the Five-Track Employment Process, VR&E has strengthened its focus on employment, but program incentives have not been updated to reflect this emphasis. VR&E has delineated its services into five tracks to accommodate the different needs of veterans, such as those who need immediate employment as opposed to those who need training to meet their career goal. However, program incentives remain directed toward education and training. Veterans who receive those services collect an allowance, but those who opt exclusively for employment services do not. While VR&E officials said they believed it would be helpful to better align incentives with the employment mission, they have not yet taken steps to address this issue. VR&E has improved its capacity to provide services by increasing its collaboration with other organizations and by hiring more staff, but it lacks a strategic approach to workforce planning. Although there have been staff increases, many of VR&E's regional offices still reported staff and skill shortages. The program is not addressing these workforce problems with strategic planning practices that GAO's prior work has identified as essential. For example, VR&E officials have not fully determined the correct number of staff and the skills they need to serve current and future veterans. VA does not adequately report program outcomes, which could limit understanding of the program's performance. Specifically, it reports one overall rehabilitation rate for veterans pursuing employment and those trying to live independently. Computing each group's success rate for fiscal year 2008, GAO found a lower rate of success for the majority seeking employment and a higher rate of success for the minority seeking independent living than the overall rate. GAO also found that VR&E changed the way it calculates the rehabilitation rate in fiscal year 2006, without acknowledgments in key agency reports. VA noted the change in its fiscal year 2006 performance report, but did not do so for its fiscal year 2007 and 2008 reports, or for its fiscal year 2008and 2009 budget submissions. Such omissions could lead to misinterpretation of program performance over time.

section_title: Letter
paragraphs: The Department of Veterans Affairs (VA) Vocational Rehabilitation and  Employment (VR&E) program helps veterans who have service-connected  disabilities and employment barriers obtain employment or live  independently if employment is not currently feasible. VR&E’s services are  particularly critical now that more than 33,000 military servicemembers  have been wounded in Afghanistan and Iraq since 2001. Many are surviving  with multiple serious injuries and illnesses, including amputations,  traumatic brain injury, and post-traumatic stress disorder. While some  servicemembers will be able to remain on active duty, others will need  comprehensive services as they transition into civilian life and work.  According to the 2007 Veterans’ Disability Benefits Commission, the VR&E  program is pivotal in helping veterans make this transition.
While the program’s historical focus was to provide training for veterans,  legislation in 1980 changed the mission to ensuring that veterans obtain  and maintain suitable employment to the maximum extent possible or  achieve independence in daily living. Since then, we, along with others  who have reviewed the program, have repeatedly raised concerns about  the ability of VR&E to fully meet this charge. In 2004, a VR&E Task Force,  chartered by the Secretary of Veterans Affairs, issued a comprehensive  report with more than 100 recommendations. Key concerns included  VR&E’s lack of emphasis on employment early in the rehabilitation  process and its limited capacity to manage its workload. To refocus  VR&E’s organization and make it responsive to the needs of 21st century  veterans, the Task Force recommended, among other actions, that VR&E  implement a new service delivery approach through a Five-Track  Employment Process to focus on employment early in the rehabilitation  process and give veterans five different program options to achieve  employment based on their individual needs. The Task Force also  recommended expanding collaboration with other organizations and  increasing staffing levels at the central and regional offices to increase  capacity. Since 2004, VR&E has redesigned its program around the Task  Force’s recommendations.
At your request we reviewed (1) how the implementation of the Five-Track  Employment Process has affected VR&E’s focus on employment, (2) the  extent to which VR&E has taken steps to improve its capacity, and (3) how  program outcomes are reported. To assess how the Five-Track process  has affected VR&E’s focus on employment and the extent to which VR&E  has taken steps to improve its capacity, we evaluated VR&E’s  implementation of key recommendations from the 2004 Task Force by  interviewing VR&E officials, Task Force members, and veteran service  organization representatives as well as reviewing agency documents. We  also conducted site visits to four of VA’s regional offices—Houston, Tex.;  Pittsburgh, Pa.; Seattle, Wash.; and St. Petersburg, Fla. We selected our  site visit locations based on several criteria including their proximity to  major military installations, the number of program participants, and  overall performance scores on various VR&E management reports.  Additionally, we surveyed the VR&E officers in all 57 regional offices  regarding their workload. To review how program outcomes are reported,  we examined the agency’s annual performance and accountability reports  and congressional budget submissions. We also analyzed data from  VR&E’s Corporate WINRS database and VA’s Benefits Delivery Network  (BDN). Before doing so, we assessed the reliability of the data and found  them to be sufficiently reliable for the purposes of this analysis. See  appendix I for more information regarding our methods.
We conducted this performance audit from July 2007 to January 2009 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: Since the 1940s, VA has provided vocational rehabilitation assistance to  veterans with service-connected disabilities. In 1980, Congress enacted the  Veterans’ Rehabilitation and Education Amendments, which mandated a  change in the mission of VA’s vocational rehabilitation program from  primarily providing training to helping veterans find and maintain  employment, or achieve independence in their daily lives if employment is  not currently feasible. VA reported that VR&E served 90,600 participants  in fiscal year 2007 at a cost of $722 million.
subsections: section_title: VR&E Structure and Eligibility Requirements
paragraphs: There are 57 VA regional offices—roughly about 1 in each state—and  about 1,000 VR&E staff who work in these regional offices and at the  program’s central office in Washington, D.C. VR&E regional office  personnel include rehabilitation counselors, employment coordinators,  and management and support staff who provide personal, face-to-face  services to veterans. VR&E services can include vocational counseling,  vocational evaluation, case management, education and training, job  placement assistance, and independent living services. VR&E can also pay  tuition, subsistence, and other expenses for veterans pursuing education  and training. An allowance is provided to veterans who have completed  training programs for up to 2 months as they seek employment. When  necessary, VR&E can also direct veterans to other vocational and  employment counselors and specialists who perform services under  contract. To receive VR&E services, veterans with disabilities generally  must have a 20 percent disability rating and an employment handicap.  Veterans with a 10 percent disability rating may also be entitled to receive  services if they have a serious employment handicap. In addition, injured  servicemembers may be eligible for VR&E services before being  discharged from the military if they request a memorandum rating from VA  and are found to have one or more service-connected disabilities that are  20 percent or higher. VR&E vocational rehabilitation counselors  determine entitlement to services, which generally provides a 12-year  period of eligibility and up to 48 months of benefits.
section_title: Federal and State Programs for Veterans with Disabilities
paragraphs: VR&E is one of many federal and state programs available to veterans with  disabilities in their transition from the military to civilian life and work.  Injured servicemembers can receive medical treatment from the  Department of Defense (DOD) military treatment facilities or Veterans  Health Administration facilities, such as polytrauma centers, which may  also provide vocational rehabilitation services. Within VA, the  Compensated Work Therapy program primarily helps veterans with  mental health diagnoses by integrating vocational rehabilitation into their  overall medical treatment plan and placing them in jobs. In addition, VA  works with DOD and the Department of Labor (Labor) to provide  presentations to servicemembers being discharged about veterans’  benefits and services through the Transition Assistance Program and  Disabled Transition Assistance Program. Labor’s Veterans’ Employment &  Training Service (VETS) also provides services to veterans. Labor and VA  have historically worked together to help veterans with service-connected  disabilities transition to the civilian workforce. Labor administers the  VETS program through grants to state workforce agencies, whose staff  provide veterans with reemployment services, such as job search and  placement assistance and also market veterans to employers. In addition,  VR&E works with state vocational rehabilitation agencies that receive  grants from the Rehabilitation Services Administration at the Department  of Education to prepare individuals with disabilities for employment  through vocational rehabilitation services.
section_title: Long-standing Critical Problems
paragraphs: For more than 25 years, we, along with others who have reviewed the  program, veteran service organizations, and VA, have found shortcomings  in the VR&E program. These reviews generally concluded that the  program had not fulfilled its primary purpose, which is to ensure that  veterans obtain suitable employment. In 1996, we reported that the  program primarily emphasized providing training and did not place enough  emphasis on providing employment services. Additionally, the 1999  Congressional Commission on Servicemembers and Veterans Transition  Assistance found that VR&E had not achieved its statutory purpose and  noted that “employment assistance is the most valuable service the Nation  can provide to personnel transitioning from active duty to the civilian  workforce.” In 2003, we designated federal disability programs, including  those at VA, as high risk because they had difficulty managing their  programs and were in need of transformation.
section_title: VR&E Has Strengthened Its Focus on Employment through the Five-Track Employment Process, but Has Not Updated Its Incentive Structure to Align with Its Mission
paragraphs: In response to recommendations from the 2004 Task Force, VR&E has  implemented the Five-Track Employment Process and strengthened the  program’s focus on employment. However, VR&E’s incentive structure for  veterans remains primarily aligned with education and training programs,  with no financial incentive for those seeking immediate employment.
subsections: section_title: VR&E Has Implemented Its Five-Track Employment Process and Strengthened Its Focus on Employment
paragraphs: In response to the 2004 Task Force report, VR&E implemented the Five- Track process by delineating its existing services into five distinct tracks  to provide a stronger focus on employment early in the rehabilitation  process. The delineation of program services into five tracks is designed to  accommodate the different needs of veterans, such as those who need  immediate employment as opposed to those who need training to meet  their career goal. Figure 1 provides details on each of the five tracks.
After veterans apply to the program and are found eligible for services,  they are introduced to the Five-Track process through a program  orientation. During orientation, VR&E shows a video that explains the  process to veterans and emphasizes that the goal of the program is to  obtain employment or to achieve independent living if employment is not  immediately feasible. At the sites we visited, we found that VR&E staff  also verbally reinforced to veterans during orientation that the primary  goal of the program is employment.
Following orientation and evaluation, veterans are assisted by VR&E staff  in selecting a track that meets their needs and employment goals. Some of  the rehabilitation counselors we interviewed told us the factors they  consider when evaluating veterans for track selection include veterans’  transferable job skills, results on various vocational tests, and how the  veterans’ disabilities affect their ability to do the work they did in the past.  Of the almost 24,000 veterans with a documented track selection who  began a plan of services from January 2007 to early May 2008, we found  that more than three-quarters chose to pursue employment through the  long-term services track, which includes education and training, while less  than one-tenth chose more immediate employment through the  reemployment or rapid access to employment tracks, and slightly more  than one-tenth entered the independent living track. Very few veterans  chose self-employment (less than 1 percent). See figure 2 for the  percentages of veterans who entered each of the five tracks.  lso as part of the Five-Track process, VR&E established an employment  A coordinator position and job labs to assist veterans with preparing for andfinding employment. Employment coordinators assess veterans’ readiness  to seek employment, develop relationships with employers, and help place  veterans in jobs. VR&E’s job labs provide computers with employment- related software that VR&E staff and veterans can use for activities such as developing job search plans, preparing for interviews, and writing  resumes.
section_title: Incentives for Veterans Remain Primarily Aligned with Training
paragraphs: Though the Five-Track Employment Process was intended to modernize  the program and increase VR&E’s emphasis on employment, VR&E did no update its financial incentive structure to align with its mission.  Specifically, the program offers a monthly subsistence allowance those veterans who are enrolled in education or training, but not to those  who receive employment services only. For example, a veteran who has  two dependents and enrolls in a full-time education or training program  receives approximately $760 in monthly assistance. That veteran would  continue to receive this allowance for 2 months following training while  or she seeks employment. One rehabilitation counselor we spoke with  noted that many veterans who have completed their training rely on this money during the job search phase. In contrast, veterans who receive  employment services only do not receive a monthly allowance while th look for employment. Our prior work has noted the need to consider basic program design, particularly those features that affect individual work  incentives and supports, when modernizing disability programs for the  21st century. Based on our prior work, we are concerned that without  properly aligned incentives and supports, veterans who need assistance  finding immediate employment may not seek out VR&E services and  others may not choose the track that is best suited for them. In our  discussions with senior VR&E officials, they acknowledged that offe ring  financial incentives for veterans receiving employment services could be  beneficial, and noted that they may review the internal incentive structure ey  as part of a program evaluation in fiscal year 2009. Additionally, in  September 2008, VA released a study on overall veterans’ compensa payments that included several options for changing the subsistence  allowance for VR&E participants. to align incentives with the program’s employment mission.
section_title: VR&E Has Collaborated with Other Organization s and Added Staff, but Lacks a Strategic Approach to Workforce Planning
paragraphs: Over the last few years, VR&E has increased its capacity to serve veterans  by engaging in a number of collaborative initiatives with other  organizations and by adding staff to its central and regional offices.  Nevertheless, the program continues to face challenges ensuring it h right number of staff with the right skills, and its workforce planning has  not strategically addressed these issues.
subsections: section_title: on a Number of Initiatives
paragraphs: VR&E has increased its collaboration with other organizations such as  federal and state agencies, as well as private and nonprofit employers  through initiatives to help injured servicemembers and disabled veterans  transition to the civilian workforce. Initiatives with DOD focus on  intervention and employment services for injured servicemembers  their recovery process, while VR&E’s partnership with VA’s Compensated  Work Therapy (CWT) program addresses the vocational rehabilitation  needs of veterans who may have mental illnesses or traumatic brain inju In addition, VR&E’s collaborative efforts with Labor, state vocational  rehabilitation agencies, and employers provide employment services to  veterans who are ready to enter the job market. VR&E’s recent efforts to  collaborate with these organizations are highlighted below. ry.
subsections: section_title: VR&E and DOD
paragraphs: Economic Systems Inc., A Study of Compensation Payments for Service-Connected  Disabilities, a special report prepared at the request of the Department of Veterans Affairs,  September 2008.
In fiscal year 2005, VR&E created a standardized presentation for the  Disabled Transition Assistance Program (DTAP), which informs disabled  servicemembers of the full range of benefits and services available to them  once they leave active duty. VR&E has assigned rehabilitation counselors  or contractors to present this information at DOD installations and  military treatment facilities. According to a senior VR&E official, VR&E is  also increasing outreach to National Guard and Reserve servicemembers  by providing information about this DTAP briefing at required post- deployment health assessments.
In fiscal year 2007, VA and DOD began to share information earlier about  seriously injured servicemembers, and VR&E now has access to a  database that allows it to identify and locate them to facilitate early  outreach.
In fiscal year 2008, VR&E rolled out the Coming Home to Work (CHTW)  initiative nationwide. This key component of VR&E’s early intervention  efforts provides counseling to individuals on active duty pending medical  separation and rehabilitation services to eligible servicemembers.  According to officials, VR&E has placed 13 full-time rehabilitation  counselors at 12 military treatment facilities to administer this program  and initiate early contact with injured servicemembers. In addition to  these 13 counselors, VR&E has designated one staff member in each  regional office as the program coordinator. As of August 2008, over 4,000  servicemembers had received counseling through CHTW and 149  servicemembers who received rehabilitation services had obtained  employment, according to VR&E officials.
section_title: VR&E and VA’s Compensated Work Therapy (CWT) Program
paragraphs: In another effort to provide services to seriously injured veterans early in  their treatment process, VR&E has taken steps to develop a partnership  with the CWT program at VA. The CWT program works primarily with  veterans that many VR&E regional officials said their staff had difficulty  serving. Such veterans might have a traumatic brain injury or mental  health diagnosis, or may need more intensive support in the structured  environment CWT provides. CWT’s early intervention model addresses  both employment goals and medical rehabilitation needs. Also, veterans  receiving services simultaneously from VR&E and the CWT program can  continue to receive services from CWT even after VR&E education and  training benefits are exhausted, according to officials. The 2004 Task  Force noted the potential advantages of increased collaboration between  VR&E and CWT. According to officials from both programs:    VR&E refers veterans to the CWT program. Regional officials at the four  sites we visited said their staff refer veterans to this program when it is  appropriate.
VR&E and CWT briefed each other’s staff at their national training  conferences in fiscal year 2008.
VR&E plans to provide a 1-hour training session for VR&E staff on the  CWT program via satellite broadcast in fiscal year 2009.
section_title: VR&E and Labor
paragraphs: The Department of Labor is VR&E’s primary employment services partner,  and an effective relationship between these agencies is important in giving  disabled veterans the best chance for successful outcomes. Recent  collaborative efforts include the following:  In fiscal year 2006, VR&E and Labor renewed their existing agreement to  improve employment services to veterans with disabilities.
In fiscal year 2006, Labor and VR&E implemented some elements of their  renewed agreement by establishing a joint work group at the national level  to develop a set of shared performance measures.
In fiscal year 2008, Labor and VR&E completed a demonstration project at  eight regional offices to develop and test joint performance measures,  tracking systems, and training curriculums for their staff who provide  employment services to veterans.
section_title: VR&E and State Vocational Rehabilitation Agencies
paragraphs: The 2004 Task Force highlighted the importance of collaboration between  VR&E and state vocational rehabilitation agencies, noting that state  vocational rehabilitation agencies have established extensive employer  networks and could provide veterans with greater access to employment  opportunities. In addition to these increased employment opportunities,  agency officials also noted that close relationships between VR&E and  these agencies could result in joint rehabilitation plans that can provide  complementary services to veterans. For example, veterans who are  jointly served by VR&E and a state vocational rehabilitation agency have  access to more and different services, such as transportation assistance or  a clothing allowance provided by state agencies, which may make the  difference in a veteran’s ability to achieve rehabilitation and employment  goals. According to officials, recent collaborative efforts with state  vocational rehabilitation agencies have included the following:  In fiscal year 2004, VR&E and the Council of State Administrators of  Vocational Rehabilitation (CSAVR), a professional association of state  vocational rehabilitation administrators, formally agreed to facilitate local  cooperative agreements between state vocational rehabilitation agencies  and VR&E regional offices. The purpose of these local agreements is to  encourage collaboration that will result in improved services and  increased employment outcomes for disabled veterans.
In fiscal year 2008, the central office staff of VR&E and CSAVR exchanged  local office contact information.
In fiscal year 2008, VR&E and state vocational rehabilitation officials  briefed each other’s staff at national conferences.
section_title: VR&E and Employers
paragraphs: VR&E has established national agreements with several private, public,  and nonprofit employers to further increase employment opportunities for  veterans. These agreements focus on joint efforts to provide career  opportunities to veterans exiting the VR&E program. VR&E central office  officials said that they inform the regional offices of new national  agreements via monthly conference calls and disseminate copies of the  agreements.
Finally, a senior VR&E official said that the program currently coordinates  individually, as opposed to jointly, with its various partners—DOD, VA’s  CWT program, Labor, and state vocational rehabilitation agencies. This  official also noted that VR&E had recently contributed to a forthcoming  report on strategies for building capacity and tools for improving  coordination among federal and state agencies, including several listed  above. The report is expected to identify promising practices for  addressing gaps in services.
section_title: VR&E Has Added Staff, but Strategic Workforce Planning Has Been Insufficient
paragraphs: VR&E has increased staffing at its central and regional offices as  recommended by the 2004 Task Force. Specifically, VR&E officials said  they increased central office staff by 67 percent, from 33 in fiscal year 2004  to 55 in fiscal year 2008, to address the concern that the central office  needed more resources to provide policy, procedures, and staff training to  the regional offices. At the four sites we visited, some regional office staff  said support and training from the central office had improved. VR&E also  increased its regional office staff by 20 percent, from 917 in fiscal year  2004 to 1,101 in fiscal year 2008. A senior VR&E official said these new  regional office staff include contracting specialists and counselors, as well  as positions to provide outreach to veterans returning from the wars in  Afghanistan and Iraq.
Despite these staff increases, the VR&E regional offices still reported staff  and skill shortages on our survey. In terms of staff shortages, more than  half of all 57 regional offices said they have fewer counselors than they  need and more than a third said they have fewer employment coordinators  than they need (see fig. 3). Some employment coordinators we  interviewed told us it is difficult for them to provide services to veterans  and reach employers throughout their entire regions, including those in  more rural locations. Exacerbating these staff shortages is the fact that  staff time may not be used efficiently as many regional office staff we  interviewed and surveyed said much of their time was spent on redundant  paperwork and data entry requirements that reduced the amount of time  they spent with veterans. In terms of skill shortages, almost one-third of  the regional offices reported that the skills of their counselors no more  than moderately meet the needs of the veterans they serve and almost one- third reported the same for their employment coordinators. Moreover,   80 percent of offices said VR&E was somewhat or less prepared to meet  the needs of veterans in the future, and, of these, 12 percent reported  VR&E was unprepared.
We found that these workforce problems were not being addressed with  some of the strategic planning practices that our prior work has identified  as essential, such as:    Using data to identify current and future human capital needs including  the appropriate number of employees, how they are deployed across the  organization, and existing opportunities to reshape the workforce by  improving current work processes; and    Determining the critical skills and competencies staff will need to  successfully achieve the organization’s mission and goals, especially as  various factors change the environment in which the organization  operates.
VR&E has not gathered data to identify the number of staff it currently  needs. The 2004 Task Force recommended a study of the time required for  key tasks and VR&E identified the need for such a study in its fiscal year  2005 - 2008 workforce plan; however, the study has not yet been  conducted. While VR&E officials told us they have plans to fund the study  in fiscal year 2009, they acknowledged that without such information they  do not know whether their current caseload target is appropriate.  Moreover, without knowing what their target caseload should be, VR&E  cannot know the total number of counselors the program needs. VR&E  officials said the current caseload target, which is one counselor for every  125 veterans, is based on a study of the state vocational rehabilitation  programs, not VR&E’s own workloads. Nevertheless, the state study  concluded that a caseload of this size would leave counselors little time to  spend with clients. We learned from our survey of VR&E regional offices  that their estimated average caseload was one counselor for every 136  veterans.
In addition, the program has not studied its work processes since the roll- out of the Five-Track process to determine whether and how to streamline  administrative activities to allow staff to use their time more efficiently.  Many survey respondents, as well as staff we interviewed, reported that  administrative paperwork was cumbersome and labor intensive.  According to staff at one regional office, paperwork requirements were a  concern when the Five-Track process was rolled out, but documentation  requirements did not ultimately change and new paperwork was added. At  another regional office, a staff member noted that the decision regarding a  veteran’s entitlement to services had to be documented multiple times. A  VR&E central office official said the program is working to transition to  one database, which will reduce redundant data entry. Additionally, the  official said that while new forms had been added to ensure consistent  documentation across all regional offices, these requirements will be  reviewed as part of the fiscal year 2009 study of counselors’ key tasks.
VR&E also does not use relevant data to identify future staffing needs.  While a VR&E official said that the program considers potential factors  such as the impact of the wars in Afghanistan and Iraq, the only data used  to project future workloads and staff needs are the program’s historical  participation rates. Moreover, while VR&E does review the numbers of  new disability claims, this official said these numbers are not formally  factored into its projections nor does the program routinely determine  what proportion of this population subsequently applies for VR&E services  or when they apply. We found a decrease in the average number of years  between a veteran receiving an initial disability rating and applying for  VR&E services from 7.9 years in fiscal year 2002 to 6.1 years in fiscal year  2007. A VR&E official said this decrease is expected due to the program’s  increased outreach to servicemembers and veterans. VR&E officials said  that their past workload projections had not been far off and, according to  VR&E data, since 2004 their projections have been within 8 percent of  actual program participation. However, new factors may be impacting  enrollment because in fiscal year 2008 the program underestimated the  number of program participants for the first time in several years.
Further, VR&E staffing projections do not account for the numbers of  veterans whose status will likely require more staff time, such as veterans  who need an extended evaluation to determine if employment is currently  feasible. Staff are allocated to the regional offices based, in part, on the  number of veterans whose status will likely take more of a counselor’s  time. However, when VR&E prepares its annual budget request for staff, it  considers only total program participants and does not take into  consideration the growing number of cases that require more staff time  due to their complexity. Yet, since the wars began in Afghanistan and Iraq,  the number of veterans who required an extended evaluation increased by  121 percent. While a senior VR&E official said the model for projecting  the program’s overall staff needs is not intended to be the same as the one  for allocating staff to regional offices, a senior VA official acknowledged  that VR&E could improve its workload management with better  projections.
In addition, VR&E officials said they have not fully determined the critical  skills and competencies needed by counselors and employment  coordinators to achieve the program’s goals. While officials in 2003  conducted an analysis of job duties and associated tasks for counselors,  this was not an analysis of the skills and competencies required to perform  those tasks or the skills that might be needed in the future.
Determining the relevant skills and competencies that counselors and  employment coordinators need may be particularly important now, given  the changing needs of veterans. About 90 percent of the regional offices  we surveyed reported that their caseloads have become more complex  since veterans began returning from Afghanistan and Iraq. They reported  dealing with multiple physical injuries as well as traumatic brain injury  and post-traumatic stress disorder among veterans returning from war.  One official noted that, while her staff are skilled, they are not experts in  traumatic injuries and psychiatric conditions, and could benefit from  additional training in these areas.
section_title: Performance and Budget Reports Lack Important Information about VR&E Program Outcomes
paragraphs: VA performance and budget reports lack important information about the  outcomes of the VR&E program. VA does not report specific performance  information for the two different groups of veterans VR&E serves—those  seeking employment and those seeking to live independently. In addition,  it has not adequately disclosed a change to its primary performance  measure. These omissions could lead to some misinterpretation of the  program’s performance.
subsections: section_title: Reports Do Not Show Separate Success Rates for Veterans Seeking Employment and Independent Living
paragraphs: Although the VR&E program works with two different groups of veterans,  most of whom are focused on employment with a smaller number seeking  independent living, VA reports an overall rehabilitation rate for all  participants. We found that this single measure masks the individual  outcome for each group of participants and may hinder oversight. For  example, VA reported a rehabilitation rate of 76 percent in fiscal year 2008.  When we computed the rates for each group of veterans we found that   73 percent of those seeking employment were successful, while 92 percent  seeking independent living were successful (see fig. 4).
Information on separate success rates would result in better information  for Congress and others to evaluate program performance and target  services. For example, reporting separate rates would show that those  participants seeking employment—the majority of people in the  program—have a lower success rate than the overall rate currently  reported. Likewise, information on separate success rates would enable  those overseeing the program to understand that the minority of  participants seeking independent living have a much higher success rate  than the reported overall rate.
Both the Task Force and VA’s Office of Inspector General (OIG) have also  noted the need for separate employment and independent living  measures. The Task Force recommended the use of separate outcome  measures because very different services are often required to serve those  seeking employment versus those seeking independent living. For  example, veterans seeking employment may need career training and  placement, while veterans trying to live independently may need to learn  to use a wheelchair or communicate with an assistive device. VR&E  officials did not implement the recommendation because, according to  officials, the existing rehabilitation rate reflected the outcomes of all  veterans in the program. For its part, VA’s OIG specifically recommended  in 2007 that VA performance and accountability reports include the  numbers of veterans who achieve employment and independent living,  given that such outcomes are used for budget and resource allocation and  in testimony to Congress. In its 2007 Performance and Accountability  Report, VA provided the absolute numbers of veterans who had found  employment (8,252) or achieved independent living (2,756), but did not  offer a separate rate for each program goal, which would have allowed for  a better assessment of VR&E’s progress. During our review, a senior VR&E  official acknowledged the merit of examining separate employment and  independent living rates and said that the program had recently begun  internally tracking separate rates. Another VR&E official told us that the  program is considering developing and reporting separate performance  measures for independent living and employment, but did not have a  specific time frame for when that decision will be finalized.
section_title: VA Altered Its Performance Measure without Adequately Disclosing the Change
paragraphs: In fiscal year 2006, VR&E changed its rehabilitation performance  measure—the way it calculates the overall rehabilitation rate—without  adequately disclosing this change in several subsequent reports even  though the change substantially increased the rate. VA noted the change in  its fiscal year 2006 Performance and Accountability Report, but did not do  so for its subsequent fiscal year 2007 and fiscal year 2008 Performance and  Accountability Reports, or for its fiscal year 2008 and fiscal year 2009  budget submissions to Congress. These reports included tables and  graphics showing a 10-point increase in the rehabilitation rate from fiscal  year 2005 to fiscal year 2006. While federal agencies may change their  performance measures, we believe that not acknowledging the change in  subsequent reports could allow for some misinterpretation of the  program’s performance over time. Our prior work on federal performance  measures found it useful to acknowledge such a change to provide a  complete picture of program performance.
Prior to fiscal year 2006, VR&E calculated the rehabilitation rate by  comparing the number of veterans who had a rehabilitation plan and  achieved their goal with the total number of veterans who had a  rehabilitation plan and either achieved their goal or discontinued the  program. In fiscal year 2006, VR&E began excluding from the total those  veterans who discontinued from the program for reasons considered  beyond VR&E’s control (see fig. 5). Specifically, VR&E excludes veterans  from the calculation who accept a position incompatible with their  disability; those they consider employable, but who are no longer seeking  employment; and those they consider unemployable due to medical or  psychological reasons.
Prior to the calculation change, VR&E was having limited success  improving its rehabilitation rate and achieving its performance goals (see  fig. 6). Changing the calculation enabled VR&E to show a 14-point increase  (from 62 percent in fiscal year 2004 to 76 percent in fiscal year 2008) in the  rehabilitation rate trend in its fiscal year 2008 Performance and  Accountability Report. According to our analysis, the increase would have  been 6 points (from 62 percent in fiscal year 2004 to 68 percent in fiscal  year 2008) without a change to the performance measure. Furthermore,  the calculation change enabled VR&E to meet its annual performance goal  in fiscal years 2006, 2007, and 2008. We are concerned that this  performance data, as currently reported without an explanation of the  calculation change, could convey a misleading picture of the program’s  performance over time.
section_title: Conclusions
paragraphs: For more than 20 years, VR&E has sought to modernize its program and  meet its employment mandate. VR&E launched its new Five-Track  Employment Process to better focus on employment; however, critical  aspects of the program have not been aligned with the employment  mission. Given the current incentive structure, veterans who most need  immediate employment services, but could also benefit from some level of  financial assistance, may be at a disadvantage. Moreover, the incentive  structure may result in some veterans not choosing the track that is best  for them and, therefore, foregoing early integration into the civilian  workforce.
VR&E has improved its capacity to serve veterans by stepping up its  collaboration with other organizations and by adding staff. However, the  lack of information about staffing needs could limit VR&E’s ability to  provide quality services to veterans returning from the wars in Afghanistan  and Iraq, as well as to veterans from prior conflicts. Without a strategic  workforce planning process that collects and uses relevant data to ensure  the right number of staff with the appropriate skills, the VR&E program  will continue to face challenges serving current veterans and could fall  short in responding to the needs of future veterans.
Finally, the lack of transparency in how VA calculates and reports  program performance is detrimental to effective oversight and VR&E’s  ability to manage the program. Without transparency in program outcomes  and how performance measures are calculated, Congress and other  stakeholders lack important information that highlights the program’s  successes and focuses their attention on its shortcomings. In addition, VA  officials lack essential information to manage and make adjustments to the  program.
section_title: Recommendations for Executive Action
paragraphs: To ensure VR&E’s employment mission is fully supported, we recommend  that the Secretary of Veterans Affairs direct VR&E to consider cost- effective options for better aligning the program’s financial incentives with  its employment mission.
To ensure that the current and future needs of veterans are met, we  recommend that the Secretary of Veterans Affairs direct VR&E to engage  in a strategic workforce planning process that collects and uses relevant  data, such as information on the appropriate counselor caseload and the  critical skills and competencies needed by staff.
To increase transparency in VR&E performance and budget reports, we  recommend that the Secretary of Veterans Affairs take actions such as  separately reporting both the annual percentage of veterans who obtain  employment and the percentage of those who achieve independent living,  and fully disclosing changes in performance measure calculations when  reporting trend data in key performance and budget reports.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to VA for review and comment. The  agency provided written comments, which are reproduced in appendix II.  VA generally agreed with our recommendations and noted the steps it will  take to act on them:  In response to our recommendation that VR&E consider cost-effective  options for better aligning the program’s financial incentives with its  employment mission, VA agreed and stated that the current law does not  permit payments of subsistence allowance to veterans receiving only  employment services. Therefore, to address this issue, VR&E has drafted a  legislative proposal for consideration by the Secretary of Veterans Affairs.
In response to our recommendation that VR&E engage in a strategic  workforce planning process that collects and uses relevant data, such as  information on the appropriate counselor caseload and the critical skills  and competencies needed by staff, VA agreed and outlined its plans to  implement the recommendation. With regard to collecting and using  information on the appropriate counselor caseload, VA stated that it plans  to complete a study by the end of fiscal year 2010 that will help it  determine the staffing levels necessary to comprehensively meet veterans’  rehabilitation needs. With regard to collecting and using information on  the critical skills and competencies needed by staff, VA noted that it has  already defined the critical skills and competencies needed for VR&E  counselors by requiring them to hold a master’s degree in rehabilitation  and has provided training to VR&E staff. While we acknowledge the value  of these efforts, the fact that many regional offices reported skill shortages  on our survey indicates that more needs to be done in this area, especially  given the increasingly complex needs of the veterans now applying for  services. VA did agree to conduct a skills assessment survey of VR&E staff  and indicated that the survey will determine the skills staff currently  possess as well as the skills staff need to successfully serve veterans.  Additionally, VA agreed to ensure staff training is targeted to the specific  skills and competencies identified on the survey.
In response to our recommendation that VA separately report the annual  percentage of veterans who obtain employment and the percentage of  those who achieve independent living and fully disclose changes in  performance measures, VA agreed and stated that it will include  employment and independent living rates in the comments of its fiscal  year 2010 budget and fiscal year 2009 Performance and Accountability  Report and will implement separate performance measures in fiscal year  2010. Additionally, VA stated that it would note the year the rehabilitation  rate calculation changed in future budget and performance and  accountability documents.
We are sending copies of this report to the Secretary of Veterans Affairs,  relevant congressional committees, and other interested parties. The  report will also be available at no charge on the GAO Web site at  http://www.gao.gov.
A list of related GAO products is included at the end of this report. If you  or your staff have any questions about this report, please contact me at  (202) 512-7215 or bertonid@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our review examined (1) how the implementation of the Five-Track  Employment Process has affected the Vocational Rehabilitation and  Employment (VR&E) program’s focus on employment, (2) the extent to  which VR&E has taken steps to improve its capacity, and (3) how program  outcomes are reported. To address these objectives, we:  reviewed agency documents and relevant recommendations from key  reports, such as the 2004 VR&E Task Force;   analyzed data from the Department of Veterans Affairs’ (VA) Corporate  WINRS and Benefits Delivery Network (BDN) data systems; interviewed VA and VR&E staff knowledgeable about VR&E planning and  operations, and others such as disability experts, members of the 2004  Task Force, veteran service organization representatives, and staff from  agencies and organizations that collaborate with VR&E;   visited four VA regional offices and conducted interviews with VR&E  officers and staff to observe and gather information on workforce  planning and how services are provided to veterans; and   conducted a survey of VR&E officers at all 57 regional offices to follow up  on several key issues relevant to our research objectives.
We conducted this performance audit from July 2007 to January 2009 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Review of Key Documentation
paragraphs: To evaluate our objectives, we reviewed agency documentation and prior  evaluations of the VR&E program and recommendations made by the 2004  VR&E Task Force, key commissions, the VA Office of Inspector General,  as well as our own previous work. To evaluate how the Five-Track  Employment Process has affected VR&E’s focus on employment and the  extent to which VR&E has taken steps to improve its capacity, we  identified key recommendations from the 2004 Task Force report by  reviewing and selecting recommendations related to the following areas:  program focus on employment; workforce and workload management;  collaboration with outside agencies and organizations; and performance  measures. We assessed VR&E’s implementation of completed  recommendations and reviewed recommendations it had not yet  completed. We also referred to our previous work on strategic workforce  planning and the Office of Personnel Management’s Human Capital  Assessment and Accountability Framework. To evaluate reports on  VR&E’s program outcomes, we reviewed recent agency performance data  in VA’s fiscal year 2006, 2007, and 2008 annual performance and  accountability reports and congressional budget submissions for fiscal  years 2008 and 2009.
section_title: Review of VA Data on Veterans’ Case Statuses, Employment Track Usage, and Program Performance
paragraphs: We used data from VA’s Corporate WINRS case management system and  its BDN system to evaluate the number of veterans in each case status, the  number of veteran’s enrolled in each of the five tracks, the amount of time  between veterans receiving an initial disability rating and applying for  VR&E services, and VR&E program outcomes reporting. To evaluate the  number of veterans in each case status over time, we used BDN fiscal  year-end national reports from fiscal year 2001 to fiscal year 2007 to  capture changes since the beginning of the conflicts in Afghanistan and  Iraq. We also analyzed BDN and Corporate WINRS data to determine the  change in the average length of time between a veteran receiving an initial  disability rating and applying for VR&E services from fiscal year 2002 to  fiscal year 2007. We began our analysis with fiscal year 2002 because an  agency official told us that regional office data were uploaded into the  Corporate WINRS database in fiscal year 2001 making data prior to fiscal  year 2002 less reliable. For performance outcomes reporting, we analyzed  data from fiscal year 2004 through fiscal year 2008, as these were the years  of data reported in the agency’s fiscal year 2008 Performance and  Accountability Report.
To assess the reliability of this data, we performed the following steps:   (1) reviewed the existing information about the data and the system that  produced them, (2) observed data entry and reviewed input controls,   (3) performed electronic testing of required data fields, and   (4) interviewed agency officials knowledgeable about the data and  systems. For BDN data, we also reviewed the programming logic that was  used to produce selected workload data and applied the same logic  contained in the programming against a file of raw data. We were able to  replicate two workload indicators that we chose to examine. This gave us  reasonable assurance that the automated BDN reports were reliable.
Agency officials said there are two sources of data that contain  information about VR&E participant case histories. This information is  contained in both the BDN and in Corporate WINRS. Corporate WINRS is  the interface that VR&E counselors use and that data updates BDN data in  most cases. To determine rehabilitation rates, VR&E uses three variables  indicating whether a case is rehabilitated, discontinued, and/or has  achieved a maximum rehabilitation gain (MRG). These three designations  are derived for each VR&E applicant based on Corporate WINRS case  history and then stored in a summary file. This summary data is then used  to calculate rehabilitation rates. We usually choose to examine raw data  instead of summary data. In this case, an ideal test would be to examine  the raw Corporate WINRS data and see if we came up with the same  designations evidenced in this summary level data. However, complexities  associated with the business rules used to establish the key designations  in the summary data (as rehabilitated, discontinued, and/or MRG)  prevented us from calculating the rehabilitation rate using the full case  history data. For this reason, we requested that VR&E provide us the  summary data that it used to calculate its rehabilitation rate. We then used  this summary data to verify its rehabilitation rate reports and to calculate  (1) the success rates of veterans who had a plan to achieve independent  living or had a plan to become employed and (2) how the agency would  have performed if it had not changed its rehabilitation rate calculation.
To verify the summary data, we discussed with agency officials the  algorithms they used to create the case-level summary data. In addition,  we drew a random sample of 65 summary data records and looked at the  raw case history data for each to see if the designations contained in the  summary data complied with the algorithms VR&E described. During this  examination, we found one case where the raw data did not support the  summary-level data designation. This allowed us to conclude with   95 percent confidence that these problems represent no more than a   7.1 percent rate of error in the summary data. In addition, although the  Corporate WINRS data for this case did not have the correct reason code  to support the MRG designation, an examination of BDN data (the  alternate data source that contains participant case information) did  contain the correct reason code and supported the MRG designation.  Based on our assessment, we determined that the Corporate WINRS data  used were sufficiently reliable for the purposes of this report.
section_title: VA Regional Office Site Visits
paragraphs: To assess the capacity of the regional offices, we conducted site visits to  four of VA’s regional offices—Houston, Tex.; Pittsburgh, Pa.; Seattle,  Wash.; and St. Petersburg, Fla. We also visited four satellite offices, three  that serve more rural areas, in Erie, Pa.; Spokane, Wash.; and Lewiston,  Idaho; and one serving a more metropolitan area, Tacoma, Wash. At each  of the regional offices, we interviewed the VR&E officer, assistant VR&E  officer (in regional offices that had an assistant), rehabilitation counselor  supervisors (in regional offices that had supervisors in addition to the  VR&E officer), vocational rehabilitation counselors, employment  coordinators, and local veteran service organization representatives. We  also observed the program orientation provided to new veterans applying  for VR&E services and conducted a file review of cases randomly selected  for the regional offices’ local quality assurance review. We selected our  site visit locations to ensure representation from each of VA’s four  geographic areas. We also selected our sites to ensure diversity in the  following factors: (1) proximity to major military installations, (2) number  of program participants, (3) change in the number of participants over  time, and (4) overall performance scores on various management reports.
section_title: VR&E Regional Office Survey Data
paragraphs: To gather information about the program’s workload and its current  capacity to help veterans obtain employment, we conducted a survey of all  57 VR&E regional offices from May 2, 2008, to May 15, 2008. Specifically,  we collected information on each VR&E regional office’s average  counselor caseload, number of staff and their skills, extent of contracting  or partnerships with other agencies, changes in the complexity of staff  caseloads since veterans began returning from Afghanistan and Iraq,  changes in VR&E services since the 2004 Task Force report was issued,  and VR&E’s preparation to meet future demand. We developed the content  of our survey based on key areas of concern of the 2004 Task Force and  issues raised by agency officials on our site visits. Officials at VA’s Office  of Field Operations electronically distributed the survey on our behalf;  however, all survey responses were sent directly to us. We had a response  rate of 100 percent.
Since we surveyed all regional offices, there is no sampling error.  However, difficulties in conducting any survey may introduce nonsampling  error. For example, because the data were self-reported difficulties in  interpreting a particular question or differences in the way some regional  offices are managed can introduce variability into the survey results.  Additionally, because of size differences among the regional offices, we  did not quantify or assign specific numbers to the scales used in the  survey. However, we took steps in developing the questionnaire to  minimize such nonsampling error. For example, we pretested the content  and format of our survey for understandability. We then refined our survey  as appropriate. An analyst entered the survey responses into a database  and the accuracy of this data entry was verified by an independent analyst.  Qualitative responses to open-ended questions on the survey were  categorized by an analyst to identify common themes. These themes were  then independently reviewed by another analyst for verification purposes.
section_title: Appendix II: Comments from Veterans Affairs
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Melissa Emrey-Arras, Assistant  Director; Amy Anderson, Analyst-in-Charge; Julie DeVault, Nora Boretti,  and Brooke Leary made major contributions to this report; William  Doherty, Peter DelToro, Cynthia Bascetta, Patricia Owens, Brett  Fallavollita, and Randall Williamson provided guidance; Walter Vance  assisted with design study; Cynthia Grant and Wayne Turowski conducted  data analysis; Stan Stenersen, Kate van Gelder, Susan Bernstein, Julianne  Hartman Cutts, and Brittni Milam helped write the report; Mimi Nguyen  provided assistance with graphics; and Doreen Feldman and Roger  Thomas provided legal advice.
section_title: Related GAO Products
paragraphs: Multiple Agencies Provide Assistance to Service-disabled Entrepreneurs,  but Specific Needs Are Difficult to Identify and Coordination Is Weak.  GAO-09-11R. Washington, D.C.: October 15, 2008.
Federal Disability Programs: More Strategic Coordination Could Help  Overcome Challenges to Needed Transformation. GAO-08-635.  Washington, D.C.: May 20, 2008.
Disabled Veterans’ Employment: Additional Planning, Monitoring, and  Data Collection Efforts Would Improve Assistance. GAO-07-1020.  Washington, D.C.: September 12, 2007.
Highlights of a GAO Forum: Modernizing Federal Disability Policy.  GAO-07-934SP. Washington, D.C.: August 3, 2007.
Federal Disability Assistance: Wide Array of Programs Needs to Be  Examined in Light of 21st Century Challenges. GAO-05-626. Washington,  D.C.: June 2, 2005.
Vocational Rehabilitation: VA Has Opportunities to Improve Services,  but Faces Significant Challenges. GAO-05-572T. Washington, D.C.: April  20, 2005.
Vocational Rehabilitation: More VA and DOD Collaboration Needed to  Expedite Services for Seriously Injured Servicemembers. GAO-05-167.  Washington, D.C.: January 14, 2005.
VA Vocational Rehabilitation and Employment Program: GAO  Comments on Key Task Force Findings and Recommendations.  GAO-04-853. Washington, D.C.: June 15, 2004.
Human Capital: Key Principles for Effective Strategic Workforce  Planning. GAO-04-39. Washington, D.C.: December 11, 2003.
A Model of Strategic Human Capital Management. GAO-02-373SP.  Washington, D.C.: March 15, 2002.
Vocational Rehabilitation: VA Continues to Place Few Disabled Veterans  in Jobs. GAO/HEHS-96-155. Washington, D.C.: September 3, 1996.
Vocational Rehabilitation: Better VA Management Needed to Help  Disabled Veterans Find Jobs. GAO/HRD-92-100. Washington, D.C.:  September 4, 1992.
VA Can Provide More Employment Assistance to Veterans Who Complete  Its Vocational Rehabilitation Program. GAO/HRD-84-39. Washington,  D.C.: May 23, 1984.