VA Vocational Rehabilitation and Employment: Better Incentives, Workforce Planning, and Performance Reporting Could Improve Program
Released: 2009-01-26T00:00:00

Letter

Background

Since the 1940s, VA has provided vocational rehabilitation assistance to veterans with service-connected disabilities. In 1980, Congress enacted the Veterans’ Rehabilitation and Education Amendments, which mandated a change in the mission of VA’s vocational rehabilitation program from primarily providing training to helping veterans find and maintain employment, or achieve independence in their daily lives if employment is not currently feasible. VA reported that VR&E served 90,600 participants in fiscal year 2007 at a cost of $722 million.

VR&E Structure and Eligibility Requirements

There are 57 VA regional offices—roughly about 1 in each state—and about 1,000 VR&E staff who work in these regional offices and at the program’s central office in Washington, D.C. VR&E regional office personnel include rehabilitation counselors, employment coordinators, and management and support staff who provide personal, face-to-face services to veterans. VR&E services can include vocational counseling, vocational evaluation, case management, education and training, job placement assistance, and independent living services. VR&E can also pay tuition, subsistence, and other expenses for veterans pursuing education and training. An allowance is provided to veterans who have completed training programs for up to 2 months as they seek employment. When necessary, VR&E can also direct veterans to other vocational and employment counselors and specialists who perform services under contract. To receive VR&E services, veterans with disabilities generally must have a 20 percent disability rating and an employment handicap. Veterans with a 10 percent disability rating may also be entitled to receive services if they have a serious employment handicap. In addition, injured servicemembers may be eligible for VR&E services before being discharged from the military if they request a memorandum rating from VA and are found to have one or more service-connected disabilities that are 20 percent or higher. VR&E vocational rehabilitation counselors determine entitlement to services, which generally provides a 12-year period of eligibility and up to 48 months of benefits.

Federal and State Programs for Veterans with Disabilities

VR&E is one of many federal and state programs available to veterans with disabilities in their transition from the military to civilian life and work. Injured servicemembers can receive medical treatment from the Department of Defense (DOD) military treatment facilities or Veterans Health Administration facilities, such as polytrauma centers, which may also provide vocational rehabilitation services. Within VA, the Compensated Work Therapy program primarily helps veterans with mental health diagnoses by integrating vocational rehabilitation into their overall medical treatment plan and placing them in jobs. In addition, VA works with DOD and the Department of Labor (Labor) to provide presentations to servicemembers being discharged about veterans’ benefits and services through the Transition Assistance Program and Disabled Transition Assistance Program. Labor’s Veterans’ Employment & Training Service (VETS) also provides services to veterans. Labor and VA have historically worked together to help veterans with service-connected disabilities transition to the civilian workforce. Labor administers the VETS program through grants to state workforce agencies, whose staff provide veterans with reemployment services, such as job search and placement assistance and also market veterans to employers. In addition, VR&E works with state vocational rehabilitation agencies that receive grants from the Rehabilitation Services Administration at the Department of Education to prepare individuals with disabilities for employment through vocational rehabilitation services.

Long-standing Critical Problems

For more than 25 years, we, along with others who have reviewed the program, veteran service organizations, and VA, have found shortcomings in the VR&E program. These reviews generally concluded that the program had not fulfilled its primary purpose, which is to ensure that veterans obtain suitable employment. In 1996, we reported that the program primarily emphasized providing training and did not place enough emphasis on providing employment services. Additionally, the 1999 Congressional Commission on Servicemembers and Veterans Transition Assistance found that VR&E had not achieved its statutory purpose and noted that “employment assistance is the most valuable service the Nation can provide to personnel transitioning from active duty to the civilian workforce.” In 2003, we designated federal disability programs, including those at VA, as high risk because they had difficulty managing their programs and were in need of transformation.

VR&E Has Strengthened Its Focus on Employment through the Five-Track Employment Process, but Has Not Updated Its Incentive Structure to Align with Its Mission

In response to recommendations from the 2004 Task Force, VR&E has implemented the Five-Track Employment Process and strengthened the program’s focus on employment. However, VR&E’s incentive structure for veterans remains primarily aligned with education and training programs, with no financial incentive for those seeking immediate employment.

VR&E Has Implemented Its Five-Track Employment Process and Strengthened Its Focus on Employment

In response to the 2004 Task Force report, VR&E implemented the Five- Track process by delineating its existing services into five distinct tracks to provide a stronger focus on employment early in the rehabilitation process. The delineation of program services into five tracks is designed to accommodate the different needs of veterans, such as those who need immediate employment as opposed to those who need training to meet their career goal. Figure 1 provides details on each of the five tracks.

After veterans apply to the program and are found eligible for services, they are introduced to the Five-Track process through a program orientation. During orientation, VR&E shows a video that explains the process to veterans and emphasizes that the goal of the program is to obtain employment or to achieve independent living if employment is not immediately feasible. At the sites we visited, we found that VR&E staff also verbally reinforced to veterans during orientation that the primary goal of the program is employment.

Following orientation and evaluation, veterans are assisted by VR&E staff in selecting a track that meets their needs and employment goals. Some of the rehabilitation counselors we interviewed told us the factors they consider when evaluating veterans for track selection include veterans’ transferable job skills, results on various vocational tests, and how the veterans’ disabilities affect their ability to do the work they did in the past. Of the almost 24,000 veterans with a documented track selection who began a plan of services from January 2007 to early May 2008, we found that more than three-quarters chose to pursue employment through the long-term services track, which includes education and training, while less than one-tenth chose more immediate employment through the reemployment or rapid access to employment tracks, and slightly more than one-tenth entered the independent living track. Very few veterans chose self-employment (less than 1 percent). See figure 2 for the percentages of veterans who entered each of the five tracks. lso as part of the Five-Track process, VR&E established an employment A coordinator position and job labs to assist veterans with preparing for andfinding employment. Employment coordinators assess veterans’ readiness to seek employment, develop relationships with employers, and help place veterans in jobs. VR&E’s job labs provide computers with employment- related software that VR&E staff and veterans can use for activities such as developing job search plans, preparing for interviews, and writing resumes.

Incentives for Veterans Remain Primarily Aligned with Training

Though the Five-Track Employment Process was intended to modernize the program and increase VR&E’s emphasis on employment, VR&E did no update its financial incentive structure to align with its mission. Specifically, the program offers a monthly subsistence allowance those veterans who are enrolled in education or training, but not to those who receive employment services only. For example, a veteran who has two dependents and enrolls in a full-time education or training program receives approximately $760 in monthly assistance. That veteran would continue to receive this allowance for 2 months following training while or she seeks employment. One rehabilitation counselor we spoke with noted that many veterans who have completed their training rely on this money during the job search phase. In contrast, veterans who receive employment services only do not receive a monthly allowance while th look for employment. Our prior work has noted the need to consider basic program design, particularly those features that affect individual work incentives and supports, when modernizing disability programs for the 21st century. Based on our prior work, we are concerned that without properly aligned incentives and supports, veterans who need assistance finding immediate employment may not seek out VR&E services and others may not choose the track that is best suited for them. In our discussions with senior VR&E officials, they acknowledged that offe ring financial incentives for veterans receiving employment services could be beneficial, and noted that they may review the internal incentive structure ey as part of a program evaluation in fiscal year 2009. Additionally, in September 2008, VA released a study on overall veterans’ compensa payments that included several options for changing the subsistence allowance for VR&E participants. to align incentives with the program’s employment mission.

VR&E Has Collaborated with Other Organization s and Added Staff, but Lacks a Strategic Approach to Workforce Planning

Over the last few years, VR&E has increased its capacity to serve veterans by engaging in a number of collaborative initiatives with other organizations and by adding staff to its central and regional offices. Nevertheless, the program continues to face challenges ensuring it h right number of staff with the right skills, and its workforce planning has not strategically addressed these issues.

on a Number of Initiatives

VR&E has increased its collaboration with other organizations such as federal and state agencies, as well as private and nonprofit employers through initiatives to help injured servicemembers and disabled veterans transition to the civilian workforce. Initiatives with DOD focus on intervention and employment services for injured servicemembers their recovery process, while VR&E’s partnership with VA’s Compensated Work Therapy (CWT) program addresses the vocational rehabilitation needs of veterans who may have mental illnesses or traumatic brain inju In addition, VR&E’s collaborative efforts with Labor, state vocational rehabilitation agencies, and employers provide employment services to veterans who are ready to enter the job market. VR&E’s recent efforts to collaborate with these organizations are highlighted below. ry.

VR&E and DOD

Economic Systems Inc., A Study of Compensation Payments for Service-Connected Disabilities, a special report prepared at the request of the Department of Veterans Affairs, September 2008.

In fiscal year 2005, VR&E created a standardized presentation for the Disabled Transition Assistance Program (DTAP), which informs disabled servicemembers of the full range of benefits and services available to them once they leave active duty. VR&E has assigned rehabilitation counselors or contractors to present this information at DOD installations and military treatment facilities. According to a senior VR&E official, VR&E is also increasing outreach to National Guard and Reserve servicemembers by providing information about this DTAP briefing at required post- deployment health assessments.

In fiscal year 2007, VA and DOD began to share information earlier about seriously injured servicemembers, and VR&E now has access to a database that allows it to identify and locate them to facilitate early outreach.

In fiscal year 2008, VR&E rolled out the Coming Home to Work (CHTW) initiative nationwide. This key component of VR&E’s early intervention efforts provides counseling to individuals on active duty pending medical separation and rehabilitation services to eligible servicemembers. According to officials, VR&E has placed 13 full-time rehabilitation counselors at 12 military treatment facilities to administer this program and initiate early contact with injured servicemembers. In addition to these 13 counselors, VR&E has designated one staff member in each regional office as the program coordinator. As of August 2008, over 4,000 servicemembers had received counseling through CHTW and 149 servicemembers who received rehabilitation services had obtained employment, according to VR&E officials.

VR&E and VA’s Compensated Work Therapy (CWT) Program

In another effort to provide services to seriously injured veterans early in their treatment process, VR&E has taken steps to develop a partnership with the CWT program at VA. The CWT program works primarily with veterans that many VR&E regional officials said their staff had difficulty serving. Such veterans might have a traumatic brain injury or mental health diagnosis, or may need more intensive support in the structured environment CWT provides. CWT’s early intervention model addresses both employment goals and medical rehabilitation needs. Also, veterans receiving services simultaneously from VR&E and the CWT program can continue to receive services from CWT even after VR&E education and training benefits are exhausted, according to officials. The 2004 Task Force noted the potential advantages of increased collaboration between VR&E and CWT. According to officials from both programs: VR&E refers veterans to the CWT program. Regional officials at the four sites we visited said their staff refer veterans to this program when it is appropriate.

VR&E and CWT briefed each other’s staff at their national training conferences in fiscal year 2008.

VR&E plans to provide a 1-hour training session for VR&E staff on the CWT program via satellite broadcast in fiscal year 2009.

VR&E and Labor

The Department of Labor is VR&E’s primary employment services partner, and an effective relationship between these agencies is important in giving disabled veterans the best chance for successful outcomes. Recent collaborative efforts include the following: In fiscal year 2006, VR&E and Labor renewed their existing agreement to improve employment services to veterans with disabilities.

In fiscal year 2006, Labor and VR&E implemented some elements of their renewed agreement by establishing a joint work group at the national level to develop a set of shared performance measures.

In fiscal year 2008, Labor and VR&E completed a demonstration project at eight regional offices to develop and test joint performance measures, tracking systems, and training curriculums for their staff who provide employment services to veterans.

VR&E and State Vocational Rehabilitation Agencies

The 2004 Task Force highlighted the importance of collaboration between VR&E and state vocational rehabilitation agencies, noting that state vocational rehabilitation agencies have established extensive employer networks and could provide veterans with greater access to employment opportunities. In addition to these increased employment opportunities, agency officials also noted that close relationships between VR&E and these agencies could result in joint rehabilitation plans that can provide complementary services to veterans. For example, veterans who are jointly served by VR&E and a state vocational rehabilitation agency have access to more and different services, such as transportation assistance or a clothing allowance provided by state agencies, which may make the difference in a veteran’s ability to achieve rehabilitation and employment goals. According to officials, recent collaborative efforts with state vocational rehabilitation agencies have included the following: In fiscal year 2004, VR&E and the Council of State Administrators of Vocational Rehabilitation (CSAVR), a professional association of state vocational rehabilitation administrators, formally agreed to facilitate local cooperative agreements between state vocational rehabilitation agencies and VR&E regional offices. The purpose of these local agreements is to encourage collaboration that will result in improved services and increased employment outcomes for disabled veterans.

In fiscal year 2008, the central office staff of VR&E and CSAVR exchanged local office contact information.

In fiscal year 2008, VR&E and state vocational rehabilitation officials briefed each other’s staff at national conferences.

VR&E and Employers

VR&E has established national agreements with several private, public, and nonprofit employers to further increase employment opportunities for veterans. These agreements focus on joint efforts to provide career opportunities to veterans exiting the VR&E program. VR&E central office officials said that they inform the regional offices of new national agreements via monthly conference calls and disseminate copies of the agreements.

Finally, a senior VR&E official said that the program currently coordinates individually, as opposed to jointly, with its various partners—DOD, VA’s CWT program, Labor, and state vocational rehabilitation agencies. This official also noted that VR&E had recently contributed to a forthcoming report on strategies for building capacity and tools for improving coordination among federal and state agencies, including several listed above. The report is expected to identify promising practices for addressing gaps in services.

VR&E Has Added Staff, but Strategic Workforce Planning Has Been Insufficient

VR&E has increased staffing at its central and regional offices as recommended by the 2004 Task Force. Specifically, VR&E officials said they increased central office staff by 67 percent, from 33 in fiscal year 2004 to 55 in fiscal year 2008, to address the concern that the central office needed more resources to provide policy, procedures, and staff training to the regional offices. At the four sites we visited, some regional office staff said support and training from the central office had improved. VR&E also increased its regional office staff by 20 percent, from 917 in fiscal year 2004 to 1,101 in fiscal year 2008. A senior VR&E official said these new regional office staff include contracting specialists and counselors, as well as positions to provide outreach to veterans returning from the wars in Afghanistan and Iraq.

Despite these staff increases, the VR&E regional offices still reported staff and skill shortages on our survey. In terms of staff shortages, more than half of all 57 regional offices said they have fewer counselors than they need and more than a third said they have fewer employment coordinators than they need (see fig. 3). Some employment coordinators we interviewed told us it is difficult for them to provide services to veterans and reach employers throughout their entire regions, including those in more rural locations. Exacerbating these staff shortages is the fact that staff time may not be used efficiently as many regional office staff we interviewed and surveyed said much of their time was spent on redundant paperwork and data entry requirements that reduced the amount of time they spent with veterans. In terms of skill shortages, almost one-third of the regional offices reported that the skills of their counselors no more than moderately meet the needs of the veterans they serve and almost one- third reported the same for their employment coordinators. Moreover, 80 percent of offices said VR&E was somewhat or less prepared to meet the needs of veterans in the future, and, of these, 12 percent reported VR&E was unprepared.

We found that these workforce problems were not being addressed with some of the strategic planning practices that our prior work has identified as essential, such as: Using data to identify current and future human capital needs including the appropriate number of employees, how they are deployed across the organization, and existing opportunities to reshape the workforce by improving current work processes; and Determining the critical skills and competencies staff will need to successfully achieve the organization’s mission and goals, especially as various factors change the environment in which the organization operates.

VR&E has not gathered data to identify the number of staff it currently needs. The 2004 Task Force recommended a study of the time required for key tasks and VR&E identified the need for such a study in its fiscal year 2005 - 2008 workforce plan; however, the study has not yet been conducted. While VR&E officials told us they have plans to fund the study in fiscal year 2009, they acknowledged that without such information they do not know whether their current caseload target is appropriate. Moreover, without knowing what their target caseload should be, VR&E cannot know the total number of counselors the program needs. VR&E officials said the current caseload target, which is one counselor for every 125 veterans, is based on a study of the state vocational rehabilitation programs, not VR&E’s own workloads. Nevertheless, the state study concluded that a caseload of this size would leave counselors little time to spend with clients. We learned from our survey of VR&E regional offices that their estimated average caseload was one counselor for every 136 veterans.

In addition, the program has not studied its work processes since the roll- out of the Five-Track process to determine whether and how to streamline administrative activities to allow staff to use their time more efficiently. Many survey respondents, as well as staff we interviewed, reported that administrative paperwork was cumbersome and labor intensive. According to staff at one regional office, paperwork requirements were a concern when the Five-Track process was rolled out, but documentation requirements did not ultimately change and new paperwork was added. At another regional office, a staff member noted that the decision regarding a veteran’s entitlement to services had to be documented multiple times. A VR&E central office official said the program is working to transition to one database, which will reduce redundant data entry. Additionally, the official said that while new forms had been added to ensure consistent documentation across all regional offices, these requirements will be reviewed as part of the fiscal year 2009 study of counselors’ key tasks.

VR&E also does not use relevant data to identify future staffing needs. While a VR&E official said that the program considers potential factors such as the impact of the wars in Afghanistan and Iraq, the only data used to project future workloads and staff needs are the program’s historical participation rates. Moreover, while VR&E does review the numbers of new disability claims, this official said these numbers are not formally factored into its projections nor does the program routinely determine what proportion of this population subsequently applies for VR&E services or when they apply. We found a decrease in the average number of years between a veteran receiving an initial disability rating and applying for VR&E services from 7.9 years in fiscal year 2002 to 6.1 years in fiscal year 2007. A VR&E official said this decrease is expected due to the program’s increased outreach to servicemembers and veterans. VR&E officials said that their past workload projections had not been far off and, according to VR&E data, since 2004 their projections have been within 8 percent of actual program participation. However, new factors may be impacting enrollment because in fiscal year 2008 the program underestimated the number of program participants for the first time in several years.

Further, VR&E staffing projections do not account for the numbers of veterans whose status will likely require more staff time, such as veterans who need an extended evaluation to determine if employment is currently feasible. Staff are allocated to the regional offices based, in part, on the number of veterans whose status will likely take more of a counselor’s time. However, when VR&E prepares its annual budget request for staff, it considers only total program participants and does not take into consideration the growing number of cases that require more staff time due to their complexity. Yet, since the wars began in Afghanistan and Iraq, the number of veterans who required an extended evaluation increased by 121 percent. While a senior VR&E official said the model for projecting the program’s overall staff needs is not intended to be the same as the one for allocating staff to regional offices, a senior VA official acknowledged that VR&E could improve its workload management with better projections.

In addition, VR&E officials said they have not fully determined the critical skills and competencies needed by counselors and employment coordinators to achieve the program’s goals. While officials in 2003 conducted an analysis of job duties and associated tasks for counselors, this was not an analysis of the skills and competencies required to perform those tasks or the skills that might be needed in the future.

Determining the relevant skills and competencies that counselors and employment coordinators need may be particularly important now, given the changing needs of veterans. About 90 percent of the regional offices we surveyed reported that their caseloads have become more complex since veterans began returning from Afghanistan and Iraq. They reported dealing with multiple physical injuries as well as traumatic brain injury and post-traumatic stress disorder among veterans returning from war. One official noted that, while her staff are skilled, they are not experts in traumatic injuries and psychiatric conditions, and could benefit from additional training in these areas.

Performance and Budget Reports Lack Important Information about VR&E Program Outcomes

VA performance and budget reports lack important information about the outcomes of the VR&E program. VA does not report specific performance information for the two different groups of veterans VR&E serves—those seeking employment and those seeking to live independently. In addition, it has not adequately disclosed a change to its primary performance measure. These omissions could lead to some misinterpretation of the program’s performance.

Reports Do Not Show Separate Success Rates for Veterans Seeking Employment and Independent Living

Although the VR&E program works with two different groups of veterans, most of whom are focused on employment with a smaller number seeking independent living, VA reports an overall rehabilitation rate for all participants. We found that this single measure masks the individual outcome for each group of participants and may hinder oversight. For example, VA reported a rehabilitation rate of 76 percent in fiscal year 2008. When we computed the rates for each group of veterans we found that 73 percent of those seeking employment were successful, while 92 percent seeking independent living were successful (see fig. 4).

Information on separate success rates would result in better information for Congress and others to evaluate program performance and target services. For example, reporting separate rates would show that those participants seeking employment—the majority of people in the program—have a lower success rate than the overall rate currently reported. Likewise, information on separate success rates would enable those overseeing the program to understand that the minority of participants seeking independent living have a much higher success rate than the reported overall rate.

Both the Task Force and VA’s Office of Inspector General (OIG) have also noted the need for separate employment and independent living measures. The Task Force recommended the use of separate outcome measures because very different services are often required to serve those seeking employment versus those seeking independent living. For example, veterans seeking employment may need career training and placement, while veterans trying to live independently may need to learn to use a wheelchair or communicate with an assistive device. VR&E officials did not implement the recommendation because, according to officials, the existing rehabilitation rate reflected the outcomes of all veterans in the program. For its part, VA’s OIG specifically recommended in 2007 that VA performance and accountability reports include the numbers of veterans who achieve employment and independent living, given that such outcomes are used for budget and resource allocation and in testimony to Congress. In its 2007 Performance and Accountability Report, VA provided the absolute numbers of veterans who had found employment (8,252) or achieved independent living (2,756), but did not offer a separate rate for each program goal, which would have allowed for a better assessment of VR&E’s progress. During our review, a senior VR&E official acknowledged the merit of examining separate employment and independent living rates and said that the program had recently begun internally tracking separate rates. Another VR&E official told us that the program is considering developing and reporting separate performance measures for independent living and employment, but did not have a specific time frame for when that decision will be finalized.

VA Altered Its Performance Measure without Adequately Disclosing the Change

In fiscal year 2006, VR&E changed its rehabilitation performance measure—the way it calculates the overall rehabilitation rate—without adequately disclosing this change in several subsequent reports even though the change substantially increased the rate. VA noted the change in its fiscal year 2006 Performance and Accountability Report, but did not do so for its subsequent fiscal year 2007 and fiscal year 2008 Performance and Accountability Reports, or for its fiscal year 2008 and fiscal year 2009 budget submissions to Congress. These reports included tables and graphics showing a 10-point increase in the rehabilitation rate from fiscal year 2005 to fiscal year 2006. While federal agencies may change their performance measures, we believe that not acknowledging the change in subsequent reports could allow for some misinterpretation of the program’s performance over time. Our prior work on federal performance measures found it useful to acknowledge such a change to provide a complete picture of program performance.

Prior to fiscal year 2006, VR&E calculated the rehabilitation rate by comparing the number of veterans who had a rehabilitation plan and achieved their goal with the total number of veterans who had a rehabilitation plan and either achieved their goal or discontinued the program. In fiscal year 2006, VR&E began excluding from the total those veterans who discontinued from the program for reasons considered beyond VR&E’s control (see fig. 5). Specifically, VR&E excludes veterans from the calculation who accept a position incompatible with their disability; those they consider employable, but who are no longer seeking employment; and those they consider unemployable due to medical or psychological reasons.

Prior to the calculation change, VR&E was having limited success improving its rehabilitation rate and achieving its performance goals (see fig. 6). Changing the calculation enabled VR&E to show a 14-point increase (from 62 percent in fiscal year 2004 to 76 percent in fiscal year 2008) in the rehabilitation rate trend in its fiscal year 2008 Performance and Accountability Report. According to our analysis, the increase would have been 6 points (from 62 percent in fiscal year 2004 to 68 percent in fiscal year 2008) without a change to the performance measure. Furthermore, the calculation change enabled VR&E to meet its annual performance goal in fiscal years 2006, 2007, and 2008. We are concerned that this performance data, as currently reported without an explanation of the calculation change, could convey a misleading picture of the program’s performance over time.

Conclusions

For more than 20 years, VR&E has sought to modernize its program and meet its employment mandate. VR&E launched its new Five-Track Employment Process to better focus on employment; however, critical aspects of the program have not been aligned with the employment mission. Given the current incentive structure, veterans who most need immediate employment services, but could also benefit from some level of financial assistance, may be at a disadvantage. Moreover, the incentive structure may result in some veterans not choosing the track that is best for them and, therefore, foregoing early integration into the civilian workforce.

VR&E has improved its capacity to serve veterans by stepping up its collaboration with other organizations and by adding staff. However, the lack of information about staffing needs could limit VR&E’s ability to provide quality services to veterans returning from the wars in Afghanistan and Iraq, as well as to veterans from prior conflicts. Without a strategic workforce planning process that collects and uses relevant data to ensure the right number of staff with the appropriate skills, the VR&E program will continue to face challenges serving current veterans and could fall short in responding to the needs of future veterans.

Finally, the lack of transparency in how VA calculates and reports program performance is detrimental to effective oversight and VR&E’s ability to manage the program. Without transparency in program outcomes and how performance measures are calculated, Congress and other stakeholders lack important information that highlights the program’s successes and focuses their attention on its shortcomings. In addition, VA officials lack essential information to manage and make adjustments to the program.

Recommendations for Executive Action

To ensure VR&E’s employment mission is fully supported, we recommend that the Secretary of Veterans Affairs direct VR&E to consider cost- effective options for better aligning the program’s financial incentives with its employment mission.

To ensure that the current and future needs of veterans are met, we recommend that the Secretary of Veterans Affairs direct VR&E to engage in a strategic workforce planning process that collects and uses relevant data, such as information on the appropriate counselor caseload and the critical skills and competencies needed by staff.

To increase transparency in VR&E performance and budget reports, we recommend that the Secretary of Veterans Affairs take actions such as separately reporting both the annual percentage of veterans who obtain employment and the percentage of those who achieve independent living, and fully disclosing changes in performance measure calculations when reporting trend data in key performance and budget reports.

Agency Comments and Our Evaluation

We provided a draft of this report to VA for review and comment. The agency provided written comments, which are reproduced in appendix II. VA generally agreed with our recommendations and noted the steps it will take to act on them: In response to our recommendation that VR&E consider cost-effective options for better aligning the program’s financial incentives with its employment mission, VA agreed and stated that the current law does not permit payments of subsistence allowance to veterans receiving only employment services. Therefore, to address this issue, VR&E has drafted a legislative proposal for consideration by the Secretary of Veterans Affairs.

In response to our recommendation that VR&E engage in a strategic workforce planning process that collects and uses relevant data, such as information on the appropriate counselor caseload and the critical skills and competencies needed by staff, VA agreed and outlined its plans to implement the recommendation. With regard to collecting and using information on the appropriate counselor caseload, VA stated that it plans to complete a study by the end of fiscal year 2010 that will help it determine the staffing levels necessary to comprehensively meet veterans’ rehabilitation needs. With regard to collecting and using information on the critical skills and competencies needed by staff, VA noted that it has already defined the critical skills and competencies needed for VR&E counselors by requiring them to hold a master’s degree in rehabilitation and has provided training to VR&E staff. While we acknowledge the value of these efforts, the fact that many regional offices reported skill shortages on our survey indicates that more needs to be done in this area, especially given the increasingly complex needs of the veterans now applying for services. VA did agree to conduct a skills assessment survey of VR&E staff and indicated that the survey will determine the skills staff currently possess as well as the skills staff need to successfully serve veterans. Additionally, VA agreed to ensure staff training is targeted to the specific skills and competencies identified on the survey.

In response to our recommendation that VA separately report the annual percentage of veterans who obtain employment and the percentage of those who achieve independent living and fully disclose changes in performance measures, VA agreed and stated that it will include employment and independent living rates in the comments of its fiscal year 2010 budget and fiscal year 2009 Performance and Accountability Report and will implement separate performance measures in fiscal year 2010. Additionally, VA stated that it would note the year the rehabilitation rate calculation changed in future budget and performance and accountability documents.

We are sending copies of this report to the Secretary of Veterans Affairs, relevant congressional committees, and other interested parties. The report will also be available at no charge on the GAO Web site at http://www.gao.gov.

A list of related GAO products is included at the end of this report. If you or your staff have any questions about this report, please contact me at (202) 512-7215 or bertonid@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Our review examined (1) how the implementation of the Five-Track Employment Process has affected the Vocational Rehabilitation and Employment (VR&E) program’s focus on employment, (2) the extent to which VR&E has taken steps to improve its capacity, and (3) how program outcomes are reported. To address these objectives, we: reviewed agency documents and relevant recommendations from key reports, such as the 2004 VR&E Task Force; analyzed data from the Department of Veterans Affairs’ (VA) Corporate WINRS and Benefits Delivery Network (BDN) data systems; interviewed VA and VR&E staff knowledgeable about VR&E planning and operations, and others such as disability experts, members of the 2004 Task Force, veteran service organization representatives, and staff from agencies and organizations that collaborate with VR&E; visited four VA regional offices and conducted interviews with VR&E officers and staff to observe and gather information on workforce planning and how services are provided to veterans; and conducted a survey of VR&E officers at all 57 regional offices to follow up on several key issues relevant to our research objectives.

We conducted this performance audit from July 2007 to January 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Review of Key Documentation

To evaluate our objectives, we reviewed agency documentation and prior evaluations of the VR&E program and recommendations made by the 2004 VR&E Task Force, key commissions, the VA Office of Inspector General, as well as our own previous work. To evaluate how the Five-Track Employment Process has affected VR&E’s focus on employment and the extent to which VR&E has taken steps to improve its capacity, we identified key recommendations from the 2004 Task Force report by reviewing and selecting recommendations related to the following areas: program focus on employment; workforce and workload management; collaboration with outside agencies and organizations; and performance measures. We assessed VR&E’s implementation of completed recommendations and reviewed recommendations it had not yet completed. We also referred to our previous work on strategic workforce planning and the Office of Personnel Management’s Human Capital Assessment and Accountability Framework. To evaluate reports on VR&E’s program outcomes, we reviewed recent agency performance data in VA’s fiscal year 2006, 2007, and 2008 annual performance and accountability reports and congressional budget submissions for fiscal years 2008 and 2009.

Review of VA Data on Veterans’ Case Statuses, Employment Track Usage, and Program Performance

We used data from VA’s Corporate WINRS case management system and its BDN system to evaluate the number of veterans in each case status, the number of veteran’s enrolled in each of the five tracks, the amount of time between veterans receiving an initial disability rating and applying for VR&E services, and VR&E program outcomes reporting. To evaluate the number of veterans in each case status over time, we used BDN fiscal year-end national reports from fiscal year 2001 to fiscal year 2007 to capture changes since the beginning of the conflicts in Afghanistan and Iraq. We also analyzed BDN and Corporate WINRS data to determine the change in the average length of time between a veteran receiving an initial disability rating and applying for VR&E services from fiscal year 2002 to fiscal year 2007. We began our analysis with fiscal year 2002 because an agency official told us that regional office data were uploaded into the Corporate WINRS database in fiscal year 2001 making data prior to fiscal year 2002 less reliable. For performance outcomes reporting, we analyzed data from fiscal year 2004 through fiscal year 2008, as these were the years of data reported in the agency’s fiscal year 2008 Performance and Accountability Report.

To assess the reliability of this data, we performed the following steps: (1) reviewed the existing information about the data and the system that produced them, (2) observed data entry and reviewed input controls, (3) performed electronic testing of required data fields, and (4) interviewed agency officials knowledgeable about the data and systems. For BDN data, we also reviewed the programming logic that was used to produce selected workload data and applied the same logic contained in the programming against a file of raw data. We were able to replicate two workload indicators that we chose to examine. This gave us reasonable assurance that the automated BDN reports were reliable.

Agency officials said there are two sources of data that contain information about VR&E participant case histories. This information is contained in both the BDN and in Corporate WINRS. Corporate WINRS is the interface that VR&E counselors use and that data updates BDN data in most cases. To determine rehabilitation rates, VR&E uses three variables indicating whether a case is rehabilitated, discontinued, and/or has achieved a maximum rehabilitation gain (MRG). These three designations are derived for each VR&E applicant based on Corporate WINRS case history and then stored in a summary file. This summary data is then used to calculate rehabilitation rates. We usually choose to examine raw data instead of summary data. In this case, an ideal test would be to examine the raw Corporate WINRS data and see if we came up with the same designations evidenced in this summary level data. However, complexities associated with the business rules used to establish the key designations in the summary data (as rehabilitated, discontinued, and/or MRG) prevented us from calculating the rehabilitation rate using the full case history data. For this reason, we requested that VR&E provide us the summary data that it used to calculate its rehabilitation rate. We then used this summary data to verify its rehabilitation rate reports and to calculate (1) the success rates of veterans who had a plan to achieve independent living or had a plan to become employed and (2) how the agency would have performed if it had not changed its rehabilitation rate calculation.

To verify the summary data, we discussed with agency officials the algorithms they used to create the case-level summary data. In addition, we drew a random sample of 65 summary data records and looked at the raw case history data for each to see if the designations contained in the summary data complied with the algorithms VR&E described. During this examination, we found one case where the raw data did not support the summary-level data designation. This allowed us to conclude with 95 percent confidence that these problems represent no more than a 7.1 percent rate of error in the summary data. In addition, although the Corporate WINRS data for this case did not have the correct reason code to support the MRG designation, an examination of BDN data (the alternate data source that contains participant case information) did contain the correct reason code and supported the MRG designation. Based on our assessment, we determined that the Corporate WINRS data used were sufficiently reliable for the purposes of this report.

VA Regional Office Site Visits

To assess the capacity of the regional offices, we conducted site visits to four of VA’s regional offices—Houston, Tex.; Pittsburgh, Pa.; Seattle, Wash.; and St. Petersburg, Fla. We also visited four satellite offices, three that serve more rural areas, in Erie, Pa.; Spokane, Wash.; and Lewiston, Idaho; and one serving a more metropolitan area, Tacoma, Wash. At each of the regional offices, we interviewed the VR&E officer, assistant VR&E officer (in regional offices that had an assistant), rehabilitation counselor supervisors (in regional offices that had supervisors in addition to the VR&E officer), vocational rehabilitation counselors, employment coordinators, and local veteran service organization representatives. We also observed the program orientation provided to new veterans applying for VR&E services and conducted a file review of cases randomly selected for the regional offices’ local quality assurance review. We selected our site visit locations to ensure representation from each of VA’s four geographic areas. We also selected our sites to ensure diversity in the following factors: (1) proximity to major military installations, (2) number of program participants, (3) change in the number of participants over time, and (4) overall performance scores on various management reports.

VR&E Regional Office Survey Data

To gather information about the program’s workload and its current capacity to help veterans obtain employment, we conducted a survey of all 57 VR&E regional offices from May 2, 2008, to May 15, 2008. Specifically, we collected information on each VR&E regional office’s average counselor caseload, number of staff and their skills, extent of contracting or partnerships with other agencies, changes in the complexity of staff caseloads since veterans began returning from Afghanistan and Iraq, changes in VR&E services since the 2004 Task Force report was issued, and VR&E’s preparation to meet future demand. We developed the content of our survey based on key areas of concern of the 2004 Task Force and issues raised by agency officials on our site visits. Officials at VA’s Office of Field Operations electronically distributed the survey on our behalf; however, all survey responses were sent directly to us. We had a response rate of 100 percent.

Since we surveyed all regional offices, there is no sampling error. However, difficulties in conducting any survey may introduce nonsampling error. For example, because the data were self-reported difficulties in interpreting a particular question or differences in the way some regional offices are managed can introduce variability into the survey results. Additionally, because of size differences among the regional offices, we did not quantify or assign specific numbers to the scales used in the survey. However, we took steps in developing the questionnaire to minimize such nonsampling error. For example, we pretested the content and format of our survey for understandability. We then refined our survey as appropriate. An analyst entered the survey responses into a database and the accuracy of this data entry was verified by an independent analyst. Qualitative responses to open-ended questions on the survey were categorized by an analyst to identify common themes. These themes were then independently reviewed by another analyst for verification purposes.

Appendix II: Comments from Veterans Affairs

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Melissa Emrey-Arras, Assistant Director; Amy Anderson, Analyst-in-Charge; Julie DeVault, Nora Boretti, and Brooke Leary made major contributions to this report; William Doherty, Peter DelToro, Cynthia Bascetta, Patricia Owens, Brett Fallavollita, and Randall Williamson provided guidance; Walter Vance assisted with design study; Cynthia Grant and Wayne Turowski conducted data analysis; Stan Stenersen, Kate van Gelder, Susan Bernstein, Julianne Hartman Cutts, and Brittni Milam helped write the report; Mimi Nguyen provided assistance with graphics; and Doreen Feldman and Roger Thomas provided legal advice.

Related GAO Products

Multiple Agencies Provide Assistance to Service-disabled Entrepreneurs, but Specific Needs Are Difficult to Identify and Coordination Is Weak. GAO-09-11R. Washington, D.C.: October 15, 2008.

Federal Disability Programs: More Strategic Coordination Could Help Overcome Challenges to Needed Transformation. GAO-08-635. Washington, D.C.: May 20, 2008.

Disabled Veterans’ Employment: Additional Planning, Monitoring, and Data Collection Efforts Would Improve Assistance. GAO-07-1020. Washington, D.C.: September 12, 2007.

Highlights of a GAO Forum: Modernizing Federal Disability Policy. GAO-07-934SP. Washington, D.C.: August 3, 2007.

Federal Disability Assistance: Wide Array of Programs Needs to Be Examined in Light of 21st Century Challenges. GAO-05-626. Washington, D.C.: June 2, 2005.

Vocational Rehabilitation: VA Has Opportunities to Improve Services, but Faces Significant Challenges. GAO-05-572T. Washington, D.C.: April 20, 2005.

Vocational Rehabilitation: More VA and DOD Collaboration Needed to Expedite Services for Seriously Injured Servicemembers. GAO-05-167. Washington, D.C.: January 14, 2005.

VA Vocational Rehabilitation and Employment Program: GAO Comments on Key Task Force Findings and Recommendations. GAO-04-853. Washington, D.C.: June 15, 2004.

Human Capital: Key Principles for Effective Strategic Workforce Planning. GAO-04-39. Washington, D.C.: December 11, 2003.

A Model of Strategic Human Capital Management. GAO-02-373SP. Washington, D.C.: March 15, 2002.

Vocational Rehabilitation: VA Continues to Place Few Disabled Veterans in Jobs. GAO/HEHS-96-155. Washington, D.C.: September 3, 1996.

Vocational Rehabilitation: Better VA Management Needed to Help Disabled Veterans Find Jobs. GAO/HRD-92-100. Washington, D.C.: September 4, 1992.

VA Can Provide More Employment Assistance to Veterans Who Complete Its Vocational Rehabilitation Program. GAO/HRD-84-39. Washington, D.C.: May 23, 1984.