VA Vocational Rehabilitation and Employment: Additional Assessment Could Enhance Consistency among Counselors
Released: 2019-12-10T00:00:00

Letter

Background

VR&E Eligibility and Process

To be entitled to VR&E services and related benefits, veterans generally must (1) have at least a 20 percent service-connected disability rating from VA and (2) be in need of rehabilitation because of an employment handicap. Entitled veterans may generally receive up to 48 months of vocational rehabilitation services and up to an additional 18 months of employment services, which include counseling, and placement and postplacement services.

If a veteran is entitled to receive VR&E services and found to be employable, a counselor is to work with the veteran to identify a suitable employment goal, and to incorporate that goal and the needed services and benefits to achieve it into a vocational rehabilitation and employment plan (hereafter “employment plan”). To develop an employment plan, the counselor and veteran review labor market information for jobs within the veteran’s identified abilities, aptitudes, and interests that will not aggravate his or her service-connected disability or disabilities. After assessing obstacles to employment, they agree on a written employment plan that describes the employment goal and the services needed to achieve it. Common services provided by VR&E are funding for higher education, career counseling, and short-term employment services like job search assistance. Counselors have the authority to approve a wide variety of educational programs and may approve employment plans that have an annual cost of up to $25,000.

VR&E Organization

Within VA’s Veterans Benefits Administration (VBA), the VR&E central office is responsible for overseeing the VR&E program, including training staff and monitoring their work to ensure high performance and consistency. Among other elements, VR&E’s quality assurance efforts entail reviewing a subset of case files on a monthly basis to ensure that the entitlement decisions, development of plans, and delivery of services are performed and documented in accordance with VA regulations, VR&E’s operations manual, and other directives.

VR&E services are provided by field staff at 56 regional offices and about 300 satellite locations. The satellite locations include college campuses to help veterans successfully complete their training and find employment, as well as military sites to help servicemembers with disabilities as they begin their transition to veteran status and the civilian workplace. VR&E field positions include (1) VR&E officers who manage the program and its staff in each region; (2) vocational rehabilitation counselors who work directly with veterans to assess their entitlement, develop their employment plans, and manage their progress; and (3) staff to support the administration of the program.

As of June 2019, 1,394 field staff members were administering the VR&E program, of which nearly 75 percent (1,026) were counselors. From September 2013 to June 2019, VR&E’s total caseload peaked in fiscal year 2016 with almost 135,000 participants (see fig. 1).

Over the same period, the number of counselors changed little until 2019. In 2019, the number increased after VA hired an additional 88 counselors in response to a provision in an appropriations law suggesting that the agency aim to serve 125 veterans or fewer per full-time equivalent counselor. The increase in staffing helped reduce the average caseload of 130-141 cases per counselor during fiscal years 2013 through 2016 to 113 cases in June 2019 (see fig. 2).

Counselors Generally Considered Common Factors When Developing Veterans’ Plans but Noted Inconsistent Application of Those Factors Likely Occurs

Counselors in Our Review Generally Considered a Set of Common Factors When Developing Plans

VR&E counselors consider a set of common factors, including the veteran’s disability or disabilities, interests, and local labor market conditions, when developing and approving veterans’ employment plans. Program regulations require an assessment of some of these factors when the veteran is initially evaluated. VR&E quality review data from fiscal years 2016 through 2018 suggest that counselors generally documented certain plan considerations during the evaluation. For example, in 98 percent of the 1,080 cases VA reviewed for accuracy in fiscal year 2018, counselors documented the veteran’s service needs based on their functional limitations. In 95 percent of cases, counselors documented that they assessed the veteran’s abilities, aptitudes, and interests. Lastly, in nearly 99 percent of cases, counselors documented that the veteran was involved in vocational exploration activities such as career searches and labor market research.

During our more focused review of how counselors developed plans for a non-generalizable sample of 34 VR&E case files, we found that counselors generally documented a set of common factors. Consistent with program guidance stipulating that counselors are to consider a veteran’s service needs, abilities, aptitudes, and interests, we identified common consideration factors including one’s functional limitation from disability, prior education, aptitude results, and career interests. Table 1 presents these factors and the number of files in which the factors were documented.

Our case file review found that 30 of the 34 counselors also documented the estimated cost of VR&E employment plans. According to testimony from a veteran service organization, many VR&E participants are dissuaded by their counselor from pursuing education at a top tier university because of cost. VA’s VR&E operations manual states that if more than one local training or educational facility will meet a veteran’s needs, counselors must justify their decision to select a school that is more expensive than the least costly one. Counselors are not required to document all of the educational facilities that would serve a veteran’s needs; therefore, we could not determine the extent to which counselors chose the lowest cost facility. Counselors we interviewed in each of the three regional offices we visited said that while mindful of cost, they strive to develop employment plans that best meet the needs of the veteran. For example, counselors at one regional office described a situation in which a higher priced school was chosen because the school offered smaller class sizes that better suited the veteran’s particular mental health conditions. Of the 34 files we reviewed, the annual plan cost exceeded $25,000 in 3 cases.

Counselors we interviewed said that they considered the veteran’s career interests but weighed these interests against other factors, such as the veteran’s functional limitations and information about the local labor market. All 34 plans we reviewed aligned with the veteran’s stated career goals, though in some cases the veteran’s goals evolved after talking with the counselor about alternative occupations. In a few instances among these cases, the final plan’s career goal was notably different from the initial goal that the veteran had stated on the program intake form. Table 2 presents examples of how a plan can evolve as a result of career exploration activities and conversations between the veteran and their counselor.

Counselors in Our Review Stated They Strive to Develop Individualized Plans but Acknowledged That Some Unintended Differences in Plans for Similarly-Situated Veterans Likely Occurs

Counselors we interviewed described how veterans’ employment plans are individually designed to suit a veteran’s needs and, as a result, may differ from one another even when veterans have similar goals, characteristics, and circumstances. In some instances two veterans may appear to be similar, but may actually differ in some critical respect that results in appropriate variation across plans. A common difference among veterans is the geographical location where they are seeking employment. Counselors described how a veteran may be encouraged to explore an occupation with many job opportunities within a specific region, while a veteran with similar characteristics and interests living in a different area may be dissuaded from pursuing the same occupation for a lack of job opportunities in the area for that occupation. Local labor markets may also drive the need for a certain type of educational credential. For example, counselors said that some veterans will be competitive in certain labor markets with a bachelor’s degree, while others living in a different region with a more educated population may need a master’s degree. Likewise, they said that certain occupations, such as certified public accountants or school teachers, may require different forms of credentialing in different states.

Other characteristics of individual veterans may also cause counselors to develop different plans for veterans who appear to have similar circumstances. One counselor we interviewed described a scenario in which one veteran who received a high score on an aptitude test for reading comprehension skills might obtain a certain employment plan while another veteran who received a much lower score would be steered toward a different plan. If the veterans were to compare their final plans, but were unaware of the differences in their aptitude test scores, they could perceive inconsistent treatment. Counselors also described how conversations they have with veterans as they work to develop employment plans can reveal other character traits, such as interpersonal skills, which can lead them to suggest different plans to two otherwise similar veterans. The counselors said that such conversations play an important role towards the development of successful plans.

However, counselors we interviewed in each of the three regional offices we visited acknowledged that unintended variation likely occurs across plans developed for similarly-situated veterans. They explained that the reasons for such potential inconsistency can include (1) the prominent role professional judgment plays in the program and the potential for unintended bias, (2) counselors’ different VR&E experience levels, and (3) variations in regional offices’ policies.

Judgment and bias. The counseling role is inherently subjective and requires counselors to use their professional judgment in each case. The VR&E operations manual describes counselors’ responsibilities in broad terms, stating that counselors are to guide and assist the veteran in making an informed decision on an appropriate plan based on the veteran’s abilities, aptitudes, and interests. According to counselors we interviewed, professional judgment enables them to develop a plan that is best suited for the veteran’s unique needs, although it also introduces the potential for personal bias and inconsistent plans for veterans with similar circumstances. For example, a counselor we interviewed cited a case in which he saw the need to develop a plan that allowed for a school that was closer to a veteran’s home over other, less costly options because of his sensitivity to the veteran’s childcare responsibilities. Another counselor in the same office may not have seen the need for that accommodation. Further, counselors we interviewed said that some of their colleagues may be more comfortable suggesting that a veteran reconsider his or her career goal given circumstances such as the veteran’s disabling conditions or the local labor market. They explained that while some counselors would be hesitant to make the veteran unhappy, and possibly angry, other counselors would be more inclined to work through the conflict. Counselors said that they try to mitigate inconsistency by asking their fellow counselors to weigh in on these sorts of judgments, either informally, or at periodic information-sharing meetings.

Counselor experience. Although all counselors have at least a master’s degree in rehabilitation counseling or a related field, differences in counselors’ levels of VR&E experience may affect their approach to plan development. Counselors at two regional offices noted that the focus of the VR&E program has oscillated between education and employment, with employment being the current primary focus. They said, as a result, a counselor’s general approach to plan development could be influenced by the prevailing focus that existed at the time he or she was hired. Counselors also said that, in general, counselors with more experience will tend to approach plan development differently than a less seasoned counselor because they will apply lessons learned from serving many other veterans. For example, one counselor said that, based on years of prior experience and observation, he has developed a better understanding of which local educational programs offer veterans the best chance for success and which do not. He said while he is able to apply his institutional knowledge and experience to do what is best for veterans, a less experienced counselor may not have the same level of knowledge, which could lead to inconsistent plans for veterans with similar circumstances. According to counselors we interviewed, because of the recent hiring of new counselors to meet caseload targets, differences in VR&E experience among counselors may be more pronounced at this time.

Regional office variation. Differences in administrative policies specific to individual regional offices may also contribute to inconsistent plan development. For example, according to program officials, to ensure the soundness of employment plans in the local labor market, some regional offices require management to approve plans involving a master’s degree, while others do not. Counselors in one region told us that requiring management approval might dissuade a counselor from developing a plan focused on a master’s degree because of the time the extra step would require. They acknowledged that this sort of approval policy could cause inconsistency across counselors’ plans and also cause a discrepancy in the number of master’s degree programs being approved at one regional office versus another.

In general, the large number of variables involved in the development of employment plans may complicate the ability to determine the extent to which differences among counselors lead to inconsistent plans among veterans. Counselors we interviewed said that given the subjective nature of the program, such inconsistency is likely. However, counselors cautioned against making the plan development process overly structured and formulaic. In their view, a more restrictive approach would eliminate the flexibility that they need to generate plans that suit each veteran’s unique needs.

VA Trains Counselors and Monitors Their Performance but Does Not Monitor the Consistency of Employment Plans

VA Trains Counselors to Develop Sound Employment Plans for Veterans

VA trains counselors on developing sound and complete employment plans for veterans. New counselors receive a series of training courses that are developed and deployed through VR&E’s central office, and then receive additional courses and mentorship that are delivered through the regional offices. Course topics for new counselors include understanding vocational impairments, developing a rehabilitation plan, and documenting a narrative of the plan. The formal training emphasizes that plans should be individualized to accommodate the veteran’s rehabilitation needs, abilities, aptitudes, and interests. Collectively, these trainings take up to 80 hours. As of 2019, experienced counselors—those on the job for at least a year—take up to 20 hours of refresher training each year determined according to how they score on an annual assessment. The assessment evaluates counselors’ technical competencies such as knowledge of relevant regulations, vocational assessment and evaluation, and case management. If a counselor scores low on a particular topic, related courses are identified for the counselor to complete.

In designing training for counselors, VA followed principles for strategically developing training that are consistent with a related guide for federal managers. For instance, VA obtained and considered input from multiple sources—including field advisory committees, quality assurance reviewers, and internal site visit auditors—to identify needs for counselor training. For example, questions and input from the field about a policy clarification led to a training about veterans’ entitlement to VR&E services. In addition, VA built flexibility into its training curricula for counselors so they could receive training on emerging topics such as implementing new policies throughout the year as needed. VA also has evaluated its training efforts in multiple ways. For example, it has evaluated training courses by surveying counselors to get immediate feedback and by checking with attendees and their supervisors to gauge improvements in skills and knowledge.

VA Checks If Plans Are Complete but Does Not Monitor Consistency among Counselors

VA monitors employment plans to ensure that they are complete, but does not check for consistency among counselors for veterans with similar circumstances. Quality reviews occur nationally as well as locally at each regional office. The purpose of the national reviews is to monitor the quality of regional offices’ work such as plan development, whereas the purpose of the local reviews is to help evaluate the performance of individual counselors. Nationally, a centralized quality assurance team monitors the completeness of regional offices’ VR&E entitlement decisions, employment plans, and service delivery by reviewing a randomly-selected subset of case files from each regional office on a monthly basis. Among other criteria, reviewers check whether a veteran’s plan identified goals and objectives, included an employment focus, and incorporated the veteran’s need for various services. Locally, VR&E officers or their designees are to review plans using the same criteria. Officers are supposed to review at least three cases per counselor per quality category (e.g., accuracy of evaluation, planning and rehabilitation services) per quarter. Reviewers do not check for consistency among counselors for similarly-situated veterans, at either the national or local levels.

VR&E officials we interviewed identified challenges to completing and monitoring local reviews, but VA is addressing these challenges. According to the VR&E officer in each of the three regional offices we visited, it is difficult to complete local reviews given system limitations and their other job responsibilities such as implementing case management initiatives. They said that it is likely that some officers are not completing the required reviews while others are conducting them with varying degrees of thoroughness. Historically, VA has not identified the specific cases VR&E officers are supposed to review locally. Consequently, VA could not determine if VR&E officers conducted the requisite number of reviews or whether officers were selecting cases for quality review uniformly and fairly. In June 2019, during the course of our review, VA began a pilot in five regional offices to centrally and systematically identify the cases officers are to review to gauge individual counselors’ performance. The new process and system are intended to help officers conduct and track local reviews as well as to help VA monitor the completion of local reviews. VA plans to expand this process to all regional offices in fiscal year 2020.

Although VA trains counselors to develop complete employment plans and reviews the completeness of some plans, it does not monitor the consistency of plans among different counselors. The code of professional ethics for rehabilitation counselors calls for counselors to be fair in the treatment of all clients and to provide appropriate services to all. In addition, one of the objectives of VR&E’s central office is to provide training and guidance to ensure high performance and consistency among field staff. Several veteran service organizations have testified at congressional hearings that VR&E is marked by inconsistent treatment of similarly-situated veterans. For example, one testimony cited veterans who allegedly received different plan approvals, such as access to graduate level education, merely on the basis of their counselor. Unlike for VA staff members who work on disability claim decisions, VA does not compare the output of VR&E counselors by, for example, analyzing responses to identical hypothetical cases for training or monitoring purposes. As a result, in addition to missing a training opportunity for counselors about employment plan development, VA does not know the degree to which inconsistency among counselors occurs. For example, the agency does not know the extent to which counselors would agree to a particular veteran’s pursuit of a master’s degree through VR&E. Moreover, VA cannot respond in an informed way—and take mitigating steps if warranted—to criticisms of subjectivity in the program. VR&E officials explained that the agency has not yet conducted such a comparative analysis because of other priorities, but agreed that it could do so particularly through its training efforts.

Conclusions

VA uses several training and monitoring practices to help ensure that VR&E counselors develop employment plans that help veterans with disabilities obtain and sustain employment. In approving these plans, VR&E counselors use their judgment and discretion fostered in part by their formal education and professional experience in vocational rehabilitation. While our review of a non-generalizable sample of 34 cases found that counselors generally considered common factors in developing employment plans, counselors we interviewed nevertheless acknowledged that counselors may apply the factors differently because of their varying backgrounds and experience levels. The variability of counselors’ experiences and veterans’ circumstances may make it difficult to determine the full extent of any inconsistency. However, taking steps to examine the prevalence and type of any inconsistency among counselors who, for example, consider the same hypothetical case, would better position VA to mitigate any unfair differences in plans for veterans with similar circumstances. An understanding of how effectively and consistently counselors assist veterans will be even more important in the coming years as VA fully integrates the new counselors hired to decrease the average caseload.

Recommendation for Executive Action

The Secretary of VA should ensure that the Director of VR&E assesses the consistency of VR&E plans among counselors and takes mitigating steps if results warrant. For example, as part of its training efforts, VA could have counselors respond to identical hypothetical veteran cases and, if unfair inconsistencies in plans result, the agency could enhance training on plan development. (Recommendation 1)

Agency Comments

We provided a draft of this report to VA for comment, and its written comments are reproduced as appendix I in this report. VA concurred with our recommendation and said that VBA will develop a consistency study of VR&E plan development. It emphasized that no two veterans are the same. It also provided technical comments which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, and other interested parties. In addition, the report will be available at no charge on the GAO website at https://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7215 or curdae@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II.

Appendix I: Comments from the Department of Veterans Affairs

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Mark Glickman (Assistant Director), Joel Green (Analyst in Charge), and David Perkins made significant contribution to the report. In addition, Jennifer Cook, Holly Dye, Alex Galuten, Monica Savoy, Mimi Nguyen, Almeta Spencer, Jeff Tessin, Rosemary Torres Lerma, and Sonya Vartivarian made key contributions.