VA Vocational Rehabilitation and Employment Program: Improved Oversight of Independent Living Services and Supports Is Needed
Published: 2013-06-07T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The IL "track"--one of five tracks within VA's VR&E program--provides a range of benefits to help veterans with service-connected disabilities live independently when employment is not considered feasible at the time they enter the VR&E program. These benefits can include counseling, assistive devices, and other services or equipment. GAO was asked to review issues related to the IL track. This report examines (1) the characteristics of veterans in the IL track, and the types and costs of benefits they were provided; (2) the extent to which their IL plans were completed, and the time it took to complete them; and (3) the extent to which the IL track has been administered appropriately and consistently across regional offices.
To conduct this work, GAO analyzed VA administrative data from fiscal years 2008 to 2011, and reviewed a random, generalizable sample of 182 veterans who entered the IL track in fiscal year 2008. In addition, GAO visited five VA regional offices; interviewed agency officials and staff; and reviewed relevant federal laws, regulations, and agency policies and procedures.
section_title: What GAO Found
paragraphs: Of the 9,215 veterans who entered the Department of Veterans Affairs' (VA) Independent Living (IL) track within the Vocational Rehabilitation and Employment (VR&E) program in fiscal years 2008 to 2011, most were male Vietnam era veterans in their 50s or 60s. Almost 60 percent served in the U.S. Army, and fewer than 1 percent served in the National Guard or Reserve. The most prevalent disabilities among these veterans were post-traumatic stress disorder and tinnitus. GAO's review of 182 IL cases from fiscal year 2008 found that VR&E provided a range of IL benefits to veterans. Among these cases, the most common benefits were counseling services and computers. Less common benefits included gym memberships, camping equipment, and a boat. GAO estimated that VR&E spent nearly $14 million on benefits for veterans entering the IL track in fiscal year 2008--an average of almost $6,000 per IL veteran.
Most veterans completed their IL plans, which identify their individual goals to live independently and the benefits VR&E will provide. About 89 percent of fiscal year 2008 IL veterans were considered by VR&E to be "rehabilitated," that is, generally, to have completed their IL plans by the end of fiscal year 2011. VR&E discontinued or closed about 5 percent of cases for various reasons, such as the veteran declined benefits. Six percent of cases were open at the end of fiscal year 2011. Because the complexity of IL cases varied depending on veterans' disabilities and needs, some cases were fairly simple for VR&E to close. For example, one IL case only called for the installation of door levers and a bathtub rail. Another more complex case involved the provision of a range of IL benefits, including home modifications. Rehabilitation rates across regions varied from 0 to 100 percent, and regions with larger IL caseloads generally rehabilitated a greater percentage of IL veterans. While IL plans nationwide were completed in 384 days, on average, completion times varied by region, from 150 to 895 days.
VR&E exercises limited oversight to ensure appropriate and consistent administration of the IL track across its regions. First, some regions may not be complying with certain case management requirements. For instance, while VR&E is required to coordinate with the Veterans Health Administration (VHA) on IL benefits, VR&E counselors have difficulty obtaining timely responses from VHA. VHA physicians respond to VR&E's IL referrals late or not at all, resulting in delayed benefits or VR&E providing the benefits instead of VHA. Second, VR&E does not monitor regional variation in IL caseloads and benefits provided. Instead, it has relied on its quality assurance reviews and ad hoc studies, but these are limited in scope. Third, given counselors have broad discretion in selecting IL benefits, VR&E's written policies for approving IL expenditures may not be appropriate as regions were permitted to purchase a range of items without any Central Office approval, some of which were costly or questionable. For example, in one case, Central Office review would not have been required for expenditures of $18,829 for a riding mower, which is prohibited, and other items. Finally, VR&E's system does not collect IL costs and benefits provided. VR&E also lacks accurate data on the number of IL veterans served. While the law currently allows 2,700 veterans to enter the IL track annually, data used to monitor the cap are based on the number of IL plans developed, not on the number of individual veterans admitted. Veterans can have more than one plan in a fiscal year, so one veteran could be counted multiple times towards the cap.
section_title: What GAO Recommends
paragraphs: GAO recommends that VR&E explore options to enhance coordination with VHA; strengthen its oversight of the IL track; and reassess its policy for approving benefits. VA generally agreed with GAO's conclusions and recommendations.

section_title: Letter
paragraphs: The Department of Veterans Affairs’ (VA) Vocational Rehabilitation and  Employment (VR&E) program, under the Veterans Benefits  Administration (VBA), administers five “tracks” that provide a wide range  of services and assistance to support veterans with service-connected  disabilities. Four of VR&E’s five tracks are intended to help veterans  prepare for, find, and maintain suitable employment. If employment is not  considered a feasible goal for veterans when they enter the VR&E  program, they can be assigned to the Independent Living (IL) track for  separate, non-employment support within the VR&E program, and can  receive benefits such as counseling, assistive devices, and other services  or equipment to improve their ability to live independently. By law, the  number of veterans who can be assigned to the IL track each year is  limited. There are currently 56 VA regional offices with responsibility for  administering VR&E’s IL track, and almost 3,000 veterans nationwide  were served in this track during fiscal year 2011.
Given the limited information available on those who participate in  VR&E’s IL track, you asked us to review issues related to the benefits IL  track veterans receive, and their outcomes. This report examines (1) the  characteristics of veterans who have participated in the IL track, as well  as the types and costs of benefits they were provided; (2) the extent to  which their IL plans were completed, and the time it took to complete  them; and (3) the extent to which the IL track has been administered  appropriately and consistently across regional offices.
To provide information on the number, characteristics, and outcomes of  veterans who entered the IL track, as well as how long it took VR&E to  complete these cases, we analyzed VA administrative data from fiscal  years 2008 through 2011. We also used these data to estimate the  difference, if any, in the amount of time it took to complete these cases  across regional offices, as well as across groups of veterans with various  characteristics. To determine the types of goods and services provided to  veterans and the direct cost of these benefits, we reviewed the case files  of a random sample of 182 veterans who were assigned to the IL track at  some point during fiscal year 2008.
To obtain information about how the IL track has been administered, we  visited VA regional offices in San Diego, California; Atlanta, Georgia;  Detroit, Michigan; Philadelphia, Pennsylvania; and the District of  Columbia. We selected these sites because they are geographically  diverse and have had differing levels of IL caseloads and rehabilitation  rates. At each of these sites we interviewed officials from VR&E, VA’s  Specially Adapted Housing (SAH) program, and Prosthetic and Sensory  Aids Service (PSAS), as well as representatives from local veteran  service organizations. We also interviewed VR&E officials at the national  level, and relied on our case file review to determine the extent to which  VR&E, SAH, and PSAS coordinate in the provision of goods and services  to IL track veterans. Lastly, we reviewed relevant federal laws and  regulations, standards for internal controls and managerial cost  accounting, as well as VR&E policies, procedures, and other relevant  studies and documentation.
We conducted this performance audit from February 2012 to June 2013  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Additional information on our  scope and methodology is provided in appendices I and II.
subsections: section_title: Background
subsections: section_title: VR&E’s IL Track
subsections: section_title: VR&E’s Five Tracks At A Glance
paragraphs: veterans with existing skills who are looking  for immediate employment.
Self-Employment: For veterans who have  the skills to start a business, and for whom  self-employment is the best or only option  for successful rehabilitation. fiscal year 2011 and is currently 2,700. VR&E anticipates an increase in  future demand for IL benefits.
section_title: VR&E’s Eligibility Requirements
paragraphs: All veterans are required to meet VR&E’s basic entitlement criteria before  they are assigned to one of its five tracks. To be entitled to VR&E  services, veterans must generally have at least a 20 percent service- connected disability rating and be in need of rehabilitation because of an  employment handicap. The period of eligibility for which veterans can  receive benefits from VR&E is 12 years and begins on either (1) the date  of separation from active military duty, or (2) the date that VA notifies the  veteran of his or her service-connected disability rating. The 12-year  eligibility period may be deferred or extended if certain conditions prevent  the veteran from participating in a rehabilitation program or if a veteran  has a serious employment handicap. To be considered for the IL track,  veterans must meet all of VR&E’s eligibility requirements and be deemed  currently unable to pursue an employment goal.
section_title: VR&E’s Structure and Administration
paragraphs: VR&E’s Central Office, located in Washington D.C., has responsibility for  several areas, including (1) overseeing the implementation of the VR&E  program, including the IL track, (2) developing program policies and  procedures, and regional guidance, and (3) formulating and monitoring  the agency’s budget and program expenditures. Currently, there are 56  VA regional offices with responsibility for administering the IL track in their  jurisdiction. Reporting to these regional offices are 179 satellite offices  that help ensure veterans who do not live near a regional office are able  to access VR&E benefits, including those provided through the IL track  (see fig. 1).
The VR&E Officer in each regional office is responsible for ensuring that  the office complies with national policies and procedures for monitoring  performance to ensure VR&E’s national rehabilitation rate is met. In  addition, this officer supervises regional Vocational Rehabilitation  Counselors (VRCs) who are responsible for working directly with veterans  to assess their IL needs and provide benefits.
VR&E has established a formal process that all veterans must go through  to receive any of the benefits offered through its five tracks (see fig. 2). It  is not possible to be assigned to any of the five tracks without first  applying and being accepted into the VR&E program.
An individualized IL plan is developed for each veteran admitted to the IL  track. This plan identifies the veteran’s IL goals, the benefits (goods and  services) VR&E will provide to help the veteran meet those goals,  estimated time frames for providing benefits, and how improvements in  the veteran’s independence will be measured, among other areas (see  fig. 3). In general, IL plans may not exceed 24 months except under  certain circumstances. According to officials, there is also no statutory  limit on the number of IL plans a veteran can have. Veterans can reapply  to the VR&E program at any time.
Furthermore, all IL plans are required to be reviewed and approved by the  veteran, the VRC, and the VR&E Officer before the veteran is admitted to  the IL track. Once the plan is approved, VRCs are required to meet with  the veteran at least once a month to track progress in meeting the plan’s  goals. In addition, VRCs are required to coordinate with other VA  programs to determine if they can provide any of the goods and services  identified in the veteran’s plan. The VA programs that officials told us they  regularly coordinate with are:
VBA’s Specially Adapted Housing (SAH) Program, which helps  servicemembers and veterans with certain service-connected  disabilities purchase or construct an adapted home, or modify an  existing home to accommodate a disability. These modifications can  include specially adapted bathrooms or the installation of exit ramps.
VHA medical centers, which provide a wide range of health-related  services to veterans, such as physical therapy and dental and  optometry services. In addition, VHA’s Prosthetic and Sensory Aids  Service (PSAS) provides a variety of medical devices and equipment  to veterans, including artificial limbs, eyeglasses, hearing aids,  hospital beds, and wheelchairs. PSAS also administers the Home  Improvements and Structural Alterations grant to eligible veterans for  home modifications, such as the widening of bedroom or bathroom  doorways to allow wheelchair access.
If the goods and services needed cannot be obtained through other VA  sources, the VR&E program will purchase them directly or contract for  them. VR&E may declare a veteran successfully “rehabilitated” when all  goals in their IL plan have been achieved, or if not all achieved, when the  following three conditions have been met: (1) the veteran has attained a  substantial increase in the level of independence; (2) the veteran has  maintained the increased level of independence for at least 60 days; and  (3) further assistance is unlikely to significantly increase the veteran’s  level of independence. “Rehabilitation” does not necessarily mean that a  veteran’s disability has improved, but rather that the goals in the veteran’s  IL plan have been met to facilitate their independence. When veterans  are not successfully rehabilitated, their case may be closed as  “discontinued.” When circumstances arise that affect the ability of VR&E  to provide the goods and services identified in the IL plan to the veteran,  VR&E can temporarily “interrupt” the veteran’s case until it becomes  possible to continue.
section_title: The IL Track Mainly Serves Older Veterans and Provides a Wide Variety of Benefits
subsections: section_title: IL Serves Mainly Vietnam- Era Veterans and Those with Post-traumatic Stress Disorder
paragraphs: From fiscal years 2008 through 2011, the typical participant in the IL track  was an older male Vietnam-era veteran. Of the 9,215 veterans who  entered the IL track at some point during these years, most (67 percent)  were males 50 years old or older. Most women in the IL track were in  their 40s or 50s (see fig. 4). The gender and age distributions of IL track  veterans were similar to those of veterans who were receiving VA  Disability Compensation cash benefits. In fiscal year 2011, about 92  percent of these veterans were male, and almost 68 percent were age 55  or older.
Most of the 9,215 veterans who entered the IL track during fiscal years  2008 through 2011 served in the Vietnam War. However, there was a  slight increase in the number of “Global War on Terrorism” veterans— including those who served during the war in Afghanistan as part of  Operation Enduring Freedom and the war in Iraq as part of Operation  Iraqi Freedom—from 13 percent in fiscal year 2008 to 16 percent in fiscal  year 2011 (see fig. 5).
In addition, almost 60 percent of IL track veterans served in the U.S.  Army and less than 1 percent served in the National Guard or Reserves.  Forty-one percent served in the U.S Navy, U.S. Air Force, or U.S. Marine  Corps (see fig. 6).
With regard to severity of service-connected disabilities, more than three- quarters of IL track veterans had a combined service-connected rating of  at least 60 percent. Thirty-four percent of IL track veterans had a  combined service-connected disability rating of 100 percent. Many IL  track veterans may have also been eligible for the VA Disability  Compensation program’s Individual Unemployability cash benefit.  Indeed, 38 percent of IL track veterans were eligible for this supplement  to their disability compensation payments. Moreover, of those IL track  veterans who were at least 60 years old, 85 percent had combined  service-connected disability ratings of at least 60 percent and more than a  third had combined ratings of 100 percent.
IL track veterans had a wide variety of service-connected disabilities, and  most had more than one. Regardless of disability rating level, the most  prevalent disabilities among this group were Post-traumatic Stress  Disorder (PTSD), tinnitus (“ringing in the ears”), and hearing loss. As  shown in figure 7, many of these disabilities—7 of the top 10 we identified  in these cases—are similar to the most prevalent conditions among  veterans rated by VA’s Disability Compensation program in fiscal year  2011.
Our review of the case files of 182 randomly selected IL track veterans in  fiscal year 2008 shows that they were provided a wide range of goods  and services to help them live independently. Goods and services ranged  from individual counseling and the installation of ramps to improve  accessibility of veterans’ homes, to a boat, camping gear, and computers  intended by VA to help decrease veterans’ social isolation. Figure 8  provides examples of the types of benefits provided in four IL track cases  we reviewed. (See appendix III for a complete list of goods and services  we identified in 182 IL track cases.)
While VR&E can provide a wide range of benefits to IL track veterans,  VR&E’s policy as of November 2012 prohibits providing courses of study  and training to veterans that would lead to academic degrees, and  purchasing certain types of equipment, such as lawn tractors. Our  review of fiscal year 2008 IL cases shows instances where VR&E  purchased lawn tractors and a riding mower. However, VR&E officials told  us that there was no specific guidance prohibiting provision of such  equipment in that year.
As shown in table 1, we classified the wide range of goods and services  we found in the 182 IL track case files we reviewed into nine general  categories:  Although we were unable to estimate the number of times certain goods  and services were provided in the 182 IL track cases we reviewed, we  generally found that the most common type of goods or services provided  in the cases we reviewed were in three categories: 1) counseling, 2)  education and training, and 3) computer and camera equipment.
We were, however, able to estimate the costs of the goods and services  purchased for IL track cases. For all veterans who entered the IL track in  fiscal year 2008, we estimate that VR&E purchased a total of almost $14  million in goods and services. The average spent on purchases of  goods and services per IL track case that year was nearly $6,000. In the  182 IL track cases we reviewed, the total amount VR&E spent on  purchases of goods and services per veteran varied from $20 to over  $33,000. Nearly 40 percent of the 182 veterans received $4,000 or more  on purchases of goods and services (see fig. 9).
section_title: Most Veterans Completed “Rehabilitation” but with Varying Time Frames
subsections: section_title: Most Veterans in the IL Track are Ultimately Classified by VR&E as “Rehabilitated”
paragraphs: Based on our analysis of IL track veterans who began only one plan  during fiscal year 2008, we found that most of these veterans were  classified by VR&E as “rehabilitated”—successfully reaching and  maintaining the goals identified in their IL plan. By the end of fiscal year  2011, as shown in figure 10, about 89 percent of veterans who began an  IL plan in fiscal year 2008 had been classified by VR&E as “rehabilitated.”  About 5 percent of cases were “discontinued” or closed by VR&E  because the rehabilitation goals in the veteran’s IL plan were not  completed. The remaining 6 percent of veterans’ cases were still open at  the end of fiscal year 2011.
Of the 122 IL cases from fiscal year 2008 that had been discontinued, we  found that about half were discontinued because the veteran declined  benefits or did not respond when VA attempted to contact them. Another  29 percent were discontinued because the veteran died. Also, some  VR&E officials and staff we spoke with during our site visits told us that  veterans may drop out of the IL track for other reasons, such as their  condition worsens or they experience family problems.
Furthermore, while most veterans were classified by VR&E as  “rehabilitated,” we found that the nature and complexity of IL plans can  vary based on veterans’ individual disabilities and needs. As such, some  plans are easier for VR&E to close as rehabilitated than others. For  example, in one case we reviewed, the IL plan for a veteran with  rheumatoid arthritis only called for the purchase and installation of eight  door levers and a grab rail for the bathtub, so that the veteran could enter  the rooms of his home without pain or assistance, and be able to safely  enter and exit the bathtub without fear of falling. Other cases were more  complex to close as rehabilitated because of the nature of services being  provided to the veteran. For instance, another case we reviewed called  for providing a veteran who used a wheel-chair with medical, dental, and  vision care as needed and about $24,000 in modifications to the veteran’s  home, including modifying the veteran’s bathroom, widening doors and  modifying thresholds, and installing an emergency exit ramp in a  bedroom.
While the overall IL rehabilitation rate nationwide of 89 percent is fairly  high, we found that the percentage of veterans who began an IL plan in  fiscal year 2008 and were rehabilitated by the end of fiscal year 2011  varied by regional office. IL rehabilitation rates in these cases ranged  from 0 to 100 percent across regional offices, although about two-thirds  of regional offices had rehabilitated 80 percent or more of 2008 IL track  veterans by the end of fiscal year 2011. (See appendix IV for a complete  list of rehabilitation rates for all regional offices.) In addition, VR&E’s IL  rehabilitation rate was higher in regional offices with larger IL caseloads.  Among veterans who entered the IL track in fiscal year 2008, an average  of 90 percent were rehabilitated at offices with more than 25 IL entrants,  compared to an average of 79 percent at offices with 25 or fewer IL  entrants (see fig. 11).
Furthermore, we found that fiscal year 2008 IL track veterans nationwide  completed their IL plans, i.e., were classified rehabilitated, in an average  of 384 days (about 13 months). About 50 percent of these veterans  completed their IL plans within 1 year, 79 percent within 2 years, and 88  percent within 3 years. Average days for these IL track veterans to  complete their plans varied by regional office from a low of 150 days at  the St. Paul Regional Office to a high of 895 days at the Roanoke  Regional Office. However, at most regional offices (49 of 53) the  average time to complete IL plans ranged from 226 to 621 days (8 to 21  months). (See appendix V for a complete list of rehabilitation times for all  regional offices.) Also, as shown in figure 12, IL track veterans generally  completed their plans in less time at regional offices with larger IL  caseloads than at offices with smaller caseloads.
The average time for veterans to complete their IL plans also varied by  veteran age. The oldest veterans completed their plans slightly more  quickly than the youngest. More specifically, veterans who were 70 to 79  completed their plans in an average of 369 days, while those who were  20 to 29 completed them in an average of 413 days (see fig. 13).
Because differences in the length of time to complete IL plans could have  occurred by chance or be explained by many factors—including the type  and severity of disability or veteran age—we used a statistical model to  estimate the amount of time it would take certain groups of IL track  veterans to complete their IL plans while controlling for type and severity  of disability and other demographic characteristics. (See appendix II for  more detailed information on our model and analysis.) The results of our  model showed that the chance of rehabilitation within 2 years varied by  regional office. At four regional offices the chance of being rehabilitated  within 2 years was less than 50 percent. At 18 offices the chance was  between 50 and less than 90 percent. At 16 offices the chance was 90  percent or greater.
Our model also estimated that a veteran served by a regional office with a  large IL caseload had a higher probability of completing an IL plan more  quickly than a veteran served by an office with a small IL caseload. More  specifically, we determined that a typical IL track veteran at an office with  a total of 75 IL cases entering the IL track during fiscal years 2008  through 2011 had a 48 percent chance of completing an IL plan within 1  year and a 92 percent chance of completing an IL plan within 3 years,  while a veteran at a regional office with 700 IL cases had a 57 percent  chance of completing an IL plan within 1 year and a 96 percent chance of  completing their plan within 3 years (see fig. 14). Finally, based on our  model we estimated that older veterans had a higher probability of  completing an IL plan more quickly than younger veterans. For example,  a 75 year old veteran had a 41 percent chance of completing an IL plan  within one year, while a 30 year old veteran had a 33 percent chance. We  found no meaningful differences in the time it takes to complete an IL plan  as “rehabilitated” according to veteran gender and most disability types.  However, we found small differences according to length of military  service, education, and combined service-connected disability rating (see  appendix II).
section_title: VR&E Exercises Limited Oversight of the IL Track
paragraphs: National oversight of VR&E’s IL track is limited in four key areas: (1)  ensuring compliance with case management requirements; (2) monitoring  regional variation in IL track caseload and benefits provided; (3)  adequacy of policies and procedures for approving expenditures on  goods and services for IL track veterans; and (4) availability of critical  program management information.
subsections: section_title: Regional Offices May Not Be Complying with VR&E Case Management Requirements
paragraphs: Based on VR&E’s compliance reports from two of the regional offices we  visited and interviews with staff in these offices, we found that some  offices may not be complying with certain VR&E case management  requirements governing the IL track. Specifically, some Vocational  Rehabilitation Counselors (VRC) are not fulfilling VR&E’s requirement to  meet in-person each month with IL track veterans to monitor progress in  completing their IL plans. Site visit monitoring reports from two  regional offices we visited indicated that these offices faced difficulties  complying with this requirement. According to these reports, some VRCs  were not able to schedule monthly in-person meetings with IL track  veterans to check their progress in completing their IL plans. In addition,  VRCs we spoke with during our site visits said that compliance with this  requirement continues to be a challenge due to the size of their caseloads  and the distances that they have to travel to meet with veterans. While  VR&E’s Central Office acknowledges that it can be challenging for some  offices to comply with this requirement, officials stated that face-to-face  contact with IL track veterans is extremely important due to the nature of  their disabilities and the need for more intensive rehabilitation services.
Furthermore, while VR&E and VHA both have policies that require them  to coordinate on the provision of goods and services for IL track veterans,  we found that some VRCs experience challenges in doing so.  Specifically, VA regulations governing VR&E’s IL track require that if a  veteran needs special equipment and is eligible for such equipment under  another VA program, the items will be provided under that program.  Therefore, prior to VR&E purchasing services for veterans, officials told  us that VRCs are required to refer cases to other VA providers, such as  VHA’s Home Improvements and Structural Alterations (HISA) program  and VA medical centers, to determine whether veterans are eligible to  receive the goods and services identified in their plan from these  providers. In addition, VHA ‘s policy directive for coordinating with VR&E  is intended to ensure that all VR&E participants, including those in the IL  track, receive timely access to VHA health care services. The directive  requires medical center directors to (1) provide clinically appropriate care  to VR&E participants, (2) establish procedures to manage these  participants’ timely access to care, and (3) ensure that clinical staff are  trained in these procedures, including how to alert appropriate VBA  officials when timeliness of care could be an issue, among other  requirements.
Despite these VR&E and VHA policies, our review of 182 IL track records  found some instances where VR&E purchased goods and services that  appear to be medically related, such as ramps and grab bars, and could  have been provided by VHA. There may have been little coordination on  IL track cases because, as VR&E officials and staff at the five VA regional  offices we visited told us, coordinating with VHA medical center staff can  be challenging. Several VRCs indicated that when they refer IL track  cases to VHA physicians to determine whether any of the medical-related  benefits identified in the IL plans—such as grab bars, wheelchairs, and  safety rails—had been or could be provided by VHA, in many instances,  VHA physicians do not respond or respond too late. As a result, services  for IL track veterans were either delayed or purchased by VR&E instead  of VHA. According to some VHA staff, lack of or delays in coordination  are sometimes due to differences in VR&E rehabilitation goals and VHA  long-term medical goals for veterans. For example, VHA staff members  mentioned they would not approve VRCs’ requests for lift chairs unless  veterans absolutely needed them because VHA staff members said their  medical goal is to promote continued mobility.
section_title: VR&E Does Not Systematically Monitor Variation in IL Track Caseload Size and Benefits Provided
paragraphs: While IL track caseload size and benefits vary across regions, VR&E  does not systematically monitor these variations. We found regional  differences in the size of IL caseloads, the range of needs covered in a  veteran’s IL plan, and the types of goods and services provided. As  shown in figure 15, the total IL track caseload during fiscal years 2008  through 2011 ranged from over 900 cases in the Montgomery Regional  Office in Alabama to 4 cases in the Wilmington Regional Office in  Delaware. (See appendix VI for annual IL caseload data.) While one  would expect differences in caseload size based on the number of  veterans in an office’s service area, some differences in IL caseload could  not be explained by the size of the veteran population in an office’s  service area. For example, while the estimated size of the veteran  populations in Alabama and Massachusetts are similar, Montgomery’s IL  caseload of over 900 veterans was more than 100 times as large as  Boston’s IL caseload of 8 veterans. VR&E officials told us they are aware  that some offices have low IL participation levels, and attributed some of  the difference to factors, such as the presence of specialized treatment  centers in certain areas of the county that can increase caseload  numbers in those regions. However, evidence from our site visits  suggests that there may be other factors that drive the number of  veterans entering a region’s IL track. For example, officials from the  Detroit Regional Office—which had the second highest total IL caseload  across fiscal years 2008 through 2011—attributed the size of their  caseload to their office’s focus on IL cases and community outreach  efforts, including the involvement of veteran service organizations. The  Atlanta Regional Office also mentioned that in addition to the size of the  veteran population in Georgia and the presence of army bases, the size  of their IL caseload is influenced by increased community interest in the  IL track in recent years, and the informal efforts of past IL participants to  provide information about the IL track to other veterans.
Based on discussions with regional personnel and our review of the  results of past studies, IL plans and the goods and services provided can  also vary by region. According to VR&E officials, the law gives VA broad  discretion in defining the range of needs covered in an IL track plan, as  well as in choosing the types of goods and services required to achieve  the plan’s goals. VR&E’s policy does, however, prohibit the agency from  providing courses of study and training leading to academic degrees, and  purchasing certain types of equipment, such as lawn tractors. As a result,  some regions appear to develop IL plans that address a broader range of  needs while others elect to develop more focused plans. Officials we  spoke with during our site visits also confirmed that regions differ in their  approach to developing IL plans, and in one region, officials commented  that some regions may develop simpler plans with fewer benefits to  achieve VR&E’s rehabilitation goal. Moreover, VR&E’s 2008 study of the  IL track reported that some regions relied repeatedly on the same types  of benefits to meet veterans’ IL needs. For example, in the New York  and Hartford regional offices, nearly 90 percent of IL participants were  provided computer training. This was almost twice the rate of Detroit—the  office with the next highest rate of providing veterans with computer  training—in 44 percent of its IL cases. Other offices focused more heavily  on providing goods or services to support veterans’ avocational interests  or on finding veterans volunteer opportunities, while some consistently  focused on providing veterans with health club memberships. For  example, the St. Petersburg Regional Office accounted for 33 percent of  veterans receiving volunteer opportunities. In addition, the 2004 VR&E  Task Force report noted significant differences in field office philosophy  regarding the scope and administration of the IL track, and also  recommended providing VR&E Central Office greater “line-of-site  authority” over VR&E field operations. As noted earlier, VBA’s Office of  Field Operations rather than VR&E’s Central Office has direct line of  authority over regional VR&E staff and operations.
While VA officials acknowledged the existence of regional variation in  these areas, VR&E’s current oversight approach may not be adequate to  ensure consistent administration of the IL track across regions. The  standards for internal control in the federal government emphasize the  need for federal agencies to have timely information to effectively monitor  and mitigate program risks. To oversee administration of the IL track,  VR&E has relied on the information provided through its general quality  assurance (QA) activities and a series of periodic ad hoc studies. The QA  activities are conducted by each region as well as VR&E’s QA staff  located in Nashville, Tennessee. However, these activities are limited in  scope, frequency, and how the information is used. At the national level, a  team of VR&E QA staff are required to perform periodic site visits to each  VA regional office at least every 3 years, although officials told us they  have not been able to meet this goal in recent years because of  budgetary constraints. This team also periodically reviews a sample of  veterans’ records from VR&E’s IL and employment tracks in selected  regions. After each site visit and case review, the QA team produces a  report for the particular region under review. While VR&E officials told us  that QA results are “analyzed to determine trends,” it is unclear how  VR&E uses these results to ensure that consistent procedures are used  by regional offices nationwide.
In addition to these QA activities, VA has funded two studies over the  past several years on the VR&E program, including implementation of the  IL track, and a third is under way. In 2003, VA appointed a task force to  evaluate the overall VR&E program and recommend ways service  delivery could be improved. In 2008, as mentioned, a study was issued by  VR&E on the types of services received by IL track participants and the  impact those services had on veterans. This study found, among other  things, that IL participants received a broad range of services, and that  some regional offices favored providing certain types of services to IL  track veterans. Finally, in October 2012, VR&E awarded a contract for  about $890,312 to study the IL track service delivery process, barriers  and incentives for field staff when considering the provision of services,  current and future training strategies, and other areas. VR&E officials  anticipate receiving the final results from this study in July 2013.
section_title: VR&E Policy for Approving IL Track Expenditures Might Not Be Adequate
paragraphs: Considering the broad discretion VR&E provides to regions in determining  the range of needs covered in IL track plans and the types of goods and  services selected to meet these needs, VR&E’s current policy for  approving IL track expenditures may not be adequate. To guard against  fraud, waste, abuse, and mismanagement, the standards for internal  control emphasize the need for agencies to have appropriate levels of  supervision and controls in place to prevent one individual from having  responsibility for all key aspects of a transaction or event. VR&E Central  Office is responsible for managing the program’s national budget. While  officials told us that VRCs are required to include all cost estimates when  they submit veterans’ IL plans to be reviewed and approved by the  region’s VR&E Officer, VR&E was unable to provide any written policy or  guidance that explicitly states this requirement for all IL expenditures.  VR&E’s current written policies allow each region to unilaterally approve  construction-related expenditures up to $25,000 per case and non- construction expenditures up to $100,000 per case without approval from  VR&E Central Office. In addition, VRCs can spend up to $2,000 on  construction-related expenditures and up to $25,000 on non-construction  expenditures without supervisory review of any kind (see table 2). Under  this policy, regional offices are permitted to purchase a broad range of  items without any Central Office approval, resulting in some offices  purchasing goods and services that may be questionable or that are  costly. For example, in one of the 182 IL cases we reviewed, VR&E  Central Office would not have been required to approve total  expenditures of $18,829 for a riding lawn mower—which VR&E policy  prohibits—and other IL goods and services including a bed, bed frame,  desktop computer, and woodworking equipment. In another case, VR&E  Central Office approval would not have been required for the purchase of  a boat, motor, trailer, and the boat’s shipping cost to Molokai, Hawaii,  among other items, totaling about $17,500. Without appropriate  approval levels, VR&E’s IL track may be vulnerable to potential fraud,  waste, and abuse.
VR&E does not collect or report critical program management  information—such as the costs of goods and service purchased, the  types of benefits provided, and the number of IL veterans served—that  would help the agency in its oversight responsibilities. Specifically,  VR&E’s Corporate WINRS case management system—commonly  referred to as “CWINRS”—lacks information on the total amount of funds  VR&E expends on IL benefits because the agency does not maintain  specific cost information on each of its five tracks or the key program  areas these tracks represent, which are employment services and  independent living. VR&E aggregates costs across all its tracks, despite  VA’s managerial cost accounting policies that require the costs of  products and services to be captured for management purposes.  Federal financial accounting standards also recommend that costs of  programs be measured and reported. According to officials, they do not  collect cost information on the IL track alone because they view the five  tracks as a single program with the same overarching goal—to help  veterans achieve their employment goals. However, in our 2009 report on  the VR&E program, we reported that VR&E serves veterans with  fundamentally different program goals and outcomes—those seeking to  obtain employment and those seeking to live independently. As a result,  we recommended that VR&E separately track its IL rehabilitation rate.
In addition, the CWINRS system does not collect information on the types  of IL benefits provided to veterans in a standardized manner that can be  easily aggregated and analyzed for oversight purposes. Currently, VRCs  are required to write in a description of the benefits they purchased for IL  track veterans in CWINRS. We found, however, that the recorded  descriptions lack consistency and vary in their level of detail. In several of  the IL track cases we reviewed, the goods and services purchased were  grouped together under a general description, such as “IL equipment” or  “IL supplies,” without any further details. In other cases, VRCs recorded  each purchase separately. For instance, in one case, the VRC created  separate entries for each item purchased, for example, “desktop  computer,” “computer desk,” “Queen Mattress,” “all in one printer,” “USB  cable and ink,” “threshold ramp,” and “washer.” In addition, we found that  controls for data entry are not adequate to ensure that all important data  is recorded. For example, we estimated that the service provider field  was either missing or unclear for one or more services in about 15  percent of all IL cases that began in fiscal year 2008. Moreover, in  another 7 percent of IL cases that began during fiscal year 2008, we  found that service provider data was unavailable because CWINRS had  overwritten veterans’ 2008 IL plans with subsequent plans. CWINRS only  retains veterans’ current IL plan.
Furthermore, CWINRS does not provide VR&E with the information it  needs to account for the actual number of IL track veterans it serves to  monitor its statutory entrant cap and program operations. The law allows  VR&E to initiate “programs” of independent living services and assistance  for no more than a specified number of veterans each year, which is  currently set at 2,700. A VR&E policy letter to VA regional offices, dated  December 14, 2010, states that “VR&E Service will monitor new IL  programs by tracking the number of veterans entering independent living  case status in each fiscal year and notify the field of any necessary action  if the 2,700 statutory limit is being approached.” However, in analyzing  VR&E’s administrative data, we found that VR&E counts the number of IL  plans developed annually and not the number of individual veterans  admitted to the track. VR&E officials told us that the law does not limit the  number of IL plans that can be developed for a veteran in a single fiscal  year. Because multiple IL plans can be developed for an individual  veteran during the same fiscal year, veterans with multiple plans are  counted more than once towards the statutory cap by CWINRS. Based on  data from CWINRS, 403 (4 percent) of the 9,215 veterans beginning an IL  plan during fiscal years 2008 through 2011 had more than one plan  during that time. About 22 percent of these veterans had more than one  IL plan in the same fiscal year. From fiscal years 2008 through 2011,  VR&E did not exceed the statutory cap (see fig. 16).
Moreover, VR&E officials indicated that they were unaware that CWINRS  was calculating the number of plans and not individual veterans. As a  result, VR&E lacks complete information on the number of veterans it is  serving through the IL track at any given time—information it could use to  better manage staff, workloads, and program resources. According to the  standards for internal control, it is important for program managers to  have reliable and timely “operational” information to ensure agency goals  are met, resources are used efficiently and effectively, and requirements  under various laws and regulations are being complied with.
Finally, recent evaluations of CWINRS have determined that the system  does not meet VR&E’s current needs and limits its oversight abilities  because important data elements are not captured to support the  agency’s “evolving business needs.” According to a 2012 VR&E report,  VR&E plans to improve CWINRS by (1) increasing data automation to  reduce instances of human error, and (2) enhancing system controls to  improve the accuracy and comprehensiveness of the data, among other  improvements. However, it appears that the CWINRS redesign will not  enable VR&E to obtain data on IL track expenditures from the system or  to use the system to accurately and efficiently track the types of goods  and services the IL track provides. While officials told us that they plan to  modify the system to enable them to track individual veterans served  through the IL track, they could provide no time frame for this project and  noted that it could take up to 3 years to obtain funding for it.
section_title: Conclusions
paragraphs: VR&E’s IL track can provide a wide range of benefits to help veterans  with service-connected disabilities maximize their ability to live  independently when the achievement of an employment goal is not  currently feasible. Under the law, each VA region has broad discretion in  determining the range and types of goods and services each IL track  veteran should receive. Consequently, adequate oversight at the national  level is extremely important in ensuring that IL track case management  requirements are met, the track is administered consistently across  regions, expenditures for goods and services are appropriate, and critical  IL track information is collected and used.
In this context and absent better coordination with VHA, VR&E will  continue to face challenges in ensuring goods and services for IL track  veterans are provided by VHA, when appropriate, in a timely manner.  With regard to VR&E’s Central Office oversight, its current policy  governing the review and approval of expenditures might not effectively  mitigate the risk of fraud, waste, abuse, or mismanagement inherent in  activities, such as the development and implementation of IL plans, where  there are few limitations on the types and cost of goods and services that  can be provided. Furthermore, without collecting information on cost and  the types of benefits regional offices provide to IL track veterans, and  maintaining an information system that ensures IL track data are recorded  consistently and are accurate and complete, VR&E cannot effectively  oversee the performance and activities of its IL track. Finally, as more  servicemembers from the conflicts in Iraq and Afghanistan transition into  civilian life and veterans’ programs and as the overall veteran population  ages, the demand for IL services will likely increase. VR&E must,  therefore, be able to accurately count the number of individual IL track  veterans served to ensure it can effectively manage its cap and assess  program performance and resource needs.
section_title: Recommendations for Executive Action
paragraphs: The Secretary of Veterans Affairs should direct the Undersecretary for  Benefits to take the following actions:  1.  Work with the Undersecretary for Health to explore options on ways to  enhance coordination to ensure IL track veterans’ needs are met by  VHA, when appropriate, in a timely manner. This could include  improving staff education, response times to IL referrals, and the  provision of medically related goods and services.  2.  Implement an oversight approach that enables VR&E to better ensure  consistent administration of the IL track across regions. In developing  this approach, consider ensuring that VR&E’s CWINRS system:
Tracks the total cost and types of benefits provided to each  veteran in the IL track by regional office.
Accounts for the number of individual veterans served to ensure  that the agency has the information it needs to adequately  manage the IL track.
Contains stronger data entry controls to ensure that IL track  information is recorded in a consistent manner and is accurate  and complete.  3.  Reassess and consider enhancing the agency’s current policy  concerning the required level of approval for IL track expenditures,  given the broad discretion individual regional offices have in  determining the types of goods and services IL track veterans receive.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the Secretary of Veterans Affairs for  review and comment. VA generally agreed with our conclusions, and  concurred with our three recommendations. VA’s written comments are  reproduced in appendix VII. In addition, VA provided technical comments  that we have incorporated in the report where appropriate. With regards  to our recommendation that VA enhance coordination between VR&E and  VHA on the provision of IL benefits to veterans, VA noted there are efforts  underway that will permit VR&E counselors to make IL referrals to VHA  providers through VHA’s Compensation and Pension Record Interchange  system, also referred to as CAPRI. In addition to these efforts, we  encourage VR&E and VHA to continue to explore other options to  enhance coordination, such as engaging in staff education and training  efforts to improve the timely provision of IL benefits and prevent any  duplication. Furthermore, with regards to our recommendation to improve  IL track oversight, VA said that it will perform an assessment of its current  IL track oversight procedures and consider including enhancements to  CWINRS. Because having a sound information system plays such a  critical role in helping program managers monitor agency performance  and operations, we urge VA to take steps to prioritize enhancements to  CWINRS. Finally, VA noted that, in response to our recommendation that  it reassess its current policy for approving IL track expenditures, it plans  to make any needed improvements by March 2014. As part of its  assessment process, we believe that VA should review its existing cost- thresholds for approving all IL expenditures, including non-construction  related expenditures, to ensure adequate oversight of goods and services  purchased for veterans.
We are sending copies of this report to the appropriate congressional  committees, the Secretary of Veterans Affairs, and other interested  parties. The report is also available at no charge on the GAO web-site at  www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at (202) 512-7215 or at bertonid@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. Staff members who made key contributions  to this report are listed in appendix VIII.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this report were to examine (1) the characteristics of  veterans who have participated in the IL track, as well as the types and  costs of benefits they were provided; (2) the extent to which their  independent living plans were completed, and the time it took; and (3) the  extent to which the IL track has been administered appropriately and  consistently.
To answer these objectives, we used a variety of methodologies, which  are described in greater detail below. Specifically, we analyzed VA  administrative data from fiscal years 2008 through 2011 to provide  information on the characteristics and outcomes of veterans who  participate in the IL track, and how long it takes to achieve these  outcomes. We also used statistical models to determine any differences  in rehabilitation times. In addition, to determine the types of benefits  provided and their costs, we conducted a case file review of a random,  generalizable sample of 182 veterans who started in the IL track during  fiscal year 2008. Lastly, to examine the administration of the IL track, we  visited five VA regional offices and interviewed officials and staff from  VR&E, Specially Adapted Housing (SAH), Prosthetics and Sensory Aid  Service (PSAS) under the Veterans Health Administration (VHA), and  veteran service organizations. We also reviewed relevant federal laws  and regulations, standards for internal controls and managerial cost  accounting, as well as VR&E policies, procedures, and other relevant  studies and documentation.
subsections: section_title: Analysis of VA Administrative Data
paragraphs: We analyzed VA administrative data in the Corporate WINRS system— commonly referred to as “CWINRS”—from fiscal years 2008 through  2011 to describe the characteristics and outcomes of veterans in the IL  track, and how long it takes veterans to achieve these outcomes.
VR&E has used the CWINRS system since 2001 to track veteran cases through its  process and to manage program costs. The “WINRS” part of the acronym represents the  first 5 regional offices that tested the original system. These offices include Waco, Texas;  Indianapolis, Indiana; Newark, New Jersey; Roanoke, Virginia; and Seattle, Washington. we found the data to be sufficiently reliable for our purpose, unless  otherwise noted.
To identify our universe of 9,215 veterans who started in the IL track  during fiscal years 2008 through 2011, we counted the number of  individual veterans who had an “IL status” recorded in CWINRS during  our time frame. While veterans can enter into the IL track more than  once in a given fiscal year, we counted a veteran only once if he or she  had more than one IL status during these years. To determine the  characteristics of the 9,215 veterans who began in the IL track during  fiscal years 2008 through 2011, we examined several demographic  variables in CWINRS, including veterans’ age, gender, types of  disabilities, and combined disability rating. Data on the war era in which  veterans served was obtained from VA’s Benefits Delivery Network.
To examine the outcomes of veterans in the IL track, we limited our  analysis to veterans who began only one IL plan at some point during  fiscal year 2008. This accounted for about 92 percent of the 2,465  veterans who began an IL plan during fiscal year 2008. The remaining 8  percent of veterans who had more than one plan were excluded from our  analysis because of potential data reliability issues associated with how  certain case actions were recorded. In our analysis of IL veteran  rehabilitation rates, we excluded veterans whose cases were transferred  to one of VR&E’s employment tracks.
To estimate the difference in rehabilitation times across regional offices  and groups of veterans with various characteristics, we used different  statistical models. These “survival modeling” methods allowed us to  appropriately estimate rehabilitation probabilities at various points in an IL  plan, even when a significant proportion of the cases are discontinued or  in progress. Our models used the same VA administrative data noted  above for veterans in the IL track who began only one IL plan at some  point during fiscal years 2008 through 2011. Consequently, our model  results only apply to a subset of veterans, which make up about 95  percent of the veterans who began in the IL track during this time frame.  For a detailed discussion of our statistical models, see appendix II.
section_title: Analysis of IL Veteran Cases
paragraphs: To obtain information on the types of benefits provided through the IL  track, as well as the costs and coordination of these benefits, we  conducted a case file review of a random, generalizable sample of 182  veterans whose cases started in the IL track at some point during fiscal  year 2008. To track and manage IL cases, VR&E utilizes both paper  records located at each regional office, and electronic case records in the  CWINRS system. For the purpose of our file review, we used veterans’  electronic records in CWINRS, and found the information in these records  to be sufficient for our purpose. We selected fiscal year 2008 because it  allowed us to follow veterans’ progress in the IL track beyond 24  months—the authorized benefit duration time frame unless a 6-month  extension is approved.
We stratified our sample of 182 cases into three groups—those veterans’  whose cases were (1) rehabilitated, (2) discontinued, or (3) still open.  We used this approach to obtain information on any differences in IL  benefits and expenditures by veteran groups. We also elected to use a  sample design that mixes the features of a proportional allocation and  equal sample per stratum. Table 3 provides more detailed information on  our sample stratification.
To produce population estimates from our sample of cases, information  collected from each case reviewed was weighted in the analysis to  account statistically for the total population of all IL track cases started in  fiscal year 2008. Because our estimates for the entire population of IL  track cases that started in fiscal year 2008 are based on a probability  sample of these cases, they are subject to sampling error. To recognize  the possibility that other samples drawn from this population might yield  different results, we express our confidence in the precision of our  sample’s results as a 95 percent confidence interval. A 95 percent  confidence interval is expected to include the actual value for the  population in 95 out of 100 samples drawn from the population. With  regard to expenditures in IL track cases, at the 95 percent confidence  interval, the amount of total expenditures across all IL track cases that  started in fiscal year 2008 falls between $11.3 million and $16.6 million;  the actual amount expended, on average, in these cases falls between  $4,835 and $7,104.
To conduct our case file review, we developed a web-based data  collection instrument to record information from the electronic printouts  VR&E provided us with of veterans’ records. After these records were  collected, GAO staff members entered the information into the electronic  data collection instrument for each veteran case in our sample. Once the  data was entered, a second GAO staff member reviewed each entry for  clarity and accuracy. With regard to the types of benefits provided to IL  track veterans, we cannot generalize the types of goods and services  provided because we recorded in our data collection instrument  information on only eight goods and services identified in each of the 182  IL case files we reviewed. However, we were able to generalize  information we obtained on IL track service providers from the IL cases  we reviewed because in less than 3 percent of these cases, there were at  least eight service providers. Our estimate of the number of cases where  service provider information was missing, unclear, or overwritten in  CWINRS may be slightly understated. The 95 percent confidence interval  for the 15 percent of cases where the service provider information was  unclear or missing for one or more service ranged from 9 to 21 percent.  The 95 percent confidence interval for the 7 percent of cases we  reviewed where one or more service was overwritten in CWINRS ranged  from 4 to 11 percent.
section_title: Regional Site Visits and Agency Interviews
paragraphs: To better understand the extent to which the IL track is being effectively  administered, we visited five VA regional offices—San Diego, California;  Atlanta, Georgia; Detroit, Michigan; Philadelphia, Pennsylvania; and the  District of Columbia. We selected these sites because they were  geographically diverse, and have varying levels of IL track participation  and success in achieving VR&E’s IL rehabilitation goal. During each of  our site visits, we interviewed VR&E Officers and vocational rehabilitation  counselors (VRCs) using a standard set of questions to identify how IL  cases were managed and any factors that may have affected the  administration of the IL track. In particular, during our Detroit and Atlanta  site visits, we interviewed VRCs who specialized in working on IL cases.  Moreover, we obtained the perspectives of veteran service organizations  that work with IL track veterans at two regional offices we visited—Detroit  and Atlanta. Furthermore, we interviewed SAH and PSAS officials at the  national level and relied on our case file review to determine the extent to  which the IL track cases were referred to VA and non-VA service  providers. In addition, we interviewed SAH and PSAS staff during our site  visits to determine the extent to which VR&E, SAH, and PSAS coordinate  in the provision of goods and services to IL track veterans.
section_title: Appendix II: Statistical Analysis of Independent Living Plan Rehabilitation Times
paragraphs: We used statistical models to estimate the difference in rehabilitation  times across regional offices and groups of veterans with various  characteristics. These “survival modeling” methods can appropriately  estimate rehabilitation probabilities at various points in an IL plan, even  when a significant proportion of the cases are discontinued or in progress.  In addition, survival methods allowed us to estimate the unique  association between rehabilitation times and various other factors,  described below. In this appendix, we describe the data we used to fit the  models and their structure, assumptions, and estimates.
We estimated our models using VA administrative data on veterans who  began exactly one IL plan from fiscal years 2008 through 2011. Due to  the data reliability issues described in appendix I, our analysis excluded  veterans who began multiple IL plans during this period. Consequently,  our results only apply to this subset of 8,737 veterans, who made up 94.8  percent of the original population.
We measured the length of time each veteran spent in his or her IL plan  as the duration between the start of the plan and one of three later times:  entering rehabilitation status, discontinuing the plan, or being in progress  on September 30, 2011, the end of our observation period. We modeled  rehabilitation as the outcome of interest, treating discontinued or in  progress plans as having rehabilitation times that were censored. Among  the veterans starting IL plans from fiscal years 2008 through 2011, 3.2  percent discontinued their IL plans prior to rehabilitation, and 35.9 percent  had plans that were in progress on September 30, 2011. These data  imply that 91.7 percent of the censored observations were in-progress  cases, which suggests that most of the censoring is likely to be  uninformative. We found no large differences in the probability of  dropping out of an IL plan across the covariates we included in our  models.
This group also excluded veterans who transferred to a VR&E employment track and  veterans who had more than one instance of rehabilitation. Missing data on age and  length of service further reduced our sample size to 8,655 veterans for modeling  purposes.
subsections: section_title: Model Structure and Assumptions
paragraphs: We fit the data to several types of Cox proportional hazards models. The  primary model took the following form:  𝜆𝑖𝑗(𝑡) denotes the hazard rate of rehabilitation at time t after veteran i  enters an IL plan, and 𝜆0𝑗 (t) denotes the baseline hazard rate for the reference veteran with covariates equal to zero, served by one of j = 1, 2,  to 908. 𝑥ij is a vector of the following covariates: age, length of military … ,36 regional offices. We combined a number of offices that served less  𝜆𝑖𝑗(𝑡)=𝜆0𝑗 (t) exp(𝑥ij𝛽)  reference veteran with covariates equal to zero, served by one of j = 1, 2,  … ,36 regional offices. We combined a number of offices that served less  than 50 veterans into a residual group, in order to ensure sufficient  sample within each office. The sample sizes within offices ranged from 58  service, and indicators for sex, branch of service, era of service,  education, year the IL plan began, partial vs. full disability, and disability  type. We coded the covariates such that the baseline reference veteran  was substantively meaningful—a male Army veteran with a high school  education, who served in the Vietnam era, entered an IL plan in fiscal  year 2008, was processed by the Detroit Regional Office, and had a full  musculoskeletal disability—either by setting values of categorical  variables to the appropriate levels or by centering continuous variables at  their means. All categorical variables included levels for missing data.
In this application, the proportional hazard structure of the model implies  that the instantaneous rate of rehabilitation at any time during an IL plan,  or the hazard, varies across values of the covariates by a constant  amount over time. For example, the model assumes that rehabilitation  rates vary by the same amounts among veterans with different types of  disabilities after 12 months, 18 months, or any other point in an IL plan. In  our initial exploratory data analysis, Kaplan-Meier estimates of the hazard  and survival functions suggested that the hazard was not proportional  across regional offices. Using an unstratified version of the model above  that included fixed effects for regional offices, we found that the  Schoenfeld residuals for many offices were correlated with time at a  sufficiently high level to reject the hypothesis that the hazard functions  were proportional across the covariates (p < .0001). These results are  consistent with our interviews and document reviews suggesting that  program administration varied across offices, which could produce  variation in hazard functions. (The body of this report summarizes these  administrative inconsistencies in more detail.) For these reasons, we  stratified the baseline hazard function by regional office, and we used this  version of the model to calculate hazard rate ratios and elasticities, along  with their standard errors, in table 3. Under the stratified model, we failed  to reject the proportional hazards assumption for 𝛼 < .47.
The estimated rehabilitation (failure) probabilities discussed in the body of  this report came from a version of the model that did not stratify by office  and, instead, assumed office fixed effects. This allowed us to calculate  illustrative rehabilitation probabilities by various follow-up points and for  specific covariate values, without having to produce 36 sets of estimates  implied by separate hazard functions for each office. The differences in  the estimated probabilities across covariate values should resemble  estimates that we would have calculated from the stratified model,  because estimates of 𝛽 and their variances were similar in both versions.
The hazard rate ratios and elasticities in table 3 are consistent with our  estimated rehabilitation probabilities in the body of the report. To calculate  these probabilities, we fixed the value of each covariate at its sample  mode or mean, depending on whether the variable was categorical or  continuous. For this profile of covariate values, we calculated survival  probabilities at various times using the estimated parameters of the  unstratified model, and transformed them to failure probabilities by  subtracting the survival estimates from 1.
We estimated a final version of the model that used neither office strata  nor fixed effects. This allowed us to estimate the association between  office workload, measured by the total number of IL plans developed from  fiscal year 2008 through 2011, and rehabilitation timeliness. Stratifying  the model by office or estimating fixed effects would have absorbed the  variation needed to estimate the coefficient on workload, as an office- level covariate.
section_title: Results
paragraphs: Table 3 provides estimates of 𝛽 and hazard ratios derived from them,  along with standard errors and p values for the hypothesis that each ratio  equals 1. These estimates come from the stratified model above. Figure  17 presents the failure functions implied by the separate hazard functions  the model assumes for each regional office. These results are generally  consistent with those discussed in the body of this report, where we  interpret the results in more detail.
section_title: Appendix III: List of Goods and Services Provided to Sample of Independent Living Track Veterans from our Case File Review
paragraphs: The table below lists the goods and services as they were recorded in the  182 cases we reviewed in the CWINRS system.
Goods and Services Provided  Counseling Services  “(1) Specialized Contractial Counseling Services 637”  “(1) Specialized Contractual Counseling”  “(1) Specialized Contractual Counseling Services 637, Vocational, Educational, Psychological or Personal adjustment  counseling which I”  “(1) Specialized Independent Living Services 638”  “(1) Specialized Rehabilitation Service 631”  “(1) Specialized Rehabilitation Services”  “(1) Specialized Rehabilitation Services 631, Specialized services involving unique skills and techniques, such as  learning di”  “(1) Specialized contractual counseling services 637; IILP Initial Assessment”  “(316) Contract Counseling Assistance 451”  “(343) Rehab Services 556, All services not otherwise described to advance the objectives of the rehabilita”  “(343) Rehab Services 556; Connections Vocational Services; IILP Evals; All service not otherwise described to  advance the objectives of the rehabilita”  “(372) Independent Living Services 644, Provide Independent Living services (i.e. IL Evaluation, Occupational Therapy  Ev”  “(372) Psychotherapy 667; provide psychotherapy and adjustment counseling”  “Rehabilitation Counseling Services 158, Specialized counseling services as required to assist and support veteran in  ful”  “SRS/IL Special Rehab Service/Independen Li”  “SRS/V Special Rehab Service/Vocaational; Center for Independent Living; IL Case Mgmt” “SRS/V Special Rehab Service/Vocational”  “Special Rehab Service/Independent Living”  “Special Rehab. Srvs/Indepen. Living 24”  “Special rehab service/Independent Living”
subsections: section_title: Goods and Services Provided Medical Care and Health Related Services “Cora Coleman Senior Center; 9 month swim membership”
section_title: “SRS/M Special Rehab Service/Medical”
paragraphs: “VA MEDICAL CENTER - ANN ARBOR”  “YMCA East Belleville Center; YMCA Membership; YMCA annual fee”
Residential Adaption and Repair  “(317) Independent Living Residence Adaptation 645”  “(317) Independent Living Residence Adaptation 645, Residential adaptations to allow a veteran to complete ILP.”  “A&E services for home modification per ILP”  “Basic construction as approved on IL plan”  “Ch 31 One Time Miscellaneous Items Purchase—Reconstruction of front porch per IL plan”  “Gould’s Discount Medical — Home Improvements”  “Herrera Engineering; house modifications; special contractual services”  “Home Reab Equipment — Supply and set up bath mods (up)”  “Home Rehab Equipment — custom railing”
section_title: Goods and Services Provided “Home safety evaluation”
section_title: “Veterans Assistance Network; wood handrails; 18 in grab bars; installation grab bar”
paragraphs: “buyrailings.com; stair railing w/brackets”  “smoke detectors/fire extinguisher/co2 detector”  “IHL Institutions of Higher Learning – for period 2/5/08-7/23/08”  “IHL Institutions of Higher Learning; 24 training session”  “IHL Institutions of Higher Learning (College books)”  “IHL Institutions of Higher Learning; Central Community College-Hastings Degree”  “Individual/Independent Instruction / Trg 164”
Jewish Vocational Services; non college degree  “Kendall & Davis, Inc.; computer training”  “Mastering MS Office Made Easy (CD-ROM version)”  “OJT On the Job Training/Apprenticeship”
1  “1-year audio book subscription (2 CDs/month)”  “All Temperature system; IL- Heating System”  “CPI*Contour living web; wedge cushions/with massage; shipping charge”  “Ch 31 non-Contractual Speicial . . . \r\n”
Computer Evaluation Services 21, Technical evaluation for computer hardware and software needs.  “Installation of Camera Security System”  “KID 1240 talking dictionary with headphones”  “King size adjustable bed with massage”  “Pride GL 358M lift chair w/set up and delivery”  “Sports authority; Treadmill, delivery & assembly, 4 yr warranty”
Goods and Services Provided  “US Computers Inc; Shipping/handling fees for computer and furnitures”  “set and instruct on scooter lift with swing arm.”
Adaptive Equipment  “All in one accessibility - adaptive equipment”  “ALI*AMIMED INC.; Backrest air obusforme grey; shipping charge”  “Alex Orthopedic; Walking Cane; Shipping charge”  “COLDERS, INC.; Bed & Chair - Independent Living”  “Colder’s; TempurPedic King Size Solution Mattress and Box Sp; HTC-1650 Massage Chair”  “Deluxe swivel seat cushion, UpEasy lifting cushion seat assist (plus shipping)”  “Glide rail & shower wand,PT-WR28R,LABOR”  “Grab bar specialists, Inc.; handi-grip portable grab bar and shipping”  “IL Adaptive Equipment \r\nCh 31 Special Equipment”
Goods and Services Provided  “IL Adaptive equipment, Shipping and Handling\r\nCh 31 Special Equipment”  “Lamp for independent living and surge protector”  “MAXIAUDS; CO2 Alarm; shiping charge”  “Medication dispenser, phone, printer, mouse”  “Mother Earth Design INC.; Trigger point pillow; shipping charge”  “Ontime supplies; ergonomic chair, dry erase board”  “Perching Stool, Height Adjustable, 250lb Capacity”  “Portable Lift Cushion, 250 1b Capacity”  “Queen Matt, Box, Tempura Symphony pillow”  “Rehab Engineering Inc.; Large Handle cups; walker, 3-inch wheels, 300lbs. capacity; 24-inch grab bar, inside tub”  “Rolling laundry cart & ergonomic grabber”  “Sliding Shelves (for Bathroom Cabinet)”  “Special equipment — one-handed jar opener, suction cup plates, suction cup bowls”  “Tub safety bar and lever door handles (QTY 6)”  “Vocational Resource Services; good grips eating utensils; sock assist; elastic shoelaces”  “Vocational Resource Services; locking elevated toilet seat w/ arms; special equipment”  “Walker Furniture; mattress, frame and bed cover”
section_title: Goods and Services Provided “desk chair”
section_title: Computer and Camera Equipment “Adaptive computer system”
section_title: “Dell - desktop, surge protector, and printer”
section_title: Goods and Services Provided “Desktop computer system”
section_title: “XPS One, Red—Dell Desk Top computer”
paragraphs: “computer w/19 inch monitor, 4n1 printer + internet”
section_title: Goods and Services Provided “desktop computer system”
section_title: “12 inch dualbevel slide mitersaw 77-141-614 Bosc”
paragraphs: “2hp soft start fixed and plunge base router”  “Craftsman fixed base router/table combo”  “Embroidery Machine & Accessories, Classes”  “Jarvi Facetron; Facitron Machine with grit; shipping”  “Rikon 18 inch bandsaw, excess weight charge”  “Searrs; rts bit saw nailer plnr sander warrenties”  “Tracrac T3B portable miter saw sta Bosch”  “drill press, protection agreement, shipping”  “The Rock Shed; Thumlers tumbler; 5lbs 60/90 grit; 5lbs 120/220 grit”  “Accessory Kit, Bench Clamp and shipping for workbench”
section_title: Goods and Services Provided “Cookbooks”
section_title: Other Goods and Services “Ch 31 Books”
section_title: “Merry Maids; maid services; fee, intial; Refrigerator cleaning”
paragraphs: Goods and Services Provided  “Shipping/delivery charges for CD-ROM”  “Voc Rehab Panel, PO 348-08-039 dtd 7/23/08”  “McGavic Outdoot power equipment; lawn tractor and delivery”
section_title: Appendix IV: Rehabilitation Rates by Regional Office, FY 2008 Independent Living Track Entrants
subsections: section_title: Entrants with One IL Plan 2274
section_title: Entrants with One IL Plan 79
paragraphs: Transfers to a VR&E  Employment Track  4  n.a.  n.a.
The Denver Regional Office includes totals for the Cheyenne, Wyoming office. This office is  considered a satellite office of Denver.
The Washington Regional Office, located in the District of Columbia, was renamed the National  Capital Region Benefits Office in November 2012.
section_title: Appendix V: Days to Complete Independent Living Plans, Veterans Entering the Independent Living Track During FY 2008
subsections: section_title: Regional Office Roanoke
section_title: Regional Office Atlanta
paragraphs: The Denver Regional Office includes totals for the Cheyenne, Wyoming office. This office is  considered a satellite office of Denver.
section_title: Appendix VI: Independent Living Track Caseload Nationwide and by Regional Office, FY 2008-2011
paragraphs: The IL entrant total for the Denver Regional Office includes the total number of veterans who  entered the Cheyenne, Wyoming, office. This office is considered a satellite office of Denver.  The Washington Regional Office, located in the District of Columbia, was renamed the National  Capital Region Benefits Office in November 2012.
section_title: Appendix VII: Comments from the Department of Veteran Affairs
section_title: Appendix VIII: GAO Contacts and Staff Acknowledgments
subsections: section_title: Contact
paragraphs: Daniel Bertoni, (202) 512-7215 or bertonid@gao.gov.
section_title: Acknowledgments
paragraphs: In addition to the contact listed above, individuals making key  contributions to this report were Clarita Mrena (Assistant Director), James  Bennett, Melinda Bowman, David Chrisinger, Mary Ann Curran, David  Forgosh, Danielle Giese, Angela Jacobs, Mitch Karpman, John Lack,  Kirsten B. Lauber, Karen O’Conor, James Rebbe, Martin Scire, Almeta  Spencer, Jeff Tessin, Jack Warner, and Ashanta Williams.
section_title: Related GAO Products
paragraphs: Veterans’ Employment and Training: Better Targeting, Coordinating, and  Reporting Needed to Enhance Program Effectiveness. GAO-13-29.  Washington, D.C.: December 13, 2012.
VA Health Care: Spending for and Provision of Prosthetic Items.   GAO-10-935. Washington, D.C.: September 30, 2010.
Veteran Affairs: Opportunities Exist to Improve Potential Recipients’  Awareness of the Temporary Residential Adaption Grant. GAO-10-786.  Washington, D.C.: July 15, 2010.
Veterans Affairs: Implementation of Temporary Residence Adaptation  Grants. GAO-09-637R. Washington, D.C.: June 15, 2009.
Federal Information System Controls Audit Manual (FISCAM).   GAO-09-232G. Washington, D.C.: February 2009.
VA Vocational Rehabilitation and Employment: Better Incentives,  Workforce Planning, and Performance Reporting Could Improve  Program. GAO-09-34. Washington, D.C.: January 26, 2009.
VA Vocational Rehabilitation and Employment: Service Contract  Management Is Improving, but Challenges Remain. GAO-07-568R.  Washington, D.C.: April 23, 2007.
Managerial Cost Accounting Practices: Leadership and Internal Controls  Are Key to Successful Implementation. GAO-05-1013R. Washington,  D.C.: September 2, 2005.
VA Vocational Rehabilitation and Employment Program: GAO Comments  on Key Task Force Findings and Recommendations. GAO-04-853.  Washington, D.C.: June 15, 2004.
Internal Controls Management and Evaluation Tool. GAO-01-1008G.  Washington, D.C.: August 2001.
Managing For Results: Barriers to Interagency Coordination.   GAO/GGD-00-106. Washington, D.C.: March 29, 2000.
Standards for Internal Control in the Federal Government.   GAO/AIMD-00-21.3.1. Washington, D.C.: November 1999.