Military Disability System: Preliminary Observations on Efforts to Improve Performance
Published: 2012-05-23T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Since 2007, the DOD and VA have operated the IDESwhich combines what used to be separate DOD and VA disability evaluation processes and is intended to expedite benefits for injured servicemembers. Initially a pilot at 3 military treatment facilities, IDES is now DODs standard process for evaluating servicemembers fitness for duty and disability worldwide. In previous reports, GAO identified a number of challenges as IDES expanded, including staffing shortages and difficulty meeting timeliness goals.
In this statement, GAO discusses initial observations from its ongoing review of the IDES, addressing two key topics: (1) the extent to which DOD and VA are meeting IDES timeliness and servicemember satisfaction performance goals, and (2) steps the agencies are taking to improve the performance of the system. To answer these questions, GAO analyzed IDES timeliness and customer satisfaction survey data, visited six IDES sites, and interviewed DOD and VA officials. This work is ongoing and GAO has no recommendations at this time. GAO plans to issue its final report later in 2012.
section_title: What GAO Found
paragraphs: Case processing times under the Integrated Disability Evaluation System (IDES) process have increased over time, and measures of servicemember satisfaction have shortcomings. Each year, average processing time for IDES cases has climbed, reaching 394 and 420 days for active and reserve component members in fiscal year 2011well over established goals of 295 and 305 days, respectively. Also in fiscal year 2011, just 19 percent of active duty servicemembers and 18 percent of guard or reserve members completed the IDES process and received benefits within established goals, down from 32 and 37 percent one year prior. Of the four phases comprising IDES, the medical evaluation board phase increasingly fell short of timeliness goals and, within that phase, the time required for the militarys determination of fitness was especially troubling. During site visits to IDES locations, we consistently heard concerns about timeframes and resources for this phase of the process. With respect to servicemember satisfaction with the IDES process, GAO found shortcomings in how these data are collected and reported, such as unduly limiting who is eligible to receive a survey and computing average satisfaction scores in a manner that may overstate satisfaction. Department of Defense (DOD) officials told us they are considering alternatives for gauging satisfaction with the process.
DOD and Veterans Affairs (VA) have taken steps to improve IDES performance, and have other improvement initiatives in process, but progress is uneven and it is too early to assess their overall impact. VA increased resources for conducting disability ratings and related workloads. The Army is hiring additional staff for its medical evaluation boards, but it is too early to see the impact of these additional resources. DOD and VA are pursuing system upgrades so that staff and managers at IDES facilities can better track the progress of servicemembers cases and respond to delays more quickly; however, multiple upgrades may be causing redundant work efforts. DOD officials also told us they have been working with the military services to correct case data that were inaccurately entered into VAs IDES tracking system, but have not yet achieved a permanent solution. Finally, DOD is in the early stages of conducting an in-depth business process review of the entire IDES process and supporting IT systems, in order to better understand how each step contributes to overall processing times and identify opportunities to streamline the process and supporting systems.

section_title: Letter
paragraphs: I am pleased to be here today to discuss our preliminary observations on  the efforts of the Departments of Defense (DOD) and Veterans Affairs  (VA) to integrate their disability evaluation systems. DOD and VA began  piloting the Integrated Disability Evaluation System (IDES) in 2007 in  response to concerns that wounded, ill, or injured servicemembers had to  undergo two separate and complex disability assessments, and in order  to expedite the delivery of benefits to servicemembers. As of October 1,  2011, IDES had replaced the military services’ existing—or “legacy”— disability evaluation systems for almost all new disability cases. GAO has  monitored the evolution of IDES since its pilot phase and our past work  highlighted a number of challenges. For instance, we reported in  December 2010 that insufficient staff and logistical challenges contributed  to delays in completing IDES cases, and recommended the agencies take  steps to ensure adequate staffing levels and develop a systematic  process for monitoring caseloads.
VTA is a VA computer system that is used to track, among other things, the dates at  which servicemembers complete the different stages of IDES. in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The IDES process begins at a military treatment facility when a physician  identifies one or more medical conditions that may interfere with a  servicemember’s ability to perform his or her duties. The process involves  four main phases: the Medical Evaluation Board (MEB), the Physical  Evaluation Board (PEB), transition out of military service (transition), and  VA benefits.
MEB phase: In this phase, medical examinations are conducted and  decisions are made by the MEB regarding a servicemember’s ability to  continue to serve in the military. This phase involves four stages: (1) the  servicemember is counseled by a DOD board liaison on what to expect  during the IDES process; (2) the servicemember is counseled by a VA  caseworker on what to expect during the IDES process and medical  exams are scheduled; (3) medical exams are conducted according to VA  standards for exams for disability compensation, by VA, DOD, or  contractor physicians; and (4) exam results are used by the MEB to  identify conditions that limit the servicemember’s ability to serve in the  military. Also during this stage, a servicemember dissatisfied with the  MEB assessment of unfitting conditions can seek a rebuttal, or an  informal medical review by a physician not on the MEB, or both.
PEB phase: In this subsequent phase, decisions are made about the  servicemember’s fitness for duty, disability rating and DOD and VA  disability benefits, and the servicemember has opportunities to appeal  those decisions. This includes: (1) the informal PEB stage, an  administrative review of the case file by the cognizant military branch’s  PEB without the presence of the servicemember; (2) VA rating stage,  where a VA rating specialist prepares two ratings—one for the conditions that DOD determine made a servicemember unfit for duty, which DOD  uses to provide military disability benefits, and the other for all service- connected disabilities, which VA uses to determine VA benefits. In  addition, the servicemember has several opportunities to appeal different  aspects of their disability evaluations: a servicemember dissatisfied with  the decision on whether he or she is fit for duty may request a hearing  with a “formal” PEB; a member who disagrees with the formal PEB fitness  decision can, under certain conditions, appeal to the reviewing authority  of the PEB; and a servicemember can ask for VA to reconsider its  ratings decisions based on additional evidence, though only for conditions  found to render the servicemember unfit for duty.
Transition phase: If the servicemember is found unfit to serve, he or she  enters the transition phase and begins the process of separating from the  military. During this time, the servicemember may take accrued leave.  Also, DOD board liaisons and VA case managers provide counseling on  available benefits and services, such as job assistance.
VA benefits phase: A servicemember found unfit and separated from  service becomes a veteran and enters the VA benefits phase. VA  finalizes its disability rating after receiving evidence of the  servicemember’s date of separation from military service. VA then starts  to award monthly disability compensation to the veteran.
DOD and VA established timeliness goals for the IDES process to provide  VA benefits to active duty servicemembers within 295 days of being  referred into the process, and to reserve component members within 305  days (see fig. 1). DOD and VA also established interim timeliness goals  for each phase and stage of the IDES process. These time frames are an  improvement over the legacy disability evaluation system, which was  estimated to take 540 days to complete. In addition to timeliness, DOD  surveys servicemembers on their satisfaction at several points in the  process, with a goal of having 80 percent of servicemembers satisfied.
Enrollment in IDES continued to grow as IDES completed its worldwide  expansion. In fiscal year 2011, 18,651 cases were enrolled in IDES  compared to 4,155 in fiscal year 2009 (see fig 2). IDES caseload varies  by service, but the Army manages the bulk of cases, accounting for 64  percent of new cases in fiscal year 2011. Additionally, active duty  servicemembers represent the majority of IDES cases, accounting for 88  percent of new cases in fiscal year 2011.
section_title: IDES Processing Times Increased over Time, While Measures of Servicemember Satisfaction Have Shortcomings
subsections: section_title: Overall IDES Case- Processing Times Steadily Increased Since the Start of IDES
paragraphs: Overall IDES timeliness has steadily worsened since the inception of the  program. Since fiscal year 2008, the average number of days for  servicemembers cases to be processed and to receive benefits increased  from 283 to 394 for active duty cases (compared to the goal of 295 days)  and from 297 to 420 for reserve cases (compared to the goal of 305 days).  Relatedly, the proportion of cases meeting timeliness goals decreased from  more than 63 percent of active duty cases completed during fiscal year 2008  to about 19 percent in fiscal year 2011 (see table 1).
When examining timeliness across the four phases that make up IDES,  data show that timeliness regularly fell short of interim goals for three— MEB, Transition, and VA Benefits (see fig. 3). For example, for cases that  completed the MEB phase in fiscal year 2011, active and reserve  component members’ cases took on average of 181 and 188 days  respectively to be processed, compared to goals of 100 and 140 days.  For the PEB phase, processing times increased over time, but were still  within established goals.
MEB phase: Significant delays have been occurring in completing  medical examinations (medical exam stage) and delivering an MEB  decision (the MEB stage). For cases completing the MEB phase in 2011,  31 percent of active and 29 percent of reserve cases met the 45-day goal  for the medical exam stage and 20 percent of active case and 17 percent  of reserve cases met the 35-day goal for the MEB stage. Officials at some  sites we visited told us that MEB phase goals were difficult to meet and  not realistic given current resources. At all the facilities we visited, officials  told us DOD board liaisons and VA case managers had large case loads.  Similarly, some military officials noted that they did not have sufficient  numbers of doctors to write the narrative summaries needed to complete  the MEB stage in a timely manner. Monthly data produced by DOD  subsequent to the data we analyzed show signs of improved timeliness  for these two stages: for example, 71 percent of active cases met the goal  for the medical exam stage and 43 percent met the goal for the MEB  stage in the month of March 2012. However, it is too early to tell the  extent to which these results will continue to hold.
PEB phase: PEB processing times goals were also not met in fiscal year  2011 for the informal PEB and VA rating stages. For cases that complete  the PEB phase in fiscal year 2011, only 38 percent of active duty cases  received an informal PEB decision within the 15 days allotted, and only  32 percent received a preliminary VA rating within the 15-day goal. Also  during this phase, the majority of time (75 out of the 120 days) is set  aside for servicemembers to appeal decisions—including a formal PEB  hearing or a reconsideration of the VA ratings. However, only 20 percent  of cases completed in fiscal year 2011 actually had any appeals; calling  into question DOD and VA’s assumption on the number of expected  appeals and potentially masking processing delays in other mandatory  parts of the PEB phase.
Transition phase: The transition phase has consistently taken longer than  its 45-day goal—almost twice as long on average. While processing times  improved slightly for cases that completed this phase in fiscal year 2011  (from 79 days in 2010 to 76 days in fiscal year 2011 for active duty  cases), timeliness has remained consistently problematic since fiscal year  2008. DOD officials suggested that it is difficult to meet the goal for this  phase because servicemembers are taking accrued leave—to which they  are entitled—before separating from the service. For example, an Army  official said that Army policy allows servicemembers to take up to 90 days  of accrued leave prior to separating, and that average leave time was  about 80 days. Although servicemember leave is skewing the  performance data, officials said that they cannot easily back this time out  from their tracking system, but are exploring options for doing so, which  would be more reflective of a servicemember’s actual total time spent in  the evaluation process.
VA benefits phase: Processing time improved somewhat for the benefits  phase (48 days in fiscal year 2010 to 38 days in fiscal year 2011), but  continued to exceed the 30-day goal for active duty servicemembers.
Several factors may contribute to delays in this final phase. VA officials  told us that cases cannot be closed without the proper discharge forms  and that obtaining these forms from the military services can sometimes  be a challenge. Additionally, if data are missing from the IDES tracking  system (e.g., the servicemember already separated, but this was not  recorded in the database), processing time will continue to accrue for  cases that remain open in the system. Officials could not provide data on  the extent to which these factors had an impact on processing times for  pending cases, but said that once errors are detected and addressed,  reported processing times are also corrected.
section_title: Shortcomings in the Design and Administration of Servicemember Survey
paragraphs: In addition to timeliness, DOD and VA evaluate IDES performance using  the results of servicemember satisfaction surveys. However,  shortcomings in how DOD measures and reports satisfaction limit the  usefulness of these data for making IDES management decisions.
Response rates: Survey administration rules may unnecessarily  exclude the views of some servicemembers. In principle, all members  have an opportunity to complete satisfaction surveys at the end of the  MEB, PEB, and transition phases; however, servicemembers become  ineligible to complete a survey for either the PEB or transition phases  if they did not complete a survey in an earlier phase. Additionally, by  only surveying servicemembers who completed a phase, DOD may  be missing opportunities to obtain input from servicemembers who  exit IDES in the middle of a phase.
Alternate measure shows lower satisfaction: DOD’s satisfaction  measure is based on an average of responses to questions across  satisfaction surveys. A servicemember is defined as satisfied if the  average of his or her responses is above 3 on a 5-point scale, with 3  denoting neither satisfied nor dissatisfied. Using an alternate measure  that defines servicemembers as satisfied only when all of their  responses are 4 or above, GAO found satisfaction rates several  times lower than DOD’s calculation. Whereas DOD’s calculation  results in an overall satisfaction rate of about 67 percent since the  inception of IDES, GAO’s alternate calculation resulted in a  satisfaction rate of about 24 percent. In our ongoing work, we will  continue to analyze variation in satisfaction across servicemember  cases using both DOD’s and GAO’s measures of satisfaction.
In our ongoing work, we will continue to assess survey results and their  usefulness for measuring performance. In the meantime, DOD is  reconsidering alternatives for measuring satisfaction, but has yet to come  to a decision. Officials already concluded that the survey, in its current  form, is not a useful management tool for determining what changes are  needed in IDES and said that it is expensive to administer—costing  approximately $4.3 million in total since the start of the IDES pilot. DOD  suspended the survey in December 2011 because of financial  constraints, but officials told us they plan to resume collecting satisfaction  data in fiscal year 2013.
section_title: Recent Actions and Ongoing Initiatives May Improve IDES Performance, but It Is Too Early to Assess Their Overall Impact
subsections: section_title: DOD and VA Took Steps to Address Previously Identified IDES Challenges
paragraphs: DOD and VA have undertaken a number of actions to address IDES  challenges—many of which GAO identified in past work. Some actions— such as increased oversight and staffing—represent important steps in  the right direction, but progress is uneven in some areas.
Increased monitoring and oversight: GAO identified the need for  agency leadership to provide continuous oversight of IDES in 2008,  and reported the need for system-wide monitoring mechanisms in  2010. Since then, agency leadership has established mechanisms to  improve communication, monitoring, and accountability. The  secretaries of DOD and VA have met several times since February  2011 to discuss progress in improving IDES timeliness and have  tasked their agencies to find ways of streamlining the process so that  the goals can be reduced. Further, senior Army and Navy officials  regularly hold conferences to assess performance and address  performance issues, including at specific facilities. For instance, the  Army’s meetings are led by its vice-chief of staff and VA’s chief of  staff, and include reviews of performance where regional and local  facility commanders provide feedback on best practices and  challenges. Further, VA holds its own biweekly conferences with local  staff responsible for VA’s portion of the process. For example, officials  said a recent conference addressed delays at one Army IDES site  and discussed how they could be addressed. VA officials noted that  examiner staff were reassigned to this site and examiners worked on  weekends to address the exam problems at this site.
Increased staffing for MEB and VA rating: In 2010, we identified  challenges with having sufficient staff in a number of key positions,  including DOD board liaisons and MEB physicians. DOD and VA are  working to address staffing challenges in some of the IDES processes  that are most delayed. The Army is in the midst of a major hiring  initiative to more than double staffing for its MEBs over its October  2011 level, which will include additional board liaison and MEB  physician positions. The Army also plans to hire contact  representatives to assist board liaisons with clerical functions, freeing  more of the liaisons’ time for counseling servicemembers.  Additionally, VA officials said that the agency has more than tripled  the staffing of its IDES rating sites to handle the demand for  preliminary ratings, rating reconsiderations, and final benefit  decisions.
Resolving diagnostic differences: In our December 2010 report, we  identified differences between DOD physicians and VA examiners,  especially regarding mental health conditions, as a potential source of  delay in IDES. We also noted inconsistencies among services in  providing guidance and a lack of a tracking mechanism for  determining the extent of diagnostic differences. In response to our  recommendation, DOD commissioned a study on the subject. The  resulting report confirmed the lack of data on the extent and nature of  such differences, and that the Army has established guidance more  comprehensive than guidance DOD was developing on how to  address diagnostic differences, and recommended that DOD or the  other services develop similar guidance. A DOD official told us that  consistent guidance across the services, similar to the Army’s, was  included in DOD’s December 2011 IDES manual. Also, in response to  our recommendation, VA plans to modify the VTA database used to  track IDES to collect this information on cases, although the upgrade  has been delayed several times.
DOD has other actions underway, including efforts to improve sufficiency  of VA examinations, MEB written summaries and reserve component  records. We plan to review the status of these efforts as part of our  ongoing work, which we anticipate completing later in 2012.
section_title: DOD and VA Are Working on Shortcomings in Information Systems, but Efforts to Date Are Limited
paragraphs: DOD and VA are working to address shortcomings in information systems  that support the IDES process, although some efforts are still in progress  and efforts to date are limited.
Improving local IDES reporting capability: DOD and VA are  implementing solutions to improve the ability of local military treatment  facilities to track their IDES cases, but multiple solutions may result in  redundant work efforts. Officials told us that the VTA—which is the  primary means of tracking the completion of IDES cases—has limited  reporting capabilities and staff at local facilities are unable to use it for  monitoring the cases for which they are responsible. DOD and VA  have been developing improvements to VTA that will allow board  liaisons and VA case managers to track the status of their cases. VA  plans to include these improvements in the next VTA upgrade,  currently scheduled for June 2012. In the meantime, staff at many  IDES sites have been using their own local systems to track cases  and alleviate limitations in VTA. Further, the military services have  been moving ahead with their own solutions. For instance, the Army  has deployed its own information system for MEBs and PEBs Army- wide. Meanwhile, DOD has also been piloting its own tracking system  at 9 IDES sites. As a result, staff at IDES sites we visited reported  having to enter the same data into multiple systems.
Improving IDES data quality: DOD is taking steps to improve the  quality of data in VTA. Our analysis of VTA data identified erroneous  or missing dates in at least 4 percent of the cases reviewed. Officials  told us that VTA lacks adequate controls to prevent erroneous data  entry, and that incorrect dates may be entered, or dates may not be  entered at all, which can result in inaccurate timeliness data. In  September 2011, DOD began a focused effort with the services to  correct erroneous and missing case data in VTA. Officials noted that  the Air Force and Navy completed substantial efforts to correct the  issues identified at that time, but Army efforts continue. While  improved local tracking and reporting capabilities will help facilities  identify and correct erroneous data, keeping VTA data accurate will  be an ongoing challenge due to a lack of data entry controls.
DOD and VA are also pursuing options to allow the electronic transfer of  case files between facilities. We are reviewing the status of this effort as  part of our ongoing work.
section_title: DOD and VA are Pursuing Broader Solutions to Improve IDES Performance
paragraphs: Based on concerns from the agencies’ secretaries about IDES delays,  DOD and VA have undertaken initiatives to achieve time savings for  servicemembers. The agencies have begun a business process review to  better understand how IDES is operating and identify best practices for  possible piloting. This review incorporates several efforts, including,
Process simulation model: Using data from site visits and VTA, DOD  is developing a simulation model of the IDES process. According to a  DOD official, this process model will allow the agencies to assess the  impact of potential situations or changes on IDES processing times,  such as surges in workloads or changes in staffing.
Fusion diagram: DOD is developing this diagram to identify the  various sources of IDES data—including VA claim forms and narrative  summaries—and different information technology systems that play a  role in supporting the IDES process. Officials said this diagram would  allow them to better understand and identify overlaps and gaps in  data systems.
Ultimately, according to DOD officials, this business process review could  lead to short- and long-term recommendations to improve IDES  performance, potentially including changes to the different steps in the  IDES process, performance goals, and staffing levels; and possibly the  procurement of a new information system to support process  improvements. However, a DOD official noted that these efforts are in  their early stages, and thus there is no timetable yet for completing the  review or providing recommendations to senior DOD and VA leadership.
section_title: Concluding Observations
paragraphs: By merging two duplicative disability evaluation systems, IDES has  shown promise for expediting the delivery of DOD and VA benefits to  injured servicemembers and is considered by many to be an  improvement over the legacy process it replaced. However, nearly 5  years after its inception as a pilot, delays continue to affect the system  and their causes are not yet fully understood. Recent initiatives to better  understand factors that lead to delays and remedy them are promising,  however it remains to be seen what their effect will be. Given the  persistent nature of IDES performance challenges, continued attention  from senior agency leadership will be critical to ensure that delays are  understood and remedied.
We have draft recommendations aimed at helping DOD and VA further  address challenges we identified, which we plan to finalize in our  forthcoming report after fully considering both DOD and VA’s comments.
Chairman Murray and Ranking Member Burr, this concludes my prepared  statement. I would be pleased to respond to any questions that you or  other Members of the Committee may have at this time.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: For further information about this testimony, please contact Daniel Bertoni  at (202) 512-7215 or bertonid@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this statement. In addition to the individual named above, key  contributors to this statement include Michele Grgich, Daniel Concepcion,  Melissa Jaynes, and Greg Whitney. James Bennett, Joanna Chan,  Douglas Sloane, Vanessa Taylor, Jeff Tessin, Roger Thomas, Walter  Vance, Kathleen van Gelder, and Sonya Vartivarian provided key support.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.