Military Disability System: Improved Oversight Needed to Ensure Consistent and Timely Outcomes for Reserve and Active Duty Service Members
Released: 2006-03-31T00:00:00

Letter

Background

As provided by the Career Compensation Act of 1949, as amended, service members who become physically unfit to perform military duties may receive military disability compensation under certain conditions.

Compensation for disabilities can be in the form of monthly disability retirement benefits or a lump sum disability severance payment, depending on the disability rating and years of creditable service. To qualify for monthly disability retirement benefits, a service member with a permanent impairment that renders him or her unfit for duty must have (1) at least 20 years of creditable service or (2) a disability rating of at least 30 percent. Service members with less than 20 years of creditable service and a disability rating less than 30 percent receive a lump sum severance disability payment.

Service members with service connected disabilities may also be eligible for VA disability compensation. Until recently, this military benefit was offset by any VA compensation received. However, the fiscal year 2004 National Defense Authorization Act now allows some military retirees to concurrently receive VA and military benefits. Generally, military disability retirement pay is taxable. Exceptions are (1) if the disability pay is for combat-related injuries or (2) if the service member was in the military, or so obligated, on September 24, 1975.

The Disability Evaluation Process

Each of the military services administers its own disability evaluation process. According to DOD regulations, the process should include a medical evaluation board (MEB), a physical evaluation board (PEB), an appellate review process, and a final disposition. Each service member should be assigned a Physical Evaluation Board Liaison Officer (PEBLO), a counselor to help the service member navigate the system and prepare documents for the PEB.

As shown in figure 1, there are a number of steps in the disability evaluation process and several factors that play a role in the decisions that are made at each step. There are four possible outcomes in the disability evaluation system. A service member can be found fit for duty; separated from the service without benefits—service members whose disabilities were incurred while not on duty or as a result of intentional misconduct are discharged from the service without disability benefits; separated from the service with lump sum disability severance pay; or retired from the service with permanent monthly disability benefits or placed on the temporary disability retired list (TDRL).

Medical Evaluation Board

The disability evaluation process begins at a military treatment facility (MTF), when a physician identifies a condition that may interfere with a service member’s ability to perform his or her duties. The physician prepares a narrative summary detailing the injury or condition. DOD policy establishes the date of dictation of the narrative summary as the beginning of the disability evaluation process. This specific type of medical evaluation is for the purpose of determining if the service member meets the military’s retention standards, according to each service’s regulations. This process is often referred to as a medical evaluation board (MEB). Service members who meet retention standards are returned to duty, and those who do not are referred to the physical evaluation board (PEB).

Physical Evaluation Board

The PEB is responsible for determining whether service members have lost the ability to perform their assigned military duties due to injury or illness, which is referred to as being “unfit for duty”. If the member is found unfit, the PEB must then determine whether the condition was incurred or permanently aggravated as a result of military service. While the composition of the PEB varies by service, it is typically composed of one or more physicians and one or more line officers. Each of the services conducts this process for its service members. The Army has three PEBs located at Fort Sam Houston, Texas; Walter Reed Army Medical Center in Washington, D.C.; and Fort Lewis, Washington. The Navy has one located at the Washington Navy Yard in Washington, D.C. The Air Force has one located in San Antonio, Texas.

The first step in the PEB process is the informal PEB—an administrative review of the case file without the presence of the service member. The PEB makes the following findings and recommendations regarding possible entitlement for disability benefits: Fitness for duty—The PEB determines whether or not the service member “is unable to reasonably perform the duties of his or her office, grade, rank, or rating,” taking into consideration the requirements of a member’s current specialty. Fitness determinations are made on each medical condition presented. Only those medical conditions which result in the finding of “unfit for continued military service” will potentially be compensated. Service members found fit must return to duty.

Compensability—The PEB determines if the service member’s injuries or conditions are compensable, considering whether they existed prior to service (referred to as having a pre-existing condition) and whether they were incurred or permanently aggravated in the line of duty. Service members found unfit with noncompensable conditions are separated without disability benefits.

Disability rating—When the PEB finds the service members unfit and their disabilities are compensable, it applies the medical criteria defined in the Veterans Administration Schedule for Rating Disabilities (VASRD) to assign a disability rating to each compensable condition. The PEB then determines (or calculates) the service member’s overall degree of service connected disability. Disability ratings range from 0 (least severe) to 100 percent (most severe) in increments of 10 percent. Depending on the overall disability rating and number of years of active duty or equivalent service, the service member found unfit with compensable conditions is entitled to either monthly disability retirement benefits or lump sum disability severance pay.

In disability retirement cases, the PEB considers the stability of the condition. Unstable conditions are those for which the severity might change resulting in higher or lower disability ratings. Service members with unstable conditions are placed on TDRL for periodic PEB reevaluation at least every 18 months. While on TDRL, members receive monthly retirement benefits. When members on TDRL are determined to be fit for duty, they may choose to return to duty or leave the military at that time. Members who continue to be unfit for duty after 5 years on TDRL are separated from the military with monthly retirement benefits, discharged with severance pay, or discharged without benefits, depending on their condition and years of service.

Service members have the opportunity to review the informal PEB’s findings and may request a formal hearing with the PEB; however, only those found unfit are guaranteed a formal hearing. The formal PEB conducts a de novo review of referred cases and renders its own decisions based upon the evidence. At the formal PEB hearing, service members can appear before the board, put forth evidence, introduce and question witnesses, and have legal counsel help prepare their cases and represent them. The military will provide military counsel or service members may retain their own representative. If service members disagree with the formal PEB’s findings and recommendations, they can, under certain conditions, appeal to the reviewing authority of the PEB. Once the service member either agrees with the PEB’s findings and recommendations or exhausts all available appeals, the reviewing authority issues a final disability determination concerning fitness for duty, disability rating, and entitlement to benefits.

Disability Evaluation System Caseloads

In 2005, over 23,000 U.S. service members with physical injuries or other conditions went through the military disability evaluation system, according to DOD. In total, the Army, Navy, and Air Force report evaluating over 90,000 PEB cases during the fiscal years 2001 to 2005. The Army represents the largest share of disability cases, with Army reserve component members representing approximately 32 percent of all Army cases in 2005 (see table 1). PEB disability caseloads for all services have increased over time from about 15,000 in fiscal year 2002 to about 23,000 in fiscal year 2005.

Military Disability Retirement Expenditures

In fiscal year 2004, the military services spent over $1 billion in disability retirement benefits for over 90,000 service members. See table 2. This table does not include expenditures for lump sum disability payments which DOD was unable to provide.

Oversight

The Secretary of Defense oversees the military disability evaluation system through the Under Secretary of Defense for Personnel and Readiness. The Surgeons General for each service are responsible for overseeing their service’s MTFs, including the MEBs conducted at each facility. The Deputy Under Secretary of Defense for Military Personnel Policy has oversight of the PEBs, and also oversees the Disability Advisory Council. The council is composed of officials from DOD’s offices of Military Personnel Policy, Health Affairs, and Reserve Affairs, the services’ disability agencies; and the Department of Veterans Affairs. See fig. 2.

Military Disability Policies Differ among the Services, and Certain Policies May Result in Different Experiences for Reservists

The policies and guidance for disability determinations for all service members are somewhat different among the Army, Navy, and Air Force. DOD has explicitly given the services the responsibility to set up their own processes for some aspects of the disability system and has given the services much room for interpretation. Each service has implemented its system somewhat differently. For example, the composition of decision making bodies differs across the services. Additionally, the laws that govern military disability and the policies that DOD and the services have developed to implement these laws have led to reserve members having different experiences with the disability system than active duty members. Some of these experiences result from the part-time nature of reserve service while others are the consequence of policies and laws specific to reservists.

DOD Policies and Guidance Allow Services to Implement the Disability System Differently

DOD regulations establish some parameters for the disability system and provide guidelines to the services, and the services each have their own regulations in accordance with these. Specifically, the aspects of the system that differ among the services include: characteristics of the medical evaluation board (MEB) and physical evaluation board (PEB), the use of counselors to help service members navigate the system, and procedures to make line of duty determinations. Appendix III provides a compilation of these and other differences.

Medical Evaluation Boards

DOD regulations require that each service set up MEBs to conduct medical evaluations to determine if the service member meets retention standards according to each service’s regulations. The services carry out MEB procedures differently. For example, the Air Force MEB convenes an actual board of physicians who meet regularly and vote to decide whether a service member meets retention standards. In the Army and Navy, in contrast, the MEB is an informal procedure. A service member’s case file is passed among the board’s members, who separately evaluate it. In all of the services, the medical commander or his designee may sign off on the final decision. The services also differ in the qualifications and requirements for MEB board membership. The Army and Navy require that at least two physicians serve on an MEB, while the Air Force requires three.

Physical Evaluation Boards

In accordance with DOD regulations, the military services have set up PEBs to evaluate whether service members are fit for duty. DOD regulations provide no guidance concerning how much time a service member has to decide whether to accept the disability decision of his or her informal PEB. Army provides service members 10 calendar days; Navy provides 15 calendar days; and Air Force provides 3 duty days, according to their regulations. Additionally, DOD regulations provide that service members found unfit for duty by an informal PEB are guaranteed the right to appeal to a formal PEB. However, service members found fit are not guaranteed the right to appear before a formal PEB.

While DOD regulations state that a service member has the right to appeal the decision of a formal PEB, they do not state what this appeal process should look like. The services differ in how many appeal opportunities they offer service members after the formal PEB. For example, the Navy and Air Force offer two opportunities for appeal after the formal PEB. The Army also has two opportunities for appeal. However, it also has the Army Physical Disability Appeal Board, which provides appeal for only certain cases, for example, if the Army Physical Disability Agency revises the finding of the PEB during a quality or mandatory review and the soldier disagrees with the change (see table 3).

Further, the services also differ on whether they permit the same members to sit on the informal and formal PEB of the same case. The Army allows PEB members to do this while the Air Force only allows this under certain circumstances. The Navy has no written policy on the matter, although one official from the Navy PEB indicated that the members of the two boards were often the same for a case.

Physical Evaluation Board Liaison Officers

The point at which PEBLOs become involved in the disability evaluation system and the training PEBLOs receive differ between the services. DOD regulations require that each service provide members counseling during the disability evaluation process and outline the responsibilities of these counselors. For example, they are expected to discuss with service members their rights, the effects of MEB and PEB decisions, and available benefits.

Each service has created PEBLOs in accordance with these rules, but the services have placed the PEBLOs under different commands. In the Army and Air Force, PEBLOs are the responsibility of the medical command. In the Navy, in contrast, the PEBLO responsibility is shared by the PEB and MTF. Further, the services involve PEBLOs at different points in the disability process. In the Army and Air Force, PEBLOs begin counseling the service member at the MEB level of the disability process. However, while Navy, officials told us that PEBLOs provide counseling at the MEB level, some PEBLOs we interviewed told us that they begin counseling members after the informal PEB has issued its decision. At some MTFs, case managers provide counseling for service members going through the disability evaluation process. The services also differ in their training of PEBLOs. The Army holds an annual conference for PEBLOs and provides on-the-job training. The Navy relies primarily on on-the-job training and also offers quarterly and annual training. The Air Force also relies heavily on on-the-job training and, until recently, held regular training for PEBLOs.

Line of Duty Determinations

As required by law, a service member may receive disability compensation for an injury or illness that was incurred or permanently aggravated while in the line of duty. Generally, the military services document that an injury occurred in the line of duty by filling out a form or noting it in the service member’s health record. Typically, a service member’s commanding officer is responsible for this action, according to the service’s policies. Unlike the Army and Navy, the Air Force always requires a line of duty determination for reservists.

DOD regulations state that an injury is presumed to have been in the line of duty when it clearly resulted from enemy or terrorist attack, regardless of whether the member is a reserve or active duty member. However, if the injury may have resulted from misconduct or willful negligence, DOD requires the military services to investigate and determine whether the injury did, in fact, occur in the line of duty. The line of duty determination is a complicated process involving a number of people, such as the examining medical official and higher commands. DOD gives the services responsibility for creating the procedures for conducting line of duty determinations, and there are some technical differences in the processes among the services. For example, the services have different rules regarding how long this process should take. The Army and the Air Force place time frames on the process, while the Navy does not.

DOD and the Services Have Established Policies That Result in Different Experiences with the Disability System for Reservists

The laws that govern the military disability system and the policies and guidance that DOD and the services have developed to implement the laws can result in different experiences with the disability system for reservists. Some of these differences are due to the part-time nature of reserve service, while others result from laws and policies specific to reservists.

Twenty Years of Service Requirement for Disability Retirement

Because they are not on duty at all times, reservists take longer to accrue the 20 years of service that may be needed to earn the monthly disability retirement benefit when the disability rating is less than 30 percent. For example, an active duty service member who enlisted in the Army in 1985 and stayed on continuous active duty would have 20 years toward disability retirement by 2005. An Army reservist who enlisted at the same time, met his training obligations, and had been activated for 1 year would have roughly 5 years and 9 months toward disability retirement by 2005, according to the formula the Army uses to determine years of service toward disability retirement benefits. All three services use the same formula when calculating the 20 years of service requirement for disability retirement benefits.

Eight Years of Service Requirement for Compensation for Preexisting Conditions

The part-time status of reservists also makes it more difficult for reservists with preexisting conditions to be covered by the 8-year rule and therefore eligible for compensation. By law, service members with at least 8 years of active duty service are entitled to compensation even if their conditions existed before the beginning of their military service or were not service aggravated. This entitlement only applies to reservists when they are on ordered active duty of more than 30 days at the time of PEB adjudication. For reservists, accruing the 8 years necessary for a condition to be covered by this rule can be more difficult than for active duty service members. For example, an active duty service member who enlisted in the Army in 1997 and stayed on continuous active duty would have 8 years toward disability retirement by 2005. A reservist who enlisted at the same time, met his training obligations, and has been activated for 2 years would have roughly 1 year and 3 months of service, according to the Army’s 8-year rule formula and would not be eligible for compensation for a preexisting condition. Further, the services differ slightly in how they calculate the 8 years for reservists. The Army and Navy calculate the 8 years differently from the 20 year requirement, but the Air Force uses the same formula for both. The Army and Navy count only active duty time, while the Air Force also counts time spent in other activities, such as continuing education.

Line of Duty Determinations

Officials reported that commanders and others responsible for completing line of duty determinations were often uncertain as to when line of duty determinations were necessary for reservists and active duty members. Moreover, these officials noted that in some cases, the necessary line of duty determinations were not made, resulting in delays for service members. For example, Air Force officials we spoke with had different impressions as to whether line of duty determinations were always required for reservists, even though Air Force regulations state they are. Officials from the Army and Army National Guard similarly offered different perspectives on the need for line of duty determinations for reservists.

Medical Holdover

In the Army, deployed active duty soldiers return to their unit in a back up capacity when they are injured or ill. However, mobilized injured or ill Army reservists have no similar unit to return to. Consequently, they may be removed from their mobilization orders and retained on active duty in “medical holdover status” and assigned to a unit, such as a medical retention processing unit. While in medical holdover status, reservists may live on base, at a military treatment facility, at home or other locations. After their mobilization orders expire, they can elect to continue on active duty through a program such as medical retention processing, which allows them to continue receiving pay and benefits. Between 2003 and 2005 the Army reports that about 26,000 reservists entered medical holdover status (see appendix II).

Unlike most injured active duty soldiers, reservists in medical holdover generally live farther from their families than active duty members because the units at military medical facilities are often far from where their families live. In certain cases reservists in medical holdover may receive treatment and recuperate at home. The Army’s Community-Based Health Care Organizations (CBHCOs) provide medical and case management for these reservists living at home as they receive medical care in their communities. As of December 2005, about 35 percent of the reservists in medical holdover were being cared for in the CBHCO program. In order to be assigned to the CBHCO program, reservists must meet a number of criteria. For example, reservists must live in communities where they can get appropriate care, and they must also be reliable in keeping medical appointments.

DOD Has Guidance in Place to Promote Consistent and Timely Decisions, but Does Not Adequately Oversee Key Aspects of the Disability System

DOD has policies and guidance to promote consistent and timely disability decisions, but is not monitoring whether the services are compliant. Neither DOD nor the services systematically determine the consistency of decision making, which would be a key component of quality assurance. With regard to timeliness, DOD has issued goals for processing service members’ cases but is not collecting available information from the services, and military officials have expressed concerns that the goals may not be realistic. Finally, DOD is not exercising any oversight over training for staff in the disability system, despite being required to do so.

DOD Requires All Services to Use a Common Ratings System and Multiple Reviewers and Convenes a Disability Council

To encourage consistent decision making, DOD policies require that service members’ case files undergo multiple reviews and federal law requires that disability ratings be based on a common schedule. During both the MEB and PEB stages of the disability process, a service member’s case must be reviewed and approved by several officials with different roles. When rating the severity of a service member’s impairment, all services are required to use a common schedule, VA’s Schedule for Rating Disabilities (VASRD), in accordance with federal law. The VASRD is a descriptive list of medical conditions along with associated disability ratings. For example, if a service member has x-ray evidence of degenerative arthritis affecting two or more joints, “with occasional incapacitating exacerbations,” he or she should receive a rating of 20 percent according to the VASRD.

DOD also convenes the Disability Advisory Council, which DOD officials told us is the primary oversight body of the disability system. The disability council is composed of key officials from the three disability agencies of the services, the VA, and relevant DOD officials from the health affairs, reserve affairs, and personnel departments. The council’s mission is to monitor the administration of the disability system and, according to DOD officials, the council serves as a forum to discuss issues such as rules changes and increasing coordination among the services. Currently, the disability council is facilitating a review and revision of all DOD regulations pertaining to the disability system. Military officials view the council as a group that aims to meet quarterly to discuss issues raised by the services. By having these meetings, DOD hopes to bring all of the services “on the same page” when it comes to the disability system. However, military officials reported that the council has not met quarterly in the past year and generally does not produce formal reports for the DOD chain of command. Furthermore, the disability council is staffed by one person at DOD who has additional responsibilities.

Military officials also regard the appeals process as helping to ensure the consistency of disability evaluation decision making. However, not all service members appeal. In addition, during the appeals process additional evidence may be presented that may result in a different outcome for the same case. Furthermore, the appeals process is designed to determine whether the correct decision was made, rather than whether consistent decisions were made across comparable cases.

Lack of Oversight by DOD and the Services Provides Little Assurance Decisions Are Consistent

Despite this policy guidance and the presence of the disability council, both DOD and the three services lack quality assurance mechanisms to ensure that decisions are consistent. Given that one of the primary goals of the disability system is that disability evaluations take place in a consistent manner, collecting and analyzing the service members’ final disability determinations are critical for ensuring that decisions are consistent. DOD regulations recognize this and require that the agency establish necessary reporting requirements to monitor and assess the performance of the disability system and compliance with relevant DOD regulations. Yet, DOD does not collect information from the services on the final disability determinations and personal characteristics of service members going through the disability system.

In addition, DOD has not established quality parameters for the services to follow to evaluate the consistency of decision making. As a result, the services generally lack a robust quality assurance process. In our past work on federal disability programs, we have recommended that quality assurance have two components: (1) the use of multivariate regression analysis examining disability decisions along with controlling factors to determine whether the decisions are consistent and (2) an in depth independent review of a statistically valid group of case files to determine what factors may contribute to inconsistencies. However, the services were unable to provide any evidence that they are conducting statistical reviews – such as multivariate regression analysis – on their data to determine the consistency of decision making for service members with similar characteristics. Furthermore, while we found that the Army is conducting independent reviews of 25 to 30 percent of its PEB cases, the Navy and Air Force conduct these reviews only when a service member appeals the PEB’s decision. Additionally, these reviews reflect how a single case’s medical evidence supports the dispositions made (accuracy) rather than the degree to which decisions in cases, in general, with similar impairments and characteristics compare (consistency). Without such an analysis the services are unable to assure that adjudicators are making consistent decisions in reservist and active duty cases with similar characteristics.

Officials from the services said that it was very difficult to examine outcomes for consistency because each disability decision is unique and there are a multitude of factors considered when rendering a disability decision, some of which could not be captured in a database. For example, individuals’ pain tolerance varies, along with their motivation to adhere to treatment programs. Nonetheless, other federal disability programs face the same challenges, have acknowledged the importance of determining consistency of decision making, and have taken some initial steps to develop quality assurance systems. For example, the VA selects a random sample of files for independent review using a standard methodology and compiles the results of these reviews.

DOD Has Instituted Timeliness Goals for Processing Service Members’ Cases, but Does Not Oversee Compliance with Them

DOD regulations set forth timeliness goals for the two major processes of the disability system. According to DOD, the first stage of the process— the MEB—should normally be completed in 30 days or less. The second stage of the process—the PEB—should normally take 40 days or less. Despite establishing these timeliness goals for the services, DOD is not ensuring compliance with them. DOD does not regularly collect available timeliness data from the services, a necessary first step for determining compliance.

The services generally are using their databases to track the timeliness of decisions, but military officials cited confusion regarding the start date for the process. Both the Army and Navy are tracking processing times for both the MEB and PEB using their databases. The Air Force lacks a centralized database to track its MEB cases and therefore can only track PEB timeliness. However, we found that the usefulness of these timeliness data may be undermined by confusion among military officials and data entry staff regarding the starting dates for the disability process. We compared original Army PEB case files to Army electronic data from both its MEB and PEB databases, and found that the date a physician dictates a narrative summary, the beginning of the disability process and a critical data point for timeliness calculations, was frequently entered incorrectly into the Army’s databases. When we asked about these errors, Army officials said that increased training of data entry staff would help with these problems. Navy officials also said that there was some confusion about how to record starting dates for cases when additional medical information was needed to make a disability decision for a service member.

Data reported by the services on the timeliness of cases generally show that the services are not meeting DOD timeliness goals (see appendix II). Military officials said that these results stemmed in part from the unrealistic nature of the goals themselves. Navy officials told us that they do not consider the 30-day goal as a performance standard for MEB processing to be held accountable for. They said that the 30-day goal is also unrealistic, especially in certain cases when there were addendums to the narrative summary. Army officials also said that it was unrealistic for all MEB cases to be processed in 30 days because certain cases take longer. For example, cases when a line of duty determination is needed or when certain medical tests are required to diagnose some orthopedic or psychiatric conditions.

DOD’s Delegation of Training to the Services and Staff Turnover Presents Additional Challenges for the Disability System

While DOD regulations require that the agency develop and maintain training for key participants in the disability system, DOD officials told us that they had given this responsibility to the services. The Assistant Secretary of Defense for Health Affairs is given explicit instructions to develop and maintain a training program for MEB and PEB staff, but officials from the Office of Health Affairs indicated they were unaware that they had the responsibility to develop a training program. In addition, despite high turnover among military disability evaluation staff, the services do not have a system to ensure that all staff are properly trained. This turnover stems, in part, from the military requirement that personnel rotate to different positions in order to be promoted. Depending on the positions involved, military officials told us that some staff remain in their positions from 1 to 6 years, with most remaining about 3 years. This turnover and the resulting loss of institutional knowledge require that the services systematically track who has been properly trained. However, all of the services lack data systems that would allow them to do so, an issue that was highlighted in a previous report by the RAND Corporation.

Disparities May Exist in Disability Benefits and Processing Times between Army Reservists and Active Duty Soldiers, but Lack of Data Prevents More Definitive Conclusions

Our analysis of Army disability data from calendar years 2001 to 2005 indicated that after controlling for many of the differences we found between reservists and active duty soldiers, Army reservists received similar disability ratings to their active duty counterparts. We also found that reservists may be less likely to receive military disability benefits. Data on years of service and preexisting conditions were not available for this analysis, however, factors that influence disability benefit decisions. Finally, we were unable to compare processing times for reserve and active duty disability cases because we found that Army data on processing times were not reliable. However, based on these data, some Army officials conclude that reservists’ cases often take longer to process through the disability evaluation system than the cases of active duty soldiers.

Army Reservists and Active Duty Soldiers in the Disability Evaluation System Had Different Characteristics

From 2001 through 2005, the characteristics of Army reservists and active duty soldiers in the disability evaluation system differed in a number of ways. Specifically, reservists tended to have more impairments than active duty soldiers; they were more likely than active duty soldiers to have three or four impairments. Reservists also experienced higher rates of impairments affecting the cardiovascular and endocrine systems, while active duty soldiers experienced a higher rate of impairments affecting the musculoskeletal system. Reservists were more often classified in higher pay grades and more often worked as functional support and administration, crafts workers, and service and supply handlers. See appendix IV. Active component soldiers worked more often as infantry and gun crews; electronic equipment repairers; and communications and intelligence specialists. In addition, compared to the number of active duty disability cases from 2001 through 2005, which remained relatively constant, the proportion of reservists going through the PEB process rose dramatically through 2004. See figure 3.

Finally, the demographic characteristics of Army reservists and active duty soldiers in the disability evaluation system also differed. Eighty percent of reservists were male, compared to 76 percent of active duty soldiers, while, on average, reservists were 11 years older than active duty soldiers. See figure 4.

Army Reservists Received Disability Ratings Similar to Their Active Duty Counterparts, but Reservists May Be Less Likely to Receive Benefits

Before controlling for factors that could account for differences in the outcomes of the Army disability evaluation system for reserve and active duty soldiers, our analysis of Army data indicates that, from 2001 through 2005, reservists were assigned slightly higher disability ratings, but received benefits less often than active duty soldiers. See appendix V. When we controlled for many of the characteristics of reserve and active duty soldiers that could account for their difference in ratings, we found that, among soldiers who received ratings, the ratings assigned to Army reservists were comparable to those assigned to their active duty counterparts. When we controlled for a more limited number of factors, Army reservists who were determined to be unfit for duty appeared less likely to receive benefits (either monthly disability payments or severance pay). See appendix I. This analysis of benefit outcomes for Army reserve and active duty disability cases could not account for the influence that preexisting conditions and years of service can have on disability decisions. These factors are key in determining whether an injured or ill service member qualify for disability benefits. Because we could not test the effect of these factors empirically, we cannot rule out the possibility that one or the other may account for the differences we found.

Poor Quality Data Precluded GAO Analysis, but the Army Reports Reservists’ Cases Can Take Longer

While, according to the Army’s own statistics, the PEB process can take longer for reservists than active duty soldiers, we found the Army data used to calculate processing times not of sufficient quality to warrant its use in our analysis. Specifically, the dates in Army’s electronic database often did not correspond with the dates recorded in paper files. See appendix I. Nonetheless, the statistics the Army provided indicate that disability cases reviewed between fiscal years 2001 and 2005 took consistently longer than those of active duty soldiers. Over half (54 percent) of reserve soldiers took longer than 90 days while over one-third (35 percent) of active duty soldiers exceed this threshold. See appendix II for more detail.

There are several possible explanations for the differences in processing times between reservists and active duty members, according to the Army. For example, the Army officials reported that MEBs often must request medical records from private medical practitioners for reservists’ cases, which can involve considerable delays. In addition, the personnel documents for reservists are stored in facilities around the U.S., and therefore they may take longer to obtain than records for centrally located active duty soldiers. Due to the lack of data on these issues as well as the problems we encountered with the data provided by the Army we were not able to measure the differences or empirically test possible explanations for differences the Army reported in the timeliness of disability case processing for Army reservists and active duty soldiers.

Conclusions

The military disability system’s outcomes can greatly impact the future of service members, including reservists, injured in service to their country. Given the significance of these decisions as well as the latitude that services have to implement the system, it is important that DOD exercise proper oversight to make sure the system meets the needs of service members today and in the future. However, DOD is not adequately monitoring the outcomes for active duty and reservist cases in the disability evaluation system. DOD and the services do not have complete and reliable data for all aspects of the disability system. Further, neither DOD nor the services are systematically evaluating consistency and timeliness of decision making in the system.

Military officials recognize that in many cases, service members’ cases are not determined within timeliness goals and have suggested that the goals may not be appropriate in many cases. In addition, it may take longer for reservist cases to go through the system. If a goal does not reflect appropriate processing times, it may not be useful as a program management tool. Furthermore, both consistency and timeliness of decisions depend on the adequate training and experience of all participants in the disability system. Yet we found that DOD had little assurance that staff at all levels are properly trained.

Recommendations for Executive Action

To ensure that all service members—both active duty and reserves— receive consistent and timely treatment within the disability evaluation process, we recommend that the Secretary of Defense take the following five actions require the Army, Navy, and Air Force to take action to ensure that data needed to assess consistency and timeliness of military disability rating and benefit decisions are reliable; require these services to track and regularly report these data— including comparisons of processing times, ratings and benefit decisions for reservists and active duty members—to the Under Secretary of Personnel and Readiness and the Surgeons General; determine, based on these reports, if ratings and benefit decisions are consistent and timely across the services and between reservists and active duty members and institute improvements to address any deficiencies that might be found; evaluate the appropriateness of current timeliness goals for the disability process and make any necessary changes; and assess the adequacy of training for MEB and PEB disability evaluation staff.

Response to Agency Comments

We provided a draft of this report to the Department of Defense for its review. DOD agreed with our recommendations, indicating the Department will implement all of them and listing a number of steps it will take to do so. DOD also provided technical comments, which we incorporated into the report as appropriate.

We are sending copies of this report to the Secretary of Defense, relevant congressional committees, and others who are interested. Copies will also be made available to others upon request. The report is also available at no charge on GAO’s Web site at http://www.gao.gov.

Please contact me on (202) 512-7215 if you or your staff have any questions about this report. Contact points for our Offices of Congressional Relations and Public Affairs can be found on the last page of this report. Other major contributors to this report are listed in appendix VII.

Appendix I: Objective, Scope, and Methodology

The objectives of our report were to determine: (1) how current DOD policies and guidance for disability determinations compare for the Army, Navy, and Air Force, and what policies are specific to reserve component members of the military; (2) what oversight and quality control mechanisms are in place at DOD and these three services of the military to ensure consistent and timely disability rating and benefit decisions for active and reserve component members, and (3) how disability rating and benefit decisions, and processing times compare for active and reserve component members of the Army, the largest branch of the service, and what factors might explain any differences.

To address objectives 1 and 2, we reviewed relevant legislation, policy guidance, and literature; interviewed officials from DOD, Army, Navy, Air Force, Reserves, and National Guard; and visited Lackland and Randolph Air Force Bases, Fort Sam Houston and Walter Reed Army Medical Center; Washington Navy Yard and Bethesda Naval Hospital; and interviewed relevant officials. In addition, we interviewed officials from military treatment facilities.

To determine if outcomes for active duty and reserve service members’ disability cases were statistically consistent, we analyzed data provided by the physical evaluation board (PEB) of the Army. We also obtained summary information on total caseloads and processing times from the services and from the Department of Defense. Based on our assessment of the quality of the Army’s data, we concluded that data on disability determinations and ratings made by the Army’s PEB were sufficiently reliable for our analysis. On the other hand, the Army’s data on processing times were not reliable for our analysis. We did not test the reliability of statistical data provided by DOD and the services.

This appendix is organized into two sections: Section 1 describes the analyses related to our tests of data quality and reliability. Section 2 describes the empirical analyses that were used to determine if outcomes for active duty and reserve disability cases were statistically consistent.

Section 1: Data Reliability Tests

To ensure that the Army data were sufficiently reliable for our analyses, we conducted detailed data reliability assessments of the data sets that we used. We restricted these assessments, however, to the specific variables that were pertinent to our analyses. We found that all of the data sets used in this report were sufficiently reliable for use in our analyses.

To allow us to analyze the outcomes of the disability evaluation process and determine whether decisions were made in a timely fashion, we requested that the Army share data from both the Medical Evaluation Board and the Physical Evaluation Board for our review. The Army provided extracts from both the Medical Evaluation Board Internal Tracking Tool (MEBITT), used by the Medical Evaluation Board and the Physical Disability Computer Assisted Processing System (PDCAPS), used by the PEB.

During interviews with the database managers responsible for MEBITT and PDCAPS, we learned that the Army has few internal controls to ensure that the data are complete and accurate. Consequently, we conducted a trace-to-file process to determine whether the data in the electronic systems were an accurate reflection of what was recorded in the paper files. We requested that the Army provide us with the paper files for a sample of 130 cases that completed the Army’s disability evaluation process between 2000 and August, 2005. Army officials provided 93 paper files for our review. The remaining files were archived or were not found.

We checked the data in files provided against the electronic records in MEBITT and PDCAPS. We determined that the MEBITT data were not sufficiently reliable for our use. We also determined that in the PDCAPS, there was a high degree of accuracy in the data fields related to: rank, component (active duty versus reserve component), date of entry into military service, primary military occupational specialty, disposition of disability case, percentage rating for disability, location of PEB, and illness/diagnosis codes. These fields were deemed reliable for use in our report. However, this review also revealed that the data in the date fields, such as the narrative summary dates and the final decision dates, were often inaccurate and were therefore determined to be of insufficient quality for use in our report.

Section 2: Statistical Analyses

To determine if outcomes for active duty and reserve disability cases were statistically consistent, we conducted extensive statistical analyses including cross tabulations and econometric modeling. This was important because active and reserve component soldiers being evaluated differ greatly in demographic characteristics and in administrative characteristics, such as pay grade and occupational specialty. Recognizing the potential of these characteristics to influence final outcomes and disability ratings, we developed econometric models to assess whether the observed differences between active and reserve component soldiers persist after controlling for these factors.

We began with a series of bivariate cross tabulations and then expanded these cross-classifications and examined three-way and four-way tables. These allowed us to compare large groups of active and reserve soldiers, as well as to compare soldiers in specific sets of categories—such as active and reserve soldiers of different grades being evaluated at different PEBs. To control for additional factors, we supplemented the cross- tabulations with ordinary least squares (OLS) and multivariate logistic regressions. Our analyses considered both the size and significance of the relationships of interest, using means, percentages, and odds and odds ratios to assess magnitude, and f-tests, chi-square tests and Wald statistics to assess the significance of the differences.

The analyses are limited due to our inability to control for several important factors in the disability evaluation process. For example, no reliable electronic data existed to indicate whether an injury existed prior to service or was incurred outside of the line of duty, both primary reasons for separating a soldier without benefits. Similarly, Army officials told us that data on years of service for reservists in the electronic data the Army provided were unreliable. Additionally, soldiers declared fit or separated without benefits do not receive percentage disability ratings, and the Army reports no impairment codes for soldiers declared fit. As such, we could not determine whether active and reserve component soldiers were similarly likely to be declared fit controlling for impairment or percentage rating. Given these difficulties, we restricted our multivariate analyses to soldiers rated unfit.

Multivariate regression analysis

To assess factors contributing to the final rating among those members declared unfit and receiving a percentage rating (that is, excluding those separated without benefits), we ran a series of multivariate models. Army data systems report up to four impairments. Their final percentage disability rating is determined by a composite of ratings for individual impairments, the system(s) affected and how the specific impairment relates to the soldier’s ability to perform his or her duties. Regression analysis allows us to assess whether the observed differences between reserve and active soldiers’ final ratings persist controlling for factors that enter the decision process, such as military occupational specialty and system of impairment, as well as other factors such as demographic differences between the reserve component and active duty soldiers.

We began by estimating a “gross effects” (or unadjusted) model, which considers the gross difference in mean disability ratings between active and reserve component soldiers ignoring other factors. The model confirms descriptive statistics showing that reserve component members’ ratings average approximately 4 points higher than those of active component members.

We next estimated a series of alternative “net effects” (or adjusted) models to account for other factors that influence the decision process; these models estimate the impact of being a reservist on rating “net” of other factors. Our first model included number of reported impairments, physical system affected and occupational specialty; a second model added year of decision, age, race, sex, pay grade, and PEB to control for forces that may influence the decision process unofficially and certain demographic differences between components. Additionally, we ran a variety of alternative specifications to ensure the stability and robustness of the results; this included, for example, a model testing the interaction between system affected and occupational specialty and a model to account for the clustering (and potential “nonindependence”) of cases within each PEB.

Table 4 presents the coefficient representing the relationship between being a reservist on final disability ratings in models that control for a limited set of controls both relevant and external to the formal decision process. What appears to be a small difference in ratings between reserve and active component members diminishes controlling for other factors. Overall, results of our OLS regression analyses suggest that active and reserve component members receive similar disability ratings controlling for factors that enter the formal decision process formally and indirectly.

Multinomial Logistic Analysis

To assess receipt of benefits, we estimated a multinomial logistic model, a technique that allows us to estimate the likelihood of placement in one of several categories controlling for additional factors. The model produces relative risk ratios that compare the relative odds of reserve component soldiers and active duty soldiers determined unfit for duty being placed into either one of two categories (severance pay or permanent disability retirement) rather than the base or referant category (separated without benefits). With controls, the relative risk ratio compares the odds of placement in the given category for similarly situated active and reserve component soldiers.

A relative risk ratio of 1 indicates that reserve and active component members have equal odds of being placed in one category rather than the base category. A relative risk ratio of less than 1 for reserve soldiers indicates that reservists have lower odds than active members of placement in the category rather than in the base category, and a relative risk ratio of greater than 1 indicates that reservists have higher odds than active duty members of being placed in that category rather than in the base category. Because soldiers placed on the temporary disability retired list (TDRL) have not received a final benefits determination, they are excluded from the model.

The relative risk ratios in table 5 demonstrate that among those declared unfit, reserve component soldiers have significantly lower odds than active component soldiers of receiving either permanent disability retirement or lump sum disability severance pay. Prior to controlling for other factors (our “gross effects” model), reserve soldiers have significantly lower odds than active component members of receiving either permanent disability retirement or severance pay rather than being separated without benefits—the relative risk ratios of 0.5 and 0.4 in the first row of the table respectively demonstrate reservists are only half or less than half as likely to receive permanent disability retirement or severance pay, respectively.

This relationship persists after controlling for limited factors both relevant to and external to the official decision making process (“net effects” models), and in fact the estimated difference between reservists and active duty soldiers is in fact increased by the inclusion of variables such as race, sex and PEB location. While these additional factors do not directly enter the decision making process, they control for some of the administrative and demographic differences we observe between active and reserve component members. The relationship differs for the odds of receiving severance pay, where reserve soldiers have less than one third the odds of active soldiers, and the odds of receiving permanent disability retirement, where the odds of reservists’ receiving this type of benefit rather than separation without benefits is about one-tenth that of active component members.

We lacked reliable electronic data on two potentially important factors. This inability to control for length of service and injuries existing prior to service prevents us from determining whether the differences presented above are warranted or defensible.

Appendix II: Selected Data from Service Branches on Processing Time, Dispositions, and Medical Holdover

Includes “presumed fit” cases.

Appendix III: Compilation of Differences in Regulations Governing the Military Disability Evaluation System

Disability Evaluation Related Regulations

Federal Code 38 CFR Part 4: Veterans Affairs Schedule for Rating Disabilities Department of Defense DODD 1332.18 “Separation or Retirement for Physical Disability” DODI 1332.38 “Physical Disability Evaluation” DODI 1332.39 “Application of the Veterans Administration Schedule for Rating Disabilities”

Army AR 40-400 “Patient Administration” AR 40-501 “Standards of Medical Fitness” AR 600-8-4 “Line of Duty Policy, Procedures, and Investigations” AR 600-60 “Physical Performance Evaluation System” AR 635-40 “Physical Evaluation for Retention, Retirement, or Separation”

Navy SECNAV 1850.4E “Department of the Navy Disability Evaluation Manual” JAGINST 5800.7D “Manual of the Judge Advocate General” NAVMED P-117 “Manual of the Medical Department”

Air Force AFI 36-2910 “Line of Duty (Misconduct) Determinations” AFI 36-3212 “Physical Evaluation for Retention, Retirement and Separation” AFI 41-210 “Patient Administration Functions” AFI 44-157 “Medical Evaluation Boards and Continued Military Service” AFI 48-123 “Medical Examination and Standards”

Appendix IV: Characteristics of Army Service Members Entering the DES

Appendix V: Disability Evaluation Outcomes for Army Active Duty and Reserve Component Members for 2001 to 2005

This appendix compares Army disability evaluation outcomes in effect for active duty and reserve component service members as of August 2005. For the purpose of our analysis, we counted only final dispositions for service members initially placed on the temporary disability retired list (TDRL) and subsequently taken off that list when a final disposition was made by the Army’s Physical Evaluation Board (PEB). In these cases, we counted the final disposition in the year the initial TDRL decision was made. As a result, the tables in this appendix show fewer TDRL dispositions than the number issued by the PEB annually, according to the Army. The tables also show greater numbers of permanent disability retirement and other dispositions than the numbers reported by the Army PEB annually for the years 2001 through 2004. In each case, the differences are more pronounced in earlier years. Therefore, data in these tables do not represent the number of each type of disability disposition issued by the Army PEB annually.

Appendix VI: Comments from the Department of Defense

Appendix VII: GAO Contact and Staff Acknowledgments

Staff Acknowledgments

In addition to the contact named above, Clarita A. Mrena, Assistant Director; and Anna M. Kelley made major contributions to this report. In addition, Jason Barnosky, Melinda Cordero, Erin Godtland, and Scott Heacock served as team members; Lynn Milan, Anna Maria Ortiz, Doug Sloane, Mitch Karpman, and Wil Holloway provided guidance and assistance with design and analysis; Rachel Valliere advised on report preparation; and Roger Thomas provided legal advice.

Related GAO Products

Veterans’ Disability Benefits: Claims Processing Challenges and Opportunities for Improvements. GAO-06-283T. Washington, D.C.: December 7, 2005.

Military Personnel: Top Management Attention Is Needed to Address Long-standing Problems with Determining Medical and Physical Fitness of the Reserve Force. GAO-06-105. Washington, D.C.: October 27, 2005.

Military Personnel: DOD Needs to Improve the Transparency and Reassess the Reasonableness, Appropriateness, Affordability, and Sustainability of Its Military Compensation System. GAO-05-798. Washington, D.C.: July 19, 2005.

Federal Disability Assistance: Wide Array of Programs Needs to be Examined in Light of 21st Century Challenges. GAO-05-626. Washington, D.C.: June 2, 2005.

Veterans’ Disability Benefits: Claims Processing Problems Persist and Major Performance Improvements May Be Difficult. GAO-05-749T. Washington, D.C.: May 26, 2005. 21st Century Challenges: Reexamining the Base of the Federal Government. GAO-05-325SP Washington, D.C.: March 4, 2005.

Military Pay: Gaps in Pay and Benefits Create Financial Hardships for Injured Army National Guard and Reserve Soldiers. GAO-05-125 and related testimony GAO-05-322T. Washington, D.C.: February 17, 2005.

High-Risk Series: An Update. GAO-05-207. Washington, D.C.: January 1, 2005.

SSA’s Disability Programs: Improvements Could Increase the Usefulness of Electronic Data for Program Oversight. GAO-05-100R. Washington, D.C.: December 10, 2004.

Veterans Benefits: VA Needs Plan for Assessing Consistency of Decisions. GAO-05-99. Washington, D.C.: November 19, 2004 and related testimony GAO-06-120T.

Military Personnel: DOD Needs to Address Long-term Reserve Force Availability and Related Mobilization and Demobilization Issues. GAO-04-1031. Washington, D.C.: September 15, 2004.

Social Security Administration: More Effort Needed to Assess Consistency of Disability Decisions. GAO-04-656. Washington, D.C.: July 2, 2004.

SSA Disability Decision Making: Additional Steps Needed to Ensure Accuracy and Fairness of Decisions at the Hearing Level. GAO-04-14. Washington, D.C.: November 12, 2003.

Defense Health Care: Army Needs to Assess the Health Status of All Early-Deploying Reservists. GAO-03-437. April 15, 2003 and related testimony GAO-03-997T Washington, D.C.: July 9, 2003.

Veterans’ Benefits: Quality Assurance for Disability Claims and Appeals Processing Can Be Further Improved. GAO-02-806. August 16, 2002 and related testimony GAO-05-655T. Washington, D.C.: May 5, 2005.

Defense Health Care: Disability Programs Need Improvement and Face Challenges. GAO-02-73. Washington, D.C.: October 12, 2001.

DOD Disability: Overview of Compensation Program for Service Members Unfit for Duty. GAO-01-622. Washington, D.C.: April 27, 2001.

Disability Benefits: Selected Data on Military and VA Recipients. HRD-92-106. Washington, D.C.: August 13, 1992.