Veterans' Disability Benefits: VA Has Improved Its Programs for Measuring Accuracy and Consistency, but Challenges Remain
Released: 2010-03-24T00:00:00

Letter

Background

Through its disability compensation program, VA pays monthly benefits to veterans with service-connected disabilities. Under its disability pension program, VA pays monthly benefits to low-income veterans who have disabilities not related to their military service or are age 65 or older. VA also pays compensation to the survivors of certain veterans who had service-connected disabilities and of servicemembers who died while on active duty.

Veterans and their survivors claim benefits at one of the Veterans Benefits Administration’s (VBA) 57 regional offices. Once the claim is received, a service representative assists the veteran in gathering the relevant evidence to evaluate the claim. Such evidence includes the veteran’s military service records, medical examinations, and treatment records from VA medical facilities and private medical service providers. Also, if necessary for reaching a decision on a claim, the regional office arranges for the veteran to receive a medical examination. Once all necessary evidence has been collected, a rating specialist evaluates the claim and determines whether the claimant is eligible for benefits. If so, the rating specialist assigns a percentage rating. Veterans with multiple disabilities receive a single composite rating. Since 2001, VBA has created 15 resource centers that are staffed exclusively to process claims or appeals from backlogged regional offices. Most of these centers focus either on making rating decisions, or on developing the information needed to evaluate claims.

In addition to the traditional claims process, any member of the armed forces who has seen active duty—including those in the National Guard or Reserves—is eligible to apply for VA disability benefits prior to leaving military service through VA’s Benefits Delivery at Discharge (BDD) program or the related Quick Start program. In 2006, VA completed its consolidation of BDD rating activity into its Salt Lake City, Utah, and Winston-Salem, North Carolina, regional offices, to increase the consistency of BDD claims decisions. Also, under the Department of Defense (DOD)–VA disability evaluation system pilot program, servicemembers undergoing disability evaluations, if found medically unfit for duty, receive VA disability ratings. This rating covers both the unfitting conditions identified by the military service and conditions identified by the servicemember during the process. The rating is used by both DOD and VA to determine entitlement for disability benefits.

Enacted in October 2008, the Veterans’ Benefits Improvement Act of 2008 required VA to contract for an independent, 3-year review of VBA’s quality assurance program. This review is to include, among other items, assessments of the accuracy of disability ratings and their consistency across VA regional offices. VA contracted with the Institute for Defense Analyses (IDA) to conduct this study. According to VA, IDA will provide preliminary findings in the Summer of 2010, and VA is scheduled to report to the Congress in October 2011.

STAR Program

Under the STAR program, which was implemented in fiscal year 1999, VBA selects a random sample of completed claims decisions each month from each of its regional offices to review for accuracy. STAR reviewers assess decision accuracy using a standard checklist. For decisions affecting benefit entitlement, this review includes an assessment of whether (1) all issues in the claim were addressed; (2) assistance was provided to the claimant, as required by the Veterans Claims Assistance Act of 2000; and (3) the benefit entitlement decision was correct. If a claim has any error, VBA counts the entire claim as incorrect for accuracy rate computation purposes. The STAR reviewer then returns the case file and the results of the review to the regional office that made the decision. If an error was found, the regional office is required to either correct it or request reconsideration of the error determination. VBA uses the national accuracy rate from STAR reviews of compensation entitlement decisions as one of its key claims processing performance measures. VA also uses STAR data to estimate improper compensation and pension benefit payments.

Consistency Review Activities

One VA consistency review activity involves conducting studies of regional offices’ decisions on specific conditions such as post-traumatic stress disorder where VBA found differences, such as in benefit grant rates, across regional offices through comparative statistical analysis. VBA uses the results of these reviews to identify root causes of inconsistencies and to target training. Under another VA consistency review activity, called inter-rater reliability reviews, VBA provides rating specialists a sample case file to assess how well raters from various regional offices agree on an eligibility determination when reviewing the same body of evidence. These reviews allow VBA officials to target a single rating issue and take remedial action to ensure the consistent application of policies and procedures nationally.

VA Has Implemented Procedures to Address Deficiencies Identified with the STAR Program, but Continues to Face Challenges in Improving Accuracy

Over the past decade, VBA has taken several actions to improve its STAR program and to address deficiencies identified by both GAO and VA’s OIG. For example, in March 1999, we found that STAR review staff lacked sufficient organizational independence because they were also responsible for making claims decisions and reported to regional office managers responsible for claims processing. In response to our findings, VBA took steps to address this by utilizing reviewers who do not process claims and who do not report to managers responsible for claims processing. More recently, in February 2008, we found that STAR was not sampling enough initial pension claims to ensure the accuracy of pension claims decisions. Because initial pension claims constituted only about 11 percent of the combined compensation and pension caseload subject to accuracy review, few were likely to be included in the STAR review sample. We recommended that VBA take steps to improve its quality assurance review of initial claims, which could include reviewing a larger sample of pension claims. According to VBA, it has addressed this issue by consolidating pension claims processing in its three Pension Management Centers and establishing a separate STAR sample for pension claims. During fiscal year 2009, VBA began reviewing more pension claim decisions and reported that, for fiscal year 2009, its pension entitlement accuracy was 95 percent, exceeding its goal.

In a September 2008 report, we noted that VA lacked sufficient and specific performance measures for assessing the accuracy of decisions on BDD claims and recommended that VA consider options for separately estimating the accuracy of such claims decisions. VA conducted an analysis of the costs of sampling pre-discharge claims as part of STAR and concluded that the costs would outweigh possible, unquantifiable benefits. VA also noted that the two sites that rate BDD claims surpassed the national average in accuracy for claims overall. While generally responsive to our recommendation, VA’s analysis did not specifically review the accuracy of BDD claims relative to traditional claims. Moreover, because BDD claims do not comprise all claims reviewed at the two rating sites, we continue to believe VA’s analysis was not sufficient to estimate the relative accuracy of BDD claims at these sites. While we agree that the benefits of reviewing accuracy are difficult to measure, if VA had better information on the accuracy of BDD claims, VA could use such information to inform training and focus its monitoring efforts. In contrast, VA currently performs STAR reviews that target rating decisions made by its Baltimore and Seattle offices under the DOD-VA disability evaluation system pilot program. Such a targeted review could also be conducted for BDD claims.

In its March 2009 report, VA’s OIG also identified several deficiencies in the STAR program and recommended corrective actions. The OIG found that (1) regional offices did not always submit all requested sample cases for review, (2) reviewers did not evaluate all documentation in sample files, and (3) reviewers were not properly recording some errors. The OIG also found that VBA was not conducting STAR reviews of redistributed cases (for example, claims assigned to resource centers for rating). The OIG reviewed a sample of redistributed claims and found that 69 percent had accurate entitlement decisions, well below VBA’s reported rate of 87 percent for the 12-month period ending in February 2008. Further, the OIG found that VBA did not have minimum training requirements for STAR reviewers.

As of March 2010, VBA had taken actions to respond to all of the OIG’s recommendations related to STAR, including (1) implementing procedures to follow up on cases not submitted by regional offices; (2) adding a mechanism to the STAR database to remind reviewers of key decision points; (3) requiring a second-level review of STAR reviewers’ work; and (4) establishing a requirement that STAR reviewers receive 80 hours of training per year. In addition, during fiscal year 2009, based in part on the OIG’s recommendation, VBA also began monitoring the accuracy of claims decided by rating resource centers as it does for regional offices. As we noted in our January 2010 report, VBA has significantly expanded its practice of redistributing regional offices’ disability claims workloads in recent years, and gathering timeliness and accuracy data on redistributed claims could help VBA assess the effectiveness of workload redistribution.

In addition, as the Congress has provided more resources to VBA to increase compensation and pension staffing, VBA has devoted more resources to quality review. In fiscal year 2008, VBA more than doubled the size of the quality assurance staff, allowing it to increase the scope of quality assurance reviews. VA states that in the 12-month period ending in May 2009, STAR staff reviewed over 14,000 compensation and pension benefit entitlement decisions.

Although VBA has taken steps to address deficiencies in the STAR program, the accuracy of its benefit entitlement decisions has not improved. The accuracy rate was 86 percent in fiscal year 2008 and 84 percent in fiscal year 2009, well short of VBA’s fiscal year 2009 goal of 90 percent. VA attributed this performance to the relatively large number of newly hired personnel conducting claims development work and a general lack of training and experience. Human capital challenges associated with providing the needed training and acquiring the experience these new claims processors need to become proficient at their jobs will likely continue in the near future. According to VBA officials, it can take 3 to 5 years for rating specialists to become proficient.

VA Has Taken Actions to Strengthen Efforts to Monitor Consistency of Claims Decisions

VA has taken actions to address deficiencies identified with its consistency review programs, but it is still too early to determine whether these actions will be effective. In prior work, we reported that VBA did not systematically assess the consistency of decision making for any specific impairments included in veterans’ disability claims. We noted that if rating data identified indications of decision inconsistency, VA should systematically study and determine the extent and causes of such inconsistencies and identify ways to reduce unacceptable levels of variations among regional offices. Based on our recommendation, VBA’s quality assurance staff began conducting studies to monitor the extent to which veterans with similar disabilities receive consistent ratings across regional offices and individual raters. VBA began these studies in fiscal year 2008. VBA identified 61 types of impairments for consistency review and conducted at least two inter-rater reliability reviews, which found significant error rates.

In its March 2009 report, the OIG noted that, while VBA had developed an adequate rating consistency review plan, including metrics to monitor rating consistency and a method to identify variances in compensation claim ratings, it had not performed these reviews as scheduled. In fact, VBA had initiated only 2 of 22 planned consistency reviews in fiscal year 2008. The OIG reported that VBA had not conducted these reviews because STAR staffing resources were not sufficient to perform all of their assigned responsibilities and noted that VBA’s quality review office had not staffed all of its authorized positions. In addition, the OIG found that inter-rater reliability reviews were not included in VBA’s quality assurance plan. The OIG recommended that VBA (1) develop an annual rating consistency review schedule and complete all planned reviews as scheduled; (2) dedicate sufficient staff to conduct consistency reviews in order to complete planned workload and reviews; and (3) include inter- rater reliability reviews as a permanent component of its consistency review program.

VBA reported that it has developed an annual consistency review schedule and is in the process of conducting scheduled fiscal year 2010 reviews. As of January 2010, VBA also added six staff members to perform quality assurance reviews. Further, VBA incorporated inter-rater reliability reviews into its fiscal year 2009 quality assurance plan. Because VBA has only recently implemented these initiatives, it is too early to determine their impact on the consistency of claims decisions.

Conclusion

Over the years, VA has been challenged in its efforts to ensure that veterans get the correct decisions on disability claims the first time they apply for them, regardless of where the claims are decided. Making accurate, consistent, and timely disability decisions is not easy, but it is important. Our veterans deserve timely service and accurate decisions regardless of where their claims for disability benefits are processed. To fulfill its commitment to quality service, it is imperative that VA continue to be vigilant in its quality assurance efforts, as this challenge will likely become even more difficult as aging veterans and veterans returning from ongoing conflicts add to VA’s workload.

Mr. Chairman, this concludes my prepared statement. I would be pleased to respond to any questions you or Members of the Subcommittee may have at this time.

GAO Contact and Staff Acknowledgments

For further information about this testimony, please contact Daniel Bertoni at (202) 512-7215 or bertonid@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. In addition to the contact named above, key contributors to this statement include Shelia Drake, Jessica Orr, Martin Scire, and Greg Whitney.

Related GAO Products

Veterans’ Disability Benefits: Further Evaluation of Ongoing Initiatives Could Help Identify Effective Approaches for Improving Claims Processing. GAO-10-213. Washington, D.C.: January 29, 2010.

Veterans’ Disability Benefits: Preliminary Findings on Claims Processing Trends and Improvement Efforts. GAO-09-910T. Washington, D.C.: July 29, 2009.

Military Disability System: Increased Supports for Servicemembers and Better Pilot Planning Could Improve the Disability Evaluation Process. GAO-08-1137. Washington, D.C.: September 24, 2008.

Veterans’ Disability Benefits: Better Accountability and Access Would Improve the Benefits Delivery at Discharge Program. GAO-08-901. Washington, D.C.: September 9, 2008.

Veterans’ Benefits: Improved Management Would Enhance VA’s Pension Program. GAO-08-112. Washington, D.C.: February 14, 2008.

Veterans’ Benefits: Further Changes in VBA’s Field Office Structure Could Help Improve Disability Claims Processing. GAO-06-149. Washington, D.C.: December 9, 2005.

Veterans Benefits: VA Needs Plan for Assessing Consistency of Decisions. GAO-05-99. Washington, D.C.: November 19, 2004.

VA Disability Benefits: Routine Monitoring of Disability Decisions Could Improve Consistency. GAO-06-120T. Washington, D.C.: October 20, 2005.

Veterans’ Benefits: Improvements Needed in the Reporting and Use of Data on the Accuracy of Disability Claims Decisions. GAO-03-1045. Washington, D.C.: September 30, 2003.

Veterans’ Benefits: Quality Assurance for Disability Claims and Appeals Processing Can Be Further Improved. GAO-02-806. Washington, D.C.: August 16, 2002.

Veterans’ Benefits: Quality Assurance for Disability Claims Processing. GAO-01-930R. Washington, D.C.: August 23, 2001.

Veterans’ Benefits Claims: Further Improvements Needed in Claims- Processing Accuracy. GAO/HEHS-99-35. Washington, D.C.: March 1, 1999.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.