Veterans' Disability Benefits: VA Has Improved Its Programs for Measuring Accuracy and Consistency, but Challenges Remain
Published: 2010-03-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: For years, in addition to experiencing challenges in making disability claims decisions more quickly and reducing its claims backlog, the Department of Veterans Affairs (VA) has faced challenges in improving the accuracy and consistency of its decisions. GAO was asked to discuss issues surrounding VA's Systematic Technical Accuracy Review (STAR) program, a disability compensation and pension quality assurance program, and possible ways, if any, this program could be improved. This statement focuses on actions VA has taken; including those in response to past GAO recommendations, to (1) address identified weaknesses with STAR and (2) improve efforts to monitor the consistency of claims decisions. This statement is based on GAO's prior work, which examined several aspects of STAR, as well as VA's consistency review activities, and on updated information GAO obtained from VA on quality assurance issues that GAO and VA's Office of Inspector General (OIG) have identified. GAO also reviewed VA's OIG March 2009 report on STAR. GAO is not making any new recommendations.
section_title: What GAO Found
paragraphs: Over the past several years, GAO has identified several deficiencies with the Veterans Benefit Administration's (VBA) STAR program, and although VBA has taken actions to address these issues, it continues to face challenges in improving claims accuracy. For example, GAO found that STAR reviewers lacked organizational independence, a basic internal control principle. In response to our finding, VA began utilizing organizationally independent reviewers that do not make claims decisions. GAO also found that sample sizes for pension claims were insufficient to provide assurance about decision accuracy. In response to GAO's recommendation, in fiscal year 2009, VA began increasing the number of pension claims decisions it reviews annually at each of its offices that process pension decisions. VA has also taken a number of other steps to address weaknesses that VA's OIG found in the STAR program, including (1) establishing minimum annual training requirements for reviewers and (2) requiring additional supervisory review of STAR reviewers' work. Although it has made or has started making these improvements, VBA remains challenged to improve its decision accuracy for disability compensation decisions, and it has not met its stated accuracy goal of 90 percent. VBA's performance has remained about the same over the past several fiscal years. In addition, VA has taken steps to address deficiencies that GAO and the VA's OIG have identified with consistency reviews--assessments of the extent to which individual raters make consistent decisions on the same claims. For example, in prior work, GAO reported that VA did not conduct systematic studies of impairments that it had identified as having potentially inconsistent decisions. In response to GAO's recommendation, in fiscal year 2008, VBA's quality assurance staff began conducting studies to monitor the extent to which veterans with similar disabilities receive consistent ratings across regional offices and individual raters. However, last year, VA's OIG reported that VA had not followed through on its plans to conduct such reviews. In response to this and other OIG findings and recommendations, VA took a number of actions, including developing an annual consistency review schedule and hiring additional quality assurance staff. However, VBA has only recently begun these programs to improve consistency, and it is too early to assess the effectiveness of their actions.

section_title: Letter
paragraphs: I am pleased to have the opportunity to comment on the Department of  Veterans Affairs’ (VA) efforts to improve the accuracy and consistency of  its disability compensation and pension benefit decisions. As we and other  organizations have reported over the last decade, VA’s claims processing  challenges are not limited to making decisions more quickly and reducing  its claims backlog; but also includes improving the accuracy and  consistency of its decisions. The number of veterans awaiting decisions  could grow as service members returning from ongoing conflicts and aging  veterans submit claims. According to VA, about 35 percent of veterans  from ongoing hostilities file claims. It is important not only that decisions  be timely, but also accurate. Accurate initial claims decisions can help  ensure that VA is paying cash disability benefits to those entitled to such  benefits and also help prevent lengthy appeals. Meanwhile, consistent  decisions help ensure that comparable medical conditions of veterans are  rated the same, regardless of which VA regional benefits office processes  the claim.
You asked us to discuss issues surrounding VA’s disability compensation  and pension quality assurance programs; particularly, the Systematic  Technical Accuracy Review (STAR) program. My statement focuses on  STAR, which deals with accuracy, and two other VA quality assurance  activities that focus on consistency. More specifically, my remarks will  focus on actions VA has taken to (1) address deficiencies identified with  STAR and (2) improve efforts to monitor the consistency of claim  decisions. This statement is based on our prior work, which examined  several aspects of STAR, as well as VA’s consistency review programs, and  on updated information we obtained from VA on quality assurance  vulnerabilities that we and VA’s Office of Inspector General (OIG) have  identified. We also reviewed VA OIG’s March 2009 report on STAR and  consistency reviews. Our work was conducted in accordance with  generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe the evidence obtained provides  a reasonable basis for our findings and conclusions based on our audit  objectives.
subsections: section_title: Background
paragraphs: Through its disability compensation program, VA pays monthly benefits to  veterans with service-connected disabilities. Under its disability pension  program, VA pays monthly benefits to low-income veterans who have  disabilities not related to their military service or are age 65 or older. VA  also pays compensation to the survivors of certain veterans who had  service-connected disabilities and of servicemembers who died while on  active duty.
Veterans and their survivors claim benefits at one of the Veterans Benefits  Administration’s (VBA) 57 regional offices. Once the claim is received, a  service representative assists the veteran in gathering the relevant  evidence to evaluate the claim. Such evidence includes the veteran’s  military service records, medical examinations, and treatment records  from VA medical facilities and private medical service providers. Also, if  necessary for reaching a decision on a claim, the regional office arranges  for the veteran to receive a medical examination. Once all necessary  evidence has been collected, a rating specialist evaluates the claim and  determines whether the claimant is eligible for benefits. If so, the rating  specialist assigns a percentage rating. Veterans with multiple disabilities  receive a single composite rating. Since 2001, VBA has created 15 resource  centers that are staffed exclusively to process claims or appeals from  backlogged regional offices. Most of these centers focus either on making  rating decisions, or on developing the information needed to evaluate  claims.
In addition to the traditional claims process, any member of the armed  forces who has seen active duty—including those in the National Guard or  Reserves—is eligible to apply for VA disability benefits prior to leaving  military service through VA’s Benefits Delivery at Discharge (BDD)  program or the related Quick Start program. In 2006, VA completed its  consolidation of BDD rating activity into its Salt Lake City, Utah, and  Winston-Salem, North Carolina, regional offices, to increase the  consistency of BDD claims decisions. Also, under the Department of  Defense (DOD)–VA disability evaluation system pilot program,  servicemembers undergoing disability evaluations, if found medically unfit  for duty, receive VA disability ratings. This rating covers both the unfitting  conditions identified by the military service and conditions identified by  the servicemember during the process. The rating is used by both DOD  and VA to determine entitlement for disability benefits.
Enacted in October 2008, the Veterans’ Benefits Improvement Act of 2008  required VA to contract for an independent, 3-year review of VBA’s quality  assurance program. This review is to include, among other items,  assessments of the accuracy of disability ratings and their consistency  across VA regional offices. VA contracted with the Institute for Defense  Analyses (IDA) to conduct this study. According to VA, IDA will provide  preliminary findings in the Summer of 2010, and VA is scheduled to report  to the Congress in October 2011.
subsections: section_title: STAR Program
paragraphs: Under the STAR program, which was implemented in fiscal year 1999, VBA  selects a random sample of completed claims decisions each month from  each of its regional offices to review for accuracy. STAR reviewers assess  decision accuracy using a standard checklist. For decisions affecting  benefit entitlement, this review includes an assessment of whether (1) all  issues in the claim were addressed; (2) assistance was provided to the  claimant, as required by the Veterans Claims Assistance Act of 2000; and  (3) the benefit entitlement decision was correct. If a claim has any error,  VBA counts the entire claim as incorrect for accuracy rate computation  purposes. The STAR reviewer then returns the case file and the results of  the review to the regional office that made the decision. If an error was  found, the regional office is required to either correct it or request  reconsideration of the error determination. VBA uses the national  accuracy rate from STAR reviews of compensation entitlement decisions  as one of its key claims processing performance measures. VA also uses  STAR data to estimate improper compensation and pension benefit  payments.
section_title: Consistency Review Activities
paragraphs: One VA consistency review activity involves conducting studies of regional  offices’ decisions on specific conditions such as post-traumatic stress  disorder where VBA found differences, such as in benefit grant rates,  across regional offices through comparative statistical analysis. VBA uses  the results of these reviews to identify root causes of inconsistencies and  to target training. Under another VA consistency review activity, called  inter-rater reliability reviews, VBA provides rating specialists a sample  case file to assess how well raters from various regional offices agree on  an eligibility determination when reviewing the same body of evidence.  These reviews allow VBA officials to target a single rating issue and take  remedial action to ensure the consistent application of policies and  procedures nationally.
section_title: VA Has Implemented Procedures to Address Deficiencies Identified with the STAR Program, but Continues to Face Challenges in Improving Accuracy
paragraphs: Over the past decade, VBA has taken several actions to improve its STAR  program and to address deficiencies identified by both GAO and VA’s OIG.  For example, in March 1999, we found that STAR review staff lacked  sufficient organizational independence because they were also responsible  for making claims decisions and reported to regional office managers  responsible for claims processing. In response to our findings, VBA took  steps to address this by utilizing reviewers who do not process claims and  who do not report to managers responsible for claims processing. More  recently, in February 2008, we found that STAR was not sampling enough  initial pension claims to ensure the accuracy of pension claims decisions.  Because initial pension claims constituted only about 11 percent of the  combined compensation and pension caseload subject to accuracy review,  few were likely to be included in the STAR review sample. We  recommended that VBA take steps to improve its quality assurance review  of initial claims, which could include reviewing a larger sample of pension  claims. According to VBA, it has addressed this issue by consolidating  pension claims processing in its three Pension Management Centers and  establishing a separate STAR sample for pension claims. During fiscal year  2009, VBA began reviewing more pension claim decisions and reported  that, for fiscal year 2009, its pension entitlement accuracy was 95 percent,  exceeding its goal.
In a September 2008 report, we noted that VA lacked sufficient and  specific performance measures for assessing the accuracy of decisions on  BDD claims and recommended that VA consider options for separately  estimating the accuracy of such claims decisions. VA conducted an  analysis of the costs of sampling pre-discharge claims as part of STAR and  concluded that the costs would outweigh possible, unquantifiable benefits.  VA also noted that the two sites that rate BDD claims surpassed the  national average in accuracy for claims overall. While generally  responsive to our recommendation, VA’s analysis did not specifically  review the accuracy of BDD claims relative to traditional claims.  Moreover, because BDD claims do not comprise all claims reviewed at the  two rating sites, we continue to believe VA’s analysis was not sufficient to  estimate the relative accuracy of BDD claims at these sites. While we  agree that the benefits of reviewing accuracy are difficult to measure, if VA  had better information on the accuracy of BDD claims, VA could use such  information to inform training and focus its monitoring efforts. In contrast,  VA currently performs STAR reviews that target rating decisions made by  its Baltimore and Seattle offices under the DOD-VA disability evaluation  system pilot program. Such a targeted review could also be conducted for  BDD claims.
In its March 2009 report, VA’s OIG also identified several deficiencies in  the STAR program and recommended corrective actions. The OIG found  that (1) regional offices did not always submit all requested sample cases  for review, (2) reviewers did not evaluate all documentation in sample  files, and (3) reviewers were not properly recording some errors. The OIG  also found that VBA was not conducting STAR reviews of redistributed  cases (for example, claims assigned to resource centers for rating). The  OIG reviewed a sample of redistributed claims and found that 69 percent  had accurate entitlement decisions, well below VBA’s reported rate of 87  percent for the 12-month period ending in February 2008. Further, the OIG  found that VBA did not have minimum training requirements for STAR  reviewers.
As of March 2010, VBA had taken actions to respond to all of the OIG’s  recommendations related to STAR, including (1) implementing procedures  to follow up on cases not submitted by regional offices; (2) adding a  mechanism to the STAR database to remind reviewers of key decision  points; (3) requiring a second-level review of STAR reviewers’ work; and  (4) establishing a requirement that STAR reviewers receive 80 hours of  training per year. In addition, during fiscal year 2009, based in part on the  OIG’s recommendation, VBA also began monitoring the accuracy of claims  decided by rating resource centers as it does for regional offices. As we  noted in our January 2010 report, VBA has significantly expanded its  practice of redistributing regional offices’ disability claims workloads in  recent years, and gathering timeliness and accuracy data on redistributed  claims could help VBA assess the effectiveness of workload redistribution.
In addition, as the Congress has provided more resources to VBA to  increase compensation and pension staffing, VBA has devoted more  resources to quality review. In fiscal year 2008, VBA more than doubled  the size of the quality assurance staff, allowing it to increase the scope of  quality assurance reviews. VA states that in the 12-month period ending in  May 2009, STAR staff reviewed over 14,000 compensation and pension  benefit entitlement decisions.
Although VBA has taken steps to address deficiencies in the STAR  program, the accuracy of its benefit entitlement decisions has not  improved. The accuracy rate was 86 percent in fiscal year 2008 and 84  percent in fiscal year 2009, well short of VBA’s fiscal year 2009 goal of 90  percent. VA attributed this performance to the relatively large number of  newly hired personnel conducting claims development work and a general  lack of training and experience. Human capital challenges associated with  providing the needed training and acquiring the experience these new  claims processors need to become proficient at their jobs will likely  continue in the near future. According to VBA officials, it can take 3 to 5  years for rating specialists to become proficient.
section_title: VA Has Taken Actions to Strengthen Efforts to Monitor Consistency of Claims Decisions
paragraphs: VA has taken actions to address deficiencies identified with its consistency  review programs, but it is still too early to determine whether these  actions will be effective. In prior work, we reported that VBA did not  systematically assess the consistency of decision making for any specific  impairments included in veterans’ disability claims. We noted that if rating  data identified indications of decision inconsistency, VA should  systematically study and determine the extent and causes of such  inconsistencies and identify ways to reduce unacceptable levels of  variations among regional offices. Based on our recommendation, VBA’s  quality assurance staff began conducting studies to monitor the extent to  which veterans with similar disabilities receive consistent ratings across  regional offices and individual raters. VBA began these studies in fiscal  year 2008. VBA identified 61 types of impairments for consistency review  and conducted at least two inter-rater reliability reviews, which found  significant error rates.
In its March 2009 report, the OIG noted that, while VBA had developed an  adequate rating consistency review plan, including metrics to monitor  rating consistency and a method to identify variances in compensation  claim ratings, it had not performed these reviews as scheduled. In fact,  VBA had initiated only 2 of 22 planned consistency reviews in fiscal year  2008. The OIG reported that VBA had not conducted these reviews  because STAR staffing resources were not sufficient to perform all of their  assigned responsibilities and noted that VBA’s quality review office had  not staffed all of its authorized positions. In addition, the OIG found that  inter-rater reliability reviews were not included in VBA’s quality assurance  plan. The OIG recommended that VBA (1) develop an annual rating  consistency review schedule and complete all planned reviews as  scheduled; (2) dedicate sufficient staff to conduct consistency reviews in  order to complete planned workload and reviews; and (3) include inter- rater reliability reviews as a permanent component of its consistency  review program.
VBA reported that it has developed an annual consistency review schedule  and is in the process of conducting scheduled fiscal year 2010 reviews. As  of January 2010, VBA also added six staff members to perform quality  assurance reviews. Further, VBA incorporated inter-rater reliability  reviews into its fiscal year 2009 quality assurance plan. Because VBA has  only recently implemented these initiatives, it is too early to determine  their impact on the consistency of claims decisions.
section_title: Conclusion
paragraphs: Over the years, VA has been challenged in its efforts to ensure that  veterans get the correct decisions on disability claims the first time they  apply for them, regardless of where the claims are decided. Making  accurate, consistent, and timely disability decisions is not easy, but it is  important. Our veterans deserve timely service and accurate decisions  regardless of where their claims for disability benefits are processed. To  fulfill its commitment to quality service, it is imperative that VA continue  to be vigilant in its quality assurance efforts, as this challenge will likely  become even more difficult as aging veterans and veterans returning from  ongoing conflicts add to VA’s workload.
Mr. Chairman, this concludes my prepared statement. I would be pleased  to respond to any questions you or Members of the Subcommittee may  have at this time.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: For further information about this testimony, please contact Daniel  Bertoni at (202) 512-7215 or bertonid@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this testimony. In addition to the contact named above, key  contributors to this statement include Shelia Drake, Jessica Orr, Martin  Scire, and Greg Whitney.
section_title: Related GAO Products
paragraphs: Veterans’ Disability Benefits: Further Evaluation of Ongoing Initiatives  Could Help Identify Effective Approaches for Improving Claims  Processing. GAO-10-213. Washington, D.C.: January 29, 2010.
Veterans’ Disability Benefits: Preliminary Findings on Claims  Processing Trends and Improvement Efforts. GAO-09-910T. Washington,  D.C.: July 29, 2009.
Military Disability System: Increased Supports for Servicemembers and  Better Pilot Planning Could Improve the Disability Evaluation Process.  GAO-08-1137. Washington, D.C.: September 24, 2008.
Veterans’ Disability Benefits: Better Accountability and Access Would  Improve the Benefits Delivery at Discharge Program. GAO-08-901.  Washington, D.C.: September 9, 2008.
Veterans’ Benefits: Improved Management Would Enhance VA’s Pension  Program. GAO-08-112. Washington, D.C.: February 14, 2008.
Veterans’ Benefits: Further Changes in VBA’s Field Office Structure  Could Help Improve Disability Claims Processing. GAO-06-149.  Washington, D.C.: December 9, 2005.
Veterans Benefits: VA Needs Plan for Assessing Consistency of  Decisions. GAO-05-99. Washington, D.C.: November 19, 2004.
VA Disability Benefits: Routine Monitoring of Disability Decisions  Could Improve Consistency. GAO-06-120T. Washington, D.C.: October 20,  2005.
Veterans’ Benefits: Improvements Needed in the Reporting and Use of  Data on the Accuracy of Disability Claims Decisions. GAO-03-1045.  Washington, D.C.: September 30, 2003.
Veterans’ Benefits: Quality Assurance for Disability Claims and Appeals  Processing Can Be Further Improved. GAO-02-806. Washington, D.C.:  August 16, 2002.
Veterans’ Benefits: Quality Assurance for Disability Claims Processing.  GAO-01-930R. Washington, D.C.: August 23, 2001.
Veterans’ Benefits Claims: Further Improvements Needed in Claims- Processing Accuracy. GAO/HEHS-99-35. Washington, D.C.: March 1, 1999.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.