VA Disability Benefits: Board of Veterans' Appeals Has Made Improvements in Quality Assurance, but Challenges Remain for VA in Assuring Consistency
Published: 2005-05-05T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The House Subcommittee on Disability Assistance and Memorial Affairs asked GAO to update a 2002 study to determine what VA has done to (1) correct reported weaknesses in methods used by the Board to select decisions for quality review and calculate the accuracy rates reported by the Board and (2) address the potential for inconsistency in decision-making at all levels of adjudication in VA, including VA's 57 regional offices and the Board. GAO said in 2002 that VA had not studied consistency even though adjudicator judgment is inherently required in the decision-making process, and state-to-state variations in the average disability compensation payment per veteran raised questions about consistency. In January 2003, in part because of concerns about consistency, GAO designated VA's disability program as high-risk.
section_title: What GAO Found
paragraphs: The Department of Veterans Affairs (VA) has taken steps to respond to GAO's 2002 recommendations to correct weaknesses in the methods for selecting decisions by the Board of Veterans' Appeals (Board) for quality review and calculating the accuracy rates reported by the Board. Specifically, the Board now ensures that decisions made near the end of the fiscal year are included in the quality review sample, and the Board now excludes from its accuracy rate calculations any errors that do not have the potential for resulting in a reversal by or remand from the court. GAO found that the Board had not yet revised its formula for calculating accuracy rates in order to properly weight the quality review results for original Board decisions versus the results for Board decisions on cases remanded by the court. However, GAO believes correcting this calculation method will not materially affect the Board's reported accuracy rates. VA still lacks a systematic method for ensuring the consistency of decision-making within VA as a whole, but has begun efforts to understand why average compensation payments per veteran vary widely from state to state. These efforts include studies underway by VA's Office of Inspector General and the Veterans Benefits Administration, which oversees the operations of VA's regional offices. Some variation is expected since adjudicators often must use judgment in making disability decisions, but VA faces the challenge of determining whether the extent of variation is confined within a range that knowledgeable professionals could agree is reasonable.

section_title: Letter
paragraphs: Thank you for inviting me to discuss our work related to the efforts of the  Department of Veterans Affairs (VA) to assure the quality and consistency  of disability decisions. Assuring the quality and consistency of VA’s  disability decisions is vital for assuring program integrity and equitable  decisions. As you know, in January 2003, we designated VA’s disability  program, along with other federal disability programs, as high-risk. In  part, we designated VA’s program as high-risk because of concerns about  consistency of decision-making. Despite VA’s efforts to provide training  and enhance communication to improve the consistency of decisions, we  found indications of inconsistency such as the wide variation among states  in the average compensation payment per veteran.
You asked us to update our 2002 report in which we found that the Board  needed to improve its quality review program. Specifically, the Board  needed to revise its sampling methods, the way it weighted quality review  results in its calculation of accuracy rates, and the types of errors reported  in the accuracy rates. In that report, we also found that VA needed to take  action to assure consistency of decision-making within VA as a whole.  Today I would like to highlight the steps the Board has taken since our  2002 report and discuss VA’s recent efforts to address inconsistent  decision-making. To update our 2002 report, we interviewed officials of  and/or obtained pertinent documentation from the Board and the Veterans  Benefits Administration (VBA), which oversees the operations of VA’s 57  regional offices. We obtained comments from Board officials on the  updated information contained in this testimony regarding the Board's  quality assurance system, and they agreed with this information. We did  our work in accordance with generally accepted government auditing  standards in April 2005.
In summary, we found that the Board has taken action to strengthen its  system for reviewing the quality of its own decisions, but VA still lacks a  systematic method for ensuring the consistency of decision-making within  VA as a whole. With respect to the Board, it has taken steps to improve the  sampling and accuracy rate calculation methods of its quality review  system and also to assure that serious errors are not obscured by mixing  them with less significant deficiencies. Regarding consistency, VA still  lacks a plan for an ongoing, systematic assessment of decision-making  consistency at all levels of adjudication within VA. As we concluded in  2002, such an assessment is needed to provide a foundation for  determining acceptable levels of decision-making variation and to reduce  variations found to be unacceptable. Although VA has much left to do  regarding consistency of decision-making, it has begun efforts to  understand the reasons behind one indication of inconsistency: the wide  variations from state to state in the average compensation payment per  veteran. These efforts include studies being done by VA’s Office of  Inspector General and VBA.
subsections: section_title: Background: The Disability Claims and Appeals Process
paragraphs: VA’s disability compensation program pays monthly cash benefits to  eligible veterans who have service-connected disabilities resulting from  injuries or diseases incurred or aggravated while on active military duty.  The benefit amount is based on the veteran’s degree of disability,  regardless of employment status or level of earnings.
A veteran starts the claims process by submitting a disability  compensation claim to one of the 57 regional offices administered by VBA  (see fig. 1). In the average disability compensation claim, the veteran  claims about five disabilities. For each claimed disability, the regional  office adjudicator must develop evidence and determine whether each  disability is connected to the veteran’s military service. The adjudicator  then applies the medical criteria in VA’s Rating Schedule to evaluate the  degree of disability caused by each service-connected disability, and then  the adjudicator determines the veteran’s overall degree of service- connected disability.
If a veteran disagrees with the adjudicator’s decision on any of the claimed  disabilities, the veteran may file a Notice of Disagreement. If the regional  office is unable to resolve the disagreement to the veteran’s satisfaction,  the veteran may appeal to the Board. A veteran can dispute a decision not  only if the regional office denies benefits by deciding that an impairment  claimed by the veteran is not service-connected. Even for a claimed  impairment found to be service-connected, the veteran may dispute the  severity rating that the regional office assigns to the impairment and ask  for an increase in the rating.
During fiscal years 2003 and 2004, respectively, the regional offices made  about 715, 000 and 598,500 decisions involving disability compensation  claims. According to VBA, during fiscal years 2003 and 2004, respectively,  veterans submitted Notices of Disagreement in about 13.4 and 14.5 percent  of all decisions involving disability ratings, and of the veterans who filed  Notices of Disagreement, about 34.9 and 44.4 percent went on to submit  appeals to the Board. Assisted by 240 staff attorneys, the Board’s 52  veterans law judges decide veterans’ appeals on behalf of the Secretary.  The Board has full de novo review authority and gives no deference to the  regional office decision being appealed. The Board makes its decisions  based on only the law, VA’s regulations, precedent decisions of the courts,  and precedent opinions of VA’s General Counsel. During the appeals  process, the veteran or the veteran’s representative may submit new  evidence to the Board and request a hearing.
In fiscal year 2004, for all VA programs, the Board decided about 38,400  appeals, of which about 94 percent (35,900) were appeals of disability  compensation cases that contained an average of 2.2 contested issues per  case. In any given case, the Board might grant the requested benefits for  one issue but deny benefits for another. In some instances, the Board may  find that a case is not ready for a final decision and return (or remand) the  case to VBA for rework, such as obtaining additional evidence and  reconsidering the veteran’s claim. If VBA still does not grant the requested  benefits after obtaining the additional evidence, it returns the case to the  Board for a final decision. Of the appeals involving compensation cases  decided during fiscal year 2004, the Board reported that it granted  requested benefits for at least one issue in about 18 percent of the cases,  denied all requested benefits in about 23 percent of the cases, and  remanded about 58 percent of the cases to VBA for rework.
Effective February 22, 2002, VA issued a new regulation to streamline and  expedite the appeals process. Previously, the Board had remanded all  decisions needing rework directly to VBA’s regional offices. The new  regulation, however, allowed the Board to obtain evidence, clarify  evidence, cure a procedural defect, or perform almost any other action  essential for a proper appellate decision without having to remand the  appeal to the regional office. It also allowed the Board to consider  additional evidence without having to refer the evidence to the regional  office for initial consideration and without having to obtain the appellant’s  waiver. According to the Board, this change in the process reduced the  time required to provide a final decision to the veteran on an appeal,  allowed regional offices to use more resources for processing initial claims  rather than remands, and virtually eliminated multiple remands on the  same case to the regional offices. However, in May 2003, the U.S. Court of  Appeals for the Federal Circuit held that the Board could not, except in  certain statutorily authorized exceptions, decide appeals in cases in which  the Board had developed evidence. As a result, VA established a  centralized Appeals Management Center within VBA in Washington, D.C.,  to take over evidence development and adjudication work on remands.
If the Board denies requested benefits or grants less than the maximum  benefit available under the law, veterans may appeal to the U. S. Court of  Appeals for Veterans Claims, an independent federal court. Unlike the  Board, the court may not receive new evidence. It considers only the  Board’s decision; briefs submitted by the veteran and VA; oral arguments,  if any; and the case record that VA considered and that the Board had  available. In cases decided on merit (cases not dismissed on procedural  grounds), the court may (1) reverse the Board’s decision (grant contested  benefits), (2) affirm the Board’s decision (deny contested benefits) or (3)  remand the case back to the Board for rework. Of the 3,489 cases decided  on merit during fiscal years 2003-2004, the court reversed or remanded in  part or in whole about 88 percent of the cases. Under certain  circumstances, a veteran who disagrees with a decision of the court may  appeal to the U.S. Court of Appeals for the Federal Circuit and then to the  Supreme Court of the United States.
section_title: The Board Has Taken Steps to Improve Its Quality Assurance Program
paragraphs: The Board of Veterans’ Appeals has taken action to strengthen its internal  system for reviewing the quality of its own decisions. Specifically, the  Board has taken steps to improve its quality review system’s sampling  methodology and to avoid obscuring serious errors by mixing them with  less significant deficiencies. We found, however, that the Board still needs  to revise its formula for calculating accuracy rates in order to avoid  potentially misleading accuracy rates.
During our 2002 evaluation, we reviewed the Board’s methods for  selecting random samples of Board decisions and calculating accuracy  rates for its decisions. We found that the number of decisions reviewed  was sufficient to meet the Board’s goal for statistical precision in  estimating its accuracy rate. However, we pointed out some Board  practices that might result in misleading accuracy rates. These practices  included not ensuring that decisions made near the end of the fiscal year  were sampled and not properly weighting quality review results in the  formula used to calculate accuracy rates. At the time of our 2002 report,  the Board had agreed in principle to correct these practices.
We found in our most recent work that the Board took corrective action in  fiscal year 2002 to assure that decisions made near the end of the fiscal  year were sampled. The quality review program now selects every 20th  original decision made by the Board’s veterans law judges and every 10th  decision they make on cases remanded by the court to the Board for  rework. However, we found that the Board had not revised its formula for  calculating accuracy rates in order to properly weight the quality review  results for original decisions versus the results for decisions on remanded  cases. We determined that, even if this methodological error had been  corrected earlier, the accuracy rate reported by the Board for fiscal year  2004 (93 percent) would not have been materially different. However, to  avoid the potential for reporting a misleading accuracy rate in the future,  corrective action needs to be taken, and the Board agreed to correct this  issue in the very near future.
In our 2002 evaluation, we also found that the Board included  nonsubstantive deficiencies (errors that would not be expected to result in  either a remand by the court or a reversal by the court) in calculating its  reported accuracy rates. We concluded that the reported accuracy rates  understated the level of accuracy that would result if the Board, like VBA,  counted only substantive deficiencies in the accuracy rate calculation.
VBA had ceased counting nonsubstantive deficiencies in its error rate after  the VA Claims Processing Task Force said in 2001 that mixing serious  errors with less significant deficiencies could obscure what is of real  concern. Similarly, we recommended that the Board’s accuracy rates take  into account only those deficiencies that would be expected to result in a  reversal or a remand by the court. In fiscal year 2002, the Board  implemented our recommendation.
Also, during the course of our 2002 evaluation of the quality review  program, we brought to the Board’s attention the governmental internal  control standard calling for separation of key duties and the governmental  performance audit standard calling for organizational independence for  agency employees who review and evaluate program performance. These  issues arose because certain veterans law judges who were directly  involved in deciding veterans’ appeals were also involved in reviewing the  accuracy of such decisions. The Board took corrective actions during our  review in May 2002 to resolve these issues so all quality reviews from  which accuracy rates are determined are done by persons not directly  involved in deciding veterans’ appeals.
In 2002, we also found that the Board collected and analyzed issue-specific  data on the reasons that the Court remanded decisions to the Board in  order to provide feedback and training to the Board’s veterans law judges;  however, the Board did not collect issue-specific data on the errors that its  own quality reviewers found in decisions of the Board’s veterans law  judges. We recommended that the Board revise its quality review program  to begin collecting such issue-specific error data in order to identify  training that could help improve decision quality. In April 2005, the Board  said it did not implement this recommendation because it believes the  benefits would be too limited to justify the substantial reprogramming of  the data system that would be required to collect issue-specific data. The  Board also pointed out that the issue-specific data captured for court  remands have not proved to be as useful as it had expected in identifying  ways to provide training that could reduce court remands.
section_title: VA Faces Challenges in Measuring Consistency But Has Recently Initiated Studies
paragraphs: Adjudicator judgment is an inherent factor in deciding disability claims,  and it introduces the potential for variation in the process. Part of  assessing inconsistency, as we recommended in 2002, would include  determining acceptable levels of variation for specific types of disabilities.  In late 2004, in response to adverse media reports, VA initiated its first  study of consistency. Such studies are the first step in determining the  degree of variation that occurs and what levels of variation are acceptable.
subsections: section_title: Adjudicator Judgment Results in Inherent Variation in Decision- making
paragraphs: Adjudicators often must use judgment in making disability decisions.  Judgment is particularly critical when the adjudicator must (1) assess the  credibility of different sources of evidence; (2) evaluate how much weight  to assign different sources of evidence; or (3) assess some disabilities,  such as mental disorders, for which the disability standards are not  entirely objective and require the use of professional judgment. In such  cases, two adjudicators reviewing the same evidence might make differing  judgments on the meaning of the evidence and reach different decisions,  neither of which would necessarily be found in error by any of VA’s quality  reviewers.
For example, in an illustration provided by the Board, consider a disability  claim that has two conflicting medical opinions, one provided by a medical  specialist who reviewed the claim file but did not examine the veteran, and  a second opinion provided by a medical generalist who reviewed the file  and examined the veteran. One adjudicator could assign more weight to  the specialist’s opinion, while another could assign more weight to the  opinion of the generalist who examined the veteran. Depending on which  medical opinion is given more weight, one adjudicator could grant the  claim and the other could deny it. Yet, a third adjudicator could apply VA’s  “benefit-of-the-doubt” rule and decide in favor of the veteran. Under this  rule, if an adjudicator concludes that there is an approximate balance  between the evidence for and the evidence against a veteran’s claim, the  adjudicator must decide in favor of the veteran.
In the design of their quality review systems, VBA and the Board  acknowledge the fact that, in some cases, different adjudicators reviewing  the same evidence can make differing, but reasonable, judgments on the  meaning of the evidence. As a result, VBA and the Board instruct their  quality reviewers that when they review a decision, they are not to record  an error merely because they would have made a different decision than  the one made by the adjudicator. VBA and the Board instruct their quality  reviewers to not substitute their own judgment in place of the original  adjudicator’s judgment if the adjudicator’s decision is adequately  supported and reasonable.
Another example provided by the Board demonstrates how adjudicators  must make judgments about the degree of severity of a disability. VA’s  disability criteria provide a formula for rating the severity of a veteran’s  occupational and social impairment due to a variety of mental disorders.  This formula is a nonquantitative, behaviorally oriented framework for  guiding adjudicators in choosing which of the degrees of severity shown in  table 1 best describes the claimant’s occupational and social impairment.
Similarly, VA does not have objective criteria for rating the degree to  which certain spinal impairments limit a claimant’s motion. The  adjudicator must assess the evidence and decide whether the limitation of  motion is “slight, moderate, or severe.” To assess the severity of  incomplete paralysis, the adjudicator must decide whether the veteran’s  paralysis is “mild, moderate, or severe.” The decision on which severity  classification to assign to a claimant’s condition could vary in the minds of  different adjudicators, depending on how they weigh the evidence and  how they interpret the meaning the of the different severity classifications.
Consequently, it would be unreasonable to expect that no decision-making  variations would occur. But it is reasonable to expect the extent of  variation to be confined within a range that knowledgeable professionals  could agree is reasonable, recognizing that disability criteria are more  objective for some disabilities than for others. For example, if two  adjudicators were to review the same claim file for a veteran who has  suffered the anatomical loss of both hands, VA’s disability criteria state  unequivocally that the veteran is to be given a 100 percent disability rating.  Therefore, no variation would be expected. However, if two adjudicators  were to review the same claim file for a veteran with a mental disability,  knowledgeable professionals might agree that it would not be out of the  bounds of reasonableness if one adjudicator gave the claimant a 50  percent disability rating and the other adjudicator gave a 70 percent rating.  However, knowledgeable professionals might also agree that it would be  clearly outside the bounds of reasonableness if one adjudicator gave the  claimant a 30 percent rating and the other, a 100 percent rating.
section_title: VA Began Studying Decision-making Consistency in December 2004
paragraphs: Although the issue of decision-making consistency is not new, VA only  recently began to study consistency issues. In a May 2000 testimony  before the House Subcommittee on Oversight and Investigations,  Committee on Veterans’ Affairs, we underscored the conclusion made by  the National Academy of Public Administration in 1997 that VBA needed  to study the consistency of decisions made by different regional offices,  identify the degree of subjectivity expected for various medical issues, and  then set consistency standards for those issues. In August 2002, we drew  attention to the fact that there are wide disparities in state-to-state average  compensation payments per disabled veteran, and we voiced the concern  that such variation raises the question of whether similarly situated  veterans who submit claims to different regional offices for similar  conditions receive reasonably consistent decisions. In January 2003, we  reported that concerns about consistency had contributed to GAO’s  designation of the VA disability program as high-risk in 2003. Again, in  November 2004, we highlighted the need for VA to develop plans for  studying consistency issues.
Most recently, in December 2004, the media drew attention to the wide  variations in the average disability compensation payment per veteran in  the 50 states and published data showing that the average payments varied  from a low of $6,710 in Ohio to a high of $10,851 in New Mexico. Reacting  to these media reports, in December 2004, the Secretary instructed the  Inspector General to determine why average payments per veteran vary  widely from state to state. As of February 2005 the Office of Inspector  General planned to use data obtained from VBA for all regional offices to  identify factors that may explain variations among the regional offices.
In March 2005, VBA began a study of three disabilities believed to have  potential for inconsistency: hearing loss, post-traumatic stress disorder,  and knee conditions. VBA assigned 10 subject matter experts to review  1,750 regional office decisions and plans to complete its analysis of study  data in mid-May 2005, develop a schedule for future studies of specific  ratable conditions, and recommend a schedule for periodic follow-up  studies of previously studied conditions.
In our 2002 report, we recommended that VA establish a system to  regularly assess and measure the degree of consistency across all levels of  VA adjudication, including regional offices and the Board, for specific  medical conditions that require adjudicators to make difficult judgments.  For example, we said VA could create hypothetical claims for certain  medical conditions, distribute the claims to multiple adjudicators at each  decision-making level, and analyze variations in outcomes. Such a system  would identify variation in decision making and provide a basis to identify  ways, if considered necessary, to reduce variation through training or  clarifying and strengthening regulations, procedures, and policies.
Although VA agreed in principle with our recommendation and agreed that  consistency is an important goal, it commented that it would promote  consistency through training and communication. We support such efforts  but still believe VA needs to directly evaluate and measure consistency  across all levels of adjudication. Otherwise, VA cannot determine whether  such training and other efforts are directed at the causes of inconsistency  and whether such efforts actually improve consistency.
In our November 2004 report, we found that VBA’s administrative data  was insufficient to analyze inconsistency because we could not reliably  use the data to identify decisions made after fiscal year 2000, identify the  regional offices that made the original decisions, or determine service- connection denial rates for specific impairments. However, in October  2004, VBA completed its implementation of a new nationwide data system,  known as Rating Board Automation (RBA) 2000. VA said this new system  could reliably collect the types of data needed to perform the analyses we  sought to do. Therefore, we recommended that the Secretary of Veterans  Affairs develop a plan, and include it in VA’s annual performance plan,  containing a detailed description of how VA intended to use data from the  new RBA 2000 information system. We recommended that VA conduct  systematic studies of the impairments for which RBA 2000 data reveal  indications of decision-making inconsistencies among regional offices. VA  concurred with our recommendation. Because the new RBA 2000 data  system had been recently implemented, we acknowledged that VA could  not implement such a plan until it accumulated a sufficiently large body of  data under the new system. In our judgment, at least one year of data  would be needed to begin such a study.
While we believe the studies recently begun by the Office of Inspector  General and VBA are positive steps forward in addressing consistency  issues, the RBA 2000 data system, if found to be reliable, can provide VA  with the data needed to proactively and systematically target specific  impairments that have the widest variations in decision-making outcomes  among the regional offices and focus VA’s efforts to study reasons for  variations on those impairments. Building in such analytical capability to  augment its quality assurance program would help enhance program  integrity and better assure that veterans’ disability decisions are made  fairly and equitably.
Mr. Chairman, this concludes my remarks. I would be happy to answer any  questions you or the members of the subcommittee may have.
section_title: Contact and Acknowledgements
paragraphs: For further information, please contact Cynthia A. Bascetta at (202) 512- 7101. Also contributing to this statement were Irene Chu, Ira Spears,  Martin Scire, and Tovah Rom.
section_title: Related GAO Products
paragraphs: Veterans Benefits: VA Needs Plan for Assessing Consistency of  Decisions. GAO-05-99. Washington, D.C.: November 19, 2004.
High-Risk Series: An Update. GAO-03-119. Washington, D.C.: January  2003.
Veterans’ Benefits: Quality Assurance for Disability Claims and Appeals  Processing Can Be Further Improved. GAO-02-806. Washington, D.C.:  August 16, 2002.
SSA and VA Disability Programs: Re-Examination of Disability  Criteria Needed to Help Ensure Program Integrity. GAO-02-597.  Washington, D.C.: August 9, 2002.
Veterans Benefits Administration: Problems and Challenges Facing  Disability Claims Processing. GAO/T-HEHS/AIMD-00-146. Washington,  D.C.: May 18, 2000.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.