Veterans' Benefits: Improvements Needed in the Reporting and Use of Data on the Accuracy of Disability Claims Decisions
Released: 2003-09-30T00:00:00

Letter

Background

Through its disability compensation program, VBA pays monthly benefits to veterans with service-connected disabilities (injuries or diseases incurred or aggravated while on active military duty) according to the severity of the disability. VBA’s pension benefit program pays monthly benefits to wartime veterans who have low incomes and are permanently and totally disabled for reasons not service-connected. In addition, VBA pays dependency and indemnity compensation to some deceased veterans’ spouses, children, and parents and to survivors of service members who died on active duty. In fiscal year 2002, VBA paid over $22 billion in disability compensation to an average of about 2.4 million veterans and over 300,000 survivors. VBA also paid over $3 billion in pensions to an average of about 580,000 veterans and survivors.

The amount of disability compensation largely depends on the degree to which a veteran is disabled. VBA determines the degree to which veterans are disabled in 10 percent increments on a scale of 0 to 100 percent. Basic monthly payments range from $104 for 10 percent disability to $2,193 for 100 percent disability. About 65 percent of veterans receiving disability compensation have disabilities rated at 30 percent and lower; about 8 percent have disabilities rated at 100 percent. The most common impairments for veterans who began receiving compensation in fiscal year 2001 were tinnitus (ex., a ringing in the ear), auditory acuity impairment rated at 0 percent (i.e., mild hearing difficulties), skeletal conditions, arthritis due to trauma, post-traumatic stress disorder, and scars. Eligibility and priority for other Department of Veterans Affairs (VA) benefits and services such as health care and vocational rehabilitation are affected by these VA disability ratings.

When a veteran submits a claim to any of VBA’s 57 regional offices, a veterans service representative (VSR) is responsible for obtaining the relevant evidence to evaluate the claim. Such evidence includes veterans’ military service records, medical examinations and treatment records from VA medical facilities, and treatment records from private medical service providers. Once a claim is developed (i.e., has all the necessary evidence), a rating VSR, also called a rating specialist, evaluates the claim, determines whether the claimant is eligible for benefits, and assigns a rating based on degree of disability. Veterans with multiple disabilities receive a single composite rating. For veterans claiming pension eligibility, the regional office determines if the veteran served in a period of war, is permanently and totally disabled for reasons not service-connected, and meets the income thresholds for eligibility. A veteran who disagrees with the regional office’s decision for either program can appeal sequentially to VA’s Board of Veterans’ Appeals (BVA), the U.S. Court of Appeals for Veterans Claims, and the U.S. Court of Appeals for the Federal Circuit.

In fiscal year 1999, VBA implemented a quality review system to improve the measurement of the accuracy of its claims decisions. Under STAR, VBA selects a random sample of completed claims decisions each month from each of its 57 regional offices. STAR quality review staff review these claims using a standard checklist. If a claim has any error, VBA counts the entire claim as incorrect for accuracy rate computation purposes. STAR then returns the case file and the results of the review to the regional office that made the decision. If an error was found, the regional office is required to either correct it or request reconsideration of the error by STAR.

VBA made several significant changes to its STAR system in fiscal year 2002. First, in response to our March 1999 recommendation, VBA made the STAR review staff organizationally independent of the regional offices to ensure the independence of the reviews. Second, VBA more than doubled the number of rating decisions it reviewed (from 3,209 in fiscal year 2001 to 6,646 in fiscal year 2002) in order to obtain more precise regional office level accuracy scores. Third, VBA improved its key measure to focus more on whether decisions to grant or deny benefits were correct. Previously, VBA’s key measure also included decision documentation and notification issues. Figure 1 shows the areas covered by the checklist used to measure accuracy in fiscal year 2001. The revised key accuracy measure includes only the first four areas of the checklist.

Accuracy Declined in Fiscal Year 2002

From fiscal years 2001 to 2002, VBA’s accuracy of decision making in the compensation and pension programs declined from 89 percent to 81 percent. We determined these accuracy rates by comparing substantially similar questions for the 2 fiscal years that focused on whether the decision to grant or deny benefits was correct. However, the agency reported that it had improved its accuracy for that period. This discrepancy occurred because VBA did not report comparable accuracy data for the 2 fiscal years. VBA reported its fiscal year 2002 accuracy based on a revised measure that excluded two areas of the checklist—reasons and bases and notification. VBA then compared this to its fiscal year 2001 accuracy, which included these two areas of the checklist. VBA officials attributed the decline in accuracy during this period to several factors. We were not able to quantify the relative contribution of these factors. These factors included headquarters emphasis on production, the specific processing requirements of the Veterans Claims Assistance Act of 2000, and the relative inexperience of VBA’s claims processing staff.

In fiscal year 2002, VBA made progress in increasing production of claims decisions and reducing inventory. The number of claims decisions rose substantially from about 481,000 in fiscal year 2001 to about 797,000 in fiscal year 2002. During the same time period, the number of claims VBA received increased from about 674,000 to 722,000. The higher level of production permitted VBA to reduce its inventory of pending claims by 18 percent from about 421,000 to about 346,000.

Although accuracy actually declined, VBA reported in its performance report to the Congress for fiscal year 2002 that the accuracy of its rating decisions for compensation and pension benefits had improved slightly. VBA reported that accuracy improved from 78 percent in fiscal 2001 to 80 percent in fiscal 2002. However, these rates were not comparable because the new “benefit entitlement accuracy” rate for fiscal year 2002 was based on fewer areas of the checklist that focused on whether and to what extent the claimant received a benefit. In its fiscal year 2002 performance report, VA made note of this change but did not recalculate the fiscal 2001 rate to provide a correct comparison. When we recomputed the agency’s accuracy rate for fiscal 2001 using its revised criteria, the result was 89 percent, not 78 percent as reported. As a result, the correct comparison showed a decline in accuracy from 89 percent to 81 percent. The agency corroborated our finding after performing its own computation at our request.

VBA officials we spoke with cited a number of factors that affected accuracy over the fiscal years 2001 to 2002 period. A key factor cited was VBA’s emphasis on production, such that accuracy became a lesser priority. In 2001, the Secretary of Veterans Affairs set an ambitious goal to reduce the agency’s average time to complete a rating decision from 173 days (in fiscal year 2000) to 100 days by the end of fiscal year 2003. Also, the Secretary tasked VBA with reducing the inventory of rating claims to 250,000 by the end of September 2003. VBA managers responded by emphasizing the completion of more rating cases each month. The result has been a significant increase in production and a significant reduction in the rating inventory since the end of fiscal year 2001.

In fiscal year 2002, VBA incorporated its new production targets into its measures for regional office performance. Regional office directors became accountable for specific targets for production, inventory reduction, and timeliness improvement. The agency also established regional office performance award criteria that gave more weight to efficiency than accuracy. Three of the four criteria for cash awards were based on production and timeliness and one on accuracy; bonuses could be received without meeting the accuracy criteria.

VBA officials also attributed accuracy problems to the addition of more specific processing requirements under the Veterans Claims Assistance Act (VCAA) of 2000. The act broadened VBA’s responsibility to assist veterans — such as notifying them of needed evidence and helping them develop that evidence for their claims. VBA incorporated VCAA requirements into its STAR rating accuracy checklist in fiscal year 2002. In the first 4 months of fiscal year 2002, VCAA errors accounted for almost half of all errors identified by STAR rating reviewers. In April 2002, VBA required each regional office to re-train its claims processing staff on the new requirements. Over the last 8 months of fiscal year 2002, the proportion of VCAA-related errors declined to about one-third of all errors.

VBA officials also cited the relative inexperience of VBA’s claims processing staff. Regional office officials noted that they had many VSRs and rating specialists with insufficient training and experience to be fully proficient at developing and making decisions on claims. For example, officials at two offices we visited noted that their accuracy was affected by large numbers of inexperienced staff, due to a large increase in staff size or a relatively high rate of turnover. VBA has found that attrition rates for new claims processing staff hired over a 3-year period ranged from 0 percent to 49 percent at some regional offices.

VBA Could Have Done More to Hold Regional Offices Accountable for Accuracy

While VBA has established accuracy standards for its regional offices, it has not made the best use of its accuracy data on regional offices. VBA collected regional office accuracy data in fiscal year 2002 but has not fully used the information to evaluate regional office performance, correct errors, and identify needed training that could reduce errors. Unless VBA better uses these data to hold regional offices accountable for accuracy, it cannot ensure that as decisions become timelier, they also become more accurate. VBA is developing an agencywide system for evaluating individual claims processing employees’ performance against VBA’s individual accuracy standards.

In fiscal year 2002, VBA improved its STAR review system to provide independent review of decisions and more precise accuracy data at the regional office level. Regional accuracy scores are estimates based on samples of cases and revealed high and low performers relative to the agency’s fiscal year 2002 accuracy goal of 85 percent. Eleven regional offices clearly failed to meet the goal while 3 clearly met or exceeded the goal. However, VBA managers did not fully use the regional office- specific information generated by the STAR unit as a basis for improving accuracy at the regional office level. For example, VBA did not require regional offices that failed to meet the national accuracy goal of 85 percent to prepare strategies for improvement. In contrast, in fiscal year 2002, 13 regional offices that did not meet established targets for production, inventory, and timeliness were required to develop corrective action plans. Also, as noted earlier, VBA did not make rating accuracy a prerequisite for receiving a performance award. Three of the four criteria for cash awards were based on production and timeliness and one on accuracy; a bonus could be received without meeting the accuracy criteria. In fiscal year 2002, 2 offices received performance awards despite clearly not meeting VBA’s accuracy goal. These offices had accuracy rates of 71 percent and 75 percent, respectively.

VBA’s regional offices did not always properly address STAR errors that were returned to them, a violation of VBA policy that regional offices either correct errors or formally request that the STAR unit reconsider its error call. VBA survey teams, which review judgmental samples of STAR cases with errors, found that the 18 regional offices they visited from October 2001 through December 2002 had failed to respond properly to over 40 percent of the errors reviewed. At 14 of the 18 offices, the survey teams found problems significant enough to recommend improvements in STAR error handling. For example, teams recommended that several offices establish management controls to ensure that they properly address all errors identified by the STAR unit. Effective April 1, 2003, VBA established a new tracking system that required regional offices to report to headquarters on actions taken to address all errors identified by the STAR unit.

In addition to not correcting all errors, several of the regional offices visited by the survey teams were not making adequate use of STAR errors to identify trends that could be used to provide feedback and training to claims processing staff. Officials at regional offices we visited commented that this information is useful for identifying trends for feedback and training purposes. For example, in fiscal year 2002, the results of STAR reviews helped to alert management of the Boston Regional Office that many of its claims processors were having difficulties with the new VCAA legislation, permitting management to provide special training for the Boston staff. The agency survey teams recommended that 7 of the regional offices reviewed between October 2001 and December 2002 take steps to use STAR reviews for feedback and training.

In addition to regional office-level accuracy standards, VBA has also established quantifiable accuracy standards for its experienced claims processing employees, as part of these employees’ annual performance evaluations.The accuracy standards for experienced rating specialists require that they correctly decide 85 percent of 60 randomly selected rating decisions per year (5 decisions per month). VBA is developing an agencywide review process, the Systematic Individual Performance Assessment (SIPA), to assess compliance with the individual accuracy standards. VBA developed a uniform checklist, similar to the STAR checklist for reviewing rating decisions, which offices could use to assess rating specialists’ accuracy. Unlike the STAR system, reviews are conducted before decisions are finalized.

In April 2003, VBA surveyed regional offices to determine the status of their efforts to measure individual accuracy and assess the staff resources needed to implement SIPA VBA-wide. Individual reviews were being conducted by supervisors, such as team coaches, and nonsupervisory staff, such as Decision Review Officers, who are responsible for handling appeals. At the time of our review, VBA was in the process of deciding who would perform the reviews. In its efforts to develop SIPA as a consistent review process, VBA is considering issues such as ensuring that consistent feedback is provided to the employees and all regional offices are using the same review checklists.

Conclusions

While VBA’s emphasis on production succeeded in increasing the number of claims decisions made during fiscal year 2002, the accuracy of those decisions declined. Because VBA did not report comparable accuracy rates for fiscal years 2001 and 2002 in its performance and accountability report, Members of Congress, Department of Veterans Affairs management, and claimants did not know of this decline in accuracy. The lack of accurate performance data hampers stakeholders’ ability to monitor VBA’s performance in serving the nation’s veterans.

In addition, VBA could better use the accuracy data it has to hold its regional offices more accountable for accuracy. While VBA has made progress in measuring regional office accuracy, it has not made full use of the results to reward high performing offices or to improve poorly performing offices. VBA has made significant progress in getting its compensation and pension claims workload under control to improve timeliness for veterans waiting for decisions on their claims. However, VBA’s accuracy has declined. Unless VBA balances its emphasis on producing more and faster decisions with an increased emphasis on accuracy, it will be difficult to ensure that veterans are getting the benefits to which they are entitled in a timely manner.

Recommendations

We recommend that the Secretary of Veterans Affairs direct the Under Secretary for Benefits to report the accuracy of VBA disability compensation and pension claims decisions to the Congress and other stakeholders in a manner that allows for valid comparisons of accuracy across fiscal years and better hold regional offices accountable for the accuracy of their claims decisions, by increasing the use of its regional office accuracy data, while at the same time maintaining an appropriate emphasis on production.

Agency Comments and Our Response

In its written comments on a draft of this report (see app. I), VBA concurred with our recommendations. However, VBA suggested that more effective use of STAR had been made to improve performance than our report concludes. We did note that VBA used STAR data to improve accuracy. Specifically, we noted in our report that VCAA-related errors declined to about one-third of all errors after VBA identified the errors through STAR and required retraining. We support VBA’s continued efforts to emphasize accuracy and timeliness as critical components of quality.

We will send copies of this report to the Secretary of Veterans Affairs, appropriate congressional committees, and other interested parties. We will also make copies of this report available to others on request. The report will also be available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staffs have any questions regarding this report, please call me at (202) 512-7215 or Irene Chu, Assistant Director, at (202) 512-7102. In addition to those named, Susan Bernstein, Kristine Braaten, Kevin Jackson, Joseph Natalicchio, Martin Scire, and Greg Whitney made key contributions to this report.