Report to Congressional Requesters

United States Government Accountability Office

GAO

November 2004 VETERANS 

BENEFITS 

VA Needs Plan for 

Assessing Consistency 

of Decisions 

GAO-05-99 
Page i GAO-05-99 Veterans Benefits 

Letter 1

Appendix I Briefing Slides 4 

--------------------------------------------------------------------------------------------------------------------- 

Abbreviations 

BDN Benefits Delivery Network 

STAR Systematic Technical Accuracy Review 

VA Department of Veterans Affairs 

Contents 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
Page 1 GAO-05-99 Veterans Benefits 

November 19, 2004 

The Honorable Henry E. Brown, Jr. 

Chairman 

Subcommittee on Benefits 

Committee on Veterans’ Affairs 

House of Representatives 

The Honorable Mike Simpson 

House of Representatives 

In the past, we have reported concerns about possible inconsistencies in 

the disability decisions made by the 57 regional offices of the Department 

of Veterans Affairs (VA). In 2002, we reported that VA did not 

systematically assess the consistency of decision making for any specific 

impairments included in veterans’ disability claims.1

 We recommended 

that VA conduct such assessments to help reduce any unacceptable 

variations that VA might find among regional offices. VA agreed that 

decision-making consistency is an important goal and concurred in 

principle with our recommendation. However, VA did not discuss how it 

would measure consistency. 

In January 2003, in part because of concerns about consistency, we 

designated VA’s disability program, along with other federal disability 

programs, as high-risk.2

 In fiscal year 2005, VA estimates it will pay about 

$25 billion in disability compensation benefits to about 2.7 million disabled 

veterans. In this context, you asked us to determine (1) the actions that VA 

has taken to assess the consistency of regional office decisions on 

disability compensation claims and (2) the extent to which VA program 

data can be used to measure the consistency of decision making among 

regional offices. 

To address these issues, we (1) identified key data fields in VA’s Benefits 

Delivery Network system—such as the level of benefits awarded for each 

claimed impairment—which VA uses to manage the delivery of disability 

 1

GAO, Veterans’ Benefits: Quality Assurance for Disability Claims and Appeals 

Processing Can Be Further Improved, GAO-02-806 (Washington, D.C.: Aug. 16, 2002). 

2

GAO, High-Risk Series: An Update, GAO-03-119 (Washington, D.C.: Jan. 2003). 

United States Government Accountability Office

Washington, DC 20548 
Page 2 GAO-05-99 Veterans Benefits 

benefits to veterans; (2) obtained from VA an electronic file of these key 

data fields for all veterans receiving compensation benefits as of March 

2004; (3) conducted electronic testing of key data fields to determine their 

reliability for identifying indications of possible inconsistency in regional 

office decisions; and (4) reviewed VA records and documents and 

interviewed VA officials. We conducted our review from November 2003 

through October 2004 in accordance with generally accepted government 

auditing standards. On October 28, 2004, we briefed your office on the 

results of our work. This letter formally conveys the information provided 

during that briefing. Appendix I contains the briefing slides. 

In summary, we found that VA still does not systematically assess 

decision-making consistency among the 57 regional offices. We also found 

that data contained in VA’s Benefits Delivery Network system, which was 

designed for the purpose of paying benefits, do not provide a reliable basis 

for identifying indications of possible decision-making inconsistencies 

among regional offices. However, according to VA officials, as of October 

2004, a newly-implemented nationwide information system (known as 

RBA 2000) could provide VA such an opportunity if the system proves over 

time to reliably collect data needed to determine each regional office’s 

denial rates and average disability ratings for specific impairments. VA will 

need to collect several years of data with RBA 2000 in order to have 

sufficient data to reliably identify indications of impairment-specific 

inconsistencies among regional offices. Still, even if the RBA 2000 system 

permits VA to identify indications of such inconsistencies, VA will need to 

systematically study and determine the extent and causes of such 

inconsistencies and identify ways to reduce any variations among regional 

offices that VA may consider unacceptable. 

We recommend that the Secretary of Veterans Affairs develop a plan, and 

include it in VA’s annual performance plan, that contains a detailed 

description of how VA will (1) use data gathered through the new RBA 

2000 system to identify indications of possible inconsistencies among 

regional offices in the award and denial of disability compensation 

benefits for specific impairments and (2) conduct systematic studies of 

consistency for specific impairments for which RBA 2000 data reveal 

indications of possible decision-making inconsistencies among regional 

offices. 

In oral comments on a draft of this report, VA agreed with our findings and 

conclusions and concurred with our recommendation. We also made 

technical revisions as appropriate. 
Page 3 GAO-05-99 Veterans Benefits 

We are sending copies of this report to the Chairman and Ranking 

Democratic Member, House Committee on Veterans’ Affairs; the Chairman 

and Ranking Democratic Member, Senate Committee on Veterans’ Affairs; 

and the Secretary of Veterans Affairs. We will also make copies available 

upon request. In addition, the report will be available at no charge on 

GAO’s Web site at http://www.gao.gov. 

If you or your staffs have any questions about this report, please contact 

me on (202) 512-7215 or Irene Chu, Assistant Director, on (202) 512-7102. 

Ira Spears, Joseph Natalicchio, Joan Vogel, Walter Vance, and Vanessa 

Taylor also made key contributions to this report. 

Cynthia A. Bascetta 

Director, Education, Workforce, 

 and Income Security Issues 
Appendix I: Briefing Slides 

Page 4 GAO-05-99 Veterans Benefits 

Appendix I: Briefing Slides 

1

Data Limitations Hinder the Department 

of Veterans Affairs’ Ability to Assess the Consistency of 

Decisions on Disability Compensation Claims

Briefing for Staff of

Representative Henry E. Brown, Jr.

Chairman, Subcommittee on Benefits,

House Committee on Veterans’ Affairs

and

Representative Mike Simpson

Subcommittee on VA, HUD and Independent Agencies

House Committee on Appropriations

October 28, 2004
Appendix I: Briefing Slides 

Page 5 GAO-05-99 Veterans Benefits 

2

• Key Questions

• Background

• Scope and Methodology

• Results in Brief

• GAO Findings

• Conclusions

• Recommendation

Data Limitations Hinder VA’s Ability

to Assess the Consistency of Decisions

on Disability Compensation Claims
Appendix I: Briefing Slides 

Page 6 GAO-05-99 Veterans Benefits 

3

• Since the issuance of our 2002 report, what actions has 

the Department of Veterans Affairs (VA) taken to assess 

the consistency of regional office decisions on disability 

compensation claims?

• To what extent does VA have program data that can be 

used to measure the consistency of decision making 

among regional offices?

Key Questions
Appendix I: Briefing Slides 

Page 7 GAO-05-99 Veterans Benefits 

4

VA claims adjudicators use judgment in making disability 

decisions, which introduces an element of potential 

variability. Judgment is particularly crucial when the 

adjudicator must

• assess the credibility of different sources of evidence; 

• evaluate how much weight to assign differing sources 

of evidence; or

• assess some disabilities, such as mental disorders, for 

which the disability standards are not entirely objective 

and require the use of professional judgment.

Background
Appendix I: Briefing Slides 

Page 8 GAO-05-99 Veterans Benefits 

5

Background

• In 1997, the National Academy of Public Administration 

reported that VA’s regulations were subject to varying 

interpretations and said achieving consistency across 

57 decentralized regional offices is inherently difficult.

• In 2001, VA’s Claims Processing Task Force 

questioned the consistency of decisions because of 

factors such as differing interpretations of VA guidance.
Appendix I: Briefing Slides 

Page 9 GAO-05-99 Veterans Benefits 

6

Background

• In 2002, we reported that VA did not systematically 

assess decision-making consistency for any specific 

medical impairments, despite concerns about possible 

inconsistencies in disability claims decisions made by 

VA’s 57 regional offices.

• VA’s disability decision quality review program—known 

as Systematic Technical Accuracy Review (STAR)—

assesses the overall accuracy of all disability decisions, 

but not the consistency of decisions overall or for 

specific impairments.
Appendix I: Briefing Slides 

Page 10 GAO-05-99 Veterans Benefits 

7

Background

• We recommended in 2002 that VA assess decisionmaking consistency for medical conditions requiring difficult 

judgment. We said VA could, for example:

• develop hypothetical claims for a specific medical 

impairment,

• distribute these claims to multiple adjudicators, and

• analyze variations in decisions on these claims.

• VA could use these findings to reduce impairment-specific 

variations among regional offices, if considered 

unacceptable, to levels VA believes would be appropriate.
Appendix I: Briefing Slides 

Page 11 GAO-05-99 Veterans Benefits 

8

Background

• VA agreed that decision-making consistency is an 

important goal and concurred in principle with our 

recommendation. VA added that it seeks to ensure 

consistency through training and communication.

• However, VA did not discuss how it would measure 

consistency or evaluate progress toward the goal of 

decision-making consistency.
Appendix I: Briefing Slides 

Page 12 GAO-05-99 Veterans Benefits 

9

Background

To decide a disability compensation claim, the regional office

• develops evidence,

• determines service connection of each claimed 

impairment,

• applies VA’s medical criteria to evaluate the degree 

of disability due to each service-connected 

impairment, and 

• determines the veteran’s overall degree of serviceconnected disability.
Appendix I: Briefing Slides 

Page 13 GAO-05-99 Veterans Benefits 

10

Background

After deciding a disability compensation claim, the regional 

office

• notifies the veteran of the decision and

• records the decision results in VA’s Benefits Delivery 

Network (BDN) system in order to begin paying 

benefits to the veteran.
Appendix I: Briefing Slides 

Page 14 GAO-05-99 Veterans Benefits 

11

To do our work, we

• identified key BDN data fields, such as the level of benefits

awarded for each impairment, and obtained from VA

an electronic file of these data for all veterans receiving 

compensation benefits as of March 2004;

• conducted electronic testing of key BDN data fields to 

determine the reliability of using these data to study the 

consistency of decision-making among regional offices; and

• reviewed VA records and documents and interviewed VA

officials. 

Scope and Methodology
Appendix I: Briefing Slides 

Page 15 GAO-05-99 Veterans Benefits 

12

• We did not assess how well BDN supports the 

payment of benefits.

• We conducted our review from November 2003 through 

October 2004 in accordance with generally accepted 

government auditing standards.

Scope and Methodology
Appendix I: Briefing Slides 

Page 16 GAO-05-99 Veterans Benefits 

13

• Since the issuance of our 2002 report, VA has not 

systematically assessed the consistency of regional 

office decisions on specific impairments.

• Existing compensation program data have limitations 

that preclude identifying indications of decision-making 

inconsistency among regional offices. However, VA is 

implementing a new data collection system that may 

afford an opportunity to identify indications of 

inconsistency in the future.

Results in Brief
Appendix I: Briefing Slides 

Page 17 GAO-05-99 Veterans Benefits 

14

VA Has Not Assessed Consistency of

Compensation Decisions among Regional Offices

Although VA acknowledges that veterans are concerned 

about consistency, VA has not taken any action to assess 

consistency.

• When we asked VA in 2004 about actions taken 

to assess consistency, VA said it would continue to 

use its STAR system to assess the overall accuracy 

of decisions for each regional office.

• However, STAR does not provide statistically 

meaningful data about the consistency of decisions 

for specific impairments.
Appendix I: Briefing Slides 

Page 18 GAO-05-99 Veterans Benefits 

15

BDN—until recently VA’s only nationwide administrative database for 

recording the results of disability decisions—does not provide a 

reliable basis for identifying indications of inconsistency in the disability 

compensation program. BDN did not permit us to reliably

• identify dates of decisions so that we could examine only 

recent decisions rather than decisions made many years ago,

• determine each regional office’s average disability 

ratings for specific impairments, or

• determine each regional office’s denial rates for specific 

impairments.

Data Limitations Prevent Reliable Identification of 

Potential Inconsistency among Regional Offices
Appendix I: Briefing Slides 

Page 19 GAO-05-99 Veterans Benefits 

16

We intended to identify indications of possible decision-making 

inconsistency among regional offices using existing administrative data 

to

• examine original claims decisions made after fiscal year 2000 

because issues affecting decision-making consistency in the 

past may not be the issues affecting consistency today and

• for selected impairments, such as post-traumatic stress 

disorder or an undiagnosed illness, compare the 

• average disability ratings of each regional office and 

• service-connection denial rates of each regional office.

Data Limitations Prevent Reliable Identification of 

Potential Inconsistency among Regional Offices
Appendix I: Briefing Slides 

Page 20 GAO-05-99 Veterans Benefits 

17

We could not reliably group decisions by a relevant date in order to identify 

original award decisions made after fiscal year 2000.

• No specific data field exists in BDN to record the date that the

veteran submitted his or her original claim or the date that the

regional office made the original award decision.

• Information in the field for recording the original award’s effective 

date (typically the original claim’s submission date) could 

subsequently be overwritten by another date or erased because of

• routine benefit changes made in a veteran’s record or 

• requirements of new court decisions or new laws.

Data Limitations Prevent Reliable Identification of 

Indications of Inconsistency among Regional Offices
Appendix I: Briefing Slides 

Page 21 GAO-05-99 Veterans Benefits 

18

We could not compare consistency among regional offices because we 

could not reliably identify the regional office that made the original award 

decision in which each service-connected disability was rated.

• Some original decisions are made by a regional office other than

the one in the region where the veteran resides, but only the 

regional office where the veteran resides is recorded in BDN 

because it has jurisdiction over the veteran’s claim file.

• If a veteran relocates to an area under the jurisdiction of another 

regional office, the regional office identifier may be changed in 

BDN. If so, BDN can no longer identify the office that made the

original award. 

Data Limitations Prevent Reliable Identification of 

Indications of Inconsistency among Regional Offices
Appendix I: Briefing Slides 

Page 22 GAO-05-99 Veterans Benefits 

19

• does not maintain impairment-specific data for 

decisions in which regional offices deny all disability 

benefits and

• does not capture impairment-specific decision data for 

any more than six impairments per veteran, even 

though veterans may claim more than six 

impairments.

Data Limitations Prevent Reliable Identification of 

Indications of Inconsistency among Regional Offices

We could not use BDN to determine denial rates for specific 

impairments because BDN
Appendix I: Briefing Slides 

Page 23 GAO-05-99 Veterans Benefits 

20

• VA is working on a new administrative data system—known as 

VETSNET—that eventually will serve all VA benefits programs. 

According to VA, the disability program portion of VETSNET will 

replace BDN in 2006.

• However, according to VA officials, as of October 2004, a nationwide 

VETSNET subsystem known as RBA 2000, which collects disability 

decision data, could provide a reliable basis for identifying impairmentspecific indications of inconsistency among the 57 regional offices.

• Several years of data will need to be collected in RBA 2000 before 

its data can be used to assess the consistency of decision making 

among the 57 regional offices.

Data Limitations Prevent Reliable Identification of 

Indications of Inconsistency among Regional Offices
Appendix I: Briefing Slides 

Page 24 GAO-05-99 Veterans Benefits 

21

• VA cannot provide reasonable assurance that similarly situated 

veterans who submit claims for the same impairment to different 

regional offices receive reasonably consistent decisions.

• RBA 2000 could allow VA to identify indications of possible 

inconsistencies if it proves over time to reliably provide data 

that enable VA to determine impairment-specific average 

disability ratings and average denial rates for each regional office.

• However,VA still would need to take additional action, such as we 

recommended in 2002, to determine the extent and causes of 

inconsistencies for the impairments in question and to identify 

ways to reduce any unacceptable levels of variation in the award

and denial of disability compensation benefits.

Conclusions
Appendix I: Briefing Slides 

Page 25 GAO-05-99 Veterans Benefits (130341) 

22

We recommend that the Secretary of Veterans Affairs develop a plan, and 

include it in VA’s annual performance plan, that contains a detailed 

description of how VA will

• use data collected through RBA 2000 to identify indications of 

possible inconsistencies among regional offices in the award and

denial of benefits for specific impairments and

• conduct systematic studies of consistency for specific impairments 

for which RBA 2000 data reveal indications of inconsistencies 

among decisions made by the regional offices.

(130341)

Recommendations for Executive Action
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

Obtaining Copies of 

GAO Reports and 

Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To 

have GAO e-mail you a list of newly posted products every afternoon, go to 

www.gao.gov and select “Subscribe to Updates.”

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to:

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Congressional 

Relations

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548

Public Affairs Susan Becker, Acting Manager, BeckerS@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548