GAO-01-930R VBA Disability Claims Processing

United States General Accounting Office

Washington, DC 20548

August 23, 2001

The Honorable Lane Evans

Ranking Democratic Member

Committee on Veterans’ Affairs

House of Representatives

Subject: Veterans’ Benefits: Quality Assurance for Disability Claims Processing

Dear Mr. Evans:

In recent years, the accuracy of claims processing in the Veterans Benefits

Administration's (VBA) compensation and pension programs has been the subject of

concern and attention within the Department of Veterans Affairs (VA), the Congress,

and veterans' service organizations. In October 1998, VBA implemented its

Systematic Technical Accuracy Review (STAR) system to improve the measurement

of the accuracy of claims processing. As designed by VBA, the STAR system

determines accuracy rates for (1) the nation as a whole, (2) each VBA Service

Delivery Network, and (3) each VBA regional office.1

In March 1999, we reported to you that the claims-processing accuracy rates for

individual regional offices are determined through quality reviews conducted by

regional office staff, who themselves are involved in processing claims and who

report to managers responsible for claims processing.2

 As we reported, such an

arrangement does not meet the governmental internal control standard calling for

segregation of key duties, or the performance audit standard calling for

organizational independence for those who review and evaluate program

performance.3

 These standards are intended to prevent bias and avoid compromised

objectivity in the efforts of individuals who perform key duties, such as reviewing and

evaluating program performance. We recommended that VBA correct the reported

shortcomings to comply with these standards. Subsequently, the Congress enacted a

 1

 VBA has structured its 57 regional offices into nine Service Delivery Networks. VBA expects the

regional offices in each Network to collaborate, provide mutual support, share resources, use teambased principles, and share collective responsibility and accountability for the Network’s overall

performance of all assigned work.

2

Veterans’ Benefits Claims: Further Improvements Needed in Claims-Processing Accuracy

(GAO/HEHS-99-35, Mar. 1, 1999).

3

Standards for Internal Control in the Federal Government (GAO/AIMD-00-21.3.1, Nov. 1999) and

Government Auditing Standards (U.S. General Accounting Office, Washington, D.C., June 1994).
Page 2 GAO-01-930R VBA Disability Claims Processing

provision in the Veterans Millennium Health Care and Benefits Act (P. L. 106-117,

Nov. 30, 1999) requiring VBA to have a quality assurance program that meets

generally applicable governmental standards for independence and internal controls

for quality reviews of Government performance and results. The conference report

for this act specifically directed the Secretary to design VBA's quality assurance

program so that it complies with our recommendation, which stated that VBA's

quality assurance system should meet the government's internal control standard on

segregation of duties and the program performance audit standard on organizational

independence.4

You asked that we determine whether VBA has a quality assurance system for

compensation and pension claims that complies with our recommendation. To

address this issue, we reviewed VBA documents related to the STAR system. At VBA

headquarters, we also interviewed VBA officials who oversee regional office

operations and officials in VBA's Compensation and Pension Service who formulate

program policy and guidance for the compensation and pension programs and who

are responsible for STAR reviews.5 Our review was conducted between June 2001

and August 2001 in accordance with generally accepted government auditing

standards.

Results in Brief

In brief, effective October 2001, VBA plans to implement a modification to the STAR

system that would bring it into compliance with our recommendation regarding the

standards on segregation of duties and organizational independence. At that time,

VBA plans that all STAR reviews will be conducted by staff of the Compensation and

Pension Service who themselves do not process claims and do not report to

managers responsible for claims processing. Staff of VBA's regional offices would no

longer be responsible for conducting the STAR reviews that determine regional office

accuracy rates. As a result, if this modification is implemented as planned, the STAR

system would comply with our recommendation regarding segregation of duties and

organizational independence. We received comments from VA on a draft of this

letter, and VA agreed with our conclusions.

Background

The compensation program pays monthly benefits--based on degree of disability--to

veterans who have service-connected disabilities (injuries or diseases incurred or

aggravated while on active military duty), regardless of whether these veterans are

employed or have earnings. The pension program pays monthly benefits--based on

financial need--to wartime veterans who have low incomes and who are permanently

and totally disabled for reasons not service-connected.

 4

 H.R. Rep. No. 106-470, at 91-92 (1999).

5

 In addition to the Compensation and Pension Service, VBA has four other Services that formulate

policy and guidance for its insurance, loan guaranty, vocational rehabilitation and employment, and

education programs. Section 801 of the Veterans Millennium Health Care and Benefits Act permits

VBA to establish a quality assurance function in each of its five program Services. As agreed with your

office, we did not review the quality assurance systems for VBA's other four program Services.
Page 3 GAO-01-930R VBA Disability Claims Processing

Veterans may submit their compensation and pension claims to any of VBA's 57

regional offices. In processing a veteran's claim, the regional office must assist the

veteran in obtaining evidence to support the claim. For example, when a veteran

submits a compensation claim, the regional office must obtain records such as the

veteran’s existing service medical records, records of relevant medical treatment or

examinations provided at VA health-care facilities or at VA’s expense, and other

relevant records held by a federal department or agency. Also, if necessary to make a

decision on the claim, the regional office must arrange a medical examination or

obtain a medical opinion. The regional office then analyzes all the evidence,

evaluates service-connected impairments, and assigns a rating for the degree to

which the veteran is disabled. The regional office then notifies the veteran of the

decision.6

Under the STAR system, VBA considers a claim to have been processed accurately if

the regional office determines basic eligibility correctly, obtains all required medical

and nonmedical documentary evidence, decides service-connection correctly, gives

the correct rating to each impairment, determines the correct payment amount, and

properly notifies the veteran of the outcome of his or her claim. If a claim has any

errors in any of these areas, VBA counts the entire claim as incorrect for accuracy

rate computation purposes. For the nation as a whole, VBA reported an accuracy

rate of 59 percent for fiscal year 2000. VBA’s strategic goal is to achieve a national

accuracy rate of 96 percent by fiscal year 2006.

The accuracy rates determined through the STAR system are a key component of

VBA's system for holding program managers accountable for program performance.

In fiscal year 1999, VBA instituted a "balanced scorecard" for measuring program

performance at three organizational levels: the national VBA headquarters level, the

Service Delivery Network level, and the regional office level. Under the balanced

scorecard approach, claims processing performance at each level is assessed on the

basis of five performance measures: accuracy, timeliness, unit cost, customer

satisfaction, and employee satisfaction and development. The balanced scorecard

gives more relative weight (28 percent) to the accuracy rate than to any of the other

four factors.

VBA's Planned Modification Would Bring the STAR System Into Compliance

With Our Recommendation Regarding Standards on Segregation of Duties

and Organizational Independence

In October 1998, when VBA implemented the STAR system for the compensation and

pension programs, VBA established a STAR unit in its Compensation and Pension

Service at VBA headquarters. Since its inception, this STAR unit, which is staffed

with Compensation and Pension Service employees, has performed the quality

 6

 If a veteran disagrees with a regional office decision, the veteran may file an appeal asking VA’s

Board of Veterans’ Appeals to review the decision. The Board makes VA’s final decision on such

appeals and can grant or deny benefits. In some cases the Board may return (remand) the case to the

regional office to obtain further evidence and reconsider the decision. In these cases, the regional

office either grants the claim or returns it to the Board for a final VA decision. Further, beyond

appeals to the Board, veterans can appeal first to the U.S. Court of Appeals for Veterans Claims, then

to the Court of Appeals for the Federal Circuit, and finally to the Supreme Court of the United States.
Page 4 GAO-01-930R VBA Disability Claims Processing

reviews that are used to determine the accuracy rates for the nation and for each of

VBA's nine Service Delivery Networks. Because the Compensation and Pension

Service itself does not process claims and is not organizationally responsible for

claims processing in the regional offices, these national and network reviews

conducted by the Compensation and Pension Service's STAR unit meet the standards

on segregation of duties and organizational independence (see fig. 1).

Figure 1: VBA Organization Chart

Note: This is a simplified version of VBA’s organization chart showing only the components needed to

understand the issues presented in this letter.

Source: VBA.

By contrast, as implemented in 1998, the STAR system does not provide adequate

segregation of duties, or organizational independence for the quality reviewers who

determine accuracy rates for individual regional offices. Under the original design,

VBA tasked each regional office with the responsibility of self-reviewing its own work

to determine its individual accuracy rate. According to VBA, it took this approach

due to the volume of additional cases that the STAR unit in the Compensation and

Pension Service would have had to review to determine accuracy rates for each

individual regional office.

However, tasking the regional offices with reviewing their own work means that the

regional office staff members who process claims are themselves responsible for

reviewing the accuracy of their own office's claims processing. These same staff

report to regional office managers who are also responsible for claims processing.

Such an arrangement does not meet the internal control standard calling for

segregation of key duties, nor does it meet the performance audit standard calling for
Page 5 GAO-01-930R VBA Disability Claims Processing

organizational independence of program reviewers. It does not meet these standards

because both the reviewers and their managers have an inherent self-interest in

having as high an accuracy rate as possible, which potentially could bias the

reviewers and influence them to overlook or conceal errors.

Because such issues raise questions about the reliability of the 57 regional offices'

self-reported accuracy rates, VBA decided not to use these self-reported accuracy

rates in the regional offices' individual balanced scorecards. Instead, to assess the

performance of a specific regional office, VBA inserts into that office's balanced

scorecard the overall average accuracy rate of its Service Delivery Network, as

determined by the STAR unit at VBA headquarters. In this way, the accuracy rate

inserted into the balanced scorecard for each individual regional office may not

reflect its actual performance. Consequently, a regional office whose actual accuracy

rate is below its network's average rate is not held appropriately accountable for its

performance.

In March 2001, however, the Under Secretary for Benefits announced a plan that

would correct these shortcomings and comply with our recommendation regarding

segregation of duties and organizational independence. The Under Secretary stated

that, to redirect regional office resources from quality assurance reviews to claims

processing, the STAR unit in the Compensation and Pension Service would begin

reviewing statistically representative samples of cases for each regional office to

determine the accuracy rates for each office. Thus, under this planned arrangement,

all of the STAR reviews--including those that determine regional office accuracy

rates--would be done by the STAR unit in the Compensation and Pension Service at

VBA headquarters. As we recommended, this planned approach would provide

adequate segregation of duties, as well as organizational independence for the

reviewers who determine regional office accuracy rates. VBA also plans to use each

regional office's own individual accuracy rate in the balanced scorecard assessment

for each office.

VBA plans to implement this new STAR system arrangement for regional offices on

October 1, 2001. VBA projects that the samples reviewed by the Compensation and

Pension Service's STAR unit will be large enough to produce an annual accuracy rate

for each regional office at the 95-percent confidence level, plus or minus an 8-

percentage point margin of error.7

 Doing so will require assigning up to 10 new fulltime staff in addition to the 6 who have been reviewing cases since the STAR system

was originally implemented.8

 Under the original design, the Compensation and

Pension Service's STAR unit has been reviewing a total of about 7,000 cases per year

 7

 For cases involving ratings, the minimum annual sample size will be 120 cases per regional office.

With this sample size, the expected margin of sampling error at the 95-percent confidence level is plus

or minus 8 percentage points. The sample size is calculated based on the assumption that the

accuracy rate will be 70 percent. If the actual accuracy rate turns out to be 50 percent, the margin of

sampling error at the 95-percent confidence level would increase to plus or minus 9 percentage points.

8

 The STAR unit at VBA headquarters actually has nine staff members who review cases, but they

spend about one-third of their time on duties other than STAR reviews. Beginning October 1, 2001,

these nine staff members will work full-time on STAR reviews. VBA also plans to outbase six of its

new STAR review staff in the regional office in Nashville. They will report directly to the STAR unit in

VBA headquarters.
Page 6 GAO-01-930R VBA Disability Claims Processing

to determine accuracy rates for the nation and each Service Delivery Network.

However, with the additional responsibility of reviewing cases to determine accuracy

rates for each regional office, the STAR unit at VBA headquarters will be responsible

for reviewing between 10,000 and 12,000 additional cases per year.

Under this plan, the STAR unit will conduct all of the quality reviews that determine

the accuracy rates used to hold program managers accountable for organizational

performance. However, VBA also believes that regional offices should have a role in

the quality review process, and not simply be the recipient of review results. As such,

VBA plans to implement a Systematic Individual Performance Assessment (SIPA)

system in fiscal year 2002. Under the SIPA system, each regional office will assess

the accuracy performance of individual employees to identify their training needs and

ways to improve the claims process. We support this effort.

Agency Comments

We received written comments from VA on a draft of this letter (see enclosure). In its

comments, VA said it was pleased that we had concluded that planned modifications

will bring VBA's STAR system into compliance with our recommendation regarding

the generally applicable governmental standards on segregation of key duties and

organizational independence for those who review and evaluate program

performance.

- - - - -

As agreed with your office, unless you publicly announce its contents earlier, we plan

no further distribution of this letter until 21 days after its issue date. At that time, we

will send copies of this letter to the Secretary of Veterans Affairs, the Under Secretary

for Benefits, and interested congressional committees. The letter is also available on

GAO's homepage at http://www.gao.gov. If you or your staff have questions, please

call me on (202) 512-7101 or Irene Chu on (202) 512-7102. Ira Spears and Steve

Morris also were key contributors to this report.

Sincerely yours,

Cynthia A. Bascetta

Director, Education, Workforce,

 and Income Security Issues
Page 7 GAO-01-930R VBA Disability Claims Processing

ENCLOSURE I ENCLOSURE I

Comments from the Department of Veterans Affairs

(130053)