Veterans Disability Benefits: Clearer Information for Veterans and Additional Performance Measures Could Improve Appeal Process
Released: 2011-09-30T00:00:00

Letter

Background

Disability Claims Process

To apply for disability compensation, a veteran submits a claim to VA’s Veterans Benefits Administration (VBA). A Veterans Service Representative at one of VA’s regional offices reviews the claim and assists the veteran in gathering required evidence, including military service records and medical treatment records from VA facilities and private providers. A Veteran Service Representative’s responsibilities may include establishing claims files, generating notification letters to veterans, assisting veterans in obtaining the evidence needed to support their claims, and assisting in processing appeals of claim decisions. A Rating Veterans Service Representative (RVSR) then evaluates the evidence and, if the RVSR finds the veteran to be eligible, determines the percentage rating for purposes of compensation. Disability compensation varies with the degree of disability and the number of a veteran’s dependents, and is paid monthly. Monthly base benefits in 2011 for an individual range from $123 for 10 percent disability to $2,673 for 100 percent disability. A veteran may cite multiple medical issues in a claim, for example, post-traumatic stress disorder, knee impairment, and hearing loss. The RVSR may grant all, some, or none of the issues in a claim. The veteran can obtain help by submitting a claim and navigating the process with a VSO representative, private attorney, or agent accredited by VA to assist veterans. In fiscal year 2010, VA received over one million disability compensation claims, a 46 percent increase from fiscal year 2007.

Appeal Process

Regional office review of notice of disagreement (initial appeal stage). If a veteran disagrees with the regional office’s initial decision on a claim, he or she appeals by submitting a written notice of disagreement to the regional office. (See fig. 1 for a flowchart of appeals process at the VA.) A veteran may choose a traditional review or a DRO review by responding to an appeal process request letter sent by VA. VA sends this letter to a veteran after receiving his or her notice of disagreement; the letter informs the veteran of the DRO review and traditional review options, requests the veteran choose one, and states how to obtain representation if the veteran does not already have it. If the veteran chooses the traditional review, the reviewer, who may be a RVSR or DRO, examines the claim file and any new evidence that the veteran submits and may hold a formal, transcribed hearing with the veteran. The reviewer may overturn the original decision based only on (1) new evidence or (2) a clear and unmistakable error made in the original decision. However, if a veteran chooses the DRO review, a DRO conducts a de novo review of the claim, meaning a new and complete review without deference to the original decision, and can revise that decision without new evidence or a clear and unmistakable error—in other words, based on a difference of opinion. In addition to formal hearings, DROs may hold informal conferences with the veteran or the veteran’s representative to discuss an appeal, including why benefits already awarded are appropriate. Ultimately, in either process, the reviewer may: (1) award a full grant, in which all claimed benefits are awarded at the maximum level; (2) award a partial grant, in which some benefits are granted but not necessarily at the maximum level or for all claimed medical issues; or (3) confirm the original decision on the claim, in which no further benefits are granted on any issues. If a full grant is awarded, the appeal ends. Otherwise, the regional office issues a written explanation to the veteran in a statement of the case.

Regional office preparation of appeal for the Board. Regardless of whether a DRO or a traditional review is chosen, if a veteran disagrees with the regional office’s appeal decision, he or she may file what VA calls a substantive appeal to the Board, which is processed initially by the regional office. The substantive appeal is a document that the veteran completes to explain the issues being appealed and why the veteran believes the VA decided his or her case incorrectly. After receiving this document and reviewing any additional supporting evidence provided by the veteran, the regional office may award additional benefits. A veteran may submit new evidence multiple times and each submission requires the regional office to re-evaluate the claim. When the regional office determines no further work is necessary, it certifies the appeal as complete and transfers it to the Board.

Board review of appeal. When the Board receives the file, it may grant or deny the claim. If the Board finds it cannot make a decision until the regional office does additional work (e.g., requesting a more recent medical exam) it sends, or remands, the case back to the regional office or to the VBA Appeals Management Center in Washington, D.C., which develops evidence and adjudicates the claim. A veteran dissatisfied with the Board’s decision can appeal, in succession, to the U.S. Court of Appeals for Veterans Claims, to the Court of Appeals for the Federal Circuit, and finally to the Supreme Court of the United States.

VA created the DRO position and implemented the DRO review process nationally in 2001 after a 1997–1998 pilot in 12 regional offices. In the pilot, DROs were responsible for performing reviews of all appeals and had the ability to make a new decision based solely on a difference of opinion with the original decision. VA promulgated final regulations when it implemented DRO review nationwide in 2001, which made the process optional and required the veteran to expressly choose it. Later in 2001, VA headquarters issued guidance to the regional offices expanding the responsibilities of DROs to include mentoring and training other disability claims staff and working with regional office managers to identify error trends and training needs.

Veterans More Often Choose the DRO Review, but Some May Not Fully Understand Their Options

Since fiscal year 2003, when VA started tracking DRO involvement in appeals, data show more veterans have chosen a DRO review than a traditional review for appeals of decisions on their disability compensation claims. From fiscal years 2003 through 2010, veterans chose a DRO review in 534,439 appeals, or 61 percent of all appeals filed over this time period, according to Board data. The percentage of appeals in which a DRO review was chosen increased each year, from 54 percent of all appeals in fiscal year 2003 to 65 percent in fiscal year 2010 (see fig. 2). However, there was significant variation across VA’s regional offices. For example, during the 8-year time period, the regional office in Columbia, South Carolina, had the lowest percentage of DRO reviews chosen—32 percent—and San Diego, California, the highest—87 percent. More than half of the regional offices had the DRO review chosen in at least 50 percent of all appeals filed.

Our review suggests a key factor in the choice of a DRO review is whether a veteran has assistance from a third party, such as a VSO representative or a private attorney. Sixty-three percent of all appeals filed by veterans with representation chose a DRO review compared to 44 percent of those filed by veterans without representation across all regional offices. Controlling for selected factors that may affect the choice of a DRO review, veterans with representation are still more likely to choose a DRO review for their appeals. Additionally, we found that representation may also be one of the factors that contribute to the variation in the percentage of veterans choosing a DRO review across regional offices. As the percentage of appeals with representation increased, the percentage of DRO reviews also increased across regional offices. However, representation alone does not fully explain the variation in DRO reviews chosen across regional offices. Other factors that we did not analyze, such as demographic factors, may contribute to the variation across regional offices.

Representatives may recommend one appeal option over another to their veterans. Of the 40 veterans we interviewed, 22 told us that representatives from a VSO or state or county agency helped them decide between the DRO and the traditional review, and 17 of the 22 said that the assistance they received was very or somewhat helpful in deciding between the appeal options. Most VSO representatives we interviewed said they usually recommend a DRO review because they believe it is faster or because a DRO may be more likely to grant the claim. Some also said they recommend a DRO review over a traditional review because a DRO will thoroughly review all the evidence and a veteran has the opportunity to make his or her case to the regional office. VSO representatives at one regional office we visited said that they recommend a DRO review if they believe, based on experience, that the DRO likely to be assigned the case is more open to working with them.

However, some VSO representatives also said that they recommend a traditional review under certain circumstances. For example, if an appeal has ambiguous evidence, they may recommend a traditional review because it may result in an appeal going to the Board faster and because they believe the Board has more flexibility than a DRO to grant benefits in such cases. Also, VSO representatives at one site visit told us they generally recommend a traditional review because in their view a DRO review takes too much time.

Of all appeals filed since fiscal year 2003, 53 percent included the fact that they chose the DRO review option on the notice of disagreement. According to VSO representatives we interviewed, because veterans’ representatives have power of attorney, they can choose an appeal option at the same time they file an appeal for a veteran rather than waiting for the veteran to receive VA’s appeal process request letter. A DRO review may be chosen on the same day an appeal is filed in order to save time because the regional office does not have to send the appeal process request letter and wait for the veteran’s response before beginning its review.

Veterans filing 13 percent of appeals chose an appeal option without assistance from a representative, and these veterans may not fully understand their options as described in the VA appeal process request letter, some VA officials and VSO representatives told us. The VBA manual provides a template of the letter for the regional offices, which prepare and mail it to a veteran after receiving the notice of disagreement. VA headquarters officials told us that while regional offices have flexibility to add information to the template letter, they cannot exclude any information, and headquarters prefers that the regional offices use the template letter. (For VA’s template letter, see app. II.) Some VA officials and VSO representatives said veterans may not be able to make an informed decision based on the letter alone. Specifically, 29 of 55 regional office managers we surveyed (53 percent) answered that they believed no or only some veterans could make an informed decision using only the VA letter. The other regional office managers we surveyed (47 percent) said that all or most veterans could make an informed decision using only the VA letter (see fig. 3). Of the DROs we spoke with, 10 of 17 said that in their view, no or only some veterans could make informed decisions based on the letter alone, and most VSO representatives at the regional offices we visited said that few or no veterans could do so. Representatives from one VSO told us veterans may phone after receiving the letter because they do not understand its description of appeal options. Some managers in our survey and VA staff we interviewed suggested changes to the template, for example, shortening its length, using simpler language and less jargon, using graphics or tables to aid understanding, explaining the benefits of representation, and advising veterans they can request informal or formal hearings if they choose a DRO review. They also said the letter could better explain differences between DRO and traditional reviews, particularly in the treatment of evidence and the authority given a DRO to render a new decision without additional evidence.

Our analysis of the template in the VBA manual, based on criteria established for federal government communication with the public, found the letter did not clearly explain appeal options and the appeal process. Although we found the letter met several criteria for clear communication by federal agencies and programs, such as simple structure, concise headings, and avoidance of complex language overall, we found it did not meet some other criteria, including:  defining terms that could be unfamiliar to the veteran, such as “decision review officer,” “de novo reviews,” “clear and unmistakable error,” and “substantive appeal”;  highlighting key deadlines for responding by including them at the beginning of the letter, so the veteran is clear that VA needs a response within 60 days; and  using tables or graphics to explain appeal options and enhance recipient’s understanding (see fig. 4).

One of the regional offices we visited made changes to the template that address some of these issues, for example, including instructions on how to request a hearing and italicizing important information, such as the 60- day deadline for a veteran to respond with an appeal choice.

Of the 40 veterans we interviewed, 14 remembered receiving and were able to describe the appeal process request letter, and 9 of those told us that the letter explained the appeal options only somewhat clearly or not at all clearly. Five said it explained the options very clearly. Some of the veterans we spoke with offered suggestions on how VA could improve the letter, for example, by providing a point of contact for veterans with questions, explaining when they might next hear from VA about their appeal, and more thoroughly describing the two appeal options. VA headquarters officials said VBA management and attorneys reviewed the letter template but they did not test it with focus groups of veterans because VBA does not have contracts to conduct focus groups. According to federal guidelines for clear communication with the public, testing documents should be an integral part of an agency’s plain- language planning and writing process.

The DRO Review More Often Results in Additional Benefits, but Does Not Reduce Appeals Going to the Board

DRO Reviews More Often Resulted in Additional Benefits than Traditional Reviews

Veterans who chose a DRO review were more likely than those who chose a traditional review to get additional disability compensation benefits after initial review of their notices of disagreement. From fiscal years 2003 through 2010 at least some additional benefits were awarded at the regional level in 32 percent of DRO reviews compared to 23 percent of traditional regional office reviews, according to our analysis of Board data (see fig.5). We found that having a DRO review was associated with a greater chance of being awarded some additional benefits even after controlling for some other factors that may affect appeal outcomes, such as whether a veteran had a representative or the regional office in which the appeal was filed. Veterans awarded additional benefits at this appeal stage receive these benefits immediately even if they continue their appeals, according to a VA official.

DRO reviews took about a month longer than traditional reviews to complete, or 266 days compared to 235 days, on average, for the time period from fiscal years 2003 through 2009 (see fig. 6). It is possible DRO reviews take longer because they involve a more thorough examination of the evidence. For example, one DRO we spoke with said such reviews take longer because they require a completely fresh look at all evidence in the original claim. Regardless of whether veterans choose a DRO or a traditional review, their waiting time for a decision on the notice of disagreement is considerably less than if they continue their appeal to the Board. For appeals filed from fiscal years 2003 through 2007, the time from filing of the notice of disagreement to Board decision was more than 1,000 days on average.

Although veterans choosing a DRO review were more likely to gain some additional benefits at the regional office level, in comparison to a traditional review, DRO review does not appear to reduce the number of appeals continuing to the Board—which was an important goal for VA in introducing this appeal option. The resolution rate at the regional office level for appeals in which a DRO review was selected has been about the same as the rate for traditional reviews. Of the 593,526 appeals of disability claim decisions receiving either DRO or traditional reviews from fiscal years 2003 through 2008 that had a final regional office outcome, 72 percent ended at the regional office level. The remainder continued to the Board. For appeals in which a DRO review was selected, 71 percent ended at the regional office level compared to 73 percent of appeals in which a traditional review was chosen.The resolution rate for DRO and traditional reviews was roughly the same in each year during this 6-year period (see fig. 7). Resolution rates varied across VA’s regional offices. For both DRO and traditional reviews, resolution rates ranged from less than 50 percent to more than 80 percent in individual offices, with the majority having rates of more than 70 percent for both types of reviews. VA officials offered several possible explanations for the variation across offices, for example, the extent to which VSO representatives are proactive in submitting necessary evidence.

The negligible difference between appeal resolution rates at the regional office level for DRO and traditional reviews in part reflects veterans’ decisions on how to proceed if they do not receive a full grant of requested benefits. Although a slightly higher percentage of DRO reviews resulted in full grants of benefits for veterans, veterans choosing DRO review were less likely than those choosing traditional review to end their appeals when not granted full benefits. Board data show that for appeals filed from fiscal years 2003 through 2008 the most common reason for resolution at the regional office level was a veteran’s failure to return a form to continue the appeal (failure to respond) after receiving VA’s explanation of its decision and of what steps to take to move the appeal to the Board (see fig. 8). On this front, traditional reviews were somewhat more likely than DRO reviews to be resolved through veterans’ failure to respond—41 percent versus 36 percent, respectively. The second most common reason for appeal resolution at the regional level was a full grant of benefits, which automatically ends the process. In contrast to the first issue, DRO reviews were somewhat more likely than traditional reviews to be resolved through a full grant of benefits—21 percent versus 17 percent, respectively. Most decisions to grant full benefits were made at the initial appeal stage, while some were made during the stage in which the regional office prepares the appeal for the Board. The remaining appeal resolutions were due to veterans or their representatives withdrawing their appeals by contacting VA or to the death of the veteran.

While we found no difference in how veterans responded to a partial grant of benefits based on their choice of appeal option, we found that veterans who chose a traditional review appeared to respond differently than those who chose a DRO review when no additional benefits were granted at the initial appeal stage. Board data for fiscal years 2003 through 2008 show that when no additional benefits were granted (i.e., the original decision was upheld), appeals in which a traditional review was chosen were somewhat more likely than those in which a DRO review was chosen to end at the regional office level—57 percent versus 51 percent, respectively. By contrast, in cases that resulted in the award of partial benefits, there was no difference based on which appeal option had been chosen—75 percent of both traditional appeals and DRO appeals were ended at the regional level. Representation may be a factor in why veterans selecting a traditional review are more likely to end their appeals after receiving no additional benefits from the regional office. Fifty-nine percent of appeals which received no additional benefits at the initial appeal stage and in which the veteran had no representative were ended by the veteran without going on to the Board, compared to 52 percent of such appeals in which the veteran had a representative. As noted previously, veterans without representation are more likely to choose a traditional review.

VA regional office staff told us that certain aspects of the DRO review can be effective in resolving appeals before they continue to the Board. Managers we surveyed rated DROs’ authority to make a completely new (de novo) decision on a claim and to hold informal conferences with veterans and representatives as the most effective tools for resolving appeals (see fig. 9). Of the 17 DROs we interviewed, 15 rated their authority to make a completely new decision as very or moderately effective in resolving appeals, 10 rated informal conferences as very or moderately effective, and 7 rated formal hearings as very or moderately effective. Many DROs in our site visits told us their authority to make a completely new decision on a claim allows them to reverse original decisions without submission of additional evidence by the veteran. DROs also said informal conferences may help resolve appeals because they enable DROs to obtain information from a veteran’s representative or explain to a representative and veteran why no additional benefits can be granted.

While many regional staff believe DRO reviews have a greater potential to resolve appeals, we found that the appeal resolution rate for DRO and traditional reviews are quite similar, perhaps in part because VA’s criteria for assessing and rewarding individual DROs’ performance do not always encourage them to use their unique authorities to resolve appeals. VA currently assesses the performance of individual DROs using four criteria: quality of work, productivity, customer service, and timeliness. The existing criteria do not specifically encourage appeal resolution at the regional level, according to 60 percent of surveyed managers and 10 of the 17 DROs we interviewed. For example, managers in one office we visited said the current criteria encourage DROs to complete tasks that help them meet their productivity requirement but may not necessarily lead to appeal resolution.

The department is currently revising its criteria for assessing DRO performance based on a 90-day pilot program in eight regional offices conducted in fiscal year 2010, and it expects to implement new criteria nationwide in fiscal year 2012. According to VA officials, the revisions are intended in part to focus DROs’ attention more on resolving appeals at the earliest possible stage of the process. VA officials told us the new criteria under development will better encourage appeal resolution by restructuring the way work credits—needed to meet the productivity requirement—are awarded to DROs. For example, under the existing criteria, work credits are awarded for holding informal conferences, but under the new criteria additional credits would be awarded when the conference results in resolving the appeal. In addition, VA officials said the new criteria aim to promote appeal resolution by more explicitly promoting regular communication with veterans’ representatives. They said that even though such communication can help resolve appeals by helping representatives understand, for example, why further benefits cannot be granted in a particular case, DROs have become less focused on communication over the years due to production pressures.

VA Faces Challenges in Defining Role of and Training for DROs

Lack of Performance Measures for Appeal Resolution Hinders VA’s Ability to Assess Use of DROs

Since the DRO pilot program in 1997–1998, VA has expanded DRO duties to include tasks not related to appeals. According to the estimates of regional office managers who responded to our survey, DROs overall tend to spend the majority of their time on appeal-related tasks, but also spend a significant amount of time on training other staff and performing quality reviews (see fig.10). Time spent on different tasks varies across regional offices. For example, the proportion of time spent on conducting de novo reviews of appeals—that is, a completely new evaluation of a claim without deference to the original decision—ranged from 3 to 70 percent in individual offices, and on formal training ranged from 0 to 40 percent. The four offices we visited typically assigned individual DROs to specific responsibilities, such as reviewing appeals, training other staff, and conducting quality reviews of other staff’s work, rather than having each DRO perform the full range of tasks. Managers in one office said specialization is helpful because it permits DROs to focus on training without the distraction of appeals work. Regional office managers may also assign DROs to different duties at different times to meet changing office needs. For example, one office we visited had recently assigned more DROs to training and quality review to address problems with its RVSRs’ performance, and managers said preliminary results showed an improvement in RVSR quality scores after this reallocation of DRO resources.

Since the implementation of the DRO review process in 2001, there have been differing opinions within VA about the proper balance between processing appeals and performing other DRO tasks. In 2006, an internal VA study group that assessed the impact of the DRO review process recommended that VA limit the scope of DRO duties to reviewing appeals and have them perform de novo reviews of all appeals—not just those in which a DRO review was selected. Their report found that there was a sufficient number of DROs to perform de novo reviews of all appeals if DROs’ duties were limited to appeals-related tasks. Officials told us that VA management decided not to implement these recommendations because of concerns that there was an insufficient number of DROs to review all appeals and a belief that other tasks are also important DRO functions. There was also a mix of opinions on this topic among regional office managers we surveyed and interviewed. Almost two-thirds of those surveyed stated that eliminating the DRO review election process and having DROs perform only de novo reviews of all appeals would somewhat or greatly improve their effectiveness. On the other hand, managers said it could be difficult to have their DROs spend more time on such reviews. About 60 percent of surveyed managers said they would prefer to allocate more DRO time to performing de novo reviews, but some respondents noted that they cannot allocate DROs’ time optimally because of staff and resource limitations and the need to assign DROs to other duties.

While VA has expanded the role of the DROs so they are contributing to other VA goals beyond appeal resolution, the department has not developed performance measures to assess whether DROs are successful in meeting their original purpose of reducing the number of appeals to the Board. The measures that VA uses to assess national and regional office performance in processing appeals do not include the proportion of appeals resolved at regional offices. Existing VA measures for national and regional office appeals performance involve the number of appeals awaiting a decision, number of days appeals have been pending a decision, and percentage of appeals that are remanded from the Board. We have previously noted that agencies should develop performance measures that are linked to agency goals. Linking performance measures to broader agency goals at each operating level reinforces the importance of these goals and may help managers identify areas where problems exist and corrective action is required.

VA already collects some data related to appeal resolution, but some VA officials question the value of a performance measure related to appeal resolution. The department tracks data on the number of appeals resolved at different stages of the appeal process—for example, through the veteran’s failure to respond to the statement of the case or after the veteran’s submission of a substantive appeal to the Board—nationally and by regional office. It makes these data available to regional offices. About half the regional office managers surveyed said an appeal resolution goal at the regional office level would somewhat or greatly enhance DROs’ effectiveness. However, VA officials told us they have not established a performance measure for appeal resolution because this outcome is not entirely under the control of the DRO, as a veteran has a right to continue his or her appeal to the Board. Some VA managers and staff in the offices we visited also expressed a concern that such a goal could have a negative impact. For example, it could push DROs to grant benefits that are not completely justified or to pressure veterans to withdraw their appeals. While we understand this potential concern, we note that VA has quality review procedures in place to ensure the quality and accuracy of DRO decisions. Specifically, VA reviews on average five cases per month that each DRO has worked on. VA assesses each case against several criteria, including whether all claimed issues were addressed, whether all applicable evidence was discussed, and whether all issues were correctly granted or denied. Some regional office managers agreed that setting an appeal resolution goal would not encourage DROs to grant unjustified benefits, because, for example, existing quality review procedures would prevent this from happening.

VA Provides No Nationwide Specialized Training for New DROs

VA has not developed a nationwide training curriculum for DROs to help them learn the duties of the position. The training available to DROs from VA headquarters is the same training offered to RVSRs (staff who evaluate initial claims). VA requires its claims processing staff, including DROs, to take 85 hours of training annually. Headquarters and regional office officials told us DROs typically take the same courses as RVSRs. As part of its training for claims processors, VA has three courses for all regional office staff who process appeals: an orientation to the appeals team, a course on the appeal process, and a course on how to use the Board’s appeals tracking system. VA headquarters officials told us they have not developed training specifically for new DROs because they should already be technical experts in evaluating claims when promoted. However, they acknowledged that the department has not completed an analysis of DRO tasks specifically to determine the training needs of this position. According to generally accepted criteria, a key step in developing successful training in the federal government is for an agency to understand the skills that its workforce needs to achieve agency goals. Furthermore, these criteria stress the need for agencies to incorporate continuous, life-long learning into their training—even experienced staff may still need additional training in certain areas.

A number of managers and DROs in the regional offices we visited said the skills DROs learn before promotion may not be sufficient to perform their new tasks. Several managers told us new DROs, based on their prior experience, are indeed technical experts in evaluating claims; however, managers and DROs said new DROs may lack experience with certain responsibilities, such as holding formal hearings and informal conferences, training other staff, processing appeals, and balancing multiple priorities. Regional managers at three of the four offices we visited said the appeals courses offered by VA are not sufficient to teach new DROs how to perform their duties; a manager in one office said these courses provide a general overview and are not primarily used to train them. DROs in the four offices we visited learn their duties primarily on the job and by observing more experienced colleagues, according to managers. However, some offices have developed more formal training for DROs. Forty percent of surveyed regional managers said their office has developed specific training that address such topics as the appeals process, conducting formal hearings and informal conferences, reviewing other staff’s work, and policy and regulatory changes.

Despite regional office efforts to provide targeted training, our survey of regional managers found that 93 percent believe a nationally standardized training program for newly promoted DROs would improve their effectiveness, and 16 of the 17 DROs we interviewed also said a national training program would be helpful. Managers in two regional offices we visited said a training program developed by VA’s central office would ensure consistency in how DROs are trained across the nation. Managers responding to our survey indicated that tailored training on a number of topics would enhance the ability of new DROs to perform their job duties (see fig. 11).

A formal, centrally designed training program—especially for new DROs—has been proposed within VA. Under this proposal, according to one VA official involved with the management of the DROs nationally, new DROs would receive instruction in areas such as training other staff and communicating with veterans and their representatives. The training might be administered in a central location for new DROs from multiple regional offices, or developed centrally and then administered by regional offices. However, VA does not yet have formal plans to move ahead with the development of such a program, according to VA officials.

Conclusions

Review of veterans’ appeals by VA’s DROs has had some positive impacts for veterans, and it is possible that more veterans would choose a DRO review if they fully understood the distinction between the DRO and traditional reviews. Because VA’s outreach letter lacks information and clarity that veterans need to make an informed choice between the options, some—in particular, those without a representative—may not be taking advantage of the DRO review process. However, looking more broadly, VA has not achieved its original goal for the DRO review process of reducing the number of appeals continuing to the Board and thereby shortening the time that veterans wait for appeal decisions. Indeed, while VA is taking steps to place a greater focus on appeal resolution in the criteria it uses to assess the performance of individual DROs, it has yet to establish any national or regional office performance measures related to resolving appeals at the regional level. Certainly, VA must balance competing demands on DROs, senior claims processors in its regional offices, to both resolve appeals and train and supervise less experienced staff, but without a more strategic approach—which includes goals and measures for resolution of appeals—VA does not know it is fully leveraging DRO reviews and, ultimately, their effectiveness for the veterans VA serves. Regardless of the proper balance between their different responsibilities, DROs may not be contributing as much as they could be, either to the goal of appeal resolution or to the goal of developing newer claims processing staff, because they do not always receive training specific to their duties.

Recommendations for Executive Action

To clarify information for veterans and ensure the most effective use of DROs, the Secretary of Veterans Affairs should direct the Veterans Benefits Administration to take the following three actions: revise the sample appeals election letter in its policy manual to define unfamiliar terms and emphasize key deadlines, and test any revised letter’s clarity with veterans before implementing it;  establish national and regional office performance measures related to appeal resolution at the regional level and ensure that sufficient quality review procedures are in place to prevent DROs from granting unjustified benefits; and  assess the knowledge and skills that DROs need to perform their varied responsibilities, determine if any gaps exist in the training currently available, and, if necessary, develop a training curriculum or program tailored to DROs.

Agency Comments and Our Evaluation

We provided a draft of this report to VA for review and comment. In its written comments (see app. III), VA concurred fully with two of our recommendations and partially with our recommendation regarding an appeal resolution performance measure. The department concurred with our recommendation to revise its sample appeal election letter. It reported that it has formed a workgroup to assess and recommend changes to the appeal election process, potentially including changes to the letter. VA also concurred with our recommendation to assess the knowledge and skills that DROs need and if necessary develop a training curriculum for them. VA said it plans to develop additional training based on the results of skills certification tests administered to DROs and on a job task analysis conducted to aid the design of these certification tests.

VA partially concurred with our recommendation to establish national and regional performance measures related to the resolution of appeals at the regional office level. The department noted that—as explained previously in our report—it is revising the criteria used to assess the performance of individual DROs to place greater emphasis on resolving appeals at the earliest stage possible, for example through communication with veterans and their representatives. However, it also stated that resolution of appeals at the regional level should not be a performance measure used to assess the impact of the DROs, for several reasons including (1) the DROs also play an important role in training, mentoring, and quality review, so an appeal resolution measure does not capture their full impact; (2) each veteran has a right to continue an appeal to the Board if dissatisfied with the regional office decision, and whether a veteran continues is beyond a DRO’s control; and (3) establishing an appeal resolution performance measure could encourage the granting of benefits that are not justified. We acknowledge that DROs impact VA operations beyond appeals. Yet VA’s own documents—including its 2006 review of the DRO program—state that the department’s primary goal in establishing the DRO review process was to increase the percentage of appeals resolved before continuing to the Board, and the changes VA is now making to its criteria for measuring individual DRO performance indicate that appeal resolution remains a major goal. We are not recommending that an appeal resolution performance measure replace existing VA performance measures related to appeals or to other VA processes on which DROs could have an impact. Our recommendation is that VA add such a measure to gauge whether the program is meeting its specific intended goal and help the department assess whether further adjustments are needed. We also acknowledge that it is not possible to prevent all appeals from continuing to the Board; such an objective may not even be desirable. However, based on its changes to the criteria for individual DRO performance, VA clearly believes that changes in incentives can encourage behaviors that are likely to resolve more appeals early on. Finally, as VA also noted in its comments, individual DROs must meet rigorous accuracy standards in their appeals work, which should mitigate against the granting of unjustified benefits.

However, if VA does not believe that its existing quality control measures are sufficient to guard against the granting of unjustified benefits, we would encourage the department to consider what additional quality control measures it could implement to ensure that the addition of a new performance measure would not have the unintended consequence of increasing the award of unjustified benefits.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until one day from the report date. At that time, we will send copies of this report to the relevant congressional committees, the Secretary of Veterans Affairs, and other interested parties. This report is also available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions regarding this report, please contact me at (202) 512-7215 or bertonid@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Staff members who made key contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

We were asked to examine (1) the extent to which veterans choose a Decision Review Officer (DRO) review as opposed to a traditional review, (2) outcomes for veterans who choose a DRO review, and (3) challenges the Department of Veterans Affairs (VA) faces in managing DROs. To answer these objectives, we reviewed relevant laws and regulations, and VA documents including procedural manuals, internal studies, and the sample appeal election letter that VA headquarters provides for its regional offices. We interviewed cognizant officials from the Veterans Benefits Administration (VBA) and the Board of Veterans’ Appeals (Board). We interviewed managers, DROs, and veteran service organization (VSO) staff in 4 regional offices, and we conducted a web- based survey of managers in all of VA’s 57 regional offices. We obtained and analyzed administrative data from the Board on outcomes and processing times for appeals of disability compensation claims. Finally, to learn about veterans’ perspectives on the DRO review election process we conducted phone interviews with a small sample of veterans who had appealed their disability compensation claims.

We conducted this review from July 2010 to September 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Analysis of VA Appeal Process Request Letter

We analyzed the VBA’s appeal process request letter as a factor that may affect veterans’ decisions about the appeal options. For our analysis we used the sample letter in the VBA manual since VBA central office officials said that regional offices use the sample as a template for the letters they mail to veterans. We selected 16 criteria for clear federal communication using information from the Plain Language Guidelines, a previous GAO report on VBA letter clarity, and the VBA manual (see table 1). We reviewed the letter and came to a consensus about which criteria were met and which were not met.

We conducted site visits to 4 of VA’s 57 regional offices: Atlanta, Georgia; Providence, Rhode Island; Salt Lake City, Utah; and Waco, Texas. We conducted these visits to learn about regional office practices in utilizing DROs and regional staff’s opinions on such topics as how veterans choose between appeal options, factors affecting DRO review outcomes, and challenges facing VA in utilizing DROs. In each regional office, we interviewed regional office managers, at least one appeals coach, multiple DROs, and local VSO staff. In total, we interviewed 17 DROs across the 4 offices, with varying levels of experience in their position. In some offices we also interviewed the regional office training coordinator. We judgmentally selected these offices to achieve variation in several factors, including geographic location, number of staff, timeliness of appeal processing for appeals in which DRO was selected, and participation in a pilot study of revised criteria for assessing DRO performance (see fig. 12). What we report about these sites may not necessarily be representative of other VA regional offices.

To analyze data on VA’s appellate workload, including the number of appeals in which DRO and traditional reviews were selected, the outcomes for DRO and traditional reviews, and the processing times for DRO and traditional reviews, we obtained record-level appeals data extracted on April 7, 2011, from the Veterans Appeals Control and Locator System (VACOLS). We limited our analysis to appeals of disability compensation claims. We also limited our analysis to original appeals, as opposed to, for example, appeals that had been remanded by the Board. Using the record-level data, we generated nationwide annual data on appeals filed from fiscal years 2003 through 2010. The data for each fiscal year includes all notices of disagreement filed with the VA during that fiscal year. We looked at data back to fiscal year 2003, even though the DRO review process was established in 2001, because the VACOLS data element that identifies appeals in which DRO review was selected was only added to VACOLS at the end of 2002. For some analyses, we excluded data for appeals filed in recent fiscal years because a high proportion of these appeals were still pending some action by the regional office. To assess the reliability of the record-level appeals data, we (1) reviewed documentation on VACOLS including the data dictionary, (2) interviewed Board officials about VACOLS and any data reliability issues, and (3) performed electronic testing. We found the data to be sufficiently reliable for our reporting objectives.

Statistical Analysis of Factors Affecting Choice of DRO Review

To investigate the relationship between DRO review selection and representation, using VACOLS data, we first examined descriptive statistics at the regional office level. We found substantial variation in rates of DRO election, as well as rates of representation among veterans filing appeals, across regional offices. Using linear multiple regression analysis, we estimated that at the regional office level, a one percentage point increase in the proportion of veterans with representation was associated with approximately half a percentage point increase in the rate of DRO election across regional offices, after controlling for fiscal year. However, variation in representation levels across regional offices did not fully explain variation in DRO election across offices.

Our next analysis focused on appeals level data. Descriptive data show that, prior to controlling for other factors, veterans with representation were more likely to choose a DRO review than those without representation. Specifically, table 2 shows that approximately 63 percent of those veterans with representation requested a DRO reviews, compared to approximately 44 percent of veterans with no representation.

An alternative way to examine the likelihood of DRO election among veterans with and without representation is the odds of requesting a DRO review. Prior to controlling for other factors, the odds that a veteran with no representation requesting a DRO review is defined as the proportion of veterans requesting a DRO review divided by the proportion requesting a traditional review, or 0.79 (44.1/55.9). In comparison, among veterans with representation, the odds of requesting a DRO review were 1.72 (63.3/36.7). To compare the relative proportion of veterans with and without representation that requested a DRO review, we can construct an odds ratio that compares the odds of DRO selection to the reference group of veterans without representation, which is 2.18 (1.72/0.79). In other words, prior to controlling for other factors, the unadjusted odds that a veteran with representation requesting a DRO review were 2.18 times that of the odds that a veteran without representation requesting such review.

Although unadjusted odds provide useful summary information, they do not account for multiple other factors that could also affect a veteran’s choice of review. Logistic regression analysis can be used to estimate the odds ratio comparing the likelihood of DRO selection among veterans with and without representation, after also controlling for other factors that could potentially influence the choice of review type. We estimated a logistic regression model of DRO selection controlling for other factors including the fiscal year of the appeal, the regional office of the appeal, whether the veteran filing the appeal had representation, how many distinct medical issues were associated with each appeal, and what type of body systems were included in the issues under appeal. We were not able to control for certain other factors, such as the timing of DRO election, because we lacked comparative data for traditional reviews. There may be systematic differences between veterans who decide not to appeal a ruling and those who appeal a ruling and thus chose between DRO and traditional reviews. Additionally, we lacked information to control for certain factors such as a veteran’s branch of service or demographic characteristics.

After controlling for fiscal year, regional office, number of issues in each appeal, and body systems related to the issues under appeal, we found that appeals filed by veterans with representation were substantially more likely than those filed by veterans without representation to elect the DRO review process. Per the last column of table 2, we can see that, after controlling for other factors, the odds that an appeal filed by a veteran with representation underwent a DRO review were approximately two times higher than the odds that an appeal filed by a veteran without representation underwent a DRO review. This result was statistically significant at the p<0.05 level.

Other factors were also significantly associated with a veteran’s choice of review. Table 2 shows that even after controlling for representation, appeals filed at different regional offices varied substantially in the odds that they underwent a DRO review. After controlling for other factors, the odds that an appeal elected a DRO review were approximately four times higher in San Diego, California, than the odds at Albuquerque, New Mexico, and the odds that an appeal filed at Columbia, South Carolina, elected DRO were approximately 75 percent lower than those filed in Albuquerque. Additionally, compared to appeals with only one issue, veterans filing appeals with multiple issues had higher odds of electing a DRO review after controlling for other factors. After controlling for other factors, the odds that an appeal elected a DRO review increased significantly over time. For example, the odds that an appeal filed in fiscal year 2010 underwent a DRO review were about 63 percent higher than the odds for an appeal filed in fiscal year 2003. For the most part, appeals that included issues related to some body systems had odds of DRO review similar to appeals that did not mention those systems, although the difference was statistically significant for several issues.

As noted previously, we could not control for the full range of variables that could conceivably affect a veteran’s choice of review type, including the timing of DRO election, a veteran’s branch of service or demographic information, or what factors affected whether a veteran decided to appeal rather than accept an initial ruling. To the extent that these appeal- specific factors are correlated with both the choice of a DRO review rather than traditional review and other variables in our model, our model estimates may be biased. We tested several alternative specifications of the model, such as reclassifying those whose representation status was unknown to either having or not representation, and found that the direction and magnitude of our results were similar across models. We further confirmed that the model estimates were not sensitive to the inclusion or exclusion of body system or recategorization of the variable measuring the number of issues in an appeal. Overall, while we are confident that our models of the type of review provides some explanatory power and are robust to alternative specifications, they are limited in their ability to substantially explain individual veterans’ decisions to choose a DRO review rather than a traditional review.

We used appeals level VACOLS data to examine whether DRO reviews were more or less likely to have a full or partial award granted than traditional reviews. Table 3 shows the likelihood of grant awards by review type (DRO or traditional). Although 22.7 percent of appeals that underwent a traditional review received a full or partial award, 31.5 percent of appeals that underwent a DRO review received a full or partial award. Comparing the odds of a full or partial award for cases that underwent different review processes, the unadjusted odds ratio comparing the odds of an award for a DRO review to those of a traditional review is 1.56. In other words, prior to controlling for other factors that could affect the odds of an award, the odds that a DRO-reviewed appeal received an award were 56 percent higher than those for a traditional- reviewed appeal.

We used logistic regression analysis to assess if DRO reviews still had higher odds of receiving a full or partial grant, even after controlling for selected factors that could affect the likelihood of an award grant. We controlled for fiscal year; regional office; number of issues in each appeal, and body systems related to the issues under appeal; and veterans’ representation status. After controlling for these factors, we found that DRO-reviewed appeals still were significantly more likely than traditional reviews to receive an award. Specifically, table 3 shows that the odds that an appeal that underwent a DRO review received a full or partial award were still approximately 60 percent higher than an appeal that went underwent a traditional review after controlling for other factors (odds ratio 1.58). Besides a veteran’s choice of appeal option, some other variables were also correlated with higher odds of a receiving an award. For example, appeals filed in certain regional offices were significantly more likely to receive grants than appeals in other offices.

We also found that, after controlling for review type and other factors, the odds that an appeal filed by a veteran with representation won a full or partial award were quite similar to that of an appeal filed by a veteran without representation (odds ratio 1.03, or approximately 3 percent higher odds). Furthermore, compared to appeals with only one issue, veterans filing appeals with multiple issues had notably lower odds of receiving a full or partial award after controlling for other factors (between approximately 25 and 50 percent lower than the odds of appeals with one issue).

We tested a variety of alternative model specifications with different variables and populations. Given that a large portion of recent cases may not have been resolved as of the time our data were produced, we tested our model excluding the fiscal years 2009 and 2010 to avoid biasing our results towards those cases that could be quickly resolved. We confirmed that a DRO review still had higher odds of an award even after reclassifying the dependent variable into full grant versus partial or no grant. We further confirmed that the model estimates were not sensitive to the inclusion or exclusion of body system or recategorization of the variable measuring the number of issues in an appeal. We also tested a model with an interaction term between representation status and review type. These models confirmed that, compared to traditional cases with or without representation, appeals that went through DRO review were more likely to receive a full or partial grant. Overall, although our models provide some explanatory power and are robust to alternative specifications, we acknowledge that they are limited in their ability to substantially explain the outcome of the award process, especially in light of known omitted variables related to the specific case.

To obtain managerial views on the appeal process and use of DROs in VBA’s 57 regional offices, and to understand variations among offices, we conducted a web-based survey of one regional office manager, or assistant manager if so designated by their director, at each office.

Survey development. We developed survey questions with input from an official at VA’s Office of Field Operations and GAO subject matter experts on survey design and performance management. To pretest the questionnaire, we conducted in-depth probing interviews and held debriefing sessions with four regional office managers by telephone. Pretest participants were selected to represent variety in regional office sizes and to include offices that participated in VBA’s pilot program for new DRO performance measures. We conducted these pretests to determine if the questions were burdensome or difficult to understand and if they measured what we intended. On the basis of the feedback from the pretests and these other knowledgeable entities, we modified the questions as appropriate.

Survey implementation. We obtained e-mail addresses of managers from VA’s Office of Field Operations. This office sent a message to prospective respondents on December 20, 2010, encouraging participation in the upcoming survey. We then began the survey by e- mailing passwords and links to the web-based questionnaire on January 4, 2011. To obtain candid responses and a high response rate, we pledged not to link the responses presented in our report to individual survey participants, and we followed up with two e-mails to initial nonrespondents, the first on January 13, 2011, and the second on January 31, 2011. Additionally, we contacted those managers who had not responded to the survey e-mails by telephone from February 3 through February 18, 2011. We also contacted some respondents by e- mail to clarify unclear, inconsistent, or incomplete responses. We received usable responses from 56 of the 57 managers, for a 98 percent response rate, and ended the survey on February 22, 2011.

Analysis of responses. We used computer programs verified to be written correctly by an independent GAO analyst to analyze the responses. We provided respondents with an opportunity to answer several open-ended questions. The responses to those questions were categorized and coded for content by a GAO analyst, while a second analyst verified that the first analyst had coded the responses appropriately. Some comments were coded into more than one category since some respondents commented on more than one topic. As a result, the number of coded items is not equal to the number of respondents who provided comments.

Analysis of survey error. Because we identified and selected all 57 regional offices for our survey, our data is not subject to errors due to selecting only a sample or failing to include a portion of the population in the sample. The practical difficulties of conducting any survey may introduce errors due to measurement, nonresponse, or data processing; however, steps taken during survey development, implementation, and analysis minimize the chance of such errors. Additionally, because of the high response rate, the risk of nonresponse error was further minimized, and because the sole nonresponding office was also one of the pretest sites, we were able to determine that the site’s characteristics and pretest answers did not differ greatly from respondents’ answers. Inclusion of what would likely have been their final answers, as determined by their pretest responses, would not materially affect overall results. As a result, we conclude that there is no material risk of nonresponse bias in our survey.

Phone Interviews with Veterans

We conducted structured phone interviews with veterans to gather information on the factors that affect veterans’ appeals decisions, such as the clarity of the VBA letter appeals process election letter, assistance from VSOs, and other specific reasons for choosing a DRO review or a traditional review. The team worked with a survey methodologist and communication expert in the development of a phone script and interview questions for the veterans, which were pretested with six veterans. The finalized phone script and questions included screening questions to determine if veterans understood the questions or recalled information accurately.

To develop our sample, we obtained VACOLS data from the Board on veterans who had filed appeals of their disability compensation decisions between February 1, 2010 and July 31, 2010. The data provided by the Board included the date of the appeal; whether the veteran had selected a DRO or a traditional review; the date on which the DRO review was selected; the veteran’s name; and, when available, the veteran’s address and phone number. The file contained 77,542 unique appeals, which corresponds to something less than 77,542 unique veterans, because one veteran may have filed multiple appeals. We removed from the list all veterans associated with a regional office that was a participant in VA’s Expedited Claim Adjudication pilot project in 2008 since all veterans who took part in this pilot automatically received a DRO review, so our interview questions—which focus on how veterans made the decision to elect DRO review—would not have been relevant for them. Four regional offices were part of this pilot: Nashville, Tennessee; Seattle, Washington; Lincoln, Nebraska; and Philadelphia, Pennsylvania. The final file, without any veterans associated with the Expedited Claim Adjudication sites, contained 71,891 appeals. We drew a random sample of 200 appeals from this final data file. The sample of appeals represented 200 unique veterans. The team used a random sample to protect against selection bias.

Before conducting the interviews, we tried to obtain contact information for veterans with missing phone numbers or addresses, but were not able to identify contact information in all cases. When addresses were available, we sent notification letters to veterans before beginning the interviews. We conducted the interviews with veterans from December 2010 to April 2011. When contacting the veterans, the interviewer read the phone script and interview questions and documented the responses. In cases where a spouse or other family member stated that the veteran was not able to participate in the interview, we asked to speak with the person most knowledgeable of the appeal and conducted the interview with this person. We also spoke with surviving spouses or children who were appealing a disability claim of a deceased veteran. We successfully completed interviews with 40 veterans—28 of the veterans had chosen a DRO review of their appeal and 12 had chosen a traditional review. The results of our interviews cannot be generalized to the overall population of veterans who filed appeals between February 2010 and July 2010. (See table 4 for detailed information on the implementation of the phone interviews.)

Appendix II: VA Appeal Process Request Letter Template

Appendix III: Comments from the Department of Veterans Affairs

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Shelia Drake, Assistant Director; Lorin Obler; Susan Aschoff; Susan Baker; Jamie Berryhill; Jessica Botsford; Juliann Gorse; Mimi Nguyen; Anna Maria Ortiz; Patricia Owens; Carol Petersen; Carl Ramirez; and Walter Vance made key contributions to this report.

Related GAO Products

Military and Veterans Disability System: Worldwide Deployment of Integrated System Warrants Careful Monitoring. GAO-11-633T. Washington, D.C.: May 4, 2011.

High-Risk Series: An Update. GAO-11-278. Washington, D.C.: February 16, 2011.

Military and Veterans Disability System: Pilot Has Achieved Some Goals, but Further Planning and Monitoring Needed. GAO-11-69. Washington, D.C.: December 6, 2010.

Veterans’ Disability Benefits: Expanded Oversight Would Improve Training for Experienced Claims Processors. GAO-10-445. Washington, D.C.: April 30, 2010.

Veterans’ Disability Benefits: VA Has Improved Its Programs for Measuring Accuracy and Consistency but Challenges Remain. GAO-10-530T. Washington, D.C.: March 24, 2010.

Veterans’ Disability Benefits: Further Evaluation of Ongoing Initiatives Could Help Identify Effective Approaches for Improving Claims Processing. GAO-10-213. Washington, D.C.: January 29, 2010.

Military Disability System: Increased Supports for Servicemembers and Better Pilot Planning Could Improve the Disability Evaluation Process. GAO-08-1137. Washington, D.C.: September 24, 2008.

Veterans’ Benefits: Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors. GAO-08-561. Washington, D.C.: May 27, 2008.

Veterans’ Disability Benefits: VA Can Improve Its Procedures for Obtaining Military Service Records. GAO-07-98. Washington, D.C.: December 12, 2006.

Veterans’ Benefits: Further Changes in VBA’s Field Office Structure Could Help Improve Disability Claims Processing. GAO-06-149. Washington, D.C.: December 9, 2005.

Veterans’ Disability Benefits: Claims Processing Challenges and Opportunities for Improvements. GAO-06-283T. Washington, D.C.: December 7, 2005.

Managing for Results: Enhancing Agency Use of Performance Information for Management Decision Making. GAO-05-927. Washington, D.C.: September 9, 2005.

VA Disability Benefits: Board of Veterans’ Appeals Has Made Improvements in Quality Assurance, but Challenges Remain for VA in Assuring Consistency. GAO-05-655T. Washington, D.C.: May 5, 2005.

Veterans’ Benefits: More Transparency Needed to Improve Oversight of VBA’s Compensation and Pension Staffing Levels. GAO-05-47. Washington, D.C.: November 15, 2004.

Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government. GAO-04-546G. Washington, D.C.: March 2004.

Veterans’ Benefits: Improvements Needed in the Reporting and Use of Data on the Accuracy of Disability Claims Decisions. GAO-03-1045. Washington, D.C.: September 30, 2003.

Veterans Benefits Administration: Clarity of Letters to Claimants Needs to Be Improved. GAO-02-395. Washington, D.C.: April 23, 2002.

Human Capital: Practices That Empowered and Involved Employees. GAO-01-1070. Washington, D.C.: September 14, 2001.