Veterans' Benefits: Better Assessments Needed to Guide Claims Processing Improvements
Published: 1995-01-13T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed the Department of Veterans Affairs' (VA) efforts to improve its claims processing operations, focusing on the effectiveness of planned changes to veterans affairs regional offices' (VARO) claims processing structures and procedures.
section_title: What GAO Found
paragraphs: GAO found that: (1) VA has taken steps to ensure that VARO implement the changes necessary to improve overall service to veterans; (2) VA needs to implement the Blue Ribbon Panel's recommendations to improve disability claims processing; (3) VA has developed several model claims processing structures designed to reorganize staff so that fewer resources are devoted to clerical functions; (4) the models will serve as a framework for implementing other initiatives such as improving claims folder management and the use of evidence received by telephone or fax; (5) VA is also developing regulations and training materials to encourage VARO to adopt these improvement initiatives; (6) VARO have been given significant flexibility to implement initiatives in ways they believe are appropriate; (7) VA may not have a sound basis for determining what additional changes need to be made for guiding future improvements because it has not developed adequate plans for evaluating the merits of the various initiatives; and (8) VA does not have a formal mechanism to disseminate information about the effectiveness of regional initiatives and other VARO experiences with these initiatives.

section_title: Letter
paragraphs: Slow claims processing and poor service to customers have long been recognized as critical concerns for the Department of Veterans Affairs (VA). As early as 1990, VA began encouraging regional offices (VARO) to develop and implement improvements in their claims processing systems; but instead of decreasing, processing times and backlogs have increased. At the end of fiscal year 1994, almost 500,000 claims were waiting for a VA decision. About 65,000 of these claims were initial disability compensation claims. On average during fiscal year 1994, veterans waited over 7 months for their initial disability claims to be decided and, if approved, payments to begin; many waited much longer.
At your request, we examined VA’s efforts to address these problems. Specifically, we obtained information about VA’s current plans to implement changes in VARO claims processing structures and procedures and assessed VA’s plans to determine the effectiveness of those changes.
subsections: section_title: Background
paragraphs: VA’s Veterans Benefits Administration (VBA) is responsible for administering benefit programs, such as disability compensation and pension. Veterans and their families can apply for benefits at any of VA’s 58 VAROs. Significant differences exist among VAROs; for example, as of September 30, 1994, their claims processing staffs ranged in size from 11 to 219. Likewise, performance varies considerably; for example, the time needed to process initial disability claims ranged from 86 to 367 days in 1994. VA’s ability to process claims for benefits in a timely way has been a major topic of concern for many years.
In 1990, VA took steps to fundamentally change the way services are provided to veterans. A key element of those changes is modernization of VBA’s automated information systems, projected to be completed in 1998. Progress on this effort has been slow, and we have raised significant concerns about the adequacy of planning and implementation. In response to our initial work, VA agreed with the Director of the Office of Management and Budget (OMB) to, among other things, increase project oversight, establish outcome-oriented performance measures and document the system’s effect on service, and update the project’s economic analysis. The OMB agreement included timeliness goals to be met by the end of fiscal year 1998—as well as interim goals—for selected types of claims, including initial disability compensation and initial pension claims.
In 1990 the Secretary of VA also asked all VAROs to identify and implement innovative changes aimed at speeding up claims processing and reducing the growing backlog. In response, some VAROs undertook major restructuring initiatives, but most continued using the traditional “assembly-line” approach to processing. Under this approach, each claim passes through several individuals, each of whom performs a specific task. One person enters the claim into the computerized system and opens the claims file. Another then determines what information is needed and develops requests for that information. Another communicates with VA hospital staff if a physical examination is needed. These steps continue until an “authorizer” approves the decision. Often, files are centrally located and are sent back and forth from the central files to various claims processors many times before a claim is decided.
The claims backlog and processing times did not decrease but grew from 1990 to 1993. The backlog of compensation and pension claims grew from about 378,000 to about 528,000 during that period. Table 1 shows that, during the same period, average processing time increased for the four types of claims specifically included in VA’s agreement with OMB.
VA attributed its claims processing difficulties to several factors, including significantly increased workloads resulting from downsizing of the military, increased complexity of claims, and expanded responsibility resulting from decisions by the U.S. Court of Veterans Appeals created in 1988. “There is no ownership or accountability associated with the process. The claim physically moves from one location to the next, with each person responsible for a small part of the process and each movement contributing to further delay in the claim.”
The panel’s recommendations addressed what it saw as three key problem areas in claims processing: (1) inadequate claims development, (2) excessive response time for obtaining evidence, and (3) an unacceptably long time to rate cases. (See app. I for a list of the panel’s recommendations.)
The recommendations were based, in part, on initiatives already implemented in one or more VAROs and on panel members’ judgment. The panel was composed of people from both inside and outside VA with extensive experience and knowledge of VA operations and relied on expertise and judgment to identify root causes and develop recommended changes. Our work and the work of others have also identified these three areas as significant problems for VA.
During 1994 some VAROs continued or began making changes intended to improve claims processing, and VA worked to develop guidance and policies for implementing the Blue Ribbon Panel recommendations. During that year the number of claims awaiting a decision decreased somewhat, from 528,000 in 1993 to about 485,000 in 1994. However, average processing times increased and VA moved further away from, rather than closer to, the 1998 timeliness goals. Officials told us that processing times increased because, during the later part of the year, VA focused on reducing the backlog of old claims, thus increasing the average age of claims closed. Table 2 shows the average 1994 processing times for the four types of claims included in the OMB agreement compared with the average time in 1993 and the 1998 goals.
section_title: Scope and Methodology
paragraphs: To determine VA’s plans for implementing change, we examined ongoing and planned efforts to change claims processing structures and procedures in seven VAROs (see app. II). We judgmentally selected VAROs that differed in size and the number and type of changes already made. At these locations we discussed the impact of changes with officials, analyzed pertinent processing data and reports, and observed claims processing activities.
We also visited VA’s eastern and western area offices, where we discussed the initiatives that VAROs in each area had implemented and the area offices’ role in implementing and monitoring those and future initiatives.In addition, we analyzed the findings and recommendations of VA’s Blue Ribbon Panel and headquarters’ plan for implementing them.
To evaluate VA’s plans for determining the effectiveness of VARO changes, we discussed plans for assessing the impact of changes with officials from VBA’s Compensation and Pension Service and Program Analysis and Evaluation staff. We also discussed how VA plans to ensure that VAROs implement those initiatives that offer the greatest promise for solving their claims processing problems.
Our work focused on changes in VAROs’ claims processing structures and procedures and not on VBA’s computer system modernization effort or VBA’s reengineering task force—which is charged with looking beyond compensation and pension issues to improving operations throughout VBA. An ongoing GAO study is addressing VA’s systems modernization efforts and their relationship to the changes in VARO structures and procedures discussed in this report and to VBA’s reengineering task force. Additionally, at VA’s request, the Center for Naval Analyses is conducting an independent assessment of the coordination, control, and integration of key modernization activities, including their relationship to other initiatives aimed at improving claims processing.
Our review was conducted between October 1993 and August 1994 in accordance with generally accepted government auditing standards.
section_title: VA Is Developing Guidance to Help and Encourage VAROs to Implement Initiatives
paragraphs: VA has developed several model claims processing structures that incorporate some key Blue Ribbon Panel recommendations. In addition, VA is working to modify regulations and other claims processing policies to encourage and allow a variety of procedural changes. VAROs will have flexibility in deciding which initiatives to implement, given their individual circumstances. However, our review showed that some VAROs are not likely to implement some initiatives. They may be reluctant to make changes or face logistical obstacles to doing so. Also, they may not have knowledge of the experiences of other VAROs, knowledge that could overcome reluctance or show ways to get past obstacles.
subsections: section_title: Models Will Guide Implementation of Initiatives
paragraphs: The Blue Ribbon Panel concluded that VA’s traditional assembly-line claims processing system should be completely restructured. The goals of the revised structures represented in the models, as described by VA officials, are to put fewer resources into clerical functions and more into decisionmaking—especially rating claims—and to ensure good service as required by the Government Performance and Results Act.
To replace the current assembly-line system, VA developed models that reorganize staff along two basic types of team structures, one based on a case management approach and the other on a functional alignment approach. The case management approach organizes staff into small work teams responsible for all claims processing steps for all or most types of claims. This approach reduces the number of staff involved in processing each claim. The functional alignment approach organizes staff into two types of work teams. One team handles the processing of all claims that require a rating decision, thereby allowing some specialization of staff responsible for the most complex claims VA processes. The other team performs all claims-related activities for claims that do not require a rating decision.
Implementation of each approach could follow one of two paths. Option one would integrate some of the VAROs’ staff responsible for all direct contact with customers—Veterans Services Division staff—with staff responsible for processing claims—Adjudication Division staff. This would allow a veteran to talk directly, in person or by telephone, to the individuals most knowledgeable about his or her specific claim. Option two would keep the functions of the two divisions separate during a transitional phase, after which the two divisions would be fully integrated.
section_title: Other Initiatives Directed at Encouraging Revised Processing Procedures
paragraphs: In addition to changing the claims processing structure, VA is planning a variety of other initiatives—not specifically related to any one model—to improve processing procedures. These initiatives include, for example, allowing claims examiners to contact claimants by telephone, developing a system to better track and locate claim files, and having claims examiners specialize by type of claim such as initial disability compensation or pension. Although empirical data were often not available to show a positive impact from these initiatives, VARO officials we spoke to who were implementing them believed the initiatives were improving timeliness or other aspects of service.
Four VAROs that we visited allowed claims processors to contact sources of evidence by telephone rather than by the standard practice of sending a letter. Officials at all four VAROs found that using telephones was helpful. Officials at one VARO said that contacting sources by telephone shortens the time required to obtain the evidence and helps ensure that claimants and other sources of evidence understand exactly what VA needs. Officials at another VARO noted that applications frequently come in lacking critical information such as social security numbers. In such instances, processors simply telephone applicants to obtain the missing information.
Locating files is a continuing problem in VA, one that regional officials acknowledged takes considerable staff time. One VARO modified VA’s existing computer system to better track claim files. The files at this VARO were well organized, and officials said they have almost eliminated the problem of lost and misplaced files. Two other VAROs planned to modify their systems in a similar manner. In addition, VA is revising existing computer software that uses bar codes to track files. The revisions will allow more VAROs to use the bar code system.
Another initiative—specialization—allows processors to become more knowledgeable about complex issues related to a specific type of claim. According to officials, this practice increases processors’ proficiency. Data from one VARO that implemented specialized work teams late in 1993 show that processing times decreased for the four types of claims included in VA’s agreement with OMB. For example, the time to process initial disability compensation claims decreased from 161 days in 1993 to 141 days in 1994.
section_title: VAROs Have Flexibility in Choosing Improvement Initiatives
paragraphs: In trying to improve claims processing, VA is allowing VAROs to make the changes they themselves deem necessary. VA has mandated that VAROs choose one of the models for reorganizing staff as a basis for their new claims processing structures. However, VAROs can modify the chosen model. VA disseminated the models to the VAROs in late November 1994. By January 1995, each VARO must submit a proposed claims processing structure for VA approval.
In general, VA headquarters’ response to the Blue Ribbon Panel recommendations has been to amend policies to allow, but not require, VAROs to implement changes, such as using telephones in claims development or removing the requirement for review and approval of the decision on each claim. VA officials said that regional directors are in the best position to determine whether specific actions will work in their given situation. In their opinion, mandating specific actions nationwide without considering the diversity that exists among VAROs—such as size and local resources—would be counterproductive.
section_title: VAROs Might Not Implement All Needed Changes
paragraphs: Some VAROs may be reluctant to make some changes or may face difficulties in doing so. This reluctance could explain the slow progress many VAROs have made in implementing changes. Early in 1994, more than 3 years after the Secretary called for VAROs to make fundamental and innovative changes, only 35 of 58 VAROs responded positively to VA’s request for information on changes made. On average, the 35 VAROs made fewer than three changes, and some of those changes were minor. For example, one VARO simply displayed graphs showing claims processing goals and target dates in the claims processing work area.
Some VAROs we visited were reluctant to implement changes that appear to have considerable advantages. Officials at several VAROs, for example, expressed concern about allowing claims processors to use telephones to contact veterans, although VA officials believe that such contact is helpful. One VARO official said he believed this would lead staff to use the telephones for personal business. An official at another VARO was concerned that staff would spend too much time “on hold” waiting for responses from institutions such as VA hospitals.
Furthermore, VAROs that want to change may have difficulty doing so. For example, two VAROs we visited were limited physically in how much they could change. One had recently renovated its space and installed modular furniture, which limited its ability to lay out its space to accommodate work teams. The other had implemented teams but could not store their files in close proximity to the teams because the floor was not strong enough to support the weight of the files. (Colocating files is generally thought to increase the efficiency of teams and improve customer service.) Likewise, regional and headquarters officials noted that some VAROs may encounter physical limitations that would make it difficult to provide all claims processors access to telephones.
When we discussed these VARO concerns with officials in VBA’s Compensation and Pension Service, they reiterated that these are the kinds of problems that necessitate flexibility: not all VAROs can implement all changes. They said, however, that in some cases they would negotiate with VARO officials to encourage implementation of specific initiatives, such as using telephones to request information.
section_title: VAROs May Not Know of All Possible Changes
paragraphs: Some VAROs may not be fully aware of initiatives that have been implemented at other VAROs. Although VA headquarters disseminates information about regional initiatives at periodic headquarters-sponsored meetings of claims processing officials, much of the information sharing among VAROs is informal. There is no reliable mechanism by which VA either collects or disseminates complete information about regional experiences so that VAROs can learn from each other.
Much of the information sharing results from informal networking. For example, at one VARO we visited, officials had learned of other VAROs’ examples through informal contacts. Officials at the one VARO took it upon themselves to travel to another to learn about the second VARO’s efforts and results. These informal methods do not guarantee complete information sharing. One official noted that VAROs may not voluntarily share information about initiatives. Likewise, VARO officials who do not make the effort to network may not learn of many initiatives.
One area office director noted that VA headquarters needs to do a much better job of compiling and disseminating information about claims processing initiatives. The experience of one VARO demonstrates the usefulness of more formal mechanisms. Officials at that VARO said they learned of an initiative, which they subsequently implemented, during a teleconference the area office set up to discuss ways to reduce claims processing time.
Recently, VA has tried to improve information dissemination. VA focused much of its September 1994 meeting of adjudication officers on new initiatives. Much of the discussion concerned new claims processing initiatives that some VAROs have implemented or that VA has proposed—including the new claims processing structures.
However, VA’s ability to inform VAROs about initiatives is limited because VA headquarters does not have complete information about regional experiences, either the initiatives that have been tried or their effectiveness. The compensation and pension staff responsible for monitoring VAROs did not have a list showing all initiatives. That staff’s March 1994 data showed that 23 VAROs had implemented 50 initiatives, yet data obtained by the VBA reengineering task force showed that, as of January 1994, 35 VAROs had implemented 86 initiatives. Four VAROs, for example, had implemented some form of claims processing work teams on which the compensation and pension staff had no information. Also, at one VARO we visited, mail clerks processed all death notices received by mail instead of forwarding them to claims processing. This initiative reduced the workload of the claims processors and ensured timely termination of payments but was not included in the data of either the compensation and pension staff or the task force.
section_title: Evaluation Plans Are Inadequate to Ensure Ongoing Improvement
paragraphs: VA’s current evaluation plans will not provide sufficient information for it to effectively assess VARO initiatives and guide future improvements in VARO operations. This is especially critical because information currently available about the effectiveness of initiatives has been inconclusive. Better evaluation could position VA to react quickly to unsatisfactory results and more effectively disseminate needed information among VAROs.
subsections: section_title: Available Data Are Inconclusive About Effectiveness of Initiatives
paragraphs: In developing initiatives, VA relied on experience and judgment. The only empirical evidence about initiatives comes from the experience of the VAROs that have already implemented some of the initiatives. However, VA has not required VAROs to evaluate their initiatives and has not provided guidance to those wishing to do so. Not all VAROs have done evaluations, and those done have been inconclusive. An official of VBA’s Program Analysis and Evaluation staff told us that, according to his recent discussions with VARO officials, those officials want headquarters to provide this type of guidance.
Some of the VAROs we visited performed weak evaluations. For example, analyses usually considered only the initiatives’ impact on overall processing time or backlog; they did not consider other possible impacts, such as improved communications with veterans. Similarly, some evaluations had technical flaws. One VARO compared the quality of processing for a prototype, team-based unit with that of its unit using the assembly-line approach. Although the comparison showed that the prototype unit was more accurate, the study’s statistical sampling methodology did not allow a valid comparison, raising questions about its conclusion.
In other cases, VAROs experienced outcomes that were contradictory or could not clearly be explained by changed procedures. For example, two VAROs of similar size established similar types of specialized claims processing teams but had different results. For unexplained reasons, one’s processing times continued to increase while the other’s decreased. Likewise, where VAROs seemed to be improving, the reasons were unclear. VA identified four VAROs that had recently begun to meet some of the department’s claims processing goals: One used specialization and met processing goals; the other three are among VA’s smallest VAROs, and officials acknowledged that the three were among those that traditionally had the best processing times anyway. In fact, two of those VAROs had reported no changes in their processing structures and procedures.
Data are also inconclusive because some initiatives may not have been in place long enough to determine their full impact. It is not clear how long evaluations should continue to accurately assess results. The importance of this issue is demonstrated by dramatically different actions involving three VAROs that have implemented claims processing work teams. Two VAROs disbanded their claims work teams after 7 months or less because processing times or backlog had not been reduced. In contrast, another VARO is continuing to use work teams even though, after nearly 2 years, its processing times and backlog have continued to increase. Additionally, some initiatives can only be implemented fully over the long term so their full impact cannot be evaluated in the short term. For example, the panel’s recommendations included assigning and training additional staff to the rating activity and certifying rating specialists. Revised training materials, performance standards, and a method for certifying rating specialists are not scheduled to be ready until June 1995; then, officials said, it could take 2 years to fully train staff. Therefore, although interim assessments can be made, a full assessment of these initiatives will take several years.
section_title: VA Headquarters Will Have Limited Information About VARO Initiatives and Their Effectiveness
paragraphs: VA headquarters plans to continue to routinely assess each VARO’s overall performance in the areas of timeliness, quality, and productivity using national data. Monitoring each VARO’s overall performance in this way is clearly a necessary step. VA needs to know how well regional initiatives, in total, are working. But overall outcome data alone are insufficient.
Following its traditional monitoring and evaluation practices, headquarters will evaluate overall outcome data—such as total average time to process each type of claim—for each VARO, semiannually. Each VARO’s progress can be compared with its own past performance and measured against VA’s national goals. Headquarters staff also have a goal of making an on-site visit to each VARO every 2-1/2 years. Additionally, as part of ongoing oversight, area offices will continue their traditional monitoring of VARO operations, including review of outcome data. Using this approach, VA will know which VAROs are improving but will have little sense of what led to the changes or how to help VAROs that are not improving.
To guide VAROs, VA will need insight into which initiatives work best under which circumstances and what factors lie behind or obstruct improvement. For example, VA could use information on the following:
How individual VAROs implemented their initiatives to help VA interpret why VAROs implementing the same or similar initiatives get different results: For example, several VAROs have created a rating analyst technician position but are using that person differently and may obtain different results.Interim and short-term outcomes to help monitor progress and assess individual initiatives: Because some initiatives address only a part of the process, data related more directly to the initiative itself rather than overall outcomes may be more relevant. For example, for the rating analyst technician who screens claims, the more important measure might be backlogs at the rating board rather than overall backlogs.
A variety of factors that could be expected to affect outcomes: These factors might include staff turnover (implementing initiatives may actually increase staff turnover in the near term as job descriptions are changed), workload, and number of cases returned by the Board of Veterans’ Appeals for insufficient evidence.
When VA disseminated the new organizational models in November 1994, it mandated that VAROs conduct periodic assessments as part of implementing the models. VA did not, however, specify the nature or scope of those assessments or provide guidance on how they should be conducted.
section_title: VA Needs to Determine What Information It Needs and Implement a Plan to Obtain It
paragraphs: In discussing with us the need for better evaluation of initiatives, officials in the Compensation and Pension Service expressed uncertainty about how to evaluate VARO initiatives to provide headquarters with sufficient information. Although some steps have been taken to determine what information should be collected, VA still needs to (1) determine what information is most critical to interpreting results and (2) develop a plan for obtaining and analyzing the data.
VBA’s Program Analysis and Evaluation staff have recognized the need to develop performance measures that are specific to the local environment and the particular initiative. In June and July of 1994, the evaluation staff visited five VAROs to study their work teams and develop ideas for measuring the progress and success of various initiatives. The staff plan to use this information to make suggestions to senior VA management. (These suggestions will incorporate customer satisfaction considerations as required by the Government Performance and Results Act.) This work could be an important first step in developing the information needed to effectively oversee ongoing efforts to improve claims processing. At this point, however, management has not indicated what action it will take.
Once VA determines the basic information needed, it can employ a variety of evaluation methods. Ideally, VA would use control groups, possibly setting up separate sections within VAROs, one or more using the revised structure and procedures and others not. Control groups would allow VA to more confidently determine whether changes resulted from the initiatives or from unrelated factors, such as workload or staff turnover. But this method is problematic. Some portion of VARO workload would have to continue to use the existing approach at a time when management sees change as critically needed. Also, VAROs would have to operate for some time using two processing structures, which could significantly strain operations. Though evaluation based on control groups is ideal, it is not absolutely necessary, however. When making management decisions in an organization as diverse as VA, it is not always possible to obtain the definitive information gained from control group methodology.
Other evaluation approaches are acceptable. Various statistical methods, for example, would allow VA to compare change over time, using past data to project what the situation would have been—for example, average processing times—if no change in approach had been made and comparing it with the situation under the new approach. Alternatively, qualitative methods could, for example, provide detailed case study information for selected VAROs, focusing on the most important initiatives and choosing VAROs to obtain a mix of approaches and circumstances. Whatever the approach, either VARO staff or headquarters staff could develop the information.
section_title: Conclusions
paragraphs: Given the urgent need for improving claims processing, the uncertainty about which initiatives will be most effective, and the extent to which some VAROs have already begun making changes, allowing regional flexibility has merit. VAROs can be expected to have different experiences with similar initiatives and therefore need some flexibility.
However, if first efforts do not result in sufficient improvement, the VAROs and headquarters need to understand why and to have some basis for determining what other changes have a better chance of success. VA needs information to gain meaningful insight into whether initiatives are working—including whether they are addressing the most significant causes of problems—and how they are affected by regional circumstances. Different results may reflect many factors, not only differences in the types of initiatives undertaken and in VARO size and resources but differences in motivation and commitment to improvement.
Without meaningful information to interpret VARO outcome data, headquarters will be hard pressed to ensure improvements as time goes on. Valid VARO assessments of initiatives are critical. Equally important, VA headquarters must understand how results of at least the most significant initiatives were affected by individual VARO circumstances. This broader understanding will better enable VA to disseminate information to VAROs about the pros and cons of various initiatives, provide guidance about what changes to make, and, if necessary, direct VAROs to make specific changes.
section_title: Recommendations
paragraphs: To better ensure improvement in VARO claims processing, we recommend that the VA Secretary direct the Under Secretary for Benefits to improve plans to evaluate the effectiveness of claims processing initiatives. The improved plans should provide both headquarters and VAROs sufficient information about the effect of initiatives to allow quick response if results are unsatisfactory and to implement even greater improvements if possible. Therefore, the plan should require VAROs to evaluate their major improvement initiatives and provide guidance on how to do so; identify which analytical methods and which data VA headquarters will use to evaluate the various initiatives and make judgments about what changes are most likely to improve claims processing under what circumstances; and describe how VA will disseminate to VAROs information on the experiences, good and bad, that VAROs have in implementing claims processing initiatives.
section_title: Agency Comments
paragraphs: In a letter dated December 13, 1994, commenting on a draft of this report, the Secretary of Veterans Affairs disagreed with our recommendation to develop and implement an evaluation plan. He indicated that VA has in place an evaluation process that includes assessment of performance indicators and that through that process VA reviews, monitors, guides, assesses, and exports initiatives of significance. The Secretary said that the process involves all levels of VBA—from headquarters, including the Compensation and Pension Service; the area offices; and the VAROs themselves. The Secretary also noted that VBA’s project to reorganize claims processing—the major focus of this study—had been in development, testing, and evaluation for at least 2 years. On November 29, 1994, VBA issued organization models to guide VAROs in the future. VA believes its current process—including analysis of outcome data and ongoing monitoring—along with the knowledge and judgment of VA staff, is sufficient to determine the most effective initiatives and provide guidance to VAROs that are not making sufficient progress.
In response to the Secretary’s comments, we clarified our recommendation about an evaluation plan to recognize that VA has an evaluation process in place. We continue to believe, however, that the existing process is inadequate. We believe a more thorough evaluation is needed to enable VA to understand not only the outcomes but their causes and to effectively persuade—and, as appropriate, direct—VAROs to adopt the most promising changes.
In support of the effectiveness of its improvement efforts to date, VA emphasized that data on average processing times began to show improvement in fiscal year 1995. VA said that processing times for several types of claims for the month of October 1994 were shorter than the times we report for fiscal year 1994 (ended September 1994). It is not clear, however, that these recent data are indicative of an improvement trend. More important, even if they do indicate a trend, VA’s current evaluation process does not allow VA to determine whether changes to the claims processing structure caused the improvement.
Interpreting the October 1994 data as the beginning of an improvement trend is questionable because monthly average processing times fluctuate significantly. The national average monthly processing time for original disability compensation claims in fiscal year 1994 ranged from 198 to 227 days while, as we reported, the annual average for that year rose to 212 days from the 1993 average of 198 days. The problem is clearer when viewed at the VARO level. At one VARO, cited by officials as a leader in improving claims processing, the average monthly processing time for original compensation claims fluctuated during fiscal year 1994 from a low of 74 days to a high of 143 days; for 10 months of fiscal year 1994, this VARO’s average was lower than its October 1994 average.
More important, whether these data indicate the beginning of a positive trend or not, the VA’s current evaluation process cannot explain with any certainty why these changes are occurring and cannot confidently point to characteristics of VAROs or specific models that have the highest probability of success. For example, VA officials told us that during the later part of fiscal year 1994 VAROs had focused on closing the oldest claims (those over 180 days old). Because, by definition, closing older claims increases average processing times, the reduction in October 1994 may not have resulted from any claims processing initiatives, but, instead, from the 1994 focus on older claims.
VARO experiences also demonstrate the difficulty in interpreting outcome data. Although VA points out that its claims processing project has been in development and testing for 2 years, the outcome data—VBA’s key evaluation tool—are inconclusive about the effects of the models. For example, VAROs implementing similar initiatives achieved different results. In fact, the VAROs we visited that had the most experience with changed claims processing structures have not shown a trend toward improved processing times. In the New York VARO—which played a key role in VA’s testing and evaluation of one of the new organization models—data comparing the processing times of staff using the new model with the rest of the staff did not show the new model to be faster.
The Secretary also raised a concern about the possible negative impact of our recommendation. He stated that VARO staff were continually seeking ways to improve processing and that it would be “unnecessary and would stifle creativity for all levels of management to know of and to control” each of the many changes until an evaluation showed them to have positive or negative impact. We agree that local creativity should be encouraged. We have not suggested waiting to implement changes in processing structures and procedures until evaluations prove them effective, nor have we suggested that every initiative be evaluated in every VARO. Our report specifically recognizes the urgent need for change in claims processing structures and that some initiatives may be more important than others. However, absent evaluation before widespread implementation, we believe VA should position itself to evaluate at least those initiatives it believes to be the most important, and to do so in a way that allows it to understand the impact different VARO circumstances have on initiatives’ effects.
We are sending copies of this report to the Chairman, Senate Committee on Veterans’ Affairs, the Secretary of Veterans Affairs, and other interested parties. This work was performed under the direction of Ruth Ann Heck, Assistant Director. Other major contributors were Richard Wade, Steve Morris, Pamela Scott, and Charles Taylor. Please contact me on (202) 512-7101 if you have questions about this report.
section_title: Blue Ribbon Panel Recommendations
subsections: section_title: Problem: Inadequate Claims Development
subsections: section_title: Recommendations
paragraphs: 1. Prepare and implement position descriptions to consolidate responsibility for control (i.e., inputting claims into the computer system), development, and award of claims. The consolidated position would be called a rating technician. 2. Create a rating activity responsible for control, development, rating, and authorization of claims requiring a rating. Compile and distribute models for the structure of consolidated rating activities containing both rating specialists and rating technicians. Require all VA regional offices (VARO) to submit for headquarters approval a locally designed plan to restructure their claims processing systems. 3. Elevate to the level of a war effort, the creation, testing, and implementation of the Claims Processing System. This system will be used (1) to help claims processors determine the exact evidence needed to support each claim and (2) to monitor the receipt of that evidence. (VA is developing this computer software package as part of its computer modernization program.) 4. Provide automated on-line access to reference materials (that is, regulations, policy and claims processing manuals, and so forth) through implementation of the Automated Reference Material System. (VA is developing this computer software package as part of its computer modernization program.) 5. Deploy manual development checklists for all aspects of claims processing. 6. Prepare a centralized training program for developing claims. 7. Finish the redesign of the application for disability compensation and pension benefits, and field test the redesigned application. 8. Design a new form to help veterans identify issues and evidence needed to support reopened claims and claims for reevaluation of service-connected disabilities. Convene focus groups to obtain feedback on the design of the new form, and field test the form.
section_title: Problem: Excessive Response Time for Obtaining Evidence
subsections: section_title: Recommendations
paragraphs: 9. Develop, field test, and implement a standard, national package of computer generated letters using input from all VA customers to clarify/improve communications between VA and its customers. 10. Change VA guidelines/procedures to allow claims processors to use other communication modes (telephone, facsimile machine, personal contact, pager, and E-mail). Use these other modes to supplement written communications between claims processors and claimants and other evidence sources. 11. Revise forms/systems to include claimant telephone numbers—both daytime and nighttime. 12. Expand the memoranda of understanding between the Veterans Benefits Administration (VBA) and the Veterans Health Administration (VHA) to include examination quality measures. (VHA completes medical examinations for VBA.) 13. Establish a reporting scheme to monitor the quality, local and national, of VHA examinations. 14. Establish physicians’ coordinators at VA headquarters, medical centers, and VAROs to improve the timeliness and quality of examinations. 15. Establish a joint VBA/VHA education and training effort concerning disability compensation and pension examinations. 16. Improve the automated medical information exchange (AMIE) examination process. (AMIE is a computer system through which VBA requests examinations and VHA reports the results.) 17. Transfer responsibility and associated resources for disability compensation and pension examinations from VHA to VBA. 18. Establish a high-level dialogue with the Social Security Administration (SSA) to communicate VA’s evidence and other needs. 19. Update/verify VBA procedural guidance on obtaining SSA records. 20. If possible, establish a VA/SSA computer link to obtain SSA medical records. 21. Expand the current agreement with the Department of the Army branches for obtaining service medical records to all military service. 22. Assign VA personnel to Department of Defense records centers to assist in obtaining service medical records and to perform liaison activities. 23. Change VBA procedures and forward the claims of separating military personnel to the VARO serving their home state immediately, rather than waiting to send claims from the VARO serving the state where the separating personnel were located. 24. Seek guidance from the environmental support group regarding their sources and capabilities. (The environmental support group is a Department of Defense organization that assists VA in adjudicating claims involving service-connected stress.) 25. Provide guidance on use of evidence sources other than the environmental support group for development of claims involving post traumatic stress syndrome. 26. Continue to educate VBA and VHA staff and veterans service organizations regarding developing claims involving post traumatic stress syndrome. 27. Revise VA regulations to allow acceptance of photocopied documents, rather than requiring certified documents. 28. Ensure that the veterans network design incorporates tracking of case status through the appeal process. (VA is developing the veterans network as part of its computer modernization program.) 29. Initiate national VA/Department of Defense dialogue concerning examinations given to separating military personnel to ensure that the examinations meet VA requirements. 30. Educate Department of Defense medical staff concerning requirements for VA examinations.
section_title: Problem: Unacceptable Amount of Time to Rate Claims
subsections: section_title: Recommendations
paragraphs: 31. Provide personal computer processing capability for the rating staff to include standardized formats and glossaries. 32. Use specialization selectively to concentrate on certain categories of complex rating cases. 33. Expand and expedite centrally coordinated training for rating staff. 34. Develop formal training programs for rating staff, and require that the staff obtain certification for rating claims. 35. Develop centralized training for rating staff that utilizes videos, video- and teleconferencing, satellite, and interactive personal computer-based programs. 36. Conduct a special review of VA regulations, manuals, and policies to refine them. 37. Reallocate staff resources to the rating activity; and train staff in the areas of rating, development, and authorization. 38. Complete the evaluation of single-signature authority being tested. (This test eliminated the requirement that a second rating specialist review claims.) 39. Establish help teams wherein several rating specialists from one or more VAROs are temporarily assigned to a VARO with a large backlog of cases awaiting a rating. 40. Implement the veterans records control system as soon as possible. (VA is developing this computer software package as part of its computer modernization program.) 41. Develop, test, and implement the rating board automation system. (VA is developing this computer software package as part of its computer modernization program.)
section_title: Ongoing Initiatives at VA Regional Offices That GAO Visited
paragraphs: Placed all education claims files in one location for easier access and better control.
Allowed veterans benefits counselors to execute simple adjudication claims processing tasks for education claims so that adjudicators could perform more complex adjudication tasks.
Participated with another VARO in developing a computer software word processing package for preparing rating decision statements.
Converted a traditional claims processing unit to a case management team in October 1993, but disbanded the team after about 4 months of operations because processing times and backlog had not decreased.
In April 1994, reorganized all staff into specialized claims processing work teams—one to process claims requiring a rating decision and one to process claims that do not require a rating decision.
Created a rating analyst technician position to screen each claim before it is sent to the rating board to ensure that the claim has been properly developed and is ready for action by the rating board.
Allowed claims examiners to begin using telephones in lieu of letters to contact veterans and others to request evidence needed to expedite adjudication of a claim.
Established no new initiatives.
Established case management self-directed work teams in May 1993 to process 25 percent of the office’s workload. These work teams consolidated claims processing and veterans assistance functions and created 2 positions to perform tasks that had been performed by up to 10 individuals.
Placed all claims processing work under self-directed work teams in August 1994.
In June 1993, established claims processing work teams that included both adjudicators and veterans benefits counselors, but the functions of the individual team members were not changed.
Created a rating analyst technician position to assist in the initial development of claims.
Developed a check list that shows the evidence needed to support the different types of claims, with a goal of more fully developing claims. (continued)
In November 1992, established claims processing work teams along the case management approach to process selected types of claims. In April 1994, reorganized the teams to process 50 percent of all types of claims.
Allowed claims examiners to begin using telephones in lieu of letters to contact veterans and others to request evidence needed to expedite adjudication of a claim.
Created a rating analyst technician position to screen each claim before it is sent to the rating board to ensure that the claim has been properly developed and is ready for action by the rating board.
Implemented a practice of conditionally approving claims on the basis of photocopies of certified documents until certified copies are obtained.
Tested a practice of finalizing claims without independent review by a second person.
Developed a check list that shows the evidence needed to support the different types of claims, with a goal of more fully developing claims.
In early 1994, established two specialized claims processing teams—one to process claims requiring a rating decision and one to process claims that do not require a rating decision. The two teams were converted to case management claims processing work teams in June 1994.
Modified VA’s computer system to permit tracking of claims files.
section_title: Comments From the Secretary of Veterans Affairs
paragraphs: The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (301) 258-4066, or TDD (301) 413-0006.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (301) 258-4097 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.
subsections: section_title: Address Correction Requested