Veterans' Benefits: Improvements Needed in VA's Training and Performance Management Systems
Released: 2008-09-18T00:00:00

Letter

Background

The Veterans Benefits Administration (VBA) within VA administers the disability compensation and pension programs, whereby VA claims processing staff assess veterans’ applications for disability compensation and pension benefits. Aside from benefits for veterans, VBA claims processing staff make eligibility determinations for deceased veterans’ spouses, children, and parents. In short, they are responsible for ensuring that the decisions that lead to paying disability compensation and pension benefits are timely, accurate, and consistent.

The VA disability claims process involves multiple steps and usually involves more than one claims processor. When a veteran submits a claim to one of VBA’s 57 regional offices, staff in that office are responsible for obtaining evidence to evaluate the claim, such as medical and military service records; determining whether the claimant is eligible for benefits; and assigning a disability rating specifying the severity of each of the veteran’s impairments. These ratings determine the amount of benefits eligible veterans will receive.

VA has faced questions about the timeliness, accuracy, and consistency of its disability decisions. GAO designated federal disability programs, including VA and other programs, as a high-risk area in 2003. In particular, our prior work found VA relied on outmoded criteria for determining program eligibility that did not fully reflect advances in medicine and technology or changes in the labor market. As a result, VA’s disability program may not recognize an individual’s full potential to work. In addition, VA has seen processing times for their disability claims increase over the past several years, and inconsistencies in disability decisions across locations have raised questions about fairness and integrity.

Some have suggested that VA needs to address its training and guidance related to claims processing in order to improve consistency and that it should conduct periodic evaluations of decisions to ensure the accuracy of ratings across disability categories and regions. VA has reported that some of the inconsistency in its decisions is due to complex claims, such as those involving post-traumatic stress disorder, but it has also acknowledged that the accuracy and consistency of claims decisions needs further improvement.

Training Complies with Some Accepted Practices, but VBA Does Not Adequately Evaluate Training and May Be Falling Short in Training Design and Implementation

To prepare newly hired staff to perform the tasks associated with processing disability claims, VBA has developed a highly structured, three- phase program designed to deliver standardized training. The first phase is designed to lay the foundation for future training by introducing new staff to topics such as medical terminology and the computer applications used to process and track claims. The second provides an overview of the technical aspects of claims processing, including records management, how to review medical records, and how to interpret a medical exam. The third includes a combination of classroom, on-the-job, and computer- based trainings. The second and third phases in this program are designed to both introduce new material and reinforce material from the previous phase.

To help ensure that claims processing staff continually maintain their knowledge after their initial training and keep up with changing policies and procedures, VBA’s Compensation and Pension Service requires all claims processing staff to complete a minimum of 80 hours of technical training annually. This training requirement can be met through a mix of classroom instruction, electronic-based training from sources such as the Training and Performance Support System (TPSS), or guest lecturers. VBA’s regional offices have some flexibility over what courses they provide to their staff to help them meet the training requirement. These courses can cover such topics as establishing veteran status, asbestos claims development, and eye-vision issues.

We found that VBA has taken some steps to strategically plan its training for claims processors in accordance with generally accepted training practices identified in our prior work. For example, VBA has taken steps to align training with the agency’s mission and goals. In 2004, VBA established an Employee Training and Learning Board (board) to, among other things, ensure that the agency’s training decisions support its strategic and business plans, goals, and objectives. Also, VBA has identified the skills and competencies needed by its claims processing staff by developing a decision tree and task analysis of the claims process. In addition, VBA has taken steps to determine the appropriate level of investment in training and to prioritize funding. The board’s responsibilities include developing an annual training budget and recommending training initiatives to the Under Secretary of Benefits. Further, we found that VBA’s training program for new claims processing staff appears well-designed, in that it conforms to adult learning principles by carefully defining all pertinent terms and concepts and providing abundant and realistic examples of claims work.

However, while VBA has developed a system to collect feedback from new claims processing staff on their training, the agency does not consistently collect feedback on all of the training it provides. For example, none of the regional offices we visited consistently collected feedback on the training they conduct. Without feedback on regional office training, VBA may not be aware of how effective all of its training tools are.

Moreover, both new and experienced claims processing staff we interviewed reported some issues with their training. A number of staff told us the TPSS was difficult to use, often out-of-date, and too theoretical. Some claims processing staff with more experience reported that they struggled to meet the annual training requirement because of workload pressures or that training topics were not always relevant for staff with their level of experience. VBA officials reported that they have reviewed the 80-hour training requirement to determine if it is appropriate, but they could not identify the criteria or any analysis that were used to make this determination. Identifying the right amount of training is crucial. An overly burdensome training requirement may needlessly take staff away from essential claims processing duties, while too little training could contribute to processing and quality errors.

In addition to lacking a clear process for assessing the appropriateness of the 80-hour training requirement, VBA also has no policy outlining consequences for individual staff who do not complete the requirement. Because it does not hold staff accountable, VBA is missing an opportunity to clearly convey to staff the importance of managing their time to meet training requirements, as well as production and accuracy goals. In fiscal year 2008, VBA implemented a new learning management system that allows it to track the training hours completed by individual staff. Although VBA now has the capacity to monitor staff’s completion of the training requirement, the agency has not indicated any specific consequences for staff who fail to meet the requirement.

VA’s Performance Management System Generally Conforms with Accepted Practices, but May Not Clearly Differentiate among Staff’s Performance Levels

VA’s performance management system for claims processors is consistent with a number of accepted practices for effective performance management systems in the public sector. For example, the elements used to evaluate individual claims processors—such as quality, productivity, and workload management—appear to be generally aligned with VBA’s organizational performance measures. Aligning individual and organizational performance measures helps staff see the connection between their daily work activities and their organization’s goals and the importance of their roles and responsibilities in helping to achieve these goals. VA also requires supervisors to provide claims processors with regular feedback on their performance, and it has actively involved its employees and other stakeholders in developing its performance management system.

However, VA’s system may not be consistent with a key accepted practice—clear differentiation among staff performance levels. We have previously reported that, in order to provide meaningful distinctions in performance for experienced staff, agencies should preferably use rating systems with four or five performance categories. If staff members’ ratings are concentrated in just one or two of multiple categories, the system may not be making meaningful distinctions in performance. Systems that do not make meaningful distinctions in performance fail to give (1) employees the constructive feedback they need to improve and (2) managers the information they need to reward top performers and address performance issues.

VA’s performance appraisal system has the potential to clearly differentiate among staff performance levels. Each fiscal year, regional offices give their staff a rating on each individual performance element: exceptional, fully successful, or less than fully successful. For example, a staff member might be rated exceptional on quality, fully successful on productivity, and so forth. Some elements are considered critical elements, and some are considered noncritical. Staff members are then assigned to one of five overall performance categories, ranging from unsatisfactory to outstanding, based on a formula that converts a staff member’s combination of ratings on the individual performance elements into an overall performance category (see fig. 1).

However, there is evidence to suggest that the performance management system for claims processing staff may not clearly or accurately differentiate among staff’s performance. Central office officials and managers in two of the four regional offices we visited said that, under the formula for assigning overall performance categories, it is more difficult to place staff in certain overall performance categories than in others—even if staff’s performance truly does fall within that category. These managers said it is especially difficult for staff to be placed in the excellent category. In fact, at least 90 percent of all claims processors in the regional offices we visited ended up in only two of the five performance categories in fiscal year 2007: fully successful and outstanding (see fig. 2).

Some managers told us that there are staff whose performance is better than fully successful but not quite outstanding, but that under VA’s formula, it is difficult for these staff to be placed in the excellent category. To be placed in the excellent category, a staff member must be rated exceptional in all the critical elements and fully successful in at least one noncritical element. However, managers told us that virtually all staff who are exceptional in the critical elements are also exceptional in the noncritical elements, and they are appropriately placed in the outstanding category. On the other hand, if a staff member is rated fully successful on just one critical element, even if all other elements are rated as exceptional, the staff member’s overall performance category falls from outstanding to fully successful.

Neither VBA nor VA central office officials have examined the distribution of claims processing staff across the five overall performance categories. However, VA has acknowledged that there may be an issue with its formula, and the agency is considering changes to its performance management system designed to allow for greater differentiation in performance. Absent additional examination of the distribution of claims processors among overall performance categories, VA lacks a clear picture of whether its system is working as intended and whether any adjustments are needed.

In conclusion, VA appears to have recognized the importance of developing and maintaining high performing claims processors. It needs to devote more attention, however, to ensuring that its training and performance management systems are better aligned to equip both new and experienced staff to handle a burgeoning workload. Specifically, in our May 2008 report, we recommended that VA should collect feedback from staff on training provided in the regional offices in order to assess issues such as the appropriateness of the 80-hour annual training requirement and the usefulness of TPSS. We also recommended that the agency should use information from its new learning management system to hold staff members accountable for meeting the training requirement. In addition, we recommended that VA should assess whether its performance management system is making meaningful distinctions in performance. In its comments on our May 2008 report, VA concurred with our recommendations, but it has not yet reported making any significant progress in implementing them.

While hiring, training, and evaluating the performance of staff is essential, commensurate attention should be focused on reviewing and aligning disability benefits and service outcomes to today’s world. In prior work, we have noted that VA and other federal disability programs must adopt a more modern understanding of how technology and labor market changes determine an individual’s eligibility for benefits, as well as the timing and portfolio of support services they are provided. To the extent progress is made in this area, effective training and performance management systems will be of crucial importance. Moreover, the way VA’s larger workforce is distributed and aligned nationwide can also significantly impact the degree to which it succeeds in meeting the agency’s responsibilities to veterans in the future. In short, VA should seize this opportunity to think more strategically about where to best deploy its new staff and how to develop and maintain their skills.

Mr. Chairman, this concludes my remarks. I would be happy to answer any questions that you or other members of the subcommittee may have.

GAO Contact and Staff Acknowledgments

For further information, please contact Daniel Bertoni at (202) 512-7215 or bertonid@gao.gov. Also contributing to this statement were Clarita Mrena, Lorin Obler, David Forgosh, and Susan Bernstein.

Related GAO Products

Veterans’ Benefits: Improved Management Would Enhance VA’s Pension Program. GAO-08-112. Washington, D.C.: February 14, 2008.

Veterans’ Disability Benefits: Claims Processing Challenges Persist, while VA Continues to Take Steps to Address Them. GAO-08-473T. Washington, D.C.: February 14, 2008.

Disabled Veterans’ Employment: Additional Planning, Monitoring, and Data Collection Efforts Would Improve Assistance. GAO-07-1020. Washington, D.C.: September 12, 2007.

Veterans’ Benefits: Improvements Needed in the Reporting and Use of Data on the Accuracy of Disability Claims Decisions. GAO-03-1045. Washington, D.C.: September 30, 2003.

Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government. GAO-03-893G. Washington, D.C.: July 2003.

Results-Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success. GAO-03-488. Washington D.C.: March 14, 2003.

Major Management Challenges and Program Risks: Department of Veterans Affairs. GAO-03-110. Washington, D.C.: January 1, 2003.

Veterans’ Benefits: Claims Processing Timeliness Performance Measures Could Be Improved. GAO-03-282. Washington, D.C.: December 19, 2002.

Veterans’ Benefits: Quality Assurance for Disability Claims and Appeals Processing Can Be Further Improved. GAO-02-806. Washington, D.C.: August 16, 2002.

Veterans’ Benefits: Training for Claims Processors Needs Evaluation. GAO-01-601. Washington, D.C.: May 31, 2001.

Veterans Benefits Claims: Further Improvements Needed in Claims- Processing Accuracy. GAO/HEHS-99-35. Washington, D.C.: March 1, 1999.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.