Veterans' Benefits: Improvements Needed in VA's Training and Performance Management Systems
Published: 2008-09-18T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Veterans Affairs' (VA) disability claims process has long been a subject of concern because of long waits for decisions and large backlogs of claims pending decisions. To address these issues, VA has hired almost 3,000 new claims processors since January 2007. However, adequate training and performance management are essential to developing highly competent disability claims processors and ensuring that experienced staff maintain the skills needed to issue timely, accurate, and consistent decisions. The Subcommittee on Disability Assistance and Memorial Affairs, House Veterans' Affairs Committee asked GAO to present its views on 1) VA's training for its claims processors and 2) VA's performance management of this staff. This statement is based on a May 2008 report on VA's training and performance management (GAO-08-561) and has been updated as appropriate.
section_title: What GAO Found
paragraphs: Training for VA disability claims processors complies with some accepted training practices, but VA does not adequately evaluate its training and may have opportunities to improve training design and implementation. VA has a highly structured, three-phase training program for new staff and an 80-hour annual training requirement for all staff. GAO found that VA has taken steps to plan this training strategically and that its training program for new staff appears well-designed and conforms to adult learning principles. However, while VA collects some feedback on training for new staff, it does not collect feedback on all the training conducted at its regional offices. Moreover, both new and experienced staff reported problems with their training. Some new staff told us a computer-based learning tool is too theoretical and often out of date. More experienced staff said they struggled to meet the annual 80-hour training requirement because of workload pressures or could not always find courses relevant given their experience level. Finally, the agency does not hold claims processors accountable for meeting the annual training requirement. VA's performance management system for claims processing staff generally conforms to accepted practices. For example, individual performance measures, such as quality and productivity, are aligned with the agency's organizational performance measures, and VA provides staff with regular performance feedback. However, the system may not clearly differentiate among staff performance levels. In each of the regional offices we visited, at least 90 percent of claims processors were placed in just two of five overall performance categories. Broad, overlapping performance categories may deprive managers of the information they need to reward top performers and address performance issues, as well as deprive staff of the feedback they need to improve

section_title: Letter
paragraphs: I am pleased to have the opportunity to comment on training and  performance management for Department of Veterans Affairs’ (VA)  disability claims processors. In fiscal year 2007, VA paid about $37.5 billion  in benefits to more than 3.6 million veterans and their families. The  disability claims process has long been a subject of concern for VA, the  Congress, and veterans’ service organizations due to long waits for  decisions, large backlogs of pending claims, and problems with the  accuracy and consistency of decisions. Moreover, we have noted that VA’s  current disability process is in urgent need of re-examination and  transformation, especially in regard to how it assesses the work capacity  of individuals with disabilities in today’s world and in its ability to provide  timely and appropriate benefits. With an increase in claims resulting from  injuries sustained in Iraq and Afghanistan and from an aging veteran  population, these issues will likely persist. To address them, VA added  almost 3,000 new claims processors from January 2007 to July 2008 and  has plans to add even more staff by the end of September 2008. Earlier this  year, I testified before this subcommittee that enlarging VA’s disability  workforce is likely to produce certain human capital challenges for the  agency. More staff alone will not guarantee effective disability claims  processing. Among other things, adequate training and performance  management are essential to developing highly competent new disability  claims processors and ensuring that experienced staff maintain the skills  needed to issue timely, accurate, and consistent disability decisions.
My remarks today primarily draw from our May 2008 report for the  Committee on Veterans’ Affairs and focus on 1) VA’s training for its  disability claims processing staff and 2) its performance management  system for claims processors. We conducted our work in accordance with  generally accepted government auditing standards. For this testimony, we  updated information from our report, as appropriate, to reflect the current  status of VA training and performance management systems.
In summary, although we found that training for VA disability claims  processors complies with some accepted training practices, it is not  adequately evaluated, and some aspects of training design and  implementation could be improved. We found that VA has taken steps to  strategically plan its training, including the establishment of a training  board to evaluate the agency’s training needs. Also, VA’s training program  for new staff appears well-designed and conforms with adult learning  principles. However, while VA collects feedback on many of the training  methods and tools for new staff, not all the training VA conducts is  evaluated to determine how relevant or effective it is. Moreover, both new  and experienced staff reported problems with their training. Some new  staff members reported that a computer-based learning tool was not  useful. Also, VA requires 80 hours of training annually for all claims  processors, but some experienced claims processors struggled to meet  this requirement because of workload pressures, and some could not  always find relevant courses. It is not clear what criteria VA uses to justify  the number of required training hours. Furthermore, individual claims  processors are not held accountable for meeting the annual training  requirement, although according to VA, the agency has implemented a new  learning management system allowing it to monitor staff’s completion of  the training requirement.
VA’s performance management system for claims processing staff  generally conforms to accepted performance management practices. For  example, individual performance measures, such as quality and  productivity, are aligned with the agency’s organizational performance  measures, and VA provides claims processing staff with regular feedback  on their performance. However, the system may not clearly differentiate  among performance levels. Broad, overlapping performance categories  may deprive managers of the information they need to reward top  performers and address performance issues, as well as deprive staff of the  feedback they need to improve.
subsections: section_title: Background
paragraphs: The Veterans Benefits Administration (VBA) within VA administers the  disability compensation and pension programs, whereby VA claims  processing staff assess veterans’ applications for disability compensation  and pension benefits. Aside from benefits for veterans, VBA claims  processing staff make eligibility determinations for deceased veterans’  spouses, children, and parents. In short, they are responsible for ensuring  that the decisions that lead to paying disability compensation and pension  benefits are timely, accurate, and consistent.
The VA disability claims process involves multiple steps and usually  involves more than one claims processor. When a veteran submits a claim  to one of VBA’s 57 regional offices, staff in that office are responsible for  obtaining evidence to evaluate the claim, such as medical and military  service records; determining whether the claimant is eligible for benefits;  and assigning a disability rating specifying the severity of each of the  veteran’s impairments. These ratings determine the amount of benefits  eligible veterans will receive.
VA has faced questions about the timeliness, accuracy, and consistency of  its disability decisions. GAO designated federal disability programs,  including VA and other programs, as a high-risk area in 2003. In particular,  our prior work found VA relied on outmoded criteria for determining  program eligibility that did not fully reflect advances in medicine and  technology or changes in the labor market. As a result, VA’s disability  program may not recognize an individual’s full potential to work. In  addition, VA has seen processing times for their disability claims increase  over the past several years, and inconsistencies in disability decisions  across locations have raised questions about fairness and integrity.
Some have suggested that VA needs to address its training and guidance  related to claims processing in order to improve consistency and that it  should conduct periodic evaluations of decisions to ensure the accuracy of  ratings across disability categories and regions. VA has reported that some  of the inconsistency in its decisions is due to complex claims, such as  those involving post-traumatic stress disorder, but it has also  acknowledged that the accuracy and consistency of claims decisions  needs further improvement.
section_title: Training Complies with Some Accepted Practices, but VBA Does Not Adequately Evaluate Training and May Be Falling Short in Training Design and Implementation
paragraphs: To prepare newly hired staff to perform the tasks associated with  processing disability claims, VBA has developed a highly structured, three- phase program designed to deliver standardized training. The first phase is  designed to lay the foundation for future training by introducing new staff  to topics such as medical terminology and the computer applications used  to process and track claims. The second provides an overview of the  technical aspects of claims processing, including records management,  how to review medical records, and how to interpret a medical exam. The  third includes a combination of classroom, on-the-job, and computer- based trainings. The second and third phases in this program are designed  to both introduce new material and reinforce material from the previous  phase.
To help ensure that claims processing staff continually maintain their  knowledge after their initial training and keep up with changing policies  and procedures, VBA’s Compensation and Pension Service requires all  claims processing staff to complete a minimum of 80 hours of technical  training annually. This training requirement can be met through a mix of  classroom instruction, electronic-based training from sources such as the  Training and Performance Support System (TPSS), or guest lecturers.  VBA’s regional offices have some flexibility over what courses they  provide to their staff to help them meet the training requirement. These  courses can cover such topics as establishing veteran status, asbestos  claims development, and eye-vision issues.
We found that VBA has taken some steps to strategically plan its training  for claims processors in accordance with generally accepted training  practices identified in our prior work. For example, VBA has taken steps  to align training with the agency’s mission and goals. In 2004, VBA  established an Employee Training and Learning Board (board) to, among  other things, ensure that the agency’s training decisions support its  strategic and business plans, goals, and objectives. Also, VBA has  identified the skills and competencies needed by its claims processing  staff by developing a decision tree and task analysis of the claims process.  In addition, VBA has taken steps to determine the appropriate level of  investment in training and to prioritize funding. The board’s  responsibilities include developing an annual training budget and  recommending training initiatives to the Under Secretary of Benefits.  Further, we found that VBA’s training program for new claims processing  staff appears well-designed, in that it conforms to adult learning principles  by carefully defining all pertinent terms and concepts and providing  abundant and realistic examples of claims work.
However, while VBA has developed a system to collect feedback from new  claims processing staff on their training, the agency does not consistently  collect feedback on all of the training it provides. For example, none of the  regional offices we visited consistently collected feedback on the training  they conduct. Without feedback on regional office training, VBA may not  be aware of how effective all of its training tools are.
Moreover, both new and experienced claims processing staff we  interviewed reported some issues with their training. A number of staff  told us the TPSS was difficult to use, often out-of-date, and too theoretical.  Some claims processing staff with more experience reported that they  struggled to meet the annual training requirement because of workload  pressures or that training topics were not always relevant for staff with  their level of experience. VBA officials reported that they have reviewed  the 80-hour training requirement to determine if it is appropriate, but they  could not identify the criteria or any analysis that were used to make this  determination. Identifying the right amount of training is crucial. An overly  burdensome training requirement may needlessly take staff away from  essential claims processing duties, while too little training could  contribute to processing and quality errors.
In addition to lacking a clear process for assessing the appropriateness of  the 80-hour training requirement, VBA also has no policy outlining  consequences for individual staff who do not complete the requirement.  Because it does not hold staff accountable, VBA is missing an opportunity  to clearly convey to staff the importance of managing their time to meet  training requirements, as well as production and accuracy goals. In fiscal  year 2008, VBA implemented a new learning management system that  allows it to track the training hours completed by individual staff.  Although VBA now has the capacity to monitor staff’s completion of the  training requirement, the agency has not indicated any specific  consequences for staff who fail to meet the requirement.
section_title: VA’s Performance Management System Generally Conforms with Accepted Practices, but May Not Clearly Differentiate among Staff’s Performance Levels
paragraphs: VA’s performance management system for claims processors is consistent  with a number of accepted practices for effective performance  management systems in the public sector. For example, the elements used  to evaluate individual claims processors—such as quality, productivity,  and workload management—appear to be generally aligned with VBA’s  organizational performance measures. Aligning individual and  organizational performance measures helps staff see the connection  between their daily work activities and their organization’s goals and the  importance of their roles and responsibilities in helping to achieve these  goals. VA also requires supervisors to provide claims processors with  regular feedback on their performance, and it has actively involved its  employees and other stakeholders in developing its performance  management system.
However, VA’s system may not be consistent with a key accepted  practice—clear differentiation among staff performance levels. We have  previously reported that, in order to provide meaningful distinctions in  performance for experienced staff, agencies should preferably use rating  systems with four or five performance categories. If staff members’  ratings are concentrated in just one or two of multiple categories, the  system may not be making meaningful distinctions in performance.  Systems that do not make meaningful distinctions in performance fail to  give (1) employees the constructive feedback they need to improve and (2)  managers the information they need to reward top performers and address  performance issues.
VA’s performance appraisal system has the potential to clearly  differentiate among staff performance levels. Each fiscal year, regional  offices give their staff a rating on each individual performance element:  exceptional, fully successful, or less than fully successful. For example, a  staff member might be rated exceptional on quality, fully successful on  productivity, and so forth. Some elements are considered critical elements,  and some are considered noncritical. Staff members are then assigned to  one of five overall performance categories, ranging from unsatisfactory to  outstanding, based on a formula that converts a staff member’s  combination of ratings on the individual performance elements into an  overall performance category (see fig. 1).
However, there is evidence to suggest that the performance management  system for claims processing staff may not clearly or accurately  differentiate among staff’s performance. Central office officials and  managers in two of the four regional offices we visited said that, under the  formula for assigning overall performance categories, it is more difficult to  place staff in certain overall performance categories than in others—even  if staff’s performance truly does fall within that category. These managers  said it is especially difficult for staff to be placed in the excellent category.  In fact, at least 90 percent of all claims processors in the regional offices  we visited ended up in only two of the five performance categories in  fiscal year 2007: fully successful and outstanding (see fig. 2).
Some managers told us that there are staff whose performance is better  than fully successful but not quite outstanding, but that under VA’s  formula, it is difficult for these staff to be placed in the excellent category.  To be placed in the excellent category, a staff member must be rated  exceptional in all the critical elements and fully successful in at least one  noncritical element. However, managers told us that virtually all staff who  are exceptional in the critical elements are also exceptional in the  noncritical elements, and they are appropriately placed in the outstanding  category. On the other hand, if a staff member is rated fully successful on  just one critical element, even if all other elements are rated as  exceptional, the staff member’s overall performance category falls from  outstanding to fully successful.
Neither VBA nor VA central office officials have examined the distribution  of claims processing staff across the five overall performance categories.  However, VA has acknowledged that there may be an issue with its  formula, and the agency is considering changes to its performance  management system designed to allow for greater differentiation in  performance. Absent additional examination of the distribution of claims  processors among overall performance categories, VA lacks a clear picture  of whether its system is working as intended and whether any adjustments  are needed.
In conclusion, VA appears to have recognized the importance of  developing and maintaining high performing claims processors. It needs to  devote more attention, however, to ensuring that its training and  performance management systems are better aligned to equip both new  and experienced staff to handle a burgeoning workload. Specifically, in  our May 2008 report, we recommended that VA should collect feedback  from staff on training provided in the regional offices in order to assess  issues such as the appropriateness of the 80-hour annual training  requirement and the usefulness of TPSS.  We also recommended that the  agency should use information from its new learning management system  to hold staff members accountable for meeting the training requirement. In  addition, we recommended that VA should assess whether its performance  management system is making meaningful distinctions in performance. In  its comments on our May 2008 report, VA concurred with our  recommendations, but it has not yet reported making any significant  progress in implementing them.
While hiring, training, and evaluating the performance of staff is essential,  commensurate attention should be focused on reviewing and aligning  disability benefits and service outcomes to today’s world.  In prior work,  we have noted that VA and other federal disability programs must adopt a  more modern understanding of how technology and labor market changes  determine an individual’s eligibility for benefits, as well as the timing and  portfolio of support services they are provided. To the extent progress is  made in this area, effective training and performance management systems  will be of crucial importance. Moreover, the way VA’s larger workforce is  distributed and aligned nationwide can also significantly impact the degree  to which it succeeds in meeting the agency’s responsibilities to veterans in  the future. In short, VA should seize this opportunity to think more  strategically about where to best deploy its new staff and how to develop  and maintain their skills.
Mr. Chairman, this concludes my remarks. I would be happy to answer any  questions that you or other members of the subcommittee may have.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: For further information, please contact Daniel Bertoni at (202) 512-7215 or  bertonid@gao.gov. Also contributing to this statement were Clarita Mrena,  Lorin Obler, David Forgosh, and Susan Bernstein.
section_title: Related GAO Products
paragraphs: Veterans’ Benefits: Improved Management Would Enhance VA’s Pension  Program. GAO-08-112. Washington, D.C.: February 14, 2008.
Veterans’ Disability Benefits: Claims Processing Challenges Persist,  while VA Continues to Take Steps to Address Them. GAO-08-473T.  Washington, D.C.: February 14, 2008.
Disabled Veterans’ Employment: Additional Planning, Monitoring, and  Data Collection Efforts Would Improve Assistance. GAO-07-1020.  Washington, D.C.: September 12, 2007.
Veterans’ Benefits: Improvements Needed in the Reporting and Use of  Data on the Accuracy of Disability Claims Decisions. GAO-03-1045.  Washington, D.C.: September 30, 2003.
Human Capital: A Guide for Assessing Strategic Training and  Development Efforts in the Federal Government. GAO-03-893G.  Washington, D.C.: July 2003.
Results-Oriented Cultures: Creating a Clear Linkage between Individual  Performance and Organizational Success. GAO-03-488. Washington D.C.:  March 14, 2003.
Major Management Challenges and Program Risks: Department of  Veterans Affairs. GAO-03-110. Washington, D.C.: January 1, 2003.
Veterans’ Benefits: Claims Processing Timeliness Performance Measures  Could Be Improved. GAO-03-282. Washington, D.C.: December 19, 2002.
Veterans’ Benefits: Quality Assurance for Disability Claims and Appeals  Processing Can Be Further Improved. GAO-02-806. Washington, D.C.:  August 16, 2002.
Veterans’ Benefits: Training for Claims Processors Needs Evaluation.  GAO-01-601. Washington, D.C.: May 31, 2001.
Veterans Benefits Claims: Further Improvements Needed in Claims- Processing Accuracy. GAO/HEHS-99-35. Washington, D.C.: March 1, 1999.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.