Veterans' Benefits: Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors
Published: 2008-05-27T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Faced with an increase in disability claims, the Veterans Benefits Administration (VBA) is hiring a large number of new claims processing staff. We were asked to determine: (1) What training is provided to new and experienced claims processors and how uniform is this training? (2) To what extent has VBA planned this training strategically, and how well is the training designed, implemented, and evaluated? and (3) To what extent is the performance management system for claims processors consistent with generally accepted practices? To answer the questions, GAO reviewed documents including VBA policies and training curricula; interviewed VBA central office officials; visited 4 of VBA's 57 regional offices, which were selected to achieve diversity in geographic location, number of staff, and officewide accuracy in claims processing; and compared VBA's training and performance management to generally accepted practices identified by GAO.
section_title: What GAO Found
paragraphs: VBA has a standardized training curriculum for new staff and a training requirement for all staff, but does not hold staff accountable for meeting this requirement. The curriculum for new staff includes what is referred to as centralized training and training at their home offices. All claims processors must complete 80 hours of training annually, which may cover a mix of topics identified centrally and by regional offices. Individual staff members face no consequences for failing to meet the training requirement, however, and VBA has not tracked training completion by individuals. It is implementing a new system that should provide this capacity. Although VBA has taken steps to plan its training strategically, the agency does not adequately evaluate training and may be falling short in training design and implementation. VBA has a training board that assesses its overall training needs. However, the agency does not consistently collect feedback on regional office training, and both new and experienced staff GAO interviewed raised issues with their training. Some new staff raised concerns about the consistency of training provided by different instructors and about the usefulness of an on-line learning tool. Some experienced staff believe that 80 hours of training annually is not necessary, some training was not relevant for them, and workload pressures impede training. The performance management system for claims processors generally conforms to GAO-identified key practices, but the formula for assigning overall ratings may prevent managers from fully acknowledging and rewarding staff for higher levels of performance. The system aligns individual and organizational performance measures and requires that staff be given feedback throughout the year. However, VBA officials raised concerns about the formula used to assign overall ratings. Almost all staff in the offices GAO visited were placed in only two of five overall rating categories, although managers said greater differentiation would more accurately reflect actual performance differences. The Department of Veterans Affairs (VA) has not examined the ratings distribution, but acknowledges a potential issue with its formula and is considering changes.

section_title: Letter
paragraphs: The Veterans Benefits Administration (VBA) is facing an increased volume  of claims for disability benefits related to the current conflicts in  Afghanistan and Iraq as well as the aging of veterans from past conflicts.  Between fiscal years 2000 and 2006, the number of disability-related claims  filed annually with VBA increased by almost 40 percent. As a result, VBA  continues to experience challenges in processing veterans’ disability  claims. As of fiscal year 2007, VBA had approximately 392,000 disability  claims pending benefit decisions, and the average time these claims were  pending was 132 days. According to VBA, the current conflicts have also  produced more claims related to post-traumatic stress disorder and  traumatic brain injury, conditions few VBA staff have had much  experience evaluating. To process the increased volume of claims, in fiscal  year 2007 the agency began hiring a large number of new Veterans Service  Representatives (VSR), who collect evidence related to veterans’ claims,  and Rating Veterans Service Representatives (RVSR), who evaluate claims  and determine benefit eligibility. It plans to add 3,100 new claims- processing staff by the end of fiscal year 2008.
Given the increased volume of claims, the increased focus on certain types  of disabilities, and the large number of new hires, training and  performance management systems for VSRs and RVSRs now play an  especially critical role in enabling VBA to meet its organizational claims  processing goals for accuracy and productivity. Training that is properly  designed and implemented is vital both to help new staff learn their jobs  and experienced staff to update their knowledge and learn about emerging  issues. An effective performance management system would also help  VBA manage its staff on a day-to-day basis to achieve its organizational  goals. To provide Congress with information on the training and  performance management of claims processors, we were asked to  determine: (1) What training is provided to new and experienced claims  processors and how uniform is this training? (2) To what extent has VBA  developed a strategic approach to planning training for claims processors  and how well is their training designed, implemented, and evaluated? And  (3) To what extent is the performance management system for claims  processors consistent with generally accepted performance management  practices in the public sector?
To address these objectives, we collected documents and data from VBA  central office and interviewed central office staff. In addition, GAO experts  on training reviewed VBA documents related to training curriculum,  lesson plans, and course evaluations. We conducted site visits to 4 of  VBA’s 57 regional offices—Atlanta, Baltimore, Milwaukee, and Portland,  Oregon. These offices were selected to achieve diversity in geographical  location, number of staff, and officewide accuracy in claims processing.  While we examined VBA-wide policies and requirements, we primarily  assessed how the training and performance management systems are  implemented at four sites. Therefore, our results may not be  representative of how these systems are implemented across all regional  offices. We assessed VBA’s training and performance management  practices by comparing them to certain generally accepted practices for  federal agencies in these areas that have been identified by GAO. We  conducted this performance audit from September 2007 through May 2008  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. (See app. I for more detailed  information on our objectives, scope, and methodology.)
subsections: section_title: Background
paragraphs: In fiscal year 2007, the Department of Veterans Affairs (VA) paid about  $37.5 billion in disability compensation and pension benefits to more than  3.6 million veterans and their families. Through its disability compensation  program, the VBA pays monthly benefits to veterans with service- connected disabilities (injuries or diseases incurred or aggravated while  on active military duty). Monthly benefit amounts vary according to the  severity of the disability. Through its pension benefit program, VBA pays  monthly benefits to wartime veterans with low incomes who are either  elderly or permanently and totally disabled for reasons not service- connected. In addition, VBA pays dependency and indemnity  compensation to some deceased veterans’ spouses, children, and parents  and to survivors of servicemembers who died while on active duty.
When a veteran submits a benefits claim to any of VBA’s 57 regional  offices, a Veterans Service Representative (VSR) is responsible for  obtaining the relevant evidence to evaluate the claim. For disability  compensation benefits, such evidence includes veterans’ military service  records, medical examinations, and treatment records from VA medical  facilities and private providers. Once a claim is developed (i.e., has all the  necessary evidence), a Rating Veterans Service Representative (RVSR)  evaluates the claim, determines whether the claimant is eligible for  benefits, and assigns a disability rating based on degree of impairment.  The rating determines the amount of benefits the veteran will receive. For  the pension program, claims processing staff review the veteran’s military,  financial, and other records to determine eligibility. Eligible veterans  receive monthly pension benefit payments based on the difference  between their countable income, as determined by VA, and the maximum  pension amounts as updated annually by statute. In fiscal year 2007, VBA  employed over 4,100 VSRs and about 1,800 RVSRs to administer the  disability compensation and pension programs’ caseload of almost 3.8  million claims.
In 2001 the VA Claims Processing Task Force noted that VSRs were  responsible for understanding almost 11,000 separate benefit delivery  tasks, such as tasks in claims establishment, claims development, public  contacts, and appeals. To improve VBA’s workload controls, accuracy  rates, and timeliness, the Task Force recommended that VA divide these  tasks among a number of claims processing teams with defined functions.  To that end, in fiscal year 2002, VBA developed the Claims Processing  Improvement model that created six claims processing teams, based on  phases of the claims process. (See table 1.)
According to one VA official, new claims processing staff generally begin  as VSRs and typically have a probationary period of about one year. After  their probationary period ends, staff can either continue to qualify to  become senior VSRs or apply for RVSR positions. VSRs are also given the  option to rotate to other VSR claim teams to gain a broader understanding  of the claims process.
section_title: VBA Has a Uniform Training Curriculum for New Claims Processors and an Annual Training Requirement for All Claims Processors, but Staff Are Not Held Accountable for Meeting This Requirement
paragraphs: VBA has established a standardized curriculum for training new VSRs and  RVSRs on how to process claims, and it has an 80-hour annual training  requirement for both new and experienced staff; however, it does not hold  individual staff accountable for meeting this requirement. VBA has  designed a uniform curriculum for training new VSRs and RVSRs that is  implemented in three phases—initial orientation training, a 3-week  training session referred to as centralized training, and comprehensive on- the-job and classroom training after centralizing training. It also requires  all staff to meet an annual 80-hour training requirement. To ensure that  staff meet this requirement, each regional office must develop an annual  training plan, which can contain a mix of training topics identified by VBA  central office and by the regional office. However, individual staff  members are not held accountable for meeting their training requirement.
subsections: section_title: Training for New Staff Is Conducted in Three Stages Using a Uniform Curriculum
paragraphs: VBA has a highly structured, three-phased program for all new claims  processors designed to deliver standardized training, regardless of training  location or individual instructors. (See fig. 1.) For example, each topic  included in this training program contains a detailed lesson plan with  review exercises, student handouts, and copies of slides used during the  instructor’s presentation. Each phase in this program is designed to both  introduce new material and reinforce material from the previous phase,  according to a VBA official.
According to VBA policy, the first phase of training for new VSRs and  RVSRs is prerequisite training. New VSRs and RVSRs begin prerequisite  training at their home regional office as soon as they begin working.  Prerequisite training lays the foundation for future training by introducing  new VSRs to topics such as the software applications used to process and  track claims, medical terminology, the system for maintaining and filing a  case folder, and the process for requesting medical records. Although VBA  specifies the topics that must be covered during prerequisite training,  regional offices can choose the format for the training and the time frame.  New VSRs and RVSRs typically spend 2 to 3 weeks completing prerequisite  training in their home office before they begin the second program phase,  centralized training.
During what is referred to as centralized training, new VSRs and RVSRs  spend 3 weeks in intensive classroom training. Participants from multiple  regional offices are typically brought together in centralized training  sessions, which may occur at their home regional office, another regional  office, or the Veterans Benefits Academy in Baltimore, Maryland.  According to VBA officials in three of the four offices we visited, bringing  together VSRs and RVSRS from different regional offices helps to promote  networking opportunities, while VBA officials from two of these offices  also stated that it provides a nationwide perspective on VBA. Centralized  training provides an overview of the technical aspects of the VSR and  RVSR positions. Training instructors should follow the prescribed  schedule and curriculum dictating when and how material is taught. For  example, for a particular topic, the instructor’s guide explains the length of  the lesson, the instructional method, and the materials required; lays out  the information that must be covered; and provides exercises to review the  material. (See fig. 2 for a sample of an instructor’s guide from the  centralized training curriculum.) Centralized training classes have at least  three instructors, but the actual number can vary depending on the size of  the group. VBA’s goal is to maintain a minimum ratio of instructors to  students.
The first week of centralized training for VSRs focuses on key concepts,  such as security, privacy and records management; terminology; and job  tools, such as the policy manual and software applications. The final 2  weeks of training focus on the different roles and responsibilities of VSRs  on the Pre-determination and Post-determination teams in processing  claims. To practice processing different types of claims and processing  claims from start to finish, VSRs work on either real claims or hypothetical  claims specifically designed for training. Centralized training for new  RVSRs—many of whom have been promoted from the VSR position— focuses on topics such as systems of the human body, how to review  medical records, and how to interpret a medical exam. According to staff  in one site we visited, RVSRs new to VBA also take VSR centralized  training or its equivalent to learn the overall procedures for processing  claims.
To accommodate the influx of new staff it must train, in fiscal year 2007  VBA substantially increased the frequency of centralized training and is  increasing student capacity at the Veterans Benefits Academy. During  fiscal year 2007, VBA held 67 centralized training sessions for 1,458 new  VSRs and RVSRs. Centralized training sessions were conducted at 26  different regional offices during fiscal year 2007, in addition to the  Veterans Benefits Academy. By comparison, during fiscal year 2006, VBA  held 27 centralized training sessions for 678 new claims processors.
To implement centralized training, VBA relies on qualified regional office  staff who have received training on how to be an instructor. According to  VBA officials, centralized training instructors may be Senior VSRs, RVSRs,  supervisors, or other staff identified by regional office managers as having  the capability and the right personality to be effective instructors.  Potential instructors have certain training requirements. First, they must  complete the week-long Instructor Development Course, which covers the  ways different adults learn, the process for developing lesson plans, and  the use of different training methods and media. During this course,  participants are videotaped and given feedback on their presentation style.  In addition, each time instructors teach a centralized training session, they  are supposed to take the 2.5 day Challenge Curriculum Course, designed  to update instructors on changes to the curriculum and general training  issues. Between October 2006 and February 2008, about 250 VSRs and  RVSRs from regional offices completed the Instructor Development  Course, and VBA officials reported that, given the influx of new VSRs and  RVSRs, they are increasing the number of times this course is offered in  order to train more instructors. Instructors can teach centralized training  sessions in their home office, another regional office, or the Veterans  Benefits Academy.
When new VSRs and RVSRs return to their home office after centralized  training, they are required to begin their third phase of training, which is  supposed to include on-the-job, classroom, and computer-based training,  all conducted by and at their regional office. In the regional offices we  visited, managers indicated that new VSRs and RVSRs typically take about  6 to 12 months after they return from centralized training to complete all  the training requirements for new staff. During this final phase, new claims  processing staff cover more advanced topics, building on what they  learned in centralized training. Under the supervision of experienced  claims processors, they work on increasingly complex types of real claims.  On-the-job training is supplemented in the offices we visited by regular  classroom training that follows a required curriculum of courses  developed by VBA’s Compensation and Pension Service, specifically for  new VSRs and RVSRs. For example, new VSRs might complete a class in  processing burial claims and then spend time actually processing such  claims. The amount of time spent working on each type of claim varies  from a couple of days to a few weeks, depending on the complexity of the  claim. On-the-job training is also supposed to be supplemented with  modules from the Training and Performance Support System (TPSS), an  interactive on-line system that can be used by staff individually or in a  group. TPSS modules provide detailed lessons, practice cases, and tests  for VSRs and RVSRs. Modules for new VSRs cover topics such as burial  benefits and medical terminology; RVSR modules cover topics such as the  musculoskeletal system, general medical terminology, and introduction to  post-traumatic stress disorder.
section_title: New and Experienced Staff Have an Annual Training Requirement, and Regional Offices Develop Training Plans That Cover a Mix of Topics Identified Centrally and Locally
paragraphs: A policy established by VBA’s Compensation and Pension Service requires  both new and experienced VSRs and RVSRs to complete a minimum of 80  hours of technical training annually, double the number VBA requires of  its employees in other technical positions. VBA officials said this higher  training requirement for VSRs and RVSRs is justified because their jobs are  particularly complex and they must work with constantly changing  policies and procedures.
The 80-hour training requirement has two parts. At least 60 hours must  come from a list of core technical training topics identified by the central  office of the Compensation and Pension Service. For example, core topics  for VSRs in fiscal year 2007 included establishing veteran status and  asbestos claims development; topics for RVSRs included due process  provisions and eye-vision issues. VBA specifies more core topics than are  necessary to meet the 60-hour requirement, so regional offices can choose  those topics most relevant to their needs. They can also choose the  training method used to address each topic, such as classroom or TPSS  training. (See app. II for the list of core technical training topics for fiscal  year 2007.) Regional offices determine the training topics that are used to  meet the remaining 20 hours, based on local needs and input. Regional  offices may select topics from the list of core technical training topics or  identify other topics on their own.
The four regional offices we visited varied in the extent to which they  utilized their discretion to choose topics outside the core technical  training topics in fiscal year 2007. Two sites selected the required 60 hours  of training from the core requirements and identified their own topics for  the remaining 20 hours. In the other two sites, almost all the training  provided to staff in fiscal year 2007 was based on topics from the list of  core requirements. An official in one regional office, for example, said that  his office used its full 20 hours to provide training on new and emerging  issues that are not covered by the core technical training topics, as well as  training to address error prone areas. An official in another regional office  said the core requirements satisfied staff training needs in fiscal year 2007,  possibly because this regional office had a large proportion of new staff  and the core topics are focused on the needs of new staff.
Regional offices must develop training plans each year that indicate which  courses will actually be provided to staff to enable them to meet the 80- hour training requirement. The training plan is a list of courses that the  regional office plans to offer throughout the year, as well as the expected  length and number and types of participants in each course. In the regional  offices we visited, when managers develop their training plans, they solicit  input from supervisors of VSRs and RVSRs and typically also consider  national or local error trend data. Regional offices must submit their plans  to the VBA central office at the beginning of each fiscal year for review  and feedback. Central office officials review the plans to determine  whether (1) the regional office will deliver at least 60 hours of training on  the required core topics, (2) the additional topics identified by the regional  office are appropriate, and (3) staff in similar positions within an office  receive the same level and type of training. According to central office  officials, they provide feedback to the regional offices on their current  plans as well as guidance on what topics to include in the next year’s  training plans. Regional offices can adjust their training plans throughout  the year to address shifting priorities and unexpected training needs. For  example, a regional office may add or remove courses from the plan in  response to changing trends in errors or policy changes resulting from  legal decisions. (See app. III for excerpts from the fiscal year 2007 training  plans from the regional offices we visited.)
While regional offices have discretion over the methods they use to  provide training, the four offices we visited relied primarily on classroom  training in fiscal year 2007. In each of these offices, at least 80 percent of  the total fiscal year 2007 training hours completed by all claims processors  was in the form of classroom instruction (see fig. 3). Officials in two of the  regional offices we visited said they used lesson plans provided by the  Compensation and Pension Service and adapted these plans to the needs  of their staff; one regional office developed its own courses. An official in  one office said they sometimes invite guest speakers, and an official in  another regional office said that classroom training is sometimes delivered  as part of team meetings. The offices we visited generally made little use  of other training methods. Only one office used TPSS for its training more  than 1 percent of the time. Two offices used self-instruction—such as  reading memos from VBA central office—for about 10 percent of their  training, and no office used videos for more than 1 percent of their  training. The central office usually communicates immediate policy and  regulatory changes through memos called Fast Letters, which may be  discussed in team meetings or may just be read by staff individually.
section_title: Staff Are Not Held Accountable for Meeting Their Training Requirement
paragraphs: Because the agency has no policy outlining consequences for individual  staff who do not complete their 80 hours of training per year, individual  staff are not held accountable for meeting their annual training  requirement, and at present, VBA central office lacks the ability to track  training completed by individual staff members. According to VBA  officials, however, the agency is in the process of implementing an  automated system that should allow it to track the training each staff  member completes. Officials reported that this system is expected to be  implemented during fiscal year 2008. VBA officials reported that this  system will be able to record the number of training hours and the courses  completed for each individual, staff position, and regional office.  One  official said the central office and regional office supervisors will have the  ability to monitor training completed by individual staff members, but that  central office will likely not monitor the training completed by each  individual staff member, even though it may monitor the training records  for a sample of staff members. Furthermore, despite the absence of a VBA- wide tracking system, managers in two of the regional offices we visited  reported using locally developed tracking methods to determine the  number of training hours their staff had completed.
While individuals are not held accountable, VBA reported taking some  steps to ensure that staff complete the required number of training hours.  VBA central office periodically reviews the aggregated number of training  hours completed at each regional office to determine whether the office is  on track to meet the training requirement. According to a VBA official,  managers in offices where the staff is not on track to complete 80 hours of  training during the year can be reprimanded by a higher-level manager,  and if their staff do not meet the aggregate training hours at the end of the  fiscal year, managers could face negative consequences in their  performance assessments.
section_title: VBA Is Taking Steps to Strategically Plan Its Training for Staff, but Does Not Adequately Evaluate Training and May Be Falling Short in Design and Implementation
paragraphs: VBA is taking steps to strategically plan its training for VSRs and RVSRs  including the establishment of a training board to assess VBA’s training  needs. VBA has also made some effort to evaluate its training for new  staff, but does not require regional offices to collect feedback from staff  on any of the training they provide. Although some regional offices collect  some training feedback, it is not shared with VBA central office. Both new  and experienced staff we interviewed did, in fact, report some problems  with their training. A number of new staff raised issues with how  consistently their training curriculum was implemented. Experienced staff  differed in their assessments of the VBA’s annual training requirement,  with some indicating they struggle to meet this requirement because of  workload pressures or that training topics are sometimes redundant or not  relevant to their position.
subsections: section_title: VBA Is Taking Steps to Strategically Plan Its Training
paragraphs: VBA is taking steps to strategically plan its training for claims processors,  in accordance with generally accepted practices identified by GAO. (See  app. I for a detailed description of these generally accepted practices.)
subsections: section_title: Aligning Training with the Agency’s Mission and Goals
paragraphs: VBA has made an effort to align training with the agency’s mission and  goals. According to VBA documents, in fiscal year 2004 an Employee  Training and Learning Board (board) was established to ensure that  training decisions within the VBA are coordinated; support the agency’s  strategic and business plans, goals and objectives; and are in accordance  with the policy and vision of VBA. Some of the board’s responsibilities  include establishing training priorities and reviewing regional office and  annual training plans.
section_title: Identifying the Skills and Competencies Needed by the Workforce
paragraphs: VBA has identified the skills and competencies needed by VBA’s claims  processing workforce. VBA developed a decision tree and task analysis of  the claims process, which GAO experts in the field of training told us made  it possible to understand and map both the claims process and the  decisions associated with it that supported the development of VBA’s  training curriculum.
section_title: Determining the Appropriate Level of Investment in Training and Prioritizing Funding
paragraphs: VBA is taking steps to determine the appropriate level of investment in  training and prioritize funding. According to VBA documents, some of the  board’s responsibilities include developing annual training budget  recommendations and identifying and recommending training initiatives to  the Under Secretary of Benefits. VBA officials also reported developing  several documents that made a business case for different aspects of  VBA’s training, such as VA’s annual budget and the task analysis of the  VSR and RVSR job positions.
section_title: Considering Government Reforms and Initiatives
paragraphs: According to one VBA official, the agency identifies regulatory, statutory,  and administrative changes as well as any legal or judicial decisions that  affect how VBA does business and issues guidance letters, or Fast Letters,  which can be sent out several times a year, to notify regional offices of  these changes. Also, as a result of Congress authorizing an increase in its  number of full-time employees and VBA’s succession planning efforts, VBA  has increased the number of centralized training sessions for new staff and  has also increased the number of Instructor Development Courses offered  to potential centralized training instructors. As a result, VBA is taking  steps to consider government reforms and initiatives to improve its  management and performance when planning its training.
section_title: VBA Collects Feedback on Centralized Training, but Regional Offices Do not Always Collect Feedback on the Training They Provide
paragraphs: According to accepted practices, federal agencies should also evaluate  their training programs and demonstrate how these efforts help  employees, rather than just focusing on activities or processes (such as  number of training participants or hours of training). VBA has made some  efforts to evaluate its training for claims processors. During the 3-week  centralized training session for new staff, VBA solicits daily feedback from  participants using forms that experts in the training field consider well- constructed and well-balanced. According to one GAO expert, the forms  generally employ the correct principles to determine the effectiveness of  the training and ascertain whether the instructor effectively presented the  material (see fig. 4). VBA officials told us that they have used this feedback  to improve centralized training for new staff. Management at one regional  office cited the decision to separate training curricula for VSRs on Pre- determination teams and VSRs on Post-determination teams as an  example of a change based on this feedback.
Although VBA evaluates centralized training, it does not require regional  offices to obtain feedback from participants on any of the training they  provide to new and experienced staff. In a previous GAO report, VA staff  told us that new training materials they develop are evaluated before being  implemented. However, none of the regional offices we visited  consistently collect feedback on the training they conduct. Supervisors  from three of the regional offices we visited told us that they collect  feedback on some of the training their office conducts, but this feedback  largely concerns the performance of the instructor. Participants are  generally not asked for feedback on course content. Moreover, regional  offices we visited that do, to some degree, collect feedback do not share  this information with VBA.
section_title: VBA’s Training Curriculum for New Staff Appears Generally Well Designed, but Some Staff Raised Issues Concerning Its Implementation
paragraphs: According to GAO experts in the training field, VBA’s training curriculum  for new staff appears well designed. VBA’s curriculum for new staff  conforms to adult learning principles, carefully defining all pertinent terms  and concepts, and providing abundant and realistic examples of claims  work. GAO experts also determined that VBA’s training for those who  teach the curriculum for new staff was well designed and would enable  experienced claims processors to become competent trainers because  they are coached on teaching theory and have multiple opportunities to  practice their teaching skills and receive feedback.
Many of the new staff at all four sites we visited reported that centralized  training provided them with a good foundation of knowledge and prepared  them for additional training conducted by their regional office. Also,  regional office managers from three offices we visited told us that  centralized training affords new staff the opportunity to network with  other new staff at different regional offices, which imbues a sense of how  their positions fit in the organization. However, some staff reported that  VBA’s implementation of their centralized training was not always  consistent. A number of staff at three regional offices reported that during  their centralized training the instructors sometimes taught different ways  of performing the same procedures or disagreed on claim procedures.  Regional office officials told us that while centralized training instructors  attempt to teach consistently through the use of standardized training  materials, certain procedures can be done differently in different regional  offices while adhering to VBA policy. For example, regional offices may  differ on what to include in veteran notification letters. VBA officials also  told us that centralized training conducted at the regional offices may not  be as consistent as centralized training conducted at the Veterans Benefits  Academy. According to these officials, unlike the regional offices, the  Veterans Benefits Academy has on-site training experts to guide and  ensure that instructors are teaching the curriculum consistently.
New staff also gave mixed assessments about how training was conducted  at their home office after they returned from centralized training. While  some staff at all of the regional offices we visited told us that the  additional training better prepared them to perform their jobs, with on-the- job training identified as a useful learning tool, others told us that the  training could not always be completed in a timely manner due to regional  office priorities. Some management and staff at two of the regional offices  we visited reported that, because of workload pressures, some of their  RVSRs had to interrupt their training to perform VSR duties. Also, a few  new staff indicated that VBA’s TPSS was somewhat difficult to use.  Although TPSS was developed to provide consistent technical training  designed to improve the accuracy of claims ratings, a number of staff at all  of the regional offices we visited reported that TPSS was too theoretical.  For example, some staff said it provided too much information and no  practical exercises in applying the knowledge. Some staff also noted that  certain material in TPSS was out-of-date with policy changes such as how  to order medical examinations. Some staff at three of the regional offices  also reported that TPSS was not always useful in training staff, in part,  because TPSS does not use real cases. Three of the regional offices  reported using TPSS for less than 1 percent of their training and VSRs at  one regional office were unaware of what TPSS was.
section_title: Experienced Staff Expressed Mixed Views of the Design and Implementation of Their Training
paragraphs: At all of the regional offices we visited, staff we spoke with generally  noted that training enables them to keep up-to-date on changes in laws  and regulations as well as provides opportunities for obtaining refresher  training on claims procedures they perform infrequently. However,  regional office staff we spoke with differed in their assessment of the 80- hour requirement. Some regional office staff said the number of training  hours required was appropriate, while others suggested that VBA adopt a  graduated approach, with the most experienced staff being required to  complete fewer hours than new staff. VBA officials told us that, in 2007,  the Compensation and Pension Service reviewed their annual training  requirements and determined the 80-hour annual training requirement was  appropriate. However, the officials we spoke with could not identify the  criteria that were used to make these determinations. Furthermore, VBA  management does not systematically collect feedback from staff  evaluating the usefulness of the training they must receive to meet this  requirement. Consequently, when determining the appropriateness of the  80-hour requirement, VBA has not taken into account the views of staff to  gauge the effect the requirement has on them.
Experienced staff had mixed views on training provided by the regional  office. Staff at three regional offices said the core technical training topics  set by the Compensation and Pension Service are really designed for  newer staff and do not change much from year to year, and therefore  experienced staff end up repeating courses. Also, a number of staff at all  of the regional offices we visited told us some regional office training was  not relevant for those with more experience. Conversely, other regional  office staff note that although training topics may be the same from year to  year, a person can learn something new each time the course is covered.  Some VBA officials and regional office managers also noted that some  repetition of courses is good for several reasons. Staff may not see a  particular issue very often in their day-to-day work and can benefit from  refreshers. Also, regional office managers at one office told us that the  core technical training topics could be modified to reflect changes in  policy so that courses are less repetitive for experienced staff.
Many experienced staff also reported having difficulty meeting the 80-hour  annual training requirement due to workload pressures. Many of the  experienced staff we spoke with, at each of the regional offices we visited,  told us that there is a constant struggle between office production goals  and training goals. For example, office production goals can affect the  availability of the regional office’s instructors. A number of staff from one  regional office noted that instructors were unable to spend time teaching  because of their heavy workloads and because instructors’ training  preparation hours do not count toward the 80-hour training requirement.  Staff at another regional office told us that, due to workload pressures,  staff may rush through training and may not get as much out of it as they  should.
section_title: Performance Management System for Claims Processors Generally Conforms to Accepted Practices, but May Not Clearly Differentiate between Performance Levels
subsections: section_title: Performance Management System for Claims Processors Is Generally Consistent with Accepted Practices
paragraphs: The elements used to evaluate individual VSRs’ and RVSRs’ performance  appear to be generally aligned with VBA’s organizational performance  measures, something prior GAO work has identified as a well-recognized  practice for effective performance management systems (see app. I).  Aligning individual and organizational performance measures helps staff  see the connection between their daily work activities and their  organization’s goals and the importance of their roles and responsibilities  in helping to achieve these goals. VSRs must be evaluated on four critical  elements: quality, productivity, workload management, and customer  service. RVSRs are evaluated on quality, productivity, and customer  service. In addition, VBA central office requires regional offices to evaluate  their staff on at least one non-critical element. The central office has  provided a non-critical element called cooperation and organizational  support, and although regional offices are not required to use this  particular element, all four offices we visited did so (see table 2). For each  element, there are three defined levels of performance: exceptional, fully  successful, or less than fully successful. Table 2 refers only to the fully  successful level of performance for each element.
Three critical elements in particular—quality, workload management, and  productivity—are aligned with VBA’s organizational performance  measures (see table 3). According to VA’s strategic plan, one key  organizational performance measure for VBA is overall accuracy in rating  disability claims. This organizational measure is aligned with the quality  element for VSRs and RVSRs, which is assessed by measuring the  accuracy of their claims-processing work. An individual performance  element designed to motivate staff to process claims accurately should, in  turn, help VBA meet its overall accuracy goal. Two other key performance  measures for VBA are the average number of days that open disability  claims have been pending and the average number of days it takes to  process disability claims. VSRs are evaluated on their workload  management, a measure of whether they complete designated claims- related tasks within specific deadlines. Individual staff performance in this  element is linked to the agency’s ability to manage its claims workload and  process claims within goal time frames. Finally, a performance measure  that VBA uses to evaluate the claims-processing divisions within its  regional offices—and that, according to VBA, relates to the organization’s  overall mission—is production, or the number of compensation and  pension claims processed by each office in a given time period. Individual  VSRs and RVSRs are evaluated on their productivity, i.e., the number of  claims-related tasks they complete per day. Higher productivity by  individual staff should result in more claims being processed by each  regional office and by VBA overall.
subsections: section_title: Providing and Routinely Using Performance Information to Track Organizational Priorities
paragraphs: Providing objective performance information to individuals helps show  progress in achieving organizational goals and allows individuals to  manage their performance during the year by identifying performance  gaps and improvement opportunities. Regional offices are supposed to use  the critical and non-critical performance elements to evaluate and provide  feedback to their staff. Supervisors are required to provide at least one  progress review to their VSRs and RVSRs each year, indicating how their  performance on each element compares to the defined standards for fully  successful performance. In the offices we visited, supervisors typically  provide some feedback to staff on a monthly basis. For example, VSRs in  the Atlanta regional office receive a memo on their performance each  month showing their production in terms of average weighted actions per  day, their accuracy percentage based on a review of a sample of cases, and  how their performance compared to the minimum requirements for  production and accuracy. If staff members fall below the fully successful  level in a critical element at any time during the year, a performance  improvement plan must be implemented to help the staff member  improve.
section_title: Connecting Performance Expectations to Crosscutting Goals
paragraphs: Performance elements related to collaboration or teamwork can help  reinforce behaviors and actions that support crosscutting goals and  provide a consistent message to all employees about how they are  expected to achieve results. VSR and RVSR performance related to  customer service is evaluated partly based on whether any valid  complaints have been received about a staff member’s interaction with  their colleagues. And performance related to the cooperation and  organizational support element is based on whether staff members’  interaction with their colleagues is professional and constructive.
section_title: Using Competencies to Provide a Fuller Assessment of Performance
paragraphs: Competencies, which define the skills and supporting behaviors that  individuals are expected to exhibit to carry out their work effectively, can  provide a fuller assessment of an individual’s performance. In addition to  elements that are evaluated in purely quantitative terms, VBA uses a  cooperation and organizational support element for VSRs and RVSRs that  requires supervisors to assess whether their staff are exhibiting a number  of behaviors related to performing well as a claims processor.
section_title: Involving Employees and Stakeholders to Gain Ownership of the Performance Management System
paragraphs: Actively involving employees and stakeholders in developing the  performance management system and providing ongoing training on the  system helps increase their understanding and ownership of the  organizational goals and objectives. For example, VA worked with the  union representing claims processors to develop an agreement about its  basic policies regarding performance management. Also, VBA indicated  that it planned to pilot revisions to how productivity is measured for VSRs  in a few regional offices, partly so VSRs would have a chance to provide  feedback on the changes.
section_title: VA’s System May Not Clearly Differentiate between Performance Levels
paragraphs: Clear differentiation between staff performance levels is also an accepted  practice for effective performance management systems. Systems that do  not result in meaningful distinctions between different levels of  performance fail to give (1) employees the constructive feedback they  need to improve, and (2) managers the information they need to reward  top performers and address performance issues. GAO has previously  reported that, in order to provide meaningful distinctions in performance  for experienced staff, agencies should use performance rating scales with  at least three levels, and scales with four or five levels are preferable  because they allow for even greater differentiation between performance  levels. If staff members are concentrated in just one or two of multiple  performance levels, however, the system may not be making meaningful  distinctions in performance.
VA’s performance appraisal system has the potential to clearly  differentiate between staff performance levels. Each fiscal year, regional  offices give their staff a rating on each critical and non-critical  performance element using a three-point scale—exceptional, fully  successful, or less than fully successful. Based on a VA-wide formula, the  combination of ratings across these elements is converted into one of VA’s  five overall performance levels: outstanding, excellent, fully successful,  minimally satisfactory, and unsatisfactory (see fig. 5). Regional offices may  award financial bonuses to staff on the basis of their end-of-year  performance category. Prior to fiscal year 2006, VA used two  performance levels—successful and unacceptable—to characterize each  staff member’s overall performance. To better differentiate between the  overall performance levels of staff, VA abandoned this pass-fail system in  that year, choosing instead to use a five-level scale.
However, there is evidence to suggest that the performance management  system for VSRs and RVSRs may not clearly or accurately differentiate  among staff’s performance. VBA central office officials and managers in  two of the four regional offices we visited raised concerns with VA’s  formula for translating ratings on individual performance elements into an  overall performance rating. These officials said that under this formula it  is more difficult for staff to be placed in certain overall performance  categories than others, even if staff’s performance truly does fall within  one of those categories. Indeed, at least 90 percent of all claims processors  in the regional offices we visited were placed in either the outstanding or  the fully successful category in fiscal year 2007. (Fig. 6 shows the  distribution of overall performance ratings for claims processors in each  office.)
Central and regional office managers noted that, in particular, it is difficult  for staff to receive an overall rating of excellent. Managers in one office  said there are staff whose performance is better than fully successful but  not quite outstanding, but under the formula it is difficult for these staff to  be placed in the excellent category as the managers feel they should be.  An excellent rating requires exceptional ratings in all the critical elements  and a fully successful rating in at least one non-critical element. However,  according to staff we interviewed, virtually all staff who are exceptional in  the critical elements are also exceptional in all non-critical element(s), so  they appropriately end up in the outstanding category. On the other hand,  the overall rating for staff who receive a fully successful rating on just one  of the critical elements—even if they are rated exceptional in all the other  elements—drops down to fully successful. Managers in one regional office  commented that the system would produce more accurate overall  performance ratings if staff were given an overall rating of excellent when  they had, for example, exceptional ratings on three of five overall elements  and fully successful ratings on the other two.
An official in VA’s Office of Human Resources Management acknowledged  that there may be an issue with the agency’s formula. Although neither  VBA nor VA central office officials have examined the distribution of VSRs  and RVSRs across the five overall performance ratings, VA indicated it is  considering changes to the system designed to allow for greater  differentiation in performance ratings. For example, one possible change  would be to use a five-point scale for rating individual elements—probably  mirroring the five overall performance rating categories of outstanding,  excellent, fully successful, minimally satisfactory, and unsatisfactory— rather than the current three-point scale. Under the proposed change, a  staff member who was generally performing at the excellent but not  outstanding level could get excellent ratings in all the elements and  receive an overall rating of excellent. This change must still be negotiated  with several stakeholder groups, according to the VA official we  interviewed.
section_title: Conclusions
paragraphs: In many ways, VBA has developed a training program for its new staff that  is consistent with accepted training practices in the federal government.  However, because VBA does not centrally evaluate or collect feedback on  training provided by its regional offices, it lacks the information needed to  determine if training provided at regional offices is useful and what  improvements, if any, may be needed. Ultimately, this information would  help VBA determine if 80 hours of training annually is the right amount,  particularly for its experienced staff, and whether experienced staff  members are receiving training that is relevant for their positions.  Identifying the right amount of training is crucial for the agency as it tries  to address its claims backlog. An overly burdensome training requirement  needlessly may take staff away from claims processing, while too little  training could contribute to processing inaccuracies. Also, without  collecting feedback on regional office training, VBA may not be aware of  issues with the implementation of its TPSS, the on-line training tool  designed to ensure consistency across offices in technical training. Setting  aside the issue of how many hours of training should be required, VBA  does not hold its staff accountable for fulfilling their training requirement.  As a result, VBA is missing an opportunity to clearly convey to staff the  importance of managing their time to meet training requirements as well  as production and accuracy goals. With the implementation of its new  learning management system, VBA should soon have the ability to track  training completed by individual staff members, making it possible to hold  them accountable for meeting the training requirement.
As with its training program for VSRs and RVSRs, the VA is not examining  the performance management system for claims processors as closely as it  should. VBA is generally using the right elements to evaluate its claims  processors’ performance, and the performance appraisals have the  potential to give managers information they can use to recognize and  reward higher levels of performance. However, evidence suggests the  formula used to place VSRs and RVSRs into overall performance  categories may not clearly and accurately differentiate among staff’s  performance levels. Absent additional examination of the distribution of  claims processors among overall performance categories, VA lacks a clear  picture of whether its system is working as intended and whether any  adjustments are needed.
section_title: Recommendations for Executive Action
paragraphs: The Secretary of Veterans Affairs should direct VBA to:    Collect and review feedback from staff on the training conducted at the  if the 80-hour annual training requirement is appropriate for all VSRs  and RVSRs; the extent to which regional offices provide training that is relevant to  VSRs’ and RVSRs’ work, given varying levels of staff experience; and    whether regional offices find the TPSS a useful learning tool and, if not,  what adjustments are needed to make it more useful; and   Use information from its new learning management system to hold  individual VSRs and RVSRs accountable for completing whatever annual  training requirement it determines is appropriate.
The Secretary of Veterans Affairs should also examine the distribution of  claims processing staff across overall performance categories to  determine if its performance appraisal system clearly differentiates  between overall performance levels, and if necessary adjust its system to  ensure that it makes clear distinctions.
section_title: Agency Comments
paragraphs: We provided a draft of this report to the Secretary of Veterans Affairs for  review and comment. In VA’s written comments (see app. IV), the agency  agreed with our conclusions and concurred with our recommendations.  For example, VBA plans to consult with regional office staff to evaluate its  annual 80-hour training requirement and will examine if staff performance  ratings clearly differentiate between overall performance levels. VA also  provided technical comments that were incorporated as appropriate.
We are sending copies of this report to the Secretary of Veterans Affairs,  relevant congressional committees, and others who are interested. We will  also provide copies to others on request. The report is also available at no  charge on GAO’s Web site at http://www.gao.gov.
Please contact me on (202) 512-7215 if you or your staff have any  questions about this report. Contact points for the Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Key contributors are listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: We were asked to determine: (1) What training is provided to new and  experienced claims processors and how uniform is this training? (2) To  what extent has the Veterans Benefits Administration (VBA) developed a  strategic approach to planning training for claims processors and how well  is their training designed, implemented, and evaluated? And (3) To what  extent is the performance management system for claims processors  consistent with generally accepted performance management practices in  the public sector? To answer these questions, we reviewed documents and  data from the central office of the Department of Veterans Affairs’  Veterans Benefits Administration (VBA) and interviewed VBA central  office officials. We conducted site visits to and collected data from four  VBA regional offices, and visited the Veterans Benefits Academy. We also  interviewed officials from the American Federation of Government  Employees, the labor union that represents Veterans Service  Representatives (VSR) and Rating Veterans Service Representatives  (RVSR). We compared VBA’s training and performance management  systems to accepted human capital principles and criteria compiled by  GAO. We conducted this performance audit from September 2007 through  May 2008 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Regional Office Site Visits
paragraphs: We conducted site visits to 4 of VBA’s 57 regional offices—Atlanta;  Baltimore; Milwaukee; and Portland, Oregon. We judgmentally selected  these offices to achieve some diversity in geographic location, number of  staff, and claims processing accuracy rates, and what we report about  these sites may not necessarily be representative of any other regional  offices or all regional offices (see fig. 7).
During our site visits, we interviewed regional office managers,  supervisors of VSRs and RVSRs, VSRs, and RVSRs about the training and  performance management practices in their offices. The VSRs and RVSRs  we interviewed at the four regional offices had varying levels of  experience at VBA. Regional office managers selected the staff we  interviewed. We also observed a demonstration of VBA’s on-line learning  tool, the Training and Performance Support System (TPSS), and collected  data from the regional offices on, for example, the training they provided  during fiscal year 2007. In conjunction with our visit to the Baltimore  regional office, we also visited VBA’s Veterans Benefits Academy, where  we observed classes for VSRs and RVSRs and interviewed the director of  the Academy.
section_title: Assessment of VBA’s Training for Claims Processors
paragraphs: To determine whether VBA’s training program is consistent with accepted  training practices in the public sector, we relied partly on a guide  developed by GAO that lays out principles that federal agencies should  follow to ensure their training is effective. This guide was developed in  collaboration with government officials and experts in the private sector,  academia, and nonprofit organizations; and in conjunction with a review  of laws, regulations and literature on training and development issues,  including previous GAO reports. The guide lays out the four broad  components of the training and development process (see fig. 8).
The guide also provides key questions for federal agencies to consider in  assessing their performance in each component. (See table 4 for a sample  of these questions.)
In addition, GAO training experts reviewed VBA materials, including  training curricula, lesson plans, and course evaluation forms, to determine  if these materials are consistent with accepted training practices.
section_title: Assessment of VBA’s Performance Management System for Claims Processors
paragraphs: In assessing the performance management system for VSRs and RVSRs,  we relied primarily on a set of accepted practices of effective public sector  performance management systems that has been compiled by GAO. To  identify these accepted practices, GAO reviewed its prior reports on  performance management that drew on the experiences of public sector  organizations both in the United States and abroad. For the purpose of this  review, we focused on the six accepted practices most relevant for VBA’s  claims-processing workforce (see table 5).
section_title: Appendix II: Fiscal Year 2007 Core Technical Training Requirements for VSRs and RVSRs
subsections: section_title: Course title or topic
paragraphs: Additional Issue Specific Lesson Plans are under development.  (Lesson plans can be taken from the Centralized Training Curriculum  found on the C&P Intranet Training Site. If used as provided they do not  require C&P review and approval.
These plans can and often should be modified to focus in on a particular  narrow issue of training need. Modified lesson plans are to be submitted  to C&P Service for review and approval at least 30 days prior to delivery  of training.
Any Challenge-oriented original lesson plan developed by Station  personnel is to be submitted to C&P Service for review and approval at  least 30 days prior to delivery of training.)
C&P Service Broadcasts that may be provided during the course of the  FY may be substituted in place of any training scheduled on an hour by  hour basis.
60 Hours of the required 80 Hours  will be selected from the suggested  topics above. The remaining 20  hours will be selected at the Stations  discretion based upon their own  individual quality review.  (Training provided from the above  topics can be focused on a particular  aspect of the topic; i.e. Cold Injuries  and Rating Hypertension from  Cardiovascular issues could be  separate classes)
Participation in Agency Advancement  Programs (i.e., LEAD, LVA) does not  substitute for Required training  requirements.
Additional Issue Specific Lesson Plans are under development.  (Lesson plans can be taken from the Centralized Training Curriculum  found on the C&P Intranet Training Site. If used as provided they do not  require C&P review and approval.
These plans can and often should be modified to focus in on a particular  narrow issue of training need. Modified lesson plans are to be submitted  to C&P Service for review and approval at least 30 days prior to delivery  of training.
Any Challenge-oriented original lesson plan developed by Station  personnel is to be submitted to C&P Service for review and approval at  least 30 days prior to delivery of training.)
C&P Service Broadcasts that may be provided during the course of the  FY may be substituted in place of any training scheduled on an hour by  hour basis.
Drill Pay Waivers  Pension Awards Processing & BDN  Hospital Reductions  Burial Benefits  Death Pension  Accrued Benefits  Accrued Awards & the BDN  Apportionments  Special Monthly Pension  Helpless Child  Incompetency/Fiduciary  Arrangements  Claims Processing  Auto Allowance and Adaptive  Equipment  Special Adapted Housing  Special Home Adaptation Grants  Incarcerated Veterans  Processing Write Outs  FOIA/Privacy Act  Telephone & Interview Techniques  Telephone Development  IRIS  Introduction to VACOLS  Education Benefits  Insurance Benefits  National Cemetery  VR&E Benefits  Loan Guaranty Benefits  General Benefits – FAQs  Suicidal Caller Guidance  Non-Receipt of BDN Payments  Mail Handling  Income & Net Worth Determinations Bootcamp test and review of VSR  Readiness Guide (2 HRS Required)  Reference Material Training and  Navigation (1 HR Required)  Appeals and Ancillary Benefits  Ready to Rate Development  Customer Service  FNOD Info and PMC Process  Intro to Appeals Process  DRO Selection Letter  Income Adjustment Materials  Income Adjustments  60 Hours of the required 80 Hours  will be selected from the suggested  topics above. The remaining 20  hours will be selected at the Stations  discretion based upon their own  individual quality review.
Overview of VA Mission  Reference Materials: Manual  Training & WARMS  C&P Website  Claims Folder Maintenance  Records Management  POA/Service Organizations  Compensation  Original Compensation Claims  Non-Original Compensation Claims  VA Form 21-526, App. For  Compensation or Pension  Establishing Veteran Status  Claims Recognition  Duty to Assist  Selecting the Correct Worksheet for  VA Exams  Issue Specific Claim Development  Asbestos Claim Development  Herbicide Claim Development  POW Claim Development  Radiation Claim Development  PTSD Claim Development  Undiagnosed Illness Claim  Development  Dependency  Contested Claims  Deemed Valid and Common-law  Marriage  Continuous Cohabitation  Pension  Intro. To Disability Pension  Overview of SHARE (SSA)  Administrative Decision Process  Character of Discharge  Line of Duty – Willful Misconduct  Claims Development  Workload Management Utilizing  WIPP  DEA Training (req. added 4/06)
Intro to Ratings  Paragraph 29 & 30 Ratings  Ratings & the BDN  BDN 301 Interface Video  PCGL Award Letters  PCGL  Dependents & the BDN  Compensation Offsets  Drill Pay Waivers  Star Reporter  Pension Awards Processing & the  BDN  Hospital Reductions  Burial Benefits  Disallowance Processing  DIC Benefits  Death Pension  Accrued Benefits  Accrued Awards & the BDN  Apportionment  Special Monthly Pension  Helpless Child  Incompetency/Fiduciary  Arrangements  Claims Processing  Automobile Allowance and Adaptive  Equipment  Specially Adapted Housing and  Special Home Adaptation Grants  Incarceration  Processing Computer Write Outs  DEA Training (req. added 4/06)
Public Contact Team Training:  FOIA/Privacy Act  Communication Skills  Telephone Development  Inquiry Routing and Information  System (IRIS)  Intro to VACOLS  Other VBA Business Lines  Customer Service  Insurance Education (2 hrs)
Intro to Appeals Process  VACOLS  http://cptraining.vba.va.gov/  C&P_Training/vsr/VSR_  Curriculum.htm#att  http://cptraining.vba.va.gov/  C&P_Training/vsr/VSR_  Curriculum.htm#iam.
section_title: Appendix III: Excerpts from Fiscal Year 2007 Training Plans for Four Regional Offices
paragraphs: Each training plan we reviewed contained the same informational  categories, some of which were what courses were offered by the regional  office, whether or not the course was conducted, and how many  employees completed the training. Although the fiscal year 2007 training  plans we reviewed include data on whether and when the course was  actually completed, the initial training plans submitted at the beginning of  the fiscal year of course do not have this information. The lists provided  below include the first 25 courses listed on each plan alphabetically, a  small sample of the courses that the regional offices reported they  completed for the fiscal year.
section_title: Appendix IV: Comments from the Department of Veterans Affairs
section_title: Appendix V: GAO Contact and Staff Acknowledgments
paragraphs: Daniel Bertoni (202) 512-7215 bertonid@gao.gov.
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Clarita Mrena, Assistant Director;  Lorin Obler, Analyst-in-Charge; Carolyn S. Blocker; and David Forgosh  made major contributions to this report; Margaret Braley, Peter Del Toro,  Chris Dionis, Janice Latimer, and Carol Willett provided guidance; Walter  Vance assisted with study design; Charles Willson helped draft the report;  and Roger Thomas provided legal advice.
section_title: Related GAO Products
paragraphs: Veterans’ Benefits: Improved Management Would Enhance VA’s Pension  Program. GAO-08-112. Washington, D.C.: February 14, 2008.
Veterans’ Disability Benefits: Claims Processing Challenges Persist,  while VA Continues to Take Steps to Address Them. GAO-08-473T.  Washington, D.C.: February 14, 2008.
Disabled Veterans’ Employment: Additional Planning, Monitoring, and  Data Collection Efforts Would Improve Assistance. GAO-07-1020.  Washington, D.C.: September 12, 2007.
Veterans’ Benefits: Improvements Needed in the Reporting and Use of  Data on the Accuracy of Disability Claims Decisions. GAO-03-1045.  Washington, D.C.: September 30, 2003.
Human Capital: A Guide for Assessing Strategic Training and  Development Efforts in the Federal Government. GAO-03-893G.  Washington, D.C.: July 2003.
Results-Oriented Cultures: Creating a Clear Linkage between Individual  Performance and Organizational Success. GAO-03-488. Washington D.C.:  March 14, 2003.
Major Management Challenges and Program Risks: Department of  Veterans Affairs. GAO-03-110. Washington, D.C.: January 1, 2003.
Veterans’ Benefits: Claims Processing Timeliness Performance Measures  Could Be Improved. GAO-03-282. Washington, D.C.: December 19, 2002.
Veterans’ Benefits: Quality Assurance for Disability Claims and Appeals  Processing Can Be Further Improved. GAO-02-806. Washington, D.C.:  August 16, 2002.
Veterans’ Benefits: Training for Claims Processors Needs Evaluation.  GAO-01-601. Washington, D.C.: May 31, 2001.
Veterans Benefits Claims: Further Improvements Needed in Claims- Processing Accuracy. GAO/HEHS-99-35. Washington, D.C.: March 1, 1999.