VA Disability Benefits: Planning Gaps Could Impede Readiness for Successful Appeals Implementation
Released: 2018-12-12T00:00:00

Letter

Background

VA’s Current Disability Compensation Appeals Process

VA’s process for deciding veterans’ eligibility for disability compensation begins when a veteran submits a claim to VA. Staff in one of VBA’s 57 regional offices assist the veteran by gathering additional evidence, such as military and medical records, that is needed to evaluate the claim. Based on this evidence, VBA decides whether the veteran is entitled to compensation and, if so, how much. A veteran dissatisfied with the initial claim decision can generally appeal within 1 year from the date of the notification letter sent by VBA.

Under the current appeals process (now referred to by VA as the legacy process), an appeal begins with the veteran filing a Notice of Disagreement. VBA then re-examines the case and generally issues a Statement of the Case that represents its decision. A veteran dissatisfied with VBA’s decision can file an appeal with the Board. In filing that appeal, the veteran can indicate whether a Board hearing is desired. Before the Board reviews the appeal, VBA prepares the file and certifies it as ready for Board review. If the veteran requests a hearing to present new evidence or arguments, the Board will hold a hearing by videoconference or at a local VBA regional office. The Board reviews the evidence and either issues a decision to grant or deny the veteran’s appeal or refers the appeal back to VBA for further work.

VA’s New Appeals Process

According to VA’s appeals plan, VA intends to implement the Act by February 2019, by replacing the current appeals process with a process offering veterans who are dissatisfied with VBA’s decision on their claim five options. Two of those options afford the veteran an opportunity for an additional review of VBA’s decision within VBA, and the other three options afford them the opportunity to bypass additional VBA review and appeal directly to the Board.

Under the new appeals process, the two VBA options will be: 1. Request higher-level review: The veteran asks VBA to review its initial decision based on the same evidence but with a higher-level official reviewing and issuing a new decision. 2. File supplemental claim: The veteran provides additional evidence and files a supplemental claim with VBA for a new decision on the claim. The veteran can also request a VBA hearing.

The three Board options will be: 3. Request Board review of existing record: The veteran appeals to the Board and asks it to review only the existing record without a hearing. 4. Request Board review of additional evidence, without a hearing. 5. Request Board review of additional evidence, with a hearing.

In November 2017, VA initiated a test of the new VBA higher-level review and supplemental claim options. According to VA’s appeals plan, a purpose of this test—the Rapid Appeals Modernization Program (RAMP)—is to reduce legacy appeals by providing veterans with a chance for early resolution of their claims within VBA’s new process.

Participation in RAMP is voluntary, but veterans must withdraw their pending legacy appeal to participate, according to VA’s appeals plan.

VA Has Not Provided Complete Information on Four Elements in the Act

In our March 2018 report, we found that VA’s November 2017 plan for implementing a new disability appeals process while attending to appeals under way in the current (legacy) process, addressed 17 of 22 elements required by the Act. For the 5 remaining elements, we found that it partially addressed 4 elements related to implementation monitoring, productivity projecting, and workforce planning, and did not address 1 element related to identifying total resources. This element called for delineating the resources needed by VBA and the Board to implement the new appeals process and address legacy appeals.

We recommended in March 2018 that VA address all 22 required elements in the Act in VA’s appeals plan to Congress—including delineating resources required for all VBA and Board appeals options— using sensitivity analyses and results from its test, RAMP, where appropriate and needed.

Since our March 2018 report, VA has taken some action on each of the five elements that we found were not fully addressed at that time. For example, VA added details related to projecting staff productivity, identifying total resources, as well as determining personnel requirements and productivity projections for processing appeals. For identifying total resources, VA added FTE information for other offices that help implement the appeals process and prepared a model to project resource needs.

Although VA now addresses the 1 element related to projecting productivity, it only partially addresses 4 elements related to monitoring implementation, workforce planning, and delineating the total resources. For example, as of November 2018, VA’s plan does not contain metrics for monitoring implementation. Moreover, for total resources, the updated plan does not delineate the total resources required by VBA and the Board, such as the resources necessary for information technology and training. We acknowledge that in some cases delineating total resources could prove challenging, such as delineating information technology resources for the legacy and new appeals processes. We also acknowledge that implementing corrective actions to fully address these 4 elements may be challenging within the next several weeks, but we continue to believe VA has an opportunity to further address these 4 elements as part of certifying the agency’s readiness prior to the full implementation of the new process.

VA Has Addressed Some Gaps in Its Plans to Monitor and Assess Performance, though Further Steps Remain

In our March 2018 report, we found gaps in VA’s planning for how it will monitor and assess performance of the new appeals process when it is implemented. Specifically, we reported that the plan did not (1) establish timeliness goals for two of the three Board options (i.e., Board review of additional evidence without a hearing and Board review of additional evidence with a hearing); (2) articulate aspects of performance important for managing appeals, such as accuracy of decisions, veteran satisfaction with the process, or cost; (3) explain how the performance of the new appeals process would be compared to that of the legacy process; or (4) explain how the agency would monitor relative workloads of, and resources devoted to, the new and legacy appeals processes.

To address these gaps, we recommended that VA clearly articulate in its appeals plan how VA will monitor and assess the new appeals process compared to the legacy process, including specifying a balanced set of goals and measures—such as timeliness goals for all VBA appeals options and Board dockets, and measures of accuracy, veteran satisfaction, and cost—and related baseline data. Articulating a balanced set of goals that cover key aspects of managing appeals is important to avoid promoting skewed behaviors (e.g., favoring timeliness over accuracy) and to fully understanding performance.

In its progress reports, VA addressed some but not all aspects of this recommendation (see table 1).

VA has made progress in monitoring performance and addressing workload changes in its new and legacy appeals processes, but still lacks a complete set of balanced goals and measures. As we noted in our July 2018 testimony, VA has developed sensitivity models and other analyses to monitor and forecast future VBA and Board workloads, production, and staffing requirements to help VA manage the legacy and new appeals processes. However, VBA and the Board have yet to specify a complete set of balanced goals for monitoring the performance of the new appeals processes. According to the November 2018 progress report, the Board plans to develop timeliness goals after VA fully implements the new appeals process. Until VA fully develops a set of balanced goals and measures, the agency risks not fully understanding how well the reforms are performing.

Regarding comparing the performance of the new and legacy appeals processes, VA has previously reported that the agency plans to implement the reporting requirements in section 5 of the Act. This section requires VA to report performance measures related to, among other things, timeliness, productivity, and outcomes, without specifying whether or how VA should compare performance of the new versus legacy processes.

In November 2018, VBA and Board officials told us they intend to use timeliness and productivity metrics from section 5 to compare the two processes. However, in its updated plans to date, VA has been reporting average timeliness of decisions made to date under RAMP—VA’s test of the two VBA options—without reporting the average time cases are pending. Moreover, VA has not been reporting timeliness data on both decisions and pending cases according to the month that they entered into RAMP, which present a more balanced indication of performance and trends. In November 2018 VBA and Board officials told us they would consider reporting timeliness using a monthly cohort that reflects when appeals were filed.

VBA and Board officials also said they have taken steps to collect, through surveys, comparable information on veterans’ satisfaction with the new and legacy appeals processes. According to VBA and Board officials, they have pre-tested the surveys—which is considered a best practice by survey methodologists—and are coordinating the survey efforts with one another. VBA and Board officials also told us that the agency will report on accuracy and outcomes (grants and denials of claims) in the new process. However, they also stated that these measures would not provide a fair comparison with the legacy process because the Act eliminated several of the requirements formerly required in the legacy appeals administrative processes.

Although VA officials said they would develop a plan for comparing the performance of the two appeals processes after the new process is fully implemented, they did not indicate how soon they would do so. Developing such a plan would better position the agency to fully understand whether the new process is an improvement.

VA Has Augmented Its Master Schedule to a Limited Extent

Our March 2018 report identified elements of a high-quality and reliable implementation schedule that were missing from VA’s master schedule for appeals reform. Specifically, we reported that VA’s high-level master schedule—which the agency included with its November 2017 plan—did not (1) include all key activities; (2) show which activities must finish prior to the start of other activities, or the amount of time an activity could be delayed before the delay affects VA’s estimated implementation date; (3) reflect interim goals and milestones for monitoring implementation; or (4) assign resources for activities.

We recommended that VA augment the master schedule for its appeals plan to reflect all activities—such as modifications to information technology systems—as well as assigned responsibilities, interdependencies, start and end dates for key activities for each workgroup, and resources. These steps establish accountability and reduce overall risk of implementation failures.

In response to our recommendation, the Board, VBA and other VA administrations made progress over time with developing and integrating underlying plans into the integrated master schedule (IMS) in spring and summer 2018. According to VA officials, VA set a baseline schedule for implementing appeals reform in response to the potential February 2019 implementation date established in the Act. Since November 2017, VA’s plan and progress reports have stated that VA uses an agency-wide governance structure to coordinate implementation, and regularly uses the schedule as a management tool for monitoring progress on appeals reform. For example, the Board’s project manager meets regularly with those responsible for major activities to check progress, including weekly meetings with leadership, and identifies and corrects issues related to schedule execution.

In October 2018, VA provided us with lower-level schedules and information that allowed us to conduct a more detailed assessment of VA’s IMS against applicable best practices criteria. The six criteria we assessed lower-level schedules against were:

Capturing all activities: schedule should reflect all activities necessary to perform work to accomplish a project’s objective.

Sequencing activities: activities should be logically sequenced in the order they are to be carried out so that critical program dates can be met.

Assigning resources: schedule should reflect all resources necessary to complete work, verify whether resources will be available, and identify any constraints.

Verifying horizontal and vertical traceability: schedule should be rational and logically sequenced, account for interdependencies among activities, and provide a way to evaluate the current status (horizontal traceability). Also, the various levels of a schedule— summary, intermediate, and detailed—should be consistent with one another and enable different teams to work to the same schedule expectations (vertical traceability).

Updating the schedule using actual progress and logic: maintain and continually update the schedule to reflect a realistic forecast of start and end dates of activities.

Maintaining a baseline schedule: use original configuration of the program plan as a point of comparison for the current plan to manage scope, timeframes, and required resources.

We found that, while VA has made progress with providing more detail, its master and underlying schedules only minimally met sound practices for project management. Specifically, as with our March 2018 assessment, we found that the schedule does not contain enough detail to manage the work or provide a realistic representation of the resources and time needed for this project. For example, the schedule did not contain a work breakdown structure that defines the work, activities, and resources necessary to accomplish implementation. Moreover, half of all the remaining activities are missing logic that shows which activities must finish prior to the start of other activities. In addition, the schedule contains an invalid critical path, meaning that the schedule does not present the amount of time that key activities could be delayed before such delays affect VA’s estimated implementation date. Without a valid critical path, management cannot focus on activities that will detrimentally affect the key program milestones and deliveries if they slip.

To address our March 2018 recommendation, VA would need to ensure that all activities are accounted for, that scheduled activities appear in the correct order, that resources are properly allocated, that all activities appear on the critical path, and that a schedule risk analysis accounts for all risks. We provide a more detailed explanation of our assessment results in appendix I.

In addition, establishing an overly optimistic schedule can reduce capacity for carrying out a project and potentially create pressure to sacrifice the quality of work activities to meet deadlines. Moreover, many of VA’s activities are slated to be concurrently completed just before implementation, posing a significant risk to implementing reform in February. For example, according to VA’s schedule, the agency needs to complete 117 activities after January 1, 2019. Further, other VA efforts to redesign or update key aspects of VA’s disability compensation process—including the Veterans Benefits Management System (VBMS)—were not driven by robust, comprehensive planning and did not achieve their schedule goals.

While VA intends to start full implementation in February, we do not know the extent to which the lack of a robust schedule poses risks to successful and smooth implementation. Even if taking corrective actions to address our findings may not be feasible before February, incorporating such lessons learned into future project planning could help VA improve its project scheduling capabilities.

VA Has Addressed Many, but Not All Key Risks to Implementation

In our March 2018 report, we found that VA’s appeals plan could more fully assess key risks related to implementing the new appeals process. In particular, we found that VA’s plan did not include testing of new Board options or clearly define how it would assess the RAMP test of the VBA- only options before implementing them more broadly. Further, we reported that VA’s plan had not comprehensively reflected key risks because the agency had not established a complete and balanced set of goals and measures, which are a necessary pre-condition to effectively assessing risk.

We recommended that VA ensure that the appeals plan more fully addresses risk associated with appeals reform by, for example, assessing risks against a balanced set of goals and measures, articulating success criteria and an assessment plan for RAMP, and testing or conducting sensitivity analyses of all five appeals options before fully implementing the new appeals process.

In its progress reports, VA took many steps to address our recommendation, although key steps are remaining for VA to better assess risks associated with implementing appeals reform and managing appeals workloads in the legacy process (see table 2).

Sound redesign and change management practices both suggest that tests be rigorously monitored and evaluated and that further roll-out occur only after an agency takes any needed corrective action and determines that the new process is achieving previously identified success criteria. Until VA takes these remaining steps, it may not have comprehensively addressed key risks to better position the agency for successful implementation of appeals reform.

In conclusion, VA is undertaking an ambitious effort to reform its disability appeals process—while onboarding hundreds of new staff and implementing new technology—that will affect the lives of hundreds of thousands of veterans with disabilities for years to come. Consistent with our prior recommendations, VA has made concrete progress to improve its planning for disability appeals reform while it attends to legacy appeals. Efforts such as resuming sensitivity analysis to monitor workloads and testing VBA and Board appeals options will provide useful information to guide VA through the uncertainty often associated with process change.

However, VA has reported it plans to fully implement the new disability appeals process in February 2019 even though it has yet to fully address our recommendations. While fully implementing our recommendations prior to February 2019 may not be feasible, doing so would better position VA to ensure successful implementation. Nevertheless, VA should still work to increase clarity around its plans prior to fully implementing reform.

Moreover, many of the principles of sound planning practices that informed our recommendations remain relevant during process change. By continuing to improve its approach to performance measurement, scheduling, and risk management, even after implementation, VA could better ensure that the new process meets veterans’ needs.

Chairman Roe, Ranking Member Walz, and Members of the Committee, this concludes my prepared statement. I would be pleased to respond to any questions you may have at this time.

GAO Contact and Staff Acknowledgments

For further information about this testimony, please contact Elizabeth H. Curda at (202) 512-7215 or curdae@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. Other key contributors to this testimony include James Whitcomb (Assistant Director), Juaná Collymore, Michele Grgich, Sara Pelton, and Rachel Pittenger. In addition, key support was provided by Susan Aschoff, Mark Bird, Alex Galuten, Jason Lee, Sheila R. McCoy, Almeta Spencer, and Walter Vance.

Appendix I: Assessment of the Extent to Which VA Followed Aspects of Scheduling Leading Practices

For this testimony, we assessed the steps that the Department of Veterans Affairs (VA) has taken to address our March 2018 recommendations and what aspects remain unaddressed, including the extent to which VA is using sound practices for scheduling key projects. In summary, we identified several areas where VA’s most recent schedule falls short of sound practices. Further incorporating sound practices into future project planning could help VA improve its project scheduling capabilities.

We reviewed VA’s integrated master schedule (IMS) for the appeals reform effort and underlying sub-schedules to assess them against 6 of the 10 best practices, which we determined most relevant to our March 2018 recommendation that VA augment its master schedule for VA’s appeals plan to reflect all activities—such as modifications to information technology systems—as well as assigned responsibilities, interdependencies, start and end dates for key activities for each workgroup, and resources, to establish accountability and reduce the overall risk of implementation failures. Specifically, we analyzed the following related scheduling best practices: (1) Capturing all activities, (2) Sequencing all activities, (3) Assigning resources to all activities, (4) Verifying that the schedule can be traced vertically and horizontally, (5) Updating the schedule using actual progress and logic and (6) Maintaining a baseline schedule.

We assessed VA’s lower-level schedules against these 6 best practices by:

Checking for specific problems that could hinder the schedule’s ability to respond to changes. For example, we: o Examined if there are any open-ended activities (i.e., activities with no predecessor and/or successors), o Searched for activities with poor logic:

For example, Start to Start successor only or Finish to Finish predecessor only which represent dangling logic, or

Logic on summary tasks rather than attached to detailed tasks (summary tasks are for organizing the schedule and should not drive the logic). o Looked for activities with constraints which keep the schedule rigid (e.g., start no earlier than, finish no later than, etc.), o Determined if activities were resource loaded—which helps to cost out the schedule—and examine whether resources are over- allocated or not available when needed, o Examined the schedule’s critical path to determine whether or not it was reliable and logical, o Examined schedule float and determined if it was reasonable, and o Examined whether the schedule was baselined, its status cycle, and what deviations there were from the original plan. We also determined if there were any actual start or finish dates recorded in the future and whether there was any broken logic between planned tasks.

We also interviewed VA officials responsible for managing the schedule. We scored each scheduling leading practice on a five-point scale: “not met”, “minimally met”, “partially met”, “substantially met” and “fully met.” We determined the characteristic assessment rating by assigning each best practice rating a number and taking the average. Our resulting conclusions based on this assessment are as follows:

VA’s project schedule minimally meets the best practice of capturing all activities. The schedule does not have well-defined start and finish milestones and there is not a project work breakdown structure (WBS) or corresponding WBS dictionary to define the work for each WBS element. We were not able to independently verify contractor work or major handoffs and deliverables in the schedule. In addition, there were activities with duplicate names, which could make communication difficult between VA teams, particularly between team members who are responsible for updating and integrating multiple schedules.

VA’s project schedule minimally meets the best practice of sequencing activities. There are issues with missing dependencies, dangling activities, summary links, constraints and lags that affect the schedule meeting this best practice. Specifically, of the remaining activities, 55 percent have missing logic, over 12 percent are dangling, 42 percent have date constraints and 4 percent have leads assigned. When activities are not correctly linked, the program cannot use the integrated master schedule (IMS) to identify disconnects or hidden opportunities and cannot otherwise promote efficiency and accuracy or control the program by comparing actual to planned progress. When this happens, the schedule will not allow a sufficient understanding of the program as a whole, and users of the schedule may lack confidence in the dates and the critical path.

VA’s project schedule minimally meets the best practice of assigning resources. While the schedule contains ‘Task Owner’ assignments, the Task Owner information has no effect on the durations or forecasted start and finish dates of detailed activities. Information on resource needs and availability in each work period assists the program office in forecasting the likelihood that activities will be completed as scheduled. If the current schedule does not allow insight into the current or projected allocation of resources, then the risk of the program’s slipping is significantly increased.

VA’s project schedule minimally meets the best practice of verifying the schedule is traceable horizontally and vertically. There was no evidence in the schedule of hand-offs within the schedule—that is givers and receivers are easily identifiable in the schedule. We were unable to determine the relationship between lower-lever activities in the project schedule and higher-level activities and milestones in the management briefs provided to us. Specifically, we could not map the activities in the briefs to activities in the schedule. This inconsistency also prevented the verification of dates between the project schedule and higher-level management documents, even with documents that were provided from the same month as the October schedule.

Products and outcomes were not easily traced through the sequencing of effort in the project schedule. In both cases the schedule did not respond appropriately to “shocks”; that is, greatly increasing the durations of some activities to increase the overall time required to complete the project did not affect the dates of key milestones. The duration increase of each activity did not affect the overall time line because the activity in question had a constraint that would not allow the project to appropriately extend.

VA’s project schedule minimally meets the best practice of updating the schedule using progress and logic. Date anomalies, such as planned dates in the past or actual dates in the future, were found. The schedule was not current as of the date delivered to GAO. While officials report that they update the schedule regularly, a schedule narrative document does not accompany the schedule update that would detail changes to the current schedule and describe information such as the status of key milestone dates, changes in network logic, and a description of the current critical path(s).

VA’s project schedule minimally meets the best practice of maintaining a baseline schedule. Officials said that the baseline schedule is the basis for performance measurement. But while baseline start and baseline finish dates were provided in the initial schedule, its activities were too high level, obfuscating the calculation of detail variances in subsequent schedules. There is also no evidence of a schedule basis document, which would include a general overview of the purpose of the schedule, other key basis information such as an overview of assumptions, rationale for durations specific to the CMR schedule, and required software settings. There is also no evidence of performance measuring.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.