Veterans Benefits Modernization: Management and Technical Weaknesses Must Be Overcome If Modernization Is To Succeed
Released: 1996-06-19T00:00:00

Letter

Business Strategy Needed to Solve Entrenched Service-Delivery Problems

VBA’s business environment encompasses many difficult challenges. These include a backlog of disability claims, improving a number of relationships with other organizations that affect how VBA does its work, and responding to its customers who are frustrated about the long-standing need to improve the accuracy and timeliness of processing claims. To deal with these issues, as well as cope with today’s constrained budgetary climate, the agency is undertaking a number of major initiatives, including beginning a business process reengineering effort for its compensation and pension programs, restructuring its regional office responsibilities, and consolidating its data centers.

VBA has, however, been proceeding without an overall business strategy clearly setting forth how it will improve its performance and tackle entrenched service-delivery problems. For example, the reported backlog of original and reopened disability claims increased from 378,000 in fiscal year 1990 to a high of 571,000 at the end of December 1993. This rise was due to several factors, including increasing complexity in claims processing and the use of inexperienced regional claims raters. VBA instituted several conventional stopgap measures to deal with this backlog. It authorized extensive overtime, shifted workloads among regional offices, purchased information technology equipment, increased the number of claims raters by about one third (from 667 to 897), and relaxed some paperwork requirements, such as accepting photocopies of certain documents. As a result the backlog has been reduced, but it is now still about 380,000—similar to the 1990 level.

Similar trends have been experienced in the processing times for original disability compensation claims, which rose from an average of 151 days in fiscal year 1990 to 213 days in fiscal year 1994. The stopgap measures used to decrease the backlog have also reduced the average processing time in fiscal year 1995 to 161—10 days more than the level in fiscal year 1990. VBA officials acknowledge that these measures cannot be sustained over a prolonged period of time. VBA must, therefore, find other solutions to achieve greater service-delivery breakthroughs.

Other entities also affect the speed with which VBA processes claims and the agency’s overall direction. For example, VBA relies on the Veterans Health Administration for most medical information needed to substantiate a disability claim, and the Department of Defense for information relating to a veteran’s service time and conditions of discharge, as well as medical information from the veteran’s tour of active duty. Delays by either of these organizations can have a significant impact on the timeliness of VBA’s claims processing.

Judicial review organizations also affect VBA’s workload and backlog. For example, the Board of Veterans’ Appeals returns almost half of its cases to VBA regional offices for additional development and reconsideration each year. The Board itself also has a significant and increasing backlog of cases; its appeals grew from about 19,500 in fiscal year 1990 to more than 50,000 in fiscal year 1995—an increase of more than 150 percent. It takes the Board about 2 years to render a decision from the date it receives an appeal.

In addition, VBA—like most federal agencies—must deal with constrained resource levels and, at the same time, maintain existing levels of service and operations. VBA is in the process of restructuring its regional offices in an effort to cope with declining resources. At the same time, funding for VBA’s information technology initiatives is discretionary and, as such, comes under close budgetary scrutiny by the Congress and the Office of Management and Budget (OMB).

A comprehensive business strategy is needed—one that includes developing strategic and information resources management plans, setting performance goals and measures, and incorporating the results of major agency initiatives, such as business process reengineering. VBA is moving in this direction; currently, however, it has no clearly articulated business strategy.

Recent legislative changes provide the framework for VBA to develop such a strategy and identify the tools needed to implement it. For example, the Government Performance and Results Act of 1993 requires agency heads to submit to OMB and the Congress a strategic plan for program activities, including a mission statement, goals and objectives, and a description of how these will be achieved and what key factors could affect their achievement. The act also requires that agencies prepare annual performance plans for each program—performance indicators that will allow measurement of outputs and service levels. In addition, the Information Technology Management Reform Act of 1996 requires agency heads to establish goals for improving the efficiency and effectiveness of agency operations and, as appropriate, the delivery of services to the public, through more effective use of information technology and business process reengineering.

VBA’s weaknesses in planning have been well documented since 1987. VBA’s planning process has been cited by us and others for (1) not having specific, measurable goals and objectives against which progress can be assessed and (2) not analyzing the costs and benefits of alternative approaches to modernization. According to VBA officials, they are in the process of developing strategic and information resources management plans and will have them ready to use in preparing the agency’s budget submission for fiscal year 1998. Assistance in this area could come from the National Academy of Public Administration, which has recently been commissioned by the Senate Appropriations Committee. In the Committee’s September 1995 report on the 1996 appropriations bill, the Committee provided $1 million to the Academy for a comprehensive assessment of VBA, with particular emphasis on the specific steps required to make claims processing more efficient and less time-consuming. The Academy will evaluate the modernization initiative and its link to strategic goals and priorities, efforts to reengineer VBA’s claims-processing methodology, performance measures for restructuring, and the roles of the Board of Veterans’ Appeals and the Court of Veterans Appeals. As of a few weeks ago, VBA was still working out the details of this study with the Academy.

VBA also needs to develop a full set of performance goals or measures. At present, processing timeliness is the primary performance measure that VBA uses. Customer-focused goals, aimed at improving the quality of service, are needed. For example, a VBA survey of “stakeholders”indicated that , in their view, an emphasis on quality over productivity alone would be the key to service excellence at VBA. These stakeholders defined quality as making the correct award decision the first time, which would improve the timeliness of claims processing and reduce the number of appeals filed.

VBA’s current goal for claims processing was set without the benefit of any clear plan. For example, its goal is to reduce average original compensation claims processing time to 106 days by 1998; this goal was set as part of a 1993 agreement with OMB to establish outcome-oriented performance goals. The performance goal is not linked to a business strategy or plan that explains how the agency intends to achieve this goal.

Reengineering is key to achieving major performance improvements that VBA establishes as business goals. As our 1994 study pointed out,organizations that successfully develop information systems do so only after thoroughly analyzing and redesigning their current business processes. Information system projects that do not first consider business process redesign typically fail, or reach only a fraction of their full potential.

In response to concerns raised by us and others over the past 3 years, VBA is preparing to reengineer its compensation and pension claims-processing operations, and has taken several positive steps. In November 1995 the agency established a Business Process Reengineering Office, and subsequently adopted a business process reengineering methodology. It also hired a consultant to assist with reengineering. By the end of this month, a business process reengineering team comprised of VBA staff and the consultant is expected to have completed a key step in the process by developing a proposal for changing the compensation and pension business processes. This proposal will be submitted to VBA management for review and approval before implementation. VBA also plans to begin a different business analysis project each year for its other four business areas. The next area planned for such an analysis is educational assistance.

It is still too early to judge whether the current business process reengineering effort will help VBA achieve its goals, but we continue to have some concerns about VBA’s current business process reengineering focus and approach. For example, VBA has not yet set quantifiable performance measures using the experiences and performance of other leading claims-processing organizations. Also, the scope of VBA’s analysis and reengineering of its business processes in the compensation and pension area does not address the claims appeal process, which has a significant impact on the timeliness and quality of some claims-processing decisions. Finally, as I will discuss later, we are concerned that reengineering is not the driver behind all of VBA’s information technology initiatives.

To solve entrenched problems and sustain long-term improvements in service delivery and operations, VBA must first know exactly what it needs to pay attention to and where it wants to go. A business strategy containing specific goals and performance measures is absolutely essential. By effectively using the framework established in recent legislation to develop the business strategy and complete its strategic and information resources management plans, VBA will go a long way toward setting out a clear path to be followed.

Managing Information Technology Is Essential to VBA’s Success

VBA’s investment in modernization activities has yielded some improvement in hardware and software applications. However, it is difficult to measure return on any of these investments.

As shown in attachment 1, between fiscal years 1986 and 1995, VBA reported that it obligated about $688 million for information technology, of which about $284 million, or about 40 percent, was for systems modernization. In December 1992 VBA awarded the first contract in its planned three-stage procurement. During stage I, VBA acquired a number of personal computers, local area networks, minicomputers, and commercial off-the-shelf software for its 58 regional offices; during stage II, VBA procured imaging equipment and associated software. Stage III was suspended in 1994; during this stage, VBA was to procure mainframe computers for its data centers in Hines, IL, and Philadelphia.

VBA has also realized some limited benefits from the development of several short-term, targeted software applications that are being used on equipment acquired during stage I. These projects include the following:

Control of Veterans Records—used to track the location of veterans’ claims folders containing application-related information;

Rating Board Automation—used to generate letters to veterans regarding

Personal Computer-Generated Letters—used to prepare general letters to disability claimants.

To help manage its information technology investments in a way that will lead to major returns, VBA must now meet the challenges of new information technology legislation that has been modeled after the best practices of leading private and public organizations. For example, the Information Technology Management and Reform Act and the Paperwork Reduction Act require agency heads to analyze the agency’s mission and, on the basis of this analysis, revise business processes as appropriate; design and implement a process for maximizing the value and assessing and managing the risks of information technology acquisitions; integrate budgetary, financial, and program management decisions in this process; and use this process to select, control, and evaluate the results of information technology initiatives.

VBA needs to make major improvements in the way it manages its information technology investments to meet these legislative requirements. Our analysis of past and current VBA information technology initiatives shows that VBA lacks the critical cost, benefit, and risk information necessary to determine whether it has made worthwhile investments. Our analysis also shows that these initiatives preceded VBA’s business process reengineering effort, which increases the risk that they may need to be substantially changed or abandoned once reengineering results become available. For example:

Between fiscal years 1993 and 1995, VBA purchased 24 minicomputers without having a clear understanding of the software applications to be placed on the equipment or the benefits to be derived from this investment. Although VBA expected to use these minicomputers in processing claims, they were not put into use until recently, when VBA began testing its software application to track claims folders. This was done at four sites: Baltimore; St. Petersburg; San Juan, Puerto Rico; and Winston-Salem, NC.

At VBA’s educational assistance processing sites in Atlanta and St. Louis, the agency has acquired and is in the process of installing imaging equipment to scan all documents in the chapter 30 education claims folders, which contain an average of 30 documents each. VBA has not, however, performed any reengineering analysis for the educational assistance area to assess how the imaging equipment could be used to improve education claims processing. In addition, while VBA has begun to collect baseline information to compare against post-implementation data in order to determine what impact the equipment will have on its operations at the Atlanta site, such information has not been collected for St. Louis, which has been using such equipment since 1987.

Also, this past March VBA embarked on a 2-year effort at its St. Petersburg regional office to replace its current benefits payment system. The objectives of this replacement system were to (1) permit more timely updating of master benefit files through on-line access, (2) provide national access to service organizations that must respond to veterans’ questions about the status of their claims, and (3) address the potential effects of processing benefits payments and other critical information after the turn of the century.

This recent project has several inherent risks that must be assessed before VBA can determine if this initiative will be worth the investment. First, the project team, comprised of VBA staff and contractor personnel, will be using a new software development language and a rapid application development methodology. While this methodology is used more frequently in the private sector, it has not been previously used at VBA. When it is used, highly skilled and experienced people are a necessity. Given both VBA’s and the contractor’s unfamiliarity with using this methodology, the staff and contractor must learn the new tools and become proficient with them so as not to jeopardize the implementation of the replacement payment system, scheduled for 1998.

We believe that this initiative is high risk because the payment replacement system timetable was based on unrealistic assumptions about the productivity and skills of newly-trained, inexperienced people, and the level of complexity of the task. Further, as I will discuss in more detail in a few moments, although VBA is in the process of developing software for its replacement system, our evaluation found that VBA is very weak in its ability to develop software and manage software-development contracts. This factor substantially increases the risks associated with this project.

Another risk is that this project was not following sound systems-development practices. For example, VBA’s system development guidelines—policies and procedures used to design and develop computer software and systems—call for verification and validation of the system requirements before proceeding from one phase of system development to the next phase. VBA’s implementation of the standard systems-development process consists of four phases: planning, analysis, design, and construction. It has been demonstrated that proceeding to a subsequent phase without reviewing the work done in the current phase for correctness, consistency, and completeness will almost always adversely impact on the project’s cost, its performance, and the delivery schedule. VBA directed the project team to proceed into the system design phase, however, without completing this important first step. Further, the data model that is being used to develop the replacement payment system has not been completed, although this should have been done prior to proceeding into the system design phase. The incomplete requirements verification and validation and incomplete data model increases the risk that the system will be designed incorrectly. Also, VBA does not have cost-benefit information with which to assess its return on this investment. For example, it has not estimated the total amount of software that must be developed, or its cost.

In addition to lacking the information to determine whether or not specific projects will pay off, VBA also lacks a process that ranks and prioritizes its investments in information technology as a consolidated portfolio. VBA is undertaking several projects simultaneously, without a full consideration of the resources required, costs, risks, and potential impact on agency operations. Current system-development activities—including addressing the year-2000 issue, data-center consolidation and related software conversion, and replacement of the benefits payment system—are all examples of investments that have not been ranked or prioritized.

Year 2000. Like all other federal agencies—and private businesses—VBA must address the effects of processing information in light of the change of century. Most of the computer software in use today employs 2-digit date fields. Consequently, at the turn of the century, computer software will be unable to distinguish between the years 1900 and 2000, since both would be designated “00.” Industry and government experts have already gone on record saying that the effort to correct this problem will become extremely costly and time-consuming, and requires early and detailed planning. If the year-2000 problem is not addressed, it will render the vast majority of date-sensitive computer information unusable or obsolete. For example, calculations based on incorrect dates in service could result in errors in processing benefit checks in the compensation and pension programs. In VBA’s educational assistance program, VBA could send threatening debt-collection letters to veterans who do not actually owe money; charge incorrect interest rates to veterans or charge interest to veterans who do not owe money; or send debtor information to the Internal Revenue Service for refund withholding, to the federal government for wage garnishment, or to private credit firms to go on a veteran’s credit report.

In our opinion, the year-2000 issue is an absolutely critical challenge that VBA faces over the next 2-3 years. Some of the computer code was developed more than 20 years ago, using nonstandard coding techniques. In some cases, the software documentation may be incomplete or nonexistent.

It is essential that VBA develop and implement a strategy to address the inherent risks that accompany the year-2000 change. First, a sufficient number of experienced staff must be devoted to this task, especially since VBA must maintain its current software and service levels at the same time that it is correcting date-sensitive code. Second, it will need to complete the programming by 1998, since industry experts recommend that 1999 be reserved for thoroughly testing the year-2000 changes. Third, VBA must have a contingency plan that outlines alternatives for processing claims if systems are not corrected.

Data-Center Consolidation and Related Software Conversion. In response to a request from OMB, VA and VBA are in the process of developing a strategy paper to reduce operational costs by consolidating their data centers. However, critical information in terms of costs and benefits is missing—information needed to determine how and when this should be done and how this effort ranks in terms of priority with competing demands, such as the year-2000 activities.

Currently, VA’s data center is in Austin, Texas, and uses IBM computer equipment to process the Department’s accounting and financial management information related to administrative operations. VBA’s two data centers—Hines and Philadelphia—use mostly Honeywell equipment; the Hines facility primarily processes disability (compensation and pension) claims, while Philadelphia processes insurance claims. The joint VA/VBA data-center consolidation strategy paper is due to OMB in July.

Because the data-center consolidation approach must also consider converting the current software to run on more modern computer equipment, added risks must be considered. Specifically, VBA is considering converting the Benefits Delivery Networksoftware—currently in use at Hines—to more modern computer equipment. The cost and time frames for this conversion will depend upon which of the three data centers is chosen as the site for Benefits Delivery Network processing. To date, two studies have been commissioned to evaluate the software conversion. The first, commissioned by VA, estimated the cost and time frames for moving the current Benefits Delivery Network to IBM equipment; the second, commissioned by VBA, assessed the feasibility of converting the Benefits Delivery Network software. The finding was that such a conversion is feasible, and could likely take 2-3 years to complete.

Neither study, in our view, provides enough information on all three sites to adequately assess the investment needed, nor do they fully address General Services Administration (GSA) criteria for making software conversion decisions. Neither contains an analysis of alternative approaches or a full description of the cost, benefits, and risks of conversion. We have discussed our analysis with VA and VBA officials, and they agree with our assessment of these studies. VA has since hired another consultant to analyze the costs and benefits and to develop a strategy for data-center consolidation. Until the results of this study are available, VBA will not be able to identify the best approach to take.

The conversion of the Benefits Delivery Network software must be carried out correctly in order to realize the potential benefits of data-center consolidation. This conversion will require much work and a dedicated staff with in-depth knowledge of the existing network software. In-depth knowledge of the Benefits Delivery Network software currently resides at VBA’s Hines data center. It will also be necessary, despite limitations on personnel and funding, to maintain the current network software and service level of operations while converting the software. The conversion risk will be further compounded by VBA’s need to address the year-2000 issue.

Replacement of the Payment System. In addition to the previously mentioned risks associated with the replacement of the payment system, we believe that VBA did not adequately consider alternative approaches for achieving the reliability and additional functionality expected in the replacement. The Federal Information Resources Management Regulations require that agencies use their systems requirements as the basis for analyzing alternatives, commensurate with the size and complexity of the agency’s business needs. The regulation stipulates that agencies should calculate the total estimated cost of each feasible alternative, and assess the risks.

Further, VBA recently acquired excess computing equipment from GSA to replace some of the equipment at Hines and Philadelphia. According to staff at both centers, the excess equipment is more reliable, has greater capacity, and is less expensive to maintain. This newer equipment allows VBA more time to analyze and assess alternatives because it makes the computing environment more stable.

Lastly, critical to VBA’s ability to identify the true return on any of these information technology initiatives is the need for accurate and reliable cost information. Our analysis of VBA’s modernization obligations to date shows that the cost of these activities may be understated because VBA lacks a managerial cost-accounting system to track payroll benefits and indirect costs associated with modernization. VBA also appears to have miscategorized some items in its information technology budget as nonmodernization items when, in our opinion, they were modernization-related and should have been categorized in that way. In addition, VBA has not updated its modernization life-cycle cost estimate of $478 million in over 3 years. Therefore, precisely how much VBA’s systems modernization effort will ultimately cost taxpayers remains uncertain. VBA’s chief financial officer is currently in the process of developing guidance for implementing a cost-accounting methodology.

Our work indicates that VBA has much to do to develop an investment strategy that can assure the Congress that scarce information technology dollars are being spent on the highest priority projects with the greatest potential for a substantial return on investment. The recent acquisition of excess equipment now provides VBA with an opportunity to effectively develop this kind of approach. VBA must expeditiously develop an effective investment process for selecting, controlling, and evaluating information technology initiatives in terms of cost, capability of the system to meet requirements, risk, timeliness, and quality; give top priority to addressing the year-2000 problem; and improve its accounting of obligations and costs associated with the modernization.

Software Development Foundation to Execute Technology Investment Needs Urgent Attention

Once technology investment processes have identified the most beneficial information technology projects in terms of cost, benefit, and return, the focus then shifts to the technical capabilities necessary to make the projects a reality. The agency must be able to quickly determine if it has the necessary in-house capability to develop the software for the new system or whether this development should be performed by an experienced contractor.

In order to mitigate any risk of not being able to deliver high-quality software within schedule and budget, agencies must have a disciplined and consistent software-development process. Software development has been identified by many experts as one of the most risky and costly components of systems development.

To evaluate VBA’s software development processes, we applied the Software Engineering Institute’s software capability evaluation methodology to those projects identified by VBA as using the best development processes. This evaluation compares agencies’ and contractors’ software development processes against the Institute’s five-level software capability maturity model, with 5 being the highest level of maturity and 1 being the lowest. As shown in attachment 2, these levels—and the key process areas described within each—define an organization’s ability to develop software, and can be used to measure improvements in this area.

On the basis of our analysis, we determined that VBA is operating at a level-1 capability, defined as ad hoc and chaotic. At this level, VBA cannot reliably develop and maintain high-quality software on any major project within existing cost and schedule constraints, placing VBA modernization at significant risk. In this context, VBA relies solely on the various capabilities of individuals rather than on an institutional process that will yield repeatable, or level-2, results. VBA does not satisfy any of the criteria for a level-2 capability, the minimum level necessary to be able to significantly improve productivity and return on investment. For example, VBA is weak in the requirements management, software project planning, and software subcontract management areas, with no identifiable strengths or planned improvement activities. However, VBA can build upon its strengths in the software configuration-management and software quality-assurance areas.

Our report on this matter is being issued soon and will contain recommendations to better position VBA to develop and maintain its software successfully and to protect its software investments. Specifically, we recommend in that report that VBA obtain expert advice to improve its ability to develop high-quality software and expeditiously implement a plan that describes a strategy for reaching the repeatable (i.e., level 2) level of process maturity, delay any major investment in new software development—beyond what is needed to sustain critical day-to-day operations—until the repeatable level of process maturity is attained, and ensure that any future contracts for software development require the contractor to have a software development capability of at least a level 2.

VBA agreed with all but one recommendation. VBA agreed that a repeatable level of process maturity is a goal that must be attained, but disagreed that “all software development beyond that which is day-to-day critical must be curtailed.” VBA stated that the payment system replacement projects and other activities to address the change of century must continue. We agree that the software conversion and development activities required to address issues such as the year 2000 must continue; we would, in fact, characterize these as sustaining critical day-to-day operations. However, systems-development initiatives in support of major new projects, such as the replacement of the payment system, should be reassessed for the risk of potential delays, cost overruns, and shortfalls in anticipated system functions and features. We are pleased to see that VBA is already initiating positive actions relating to our other recommendations, including acquiring expert advice to assist it in improving its ability to develop high-quality software, consistent with criteria set forth by the Software Engineering Institute.

Conclusions

The business and operational problems facing VBA are complex and not easy to resolve. VBA has begun to take action to improve agency operations and service delivery, but it has not yet implemented enough of the right kinds of actions—actions that involve developing a sound business strategy and the supporting plans, approaches, and measures to guide them into the next century. The need for more rigorous management and technical methods is critical if VBA is to successfully develop modern, efficient, and cost-effective business processes and computer systems that will allow them to deliver truly improved services to veterans.

Mr. Chairman, this completes my testimony this morning. I would be pleased to respond to any questions you or other members of the Subcommittee may have at this time.

Attachments

GAO Software Capability Maturity Model:

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.