VA’s mission is to promote the health, welfare, and dignity of all veterans in recognition of their service to the nation by ensuring that they receive medical care, benefits, social support, and lasting memorials. In carrying out this mission, the department manages one of the largest health care delivery systems in the United States that provides enrolled veterans with a full range of services. These services may include primary care; mental health care; and outpatient, inpatient, and residential treatment. The Veterans Health Administration (VHA), one of the department’s three major components, is responsible for overseeing the provision of health care at all VA medical facilities. Information technology (IT) is widely used and critically important to supporting the department in delivering health care to veterans. As such, VA operates and maintains an IT infrastructure that is intended to provide the backbone necessary to meet the day-to-day operational needs of its medical centers and other critical systems supporting the department’s mission. The infrastructure is to provide for data storage, transmission, and communications requirements necessary to ensure the delivery of reliable, available, and responsive support to all VA staff offices and administration customers, as well as veterans. The Office of Information and Technology (OIT) is responsible for managing the majority of VA’s IT- related functions. The office provides strategy and technical direction, guidance, and policy related to how IT resources are to be acquired and managed for the department. VA provides health care services to approximately 9 million veterans and their families and relies on its health information system—VistA—to do so. VistA has been essential to the department’s ability to deliver health care to veterans. It was developed based on the collaboration between staff in the VA medical facilities and VHA IT personnel. Specifically, clinicians and IT personnel at the various VA medical facilities collaborated to define the system’s requirements and, in certain cases, carried out its development and implementation. As a result of these efforts, the system has been in operation since the early 1980s. VistA supports a complex set of clinical and administrative capabilities. It is comprised of an architecture that ties together servers and personal computer workstations with various applications within VA facilities and the supporting infrastructure, such as data centers, storage, and messaging technologies. The core system and database code are programmed in the MUMPS programming language. Among other things, VistA contains an EHR for each patient and supports clinics and medical centers. In addition, the system provides functionality beyond the EHR and exchanges information with many other applications and interfaces. For example, the system also provides the functionality of a time and attendance program, asset management system, library, and billing system, among other things. Users interact with VistA through a number of interfaces that connect stored health data. These interfaces enable the system to communicate (send or exchange data) with other VA systems, as well as with other federal agencies (e.g., DOD), health information exchange networks, and COTS products. According to OIT officials, applications either interface with VistA directly through a messaging protocol or extract data from the system via a reporting mechanism. The Computerized Patient Record System is a graphical user interface to VistA that runs on workstations, laptops, and tablets and enables the department to support clinical workflows. Specifically, the Computerized Patient Record System enables the department to create and update an individual EHR for each VA patient. Among other things, clinicians can order lab tests, medications, diets, radiology tests, and procedures; record a patient’s allergies or adverse reactions to medications; request and track consults; enter progress notes, diagnoses, and treatments for each encounter; and enter discharge summaries. According to VHA officials, there are also more than 100 COTS products that interface with VistA. In addition to these commercial products, medical equipment or devices at local facilities may also require interfaces to the system, and these vary on a site-by-site basis. Over the last several decades, VistA has evolved into a technically complex system that supports health care delivery at more than 1,500 locations, including VA Medical Centers, outpatient clinics, community living centers, and VA vet centers. Customization of the system by local facilities has resulted in about 130 clinical versions of VistA—referred to as instances. According to the department, no two VistA instances are identical. Further, each instance is comprised of over 27,400 routines (executable modules of code), which are logically grouped into products or modules. VistA products or modules can also be comprised of one or more software applications that support health care functions, such as providing care coordination and mental health services. The department reported that there are approximately 140 to 200 products or modules that comprise the system. The 130 clinical instances of VistA are operated from four regional VA data centers. Users interact with the system through the Computerized Patient Record System. Aggregated clinical data from every instance of the system are located on servers hosted at VA’s National Data Center. Over time, VA has identified the need for enhancements and modifications to VistA in order to ensure that the system keeps up with current technology and health care delivery. However, according to the department, the system has become difficult and costly to maintain. This is a result of, for example, being programmed in MUMPS, a language for which there is a dwindling supply of qualified software developers. It is also due to years of decentralized customization of the system by staff members who were permitted to develop and implement applications at the local level. OIT and VHA serve as the technical and functional leaders, respectively, for the department’s health care delivery and, together, they have worked to develop and maintain VistA for decades. Specifically, OIT is responsible for managing the majority of VA’s IT-related functions. The office provides strategy and technical direction, guidance, and policy related to how IT resources are to be acquired and managed for the department. According to the department, OIT’s mission is to collaborate with its business partners (such as VHA) and provide a seamless, unified veteran experience through the delivery of state-of-the-art technology. The Assistant Secretary for Information and Technology/Chief Information Officer (CIO) serves as the head of OIT and is responsible for providing leadership for the department’s IT activities. The CIO also advises the Secretary regarding the execution of VA’s IT systems appropriation, consistent with the Federal Information Technology Acquisition Reform Act. For fiscal year 2019, the department has been appropriated $4.1 billion for IT. According to VA’s budget documentation, about $1.2 billion of this amount is intended to support IT staffing and associated costs for approximately 8,100 full-time employees. VHA provides information and expertise to OIT to support the department’s health-related information systems. For example, VHA officials help identify clinical and business needs used to inform IT requirements development. The Under Secretary for Health is the head of VHA and is supported by the Principal Deputy Under Secretary for Health, four Deputy Under Secretaries for Health, and nine Assistant Deputy Under Secretaries for Health. After nearly 2 decades of pursuing multiple efforts to modernize VistA, in June 2017, the former VA Secretary announced that the department planned to acquire the same EHR system that DOD is acquiring—Cerner Millennium. According to the department, it has chosen to acquire this product because Cerner Millennium should allow VA’s and DOD’s patient data to reside in one system, thus, potentially reducing or eliminating the need for manual and electronic exchange and reconciliation of data between two separate systems. Accordingly, the department awarded an indefinite delivery, indefinite quantity contract to Cerner Corporation in May 2018 for a maximum amount of $10 billion over 10 years. Cerner is to replace the 130 instances of VistA with a standard COTS system to be implemented across VA. This new system is to support a broad range of health care functions including acute care, clinical decision support, dental care, and emergency medicine. When implemented, the new system will be expected to become the authoritative source of clinical data to support improved health, patient safety, and quality of care provided by VA. The Electronic Health Record Modernization (EHRM) program is responsible for managing the Cerner contract implementation. For fiscal year 2019, the program was appropriated about $1.1 billion for planning and managing the transition from VistA to Cerner. Further, the department has estimated that an additional $6.1 billion in funding, above the Cerner contract amount, will be needed to fund additional project management support supplied by outside contractors, government labor costs, and infrastructure improvements over the 10- year contract period. VA plans to deploy the new EHR system at three initial operating capability sites within 18 months of October 1, 2018, with a phased implementation of the remaining sites over the next decade. Each VA medical facility is expected to continue using VistA until the new system has been deployed. The three initial deployment sites, located in the Pacific Northwest, are the Mann-Grandstaff, American Lake, and Seattle VA Medical Centers and related clinical facilities that operate the same instances of VistA. These are the first locations where the system is expected to “go live.” The task order to deploy the Cerner system at the three initial sites provides a detailed description of the steps Cerner needs to take in order to reach initial operating capability at the Mann-Grandstaff site in March 2020, and at the Seattle and American Lake sites in April 2020. According to the schedule, the initial operating capability sites are expected to be operational by July 2020. In order to maintain internal control activities over an IT system and its related infrastructure, organizations should be able to define physical and performance characteristics of the system, including descriptions of the components and the interfaces. Further, consistent with GAO’s Cost Estimating and Assessment Guide, a comprehensive system definition should identify customization and the environment in which the system operates. While defining a complex IT system can be challenging, having an adequate understanding of its characteristics will better position the organization to comprehensively project and account for costs over the life of a system or program as well as identify specific technical and program risks. Definition of VistA remains important because VA plans to continue using the system during the department’s decade-long transition to the Cerner system. VA maintains multiple documents and a database that describe parts of VistA, including various components and interfaces. However, despite these existing sources, OIT officials acknowledged that there is no comprehensive definition of the VistA system. Consequently, VA has completed a number of efforts to better define VistA and understand the environment in which it operates and additional work is planned in the future. Specifically, VA has documented descriptions of the system, including the components that comprise it. These descriptions are documented in multiple sources: the VA Monograph, VA Systems Inventory, and VA Document Library. The VA Monograph is a document maintained by OIT that provides an overview of VistA and non-VistA applications used by VHA. According to VHA officials, the VA Monograph is the primary document that describes the components of the system. The Monograph describes VistA in terms of modules. For modules identified, including VistA modules, information such as the associated business functions, VA Systems Inventory identification number, and a link to the VA Document Library for additional technical information are provided. The VA Systems Inventory is a database maintained by OIT that identifies current IT systems at the department, including systems and interfaces related to VistA. For systems identified, the database includes information such as the system name, the system status (i.e., active, in development, or inactive), and related system interfaces. The VA Document Library is an online resource for accessing documentation (i.e., user guides and installation manuals) on the department’s nationally released software applications, including VistA. VA has taken additional steps to further define the system. For example, EHRM program officials recognized the need to further understand the customization of VistA components at the various medical facilities and have conducted analyses to do so. These analyses include: Variance analysis: As part of its VistA Evolution program, which has focused on standardizing a core set of VistA functionality, the department implemented a process to compare the instances of VistA installed at sites to the Enterprise Standard version. The results of this analysis allowed the department to assess the criticality of each variance, which is expected to help with VA’s transition to the Cerner system. Module analysis: EHRM program subject matter experts undertook an analysis that involved reviewing and assessing capabilities provided by VistA modules. This analysis enabled department officials to determine whether the capability provided by a VistA module could be provided by the Cerner system, or whether another COTS solution would be required to support this function going forward. Visual mapping: EHRM program officials also directed an analysis that involved developing a notional visual mapping of VA’s health care applications, components, and supporting systems within the health delivery environment. The results of this analysis provided a description of the current state of one instance of VistA and the VA health environment, which is intended to inform the department of possible opportunities for business process and IT improvements as it proceeds with the Cerner acquisition. Nevertheless, even with these analyses, VA has not yet fully defined VistA, including, for example, identifying performance characteristics of the system and describing the environment in which it operates. The department’s three sources that describe VistA and the additional analyses undertaken do not provide insight into site specific customizations of the system. For example, the VA Monograph does not include information on module customization at local facilities. In addition, according to OIT officials, the systems inventory does not reflect differences among the 130 different instances of VistA and does not take into consideration regional and local customizations of related components. Further, the visual mapping analysis noted that there was not full insight of the intertwined structure of data and applications or the various local customizations of VistA. EHRM program officials stated that they have not been able to fully define VistA and understand all local customizations due to the decentralization of the development of the system and its evolution over more than 30 years. They explained that VistA’s complexity is partly due to the various instances of the system, compounded by local customizations, which have resulted in differences in VistA instances operating at various facilities. According to EHRM program documentation, Cerner’s contract calls for the company to conduct comprehensive assessments to capture the current state of technical and clinical operations at specific facilities, as well as identify site-specific requirements where the Cerner system is planned to be deployed. As of June 2019, Cerner had completed site assessments for the three initial operating capability sites in the Pacific Northwest and had planned additional assessments at future deployment sites. The initial site assessments included, among other things, an assessment of the unique VistA instances and the environment in which the system operates. The continuation of planned site assessments should provide a thorough understanding of the 130 VistA versions, help the department better define VistA, and position it for transitioning from VistA to Cerner’s COTS solution. When using public funds, an agency must employ effective management practices in order to let legislators, management, and the public know the costs of programs and whether they are achieving their goals. To make those evaluations for a program or for a system as large and complex as VistA, a complete understanding of the system and reliable cost information is required. By following a methodology and utilizing reliable data, an agency can ensure that all costs are fully accounted for, which in turn, better informs management decisions, establishes a cost baseline, and enhances understanding of a system’s performance and return on investment. Fundamental characteristics of reliable costs are that they should be accurate (unbiased, not overly conservative or optimistic), well- documented (supportable with source data, clearly detailed calculations, and explanations for choosing a particular calculation method), credible (identifying any uncertainty or biases surrounding data or related assumptions), and comprehensive (costs are neither omitted nor double counted). Identification of VistA’s costs remains important because VA plans to continue using the system during the department’s transition to the Cerner system over the next decade. VA identified costs for VistA and its related activities adding up to approximately $913.7 million, $664.3 million, and $711.1 million in fiscal years 2015, 2016, and 2017, respectively—for a total of about $2.3 billion over the 3 years. However, the department could not sufficiently demonstrate the reliability of certain costs that were identified. In addition, VA identified other categories of VistA-related costs, but omitted these costs from the total. Of the $2.3 billion total costs for VistA, VA demonstrated that only approximately $1 billion of these costs were reliable. Specifically, OIT officials identified VistA-related costs within seven categories. The officials were able to sufficiently explain why these categories were included in the development and sustainment costs for VistA and how they were documented by the department; the officials also presented detailed source data for our examination. As a result of our review, we determined that the cost data for these seven categories were accurate, well-documented, credible, and comprehensive and, thus, sufficiently reliable. Table 1 provides a summary of the program costs identified for VistA by OIT and VHA for fiscal years 2015 through 2017 that we determined to be reliable. As shown in the table, VA identified costs for the following seven categories for fiscal years 2015 through 2017: VistA Evolution – The VistA Evolution program costs were associated with VistA strategy, system design, product development, and program management. These costs totaled approximately $549.6 million. Interoperability – The Interoperability program focused on sharing electronic health data between VA and non-VA facilities, including private sector providers and DOD. For example, interoperability costs were associated with architecture, strategy, the Interagency Program Office, product development, and program management. These VistA-related costs totaled approximately $140.2 million. Virtual Lifetime Electronic Record (VLER) Health – This program focused on streamlining the transition of electronic medical information between VA and DOD. These VistA-related costs were associated with product development and program management and totaled approximately $81.2 million. Contracts – Contract costs for VistA Evolution included VHA’s obligations associated with workload management, change management, clinical requirements, and clinical interoperability. These VistA-related costs totaled approximately $202.8 million. Intergovernmental personnel acts – Intergovernmental personnel acts are agreements for the temporary assignment of personnel between the federal, state, and local governments; colleges and universities; Indian tribal governments; federally funded research and development centers; and other eligible organizations. These costs accounted for VHA’s need to use outside experts from approved entities for limited periods of time to work on VistA Evolution assignments. The total VistA-related costs were approximately $2.4 million. Memorandums of understanding – According to VHA, memorandums of understanding are agreements used by the administration to obtain the services of personnel between VA entities for VistA-related activities. These agreements accounted for approximately $2.3 million. Pay – Costs in this category included salaries for VHA staff who worked on VistA-related projects as well as travel, training, and supply costs associated with employment. These costs totaled approximately $34.1 million. However, VA was not able to sufficiently demonstrate the reliability of approximately $1.3 billion in costs related to VistA. Specifically, OIT officials identified the additional legacy VistA costs that generally fell into three categories: Legacy VistA: Infrastructure, hosting, and system sustainment – Legacy VistA costs are generally related to the maintenance of fully operational items, such as VistA Imaging and Fileman—two key components related to VistA’s operation. The costs also included obligations for costs related to hosting health data in both VA and non-VA facilities. The OIT officials and subject matter experts estimated these total costs to be approximately $343 million during fiscal years 2015 through 2017. However, we were not able to determine the reliability of these costs because, for example, source data were not well documented; changes in the cost information provided to us during our review indicated that the cost data may not be credible; and subject matter experts were unclear about how to separate VistA costs from non- VistA costs. Related software – Related software costs are associated with the software supporting, or closely integrated with, VistA that were identified by EHRM officials, yet not tracked directly for one of the VistA-related programs. Both OIT and VHA identified software licensing costs as VistA-related obligations. The EHRM program reported these costs to be approximately $389 million in total during fiscal years 2015 through 2017. However, we were not able to determine the reliability of the costs in this category for a variety of reasons, including that source data were not well documented. In addition, VA officials were not clear regarding how the total amounts in each category should be divided between OIT and VHA. Given this confusion, we were not able to determine if the costs were fully accurate or credible. OIT personnel (pay and administrative) – According to EHRM officials, OIT does not track labor costs by program. Instead, the department provided estimations of the amount of salaries paid to OIT government staff working on activities such as VistA Evolution, program management, and overall support of VistA and related applications. OIT personnel costs were estimated by the EHRM program office to be approximately $544 million total during fiscal years 2015 through 2017. However, we were not able to determine the reliability of costs in this category because assumptions made for estimating the personnel and salary costs were not well documented and could not be verified. In addition, VA omitted certain VistA costs from the total costs identified for fiscal years 2015, 2016, and 2017. Specifically, VA omitted the following costs: Additional hosting – OIT officials stated that additional costs related to hosting health data by an outside vendor, as well as hosting backup VistA instances at each of the medical center sites, should also be included in the total costs for VistA; however, VA omitted these costs from the total for fiscal years 2015 through 2017. Specifically, according to the officials, calculating costs for these hosting activities requires subject matter experts to identify equipment, space, utilities, and maintenance costs for resources allocated specifically for VistA. However, the department has not yet developed a methodology to calculate the costs. The officials said they were working on identifying a reliable approach for calculating these costs in the future. Data standardization and testing – OIT officials stated that additional costs related to work on clinical terminology mapping and functional testing were not included in the total costs for VistA for fiscal years 2015 through 2017. This work related to mapping existing clinical data to national standards and making updates to VistA or the Joint Legacy Viewer and included mapping data and building test scripts and reports. OIT officials noted that this work had been critical to the VistA Evolution program, but they did not provide actual cost data in this category. The lack of sufficiently reliable and comprehensive costs indicates that the department is not positioned to accurately report the annual costs to develop and sustain VistA. This is due in part to VA not following a well- documented methodology that describes how the department determined the total costs for the system. In lieu of a methodology, OIT officials said that leadership and staff from the program took efforts to identify and track the cost components and contracts associated with the system. However, they noted that costs associated with VistA were not all clearly labeled as VistA in an IT system and it was necessary to estimate other costs. The officials were also unable to verify how VistA-related costs were separated from other department costs in all areas and subject matter experts were not consistently familiar with the estimation methods employed and how VistA was defined for the purposes of calculating costs. Further, VA officials noted that they were still working on the best approach to identifying and calculating omitted costs. Without documenting the methodology for what costs are to be included and how they were identified and calculated, VA’s total does not accurately reflect the development and sustainment costs for VistA. As a result, the department, legislators, and the public do not have the comprehensive, reliable information needed to understand how much it actually cost to develop and maintain the system. Further, VA does not have the reliable information needed to make critical management decisions for sustaining the many versions of VistA over the next 10 years until the Cerner system is fully deployed. In our report, we are making a recommendation for VA to improve its reporting of VistA’s costs. Specifically, we are recommending that the department develop and implement a methodology for reliably identifying and reporting the total costs of VistA. The methodology should include steps to identify the definition of VistA and what is to be included in its sustainment activities, as well as ensure that comprehensive costs are corroborated by reliable data. In written comments on a draft of the report, the department agreed with the recommendation and stated that it will provide the actions it plans to take to address this recommendation within 180 days. In conclusion, although VA is not likely to be positioned to retire VistA for at least another 10 years, the department lacks the comprehensive and reliable cost information needed to make critical management decisions for sustaining the system. As the department continues to work toward acquiring a new electronic health record, it will be important for VA to take actions to address our recommendation for improving the reporting of VistA costs. Doing so is essential to helping ensure that decisions related to the current system are informed by reliable cost information and that there is an accurate basis for reporting on the return on its investment for replacing VistA. Chair Lee, Ranking Member Banks, and Members of the Subcommittee, this completes my prepared statement. I would be pleased to respond to any questions that you may have. If you or your staffs have any questions about this testimony, please contact Carol C. Harris, Director, Information Technology Management Issues, at (202) 512-4456 or harriscc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony statement. GAO staff who made key contributions to this testimony are Mark Bird (Assistant Director), Rebecca Eyler, Jacqueline Mai, Monica Perez-Nelson, Scott Pettis, Jennifer Stavros-Turner (Analyst in Charge), and Charles Youman. This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.

Summary: VA provides health care services to approximately 9 million veterans and their families and relies on its health information system—VistA—to do so. However, the system is more than 30 years old, is costly to maintain, and does not fully support exchanging health data with DOD and private health care providers. Over nearly 2 decades, VA has pursued multiple efforts to modernize the system. In June 2017, the department announced plans to acquire the same system—the Cerner system—that the Department of Defense is implementing. VA plans to continue using VistA during the department's decade-long transition to the Cerner system. GAO was asked to summarize its report that is being released today which discusses, among other things, (1) the extent to which VA has defined VistA and (2) the department's annual costs to develop and sustain the system. In preparing the report on which this testimony is based, GAO analyzed documentation that defines aspects of VistA and identifies components to be replaced; and evaluated the reliability of cost data, including funding obligations associated with the development and sustainment of VistA for fiscal years 2015, 2016, and 2017. The Department of Veterans Affairs (VA) has various documents and a database that describe parts of the Veterans Health Information Systems and Technology Architecture (VistA); however, the department does not have a comprehensive definition for the system. For example, VA has identified components that comprise VistA, identified interfaces related to the system, and collected system user guides and installation manuals. VA has also conducted analyses to better understand customization of VistA components at various medical facilities. Nevertheless, the existing information and analyses do not provide a thorough understanding of the local customizations reflected in about 130 versions of VistA that support health care delivery at more than 1,500 sites. Program officials stated that they have not been able to fully define VistA due to the decentralization of the development of the system for more than 30 years. Cerner's contract to provide a new electronic health record system to VA calls for the company to conduct comprehensive assessments to identify site-specific requirements where its system is planned to be deployed. Three site assessments have been completed and additional assessments are planned. If these assessments provide a thorough understanding of the 130 VistA versions, the department should be able to define VistA and be better positioned to transition to the new system. VA identified costs for VistA and its related activities adding up to approximately $913.7 million, $664.3 million, and $711.1 million in fiscal years 2015, 2016, and 2017, respectively—for a total of about $2.3 billion over the 3 years. However, of the $2.3 billion, the department was only able to demonstrate that approximately $1 billion of these costs were sufficiently reliable. In addition, the department omitted VistA-related costs from the total. The lack of a sufficiently reliable and comprehensive total cost for VistA is due in part to not following a well-documented methodology that describes how the department determined the costs for the system. As a result of incomplete cost data and data that could not be determined to be sufficiently reliable, the department, legislators, and the public do not have a complete understanding of how much it has cost to develop and maintain VistA. Further, VA lacks the information needed to make decisions on sustaining the many versions of the system. In its report being issued today, GAO is recommending that VA develop and implement a methodology for reliably identifying and reporting the total costs of VistA. The department agreed with the recommendation.