Electronic Health Records: VA and DOD Need to Establish Goals and Metrics for Their Interoperability Efforts
Released: 2015-10-27T00:00:00

Letter

Background

Historically, patient health information has been scattered across paper records kept by different caregivers in many different locations. Thus, the move toward collecting, storing, retrieving, and transferring these records electronically can significantly improve the quality and efficiency of care. This is especially true in the case of military personnel and veterans, because they tend to be highly mobile and may have health records at multiple facilities both within and outside the United States.

Interoperability allows patients’ electronic health information to be available from provider to provider, regardless of where it originated. Achieving this depends on, among other things, the use of agreed-upon health data standards (e.g., standardized language for prescriptions and laboratory testing) and the ability of systems to use the information that is exchanged.

Currently, both VA and DOD operate separate electronic systems to create and manage electronic health records. VA uses its Veterans Health Information Systems and Technology Architecture (VistA), a system that the department developed in-house and that consists of 104 separate computer applications; while DOD uses the Armed Forces Health Longitudinal Technology Application (AHLTA), which consists of multiple legacy medical information systems developed from customized commercial software applications.

A Long History of Efforts to Achieve Electronic Health Record Interoperability

Since 1998, VA and DOD have undertaken a patchwork of initiatives intended to allow their health information systems to exchange information and increase interoperability. Among others, these have included initiatives to share viewable data in existing (legacy) systems, link and share computable data between the departments’ updated heath data repositories, and jointly develop a single integrated system. Table 1 below summarizes a number of the departments’ key efforts.

In addition to the initiatives mentioned in table 1, the departments took a variety of actions to respond to provisions in the National Defense Authorization Act (NDAA) for Fiscal Year 2008, which required them to jointly develop and implement fully interoperable electronic health record systems or capabilities in 2009. The act also directed them to set up an interagency program office (referred to as the IPO) to serve as a single point of accountability for these efforts.

Department officials stated that their previous initiatives, along with meeting six interoperability objectives established by their Interagency Clinical Informatics Board, had enabled them to meet the deadline for full interoperability established by the act. However, we previously identified a number of challenges that the departments faced in managing their efforts in response to the act and to address their common health IT needs. In particular, although these initiatives have helped to increase data-sharing in various ways, they have been plagued by persistent management challenges that have hampered progress toward fully interoperable electronic health record capabilities.

In March 2011, the secretaries of the two departments announced that they would develop a new, joint integrated electronic health record system (referred to as iEHR). This was intended to replace the departments’ separate systems with a single common system, thus sidestepping many of the challenges they had previously encountered in trying to achieve interoperability. However, in February 2013, about 2 years after initiating iEHR, the secretaries announced that the departments were abandoning plans to develop a joint system, due to concerns about the program’s cost, schedule, and ability to meet deadlines. The IPO reported spending about $564 million on iEHR between October 2011 and June 2013.

In place of the iEHR initiative, VA stated that it would modernize VistA, while DOD planned to buy a commercially available system to replace AHLTA. The departments stated that they would ensure interoperability between these updated systems, as well as with other public and private health care providers. In December 2013, the IPO was re-chartered and given responsibility for establishing technical and clinical standards and processes to ensure that health data between the two departments (and other providers) are integrated.

We issued several prior reports regarding this approach, in which we noted that the departments did not substantiate their claims that it would be less expensive and faster than developing a single, joint system. We also noted that the departments’ plans to modernize their two separate systems were duplicative and stressed that their decisions should be justified by comparing the costs and schedules of alternate approaches. We therefore previously recommended that the departments develop cost and schedule estimates that would include all elements of their approach (i.e., modernizing both departments’ health information systems and establishing interoperability between them) and compare them with estimates of the cost and schedule for the single-system approach. If the planned approach was projected to cost more or take longer, we recommended that they provide a rationale for pursuing such an approach.

VA and DOD agreed with our prior recommendations and stated that initial comparison indicated that the current approach would be more cost effective. However, as of October 2015, the departments have not provided us with a comparison of the estimated costs of their current and previous approaches. On the other hand, with respect to their assertions that separate systems could be achieved faster, both departments have developed schedules that indicate their separate modernizations are not expected to be completed until after the 2017 planned completion date for the previous single system approach.

In light of the departments’ not having yet implemented a solution that allows for seamless electronic sharing of health care data, the National Defense Authorization Act (NDAA) for Fiscal Year 2014 included requirements pertaining to the implementation, design, and planning for interoperability between VA’s and DOD’s electronic health record systems. Among other actions, provisions in the act directed each department to (1) ensure that all health care data contained in their systems (VA’s VistA and DOD’s AHLTA) complied with national standards and were computable in real time by October 1, 2014, and (2) deploy modernized electronic health record software to support clinicians while ensuring full standards-based interoperability by December 31, 2016.

Interoperability Efforts Continue, but DOD and VA Need to Develop Goals and Metrics for Assessing Their Progress

Our August 2015 report noted that the departments have engaged in several near-term efforts focused on expanding interoperability between their existing electronic health record systems. For example, the departments analyzed data related to 25 “domains” identified by the Interagency Clinical Informatics Board and mapped health data in their existing systems to standards identified by the IPO. The departments also expanded the functionality of their Joint Legacy Viewer—a tool that allows clinicians to view certain health care data from both departments in a single interface.

In addition, VA and DOD have both moved forward with plans to modernize their respective electronic health record systems. VA has developed a number of plans for its VistA modernization effort (known as VistA Evolution), including an interoperability plan and a road map describing functional capabilities to be deployed through fiscal year 2018. According to the road map, the first set of capabilities was to be delivered in September 2014, and was to include access to the Joint Legacy Viewer, among other things.

For its part, DOD issued a request for proposals and developed a series of planning documents for its systems modernization effort (referred to as the Defense Healthcare Management System Modernization (DHMSM) program). Further, the department announced that the DHMSM contract was awarded on July 29, 2015, and that it plans for the new system to reach initial operating capability by December 2016.

The IPO has also taken actions to facilitate departmental interoperability efforts. These included developing technical guidance that details how VA and DOD systems are to exchange information consistent with national and international standards. The office also developed a joint interoperability plan, which summarizes the departments’ actions in this area, and a health data interoperability management plan, which outlines a high-level approach and roles and responsibilities for achieving health data exchange and terminology standardization.

While these are important steps toward greater interoperability, VA and DOD nonetheless did not, by the October 1, 2014, deadline established by the 2014 National Defense Authorization Act for compliance with national data standards, certify that all health care data in their systems complied with national standards and were computable in real time.

Additionally, the departments acknowledged that they do not expect to complete a number of key activities related to their electronic health record system efforts until sometime after the December 31, 2016, statutory deadline for deploying modernized electronic health record software with interoperability. Specifically, deployment of VA’s modernized VistA system at all locations and for all users is not planned until 2018. Meanwhile, DOD has yet to define all the additional work that will be necessary beyond 2016 to fully deploy the DHMSM system, and full operational capability is not planned to occur until the end of fiscal year 2022. Thus, for the departments, establishing modernized and fully interoperable health record systems is still years away.

A significant concern is that VA and DOD had not identified outcome- oriented goals and metrics that would more clearly define what they aim to achieve from their interoperability efforts and the value and benefits these efforts are intended to yield. As we have stressed in prior work and guidance, assessing the performance of a program should include measuring its outcomes in terms of the results of products or services. In this case, such outcomes could include improvements in the quality of health care or clinician satisfaction. Establishing outcome-oriented goals and metrics is essential to determining whether a program is delivering value.

The IPO is responsible for monitoring and reporting on the departments’ progress in achieving interoperability and coordinating with VA and DOD to ensure that these efforts enhance health care services. Toward this end, the office issued guidance that identified a variety of process- oriented metrics to track, for example, the percentage of data domains that have been mapped to national standards. The guidance also identified metrics to be reported that relate to tracking the amount of certain types of data being exchanged between the departments’ existing initiatives, such as laboratory reports exchanged from DOD to VA through the Federal Health Information Exchange and patient queries submitted by providers through the Bidirectional Health Information Exchange.

Nevertheless, as we reported in August 2015, the IPO had yet to specify outcome-oriented metrics and goals that would gauge the impact interoperable health record capabilities will have on the departments’ health care services. The acting director of the IPO stated that the office was working to identify metrics that would be more meaningful, such as metrics on the quality of a user’s experience or improvements in health outcomes. However, the IPO had not established a time-frame for completing such metrics and incorporating them into the office’s guidance.

In our August 2015 report, we stressed that using an effective outcome- based approach could provide DOD and VA with a more accurate picture of their progress toward achieving interoperability and the value and benefits generated. Accordingly, we recommended that the departments, working with the IPO, establish a time frame for identifying outcome- oriented metrics, define related goals as a basis for determining the extent to which the departments’ modernized electronic health records systems are achieving interoperability, and update IPO guidance accordingly. Both departments concurred with our recommendations.

In conclusion, VA and DOD are continuing to pursue their nearly 2 decades-long effort to establish interoperability between their electronic health records systems. Yet while the departments’ various initiatives over the years have increased the amount of patient health data exchanged by the departments and made accessible to providers, these efforts have been beset by persistent management challenges and uncertainty about the extent to which fully interoperable capabilities will be achieved and when. The 2013 decision to pursue separate modernizations, rather than a single, joint system, indicates that achieving interoperability will be an ongoing concern for years to come. Moreover, it has once again highlighted the criticality of these departments needing to define what they aim to accomplish through these efforts, and identify meaningful outcome-oriented goals and metrics that indicate not only the extent to which progress is being made toward achieving full interoperability, but also the measures to which they will be held accountable. As we stressed in our report, establishing measurable goals for improving the care that VA and DOD provide to the millions of service members, veterans, and their beneficiaries is essential to more effectively position the departments to do so.

Chairmen Hurd and Coffman, Ranking Members Kelly and Kuster, and Members of the Subcommittees, this concludes my prepared statement. I would be pleased to respond to any questions that you may have.

Contact and Acknowledgments

If you or your staffs have any questions about this statement, please contact Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov. Additional staff who made key contributions to this statement include Mark T. Bird (assistant director), Lee McCracken, Jacqueline Mai, Scott Pettis, and Jennifer Stavros-Turner.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.