Electronic Health Records: VA and DOD Need to Establish Goals and Metrics for Their Interoperability Efforts
Published: 2015-10-27T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: VA and DOD operate two of the nation's largest health care systems, serving millions of veterans and active duty members and their beneficiaries. For almost two decades the departments have undertaken various efforts to advance interoperability between their respective electronic health record systems. While the departments have made progress, these initiatives have also faced significant management challenges. Among their recent initiatives, the secretaries of the two departments committed to establishing interoperability between their separate electronic health record systems, which they are working to modernize.
This statement summarizes GAO's August 2015 report (GAO-15-530) on VA and DOD's efforts to achieve interoperability between their health records systems.
section_title: What GAO Found
paragraphs: As GAO reported in August 2015, the Departments of Veterans Affairs (VA) and Defense (DOD), with guidance from the Interagency Program Office (IPO) tasked with facilitating the departments' efforts to share health information, have taken actions to increase interoperability between their existing electronic health record systems. These actions have included work on near-term objectives such as standardizing certain health data and making them viewable by clinicians in both departments in an integrated format. The departments also developed plans for their longer-term initiatives to modernize their respective electronic health record systems. In accordance with its responsibilities, the IPO issued guidance outlining the technical approach for achieving interoperability between the departments' systems.
Having taken these actions, however, the departments did not, by the October 1, 2014, deadline established in the National Defense Authorization Act (NDAA) for Fiscal Year 2014 for compliance with national standards, certify that all health care data in their systems complied with national data standards and were computable in real time. Moreover, the departments do not plan to complete the modernization of their electronic health record systems until well after the December 2016 statutory deadline by which they are to deploy modernized electronic health record software while ensuring full interoperability. Specifically, VA plans to modernize its existing system, while DOD plans to acquire a new system; but their plans indicate that deployment of the new systems with interoperable capabilities will not be complete until after 2018.
Consistent with its responsibilities, the IPO took steps to begin developing metrics to monitor progress related to the standardization of the departments' data and their exchange of health information. For example, it called for the development of tracking metrics to gauge the percentage of data domains within the departments' current systems that have been mapped to national standards. However, the office had not defined outcome-oriented metrics and related goals to measure the effectiveness of interoperability efforts in terms of improving health care services for patients served by both departments. IPO officials said that work was ongoing to develop more meaningful measures of progress, but the office had not established a time frame for completing this work or incorporating the outcome metrics and associated goals into its guidance. GAO concluded that without defining outcome-oriented metrics and related goals and incorporating them into their current approach, the departments and the IPO will not be in a position to effectively assess their progress toward further achieving interoperability and identifying the benefits that their efforts yield.
section_title: What GAO Recommends
paragraphs: In its August 2015 report, GAO recommended that VA and DOD, working with the IPO, establish a time frame for identifying outcome-oriented metrics, define goals to provide a basis for assessing and reporting on the status of interoperability, and update the IPO's guidance accordingly. The departments concurred with GAO's recommendations.

section_title: Letter
paragraphs: Thank you for inviting me to testify at today’s hearing on the Department  of Veterans Affairs’ (VA) and the Department of Defense’s (DOD)  progress toward developing interoperable electronic health records. As  you know, these departments operate two of the nation’s largest health  care systems, serving millions of veterans, active duty service members,  and their beneficiaries. For nearly two decades, the departments have  been working on efforts to better share data in their health care systems  and make patient information more readily available to health care  providers. Yet, while the departments have undertaken numerous  initiatives, they have faced significant challenges and their progress has  been slow.
In August 2015, we issued a report summarizing the findings from our  most recent review of VA’s and DOD’s electronic health record efforts.  The objective of that review was to evaluate the actions taken by the  departments, and the Interagency Program Office (IPO) tasked with  facilitating their efforts, to plan for and measure progress toward  achieving interoperability between the VA and DOD electronic health  record systems. My statement today summarizes the findings of our  report.
For the August 2015 report, we obtained and reviewed relevant program  documentation, such as the IPO Health Data Interoperability  Management Plan, to identify planned metrics tracked and reported by  the IPO. In addition, we reviewed reports, such as the IPO Executive  Committee quarterly reports and the DOD/VA quarterly data sharing  reports to Congress, to determine if metrics and goals related to  interoperability were consistent with GAO guidance that discussed  process and outcome metrics used for performance measurement.  Further, we interviewed cognizant officials, such as the IPO Acting  Director, Deputy Director, and other VA and DOD program officials to  understand efforts related to improving interoperability between the  departments. The report on which this testimony is based includes a more  detailed discussion of the scope and methodology for our review.
The work on which this testimony is based was conducted in accordance  with generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.
subsections: section_title: Background
paragraphs: Historically, patient health information has been scattered across paper  records kept by different caregivers in many different locations. Thus, the  move toward collecting, storing, retrieving, and transferring these records  electronically can significantly improve the quality and efficiency of care.  This is especially true in the case of military personnel and veterans,  because they tend to be highly mobile and may have health records at  multiple facilities both within and outside the United States.
Interoperability allows patients’ electronic health information to be  available from provider to provider, regardless of where it originated.  Achieving this depends on, among other things, the use of agreed-upon  health data standards (e.g., standardized language for prescriptions and  laboratory testing) and the ability of systems to use the information that is  exchanged.
Currently, both VA and DOD operate separate electronic systems to  create and manage electronic health records. VA uses its Veterans  Health Information Systems and Technology Architecture (VistA), a  system that the department developed in-house and that consists of 104  separate computer applications; while DOD uses the Armed Forces  Health Longitudinal Technology Application (AHLTA), which consists of  multiple legacy medical information systems developed from customized  commercial software applications.
subsections: section_title: A Long History of Efforts to Achieve Electronic Health Record Interoperability
paragraphs: Since 1998, VA and DOD have undertaken a patchwork of initiatives  intended to allow their health information systems to exchange  information and increase interoperability. Among others, these have  included initiatives to share viewable data in existing (legacy) systems,  link and share computable data between the departments’ updated heath  data repositories, and jointly develop a single integrated system. Table 1  below summarizes a number of the departments’ key efforts.
In addition to the initiatives mentioned in table 1, the departments took a  variety of actions to respond to provisions in the National Defense  Authorization Act (NDAA) for Fiscal Year 2008, which required them to  jointly develop and implement fully interoperable electronic health record  systems or capabilities in 2009. The act also directed them to set up an  interagency program office (referred to as the IPO) to serve as a single  point of accountability for these efforts.
Department officials stated that their previous initiatives, along with  meeting six interoperability objectives established by their Interagency  Clinical Informatics Board, had enabled them to meet the deadline for full  interoperability established by the act. However, we previously identified a  number of challenges that the departments faced in managing their  efforts in response to the act and to address their common health IT  needs. In particular, although these initiatives have helped to increase  data-sharing in various ways, they have been plagued by persistent  management challenges that have hampered progress toward fully  interoperable electronic health record capabilities.
In March 2011, the secretaries of the two departments announced that  they would develop a new, joint integrated electronic health record  system (referred to as iEHR). This was intended to replace the  departments’ separate systems with a single common system, thus  sidestepping many of the challenges they had previously encountered in  trying to achieve interoperability. However, in February 2013, about 2  years after initiating iEHR, the secretaries announced that the  departments were abandoning plans to develop a joint system, due to  concerns about the program’s cost, schedule, and ability to meet  deadlines. The IPO reported spending about $564 million on iEHR  between October 2011 and June 2013.
In place of the iEHR initiative, VA stated that it would modernize VistA,  while DOD planned to buy a commercially available system to replace  AHLTA. The departments stated that they would ensure interoperability  between these updated systems, as well as with other public and private  health care providers. In December 2013, the IPO was re-chartered and  given responsibility for establishing technical and clinical standards and  processes to ensure that health data between the two departments (and  other providers) are integrated.
We issued several prior reports regarding this approach, in which we  noted that the departments did not substantiate their claims that it would  be less expensive and faster than developing a single, joint system. We  also noted that the departments’ plans to modernize their two separate  systems were duplicative and stressed that their decisions should be  justified by comparing the costs and schedules of alternate approaches.  We therefore previously recommended that the departments develop cost  and schedule estimates that would include all elements of their approach  (i.e., modernizing both departments’ health information systems and  establishing interoperability between them) and compare them with  estimates of the cost and schedule for the single-system approach. If the  planned approach was projected to cost more or take longer, we  recommended that they provide a rationale for pursuing such an  approach.
VA and DOD agreed with our prior recommendations and stated that  initial comparison indicated that the current approach would be more cost  effective. However, as of October 2015, the departments have not  provided us with a comparison of the estimated costs of their current and  previous approaches. On the other hand, with respect to their assertions  that separate systems could be achieved faster, both departments have  developed schedules that indicate their separate modernizations are not  expected to be completed until after the 2017 planned completion date for  the previous single system approach.
In light of the departments’ not having yet implemented a solution that  allows for seamless electronic sharing of health care data, the National  Defense Authorization Act (NDAA) for Fiscal Year 2014 included  requirements pertaining to the implementation, design, and planning for  interoperability between VA’s and DOD’s electronic health record  systems. Among other actions, provisions in the act directed each  department to (1) ensure that all health care data contained in their  systems (VA’s VistA and DOD’s AHLTA) complied with national  standards and were computable in real time by October 1, 2014, and (2)  deploy modernized electronic health record software to support clinicians  while ensuring full standards-based interoperability by December 31,  2016.
section_title: Interoperability Efforts Continue, but DOD and VA Need to Develop Goals and Metrics for Assessing Their Progress
paragraphs: Our August 2015 report noted that the departments have engaged in  several near-term efforts focused on expanding interoperability between  their existing electronic health record systems. For example, the  departments analyzed data related to 25 “domains” identified by the  Interagency Clinical Informatics Board and mapped health data in their  existing systems to standards identified by the IPO. The departments also  expanded the functionality of their Joint Legacy Viewer—a tool that allows  clinicians to view certain health care data from both departments in a  single interface.
In addition, VA and DOD have both moved forward with plans to  modernize their respective electronic health record systems. VA has  developed a number of plans for its VistA modernization effort (known as  VistA Evolution), including an interoperability plan and a road map  describing functional capabilities to be deployed through fiscal year 2018.  According to the road map, the first set of capabilities was to be delivered  in September 2014, and was to include access to the Joint Legacy  Viewer, among other things.
For its part, DOD issued a request for proposals and developed a series  of planning documents for its systems modernization effort (referred to as  the Defense Healthcare Management System Modernization (DHMSM)  program). Further, the department announced that the DHMSM contract  was awarded on July 29, 2015, and that it plans for the new system to  reach initial operating capability by December 2016.
The IPO has also taken actions to facilitate departmental interoperability  efforts. These included developing technical guidance that details how VA  and DOD systems are to exchange information consistent with national  and international standards. The office also developed a joint  interoperability plan, which summarizes the departments’ actions in this  area, and a health data interoperability management plan, which outlines  a high-level approach and roles and responsibilities for achieving health  data exchange and terminology standardization.
While these are important steps toward greater interoperability, VA and  DOD nonetheless did not, by the October 1, 2014, deadline established  by the 2014 National Defense Authorization Act for compliance with  national data standards, certify that all health care data in their systems  complied with national standards and were computable in real time.
Additionally, the departments acknowledged that they do not expect to  complete a number of key activities related to their electronic health  record system efforts until sometime after the December 31, 2016,  statutory deadline for deploying modernized electronic health record  software with interoperability. Specifically, deployment of VA’s  modernized VistA system at all locations and for all users is not planned  until 2018. Meanwhile, DOD has yet to define all the additional work that  will be necessary beyond 2016 to fully deploy the DHMSM system, and  full operational capability is not planned to occur until the end of fiscal  year 2022. Thus, for the departments, establishing modernized and fully  interoperable health record systems is still years away.
A significant concern is that VA and DOD had not identified outcome- oriented goals and metrics that would more clearly define what they aim  to achieve from their interoperability efforts and the value and benefits  these efforts are intended to yield. As we have stressed in prior work and  guidance, assessing the performance of a program should include  measuring its outcomes in terms of the results of products or services. In  this case, such outcomes could include improvements in the quality of  health care or clinician satisfaction. Establishing outcome-oriented goals  and metrics is essential to determining whether a program is delivering  value.
The IPO is responsible for monitoring and reporting on the departments’  progress in achieving interoperability and coordinating with VA and DOD  to ensure that these efforts enhance health care services. Toward this  end, the office issued guidance that identified a variety of process- oriented metrics to track, for example, the percentage of data domains  that have been mapped to national standards. The guidance also  identified metrics to be reported that relate to tracking the amount of  certain types of data being exchanged between the departments’ existing  initiatives, such as laboratory reports exchanged from DOD to VA through  the Federal Health Information Exchange and patient queries submitted  by providers through the Bidirectional Health Information Exchange.
Nevertheless, as we reported in August 2015, the IPO had yet to specify  outcome-oriented metrics and goals that would gauge the impact  interoperable health record capabilities will have on the departments’  health care services. The acting director of the IPO stated that the office  was working to identify metrics that would be more meaningful, such as  metrics on the quality of a user’s experience or improvements in health  outcomes. However, the IPO had not established a time-frame for  completing such metrics and incorporating them into the office’s  guidance.
In our August 2015 report, we stressed that using an effective outcome- based approach could provide DOD and VA with a more accurate picture  of their progress toward achieving interoperability and the value and  benefits generated. Accordingly, we recommended that the departments,  working with the IPO, establish a time frame for identifying outcome- oriented metrics, define related goals as a basis for determining the  extent to which the departments’ modernized electronic health records  systems are achieving interoperability, and update IPO guidance  accordingly. Both departments concurred with our recommendations.
In conclusion, VA and DOD are continuing to pursue their nearly 2  decades-long effort to establish interoperability between their electronic  health records systems. Yet while the departments’ various initiatives  over the years have increased the amount of patient health data  exchanged by the departments and made accessible to providers, these  efforts have been beset by persistent management challenges and  uncertainty about the extent to which fully interoperable capabilities will  be achieved and when. The 2013 decision to pursue separate  modernizations, rather than a single, joint system, indicates that achieving  interoperability will be an ongoing concern for years to come. Moreover, it  has once again highlighted the criticality of these departments needing to  define what they aim to accomplish through these efforts, and identify  meaningful outcome-oriented goals and metrics that indicate not only the  extent to which progress is being made toward achieving full  interoperability, but also the measures to which they will be held  accountable. As we stressed in our report, establishing measurable goals  for improving the care that VA and DOD provide to the millions of service  members, veterans, and their beneficiaries is essential to more effectively  position the departments to do so.
Chairmen Hurd and Coffman, Ranking Members Kelly and Kuster, and  Members of the Subcommittees, this concludes my prepared statement. I  would be pleased to respond to any questions that you may have.
section_title: Contact and Acknowledgments
paragraphs: If you or your staffs have any questions about this statement, please  contact Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov.  Additional staff who made key contributions to this statement include  Mark T. Bird (assistant director), Lee McCracken, Jacqueline Mai, Scott  Pettis, and Jennifer Stavros-Turner.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.