Electronic Health Records: Outcome-Oriented Metrics and Goals Needed to Gauge DOD's and VA's Progress in Achieving Interoperability
Released: 2015-08-13T00:00:00

Letter

Background

Historically, patient health information has been scattered across paper records kept by many different caregivers in many different locations, making it difficult for a clinician to access all of a patient’s health information at the time of care. Lacking access to these critical data, a clinician may be challenged in making the most informed decisions on treatment options, potentially putting the patient’s health at risk. The use of technology to electronically collect, store, retrieve, and transfer clinical, administrative, and financial health information has the potential to improve the quality and efficiency of health care.

Electronic health records are particularly crucial for optimizing the health care provided to military personnel and veterans. While in active military status and later as veterans, many DOD and VA personnel, along with their family members, tend to be highly mobile and may have health records residing at multiple medical facilities within and outside the United States.

Each department operates separate electronic health record systems that they rely on to create and manage patient health information. In particular, DOD currently relies on AHLTA, which comprises multiple legacy medical information systems that were developed from commercial software products and customized for specific uses. For example, the Composite Health Care System (CHCS), which was formerly DOD’s primary health information system, is used to capture information related to pharmacy, radiology, and laboratory order management. In addition, the department uses Essentris (also called the Clinical Information System), a commercial health information system customized to support inpatient treatment at military medical facilities.

For its part, VA currently uses its integrated medical information system— VistA—which was developed in-house by VA clinicians and information technology (IT) personnel. The system consists of 104 separate computer applications, including 56 health provider applications; 19 management and financial applications; 8 registration, enrollment, and eligibility applications; 5 health data applications; and 3 information and education applications.

Interoperability: An Overview

The sharing of health information among organizations is especially important because the health care system is highly fragmented, with care and services provided in multiple settings, such as physician offices and hospitals, that may not be able to coordinate patient medical care records. Thus, a means for sharing information among providers, such as between DOD’s and VA’s health care systems, is achieving interoperability.

The Office of the National Coordinator for Health IT (ONC), within the Department of Health and Human Services, has issued draft guidance,describing interoperability as: 1. the ability of systems to exchange electronic health information and 2. the ability to use the electronic health information that has been exchanged from other systems without special effort on the part of the user.

Similarly, the fiscal year 2014 NDAA defines interoperability, as used in the provision governing the departments’ electronic health records, as “the ability of different electronic health records systems or software to meaningfully exchange information in real time and provide useful results to one or more systems.” Thus, in these contexts, interoperability allows patients’ electronic health information to be available from provider to provider, regardless of where the information originated.

Achieving interoperability depends on, among other things, the use of agreed-upon health data standards to ensure that information can be shared and used. If electronic health records conform to interoperability standards, they potentially can be created, managed, and consulted by authorized clinicians and staff across more than one health care organization, thus providing patients and their caregivers the information needed for optimal care. Information that is electronically exchanged from one provider to another must adhere to the same standards in order to be interpreted and used in electronic health records, thereby permitting interoperability. In the health IT field, standards may govern areas ranging from technical issues, such as file types and interchange systems, to content issues, such as medical terminology. On a national level, ONC has been assigned responsibility for identifying health data standards and technical specifications for electronic health record technology and overseeing the certification of this technology.

In addition to exchanging the information, systems must be able to use the information that is exchanged. Thus, electronic health record technology has the potential to improve the quality of care that patients receive and to reduce health care costs, if the technology is used in a way that improves providers’ and patients’ access to critical information. For example, with interoperability, medical providers have the ability to query data from other sources while managing chronically ill patients, regardless of geography or the network on which the data resides.

DOD and VA Have a Long History of Efforts to Achieve Electronic Health Record Interoperability

Since 1998, DOD and VA have relied on a patchwork of initiatives involving their health information systems to exchange information and to increase electronic health record interoperability. These have included initiatives to share viewable data in existing (legacy) systems; link and share computable data between the departments’ updated health data repositories; develop a virtual lifetime electronic health record to enable private sector interoperability; implement IT capabilities for the first joint federal health care center; and jointly develop a single integrated system. Table 1 provides a brief description of the history of these various initiatives.

In addition to the initiatives mentioned in table 1, DOD and VA previously responded to provisions in the fiscal year 2008 NDAA directing the departments to jointly develop and implement fully interoperable electronic health record systems or capabilities in 2009.called for the departments to set up an interagency program office to be a single point of accountability for their efforts to implement these systems or capabilities by the September 30, 2009, deadline. In January 2009, the IPO completed its charter, articulating, among other things, its mission and functions with respect to attaining interoperable electronic health data.

The act also Further, the departments’ Interagency Clinical Informatics Boardestablished the following six interoperability objectives for meeting the departments’ data-sharing needs and facilitating compliance with the fiscal year 2008 NDAA: demonstrate initial network gateway operation, expand questionnaires and self-assessment tools, expand Essentris in DOD, and demonstrate initial document scanning between the departments. refine social history data, share physical exam data, The departments’ officials, including the co-chairs of the group responsible for representing the clinician user community, asserted that they had met the priorities established by the Interagency Clinical Informatics Board and, in conjunction with capabilities previously attained (e.g., the Federal Health Information Exchange and the Bidirectional Health Information Exchange), had met the deadline for achieving full interoperability as required by the act. Nonetheless, in prior reviews, we have identified a number of challenges the departments have faced in managing their efforts in response to the fiscal year 2008 NDAA and to address their common health IT needs.

While these initiatives, collectively, have yielded increased data sharing in various capacities, we previously reported that they nonetheless experienced persistent management challenges and did not result in the fully interoperable electronic health record capabilities that the departments had long sought. We have also noted that the manner in which DOD and VA reported progress toward achieving interoperability lacked results-oriented (i.e., objective, quantifiable, and measurable) goals. Specifically, we noted that the departmental plans lacked associated performance goals and measures that are a necessary basis to provide the departments and their stakeholders with a comprehensive picture to effectively manage their progress toward realizing increased interoperability.

DOD and VA Committed to a Joint System to Address Common Business Needs

In March 2011, the Secretaries of Defense and Veterans Affairs committed the two departments to the development of a new, joint integrated electronic health record (iEHR) system. Further, in May 2012, they announced their goal of implementing the integrated health record across both departments by 2017. According to program documentation, pursuing iEHR was expected to enable the departments to align resources and investments with common business needs and programs, resulting in a platform that would replace the two departments’ separate electronic health record systems with a common system. In addition, because it would involve both departments using the same system, this approach was expected to largely sidestep the challenges they had historically encountered in trying to achieve interoperability between separate systems.

Toward this end, initial development plans called for the single, joint iEHR system to consist of 54 clinical capabilities that would be delivered in six increments between 2014 and 2017, with all existing applications in VistA and AHLTA continuing uninterrupted until full delivery of the new capabilities. The initiative was to deliver several common infrastructure components—an enterprise architecture;user interface; data centers; and system interface and data exchange standards. The system was to be primarily built by purchasing commercially available solutions for joint use, with noncommercial solutions developed or adopted only when a commercial alternative was unavailable.

However, in February 2013, about 2 years after initiating iEHR, the departments’ Secretaries announced changes to their approach— essentially abandoning their effort to develop a single, integrated electronic health record system for both departments. This decision resulted from an assessment of the iEHR program that the Secretaries had requested in December 2012 because of their concerns about the program facing challenges in meeting deadlines, costing too much, and taking too long to deliver capabilities. The IPO reported spending about $564 million on iEHR between October 2011 and June 2013.

Departments Return to Two- System Approach with a Need to Ensure Interoperability

In place of the iEHR initiative, DOD determined that it would buy a commercially available system to replace its existing AHLTA system and VA decided that it would modernize its existing VistA health information system. In this regard, DOD is pursuing the acquisition of a replacement system for its multiple legacy electronic health record systems under a new program—the DHMSM program. For its part, VA intends to enhance and modernize its existing VistA system under a new program called VistA Evolution. The departments indicated that they would ensure interoperability between their two new systems, and with other public and private health care providers.

In December 2013, the IPO was re-chartered and recognized as the single point of accountability in the development and implementation of electronic health records systems or capabilities that allow for full interoperability of health care information between DOD and VA. According to the IPO charter, the office is responsible for, among other things, establishing technical and clinical standards and processes to ensure integration of health data between the two departments and other public and private health care providers. Further, it is to monitor and report on the departments’ progress in implementing the technical standards during the development of their respective systems. It is also to coordinate with the departments to ensure that advances in interoperable capabilities enhance the quality, safety, efficiency, and effectiveness of health care services.

While the departments have chosen their current approach to modernize two separate systems, we have previously reported that they did not substantiate their claims that the current approach would be less expensive and faster than the single-system approach. Further, we have noted that the departments’ efforts to modernize their two separate systems were duplicative. We stressed that major investment decisions—including terminating or significantly restructuring an ongoing program—should be justified using analyses that compare the costs and schedules of alternative proposals.

Accordingly, we recommended that DOD and VA develop a cost and schedule estimate for their current approach from the perspective of both departments that includes the estimated cost and schedule of VA’s VistA Evolution program, DOD’s DHMSM program, and the departments’ joint efforts to achieve interoperability between the two systems, and then compare the cost and schedule estimates of the current and previous (i.e., single-system) approaches and, if applicable, provide a rationale for pursuing a more costly or time-consuming approach. The departments agreed with our recommendation and stated that, while initial comparisons indicated that their current approach would be more cost effective, they would continue to refine cost estimates as part of both departments’ acquisition programs.

DOD, VA, and the IPO Have Taken Steps toward Achieving Interoperability, but All Key Activities Will Not Be Completed until Future Years, and Plans Lack Outcome- Oriented Metrics and Goals

DOD, VA, and the IPO have undertaken various activities aimed at achieving interoperability between the two departments’ electronic health record systems. In this regard, DOD and VA have initiated work focused on near-term objectives including standardizing their existing health data and making them viewable by both departments’ clinicians in an integrated format. The departments have also developed plans related to their efforts to modernize their respective electronic health record systems. Further, the IPO has issued guidance outlining the technical approach for achieving interoperable capabilities between the departments’ systems. Even with the actions taken, however, the two departments did not, by the October 2014 deadline established in the fiscal year 2014 NDAA for compliance with national data standards, certify that all health care data in their systems complied with national standards and were computable in real time. Further, the departments’ system modernization plans identify a number of key activities to be implemented beyond the 2016 deadline established in the act, suggesting that deployment of the new systems with interoperability capabilities will not be completed across the departments until years later. In addition, while the IPO has begun steps to measure and report on the progress of the exchange of health data, its efforts have not included the use of outcome-oriented metrics and established goals essential to gauging the extent to which interoperability is being delivered and having an impact on improving health outcomes.

DOD, VA, and the IPO have Taken Actions to Achieve Interoperability

As part of the approach toward achieving interoperability between their electronic health record systems, both DOD and VA, along with the IPO, have taken actions focused on near-term objectives including standardizing data and expanding the functionality and availability of patient health information. Specifically, both departments have taken actions to:

Analyze data related to 25 health data domains that were identified and prioritized by the Interagency Clinical Informatics Board and map the data in their respective electronic health record systems—AHLTA and VistA—to health data standards identified by the IPO. Such standards include RxNorm and the Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT).

Expand functionality of the Joint Legacy Viewer—a tool that provides a real-time, integrated, categorized, and chronological view of electronic health record information contained in existing DOD and VA systems. For example, it allows both departments to share certain healthcare data (e.g., patient demographics, allergies, medications) in a viewable interface that is available to clinicians. In expanding its functionality, the departments took steps to make more computable (mapped) data from both departments available via the Joint Legacy Viewer and to increase user access to this tool.

Issue plans for their respective electronic health records modernization or acquisition programs consistent with IPO guidance that describes the technical approach to interoperability.

With regard to its specific actions, DOD’s recent work on interoperability has been implemented by the Defense Medical Information Exchange program. This program is focused on increasing the amount of data shared with VA and with the private sector. The program has also been working to make more data viewable through the Joint Legacy Viewer, consolidate DOD data flow through a single exchange mechanism to achieve the near-term interoperability objectives, and prepare for the deployment of the department’s new electronic health record.

According to the program manager, the program has performed the infrastructure testing intended to ensure that DOD and VA have the necessary capacity to accommodate the increasing number of users of the Joint Legacy Viewer. According to DOD documentation, during fiscal years 2015 and 2016, the program’s plans include continuing to enhance the Joint Legacy Viewer by adding additional private sector data and incorporating capabilities for viewing data from existing tools, such as the Bidirectional Health Information Exchange.

DOD also issued a request for proposals for the DHMSM program in August 2014 that described the department’s plans to replace its legacy systems and acquire a modernized electronic health record system with interoperable capabilities across the military operations. The department has developed a series of planning documents, including an acquisition strategy that called for awarding the DHMSM contract by the end of fiscal year 2015. The department noted in a January 2015 briefing to Congress that it plans to reach initial operational capability for the modernized system by December 2016. This is expected to include the deployment of modernized electronic health record software at eight locations. The time frame for reaching full operating capability for the new system is to be determined after contract award.

For its part, VA has developed plans, such as the VA Interoperability Plan and the VistA 4 Roadmap. Both documents describe the department’s approach for modernizing its existing electronic health record system through the VistA Evolution program, while helping to facilitate interoperability with DOD’s system and the private sector. For example, the VA Interoperability Plan, issued in June 2014, describes activities intended to improve VistA’s technical interoperability, such as standardizing the VistA software across the department to simplify sharing data.

In addition, the VistA 4 Roadmap, last revised in February 2015, describes four sets of functional capabilities that are expected to be incrementally deployed during fiscal years 2014 through 2018 to modernize the VistA system and enhance interoperability. According to the road map, the first set of capabilities was delivered by the end of September 2014 and included access to the Joint Legacy Viewer and a foundation for future functionality, such as an enhanced graphical user interface and enterprise messaging infrastructure. Another interoperable capability that is expected to be incrementally delivered over the course of the VistA modernization program is the enterprise health management platform. This platform is intended to provide clinicians with a customizable view of a longitudinal health record that can integrate data from DOD, VA, and third-party providers. Also, when fully deployed, VA expects the enterprise health management platform to replace the Joint Legacy Viewer.

Additionally, with regard to its actions to facilitate department interoperability efforts, the IPO has developed guidance that describes a technical approach for standardizing health data, to include related roles and responsibilities and near-term actions intended to increase interoperability. Specifically,

The IPO’s Healthcare Information Interoperability Technical Package, first issued in July 2014 and subsequently updated twice, describes a standards-based approach and technical objectives for interoperability. This guidance addresses issues such as how the DOD and VA systems are to exchange information consistent with national and international standards. It also identifies the standards that the IPO has selected for the 25 prioritized health data domains.

The Joint Interoperability Plan, most recently updated in January 2015, is characterized by the IPO as a working document to be regularly updated. It summarizes the departments’ actions to increase interoperability, including discussions of completed actions as well as near-term goals and deliverables through 2016, such as improvements to the Joint Legacy Viewer and VistA Evolution capabilities and the DHMSM contract award, among others. In addition, the plan identifies challenges to achieving interoperability, such as the evolving nature of health care data standards and the complexity of integrating interoperable data with clinicians’ current workflows. Further, the plan includes four approved “use cases,” which are scenarios that help describe areas where increased interoperability would be most valuable. The use cases are intended to help identify additional requirements for the development and testing of interoperability capabilities beyond 2016.

The IPO’s Health Data Interoperability Management Plan, issued in September 2014, outlines a high-level approach and roles and responsibilities for achieving electronic health data exchange and terminology standardization for DOD, VA, other government entities, and private sector healthcare partners. The plan also establishes the Health Data Interoperability Standards Lifecycle Model, which describes the process by which data standards are expected to be selected—consistent with national and international health data standards as they evolve and are implemented by the departments.

Overall, the recent actions taken by DOD, VA, and the IPO have focused on ensuring that health care data used by the two departments’ existing systems, AHLTA and VistA, are compliant with national standards. In particular, following the technical approach outlined by the IPO, the departments have increased the amount of data from these systems that are mapped to national standards; they also have made that data available in the Joint Legacy Viewer.

Nonetheless, DOD and VA program officials acknowledged that these actions did not result in the two departments meeting the October 1, 2014, deadline established in section 713 of the fiscal year 2014 NDAA for certifying that all health care data in their systems complied with national standards and were computable in real time. While the departments did provide Congress with an update on their progress, DOD officials stated that the department plans to certify that it has met the requirement in the next several months. VA officials stated that the department plans to certify that it has met the requirement later in calendar year 2015.

Additional Actions Planned Beyond 2016 to Meet Interoperability Requirements

While important actions are being taken, the departments have indicated that they do not intend to complete a number of key activities related to the deployment of their modernized systems and interoperability until after the December 31, 2016, statutory deadline for deploying modernized electronic health record software. DHMSM program officials have acknowledged that additional project details to guide the department’s efforts towards achieving full operating capability for the modernized system have yet to be determined. The officials said that they expect these details to be developed after the contract for the DHMSM service provider integrator and system solution has been awarded. The department also currently estimates full operational capability to occur at the end of fiscal year 2022. Thus, while initial deployment of an interoperable electronic health record system is expected at eight locations by the statutory deadline, additional work beyond 2016, which DOD has yet to fully define, will be required to extend access to the modernized software, provide interoperable capabilities throughout the department, and include all users who would benefit from access.

In addition, deployment of VA’s modernized VistA system at all locations and for all necessary users is not scheduled until 2018. The department plans to deliver functionality in a phased approach in four product releases over 5 years to improve performance and increase interoperable capabilities in additional clinical areas. For example, according to its plans, the department intends to deliver additional features through fiscal year 2018 to improve the functionality of the enterprise health management platform. It also plans to increase interoperable capabilities in additional clinical areas as it replaces its legacy scheduling system that is intended to reduce VA patient wait times. Thus, additional actions beyond the 2016 statutory deadline are still planned for deploying a modernized electronic health record system.

DOD, VA, and the IPO Lack Outcome-Oriented Metrics and Goals for Defining and Measuring Interoperability Progress

Prior work and guidance that we have issued stress the importance of measuring program performance,reporting of accomplishments. This guidance further states that performance measurement should evaluate both processes and outcomes related to program activities. Specifically, process metrics address the type or level of program activities conducted and the direct products or services delivered by a program, such as the number of electronic health records queried in an hour or day. Outcome metrics address the results of products and services, such as improvements on the quality of health care services or clinician satisfaction. Outcome which is the ongoing monitoring and metrics can help in assessing the status of program operations, identifying areas that need improvement, and ensuring accountability for end results. Further, measuring program performance is essential for monitoring progress toward pre-established goals and should be tied to program goals that allow organizations to demonstrate and report the degree to which desired results are achieved.

The IPO’s responsibilities include monitoring and reporting on progress made by the departments to standardize their health care data and coordinating with the departments to ensure that interoperable capabilities enhance health care services. With DOD and VA continuing their activities to increase the sharing of health care data, the IPO has begun taking steps to measure and report on the progress of the two departments’ efforts.

Toward this end, the office has issued guidance describing process metrics that are to be tracked and formally reported to the Health Executive Committee and congressional stakeholders. For example, among these metrics, the Health Data Interoperability Management Plan calls for tracking the percentage of data domains within the departments’ current health information systems that are mapped to selected national standards.

The plan also identifies metrics to be collected and reported that relate to tracking health information exchanges through the departments’ existing initiatives. These metrics include, for example, the number of laboratory reports and the number of consultation reports exchanged from DOD to VA through the Federal Health Information Exchange for separated service members and the number of patient queries by providers from both departments through the Bidirectional Health Information Exchange. The measurements are included in a DOD/VA quarterly data sharing report that the departments prepare and send to Congress. The report is intended to provide a snapshot of the amount of data shared between the departments.

While the IPO has developed process metrics and begun reporting the departments’ progress related to standardizing and exchanging health data, the office has not specified outcome-oriented metrics and established goals that are important to gauging the impact that increased interoperability has on improving health care services. The IPO’s Health Data Interoperability Management Plan indicates that the DHMSM and VistA Evolution programs are to develop outcome metrics related to their respective acquisition and modernization programs. However, the guidance does not identify outcome metrics or establish goals that the IPO expects to use to measure progress toward improving health care services resulting from the departments’ interoperable capabilities.

The IPO Acting Director said that he has tasked a team with working to identify better metrics to capture both the technical and clinical progress resulting from interoperability efforts between the departments. According to the official, this team is working with DOD, VA, and ONC subject matter experts to identify metrics that would be more meaningful for determining the impact of increased interoperability, such as metrics on the quality of a user’s experience and improvements in health outcomes.

However, as of late May 2015, the IPO had not established a time frame for when the metrics would be completed and incorporated into their guidance. Officials of the departments and the IPO explained that defining appropriate outcome metrics for interoperability is not just a DOD and VA issue; rather, it is a national challenge to identify how to measure interoperability and what data are needed.

Using an effective outcome-based metrics approach could provide DOD and VA a more accurate, ongoing picture of their progress toward achieving interoperability and the value and benefits generated. Doing so would also better position them to assess and report on the status of interoperability-related activities in terms of results, and to determine areas that need improvement. Until they establish a time frame, complete steps to define outcome metrics and goals, and incorporate these into IPO guidance, the departments and the IPO risk not knowing the status of program operations and areas that need improvement, and ensuring accountability for end results.

Conclusions

DOD and VA, with guidance from the IPO, have taken actions to increase interoperability between their electronic health record systems, as called for in the fiscal year 2014 NDAA. However, the departments have indicated that they do not intend to complete a number of key activities related to the deployment of their modernized systems and interoperability until after the December 31, 2016, statutory deadline for deploying modernized electronic health record software. To address the 2016 requirement, DOD has issued plans and announced the award of a contract for acquiring a modernized system to include interoperability capabilities across military operations. VA, for its part, has issued plans describing an incremental approach to modernizing its existing electronic health records system. However, these plans—if implemented as currently described—show that interoperability delivered by the new systems is not expected to be completely deployed until after 2018, which is beyond the statutory deadline. To date, the departments have kept Congress informed of their efforts, and we believe it is critical that they continue to do so.

Further, the IPO has taken steps to develop guidance that includes process-oriented metrics for monitoring and reporting on the increasing exchange of health information between the departments. However, it has yet to develop outcome-oriented metrics that are important to gauging the impact that increased interoperability has on improving health care services. While IPO officials have said that a team has been tasked to identify metrics that would be more meaningful for determining the impact of increased interoperability, no time frame has been identified for when this team will report its results and when the IPO plans to incorporate these metrics into its guidance. Further, the office has yet to identify goals that can be used to indicate the status of interoperability-related activities and the extent to which progress is being made to achieve full interoperability of health care information by the departments. Without defining outcome-oriented metrics and related goals and incorporating these into the current approach, the departments and the IPO will not be positioned to assess and report on the status of interoperability-related activities and determine areas that need improvement.

Recommendations for Executive Action

To facilitate oversight and inform decision making regarding their respective department’s interoperability-related activities, we recommend that the Secretaries of Defense and Veterans Affairs, working with the Interagency Program Office, take the following three actions: establish a time frame for identifying outcome-oriented metrics, ensure related goals are defined to provide a basis for assessing and reporting on the status of interoperability-related activities and the extent to which interoperability is being achieved by the departments’ modernized electronic health record systems, and update IPO guidance to reflect the metrics and goals identified.

Agency Comments and Our Evaluation

We provided a draft of this report to VA and DOD and received written comments, which are reprinted in appendixes II and III, respectively. In addition, VA provided technical comments, which we incorporated, as appropriate.

In its comments, VA generally agreed with our conclusions and concurred with our recommendations. With regard to our recommendation to establish a time frame for identifying outcome-oriented metrics, the department described recent actions it has taken toward the development of interoperability milestones and metrics, which are intended to serve as a blueprint for the IPO’s efforts to synchronize outcome-oriented metrics between DOD and VA. In addition, VA noted that it has begun to develop standardized metrics related to VistA Evolution that are tied to desired business goals. VA also described its collaborative efforts with DOD and the IPO to mature interoperability metrics into more meaningful, outcome- oriented metrics and to establish timelines for formal reporting through IPO guidance and data sharing reports.

In its comments, DOD also concurred with our recommendations and stated that the department continues to work with the IPO and VA stakeholders to develop baseline interoperability metrics. The department added that it plans to meet with the IPO and VA on a regular basis to mature these metrics into more meaningful, outcome-oriented measures.

Nevertheless, DOD took issue with selected aspects of our discussion related to requirements in section 713 of the fiscal year 2014 NDAA. For example, the department contended that its limited initial deployment of the new system, combined with the pending documentation of interoperability, will satisfy the statutory requirement to “deploy modernized electronic health record software supporting clinicians of the departments by no later than December 31, 2016, while ensuring continued support and compatibility with the interoperability platform and full standards-based interoperability.” We disagree with DOD’s position and reaffirm our finding that DOD’s and VA’s plans—if implemented as currently described—indicate that deployment of the new systems will not be completed across the departments until after 2016 and that much additional work is needed to extend access to the modernized software to all relevant users and department locations.

The history and framework of statutory requirements has long called for the departments to take steps to achieve interoperable health record capabilities. In this regard, the fiscal year 2008 NDAA included a mandate to achieve fully interoperable health record capabilities. Further, in section 713 of the fiscal year 2014 NDAA, Congress stated that the departments “…have failed to implement a solution that allows for seamless electronic sharing of medical health care data…” and that “most of the information shared…is not standardized or available in real time to support all clinical decisions.” We recognize that section 713 of the fiscal year 2014 NDAA does not qualify the degree of deployment. Nevertheless, we believe it is reasonable to expect that electronic health records interoperability between the departments should be demonstrated by more than a limited number of users and locations and, more importantly, should be made available to all relevant clinicians at all relevant locations as expeditiously as possible.

DOD also disagreed that the fiscal year 2014 NDAA specified a date by which certification of compliance with existing national data standards was required and stated that the timetable for certification is distinct from the October 1, 2014 deadline for compliance with national standards. We assert that the October 1, 2014 deadline in section 713(g)(1) of the fiscal year 2014 NDAA is linked to the departments’ certification that is required in section 713(g)(2). As DOD points out, the certification requirement depends on achieving the capability described in section 713(b)(1). This capability is to be “interoperable with an integrated display of data…by complying with the national standards…”. The national standards referred to here are those national data standards described in section 713(g)’s certification provision. Section 713(g)(2) requires a certification that DOD and VA have complied with data standards referred to in section 713(g)(1). These data standards are described in section 713(g)(1) as the existing national data standards to which all health data in the DOD and VA systems must comply with by October 1, 2014. DOD and VA’s required certification involves certification of compliance with existing national data standards and that compliance was required by October 1, 2014. Thus, we stand by our statement in the report on this matter.

Further, the department expressed concern about our use of the terms “full” deployment and “enhanced” interoperability, stating that these terms are not cited in the act. We agree that section 713 of the fiscal year 2014 NDAA does not use the term “full” deployment and we have revised our report accordingly to remove references to this term. On the other hand, “interoperability enhancements” is a term that VA has used when describing features to be delivered throughout its phased approach to the VistA Evolution modernization. We used the term “enhanced” interoperability to describe the improvements planned as the modernized systems mature beyond October 1, 2014. However, to resolve any ambiguity, we have revised our report to remove the term.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, the Secretary of Defense, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have questions about this report, please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IV.

Appendix I: Objective, Scope, and Methodology

Our objective was to evaluate the actions taken by the Department of Defense (DOD), the Department of Veterans Affairs (VA), and the Interagency Program Office (IPO) to plan and measure the progress toward achieving interoperability between the departments’ electronic health record systems.

To evaluate the actions taken with regard to planning for electronic health record interoperability between the departments’ systems, we reviewed our previous work related to electronic health records and DOD and VA efforts to develop health information systems, interoperable health records, and interoperability standards to be implemented in federal health care programs. We obtained and analyzed DOD, VA, and IPO documentation to evaluate the departments’ plans and identify recent actions taken toward achieving interoperability consistent with the requirements specified in the National Defense Authorization Act (NDAA) for Fiscal Year 2014. Specifically, we analyzed DOD, VA, and IPO plans to evaluate how they relate to NDAA requirements that direct the DOD and VA to each (1) ensure that all health care data contained in DOD’s Armed Forces Health Longitudinal Technology Application (AHLTA) and VA’s Veterans Health Information Systems and Technology Architecture (VistA) systems complied with national standards and were computable in real time by October 1, 2014, and (2) deploy modernized electronic health record software supporting clinicians by no later than December 31, 2016, while ensuring continued support and compatibility with the interoperability platform and full standards-based interoperability.

Further, we reviewed and analyzed DOD’s request for proposals, the Defense Healthcare Management System Modernization Program Acquisition Strategy, VA’s VistA 4 Roadmap, the VistA Evolution Program Plan, VA’s Interoperability Plan, and the IPO guidance including the Healthcare Information Interoperability Technical Package, the Health Data Information Management Plan, and the Joint Interoperability Plan. Further, we identified what the departments plan to deliver by the 2014 and 2016 deadlines and compared planned activities to the statutory requirements.

To evaluate the actions taken by DOD, VA, and IPO to measure the progress toward achieving interoperability between the departments’ electronic health record systems, we reviewed the December 2013 IPO Charter. Our review determined that DOD and VA have assigned the IPO responsibility for, among other things, monitoring and reporting on the progress of the departments’ use of national and international health data standards; compliance with the implementation of IPO’s technical standards; and coordinating and communicating with the departments to ensure advances in interoperability capabilities enhance the quality, safety, efficiency, and effectiveness of health care services. Accordingly, we reviewed IPO documentation, such as the Healthcare Information Interoperability Technical Package, the Health Data Interoperability Management Plan, the Joint Interoperability Plan, the IPO Executive Committee quarterly reports, and DOD/VA quarterly data sharing reports to identify performance measures related to measuring and reporting progress toward achieving interoperability. We compared the performance measures identified in program documentation and reported to Congress on our guidance related to process and outcome-oriented metrics and goals reported in our prior work.

We supplemented our analyses with interviews of DOD, VA, and IPO officials with knowledge of the interoperability efforts, including the IPO Acting Director (also the current Program Executive Officer for the Defense Healthcare Management Systems), IPO Deputy Director, DOD officials from the Defense Medical Information Exchange program, VA officials from the Office of Information and Technology and the Veterans Health Administration with knowledge of the VistA Evolution program, and members of the Interagency Clinical Informatics Board.

We conducted this performance audit from September 2014 to August 2015, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.

Appendix II: Comments from the Department of Veterans Affairs

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Mark Bird (Assistant Director), Neelaxi Lakhmani (Assistant Director), Kami Brown, Nancy Glover, Jennifer Stavros-Turner, and Marshall Williams, Jr., made key contributions to this report.