Understanding Meaningful Use with a Focus on Testing the HL7 V2 Messaging Standards

Robert Snelick, National Institute of Standards and Technology (NIST)

Sheryl Taylor, Booz-Allen Hamilton (BAH)

Use of electronic health records (EHRs), especially systems with clinical decision support capabilities, has 

been shown to enable quality improvement in healthcare as well as to help reduce the cost of that care 

when used regularly in the practice of medicine. These facts contributed to the impetus for Congress to

enact the Health Information Technology for Economic and Clinical Health (HITECH) Act, a component of 

the American Recovery and Reinvestment Act (ARRA) of 2009, and which provides funding for incentive 

payments to physicians and hospitals that adopt health information technology (HIT). Initially focusing on 

adoption of EHRs, approximately $17 billion in the Center of Medicare and Medicaid Services (CMS)

incentive payments are available through CMS’s HITECH-based EHR Meaningful Use (MU) Program, to 

be paid to providers that attest to or demonstrate “meaningful use” of “certified” EHR technology 

(CEHRT). In response to this mandate for use of CEHRT, the Office of the National Coordinator (ONC) 

established a certification program and published EHR certification criteria. The National Institute of 

Standards and Technology (NIST) developed test procedures and conformance test tools based on the 

ONC’s EHR certification criteria. This article briefly explains the history and purpose of the ONC 

certification program, and provides insight into the test procedures and testing process for the Health 

Level 7 (HL7) Version 2 (V2) messaging standards. It also presents an overview of the test tools that are 

used by testing laboratories to ensure that vendors’ EHR technologies meet the requirements of CEHRT. 

Adoption and use of CEHRT are required for eligible professionals (EPs) and eligible hospitals (EHs) to 

become “meaningful users” and to receive payments from the CMS EHR Incentive Program. On January 

3, 2011, the ONC released the Final Rule to establish a permanent program for certification of Health IT. 

This “ONC HIT Certification Program” was launched on October 4, 2012, replacing the previous 

temporary program.

As shown in Figure 1, the ONC HIT Certification Program correlates directly to the CMS MU Final Rule as 

well as the ONC Health IT Standards, Implementation Specifications and Certification Criteria Final Rule

(ONC Final Rule). CMS MU requirements are behavioral in scope, specifying how the EPs and EHs must 

use the CEHRT in order to receive the incentive payments. Based on the MU requirements, ONC defines 

the technical criteria CEHRT must meet and manages the development and implementation of 

certification testing procedures.

Thus far, CMS and ONC have published two sets of requirements for MU and for EHR certification known 

as Stage 1 or 2011 Edition and Stage 2 or 2014 Edition. Included in the ONC criteria are interoperability 

standards, such as HL7 V2 implementation guides and vocabulary standards. ONC CEHRT requirements 

do not represent a comprehensive list of all EHR capabilities that a physician’s office or a hospital might 

want or need, but they are a baseline of capabilities that CEHRT must support. Additional MU Stages are 

planned, and the requirements are anticipated to become more challenging, demanding more 

capabilities.
Figure 1: Correlation of CMS MU Final Rule, ONC Final Rule, and ONC HIT Certification Program

The ONC HIT Certification Program manages the testing and certification of EHR technologies by 

Accredited Testing Laboratories (ATLs) and Authorized Certification Bodies (ACBs). “Testing” is the 

process used by an ATL to determine the degree to which the EHR technology meets the specific 

certification criteria. “Certification” is the assessment and affirmation made by an ACB once it has 

analyzed the quantitative results produced by the testing and determined that the EHR technology has 

met all of the applicable certification criteria. ONC reviews the product certification provided by the ACB, 

and then posts it to the online Certified HIT Product List (CHPL), thereby designating the product as a 

CEHRT.

In order to verify that EHR technologies meet the ONC certification criteria, a test method was developed 

for use by the ATLs. The ONC test method is composed of test procedures, test data, and (where 

applicable) conformance test tools. Test procedure documents provide a standard structure for the 

content related to the certification testing steps; the test data work in conjunction with the test procedures 

to further indicate the intent of the ONC policies behind the certification criteria; and the conformance test 

tools work with the test procedures and test data to enable automated and semi-automated validation of 

the EHR technologies’ conformance to the certification criteria and the named interoperability standards.

Conformance Test Tools

The 2014 Edition/Stage 2 ONC EHR certification standards and criteria specify four HL7 V2 

implementation guides that apply to five certification criteria; (1) transmission to immunization registries, 

(2) syndromic surveillance to public health agencies, (3) transmission of reportable lab results to public 

health agencies, (4) transmission of electronic lab results to ambulatory providers, and (5) incorporation 

of lab tests and results. NIST has developed the test procedure and conformance test tool for each of 

these criteria. The links to these test tools are provided along with standards they target:
• Immunization: http://hl7v2-iz-testing.nist.gov/mu-immunization/

o HL7 2.5.1 Implementation Guide for Immunization Messaging, Release 1.4

o HL7 Standard Code Set CVX -- Vaccines Administered

• Syndromic Surveillance: http://hl7v2-ss-testing.nist.gov/mu-syndromic/

o HL7 2.5.1 PHIN Messaging Guide for Syndromic Surveillance

• Reportable Laboratory Results (ELR): http://hl7v2-elr-testing.nist.gov/mu-elr/

o HL7 Version 2.5.1 Electronic Laboratory Reporting to Public Health, Release 1

o SNOMED CT® and Logical Observation Identifiers Names and Codes (LOINC®)

• Laboratory Results Interface (LRI): http://hl7v2-lab-testing.nist.gov/mu-lab/

o HL7 Version 2.5.1 Implementation Guide: S&I Framework Lab Results Interface 

o Logical Observation Identifiers Names and Codes (LOINC®)

o Note the LRI test tool covers both the transmission of laboratory results criteria and the 

incorporation of laboratory results criteria

Central to certification is conformance testing of the technical requirements and capabilities of the EHR 

technology. Testing is facilitated with the use of specific test cases and data. Through consultation and 

collaboration with subject matter experts from the ONC Standards and Interoperability (S&I) Framework, 

the Centers for Disease Control (CDC), the Association of Public Health Laboratories (APHL), and the 

International Society for Disease Surveillance (ISDS), NIST developed test cases that targeted the most 

important use cases and capabilities specified in the referenced standards.

The test data is provided to assist the Tester in verifying that the vendors’ EHR technologies are capable 

of supporting the required functions; verifying the ability to support the specific test data content is a 

secondary aspect to testing. Testing and verification related to specific content usually is more 

appropriate for local installations of the EHR technologies; however, for certain aspects of a certification 

test case, examining exact content is necessary to verify that a capability exists in the EHR technology.

An added benefit of providing realistic test data for common use cases is that it reinforces the expected 

interpretation and use of the referenced standards.

Testing the EHR for Transmitting a Message

When testing the ability to transmit HL7 V2 messages, the focus of the conformance testing strictly 

centers on validating the message produced by the EHR system (the sender). The System Under test 

(SUT), the EHR, is treated as a “black box”—how the message is created or transformed is not in scope. 

If we consider a Laboratory Information System (LIS) or Laboratory module as part of the EHR (hereafter 

lab component), testing is not concerned with the detailed architecture of the lab component, but rather 

what it produces (a message) based on a given set of inputs (i.e., the test case). The “black box” can be a 

self-contained lab component or be composed of multiple modules with a data flow between them, for 

example, a lab component and an integration engine; however, the “black box” must contain a lab 

component, and test case data must be entered into and originate from the lab component.
Figure 2: Context-free Validation Test Flow

The NIST HL7 V2 conformance test tools for validating sending systems have two operational modes:

• Context-free

• Context-based 

The context-free mode validates any message created by the SUT. It is not dependent on any use case, 

test case, or specific test data content. Figure 2 illustrates the context-free test flow.

Figure 3: Context-based Validation Test Flow

The context-based mode validates messages that are associated with a given use case and test case

that includes specific test data that are to be entered into the SUT. The SUT creates a message that 

corresponds to the test data provided in the test case. Testing assesses the technical requirements and 

content-specific requirements specified in the test case. Context-based validation expands the test space, 

enabling more comprehensive testing (e.g., conformance usage constructs such as conditional elements

and required, but may be empty elements). Figure 3 shows the context-based test flow. 

Although the NIST conformance test tools support both modes of validation, MU EHR certification testing 

is context-based. However, the context-free validation functionality provided in the test tool can be used 

to assist certification testing in certain circumstances and provides a useful tool for local installation

testing.
Testing the EHR for Incorporation of a Message

Testing the incorporation of laboratory results presents a set of challenges, since there is no output 

artifact that can be assessed directly. For this criterion the EHR, as the receiving system, is being 

examined for the incorporation of laboratory results and seven display requirements (adopted from CLIA, 

the Clinical Laboratory Improvement Amendments which are regulatory US standards for clinical 

laboratory testing). The MU criterion specifies the Laboratory Results Interface (LRI) implementation 

guide for import of the laboratory results. Conformance testing employs the use of a juror document 

(inspection check list) and a human inspector. The content of the juror document is derived from the test 

case and test message. Figure 4 illustrates the test flow of the NIST conformance test tool for the 

incorporation of laboratory results test procedure.

Figure 4: Test Flow for Testing EHR Receiving Transmission of Lab Results

The test tool provides a test harness that interacts with the EHR system, simulating the functionality of an 

LIS. Test cases are developed for the various use cases described in the LRI implementation guide, and 

for each test case its respective test data are created. Based on the test case and test data, a test 

message and juror document is automatically generated by the test tool. The test message is sent to the 

EHR for incorporation. The ATL Tester uses the juror document to examine the EHR system. The juror 

document is the test case-specific checklist the Tester uses to document the presence or absence of the 

data in the EHR for data elements transmitted to the EHR from the LRI Test Harness. The data elements 

are categorized by how they are verified. For example, some elements are required to be displayed to the 

clinical user on the EHR screen and stored, while other data elements are required to be stored or 

derivable. Not all data elements are subject to incorporation testing since they are not relevant to the 

incorporation of laboratory results (e.g., message processing elements).

Certification Testing: What does it mean?

Certification of EHR technology is a critical step towards achieving interoperability for exchanging 

healthcare information, but it is not the end-all. ONC certification testing for meaningful use focuses on 
assessing the capabilities of the EHR product for a given set of requirements. It is not directed at sitespecific installations. Purchasing a certified EHR system provides a degree of certainty that the buyer has 

obtained a product that meets a level of capabilities established by the ONC. End users of certified EHR 

products will need to configure their products according to their local requirements. Although the ONC 

HIT Certification Program by design does not extend to installation bases, it provides the foundation and 

a shorter pathway to achieving site-specific interoperability. Incremental progress in terms of the EHR 

capabilities and interoperability is the intent as advancement is made through the MU stages.

It is also important to recognize that the ONC certification program targets the EHR technologies and not 

the systems that interact with them. For example, the transmission to immunization registries certification 

criterion and associated tests assesses whether the EHR product can create HL7 V2 messages in 

accordance with the requirements given in the referenced standard. It does not, however, place any 

requirements on the receiving system, in this case the immunization information systems (IIS) used by the 

registries. In order to achieve interoperability the receiving systems must implement the corresponding 

requirements. The Meaningful Use program envisions that these systems will, over time, acquire such 

capabilities due to market forces and the benefits gained (e.g., the public health agencies will want to take 

advantage of the standardized interfaces).

One of NIST’s roles in developing the test tools is to strictly interpret the requirements as written in the 

standards and the ONC EHR certification criteria, and then develop test cases to test to those 

requirements accordingly. This process has led to the discovery of underspecified and ambiguous 

requirements in the standards and gaps in the certification criteria. These discoveries have necessitated 

the collective efforts of numerous clinical subject matter experts and NIST to resolve, making the MU 

program a useful endeavor in perhaps unexpected ways. The findings have been recorded in clarification 

documents that supplement the HL7 V2 implementation guides. NIST, the ONC S&I Framework, the 

CDC, the AHPL, the ISDS, and the HL7 Conformance and Guidance for Implementation and Testing 

(CGIT) Working Group have begun efforts to establish and apply more concise rules for specifying 

conformance and best practices for writing HL7 V2 implementation guides. The result of these efforts will 

improve the HL7 V2 referenced standards for future MU editions.

In summary, meaningful use of EHRs is a key element in improving both quality and cost of healthcare. 

Two stages of the CMS MU and ONC HIT Certification Programs have been initiated, each stage 

representing incremental progress in the improvement of EHR systems’ interoperability and capabilities 

based on the named standards in the ONC certification criteria. NIST develops the HL7 V2 conformance 

test tools in accordance with implementation guide requirements and vocabulary standards mandated by 

the ONC. Knowing that unambiguous, “testable” requirements are critical to development of robust 

certification testing tools, NIST as a neutral participant devotes time and resources to the collaborative 

process of creating the HL7 V2 implementation guides, reconciling the ballot comments for the draft 

versions, and determining content for the various clarification supplements. The ONC-ATLs use the NIST 

test tools to perform automated and semi-automated validation in a consistent manner for certification of 

EHRs based on requirements included in the standards. It cannot be over overemphasized that the 

effectiveness of a certification program is dependent upon a combination of the quality of the underlying 

standards and the thoroughness of the test tools based on those standards.
Certification of EHR technologies is a critical step in achieving interoperability for exchanging healthcare 

information, as it provides a foundation that can be built upon. Much progress has been made and 

significant knowledge has been gained in the first two MU stages. MU Stage 3 promises to build further 

upon this foundation.