Software and Systems Division

IHE-PCD , HL7 Dev WG,

ISO/IEEE 11073, 

and

NIST

NIST MDC Test Tooling

IHE-PCD/HL7/IEEE WG Meetings

(@ Phoenix, Az)

19 January 2010
Software and Systems Division

Medical Device CommunicationTest Effort

NIST Contacts

• John Garguilo (john.garguilo@nist.gov, 

301-975-5248)

• Sandra Martinez (sandra.martinez@nist.gov, 

301-975-3579)

NIST Tool Sites

Pre-Connectathon Web site:

http://xreg2.nist.gov:8080/PCD-HL7Web/

Connectathon Web site: 

http://xreg2.nist.gov:8080/PCD-HL7WebCon/

MDC Testing Web site: www.nist.gov/medicaldevices
Software and Systems Division

Topics

• HL7 Tooling Status (John)

– NIST‟s HL7 V2 Tools

– Supported IHE-PCD Cycle 4 Pre- & VirtualConnectathons, and Connectathon

• ISO/IEEE 11073 Tooling Status (Sandra)

– ICSGenerator Interface Update

– Discussion (Cardinality)

• HIT Testing Infrastructure - Coordination with IHE 

“Gazelle Project” (Rob)

• Discussion – Testing Strategy
Software and Systems Division

IHE-PCD Testing – Key Objectives

• Increase test comprehensiveness & quality 

• Support both conformance & interoperability 

testing

• Support for Pre- & Virtual- Connectathons, actual 

Connectathon & enable year round testing

• Remain in alignment with IHE-PCD integration 

profile development road map

• Establish single framework for PCD covering 

increasing complexity and technologies over next 5 

years

• Coordinate with IHE “Gazelle Project”

• Generate work products that companies can use in 

their regulatory submissions
Software and Systems Division

Test Environments

• Instance Testing

– Conformance (e.g., against HL7 2.x or CDA)

• Implementation conforms to Spec. on which it is based

• IHE Model: ~Virtual and Pre-Connectathon

• Isolated System Testing

– Includes Instance Testing Activities

– Protocol Conformance 

– Functional Behavior Conformance

• Features and Operational behavior correspond to Specs.

• IHE Model: ~Virtual and Pre-Connectathon

• Peer-to-Peer System Testing

– Includes Isolated System Testing Activities

– Interoperability Testing

• Testing complete application environment 

• May include interacting w/ Database, using Network 

Communications, or interacting w/ other hardware, apps, or 

systems if appropriate

• IHE Model: ~Connectathon
Software and Systems Division

IHE-PCD Testing

IHE-PCD Testing – Key Ingredients

• Well Defined Integration Profiles

– Technical Framework

– Supplements

• Test Guidelines

• Test Plans

– Test Scenarios, Actors, Transactions, Validation Criteria

• Test Artifacts, including:

– HL7 Profile(s) (and eventually x73 Device Specializations?) 

– Repositories

– Nomenclature (e.g., RTM)

– Value Tables (e.g., HL7, units, local, etc.), Default and Sample Values

• Test Cases
Software and Systems Division

NIST IHE-PCD Test Guidelines

(Cycle 4 ‟09-‟10 Pre-connectathon)

• Quick, executive summary level view

• Defines testing requirements for those performing testing

– HL7 Messaging version (e.g., V2.6)

– HL7 profile 

• Identifies test environment

– See „Test Environment‟ slides below...

• Identifies what types of testing shall occur

• Links test artifact(s) (needed) to validate specific test case(s)

• Syntax Validation

– HL7 Syntax/structure – basic syntax via HL7 V2.6 standard

– HL7 IHE-PCD Profile Validation – required fields, order, data types, 

cardinality via message profile

• Semantic Validation(via context validation file)

– Integration Profile Validation – (semantic) via message profile file (XML)

– Rosetta Validation – (semantic) terminology w/ co-constraints from RTM 

Integration Profile

– Test case validation – (semantic) specific field values
Software and Systems Division

NIST IHE-PCD Test Plans

(‟09-‟10 Pre- and Virtual- Connectathon)

• Clear and concise documentation

• Mirror (point to) and/or include test cases 

– Already defined and new test cases as developed/introduced

• Necessary for rigorous HL7 profile and message validation

– Defines “validation context” specifications used by tooling

• Recommended and used document template...

– Used by IHE-PCD Integration Profile working groups to establish 

exactly what‟s to be tested

• Communicates specific validation criteria to participating vendors

– I.e., no surprises

• ...Supporting documentation for work products

– ...that perhaps companies can use in their regulatory 

submissions

• Start w/ static messages and move into scenarios

– Way to identify appropriate Rosetta Terminology according to 

IHE use case
Software and Systems Division

IHE-PCD Cycle 4 Testing

Documentation

Test Guideline Documents

• NIST DEC Pre-Connectathon Test Guidelines.docx

• NIST ACM Pre-Connectathon Test Guidelines.docx

• NIST IDCO Pre-Connectathon Test Guidelines.docx

• NIST PIV Pre-Connectathon Test Guidelines.docx

Test Plan Documents

• DEC_TEST PLAN.docx

• DEC_SPD_TEST PLAN.docx

• ACM_TEST PLAN.docx

• IDCO_TEST PLAN.docx

• PIV_TEST PLAN.docx

• Guidelines/Test Plans and Validation Context Files (XML) 

available at IHE Patient Care Devices‟ FTP site:

– ftp://ftp.ihe.net/Patient_Care_Devices/Tools/NISTTools/NIST_HL7_V2_Tools/
Software and Systems Division

NIST IHE-PCD HL7 Profiles (cycle 4)

• NIST_DEC_DOR_PCD-01_ORU^R01.xml

• NIST_SPD_DOC_PCD-02_QSB^Q16.xml

• NIST_PIV_IOP_PCD-03_RGV^O15.xml

• NIST_ACM_AR_PCD-04_ORU^R01.xml

• NIST_IDCO_IDCR_PCD-09_ORU^R01.xml

• Key: Msg_type^Event_type^ e.g., ORU^R01^...)
Software and Systems Division

Message Validation Context

Technical Framework/Supplement Validation

• Naming Convention: w...w-yyy-nn-ValidationContext.xml

– [w...w]: Integration Profile Group/Option 

(e.g., “PIV”)

– yyy-nn: IHE-PCD transaction-Number 

(e.g., “PCD-03”)

• DEC-PCD-01-ValidationContext.xml

• SPD-PCD-02-ValidationContext.xml

• PIV-PCD-03-ValidationContext.xml

• ACM-PCD-04-ValidationContext.xml

• IDCO-PCD-09-ValidationContext.xml
Software and Systems Division

Message Validation Context

hRTM Validation

• hRTM

– Integrated via context validation files for this year‟s 

cycle

– Future, likely a web service (call to repository)

• hDEC-PCD-01-ValidationContext.xml

• hPIV-PCD-03-ValidationContext.xml

• hACM-PCD-04-ValidationContext.xml

• hIDCO-PCD-09ValidationContext.xml
Software and Systems Division

NIST Test Case

Message Validation Context (‟09-‟10 Pre-connectathon)

• NIST validation of specific values 

(derived from the test cases for each Integration Profile)

– Based on Use Case

– Instance of Use Case ultimately demonstrated at Showcase

• E.g., PIV(for PCD-03)

– 60101-ValidationContext.xml

– 60102-ValidationContext.xml

– 60103-ValidationContext.xml

– 60104-ValidationContext.xml

– 60105-ValidationContext.xml

– 60106-ValidationContext.xml

– 60107-ValidationContext.xml

– 60108-ValidationContext.xml

– 60109-1-ValidationContext.xml

– 60109-2-ValidationContext.xml

– 60109-3-ValidationContext.xml

– 60109-4-ValidationContext.xml

– 60110-ValidationContext.xml
Software and Systems Division

Test Environment Message Validation

NIST V2 Testing Tools: IHE-PCD

• Validation of IHE-PCD message(s) and 

corresponding HL7 *Profile(s)

* Profiles developed by NIST to control testing – relieve 

vendors of burden 

• Syntax and Semantic Content Validation

– Against HL7 V2 message (e.g., PCD-01)

• Message structure (e.g., 

MSH,PID,PV1,OBR,NTE,{{OBX},OBX,OBX,OBX,...})

– Against HL7 profile

• (Msg_type^Event_type^ e.g., ORU^R01^...)

– Against HL7 and/or user provided tables

• Example of user provided table is RTM for Ref_IDs, Units, etc.

– Against „validation context‟, including specific values

• Defined in XML (e.g., specific test case values)
Software and Systems Division

NIST IHE-PCD HL7 V2

Pre- & Connectathon Test Tool: Reports

• Resultant ‘Validation Report’ 

• Provides syntax and semantic results of checks against:

 HL7 V2 Structure Validation

 IHE-PCD Technical Framework/Supplements

 Harmonized Rosetta (hRTM)

 Test Case (specific value(s))

• Resultant Test reports may be saved (MS Doc)
Software and Systems Division

• Validation against „failure types‟: 

– VERSION*: The version in the message and in the profile should match. 

– MESSAGE_STRUCTURE_ID*: The message type (MSH.9 element) in the profile 

and in the message should match. 

– MESSAGE_STRUCTURE: The message should have a valid message structure 

(correct usage, correct cardinality, and correct element name). 

– USAGE: R elements should be present; X elements should not be present in the 

message. 

– CARDINALITY: Elements should be present at least the minimum times and at 

most the maximum times specified in the profile. It should also take into account 

the usage of the element (X element with a minimum of 4 should not be present in 

the message). 

– LENGTH: The value of the element should have a length equal or less than the 

value specified in the profile. 

– DATATYPE: For the data type NM, DT, DTM, SI and TM, the value of the element 

should match the regular expression defined in the standard. 

– DATA: The value of the element should match a constant specified in the profile, a 

value set specified in a table, a value or a regular expression specified in the 

message validation context. 

– MESSAGE_VALIDATION_CONTEXT*: This is a user input error when the 

location specified in the message validation context can't be found in the message. 

– TABLE_NOT_FOUND*: This is a user input when a table can't be found in the 

table files (TableProfileDocument). 

– AMBIGUOUS_PROFILE*: The profile should not be ambiguous. 

NIST IHE-PCD HL7 V2 Pre-Connectathon Test Tool: 

Testing Validation Types
Software and Systems Division

HL7 V2 Tools: Next Steps

• Re-visit Lessons Learned from Pre-Connectathon 

and Connectathon

• Update NIST HL7 tools/documentation accordingly

• Test Case revision/composition

• Start addressing Isolated (Scenario) and Peer-toPeer type testing for IHE-PCD cycle 5

• Integrate ICS XML resultant files w/ IHE-PCD 

– Volume III/IV – Device-level profiles?
Software and Systems Division

Topics

• HL7 Tooling Status (John)

– NIST‟s HL7 V2 Tools

– Supported IHE-PCD Cycle 4 Pre- & VirtualConnectathons, and Connectathon

• ISO/IEEE 11073 Tooling Status (Sandra)

– ICSGenerator Interface Update

– Discussion (Cardinality)

• HIT Testing Infrastructure - Coordination with IHE 

“Gazelle Project” (Rob)

• Discussion – Testing Strategy
Software and Systems Division

Tooling Status: ICSGenerator

• Interface update:

– Modified attribute list display for selected MOC in the right pane.

 The value and unit are not included in this enhancement, but will be 

included when Rosetta dbase is incorporated.

– Added status bar to show the nature of medical device profile 

(e.g. Application Profile Type).

– Added tool bar to directly access Tool‟s functionality.

– Added a tree panel tool bar to aid in managing and operating 

ICSGenerator modeling capabilities (also available when right 

clicking the objects in the tree).

• Incorporated the x73 Nomenclature Dbase 

– Added drop down menu for any text box where data is derived 

from the database. 

• Infrastructure terms (e.g., object & attribute names)

• Object Type ID

• Object Class

– Term codes auto-filled.

– The drop down includes auto completion and keyword searching.

– ICSGenerator does not depend on dbase installation; data 

accessed via an XML file that contains the x73 Nomenclature 

Dbase.
Software and Systems Division

Tooling Status: ICSGenerator

(continued)

• Added capability for device specialization to constrain attribute 

status - when the status is not “mandatory”.

• Added a transformation capability to generate “proposed” 

Rosetta Containment Hierarchy (RCH). 

– ICSGenerator supports the generation of:

• a device specific observation and general attribute observation 

hierarchy.

• OBXV visibility (user supplied)

• OBX-4 representation based on visibility and containment.

• The ability to save RCH in xml format.

 ICSGenerator must be enhanced to support device specialization 

cardinality constraints.

• Code restructured; Improves:

– maintainability, expandability, and performance.

• Issues/bugs Fixed

– Label now fully displayed on Jtree.

– Attribute update panel no longer hangs when removing attributes and/or 

adding the changes.
Software and Systems Division

Tooling Status: ICSGenerator 

Old Interface: New Interface:

Major tool bar

Status bar

Tree panel 

bar
Software and Systems Division

ICSGenerator

Tooling Status

Drop down menu 

RefIds derived from the 

x73 Nomenclature 

database.
Software and Systems Division

ICSGenerator

Tooling Status

Rosetta 

Containment 

Hierarchy Report.
Software and Systems Division

Tooling Status/Next Steps

ICSGenerator

• Incorporate Rosetta Dbase.

– Object Type, unit code, metric id (?)- in Nu-ObservedValue value type.

• Add the capability for specializations to constrain object 

cardinality.

• Value display enhancement.

• Continue improving application interface to increase usability.

• Provide initial guidance (on the right panel) when starting 

ICSGenerator.

• Finalize implementation of RCH (based on standards-body 

approval).

– Implementing OBXV and derived OBX-4 

• Update PHD specialization profiles as approved by ISO/IEEE
Software and Systems Division

RTMMS and x73-10201 Status

• RTMMS

– No update, updates forthcoming as more Rosetta terms identified

– Management and Hosting Discussion – later in day (Q2/Q3)

• X73-P10201 Electronic DIM (XML Schema)

– No update
Software and Systems Division

Topics

• HL7 Tooling Status (John)

– NIST‟s HL7 V2 Tools

– Supported IHE-PCD Cycle 4 Pre- & VirtualConnectathons, and Connectathon

• ISO/IEEE 11073 Tooling Status (Sandra)

– ICSGenerator Interface Update

– Discussion (Cardinality)

• HIT Testing Infrastructure - Coordination with IHE 

“Gazelle Project” (Rob)

• Discussion – Testing Strategy
Software and Systems Division

Topics

• HL7 Tooling Status (John)

– NIST‟s HL7 V2 Tools

– Supported IHE-PCD Cycle 4 Pre- & VirtualConnectathons, and Connectathon

• ISO/IEEE 11073 Tooling Status (Sandra)

– ICSGenerator Interface Update

– Discussion (Cardinality)

• HIT Testing Infrastructure - Coordination with IHE 

“Gazelle Project” (Rob)

• Discussion – Testing Strategy
Software and Systems Division

Topics

• HL7 Tooling Status (John)

– NIST‟s HL7 V2 Tools

– Supported IHE-PCD Cycle 4 Pre- & VirtualConnectathons, and Connectathon

• ISO/IEEE 11073 Tooling Status (Sandra)

– ICSGenerator Interface Update

– Discussion (Cardinality)

• HIT Testing Infrastructure - Coordination with IHE 

“Gazelle Project” (Rob)

• Discussion – Testing Strategy
Software and Systems Division

Test Environments

• Instance Testing

– Conformance (e.g., against HL7 2.x or CDA)

• Implementation conforms to Spec. on which it is based

• IHE Model: ~Virtual and Pre-Connectathon

• Isolated System Testing

– Includes Instance Testing Activities

– Protocol Conformance 

– Functional Behavior Conformance

• Features and Operational behavior correspond to Specs.

• IHE Model: ~Virtual and Pre-Connectathon

• Peer-to-Peer System Testing

– Includes Isolated System Testing Activities

– Interoperability Testing

• Testing complete application environment 

• May include interacting w/ Database, using Network 

Communications, or interacting w/ other hardware, apps, or 

systems if appropriate

• IHE Model: ~Connectathon
Software and Systems Division

Test Artifacts

•Conformance Profiles

•HL7 Tables

•Validation Context Files

•Generation Context Files

HL7 V2

Message

Validation

Services Test Management

Test

Harness

(Java Code)

HL7 V2 Message

Validation Criteria

Results

HL7 V2

Message

Validation

Report User

Instance Testing 

HL7 V2 Message Validation

Test Execution

Web 

Application

Report Client

HL7 V2 

Message

Conformance Testing: Showing that messages are in

compliance w/ HL7 V2.x and constraints specified in 

IHE-PCD framework documents.
Software and Systems Division

System Under Test

IHE-PCD

DOR

System

Future: Isolated System Testing

IHE-PCD Application Functional Behavior Testing

Test Artifacts

•Conformance Profiles

•HL7 Tables

•Validation Context Files

•Generation Context Files

IHE-PCD

DOC

Test Agent

HL7 V2

Message

Generation

IHE-PCD

DOR

Test Agent

HL7 V2

Message

Validation

Services Test Management

Test

Harness

(Java Code)

Router

IHE-PCD DEC

DOC

Test Scenario

Results

Message

Validation

Reports

Vendor

Test Execution

Web 

Application

Client

Report

RTM

Message

Validation

Conformance Testing: May include HL7 message syntax 

validation, RTM, and/or application functional behavior testing

Time Security
Software and Systems Division

Test Artifacts

•Conformance Profiles

•HL7 Tables

•Validation Context Files

•Generation Context Files

Services Test Management

Test

Harness

(Java Code)

HL7 V2 Message

RTM

Validation Criteria

Results

RTM

Message

Validation

Report

User

Future: Isolated System Testing

IHE-PCD RTM Validation

Test Execution

Web 

Application

Report Client

HL7 V2 

Message

HL7 V2

Message

Validation

RTM

Message

Validation

IHE PCD

DOC

Test Agent

Conformance Testing: Showing that messages are in

compliance w/ Harmonized-Rosetta terminology

Time Security

System Under Test

IHE-PCD

DOR

System

HL7 V2 

Message
Software and Systems Division

Future: Peer-to-Peer System Testing 

IHE-PCD Application Functional Behavior Testing

Test Artifacts

•Conformance Profiles

•HL7 Tables

•Validation Context Files

•Generation Context Files

IHE-PCD

DOC

Test Agent

HL7 V2

Message

Generation

IHE-PCD

DOR

Test Agent

HL7 V2

Message

Validation

Services Test Management

Test

Harness

(Java Code)

Router/Logger/Proxy

IHE-PCD DEC

DOC

Test Scenario

Results

Message

Validation

Reports

Vendor

Test Execution

Web 

Application

Client

Report

RTM

Message

Validation

Conformance Testing: May include HL7 message syntax 

validation, RTM, and/or application functional behavior testing

Interoperability Testing: system(s) (functional) behavior

Time Security

System Under Test

IHE-PCD

DOC

System

System Under Test

IHE-PCD

DOC

System