Software and Systems Division

IHE-PCD

ISO/IEEE 11073, 

and

NIST

NIST Medical Device Connectivity

Test Tooling

IHE-PCD/IEEE WG Meetings

(F2F @ Berkley, California)

May 4, 2010
Software and Systems Division

NIST Contacts

• John J. Garguilo

301-975-4248

john.garguilo@nist.gov

• Sandra Martinez

301-975-3579

sandra.martinez@nist.gov

• Maria Cherkaoui (Guest Researcher)

• Jing Gao (GR)

Project Web site: 

www.nist.gov/medicaldevices

Semantic Interoperability of Medical Devices
Software and Systems Division

Topics 

• Areas being addressed by Test Tooling Effort

• HIT Test Infrastructure

– Conformance testing across various test 

environments

• IHE-PCD HL7 Message Verification

– Using Profiles (constraints  assertions)

• IHE-PCD Tooling (2010 cycle 4) 

& going forward...(2010-11 cycle 5)

• ISO/IEEE 11073 Tooling

– RTMMS

– ICSGenerator
Software and Systems Division

NIST Test Effort

• Medical Device Standards 

Work

– Device and Enterprise-level

• Integrating Health 

Enterprise - Patient Care 

Devices (IHE-PCD)

– Enterprise-level

• Personal Health Devices

– Personal Tele-health-level

• Facilitate the efficient 

exchange of medical device 

and vital signs data 

throughout the HC 

enterprise

– Test Research Methods

– Conformance 

Interoperability (based on 

Standards)

– Ultimately: Real-time plugand-play interoperability

Medical Device Communication
Software and Systems Division

IHE-PCD Testing

IHE-PCD Testing – Key Objectives

• Increase test comprehensiveness & quality 

• Support both conformance & interoperability testing

• Support for pre- & virtual- connectathons, actual connectathon

& enable year round testing

• Remain in alignment with IHE-PCD integration profile 

development road map

• Establish single framework for PCD covering increasing 

complexity and technologies over next 5 years

• Coordinate with IHE ―Gazelle Project‖ and NIST‘s HIT Test 

Infrastructure

• Generate work products that companies can use in their 

regulatory submissions
Software and Systems Division

IHE-PCD Testing

IHE-PCD Testing – Key Ingredients

• Well Defined Integration Profiles

– Technical Framework

– Supplements

• Unambiguous Standards

• Test Guidelines

• Test Plans

– Test Scenarios, Actors, Transactions, Validation Criteria

• Test Artifacts, including:

– HL7 Profile(s) (and eventually x73 Device Specializations?) 

– Repositories

– Nomenclature (e.g., RTM)

– Value Tables (e.g., HL7, units, local, etc.), Default and Sample Values

• Test Cases
Software and Systems Division

Conformance Testing: 

Using  ̳Profiles‘ to Advance Rigorous Testing

Specification 

Constraints

User / Device

Message

E.g., HL7 V2 

Terminology/

Nomenclature

Standards

Profile

Domain

Framework

Test Case/

Value(s)

Validation

Test Management

Test Services

Test System 

Development 

Components

Test Harness

Test Resources

Based on Use Case(s)

Report

Test System 

Instance

Patient Care Devices (PCD)
Software and Systems Division

Validation Operational Process: 

Origin of Test Assertions

HL7 v2

Standard

Message

Definition

IHE TF

Message

Transaction

Constraints

HL7 v2

Standard

Value Sets

IHE TF

Message

Transaction

Value Set

Constraints

Harmonized

Rosetta

Terminology

Mapping

Constraints

ISO/IEEE 

11073 

Nomenclature

Assertions

Validation

Context

File

(XML)

Table

Library

(XML)

Conformance

Profile

(XML)

Validation

Engine

Test Case

Specific

Test

Assertions

IHE-PCD TF

Message

Transaction

Test

Assertions

Validation

Context

File

(XML)

Testable Assertions: 

IHE-PCD Validation Requirements 

Used by NIST Test Tools

Patient Care Devices (PCD)
Software and Systems Division

• Validation against  ̳failure types‘: 

– VERSION*: The version in the message and in the profile should match. 

– MESSAGE_STRUCTURE_ID*: The message type (MSH.9 element) in the profile 

and in the message should match. 

– MESSAGE_STRUCTURE: The message should have a valid message structure 

(correct usage, correct cardinality, and correct element name). 

– USAGE: R elements should be present; X elements should not be present in the 

message. 

– CARDINALITY: Elements should be present at least the minimum times and at 

most the maximum times specified in the profile. It should also take into account 

the usage of the element (X element with a minimum of 4 should not be present in 

the message). 

– LENGTH: The value of the element should have a length equal or less than the 

value specified in the profile. 

– DATATYPE: For the datatype NM, DT, DTM, SI and TM, the value of the element 

should match the regular expression defined in the standard. 

– DATA: The value of the element should match a constant specified in the profile, a 

value set specified in a table, a value or a regular expression specified in the 

message validation context. 

– MESSAGE_VALIDATION_CONTEXT*: This is a user input error when the 

location specified in the message validation context can't be found in the message. 

– TABLE_NOT_FOUND*: This is a user input when a table can't be found in the 

table files (TableProfileDocument). 

– AMBIGUOUS_PROFILE*: The profile should not be ambiguous. 

NIST V2 Testing Tools and Services

Testing Validation Types
Software and Systems Division

Test Environment Message Validation

NIST V2 Testing Tools: IHE-PCD

• Validation of IHE-PCD message(s) and 

corresponding HL7 Profile(s) 

• Syntax and Semantic Content Validation

– Against HL7 V2 message (e.g., PCD-01)

• Message structure (e.g., 

MSH,PID,PV1,OBR,NTE,{{OBX},OBX,OBX,OBX,...})

– Against HL7 profile

• (Msg_type^Event_type^ e.g., ORU^R01^...)

– Against HL7 and/or user provided tables

• Example of user provided table is RTM for Ref_IDs, Units, etc.

– Against  ̳validation context‘, including specific values

• Defined in XML (e.g., specific test case values)
Software and Systems Division

Test Environments

• Instance Testing

– Conformance (e.g., against HL7 V2.x or CDA)

• Implementation conforms to Spec. on which it is based

• Isolated System Testing

– Includes Instance Testing Activities

– Protocol Conformance 

– Functional Behavior Conformance

• Features and Operational behavior correspond to Specs.

• Peer-to-Peer System Testing

– Includes Isolated System Testing Activities

– Interoperability Testing

• Testing complete application environment 

• May include interacting w/ Database, using Network 

Communications, or interacting w/ other hardware, apps, 

or systems if appropriate

NIST HIT Test Infrastructure
Software and Systems Division

Conformance Testing of an HL7 V2 Message

Report

Test Artifacts

• Conformance Profile

• HL7 Tables

• ‘Device’ Test Agents

• ISO/IEEE 11073/Rosetta 

Terminology

HL7 V2

Message

Validation

Services Test Management

HL7 V2 

Message

Validation 

Test Case

Results

HL7 V2 

Message

Validation 

Report

Test Harness

(Java Code)

Test 

Execution

User

Web 

Application

Client

HL7 V2 

Message

Registry/

Repository

Instance System Test Environment
Software and Systems Division

Test Environments

• Instance Testing

– Conformance (e.g., against HL7 V2.x or CDA)

• Implementation conforms to Spec. on which it is based

• Isolated System Testing

– Includes Instance Testing Activities

– Protocol Conformance 

– Functional Behavior Conformance

• Features and Operational behavior correspond to Specs.

• Peer-to-Peer System Testing

– Includes Isolated System Testing Activities

– Interoperability Testing

• Testing complete application environment 

• May include interacting w/ Database, using Network 

Communications, or interacting w/ other hardware, apps, 

or systems if appropriate

NIST HIT Test Infrastructure
Software and Systems Division

IHE-PCD Testing using a Web Application Client

Report

IHE-PCD

DOR/DOF

Test Agent

HL7 V2

Message

Generation

IHE-PCD

DOC

Test Agent

HL7 V2

Message

Validation

Services

Test Management

Router/Logger/Proxy

Vendor

System Under Test

Test Artifacts

•Conformance Profiles

•HL7 Tables

•Validation Context Files

•Generation Context Files

IHE-PCD Client

Test Scenario

Results

HL7 V2 Message 

Validation 

Reports

Test Harness

(Java Code)

Test Execution

Web 

Application

Client

IHE-PCD

IOR

Test Agent

IHE-PCD

AM

Test Agent

IHE-PCD

IOC

Test Agent

IHE-PCD

AR

Test Agent

IHE-PCD

IDCC

Test Agent

IHE-PCD

IDCR

Test Agent

Isolated System Test Environment
Software and Systems Division

Test Environments

• Instance Testing

– Conformance (e.g., against HL7 V2.x or CDA)

• Implementation conforms to Spec. on which it is based

• Isolated System Testing

– Includes Instance Testing Activities

– Protocol Conformance 

– Functional Behavior Conformance

• Features and Operational behavior correspond to Specs.

• Peer-to-Peer System Testing

– Includes Isolated System Testing Activities

– Interoperability Testing

• Testing complete application environment 

• May include interacting w/ Database, using Network 

Communications, or interacting w/ other hardware, 

apps, or systems if appropriate

NIST HIT Test Infrastructure
Software and Systems Division

A Framework for Building Test Systems—an SOA Approach

Test Data

Services Test Management

Test

Description Results

Test

Harness

Test Execution

Router/Logger/Proxy

User

Facilitator

Monitor

System Under Test

System Under Test

Validation

Generation Test Agent

Evaluation 

Agent

Aggregated 

Services

Test Artifacts

Report Time

Logging Security

Log Analyzer Proxy

Specialized

Validation

External

Services

Other 

Services

Test

Analyzer

Peer-to-Peer Test Environment
Software and Systems Division

NIST‘s IHE-PCD HL7 V2 Tools

IHE-PCD Pre-Connectathon Tool

http://xreg2.nist.gov:8080/PCD-HL7Web/
Software and Systems Division

NIST‘s IHE-PCD HL7 V2 Tools

IHE-PCD Connectathon Tool

http://xreg2.nist.gov:8080/PCD-HL7WebCon/
Software and Systems Division

ISO/IEEE 11073 – How Are We Involved?

• MDC Standards development 

– ISO/IEEE 11073 Point-of-care Medical Device Communication

– Co-chair new normative chapter - addition provides NIST developed 

electronic information model 

– Assist development of more complete and correct specifications prior to 

balloting

– Work with SDOs (testing perspective), clinicians, clinical engineers

• Device Communication Test Tooling

– XML Schema of the ISO/IEEE 11073 Domain Information Model

– ICSGenerator Tool

• Produces standard-compliant device profiles and specializations

• Generates Implementation Conformance Statements

– ValidatePDU Tool

• Provides message syntax and semantic validation

– Java Class Library (of standard‘s syntax notation) 

• Implementable-code of abstract types defined in standard

• Coder (encodes and decodes APDUs/messages)

–  ̳Rosetta‘ Terminology Management System

• Standardized terminology across MD manufacturers
Software and Systems Division

Rosetta Terminology Mapping

Management System

(RTMMS)

National Institute of Standards and Technology (NIST)

MARIA CHERKAOUI, John Garguilo, Sandra Martinez

April 2010

20
Software and Systems Division

NIST Tooling To Support RTM Process

(*Championed by Paul Schluter [GE Healthcare])

What is RTMMS? 

• A web application that allows vendors and reviewers access, 

retrieval, and reporting of Rosetta Tables over the internet in 

conformance to RTM

• The tool provides the capability of saving the data in the XML 

format as defined by RTM

• Aid in The harmonization process by:

– Identifying missing terms 

– Facilitate the proposal of new terms

– Facilitate discussion of the proposed term

– Automatic generation of the ―Harmonized Rosetta Table‖

• Database/XML Server initially prototyped and located at NIST

• A web service/tool used as part of SDO‘s ballot / approval 

process

“RTM Management System”
Software and Systems Division

NIST Tooling To Support RTM Process

What is RTMMS? (Continued)

• Facilitate Conformance Tooling

– Message verification and conformance

– Leading to interoperability...

“RTM Management System”
Software and Systems Division

RTMMS vs. RTM Excel process

• For Vendors

– Facilitate input of entries by vendors

• Tooltips providing supplementary information

• Available Interface to lookup values from the database

• Automatic completion of codes

• Validation of required content

– Reduce errors made by vendors while submitting entries

• For Reviewers and SDO

– Facilitate the generation of the Harmonized Rosetta

– Help the review process of Rosetta entries

• Highlighting discussed entries

• Highlighting proposed REFIDs

• Adequate interface to view discussions and add comments

• For all users

– Rosetta data available to everyone any time

– Provide XML version of tables

• All XSLT transformations can still be used

23
Software and Systems Division

Use case

24
Software and Systems Division

RTMMS Roles

• ―Non-registered User‖

– Able to download RTM (latest approved version)

• ―Vendor‖ (Registered)

– Able to view, edit and propose vendor terms (only for vendor they are 

registered with)

• ―Reviewer‖ (Registered)

– Able to view all vendor terms and make annotations (discussion points)

• ―Expert Reviewer‖ (Registered and approved by SDO)

– Technical expert (e.g., Jan Wittenber (Phillips) and Paul Schluter (GE)) 

who can view and comment on all vendor terms

– Expert has the ability to generalize term to overall specification

• ―SDO‖ (Standards Development Organization approved official)

– Authority to approve/decline new or edited term proposed by vendor

– Approval based on SDO ballet rules

– Considers working group and especially  ̳Expert Reviewer‘ input

– Usually chair/co-chair of standards body and or working group

• ―Administrator‖ (Approved by  ̳SDO‘)

– Provides administrative support to database

– Generates and provides new user account information (user name and 

password), delete accounts, etc.

25
Software and Systems Division

Database

• Models RTM data and relationships

• Uses x73 Nomenclature database

– REFIDs

– Term codes

– Partition numbers

– (New terms added as approved/normative additions to standard)

• Stores RTM data

– Rosetta table

– Units and Unit Groups

– Enumerations and Enumeration Groups

– hRTM table

– (New terms proposed – may be missing from x73 Nomenclature 

Database)

26
Software and Systems Division

Available Features of RTMMS

• Features based on the Rosetta Supplement

– Units table management

– Unit groups management

– Rosetta table management

– Handling uncertain REFIDs

– Automatic generation of the Harmonized Rosetta

• Need additional requirements

– XML Rosetta download

• User oriented features

– User management module (Implemented user (roles) 

and privileges)

– Columns filtering

– REFIDs lookup in database

– Group lookup in database

– Units and Enumerations lookup in database

– Term codes completion from database

– Saving browsing history

– User registration 

27
Software and Systems Division

Features of RTMMS (continued)

• RTMMS Architecture improvements

• New Features (enhancements from May 2009 WG meetings)

– New x73 Nomenclature

• Terms in both annexes A and B. 

• IDC Nomenclature

– Highlighted New Terms in Rosetta, Units and Enumerations Tables

– Added Interface for selecting REFID from x73 Nomenclature database

– Added Interface to propose New Terms

– Added New Term View for SDO users

– Implement Rosetta validation against H-Rosetta 

– Enhanced registration process.

• Email confirmation, approval...

– Added Admin Type of users to manage users and enable new users

– XML Units Download

– Added ranking capabilities to assess probability of valid terms in the 

Rosetta table

• Scale from 1 to 10

– Include column filtering based on regular expressions 28
Software and Systems Division

User Registration

29
Software and Systems Division

Units Table

30
Software and Systems Division

Enumeration Groups Table

31
Software and Systems Division

Edit Entry Form

32
Software and Systems Division

Add Comment Dialog

33
Software and Systems Division

Rosetta Table

34
Software and Systems Division

User Management Table

35
Software and Systems Division

REFID Selection Dialog (1/3)

36
Software and Systems Division

REFID Selection Dialog (2/3)

37
Software and Systems Division

REFID Selection Dialog (3/3)

38
Software and Systems Division

New Terms Table

39
Software and Systems Division

Next Steps

• Features based on the Rosetta Supplement

– Incorporate ―Enumerations‖ management capabilities

– Add ―containedBy‖, ―contains‖ and ―External_Sites‖ to the interface

– Ability to edit Harmonized Rosetta table and save changes

• User oriented features

– Implement ―change trailing‖ capabilities

• To identify occurred changes, time they were made, users who made them...

– Incorporate enhanced X73 Nomenclature database

• Includes Systematic name, description...

– Automate generation of the ―Harmonized Rosetta Table‖

– Adding new user role ―Expert (or Technical) Reviewer‖ 

• Continue discussion of approving and adding normalized terminology 

to IEEE x73

– Build on April 23 Discussion (w/ Jan, Paul, Melvin, Todd, John R, others?)

40
Software and Systems Division

ICSGenerator Capabilities

• Generates Implementation Conformance Statements (ICSs)

– Required in conformance section (10) of DIM x73 document

– Ensures common format for ICS generation

• Builds Device Profile (XML)

– Generates an electronic (XML) version of device data model based 

strictly on the IEEE x73 DIM

– Includes private or manufacturer-specific extensions

• Provides validation against DIM Schema

– A device data model generated using this tool can be validated against an 

updated version of the DIM XSchema

• Provides high level semantic interoperability

– Ensures correct containment relationship and terminology at the object 

class and related attribute, notification, and behavior level

– Compare Device ICSs

• Device ICSs comparison capability aids in identifying potential 

interoperability issues

• Generates HL7 OBX Segments

• Generates Device UML Diagram
Software and Systems Division

Tooling Status

ICSGenerator

– Interface update: • Make list of attributes visible for selected MOC in the right pane. – The value and unit are not included in this enhancement, but will be included when 

Rosetta dbase is incorporated.

• Added status bar to show the nature of medical device profile. • Added a direct tooling accessibility tool bar. • Added a tree panel tool bar to aid in managing and operating ICSGenerator

modeling capabilities. (also available when right clicking the objects in the 

tree) 

– Incorporated the x73 Nomenclature Dbase • Added drop down menu for any text box where the data could be extracted 

from the database. – Infrastructure terms (e.g., object & attribute names) – Object Type ID (if not in hRTM) – Object Class

• Term code auto-filled when object type is available from dbase.

• The drop down include auto completion and keyword searching, no case 

sensitive.

• ICSGenerator does not depend on dbase installation, it access the data from 

and XML file that contains the data from the x73 Nomenclature Dbase.
Software and Systems Division

Tooling Status

ICSGenerator

• Status (cont.) – Added a drop down to allow user to change the attribute 

status when the status is not ―mandatory‖.

– Initial implementation of RCH (Rosetta Containment 

Hierarchy).

– Code restructuring to improve maintainability, expandability 

and performance.

– Fixed bugs • Label not fully displayed on Jtree. • Attribute update panel hanging when removing attributes and adding the 

changes.
Software and Systems Division

Tooling Status/Next Steps

ICSGenerator

– Incorporate Rosetta Dbase. • Object Type, unit code, metric id (?)

- in Nu-Observed-Value 

value type

– Finalize implementation of RCH. • Implementing OBXV and derived OBX

-4 

– Value display enhancement – Add information description to tabs – Provide initial guidance on the right panel when stating 

ICSGenerator

– Update PHD specialization profiles
Software and Systems Division

Summary / Discussion

• Develop ―Conformance Test‖ WG for this cycle

– Bi-weekly, meet w/ individual Integration Profile Groups

– Update test cases

– Update/Continue work on Test Plans / Conformance Guide

• Develop Test Agents across Integration Profile Actors

– Continue work on TF and Supplements

– Further define  ̳scenarios‘ (message transaction sequences)

• RTMMS

– Continue discussion of approving and adding normalized terminology to 

IEEE x73

– Build on April 23 Discussion (w/ Jan, Paul, Melvin, Todd, John R, others?)

– Add two columns to support mapping to ITSDO work (w/ Jan)

• ICSGenerator

– Start developing IHE-PCD Device  ̳specializations‘ for devices across 

various IHE-PCD Integration Profiles

• Explore OHT work (w/ Ioana, David Carlson)

– http://mdht.projects.openhealthtools.org

• Questions? / Discussion...

• Thank-you!

45