NIST

Medical Device Communication Testing

Semantic interoperability of Medical Devices

Test Tool Update

Joint HL7/IEEE 11073 Healthcare Devices Working Group

National Institute of Standards and Technology

15 January, 2013 – Phoenix, Arizona

Contact: john.garguilo@nist.gov, 301-975-5248
2

NIST MDC Testing Staff 

• John J. Garguilo

301-975-5248

john.garguilo@nist.gov

• Sandra Martinez 

301-975-3579

sandra.martinez@nist.gov

• Julien Deshayes (Guest Researcher) – HL7 V2 Tools

• Nicolas Crouzier(GR) - RTMMS

• Jing Gao (GR) – ICSGenerator

• Art Griesser (Prometheus Computing ) – DIM 

Modeling

Semantic Interoperability of Medical Devices

2
3

Discussion Topics 

• HL7 V2 (2.6) IHE-PCD Validation Tools

– Cycle 7 – 2012-13; Fall/Winter Pre-Connectathon [completed] + 

Virtual Testing, January/February 2013 Connectathon, March 

2013 HIMSS13

– IHE-PCD Pre-Connectathon, “Isolated Environment”

– IHE-PCD Connectathon, “Instance Environment

• Rosetta Terminology Mapping Management System 

(RTMMS) Overview 

– RTMMS Deployment Update and service status

• IEEE Agreement to make available the “Works”

• ISO/IEEE 11073 ‘Domain Information Model Editor’

– Proposed Roadmap

– UML Defined x73 DIM, Provides DIM Objects

– Art Griesser: issues/questions worked on via Tooling and Standards 

weekly WG

3
HL7 V2.6 Validation Tooling

IHE-PCD Pre-Connectathon Cycle 7

Semantic interoperability of Medical Devices

Test Tool Update

Joint HL7/IEEE 11073 Healthcare Devices Working Group

National Institute of Standards and Technology

15 January, 2013 – Phoenix, Arizona

Contact: john.garguilo@nist.gov, 301-975-5248
5

2012-12 IHE-PCD Pre-Connectathon Status 

• IHE-PCD: NIST “Isolated Environment” Test Tools required

– HL7 V2.6

– Supports 6 Integration Profiles

• DEC – Device Enterprise Communication (9)

– SPD – Filter (4)

– POI – Pulse Ox (3)

• PIV – Patient Infusion Verification (7)

• IPEC – Infusion Pump Event Communication (6)

• IDCO – Implantable Device Cardiac Observation (4)

• ACM – Alarm Communication Management (1)

• RDQ – Retrospective Data Query (1)

• 28 Test Cases

• 60+ existing User Accounts 

– In total since tool made available
6

IHE-PCD 2012/13 Pre-Connectathon + Connectathon 

 NIST Testing Support 

The NIST V2 (2.6) Tools perform the following validation:

• Syntax and Semantic Content Validation

– Against IHE-PCD Technical Frameworks/Supplements (e.g., 

PCD-01, Communicate Device Data)

– Against HL7 conformance profile

– Against HL7 and/or user [local] provided tables

• Example of user provided table is RTM for Ref_IDs, Units, etc.

– Against ‘validation context’, including specific values

• Defined in XML (e.g., specific test case values)
7

Testing Environments 

• Instance Testing

– Conformance (e.g., against HL7 2.x or CDA)

• Test object conforms to specification on which it is based

• IHE Model: ~Virtual and Pre-Connectathon 

• NIST IHE-PCD v2 Message Validation Test Tool

• Isolated System Testing

– Includes Instance Testing Activities

– Protocol Conformance 

– Functional Behavior Conformance

• Features and operational behavior correspond to specifications

• IHE Model: ~Virtual and Pre-Connectathon

• NIST IHE-PCD v2 Test Tool

• Peer-to-Peer System Testing

– Includes Isolated System Testing Activities

– Interoperability Testing

• Testing complete application environment 

• May include interacting with Database, using Network Communications, or interacting with 

other hardware, applications, or systems if appropriate

• IHE Model: ~Connectathon
8

NIST V2 HL7 IHE-PCD Test Tool: Operational Process 

END-USER

(VENDOR)

SYSTEM UNDER TEST 

(SUT)

NIST IHE-PCD 

HL7 v2/v3 TEST TOOL

SPECIFICATIONS

(test material that defines test assertions)

INTERACTION/REPORTS

MESSAGES (TEST OBJECTS)

MANUAL OR AUTOMATED SUT

Web Application 

Interface

(via the communication protocol

currently only MLLP)

V3 – Future Work

STIMULUS OR RESPONSE (MESSAGES)
9

IHE-PCD Cycle 7 2012-13 Pre-Connectathon 

 Test Result Statistics – Organizations and Tests Executed 

• Total Organizations (by User Accounts): 19

• Total Organizations (by *unique Company Name): 14

*Org No. followed by dash indicates more than one account 

from same organization in following slide

Registered (stored) at NIST site (over Pre-Con 2012-13 cycle)

• Total number of Tests Executed:82

• Total Number of Tests Passed: 81

• Total Number of Tests Failed: 1
10

Organization running DEC Reporter: 16

Organization running DEC Consumer: 1 (1 Test Case) – OZ Systems

Organization running DEC Filter: 1 (1 Test Case) – B Braun 

Organization running PIV Programmer: 0

Organization running PIV Consumer: 5

Organization running ACM Reporter: 6

Organization running ACM Manager: 1 (1 Test Case) – Philips Medical

Organization running IDCO Reporter: 0

Organization running IDCO Consumer: 0

Organization running IPEC Reporter: 4

Organization running IPEC Consumer: 0

Organization running RDQ Reporter: 0

Organization running RDQ Consumer: 0

Organizations running DEC Option POI: 1 (3 Test Cases) - Massimo

IHE-PCD Cycle 7 2012-13 Pre-Connectathon 

 Test Result Statistics – by Actor 
11

2012-13 IHE-PCD Pre-Connectathon Results, 1 of 3 

Org No. Organization Result Integration 

Profile/Actor

1-1 GE (Yang) 1 Pass DEC Reporter

1-2 GE Healthcare

(Flanders)

1 Pass DEC Reporter

2 B Braun 

(Engelbert)

1 Pass

4 Pass

4 Pass

1 Pass

6 Pass

DEC Report

DEC Filter

PIV Consumer

ACM Reporter

IPEC Reporter

3 True Process, 

Inc

(Anggara)

1 Pass

5 Pass

DEC Reporter

PIV Consumer

4 Carefusion

(Lunde)

1 Pass

5 Pass

3 Pass/1 Fail

DEC Reporter

PIV Consumer

IPEC Reporter
12

2012-13 IHE-PCD Pre-Connectathon Results, 2 of 3 

Org No. Organization Result Integration Profile/Actor

5-1 Philips Medical

(Gold)

1 Pass ACM Manager

5-2 Philips 

Healthcare

(Rhoads)

1 Pass DEC Reporter

5-3 Philips 

Healthcare

(K, Raja)

1 Pass DEC Reporter

5-4 Philips 

Healthcare

(Shekhar)

1 Pass DEC Reporter

6-1 West Health 

Institute 

(Mellott)

1 Pass ACM Reporter

6-2 WHI (West 

Health Institute

1 Pass

1 Pass

6 Pass

DEC Reporter

ACM Reporter

IPEC Reporter
13

2012-13 IHE-PCD Pre-Connectathon Results, 3 of 3 

Org No. Organization Result Integration Profile/Actor

7 Nuvon

(Neto)

1 Pass

1 Pass

DEC Reporter

ACM Reporter

8 Baxter

(Yang)

1 Pass

4 Pass

DEC Reporter

PIV Consumer

9 Hospira

(Walton)

1 Pass

5 Pass

1 Pass

6 Pass

DEC Reporter

PIV Consumer

ACM Reporter

IPEC Reporter

10 Mindray

(Fuchs)

1 Pass DEC Reporter

11 OZ Systems

(Kunte)

1 Pass

5 Pass

DEC Reporter

DEC Consumer

12 Masimo Corp.

(Housel)

4 Pass DEC Reporter

13 Medtronic Inc.

(Stomberg)

4 Pass DEC Reporter

14 Baxter

(Cuti)

1 Pass ACM Reporter
Rosetta Terminology Mapping Management 

System - RTMMS

Semantic interoperability of Medical Devices

Test Tool Update

Joint HL7/IEEE 11073 Healthcare Devices Working Group

National Institute of Standards and Technology

15 January, 2013 – Phoenix, Arizona

Contact: john.garguilo@nist.gov, 301-975-5248
15

RTMMS Overview 

• A web application* that allows vendors and reviewers access, 

retrieval, and reporting of Rosetta Tables over the internet in 

conformance to IHE-PCD RTM Profile

• An electronic resource/tool providing the capability of saving 

data in xml format (as defined by RTM Profile)

• Aids the harmonization process by:

– Identifying missing terms

– Automatic generation of the “Harmonized Rosetta Table”

– Providing latest up-to-date view of hRTM table

• Facilitates the proposal of New Terms to IEEE 11073 

Nomenclature standard

• Facilitates Conformance Tooling

– Message verification and conformance (syntax and semantics)

– Leading to interoperability...

– *developed by and currently hosted at NIST

– Integrated with ICSGenerator Tool (hRTM also imported into DIM Editor)
16

Rosetta Terminology Mapping Data Base

• Rosetta Table

– Maps vendor supported observations, units and enumerations to 

ISO/IEEE x73 nomenclature

• Units Table

– Defines allowed units-of-measure

– Defines groups of related units-of-measure

• Enumerations Table

– Defines groups of enumerated values

• hRTM Table

– Generated from the original Rosetta 

X73 Nomenclature DB

Security DB - Stores users information

RTMMS Key Tables 
17

• Access to NIST IEEE 11073 Nomenclature database

– Appendix A terms (from ISO/IEEE 11073 10101: Nomenclature)

– Appendix B terms (from ISO/IEEE 11073 10101: Nomenclature)

– IDCO terms (ISO/IEEE 11073-10301 – implantable device cardiac)

– aECG (annotated Electrocardiography)

– PHD terms (personal health domain)

• Access to RTM database

• Ability to propose terms in Rosetta

• hRTM, units, and enumeration download-able in XML format

• User registration

– Email confirmation, approval process... controlled through ‘admin’

• Filtering based on regular expressions

• Rosetta validation against hRTM

• Management capabilities for SDO users

 Integrated w/ ICSGenerator / exported to ‘DIM Editor’

RTMMS Key Features 
18

• General user

– Views Rosetta Tables

• Reviewer

– Participates in discussions

• Contributing Organizations

– Vendor/Contributer ‘sandbox’

– Modifies Vendor Rosetta Table

– Suggests new terms

– Modifies Units and Enumerations Table

• SDO (Standard Development Organization)

– Modifies Units and Enumerations Table

– Register new terms

• Admin

– Manages User Accounts

RTMMS Users 
19

RTMMS (as of 15 January 2013) 

• 605 hRTM terms

• 34 Contributing Organizations now Registered!
20

RTMMS Primary Updates 

 Agreement finalized between IEEE and NIST (Dec 2012)

• Allows users free and public access of terminology attributes that were 

formally ‘pay-for’ via the NIST tool

• The “Works”

Definition of the “Works” [from IEEE/NIST Agreement]

The following data items within the approved standards , existing and future 

versions, currently designated as:

1 – IEEE 11073-10xxx

2 – IEEE 11073-20xxx

Limited to the following data items within, currently designated as:

1 – Ref ID

2 – Code

3 – Description

4 – Systematic Name

 Revised “term approval” Functionality
21

Term Approval 

Revised “term approval” tables/tab

“New proposed terms” tab (205 rows)

Re-groups all the newly proposed terms with “MDCX_” prefix

The ‘Assign REFID’ button allows to change the REFID from an “MDCX_” term 

to a “MDC_ term” (or anything else) 
22

• “Ready to review terms” tab (220 rows):

– Terms with MDC_ prefix and null or 0 term code 

– The ‘Register’ button allows to register a term 

(define a Block, partition, code, systematic name etc...) to the x73 database

 These processed terms would become / be added to the 

harmonized Rosetta terms (hRTM) upon approval by SDO user 

and balloting

Term Approval, cont. 
23

RTMMS Updates, cont. 

• User can now download an HTML version of:

– “hRTM”

– “New proposed terms” 

– “New proposed terms” table, 

– Note: hope is users find this version to be easier to work with when working 

outside the tool

• “Standard Table”

– A new column “Standard table” is now present in all Rosetta, hRTM and review 

tables - 

– Provides association of the term with corresponding table in the base standard 

– Only the terms given by Jan W. (275) REFIDs are currently linked

– NIST can add more if community provides the/additional mapped terms.
Domain Information Model

UML / DIM Editor, Proposed Roadmap

Semantic interoperability of Medical Devices

Test Tool Update

Joint HL7/IEEE 11073 Healthcare Devices Working Group

National Institute of Standards and Technology

15 January, 2013 – Phoenix, Arizona

Contact: Arthur Griesser <a.griesser@prometheuscomputing.com>
25

How Should Road Map be Built? 

• Suggested process

– First figure out where we want to go (goals)

– Then figure out how to get there

• Suggested principles

– Get fundamental artifacts fast

– Prove one implementation strategy fast

– Iterative improvement

• Basic functionality first, frills next, ‘pretty’ last

– Rigorously OO

– Add additional implementations
26

Goals 

• Widespread use of standard

– Clear, easy implementation

• Machine interpretable artifacts

• Artifacts in multiple forms, for multiple technologies

– Self consistent, few if any conflicts

• All artifacts derived from one fundamental source

– Useful, meets user needs (use cases as tests)

• Compliance tests

• Interoperability tests
27

Progress I 

• Made UML model from DIM standard

• Made XML Schema from UML model of DIM

• Made web-based editor for DIM data

• Added to DIM UML and web app:

– RTMMS data

– Conformance statements

– Meta-information

• Demonstrated an example (non-DIM) standard derived from a 

model
28

Progress II 

• Created interface for DIM section 8

• Finished testing framework document

• Injected additional metainformation into UML

– Encoding values, clause numbers & titles, name binding, registered as, 

description, update type, attribute id

• Discussed semantics of DIM groups

• Discussed ASN.1 representation of inheritance

• Generated JSON and YAML files of meta-info
29

Next Steps 

• Understand ASN.1 generation requirements

• Derive ASN.1 syntax programmatically

• Extract normative text, with formatting

• UML model elements associated with normative text

• Programmatically generated IEEE compliant standard

• Add containment tree browser to DIM GUI

• Executable use cases

• Review ASN.1/UML/XML mapping

• Generate HL7 messages

• Review & improve models

• Encoded versions of schema
30

Proposed x73 DIM UML (from Art Greisser) 
40

NIST MDC Testing Project 

Web Sites 

• Project Web site: www.nist.gov/medicaldevices

• NIST HL7 V2 Test Tooling Web sites:

IHE-PCD Pre-Connectathon:

http://hit-testing.nist.gov:13100/PCD-HL7WebPreCon/

 IHE-PCD Connectathon: 

http://hit-testing.nist.gov:13100/PCD-HL7WebCon/

• NIST Medical Device Terminology Service:

 Rosetta Terminology Mapping Management System (RTMMS): 

http://hit-testing.nist.gov:13110/rtmms/

• NIST Implementation Conformance Statement 

Generator (ICSGenerator):

– http://hit-testing.nist.gov/medicaldevices/ICSGenerator/ics_download.html

Semantic Interoperability of Medical Devices

40