# Presentation: 701560

## Abstract Data Types Workshop

- NCI, 9 – 11 April 2008
- Grahame Grieve

## Workshop Overview

- Part1: General Considerations
- Specifications and Architectural perspectives.
- Part 2: Semantics of the Data types
- Each data type in depth.
- Part 3: Implementation in Depth
-  XML and Java (Eclipse and javaSIG code).

## Administration

- In grand HL7 tradition...
- Quarter 1: 	  9:00 - 10:30  	*Coffee Break*
- Quarter 2: 	11:00 - 12:30    	*Lunch*
- Quarter 3: 	13:30 - 15:00 	*Coffee Break*
- Quarter 4:	15:30 - 17:00

## Scope

- Explain the Data types
- Semantics,
- modelling,
- syntax,
- implementation,
- architectural principles..
- Get real with UML, XML, and Java

**Notes:**
Tip: Add your own speaker notes here.

## Scope

- Explain the ISO Healthcare Data types
- Semantics,
- modelling,
- syntax,
- implementation,
- architectural principles..
- Get real with UML, XML, and Java
- Everything I had to find out the hard way!

**Notes:**
Tip: Add your own speaker notes here.

## Questions

- Data Types is an arcane area
- I’ve lived and breathed it for years
- I don’t know how to do this without using forwards references
- Don’t let things go by – interrupt!
- I’ve planned to be interrupted
- We’ll maintain a list of things I’ve promised to discuss

## Day : Overview

- Introduction
- Welcome & Background
- Abstract & ISO Data Types
- What is a data type?
- Architecture
- Chestnuts: nullFlavor, updateMode, equality
- Character sets & Unicode
- Collections, Cardinality, Conformance
- Semantics of Data Types
- Collection Types

**Notes:**
Tip: Add your own speaker notes here.

## General Context for V3 Data types

- Overview of HL7 V3
- Semantic and Implementable data types
- Politics
- Review of alternative approaches

## What’s a Data Type?

- Multiple Answers
- In this context:
- What you use for an attribute type
- Has no identity, so cannot have an association
- does not have associations (err, all associations are compositions)

## Overview of  V3

- RIM (Reference Information Model)

- Constraint Models
- (DMIM, DIM, RMIM, HMD, MT, CIM, LIM, TIM)

- Wire Formats
- (XML ITS)

- Solid Ontological
- Foundation

- Adaptation to a context of use

- How to make it work

- Data Types

- Vocabulary Definitions

- Implementation

- Concept

## Overview of  V3

- RIM (Reference Information Model)

- Service Functional
- Models

- OMG
- HSSP
- Specifications

- Solid Ontological
- Foundation

- Adaptation to a context of use

- How to make it work

- Data Types

- Vocabulary Definitions

- Implementation

- Concept

## CEN 13606

- RIM (Reference Model)
- Part #1

- Archetypes
- Part #2

- Exchange
- Format
- Part #5

- Solid Ontological
- Foundation

- Adaptation to a context of use

- How to make it work

- Data Types

- Vocabulary Definitions

- Implementation

- Concept

## RIM

## HL7 / ISO data types

- The data types are ubiquitous
- All actual data is stored in data types
- Plus content model bindings
- RIM is a general model for everything
- The data types are where the pedal hits the metal
- The data types are designed with rigor for rebust re-use in multiple contexts

## In the data types...

- Infrastructural things (Lists, Sets, Booleans)
- Text & multimedia content
- Coded Concepts / Terminology References
- Identification
- Names and Addresses
- Quantities
- Intervals and Complex Set Expressions
- Uncertain Values

## Slide 16

## History: v2

- HL7 v2 there is an ever growing number of data types
- Some are almost identical
- Some represent different concepts of how to solve the same problem
- Some are badly designed
- Framework does not allow proper componentization
- Some are complex data elements
- Rigorous semantics are not defined

## History: v3

- These problems are generally true about all of v2
- HL7 initiated a new version (v3). Goals:
- Defined with rigorous consistency
- Not Semantically ambiguous
- Allow high level of reuse
- Eventually become Reference/Constraint model

## R1: 2 Data Types Standards

- Abstract Specification
- Describes the data types independent of everything else
- Only depends on the notion of true and false
- Self contained mathematically rigorous definition of all the data types
- Not just “platform independent”
- XML ITS – Data Types
- “Implementation Technology Solution”
- Defines how to encode a representation in XML
- A compromise between efficiency, human readability and schema validation support

## RM-ODP View points

- Enterprise - purpose, scope and policies of system
- Information – kinds of information, and constraints & use of it
- Computational - functional decomp-osition into objects that interact
- Engineering - infrastructure required to support system distribution
- Technology - technology to support system distribution

## RM-ODP Mappings

- Enterprise
- Information
- Computational
- Engineering
- Technology

- Abstract Data Types
- XML ITS Data Types

## RM-ODP Mappings

- Enterprise
- Information
- Computational
- Engineering
- Technology

- Abstract Data Types
- XML ITS Data Types

- Gunther claims this is so!

## R2 Data types

- Learn from implementation experience
- Add new features (more requirements gathering)
- Clean up many inconsistencies
- New implementable but platform & technology neutral specification
- Resolve Political battles between standards organisations

## R2 Data Types - Progress

- Work began in 2005
- ISO data types conceived in UK
- ISO data types balloted first
- Abstract & ISO must be done together
- R2 taskforce: HL7 insiders – meet by skype
- Abstract & ISO currently in (expected) last round of balloting.
- Implementation trials are under way

## What’s the Political Discord

- HL7 V3 Data Types highly controversial
- Improper UML diagrams
- Modeling by constraint
- Gap between specification and implementation
- Open war about null flavors
- CEN rejected V3 Data types
- Strong input from OpenEHR
- Several candidate standards proposed
- ISO: Several candidate standards proposed
- A mess: personal, and a disaster

## ISO Data Types

- A computable/implementable standard
- Proper implementation of abstract data types
- Mappings to openEHR
- Acceptable to everyone
- Grudging acceptance!
- The ISO data types are still full of compromise
- The Eclipse reference implementation demonstrates this

## Data Types Requirements

- Massive requirements gathering:
- v2 semantics must all be supported
- Years of requirements gathering
- many MBs of archived meeting minutes, wiki content, ballot reconciliation, emails, skype chats
- review by other standards organisations, experts
- most available publicly, but not gathered together

## Slide 28

## Slide 29

## Abstract Data Types

- Overview
- Explanation of Definitions
- Data Type Definitions
- Foundation / Basic
- Quantities / Quantity Collections
- Uncertainties
- Data Type Flavors
- Appendices
- Informative Types
- Representational Properties

## Introduction and Scope

- This document provides the semantic definitions for the data types used in the creation of HL7 V3 specifications.  These "abstract" semantic definitions are also able to be used as constraints in the creation of implementation guides, and implementation technology specifications that enable actual exchange of data are based on these semantic definitions.
- The audience for this document is anyone who uses HL7 V3 data types - both implementers and designers of HL7 specifications.  For implementers, this document should be treated as secondary supplemental information to the ITS data types, which provide an basis for actual implementation of the data types.
- The data types defined in this specification are not intended to be implemented based on the details presented herein.  For example, this specification does not differentiate between the information that should be represented in a payload and the information that should be derived from the payload.

## Overview

- What is a data type?
- To be dealt with later
- Difference between Semantics and Representation
- Laboring the point...
- Properties
- “answers to questions”
- “Many properties are never explicitly represented in an implementation...”

## Why Abstract?

- Any other approach depends on some technology
- instability
- binding problems,
- choices...
- Was it worth it – yes
- solid base with no ambiguity
- internally rigorous

## Architecture & Semantics

## Data Type Definitions

- Full name,  code name & specializations
- UML diagrams
- A prose definition & explanation
- A table of "primary" properties
- Data Type Definition Language (DTDL) definition
- prose explanation of key points
- Vocabulary definitions
- Notes identifying common misunderstandings, usage expectations, open questions, and other important points

## UML Stereotypes

*interface*:  standard UML interface stereotype
*bind*: When specialization includes binding the generic to a particular type, a "realize" association with a "bind" stereotype . (The type is indicated as a parameter on the bind stereotype.)
*mixin*: denotes that the type is an interface, and that it specializes the parameter type
*flavor*:  not a type: a flavor expresses a set of constraints on a data type.
*constrain*: Indicates the relationship between a flavor and the type or flavor that it constrains.
*iteratable*: Indicates that it makes sense to enumerate the items of the collection using a classic iterator interface.

## UML Colours

| Colour | Category |
| --- | --- |
| White | Infrastructure |
| Salmon | Text & Multimedia |
| Green | Coded Concepts |
| Orange | Names and Addresses |
| Purple | Identifiers & Contacts |
| Grey | Quantities |
| Yellow | Generics |
| Blue | Generic Type Extensions |
| Mauve | Comparators |

## Concept Domains

## Vocab Structure

- Code system: a collection of coded concepts. each having associated designations and meanings
- Concept Domain: a named category of like concepts
- Value Set : a uniquely identifiable set of valid concept representations
- Binding : associating a value set with a domain
- CD : coded data types are constrained to a domain

## Vocab Structure

## DTDL

- formal definition of data types
- other syntaxes
- technology specific
- dependent on their own type libraries
- accessible to formal analysis
- not an API specification.
- Does not define the means of implementation.
- language need not be implemented or used
- The formal definition only specifies the meaning of the data values through statements defining semantic relationships and behavior.

## DTDL Definitions

- Full name, code name
- Specialisations
- Property signatures
- Invariants
- Type Conversions
- Literal Formats

## DTDL Type Declaration

- abstract type DataValue alias ANY {
- type Boolean alias BL specializes ANY 		values(true, false) {
- template<ANY T, COMP C = CEQ> abstract type Set<T, C<T>>
-  alias SET<T> specializes ANY {

## Types of Types

| Keyword | Meaning |
| --- | --- |
| abstract  | no non-exceptional value can be of this type: a proper value must belong to a concrete specialization of the abstract type
 |
| protected  | type that is used inside this specification but no property outside this specification should be declared of a protected type
 |
| private  | is an internal "helper" abstraction, defined only for the purpose of defining some aspect of the semantics of data types but that is not used even as the type of another protected or public type's property. |

## DTDL Type Declaration

- abstract type DataValue alias ANY {
- type Boolean alias BL specializes ANY 		values(true, false) {
- template<ANY T, COMP C = CEQ> abstract type Set<T, C<T>>
-  alias SET<T> specializes ANY {

## Property Definitions

- abstract type ANY {
-   TYPE dataType;
-   protected BN identical(ANY x);
- };
- abstract type SET<T, C<T>> {
-   BL contains(T element);
-   BL contains(SET<T> subset);
-   SET<T> union(SET<T> otherset);
-   SET<T> union(T element);
- }

## Invariant Definitions

- Invariant[(<Type> <name>)  [, <Type> <name>]*]
-   [where condition] {
-   rule [; rule]*
- };
- invariant(BL x) where x.nonNull {
-   x.and(true).equal(x);
-   x.or(false).equal(x);
- };

## Invariant Definitions

- All Invariants must always be true
- All have meaning for conformant systems
- 3 kinds of meaning:
- must logically be true
- Mainly mathematical
- i.e. true != false, 1+1=2
- must implement correctly in code
- i.e. set.isEmpty() != set.notEmpty()
- Data rules
- Rules about what is valid data
- i.e. A proper set must have a value

## Base Semantic Invariants

- invariant {
-   true.not.equal(false);
-   false.not.equal(true);
- };
- invariant(INT x, y, z)where
-   x.successor(y).and(y.successor(z)) {
-   x.lessThan(y);
-   y.lessThan(z);
-   x.lessThan(z);
- };

## Must Implement Correctly

- invariant(ANY x) {
-    x.notApplicable.equal(x.nullFlavor
-          .implies(NullFlavor.NA));
- };
- invariant(SET<T> set) where set.nonNull {
-   set.isEmpty.equal(notEmpty.not);
-   set.notEmpty.equal(exists(T element) {
-      set.contains(element);
-   });
- };

## Data Rules

- invariant(ANY x) {
-   x.nullFlavor.implies(
-    NullFlavor.PINF).implies(
-      x.dataType.implies(QTY);
- }
- invariant(BIN x)
-       where x.nonNull {
-    x.notEmpty;
- };

## Invariant Definitions

- All Invariants must always be true
- All define the semantic meaning of the data
- Are the final arbiter in any doubt

## Promotion / Demotion

- template<ANY T>
- Type DSET<T> specializes SET<T>,
- 	COLL<T> {
-   promotion  DSET<T>  (T x);
- };
- template<ANY T>
- type Sequence<T> alias LIST<T> {
-   promotion  LIST<T>  (T x);
-   demotion   BAG<T>;
- };

## Promotion / Demotion

- If X can be promoted to Y
- X can be substituted for Y
- If X can be demoted to Y
- X can be substituted for Y
- This is true in concept
- But difficult to implement
- only allowed in some circumstances

## Literals

- A simple text representation of the data type
- used in the DTDL statements	(true.not.equal(false))
- Sometimes used in the semantics of the definition (precision)
- For definitional purposes
- Not used in production systems
- ITS’s may be guided by the literal form

## Literals

- BAG<T>.literal ST.SIMPLE {
-   BAG<T>          : "{" elements "} { $.equal($2); };
-   BAG<T> elements : elements ";" T  { $.except($2).equal($1); }
-                   | T               { $.contains($1);
-                                       $.except($1).isEmpty; };
- };

| literal | meaning |
| --- | --- |
| {1; 3; 3; 5; 7; 19} | a bag of integer numbers or real numbers |
| {3; 1; 5; 19; 3, 7} | the same bag of integer numbers or real numbers |
| {1.2 m; 2.67 m; 17.8 m} | a bag of discrete physical quantities |
| {"apple"; "orange"; "banana"; "strawberry"; "apple"} | a bag of character strings (for strings, use quoted form to prevent problems with ";") |

## Flavors

- No new semantics
- But some extra rules
- Purposeful limitations
- No need to know the rules to process correctly
- Only one at once
- May always be ignored

- flavor BooleanNonNull alias BN constrains BL;
- invariant (BN x) {
-    x.isNull.not;
- };

## Generic Type Extensions

- template<ANY T>
- type Sequence<T> alias LIST<T>
- 	 specializes COLL<T> {
-   T        head;
- };
- template<ANY T>
- type HistoryItem<T> alias HXIT<T>
- 	 specializes T {
-    IVL<TS> validTime;
-    II controlActIdRef;
- };

## Generic Type Extensions

- May extend any type that specialises the parameter
- HXIT<BL>
- HXIT<SET<T>>
- HXIT<ANY> - still abstract
- Mix-ins
- Powerful technique
- Hard to implement (technology, semantics)
- Extremely limited use

## Architecture & Semantics

## Slide 61

## Morning Tea?

## Slide 63

## ISO Data Types

- Scope
- Normative references, terms and definitions, abbreviations
- Conformance
- Overview of Data types
- Data types in depth
- XML Representation
- Abstract Data Types Mapping
- Schema

## Slide 65

## Introduction and Scope

- provides set of data type definitions for representing and exchanging basic concepts that are commonly encountered in healthcare environments in support of information exchange in the healthcare environment,
- specifies a collection of healthcare related data types suitable for use  in a number of health related information environments,
- declares the semantics of these data types using the terminology, notations and data types defined in ISO 11404 rev 20035,.
- provides UML definitions of the same data types using the terminology, notation and types defined in Unified Modeling Language (UML) version 2.0,.
- defines an XML (Extensible Markup Language) based representation of the data types suitable for use when exchanging information between information processing entities.

## Introduction and Scope

- The requirements which underpin the scope reflected are a mix of requirements gathered primarily from HL7 Version 3 and ISO 11404, and also from CEN 13606, openEHR , and past ISO work on healthcare data types.
- Although the focus with regard to requirements and design approaches has focused on the information interchange, these data types may be adopted for other purposes in healthcare, but this is not mandated by this standard.

## Overview

- What is a data type?
- Definition Methodology
- Note about Names
- General Issues
- Generalization / Specialization
- Enumeration Definitions
- Character Encoding
- Immutability
- Equality
- Audit Trail
- Null Flavors
- Conformance

## Architecture & Semantics

## Data Type Definitions

- UML diagrams
- name,  prose description
- ISO 11404 Syntax
- Attributes (name : type, description / discussion + Notes)
- Equality (prose description of equality)
- Invariants (in OCL, validated)
- Operations (name(params):type, description / discussion + Notes)
- Examples in XML with discussion

## UML Stereotypes

*enumeration*:  standard UML enumeration stereotype
*enum*: standard UML enum stereotype
*constraint*: This UML Classifier describes a data type flavor
*Choice*: denotes that only one of Classifier attributes may be not null
*XMLText*:  Represented as a text node in XML
*Anonymous*:  There is no element for this the XML.
*XMLID*:  This attribute is represented using an XML ID
*XMLIDRef*:  This attribute is refers to an attribute that is represented as an XML ID

## UML Colours

| Colour | Category |
| --- | --- |
| Peach | Proper Types & Flavors |
| Grey | Protected / Private Types |
| Aqua | Enumerations |

## Enumerations

## 11404 Definition

-  type ANY = class (
-    validTimeLow : characterstring,
-    validTimeHigh : characterstring,
-    controlActRoot : characterstring,
-    controlActExtension : characterstring,
-    nullFlavor : NullFlavor,
-    updateMode : UpdateMode,
-    flavorId : characterstring
-  )
- type UpdateMode = enumerated (A, AU, R, AR, N, U, K)

- No Specialization
- “class”
- ISO 11404 primitive types

## Definition of ANY & BL

- Consider in detail
- ANY Notes:
- Not Abstract
- Prose Conformance Statements
- OCL definitions
- Difference in intent of invariants
- BL Notes:
- BL Operations are special
- BL.NONNULL - flavor

## XML

- The UML definitions are good
- UML is widely understood
- UML can be used for software generation
- The UML/OCL definitions are comprehensive
- But need a transfer format
- Everyone must exchange data
- Not everyone uses UML
- Multiple views on mapping UML to XML

## ISO 21090 XML Format	

- ISO 21090 defines it’s own XML format
- Tried to be conservative (standard webservices techniques)
- Failed where a pedantic XML pre-exists and must be followed
- Attributes must be used
- Rules defined in Appendix A (consult)

## ISO 21090 Conformance

- “Information Processing Entity”
- anything which processes information and contains the concept of data type.
- other standards
- Specifications
- data handling facilities and services
- Etc
- Can be conformant to ISO 21090
- Directly
- Indirectly
- May make a “conformance statement”

## Direct Conformance

- Uses Healthcare Data types as defined in ISO 21090
- Attributes
- Invariants
- Operations
- Conformance Statements
- XML format rules (may define namespace)

## Direct Conformance

- IPE’s claiming direct conformance may:
- Constrain the data types
- i.e. validTimeLow & validTimeHigh SHALL be null
- Not make use of some data types
- Do not use URG
- Add other operations to data types
- i.e. Decorate ST with java.lang.String operations
- Use other representations for the concepts as well
- i.e. CEN 13606 has other ways of doing address. These are still allowed

## Indirect Conformance

- Defines Mappings to Healthcare Data types as defined in ISO 21090
- Inwards (from 21090 to internal types)
- Outwards (From internal types to 21090)
- Must be specific about is mapped

## Conformance

- Direct Conformance
- HL7 V3 claims direct conformance to ISO 21090
- CEN 13606 will claim direct conformance to ISO 21090
- Eclipse OHF V3 code will claim direct conformance to ISO 21090
- Indirect Conformance
- OpenEHR will claim indirect conformance to ISO 21090 (I will publish the mappings)

## ISO Data Types Guide

- Part of XML ITS R2 Guide
- Sections:
- Conformance Statement
- How extra constraints are done
- Change Summary from XML Data types R1
- Design Philosophy

## 21090 Conformance Statement

| 
HL7 V3 declares direct conformance to the ISO data types standard
 |  |
| --- | --- |
| Supported Data types | All Data types defined in the ISO 21090 are supported by HL7 V3 |
| Value Spaces | All the value spaces of the types are supported. 
HL7 V3 almost always further constrains the data types.  |
| Operations | V3 abstract data types defines additional operations, but they are either derived from or consistent with the operations defined in 21090. |
| XML | XML format is that described in the ISO data types. 
The namespace is “urn:hl7-org:v3” |

## Constraints Guide

| ISO Features | Constraint |
| --- | --- |
| HXIT properties | HXIT properties are fixed to null unless the generic type extension HXIT is part of the type (either directly, or applied using HIST) |
| ANY.nullFlavor | Must be used consistently within the V3 conformance framework |
| ANY.updateMode | Fixed to null unless the static model (constraining model) defines allowable values for updateMode |
| ANY.flavorId | May only be used consistently with the rules in Refinement, Constraint and Localisation |
| SC.code | The code is always constrained to a CD.CV within the context of HL7 V3 |
| QTY.expression & QTY.expressionLanguage | These are fixed to null unless the generic type extension EXPR is part of the type |
| QTY.uncertainty & QTY.uncertaintyType | These are fixed to null unless the generic type extension PPD is part of the type |

## XML Changes

- 44 Changes to XML format. Reasons:
-  Changes to the data type semantics due to R2 proposals
-  Fixing mistakes in the R1 definitions
-  Changes to the XML format to introduce more consistency (to assist with automated processing)
-  Deliberate changes to the XML format to resolve known issues
-  Changes to allow for harmonization in ISO
-  A desire to represent more of the constraints from the abstract model in the schema (mostly as schematron)

## Design Philosophy

- ISO 21090 Design Philosophy
- Platform Independence, UML
- ITS R2 Design Philosophy
- What was wrong with R1
- Platform Independence, UML(!)
- Relationship between Abstract and ISO
- ISO is an implementation of Abstract
- Interface / Class
- Implementation is incomplete
- Attributes
- Reasons why attributes chosen

## ISO / Abstract Mapping

## Slide 89

## Criticisms and Alternatives

- The most trenchant critic of the V3 Data types is Thomas Beale (OpenEHR/Ocean Informatics)
- OpenEHR has it’s own data types, partially based on V3 data type considerations
- Other alternatives?
- CEN / ISO in a mess
- HL7 V2.....

## Criticisms of V3 Data Types

- Abstract Spec is not appropriate
- No published requirements
- Naming (1): clashes with built-in types
- Naming (2): acronyms & words
- BL is a tristate, not a bistate
- ST/ED inheritance
- ED/BIN inheritance
- Flavours of null design approach
- Flavours of null vocabulary

## Criticisms of V3 Data Types

- CodedOrdinal not a quantitative type
- CD uncontrolled post-coordination (qualifiers)
- Interval<T> & Set<T> modelling
- Update mode
- Identifiers
- Semantics of partial date/times defined?
- Convertability of T/Set<T>/IVL<T>
- Is ‘tablets’ a kind of unit?
- Quantities versus quantity specifications

## OpenEHR Data Types

- Support
- “Assumed” Data types
- Primitives from other technologies
- Various particular constraints for particular applications

## OpenEHR Primitive types

## OpenEHR Primitive Types

## OpenEHR Assumed Types

## OpenEHR Identification Types

## Open EHR Data Types

## Open EHR Data Types

## DV_BOOLEAN

## OpenEHR Data Types

- Are the most serious alternative to the V3 Data types
- Are deeply enmeshed in the openEHR Paradigm
- Are well documented and well designed
- Are always worth consulting
- R2 Data types have been informed by the openEHR data types, and have migrated towards them
- We will consult them occasionally later

## HL7 V2 Data Types

- Well, Data something or others
- Backwards Compatibility for ever
- Field / Component / Sub-Component:Types cannot nest properly
- Nothing can be fixed. Ever.
- Just add more components
- Variations on a theme: multiple different ways to do identification, naming, concepts.
- Committees work hard to clean up, but backwards compatibility.....

## HL7 v2 Overlapping Data Types

**CF - coded element with formatted values**
- CNE – coded with no exceptions
**CWE – coded with exceptions**
- ID - coded value for HL7 defined tables
- IS - coded value for user-defined tables
- AD – address
- SAD – street address
- XAD - extended address
- LA1 - location with address variation 1
- LA2 - location with address variation 2
- CNN - composite ID number and name simplified
- CX - extended composite ID with check digit
- EI - entity identifier
- EIP - entity identifier pair
- HD - hierarchic designator
- XCN - extended composite ID number and name for persons
- XON - extended composite name and identification number for organizations

## HL7 V2 CWE

| SEQ | LEN | DT | OPT | TBL# | COMPONENT NAME | COMMENTS | SEC.REF. |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | (12..20) | ST | O |  | Identifier |  | 2.A.75 |
| 2 | (0..199) | ST | O |  | Text |  | 2.A.75 |
| 3 | (20..20) | ID | C | 0396 | Name of Coding System |  | 2.A.35 |
| 4 | (1..20) | ST | O |  | Alternate Identifier |  | 2.A.75 |
| 5 | (1..199) | ST | O |  | Alternate Text |  | 2.A.75 |
| 6 | (20..20) | ID | C | 0396 | Name of Alternate Coding System |  | 2.A.35 |
| 7 | (1..10) | ST | C |  | Coding System Version ID |  | 2.A.75 |
| 8 | (1..10) | ST | O |  | Alternate Coding System Version ID |  | 2.A.75 |
| 9 | (1..199) | ST | O |  | Original Text |  | 2.A.75 |
| 10 | (1..20) | ST | O |  | Second Alternate Identifier |  | 2.A.75 |
| 11 | (1..199) | ST | O |  | Second Alternate Text |  | 2.A.75 |
| 12 | (20..20) | ID | C | 0396 | Second Name of Alternate Coding System |  | 2.A.35 |
| 13 | (1..10) | ST | O |  | Second Alternate Coding System Version ID |  | 2.A.75 |
| 14 | (1..199) | ST | C |  | Coding System OID |  | 2.A.75 |
| 15 | (1..199) | ST | O |  | Value Set OID |  | 2.A.75 |
| 16 | (4..24) | DTM | C |  | Value Set Version ID |  | 2.A.22 |
| 17 | (1..199) | ST | C |  | Alternate Coding System OID |  | 2.A.75 |
| 18 | (1..199) | ST | O |  | Alternate Value Set OID |  | 2.A.75 |
| 19 | (4..24) | DTM | C |  | Alternate Value Set Version ID |  | 2.A.75 |
| 20 | (1..199) | ST | C |  | Second Alternate Coding System OID |  | 2.A.75 |
| 21 | (1..199) | ST | O |  | Second Alternate Value Set OID |  | 2.A.75 |
| 22 | (4..24) | DTM | C |  | Second Alternate Value Set Version ID |  | 2.A.22 |

## HL7  V2 Data Types

- Source of Requirements
- Sift through the chaff of self-imposed exceptions
- On going convergence with v3
- Over my dead body (!)
- Come to my funeral
- But not a good basis for design

## V3 Data Types Architecture

- Scope of Data types
- Type Issues
- Incompleteness – NullFlavors
- Identity & Equality
- Modes of Use – Update Mode & History
- Character Sets & Unicode
- Translations
- Conformance & Cardinality
- Flavors

## Scope of Data Types

- Question:
- What things should be data types?
- What’s a data type? How are they different?

## Abstract Data Types says...

- What, then, is the difference between a data type and a message, document, or business object?
- Data type values stand for themselves: the value is all that counts. Neither identity nor state or changing of state is defined for a data value.
- Conversely in business objects, we track state and identity; the properties of an identical object might change between now and later. Not so with data values: a data value and its properties are constant.

## Example

- number 5 is always number 5: there is no difference between this number 5 and that number 5 (no identity distinguished from value), number 5 never changes to number 6 (no change of state).
- Data types are immutable objects where identity does not matter (identity and equality are the same).

## ISO Data Types says:

- Generally the definitions of the scope of data types revolve around either or both of the following notions:
- Immutability (the properties of the data type cannot change, instead a new instance is created: data types have no lifecycle)
- The relationship between equality and identity (if two data types are equal they are the same instance)
- Coherency of a single concept (each data type should represent a single concept space)

## ISO Data Types says:

- Application of principles is a matter of perspective
- Based on past work (i.e. No good reason)
- Process has been deliberately inclusive

## Obvious Candidates

- Boolean
- Integer
- Real
- URLs
- Dates and Times
- Physical Measurements
- Monetary Values
- If you change them, they become a “different value”

## Not so Obvious

- LIST
- SET
- String
- This starts being an implementation choice – if you insert a character in a string, do you get a new value?
- In Java: String, StringBuilder, StringBuffer...
- In UML/OCL: All are immutable

## Not So Obvious

- II
- CD
- When you change properties, do you have a new code?
- Or would you rather define a lifecycle with rules
- Can’t change the code or code-System
- Can change the displayName
- We choose to consider them immutable

## Controversial

- Names & Addresses
- Does your address change while still being the same address?
- Are different representations the “same value”?
- Grahame D Grieve
- Grieve, Grahame David
- Junior (nick)
- We say: we are not interested in defining lifecycle and reference identity: they are data types

## And then there’s EXPR...

- PQ is a data type
- Is an expression that tells you how to calculate a data type a value that doesn’t change?
- Is it a value that is equal to the calculated value?
- Hands up volunteers to define the lifecycle...

## The trouble with translations

- The following types have translations:
- CD
- ED
- PQ
- These are representations of the concept in a different concept representation system (Code System, language, units system)
- Does changing the translations change the value?

## Type Issues

- Wrapping
- Specialization
- Promotion / Demotion
- Substitution

## Primitive Type Pattern

- Desire primitive values with common types:Booleans, integers, floats, strings (?)
- Need to bring them into a consistent  architecture
- Architecture is established by gen/spec from ANY
- Abstract: Define a representation of the primitive type, give it the expected operations, specialize ANY
- ISO: Define a class with a standard name, specialize ANY, give it an attribute value with appropriate choice from technology’s type library

## Primitive Type Pattern

## Names

|  | BL | INT | REAL | ST |
| --- | --- | --- | --- | --- |
| V3 Wrapper | BL | INT | REAL | ST |
| Schema | boolean | integer | double + | string |
| UML | Boolean | Integer | Real | String |
| Java | bool/Boolean | int/Integer | float/Float | String |
| C# | bool | int | float | String |
| Pascal | boolean | Integer | Real/Double | String |
| 11404 | Boolean | Integer | Real | CharacterString |

## So much trouble

- “The types INT (Integer), REAL (Real), Set<T>, Bag<T>, BL clash with inbuilt types of the same names in ISO 11404 & most implementation technologies. This has caused a lot of confusion. Such inbuilt types do not inherit from the HL7 type Data_value”
- Duh. Names might clash. Deal with it. In your own way

## Specialization

- “Gen/Spec”
- Has normal UML meaning
- Often criticised: “Normal UML use does not let you take things away”’
- i.e. ST “takes away reference”(*x.reference.notApplicable*)
- But this is just a fixed value, like *x.mediaType.code.equal("text/plain");*
- This is a time honoured OO principle

## Promotion/Demotion

- Convert from one type to another
- A single item can be converted to a list or a set with a single item
- An integer can be converted to a real and vice versa
- Sometimes some default information must be added (promotion)
- Sometimes some information lost (demotion)
- Should be seamless (Real / Int)
- Common Practice

## Promotions

- ST  EN
- INT  REAL
- REAL  PQ
- BAG<T>  SET<T>
- QTY  IVL<QTY>
- QTY  RTO<QTY, INT>
- ANY  LIST<ANY>
- ANY  BAG<ANY>
- ANY  SET<ANY>

- Type REAL {
-   REAL negated;
-   demotion INT;
-   promotion REAL (INT x);
-   promotion PQ;
- };

## Demotions

- EN  ST
- REAL  INT
- PQ  REAL
- LIST<T>  BAG<T>
- IVL<QTY>  QTY
- RTO<N, D>  REAL
- RTO<N, D>  PQ
- HIST<T>  HXIT<T>
- QSET<T>  LIST<IVL<T>>

- type RTO<N, D> {
-   demotion  REAL;
-   demotion  PQ;
- };

## Promotions / Demotions

- ST  EN
- INT  REAL
- REAL  PQ
- BAG<T>  SET<T>
- QTY  IVL<QTY>
- QTY  RTO<QTY, INT>
- ANY  LIST<ANY>
- ANY  BAG<ANY>
- ANY  SET<ANY>

- EN  ST
- REAL  INT
- PQ  REAL
- LIST<T>  BAG<T>
- IVL<QTY>  QTY
- RTO<N, D>  REAL
- RTO<N, D>  PQ
- HIST<T>  HXIT<T>
- QSET<T>  LIST<IVL<T>>

## Substitution

- If an element has an assigned type T:
- You can use any type that is a specialization of T
- You can use any type that can be demoted to T
- You can use any type that can be promoted to T
- All these rules are transitive, can link them along. (Game: what can you substitute for T?)

## Substitution

- Actually, the Data types spec does not mandate what can be substituted
- The spec mandates how things can be done
- The context of use mandates what is expected
- V3 Messaging / CDA: A set list is published in Refinement, Constraint, and Localization

## V3 Allowed Substitutions

| Type | Allowable substitution types |
| --- | --- |
| ANY | Any data type, including 
Generic Type Extensions |
| ED | ST, UID |
| ST | UID |
| CD | CV, CE, CO |
| CE | CV, CO |
| SC | ST |
| EN | PN, ON |
| PN | TN |
| ON | TN |
| QTY | TS, MO, REAL, INT, TS, RTO <QTY, QTY>, PQ |
| REAL | INT |
| PQ | REAL |

| Type | Allowable substitution types |
| --- | --- |
| SET<T> | T, substitutions of T, LIST<T> and IVL<T> |
| LIST<T> | T |
| GLIST<T> | T |
| SLIST<T> | T |
| BAG<T> | T |
| IVL<T> | T |
| HIST<T> | HXIT<T> |
| UVP<T>  | T |
| NPPD<T> | T |
| PPD<T> | T |
| EIVL<TS> | IVL<TS> |
| PIVL<TS> | IVL<TS> |
| QSET<TS> | SET<PIVL<TS>>, SET<EIVL<TS>> |

## Reference vs Constraint

- General approach has problems
- You can’t have your cake and eat it too
- Magnitude of impact not fully understood
- openEHR is doing exactly the same thing

## NullFlavor

- Incomplete / Partial Data is common in healthcare
- Patient is unconscious
- Patient was unwilling to provide information
- Data is not (yet?) available
- User does not have access to the information
- Other system does not collect or share data item
- Actual value is not a proper value

## Not a proper value?

- Main case: Coded values
- Code must come from a code system
- Code system doesn’t contain concept
- Text alternative is “not a proper value”
- Other cases:
- Infinity (+/-) is not a valid integer
- A trace detected
- Water to 100mL (in a formulation)

## Architectural Choice

- How to handle this?
- By Exception
- extra field for providing a reason when appropriate
- Infrastructurally
- Slot for reason is always present even when not appropriate
- May be banned when inappropriate
- Choice: what does always present mean?

## HL7 Solution: nullFlavor

- Every data element (Data Type or RIM Class) may have a nullFlavor
- Abstract:
- The nullFlavor values are part of the value domain of every type or class
- ISO
- Every type or class has a nullFlavor attribute

## nullFlavor != null

- Types or classes that have a nullFlavor may also have other information:<CD nullFlavor=“OTH” codeSystem=“SNOMED”>   <originalText>Burnt Ear</originalText></CD>
- null in implementation technologies (SQL null, OCL null, pointers) mean that no extra information is found
- nullFlavor is not the same as null!

## Why “null”Flavor?

- If it’s not the same as null, why give it such a confusing name?
- “ExceptionalValueStatus” is a pig to say
- It is connected to the notion of a null value in v2 (Internal Continuity)
- It *behaves* like an SQL or OCL null (External Continuity)

## SQL null

| Key | Name | Count |
| --- | --- | --- |
| 1 | USA | 0 |
| 2 | Canada | 1 |
| 3 | Australia | null |

## HL7 null and equal

- HL7 null values’ equality is unknown<CD nullFlavor=“OTH” codeSystem=“SNOMED”>   <originalText>Burnt Ear</originalText></CD><CD nullFlavor=“OTH” codeSystem=“SNOMED”>   <originalText>Burnt Ear</originalText></CD>
- HL7 null values may be not equal<INT nullFlavor=“PINF”/><INT nullFlavor=“NINF”/>  Not Equal<INT nullFlavor=“UNK”/>   Unknown<INT nullFlavor=“PINF”/>  No Information

## OCL Null

- OCL null values propagateCD.displayName.language.equal(x) == null
- Unless special OCLType properties are used:CD.displayName.language.isNull == true
- HL7 null values work exactly the same way: CD.displayName.language.equal(x) == NI CD.displayName.language.isNull == true

## Null = NullFlavor.NI

- When a value is not present (i.e. null) in an implementation, the nullFlavor NI (No information) is considered to apply.
- No information = no information

## NullFlavor

- NullFlavor does not mean “null”
- But behaves like it in many ways
- Every data type (including properties) implicitly carry information about how or why they are incomplete or an improper value
- Every data type may carry other information even when null
- Implementers must always consider null value!

## NullFlavor Domain

| code | Name |
| --- | --- |
| NI | no information |
|   INV | Invalid |
|     OTH | Other |
|       NINF | negative infinity |
|       PINF | positive infinity |
|     UNC | Unencoded |
|     DER | Derived |
|   UNK | Unknown |
|     ASKU | asked but unknown |
|       NAV | temporarily unavailable |
|     QS | sufficient quantity |
|     NASK | not asked |
|     TRC | Trace |
|   MSK | masked |
|   NA | not applicable |

## Related Properties

- nullFlavor : CS – the actual code
- isNull : BN – whether value is null
- nonNull : BN – whether value is not null
- notApplicable : BL – whether value is NA
- unknown : BL – whether value is UNK
- other : BL – whether value is OTH
- value is null if nullFlavor code is nonNull
- value is OTH if nullFlavor code is OTH

## Rules for nullFlavors

- invariant(ANY x) {
-   x.nullFlavor.implies(NullFlavor.PINF).implies(x.dataType.implies(QTY);
-   x.nullFlavor.implies(NullFlavor.NINF).implies(x.dataType.implies(QTY);
-   x.nullFlavor.implies(NullFlavor.QS).implies(x.dataType.implies(QTY);
-   x.nullFlavor.implies(NullFlavor.TRC).implies(x.dataType.implies(QTY);
-   /* if whatever type x is doesn't have an originalText property,
-      the invariant will be null (not true) */
-  x.nullFlavor.equals(UNC).implies(((x.dataType)x).originalText.nonNull);
-   x.nullFlavor.equals(DER).implies(x.dataType.implies(EXPR<QTY>)
-     .and((EXPR<QTY> x).expression.nonNull));
- };

## Null is not Null

- Consider attribute nullFlavor
- Domain values of the attribute are the nullFlavor codes
- Like all types, the nullFlavor values are also in the domain values
- The nullFlavors are in it’s domain values twice: once as codes, once as values
- i.e. The value “NI” has the exact opposite meaning of the code “NI”

## NullFlavors & Conformance

- Although Null Flavors are called “exceptional” they are not exempt from the rules: they SHALL conform to all the formal statements in the data types.

- invariant(ANY x) {
-  x.nonNull.equal(x.isNull.not);
- };
- invariant(ANY x) where x.nonNull {
-  x.dataType.nonNull;
- };

## NullFlavors and Conformance

- NullFlavors are tied to the conformance framework
- A null value (has a nullFlavor) is an exception to the value domain assigned in the context of use
- (never an exception to the value domain assigned in the data types)
- A mandatory value cannot be null (must be a value from the value domain)

## NullFlavors and Conformance

- If value domain is 1-100 and the attribute is not mandatory, can send<INT nullFlavor=“NI” value=“120”/>
- What is an implementation supposed to do?
- Data types specification does not say. You can choose.

## NullFlavors and Types

- All Data Values have a Data Type
- All non-null values have a type not ANY
- All values from the NullFlavor Domain have a type (not like OCL)
- May be of type ANY
- Attribute type is ANY (i.e. Observation.value  : ANY)
- Attribute value is null in implementation
- What type does it have?
- ANY is abstract in Abstract Spec, but concrete in the ISO data types

## NullFlavors and Properties

- Abstract Spec:“NULL-flavors are applicable to any property of a data value or a higher-level object attribute. Where the difference of null flavors is not significant, ITS are not required to represent them. If nothing else is noted in this specification, ITS need not represent general NULL-flavors for data-value properties.”

## NullFlavors and Properties

## NullFlavors and Properties

- Sometimes the nullFlavor just doesn’t matter
- The ITS doesn’t need to convey it.
- It defaults to NI
- This applies to other properties too
- The ITS simplifies many properties to primitives where some of the semantic properties are inferred

## ITS Simplification

- No nullFlavor:
- root
- extension
- displayable
- identifierName
- scope
- reliability
- No language:
- identifierName

## Nested nullFlavors

- What nested nullFlavors are allowed?
- Translations always have their own nullFlavor
- originalText has it’s own nullFlavor
- QSETs always have their own nullFlavor
- LIST, SET, BAG elements (not inside data types except for translations)
- Ratio Numerator and Denominator
- Complex properties always have their own nullFlavor by definition, but mostly prohibited in ITS

## Criticism of NullFlavors

- Needed only for values from outside world (Element.value, Obs.value); not structural attrs. DTs force them everywhere
- NF only make sense once for a data value
- Only useful if actually set. One example: who is going to set NAV (temporarily unavailable); how. Will they do it for all 20 items on the screen?
- Consequences for systems using HL7 NFs:
- If NF are optional at data entry, they’ll never be set
- If mandatory, they cost time, require a decision, else just choose any old value for expediency
- Can only work if settable by the software

## OpenEHR NullFlavors

- Every element has two properties:
- nullFlavor
- Value : DV_ANY
- No NullFlavors inside ANY
- Properties are still nullable (but less often)
- NullFlavor only allowed if author explicitly makes it so

## OpenEHR Differences

- NullFlavor not allowed where it should be
- Implementer must still check for null but slightly less often
- nullFlavor may be lost from null
- Back to the original question:
- Handle by Exception
- Handle by Infrastructure
- HL7 forces you to always ask null *and nullFlavor*

## NullFlavor Criticisms (2)

- Vocabulary wrongly includes +infinity and –infinity, which are mathematical value concepts, and have nothing to do with lack of data*Comment: NullFlavor is not about lack of data*
- Values other than UNK, NA, NI are probably of no use; guidance is needed on how they could be set*Comment: yeah, definitions are confusing*

- _[http://www.openehr.org/wiki/display/spec/Null+Flavours+and+Boolean+data+in+openEHR](http://www.openehr.org/wiki/display/spec/Null+Flavours+and+Boolean+data+in+openEHR)_

## More nullFlavors?

- This data item was not possible to capture, on this occasion or indefinitely, because of this patient’s clinical situation e.g. amputation or bandaged injury preventing measurement of a foot pulse.
- This data item was omitted (as a clinical judgement) because of this patient’s clinical/social situation e.g. to avoid psychological distress, or at the patient’s request.
- This data item was omitted because equivalent information had already been captured and was present in the patient’s record e.g. height.
- This data item was not populated because relevant (background) information was not available at the time e.g. the patient could not recall a fact or date, a relevant measurement result or instruction was not to hand to permit a procedure to be performed.
- This data item was not captured because logistic or facility resource problems prevented the clinical workflow step from being performed to capture it e.g. not enough time, equipment faulty, nobody to facilitate language translation with patient, no female chaperone.
- This data item was pending at the time of committing the data set: the clinical workflow item was performed but the data value needed as the outcome was not yet available.
- The most correct response for this data item is null: i.e. it is a positive assertion that this this data item does not have a value.
- This data item was omitted for other reasons - which need to be elaborated.

## Identity and Equality

- “identity and equality are the same”
- This is not true!
- HL7 defines equality very carefully for the data types

## Simple Example

- A:
**public**** ****boolean**** equals(Object other) {**
**  ****return**** other ****instanceof**** A && **
**   ****((A) other).****a**** == ****a****;**
- }
- B:
**public**** ****boolean**** equals(Object other) {**
**return**** ****super****.equals(other) && **
**    ****other ****instanceof**** B && **
**   ****((B) other).****b****.equals(****b****);**
- }

## JavaDoc about equals

- The equals method implements an equivalence relation on non-null object references:
- It is *reflexive*: for any non-null reference value x, x.equals(x) should return true.
- It is *symmetric*: for any non-null reference values x and y, x.equals(y) should return true if and only if y.equals(x) returns true.
- It is *transitive*: for any non-null reference values x, y, and z, if x.equals(y) returns true and y.equals(z) returns true, then x.equals(z) should return true.
- It is *consistent*: for any non-null reference values x and y, multiple invocations of x.equals(y) consistently return true or consistently return false, provided no information used in equals comparisons on the objects is modified.
- For any non-null reference value x, x.equals(null) should return false.

## Data Types About Equals

- A reflexive, symmetric, and transitive relation between any two data values.
- invariant(ANY x, y, z) where
-    x.nonNull.and(y.nonNull).and(z.nonNull) {
-  x.equal(x);                   /* reflexivity */
-  x.equal(y).equal(y.equal(x)); /* symmetry */
-  x.equal(y).and(y.equal(z)).
-           implies(x.equal(z)); /* transitivity*/
- };

## Eqaulity & Polymorphism

## Identity & Equality

- Data type definitions are very careful to make equality reflexive, symmetric, and transitive
- Data type definitions also deal with nullFlavors at length
- Data type definitions also exclude some properties
- Required to maintain symmetry
- Not relevant to semantic equality

## Equality for CD

- invariant(CD x, y) where x.nonNull.and(y.nonNull) {
-   x.equal(y).equal(x.code.equal(y.code)
-        .and(x.codeSystem.equal(y.codingSystem));
- };
- The equality of two *CD* values is determined solely based upon code and codeSystem.
- codeSystemVersion is excluded from the equality test.
- displayName is excluded from the equality test
- Translations are not included in the equality test.
- Exceptional *CD* values are not equal even if they have the same NULL-flavor or the same original text.

## Equality for CO

- The definition of equality for *CO* is taken from the CD data type.
- The value property is not included in the equality test.
- This means that values of type CO and CD may be equal
- i.e. SET<CO> may contain a CD value

## Identity

- There’s still a place for real identity (Java “=” vs .equals())
- invariant(SET<T> x)
-       where x.nonNull {
-  forall(T a) where a.nonNull {
-    forall(T b) where b.nonNull {
-      /* if the comparator cannot compare a or b,
- 	    both cannot be in the Set */
- 	 x.contains(a).and(x.contains(b)).implies.
- 	    x.comparator.compare(a, b).nonNull;
-      /* if both a and b are in the set, they must be different
-        (unless they are the are the same instance)*/
- 	 x.contains(a).and(x.contains(b)).implies(
- 	   a.identical(b).or(x.comparator.compare(a, b).not));
- 	 }
-    }
- };

## Update Mode

- Data types deal with semantics of the information
- HL7 Modellers and implementers are also interested in metadata about the information
- What changed this time?
- When did it change?
- Who changed it?
- Do I have to do anything about it?

## 3 ways to exchange

- Snapshot
- Sending system sends everything it has
- Receiving system merges with everything it has
- UpdateMode
- Sending system sends what has changed,  all explicitly marked
- Receiving system makes same changes
- Reference
- Sending system sends enough to identify
- Receiving system finds the object

## Snapshot

- When destination system identity or state is not known
- Receiving application
- matches objects
- appropriately process the information (e.g. merge all the attributes and associations of the objects).
- Potential Advantages:
- Can be easier for senders to implement
- Many sending systems implement Version 2 messages in this fashion
- Potential Disadvantages
- Typically more complicated for receivers to process appropriately
- Easier for relevant data to be deleted

## Update Mode

- When Destination system is well known + there is an implicit or explicit contract
- changes may be additions, deletions, and revisions to existing data.
- Potential Advantages (depending how it is used):
- reduced instance size
- Easy to determine what changes the sender has made : reduced processing time
- No need to store ‘images’ of data determining change received from a particular sender
- Simpler implementation decision making
- Easy to audit changes
- Potential Disadvantages:
- systems can get out of sync : be careful.
- More effort for the source system

## Update Mode

- Alternatively:
- same data items as for snapshot node
- marks each value and indicate whether
- it is added,
- replaces another item,
- or has not changed (but not deleted)
- Normal mode in V3 is snapshot
- update mode is only allowed when the constraining model allows

## UpdateMode values

| code | name | definition |
| --- | --- | --- |
| A | Add | The item was (or is to be) added, having not been present immediately before. (If it is already present, this may be treated as an error condition.)  |
| D | delete | Delete any previously received value. If the item is part of a collection, delete any matching items. |
| R | replace | The item existed previously and has (or is to be) revised. (If an item does not already exist, this may be treated as an error condition.)  |
| AR | Add or Replace | The item was (or is to be) either added or replaced.  |
| N | No Change | There was (or is to be) no change to the item. This is primarily used when this element has not changed, but other attributes in the instance have changed.  |
| U | Unknown | It is not specified whether or what kind of change has occurred to the item, or whether the item is present as a reference or identifying property.  |
| REF | reference | This item provides enough information to allow a processing system to locate the full applicable record by identifying the object.  |
| K | key | This item is part of the identifying information for this object. |

## UpdateMode and Data Types

- REF cannot apply to data types
- There is no way to refer to a data type
- Update Mode cannot apply to data types: they are immutable
- Except (!):
- UpdateMode may be applied to items within a DSET, BAG, or LIST

## Attributes and Collections

- Attributes with Cardinality > 1
- How is the collection handled: implicitly or explicitly?
- UML/OCL is ambiguous in this regard

## V3 is (Nearly) Clear

- There is only ever instance of an attribute
- Attribute cardinality is 0..1 or 1..1
- If the attribute type is a specialisation of COLL<T>, then cardinality can be >1 (i.e. 0..*)
- Except for updateMode. Update Mode, which is a property of attributes, can apply to each element in attribute collections
- Can add, replace, and delete in collections

## Implementing Mix-ins

- Generic Type Extensions: HXIT<ANY T> extends T
- So there can be HXIT<anything>
- Even AOP doesn’t quite implement Generic Type Extensions in the way the abstract data types uses them (like Java & Generics)

## HXIT Implementation

## HXIT Implementation

- ANY specializes HXIT
- HXIT properties are always available
- Not always able to be used (only when type is HXIT<T>)
- Not always the appropriate solution
- Consider each mix-in independently

## Mixins and equality

- x.equals(y).equals(y.equals(x))
- PPD<T> - adds the notion of uncertainty
- so is PPD<T> = T?
- is 3 +/1 = 3?
- is 3 +/- 1 = 2.8 +/- 0.05?
- How do you define this?

## Audit Trail

## HXIT.validTime

- The time interval during which the given information was, is, or is expected to be valid.
- 	The interval can be closed-- i.e. finite and defined—or open—i.e. infinite or undefined —on either side.
- 	The interval cannot be just a width.

## HXIT. controlActIdRef

- The identifier of the event associated with setting the data type to its specified value.  By referencing a particular event, the property links to all of the information surrounding that event, such as who made the change, when it was made, why it was made, what system originated the change, etc.

## HXIT.validTime

- validTime is the time the information was *actually* valid (i.e. When a particular name was actually used)
- not the time the information was considered to be valid
- EN has validTime too – this is the same attribute (err, maybe)
- Have to consult the event history to determine when the information was considered to be valid (not for faint-hearted)

## Unicode & Character Sets

- Character Set : a logical set of characters, each assigned an identifier and a glyph
- Character Encoding : how you represent a character set in a sequence of bytes

## Unicode

- Unicode is a character set
- UTF-8, UTF-16BE, UTF-16LE, UTF-32 are all character encodings
- All are unicode, just different efficiencies with different languages
- Unicode is not a stateless character encoding
- It is a complex structure with planes, bi-di controls, and other things

## Japan and Unicode

- The Japanese have big problems with the whole notion of a fixed character set
- The Japanese have big problems with Unicode, and refuse to use it
- Nevertheless, main OSes use unicode internally, and are used in Japan (but some hacking involved)
- Japanese prefer Shift-JIS encoding (mix ASCII, and variety of other character sets and encodings with state change sequences)
- _[http://www.jbrowse.com/text/unij.html](http://www.jbrowse.com/text/unij.html)_

## Data Types and Unicode

- Abstract does not use Unicode
- ST is a series of bytes.
- mediaType = text/plain
- charset must be known – but not fixed

## Character Sets

| code | definition |
| --- | --- |
| EBCDIC | HL7 is indifferent to the use of this Charset. |
| ISO-10646-UCS-2 | Deprecated for HL7 use. |
| ISO-10646-UCS-4 | Deprecated for HL7 use. |
| ISO-8859-1 | HL7 is indifferent to the use of this Charset. |
| ISO-8859-2 | HL7 is indifferent to the use of this Charset. |
| ISO-8859-5 | HL7 is indifferent to the use of this Charset. |
| JIS-2022-JP | HL7 is indifferent to the use of this Charset. |
| US-ASCII | Required for HL7 use. |
| UTF-7 | HL7 is indifferent to the use of this Charset. |
| UTF-8 | Required for Unicode support. |

## ISO Data Types

- ST contains a sequence of logical characters
- By default, ST contains Unicode characters.
- Unicode SHOULD be used
- For Japan (and a few others): regulatory domains which may mandate some other character set than Unicode.
- Note to count costs properly in this case
- All character encodings may contain control bytes

## Translations

- CD, PQ, and ED all contain translations
- All contain translations for different reasons
- Slightly different design and implementation patterns
- All present similar challenges
- Specifying Translations
- Choosing Root / Conformance

## CD

## ED / ST

## PQ

## PQR Implementation

*Abstract is broken on source()*

## Specifying Translations

- What is a translation?
- ED: same content, different language or mediaType (e.g. English / Spanish, PDF/HTML, Video / Document)
- CD: different code/codeSystem
- PQ: non-UCUM units
- Do you need to know what is translated from what?
- Very controversial

## Tracing Translations

- Source : X : allow sender to specify which was the source
- codingRationale : allow sender to specify why translation is provided

| code | name | definition |
| --- | --- | --- |
| O | Original | Originally produced code |
| P | Post-coded | Post-coded from free text source |
| R | Required | Required standard code for HL7 |

## Which is Root?

- Root is the one to which conformance expectations apply
- CD – the code (if one exists) that is in the specified domain
- PQ – the one with the UCUM code
- ED – no guidance
- In the past, root was the first so tracing was possible (introduced source instead)

## Coffee?

## Conformance

- Two meanings to conformance
- Does the Information Processing Entity conform to this specification?
- How does a standard or specification make rules about the data types
- The Data types define what rules can be applied

## Conformance Properties

| Property | Type | Meaning |
| --- | --- | --- |
| Mandatory | ANY | a mandatory data item cannot have a nullValue (no nullFlavor, and no null in implementation)
 |
| Cardinality | COLL<T> | specifies how many items may be in the one collection
 |
| Domain | CD | specifies the vocab domain from which the root code must be taken from
 |

## Conformance & Translations

- Translations are not really subject to conformance
- By convention, code domains specified in templates apply to translations, but waiting for implementation comment on this one.

## Other constraints

- Use ST Literal to fix ST, UID, INT, REAL, PQ, CS, BL
- CfH: II fixed value implies II.root fixed value
- Constraint Languages:
- OCL
- HPath
- ADL
- Schematron
- Data Type Flavors

## Constraint Languages

- First Problem:  Abstract or ITS?
- Abstract Spec
- Avoids implementation specific artefacts
- Supports Re-use across technologies
- Whole point of modelling
- Where is the data?
- Technology Specific
- They can be tested and can be useful
- Generally limited reuse
- Different tools
- Different models

## OCL (ISO)

- OCL can be used directly on the ISO data types
- ISO data types standard is full of it
- Can say anything that OCL allows
- Powerful for collection related operations (OCL iterators way cool)
- Implementation Specific: ISO data types UML model only

- Context CD: displayName().language().isNull()

## OCL (Abstract)

- OCL can be used against the abstract data types
- Have to define an OCL mapping. Issues
- Equality & Identity + nullFlavors
- Type rules + type spoofing for primitives
- InFix operators
- Collection spoofing
- Class Factory + utility functions from OCL
- More Type spoofing to do with cloning
- Association Name spoofing
- I’d have to rewrite my OCL engine to violate basic UML rules

- Context CD: displayName().language().isNull()

## HPath

- Gunther & Grahame designed this on a napkin at lunch one day
- Simple premise:
- XPath-like statements (JPath)
- Use abstract properties & string literals instead of element/attribute syntax
- Statements attached to attributes
- JavaSIG has an implementation
- Purely based on abstract spec
- No traction: It’s not a standard

- displayName.language.isNull

## Schematron

- Use against XML (or ISO data types)
- It’s what most people are doing now
- Poor support for type based schematron
- Poor support for semantics – you can only address the XML.
- Can use web service based terminology services (CTS 2 for this) (slow)
- No one writes proper schematron statements

- Not(displayName) or displayName/@nullFlavor or Not(displayName/@language)

## Constraints

- All constraint languages have the same problem:
- They are black boxes – they can test for compliance
- But you can’t design from them
- To design, you need some positive statement (i.e. Static Model)
- There’s nothing out there for that, except for custom stuff in various stds (HL7, DITA, openEHR)
- Please, OMG, rescue us!

## Data Type Flavors

- Are a systematic way to index, reference and share constraints on data types
- Constraints actually still need to be expressed...
- Schema
- Table form (lite static model)
- Prose
- Data types defines some common data types flavors by demand

## MIF

- MIF = Model Interchange Format
- HL7’s private pre-publishing format
- very semantically focused
- proprietary format – HL7’s wants come first
- can be used by non-HL7 users
- hard and easy to use
- MIF for data type flavors up in the air

## Part 2: Semantics of Data Types

- Collections (*SET, DSET, BAG, LIST*)
- Text/Media (*ED, ST, SC*)
- Concept References (*CD, CS*)
- Identifiers (*II, TEL*)(lunch)
- Demographics (*AD, EN*)
- Quantities (*INT, REAL, MO, CO, PQ, TS, RTO)*
- Quantity Sets (*QSET, QSS, QSP, QSI, QSD, QSU, PIVL, EIVL, IVL*)

## Collections

- COLL (Collection) A collection of values which can be enumerated using an iterator
- DSET(Discrete Set) :  A collection that contains distinct and discrete values in no particular order
- LIST (Sequence) : A collection that contains other discrete (but not necessarily distinct) values in a defined sequence
- BAG (Bag) : An unordered collection of values, where each value can be contained more than once in the collection

## OrderedSet

- Where’s OrderedSet – UML/OCL has it
- “A collection that contains distinct and discrete values in a defined sequence”
- Difficult beast
- Outcomes are poorly defined. What happens if you insert a duplicate value at a particular point in the sequence?
- It’s been proposed, but no convincing use case (not enough to overcome the beastliness)

## Other “Collections”

- Map
- Name / Key value pairs
- Tree
- Recursively nested list
- Array
- Fixed length random access
- Shudder

## Bag

## Bag - Equality

- Two bags are equal if and only if they are both empty, or if they both contain the same items, with the same number of each item present.
- Note: the determination of element content is based on the same semantic equals as defined in this specification, so it is possible that a BAG(CD) can be equal to BAG(CO), for instance.

## Bag Examples

- <example xsi:type="BAG_TEL">
-   <item value="tel:+15556667777" use="H"/>
-   <item nullFlavor="UNK" use="W"/>
- </example>
- <example xsi:type="BAG_TEL">
-   <item nullFlavor="UNK" use="W"/>
-   <item value="tel:+15556667777" use="H"/>
- </example>
- 2 examples not equal (equality is unknown)

## List

## List

- The sequence is an ordered collection of values, but no particular order is inherently associated with the sequence.
- The meaning of the order of the items SHOULD be defined where a LIST is used.
- Note that in some cases, the order is fixed ( e.g. HIST), but in other cases, the order is not fixed: only the meaning associated with the order in the instance is defined (e.g. EN, AD).

## List - Invariants

- If the list is not null, it cannot be empty

## List - Equality

- Two lists are equal if and only if they are both empty, or if they contain the same elements in the same order.
- Note 1: the determination of element content is based on the same semantic equals as defined in this specification, so it is possible that a LIST(ED) can be equal to LIST(ST), for instance.
- Note 2: SLIST equality is based on the equality of a sequence of values, so a SLIST<T> may be equal to a LIST<T>.

## List Example

- <example xsi:type="LIST_INT">
-   <item value="3"/>
-   <item value="11"/>
-   <item value="6"/>
-   <item value="9"/>
- </example>

## Specialized Lists

- GLIST (Generated Sequence)A periodic or monotone sequence of values generated from a few parameters, rather than being enumerated. Used to specify regular sampling points for biosignals.
- SLIST (Sampled Sequence)A sequence of sampled values scaled and translated from a list of integer values. Used to specify sampled biosignals.

## GLIST

## GLIST - Attributes

| Property | Type | Description |
| --- | --- | --- |
| head | T | The first item in this sequence
 |
| increment | QTY | The difference between one value and its previous different value
 |
| denominator | Integer | the number of times the sequence generates the same sequence item value before incrementing to the next sequence item value
 |
| period | Integer | If non-NULL, specifies that the sequence alternates, i.e., after this many increments, the sequence item values start again
 |

## GLIST Invariants

- If the GLIST is not null, all attributes but the period are required
- Denominator must be positive
- The period must be positive

## GLIST - Equality

- GLIST is a list generator. Two GLISTs are equal if they specify the same sequence of elements.

## GLIST Examples

- <example xsi:type="GLIST_PQ" period="100" denominator="100">
-   <head value="0" unit="V"/>
-   <increment xsi:type="PQ" value="1" unit="mV"/>
- </example>
- The x-wave of a digital oscillograph scanning between 0 and 100 mV in 100 steps of 1 mV.

## GLIST Examples

- <example xsi:type="GLIST_TS" denominator="1">
-   <head value="2002072920300"/>
-   <increment xsi:type="PQ" value="100" unit="us"/>
- </example>
- A timebase from June 29, 2002 at 8:30 PM with 100 us between each steps of the index.

## SLIST

## SLIST - Attributes

| Property | Type | Description |
| --- | --- | --- |
| origin | T | The first item in this sequence
 |
| scale | QTY | The difference between one value and its previous different value
 |
| digits | Sequence(INT) | the number of times the sequence generates the same sequence item value before incrementing to the next sequence item value

Note nullability! |

## SLIST Invariants

- scale must be a difference from the origin
- If the SLIST is not null, an origin and at least one digit is required

## SLIST Equality

- SLIST specifies a LIST. Two SLISTs are equal if they specify the same sequence of values.
- Note: Because SLIST specifies a LIST, and equality is basd on the LIST that is specified, it is possible that a LIST<PQ> will be equal to a SLIST<PQ>.

## SLIST Example

- <example xsi:type="SLIST_PQ">
-   <origin value='0' unit='uV'/>
-   <scale xsi:type="PQ" value='2.5' unit='uV'/>
-   <digit value="-4"/>
-   <digit value="-13"/>
-   <digit value="-18"/>
-   <digit value="-18"/>
-   <digit nullFlavor="UNK"/>
- </example>
- Lead II of an EKG tracing, with origin calibrated at 0 μV and with a scale factor of 2.5 μV.  The last measurement failed

## Sets: Two Kinds

- Mathematical
- i.e. Set of real numbers between 1 and 2
- Can test set membership
- Cannot be iterated or counted
- Cannot be specified by listing members
- Needs Type with order
- Computational
- i.e. “Washington, Chicago, Los Angeles”
- Can test set membership
- Can be iterated and counted
- Can only be specified by list members
- Needs Discrete Type

## Sets

## DSETs & QSETs

- DSETs
- General rule: use for anything that is not QTY
- QSETs
- General rule: use for all QTY specializations
- Special Case: Integer is both ordered and discrete.
- DSET(INT) = 2,3,4,5 equals QSET(INT) = 2..5