XML Terms and Con

Primer 

Susan Malaika, IBM 

malaika@us.ibm.com 

NIEM XML & Mobile ID Workshop, Gaith

Sep 2007 

September 2007 Copyright IBM 
Agenda 

• What is XML? • Why XML? • XML Use Cases • More on XML • Well-formed XML documents • Namespaces • XPath • XML Schema • Various XML Industry Formats • Summary and Further Reading September 2007 Copyright IBM 
Exchanging Data 

Let's start with an example... Here is some data from an ordinary delimited 

47; John Doe; 58; Peter Pan; Database systems; 

What does this data meanWho can process this dat

September 2007 Copyright IBM 
An XML Document 

XML = e

Xtensible 

Markup 

Language 

<book> 

<authors> 

<author id=“47

”

>John Doe

<author id=“58

”

>Peter Pan

</authors> <title>Database systems</title<price>29</price> <keywords> 

<keyword>SQL</keyword> <keyword>relational</keyw

</keywords> 

</book> 

XML: Des

XML Document, Message, or Instance 

September 2007 Copyright IBM 
XML Time Line 

These formats u

Schema 

GML 

Generalized Markup Language 

1970s 

SGML 

Standard Generalized 

Markup Language 1986 

HTML has an SGML DTD 

& is an SGML vocabulary 

HTML 

1991 

XML 1.0 (SGML “subset”) 

Extensible Markup Languag

1998 

XHTML 

2000 

HTML Forms 

1994 A

Feed 

XForms 2001 

DTD: Docume

September 2007 Copyright IBM 
Some XML Terminology 

Start Tag 

End Tag 

Start Tag 

<book> 

<authors> 

<author id=“47

”

>John Doe

<author id=“58

”

>Peter Pan

</authors> <title>Database systems</title<price>29</price> <keywords> 

<keyword>SQL</keyword> 

<keyword>relational</keyw

</keywords> 

</book> 

Data or Content 

Attribute 

End Tag 

XML: DescrHTML: Desc

September 2007 Copyright IBM 
XML Document - 2 Represen

Two different representations of the same hierarchical X

Text Tree 

<book> 

<authors> 

<author id=“47”>John Doe</author> 

<author id=“58”>Peter Pan</author> 

</authors> 

<title>Database systems</title> 

<price>29</price> 

<keywords> 

<keyword>SQL</keyword> <keyword>relational</keyword> 

</keywords> 

</book> 

Other representations: Event streams etc. 

author 

authors 

author 

id=47 id=58 

D

John Doe Peter P

September 2007 Copyright IBM 
The XML Document Tree 

XML Parser•DOM: Doc•SAX: Simp

<book> 

<authors> 

<author id=“47

”

>John Doe</author> 

<author id=“58”>Peter Pan</author> 

</authors> <title>Database systems</title> <price>29</price> <keywords> 

<keyword>SQL</keyword> <keyword>relational</keyword> 

</keywords> 

</book> 

X

book 

title 

author 

authors 

author 

id=47 id=58 

Database 

Systems 

John Doe Peter Pan 

Serialization 

September 2007 Copyright IBM 
Why XML? 

• XML: a notation for data exchange betw

and applications that have not necessaformally introduced to each other. 

• XML: enables the creation of precise de

admissible content - you can define th

– Encoding: Allowable characters and allo

encodings - to support diverse platformenvironments and hardware). 

• Ways to discover the particular enco

– Schemas and Constraints: Methods for 

specific and general purpose content 

-

• Ways to discover, access and proce

and constraints 

– Namespaces: Mixing of domain specific conten

sources 

• Ways to use different schemas in a s

document 

September 2007 Copyright IBM 
More XML Characterist

 XML has a flexible data model: 

For structured data, semi-structured data, sc

For data whose structure evolves 

Easy to extend: define new tags as neede

XML is self-describing: any XML parser can "undersXML is vendor and platform independent Easy to "validate" XML, e.g., to check compliance w

- any XML parser can do it! 

Easy to transform XML documents into other format

Plenty of transformation tools 

Easy to share XML between applications, businesse

September 2007 Copyright IBM 
The Success of XM

The success of XML has resulted in the wid• General purpose software for processing 

parsers, XML transformers, XML databaprocessors 

• XML content – e.g., XML exchange mess

configuration files, XML audit data, XML 

• XML based IT standards – such as XHTM

WSDL for Web Services 

• XML based industry standards – agreed 

use in specific industries and across induChemical Markup Language, Healthcare products markup language, Music XML, 

September 2007 Copyright IBM 
Architecture of Standards 

Composite Industry Standards 

Information Technology Standards 

(cross-industry) 

Horizontal Industry Standards 

(cross-industry) 

Industry-Specific Standards 

ExamIHE, AFormaACORGJXDNewsSTARebXMFormaCommAddresProgra

September 2007 Copyright IBM 
XML can be a very goo

• Data that needs to be exchanged in a variety of ways 

– Example: e-mail, File transfer, Web Services, Fax 

• Data that’s inherently hierarchical or nested in nature 

– Example: Medical data, Bill-of-materials 

• Data sets with sparsely populated attributes 

– Example: FIXML (Financial Information Exchange), FpML (Financ

Language), Customer profiles, Configuration information 

• Data whose structure evolves 

– Example: Frequently changing services/products/processes, prod

files 

• Data that has many variable schemas 

– Example: Data integration, consolidation of diverse data sources 

• Data that combines structured & unstructured information 

– Example: Life Sciences, News & Media 

• Data that needs to be kept a long time 

– Example; Legal records 

September 2007 Copyright IBM 
Typical XML Use Cas

• Information exchange between applications & organization• Message-based transactions, Web Services • Electronic forms and workflow processing • Integration of diverse data sources • Descriptions that change often (configuration files, product • Content that needs to be human-readable as well as proce• Content that needs to be transmitted in different ways, e.g

Web Service, feed, fax 

• XML documents as business objects / transaction records 

auditing, regulatory compliance) 

• XML as a flexible data model (for multi-values, hierarchica

September 2007 Copyright IBM 
Who uses XML? Everybo

Financial 

ACORD XML for Insurances http://www.acord.oFIXML Financial Information eXchange protocol http://www.fixproto

FPML Financial Product ML http://www.fpml.orFUNDSML Funds Markup Language http://www.funds-xXBRL eXtensible Business Markup Language http://www.xbrl.orgLife Sciences AGAVE Architecture for Genomic Annotation, 

Visualization and Exchange 

http://www.lifecde.

BSML Bioinformatic Sequence Markup Language http://www.bsml.orCML Chemical Markup Language http://www.xml-cmPublication etc. SportML Sport Markup Language http://www.sportsmNewsML News Markup Language http://www.newsmXBITS XML Book Industry Transaction Standards http://www.xmlbitsXPRL eXtensible Public Relations Language http://www.xprl.orgOther LandML Land Development Markup Language http://www.landxmMODA-ML Middleware tOols and Documents to 

Enhance the textile/clothing supply chain through xML 

http://www.moda-mml/repository/sche1/default.asp?lingu

MatML Materials Property Data Markup Language http://www.matml.oJXDM Global Justice XML Data Model http://it.ojp.gov/jxdebXML Electronic Business using eXtensible Markup 

Language 

http://www.ebxml.o

... ... ... 

September 2007 Copyright IBM 
(

C 

Use Case 1: Financial Data (F

Buying 1000 Shares of IBM Stock.. 

8=FIX.4.2^9=251^35=D^49=AFUNDMGR^56=ABROK^52=20030615-01:14:49^11=12345^1=111111^63=0^0621^21=3^110=1000^111=50000^55=IBM^48=459201^54=1^60=2003061501:14:4938=5000^40=1^44=15.^59=0^10=127 

Old FIX Protocol 

Financial Information eXchange) 

New FIXML 

Protocol 

extensible lower appl development & 

maintenance cost 

See http://www.fixprotocol.org/ 

September 2007 opyright IBM 
Use Case 2: newsML – News D

Worldwide News exchange in XML forma

ƒbetween news agencies, publishers, m

companies, news editing systems, etc. 

Typically semistructured data 

ƒsome structured data items (date, timeƒfull text 

Easily searchable Easily transformable into any target formaSimilar use case: RSS or Atom 

 Online news feeds in XML format 

See http://www.newsml.org/ 

September 2007 Copyright IBM 
Use Case 3: FpML - OTC Deri

Trading 

• 

Financial 

products 

Markup 

Language def

(International 

Swaps and 

Derivatives 

Ass

http://www.fpml.org/ 

• XML vocabulary for describing derivatives

and their risks 

• Derivatives: risk-shifting agreement, base

tradable instrument (interest rate, stock, incurrency,...) 

• OTC (over-the-counter) derivatives: privat

no standards, customized contracts 

– Large variety & rapid changes in deriv

transactions 

– Not manageable in a relational databa

September 2007 Copyright IBM 
ght IBM 

Derivatives 

Trading before 

FpML 

• Highly manual, i.e. error-prone and 

of poor timeliness 

• No automated system • Concern: system not able to 

handle variety & rapid change 

• Concern: too costly to build 

automated trading system 

• Concern: system is obsolete by the 

time it’s implemented 

• Solution: XML-based trading 

system, automated, able to evolve rapidly -> FpML 

September 2007 Copyri 
Benefits of FpML 

• Integration of trading services acros

systems and applications 

• Hardware and Software independe• Lower system implementation & ma

cost 

• Higher trading volumes with higher • Increased business opportunities • Reduced operational risks 

September 2007 Copyright IBM 
Use Case 4: ACORD – Insu

• ACORD =Association for Cooperative Oper

Research and Development http://www.acord

• Non-profit standards body for insurance data exchan• 1970: Forms development for property & casualty ins• 1980: EDI standards for Property & Casualty industry 

• 1996: Standards for Life Insurance • 2000+: 

– XML-based standards for Property & Casualty, 

– Data and application integration – Real-time information exchange for B2B and B2

September 2007 Copyright IBM 
Why did ACORD move to 

• eBusiness and Internet-based business: 

back offices, agents, brokers, consumers

• Diversified & multi-channel distribution • Streamlined & simplified data transfer • Straight-through processing of applicatio• Cross platform and cross-system data ex• Integration of diverse data sources • Extensible: for hybrid and aggregate insu

September 2007 Copyright IBM 
Well-formed XML Docu

An XML document is wellformed, if: not well-formed We

Has exactly one root element <b>bla</b> 

<c>blub</c> 

<a> 

<b<c

</a

Each opening tag is matched by a closing tag 

<a><b>bla</a> <a

All elements are properly nested <a><b>bla</a></b> <aAttribute values must be quoted <a id=15></a> <a Does not use disallowed characters in tags or values 

<a> 3<5 </a> <a

... ... ... 

Note: xml header <?xml version=“1.0”?> is NOT required for well-formednesSee http://www.w3.org/TR/REC-xml for full definition. 

September 2007 Copyright IBM 
“Well-formed” or “Vali

• An XML document is well-formed, if... 

– ...it complies with the rules on the previous p– i.e. it can be parsed by an XML parser withou

• An XML document is valid, if... 

– ...it is well-formed AND – ...it complies with a specific DTD or XML Sch

• XML Parsers can optionally perform “validatio

• DTDs (Document Type Definitions) and XML 

specific XML document structure 

September 2007 Copyright IBM 
Problem: Name Colli

Three different XML elements: 

<title>Database Administrator</title> <title>Your majesty</title> <title>Gone with the wind</title> 

• Same element name, but different meaning ! • Can result in processing/application errors. • Need to distinguish between different domains. 

September 2007 Copyright IBM 
Solution: Namespac

A prefix identifies the domain (“namespacedistinguishes between duplicate element na

<job:title>Database Administrator</job:titl

<person:title>Mr</person:title> <movies:title>Gone with the wind</movies

Namespaces need to be uniquely identified....-> UR

September 2007 Copyright IBM 
URI = Universal Resource 

URI Examples: 

http://www.ibm.com/db2xml http://abcdefghijklmn.xyz 

• URIs uniquely identify a namespace • URIs typically look like a URL 

• URIs are just an identifier, they may to po

page, but don’t have to ! 

For more details on URIs see http://www.ietf.org/rfc/rfc2396

September 2007 Copyright IBM 
“ ”

Namespace Declaration a

• Example: 

– element name: person – namespace URI: http://www.foobar.o

– namespace prefix: foo 

<foo:person xmlns:foo="http://www.foobar.org"

<foo:name>John Doe</foo:name> 

</foo:person> 

The reserved attribute xmlns defines namespaces

assigns them to a namespace prefix 

The namespace applies to the current element and and attributes that it contains. 

September 2007 Copyright IBM 
Multiple Namespac

<cust:person xmlns:cust=“http://www.foobar.com/cu

<cust:name>John Doe</cust:name> <prod:product xmlns:prod=“http://www.foobar.com

<prod:name>Thinkpad T40</prod:name> 

<prod:orderdate>2004-11-18</prod:orderdate

</prod:product> 

</cust:person> 

The namespace applies to the current element and aland attributes that it contains – unless it’s overridd

September 2007 Copyright IBM 
Default Namespac

A namespace declaration without prefix defines a defnamespace. The namespace is implicit for all elemein scope, without using a prefix. 

<person xmlns=“http://www.foobar.org”> 

<age>45</age> <name> 

<first>John</first> <last>Doe</last> 

</name> 

</person> 

The default namespace applies to the current elemeelements and attributes that it contains. 

September 2007 Copyright IBM 
XPath 

 XPath is used in: 

<dept bldg=“101”> XSLT 

<employee id=“901”> XQuery 

<name>John Doe</name> SQL/XML

<phone>408 555 1212</phone> 

<office>344</office> 

</employee> 

<employee id=“902”> 

<name>Peter Pan</name> <phone>408 555 9918</phone> 

<office>216</office> 

</employee> 

</dept> 

dept 

name 

employee 

id=901 phone

John Doe 

office 

408-555-1212 344 

n

id=902 

Peter 

September 2007 Copyright IBM 
XPath: 

Simple XPath Expressions 

 Use fully qualified paths to specify elements/attributes  “@” is used to specify an attribute  use “text()” to specify the text node under an element 

XPath Result 

/dept/@bldg 101 /dept/employee/@id 901 

902 

<dep

<e

<<<

</dep

/dept/employee/name <name>Peter Pa

<name>John Do

/dept/employee/name/text() Peter Pan 

John Doe 

September 2007 Copyright IBM 
XPath: 

Wildcards 

 * matches any tag name  // is the “descendent-or-self” wildcard 

XPath /dept/employee/*/text() /dept/*/@id //name/text() /dept//phone 

<dept 

<em

<

</e<e</e

</dept

ResulJohn Doe 408 555 1212 344 Peter Pan 408 555 9918 216 901 902 Peter Pan John Doe <phone>40<phone>40

September 2007 Copyright IBM 
XPath: 

Predicates 

 Predicates are enclosed in square brackets [...]  Can have multiple predicates in one Xpath  Positional predicates: [n] selects the n-th child 

XPath /dept/employee[@id=“902”]/name 

/dept[@bldg=“101”]/employee[office >“300”]/name //employee[office=“344” OR office=“216”]/@id /dept/employee[2]/@id 

<dep

<e

<<<

</dep

Resu<nam

<nam

901 902 902 

September 2007 Copyright IBM 
XPath: <dep

<e

The Parent Axis 

<<

 Current context: “.”  Parent context: “..” <

</dep

XPath Result 

/dept/employee/name[../@id=“902”] <name>

/dept/employee/office[.>“300”] <office>/dept/employee[office > “300”]/office <office>

/dept/employee[name=“John Doe”]/../@bldg 101 /dept/employee/name[.=“John 101 Doe”]/../../@bldg 

September 2007 Copyright IBM 
Orderschema

What is an XML Schem

• Defines structure, content, data types for • Consists of 1 or more schema document

known as XSDs 

• A schema document can define a names• Example: 

– 1 XML Schema, 3 Schema Documents, 2 Na

order.xsd

Order Lineitem 

lineitem.xsd 

import 

include 

Namespace: Namespace: 

http://www.w3.org/TR/xmlschema-0/ 

September 2007 Copyright IBM 
Some XML Schema Characte

• Can define data types for elements

– Basic types: integer, date, decimal, – User Defined Types, Complex Eleme– Allowed length & patterns for string – Supports derived data types 

• Detailed occurrence and value range def• XML Schema supports XML Namespace

• A Schema can be composed of multiple sche

– Import/Include of other schemas is supported 

September 2007 Copyright IBM 
Types in XML Sche

Built-In Simple Types: •string •boolean •float •double •decimal •integer •positiveInteger •byte •date •datetime •anyType •.... 

Derived Simple Types: • Restriction of a simple type 

“integer between 5 and 10” 

• Union of simple types 

“integer 

∪ string” 

• Enumerations • etc. 

September 2007 Copyright IBM 
XML Schema: Examp

<xsd:schema targetNamespace=“http://www.mycompany/pr

xmlns:xsd="http://www.w3.org/2001/XMLSchem

<xsd:simpleType name=“PriceType"> 

<xsd:restriction base="xsd:decimal"> 

<xsd:minInclusive value="

0"/> 

<xsd:maxInclusive value=“100000"/> 

<xsd:totalDigits value=“

9"/> 

<xsd:fractionDigits value=“

3"/> 

</xsd:restriction> 

</xsd:simpleType> <xsd:complexType name=“StockPriceType"> 

<xsd:sequence> 

<xsd:element name="Ask" type="PriceType"/> 

<xsd:element name="Bid" type="PriceType"/> 

<xsd:element name="P50DayAvg" type="PriceType

</xsd:sequence> 

</xsd:complexType> <xsd:element name=“StockPrice" type=“StockPriceType"/> 

</xsd:schema > 

September 2007 Copyright IBM 
XML Schema: Simple & Comple

<xsd:complexType name=“StockPriceType"> 

<xsd:sequence> 

<xsd:element name="Ask" type="PriceType"/> 

<xsd:element name="Bid" type="PriceType"/> 

<xsd:element name="P50DayAvg" type="PriceType"/> 

<xsd:restriction base="xsd:decimal"> 

<xsd:minInclusive value="

0"/> 

<xsd:maxInclusive value=“100000"/> 

<xsd:totalDigits value=“

9"/> 

<xsd:fractionDigits value=“

3"/> 

<xsd:simpleType name=“PriceType

PriceType: d“Decimal” by additional re

Oct

<St

<<<

</St

"> 

</xsd:restriction> 

</xsd:simpleType> 

</xsd:sequence> A 

</xsd:complexType> 

<xsd:element name=“StockPrice" type=“StockPriceType"/> 

September 2007 Copyright IBM 
XML Schema Characteristics for Some I

Formats [1] 

Industry Format Version Types Elements Attribut

ACORD (Association for Cooperative Operations Research and Development) 

(XMLife) 

2.16.01 1369 9378 12

ARTS (Association for Retail Technology Standards) 1.0 - 3.0 4825 6305 20CDISC (Clinical Data Interchange Standards Consortium) 00-9-03 98 84 FpML (Financial Products Markup Language) 4.2 686 1867 1

FIXML (Financial Information Exchange) 4.4 1310 619 25

HL7CDA (Clinical Document Architecture) 3 1953 945 4IRS1120 (IRS e-File Form 1120) 2006v3.3 3415 11591 26

MISMO (Mortgage Industry Standards Maintenance Organization) 2.3 - 2.4 2899 1087 137

September 2007 Copyright IBM 
XML Schema Characteristics for Som

Formats [2] 

Industry Format Version Types Elements Attrib

MCJE (NIEM) (Minnesota Criminal Justice Event (National Information Exchange Model)) 1 415 936 OTA (OpenTravel Alliance) 2003/5 27293 24893 4STAR (Standards for Technology in Automotive Retail – OAGIS) 5.0.4 5846 77319 TWIST (Transaction Workflow Innovation Standards Team) 3.1 1016 2314 UBL (Universal Business Language) 2 682 2665 UNIFI (ISO20022 UNIversal Financial Industry message scheme) 1.01 - 2.01 5082 9747 

XBRL - GL (Extensible Business Reporting Language - Global Ledger) 2006-10-25 1858 2847 

These tables are intended to give an idea of the different des– The schemas may have moved on since these tables was 

September 2007 Copyright IBM 
Some XML Design Consid

• Elements and Attributes • Document size • Nesting level • XML schema features used, 

– e.g., substitution groups, extension elements, complex ty

• Schema versioning, 

– e.g., specifying version in schema, in instance document 

• Number and structure of XSD files • Number of message types • Number and structure of namespaces • Digital signature provision • Use of other XML schemas • Useful complimentary items, such as: 

– forms, e.g., through XForms – transformations, e.g., through XSLT – queries, e.g., through XQuery – constraint check specification e.g., through Schematron 

September 2007 Copyright IBM 
XML Summary [1

XML is a key technology for... 

– ...data exchange – ...data integration – ...data evolution and flexibility – ...Web applications & Web servi– ...Service Oriented Architectures 

• Software that supports XML is 

– Hardware that supports XML is a

September 2007 Copyright IBM 
XML Summary [2] 

• Well-formed XML documents 

– Valid XML documents are well-formed XML t

schema 

• Namespaces help: 

– Avoid naming collisions – Identify XML vocabularies 

• Applying constraints on XML can be ach

through: 

– XML Schemas support data types & namesp– Constraint languages such as Schematron 

• XPath, XSLT, XQuery, SQL/XML 

– Search, retrieve, transform modify and shred 

September 2007 Copyright IBM 
Further Reading (XML Fundamentals) 

Many online tutorials available.... 

http://www.w3schools.com/ http://www.w3schools.com/xml/ http://www.w3schools.com/dtd/ http://www.w3schools.com/schema/ http://www.w3schools.com/xpath/ http://www.w3schools.com/xsl/ http://www.w3schools.com/xml/xml_namespaces.asp ... 

September 2007 Copyright IBM 
The Interactive Industry Forma

[1] You can read an article about the demo and bundleGet started with Industry Formats and Services with pu

http://www.ibm.com/developerworks/db2/library/techarticl

[2] You can run the demo here: 

http://www.alphaworks.ibm.com/tech/pur

Scroll down on the page to locate the PDF entitled “Gethe Demo” here. Please follow the “Before You Begin” PDF. The Interactive Demo (which includes NIEM) can selecting View Demo. [3] You can locate the NIEM Industry bundle by selecting 

http://www.alphaworks.ibm.com/tech/purexml 

The bundle includes scripts to store, index, validate, adatabase 

We welcome feedback from t

community on the demo and 

September 2007 Copyright IBM