# Presentation: 116624

## 4. Metadata syntaxes and containers

- Metadata Standards and Applications Workshop

## Goals of session

- Understand syntaxes used for encoding information, including HTML, XML and RDF
- Discover how container formats are used for managing digital resources and their metadata

## Overview of Syntaxes

- HTML, XHTML: Hypertext Markup Language; eXtensible Hypertext Markup Language
- XML: Extensible Markup Language
- RDF/XML: Resource Description Framework

**Notes:**
[Point out the handout that defines some of the XML terms.]
Mostly, in order to use these syntaxes you will want to find someone who understands this stuff and nail them to the floor.

## HTML

- HyperText Markup Language
- HTML 4 is the current standard
- HTML is an SGML (Standard Generalized Markup Language) application conforming to International Standard ISO 8879
- Widely regarded as the standard publishing language of the World Wide Web
- HTML addressed the problem of SGML complexity by specifying a small set of structural and semantic tags suitable for authoring relatively simple documents

**Notes:**
HTML 4 is the current standard.
SGML has not translated well to the web.
Use HTML for simple documents and web presentation.

## XHTML

- XML-ized version of HTML 4.0, tightens up HTML to match XML syntax
- Requires ending tags, quoted attributes, lower case, etc., to conform to XML requirements
- XHTML is a W3C specification, redefining HTML as an XML implementation, rather than an SGML implementation
- Imposes requirements that are intended to lead to more well-formed, valid XML, easier for browsers to handle

**Notes:**
HTML does not require ending tags.  XHTML makes HTML look more like XML.  It is important to know that XHTML is a subset of XML to know what it works for and what it does not work well for.

## An XHTML Example

**Notes:**
This example was created using the DC-Dot metadata generator. Note that it has a link to 2 namespaces: DC elements, DC terms
Note also the variable usefulness of the subject keywords.
XHTML is commonly used for digital objects on the web

## XML

- Extensible Markup Language
- A ‘*metamarkup’* *language*: has no fixed tags or elements
- Strict grammar imposes structure designed to be read by machines
- Two levels of conformance:
- well-formed--conforms to general grammar rules
- valid--conforms to particular XML schema or DTD (document type definition)

**Notes:**
XML on its own is just a box.  You use whatever element sets you want to use.  The structured grammar is for machines, not to be read by humans, although some of have adapted.

## XML: Extensible Markup Language

- A technical approach to convey meaning with data
- Not a natural language, although uses natural languages
- <姓名>Louis Armstrong</姓名>
- <name>Louis Armstrong</name>
- Not a programming language
- Language in the sense of:
- A limited set of tags defines the elements that can be used to markup data
- The set of tags and their relationships need to be explicitly defined (e.g., in XML schema)
- We can build software that uses XML as input and processes them in a meaningful way
- You can define your own markups and schemas

## XML is the lingua franca of the Web

- Web pages increasingly use at least XHTML
- Business use for data exchange/ messaging
- Family of technologies can be leveraged
- XML Schema, XSLT, XPath, and XQuery
- Software tools widely available (open source)
- Storage, editing, parsing, validating, transforming and publishing XML
*Microsoft Office 2003* supports XML as document format (WordML and ExcelML)
*Web 2.0* applications based on XML (AJAX, Semantic Web, Web Services, etc.)

## An XML Schema Defines:

- What elements may be used
- Of which types
- Any attributes
- In which order
- Optional or compulsory
- Repeatability
- Subelements
- Logic

## Anatomy of an XML Record

- XML declaration--prepares the processor to work with the document and states the XML version
- Namespaces (uses xmlns:*prefix* and a URI to attach a prefix to each element and attribute)
- Distinguishes between elements and attributes from different vocabularies that might share a name (but not necessarily a definition) using association with URIs
- Groups all related elements from an application so software can deal with them
- The URIs are the standardized bit, not the prefix, and they don’t necessarily lead anywhere useful, even if they look like URLs

**Notes:**
URI might not be resolvable– there might not be something at the other end, but the important thing is that it is there to resolve the ambiguity.  Many people think that there needs to be more of a push for resolvable URIs.

## XML Namespaces

- xmlns:dc=”_http://purl.org/dc/elements/1.1/”_

- XML Namespace

- Namespace Prefix

- Namespace Identifier

## XML Anatomy Lesson

-  <mods:genre authority=“marcgt”>bibliography /</mods:genre>

- Attribute

- End Tag

- Content

- Name

- Start Tag

**Notes:**
The pieces that come together to build an XML statement (nested within another element).

## XML Validation

- XML Instance

- XML Schema

- Validator

- Valid

- Invalid

## XML Schema Example

- <xs:element name="software" minOccurs="0" maxOccurs="unbounded">
- 	<xs:complexType>
- 		<xs:sequence>
- 			<xs:element name="swName" minOccurs="1" maxOccurs="1" type="xs:string"></xs:element>
- 			<xs:element name="swVersion" minOccurs="0" maxOccurs="1" type="xs:string"></xs:element>
- 			<xs:element name="swType" minOccurs="1" maxOccurs="1" type="xs:string"></xs:element>
- 			<xs:element name="swOtherInformation" minOccurs="0" maxOccurs="unbounded" type="xs:string"> </xs:element>
- 			<xs:element name="swDependency" minOccurs="0" maxOccurs="unbounded" type="xs:string"> </xs:element>
- 		</xs:sequence>
- 	</xs:complexType>
- </xs:element>

**Notes:**
minOccurs specifies whether it is mandatory or optional (minOccurs=“0” is optional; minOccurs=“1” is mandatory
maxOccurs specifies repeatability (maxOccurs=“1” is not repeatable; maxOccurs=“unbounded” means it can be repeated an unlimited number of times)
complexType defines additional structure
xs:sequence specifies order
Type specifies a data type (string is text; integer is number; xs:date is a special structured form of date)

## Will the following XML instance validate?

- <software>
**   ** <swName>**Windows**</swName>
** **   <swVersion>**2000**</swVersion>
** **   <swType>**Operating System**</swType>
- </software>
**How about this?**
- <swVersion>**2000**</swVersion>

**Notes:**
The second one won’t if the schema specifies that the element is in a container.

## Resource Description Framework

- Application for describing and exchanging metadata
- Structure consists of “triples”
- Designed to be read by computers, not humans
- An ontology language to support semantic interoperability—understanding meanings
- Considered an essential part of the Semantic Web
- Can be expressed using XML

- Subject

- Object

- Predicate

- http://www.w3.org/RDF

**Notes:**
RDF was created in 1999 as a standard that sat on top of XML; has evolved away from encoding metadata to describing the relationships between concepts (names (people, corporate bodies, place), topics, objects

Subject = Topic
Predicate = Relationship (hierarchical, equivalent, associative)
Expressed in XML

Permits infinite number of combinations which can provide a framework for the development taxonomies (thesauri, controlled vocabularies, faceted classifications)

## Some RDF Concepts

- A **Resource** is anything you want to describe
- A **Class** is a category; it is a set that comprises individuals
- A **Property** is a Resource that has a name, such as "creator" or "homepage"
- A **Property value** is the value of a Property, such as “Barack Obama" or "_http://dublincore.org_" (note that a property value can be another resource)

**Notes:**
Simple RDF concepts.
Something that is an element-property is also a conceptual resource, very much like a controlled vocabulary term.

## RDF Statements

- The combination of a **Resource**, a **Property**, and a **Property value** forms a *Statement* (known also as the subject, predicate and object of a Statement), also known as “triples”
- An example *Statement*: "The editor of _http://dublincore.org/documents/usageguide_/ is Diane Hillmann"
- The subject of the statement above is: _http://dublincore.org/documents/usageguide/_
- The predicate is: editor
- The object is: Diane Hillmann

**Notes:**
Resource = subject
Property = predicate
Property value = object
... of an RDF statement.
These three pieces are called “triples.”

## RDF and OWL

- RDF does not have the language to specify all relationships
- Web Ontology Language (OWL) can specify richer relationships, such as equivalence, inverse, unique
- RDF and OWL may be used together
- RDFS: a syntax for expressing relationships between elements

## An RDF/XML Example (with DC)

- <rdf:RDF
-     xmlns:rdf="_http://www.w3.org/1999/02/22-rdf-syntax-ns#_"
-     xmlns:dc="_http://purl.org/dc/elements/1.1/_">
-     <rdf:Description rdf:about="_http://www.dlib.org_">
-       <dc:title>D-Lib Program - Research in Digital Libraries</dc:title>
-       <dc:description>The D-Lib program supports the community of people
-        with research interests in digital libraries and electronic
-        publishing.</dc:description>
-       <dc:publisher>Corporation For National Research Initiatives</dc:publisher>
-       <dc:date>1995-01-07</dc:date>
-       <dc:subject>
-         <rdf:Bag>
-           <rdf:li>Research; statistical methods</rdf:li>
-           <rdf:li>Education, research, related topics</rdf:li>
-           <rdf:li>Library use Studies</rdf:li>
-         </rdf:Bag>
-       </dc:subject>
-       <dc:type>World Wide Web Home Page</dc:type>
-       <dc:format>text/html</dc:format>
-       <dc:language>en</dc:language>
-     </rdf:Description>
- </rdf:RDF>

- Note unordered list

**Notes:**
RDF does not need XML.  This is just a way of expressing it so that computers (and sometimes humans) can understand it based on their 
Under <subject>, we have a list.  You can make an order of the list, or make it an unordered list as here.
Ordered list ... RDF seq [sequence]

## Overview of container formats

- A container format is needed to package together all forms of metadata and digital content
- Use of a container is compatible with and an implementation of the OAIS information package concept
- METS: packages metadata with objects or links to objects and defines structural relationships
- MPEG 21 DID: represents digital objects using a flexible and expressive model

**Notes:**
OAIS=Open Archival Information System, a widely used standard for understanding digital repositories and functions

## MEMetadata Encoding & Transmission StandardTS

- Developed by the Digital Library Federation, maintained by the Library of Congress
- “... an XML document format for encoding metadata necessary for both management of digital library objects within a repository and exchange of such objects between repositories (or between repositories and their users).”
- Records the (possibly hierarchical) structure of digital objects, the names and locations of the files that comprise those objects, and the associated metadata
**http://www.loc.gov/standards/mets/**

## What is 			used for?

- To package metadata with digital object in XML syntax
- For retrieving, storing, preserving, serving resource
- For interchange of digital objects with metadata
- As information package in a digital repository (may be a unit of storage or a transmission format)

## Characteristics of METS 

- Open non-proprietary standard
- Extensible
- Modular
- Developed by the digital library community

**Notes:**
Two sections (dmdSec and amdSec) serve as “wrappers” or “sockets” where elements from other schemas, called "extension schemas” can be plugged in. This is the mechanism by which METS is extensible. It is accomplished by using the XML Schema facility for combining vocabularies from different Namespaces.
METS Editorial Board endorsed extension schemas for bibliographic data (MARCXML, MODS, DC), for technical metadata for still images (MIX), and technical metadata for text.

##   METS Sections 

- Defined in METS schema for navigation & browsing
- Header (XML Namespaces)
- File inventory,
- Structural Map & Links
- Descriptive Metadata (not part of METS but uses an externally developed descriptive metadata standard, e.g. MODS)
- Administrative Metadata (points to external schemas):
- Technical, Source
- Digital Provenance
- Rights

**Notes:**
Descriptive metadata describes the package itself.
File section: includes a list of all the files in the METS package
Structure map: what this thing is; links the digital object to content files and the metadata

## The structure of a METS file

## Linking in METS Documents(XML ID/IDREF links)

- DescMD
- 	mods
- 		relatedItem
- 		relatedItem

- AdminMD
- 	techMD
- 	sourceMD
- 	digiprovMD
- 	rightsMD

- fileGrp
- 	file
- 	file

- StructMap
- 	div
- 	   div
- 	      fptr
- 		div
- 		   fptr

**Notes:**
The structMap (structural map) is the heart of the METS document and specifies its structure by using nested “div” elements.

## Linking in METS Documents(XML ID/IDREF links)

- DescMD
- 	mods
- 		relatedItem
- 		relatedItem

- AdminMD
- 	techMD
- 	sourceMD
- 	digiprovMD
- 	rightsMD

- fileGrp
- 	file
- 	file

- StructMap
- 	div
- 	   div
- 	      fptr
- 		div
- 		   fptr

## Linking in METS Documents(XML ID/IDREF links)

- DescMD
- 	mods
- 		relatedItem
- 		relatedItem

- AdminMD
- 	techMD
- 	sourceMD
- 	digiprovMD
- 	rightsMD

- fileGrp
- 	file
- 	file

- StructMap
- 	div
- 	   div
- 	      fptr
- 		div
- 		   fptr

## Linking in METS Documents(XML ID/IDREF links)

- DescMD
- 	mods
- 		relatedItem
- 		relatedItem

- AdminMD
- 	techMD
- 	sourceMD
- 	digiprovMD
- 	rightsMD

- fileGrp
- 	file
- 	file

- StructMap
- 	div
- 	   div
- 	      fptr
- 		div
- 		   fptr

## Linking in METS Documents(XML ID/IDREF links)

- DescMD
- 	mods
- 		relatedItem
- 		relatedItem

- AdminMD
- 	techMD
- 	sourceMD
- 	digiprovMD
- 	rightsMD

- fileGrp
- 	file
- 	file

- StructMap
- 	div
- 	   div
- 	      fptr
- 		div
- 		   fptr

## METS extension schemas

- “wrappers” or “sockets” where elements from other schemas can be plugged in
- Provides extensibility
- Uses the XML Schema facility for combining vocabularies from different Namespaces
- Endorsed extension schemas:
- Descriptive: MODS, DC, MARCXML
- Technical metadata: MIX (image); textMD (text)
- Preservation related: PREMIS

## 	Descriptive Metadata Section (dmdSec)

- <mets>
-   **<dmdSec></dmdSec>**
-   <fileSec></fileSec>
-   <structMap></structMap>
- </mets>

- Two methods: Reference and Wrap

## METS examples

- METS with MODS
- Patriotic melodies
- Recorded event
- METS with MODS, PREMIS and MIX
- Portrait of Louis Armstrong (XML)
- Portrait of Louis Armstrong (presentation)

## MPEG-21 Digital Item Declaration (DID)

- ISO/IEC 21000-2: Digital Item Declaration
- A promising alternative to represent Digital Objects
- Starting to get supported by some repositories, e.g., aDORe, DSpace, Fedora
- A flexible and expressive model that easily represents compound objects (recursive “item”)
- Attach well-formed XML from persistent namespaces as metadata

## Abstract Model for MPEG-21 DID

*resource*

*resource*

*resource*

*component*

*component*

*descriptor/statement*

*descriptor/statement*

*descriptor/statement*

*descriptor/statement*

*item*

*item*

*container*

***resource***: datastream

***component***: binding of *descriptor/statements* to datastreams

***item***: represents a Digital Item aka Digital Object aka asset. *Descriptor/statement* constructs convey information about the Digital Item

***container***: grouping of *items* and descriptor/statement constructs pertaining to the container

## Summary: METS vs. MPEG 21 DID

- METS and MPEG DID are similar types of container formats in that both are expressed in XML, both represent the structure of digital objects, and both include metadata
- MPEG DID doesn’t have the segmentation in metadata sections that METS does, so this implementation decision need not be made in DID
- METS is open source and developed by open discussion, mainly cultural heritage community
- MPEG DID is an ISO standard and has industry support, but is often implemented in a proprietary way and standards  development is closed
- It would be possible to transform a METS container to a MPEG DID and vice versa; development of stylesheets will enable transformations

## Exercise	

- Encode your resource in DC and MODS using XML
- Use the template forms provided