# Presentation: 535879

## Evolution of Metadata Standards: New Features in ISO 19115

- Ted Habermann
- NOAA National Data Centers
- December, 2007
- Ted.Habermann@noaa.gov

**Notes:**
I am Ted Habermann from NOAA's National Data Centers. Creating and maintaining high quality metadata that document data completely is an important task at the Data Centers and metadata standards are changing. This presentation an overview of some interesting capabilities of the ISO 19115 metadata standard.

Click on a tab at the top of this presentation to see a glossary that defines many important metadata terms and a timeline that provides a context for changing metadata standards. Also, make sure you see the buttons on the left. Use these to switch between the outline or thumbnails view, and to read or search the notes. 

## Evolution of  Documentation Standards

**Interchange**
**Format**

**ISO 19115 – 2**

**ISO 19115/NAP**

**FGDC Remote **
**Sensing Extensions**

**FGDC**

**NOAA**
**Supplemental**

**NASA**
**EOS Core**
**System**
**Metadata**

## Question #1

**New Standard Emerges:**

*What is the minimum that I need to do to conform to this standard?*

*What is the most that this standard can do for me?*

- The answer is a short list of metadata elements.

- Depends on 1) understanding of the standard and 2) ingenuity..

- The goal of this presentation is to increase understanding of the standard. The ingenuity is up to you.

**Notes:**
There are several questions that can be asked by users of a new standard.
One that is quite common is “What is the minimum that I need to do to conform to this standard?”. This question can usually be answered with a fairly short list of metadata elements. These elements provide some important information and might support limited data discovery requirements, but they generally do not contribute much to understanding of the dataset. Filling in these elements can result in a record that is compliant with a standard, but it is not a high-quality record.

A second question is “What is the most that this standard can do for me?”. The answer to this question depends on understanding the standard and applying the standard in ingenious ways. It may even involve creating extensions to the standard for a particular community or application.

The goal of this presentation is to increase understanding of the standard. The ingenuity is up to you.

Ingenuity: skill or cleverness in devising or combining : inventiveness b : cleverness or aptness of design or contrivance 

## Metadata Needs

- The rest of this presentation is made up of a series of questions about your metadata needs. Suggested answers using ISO 19115 and 19115-2 are provided.
- These suggested approaches are presently being tested at NGDC and some may survive to become recommended best practices. Others may evolve significantly as our understanding and experience increases.

**Notes:**
The rest of this presentation is made up of a series of questions about your metadata needs. Suggested answers to those questions using ISO 19115 and 19115-2 are provided. These suggested approaches are being tested at NGDC and some may survive to become recommended best practices. Others may evolve significantly as our understanding and experience increases.

## Do you need to unambiguously identify things using your own namespace?

**MD_Identifier**
- + authority [0..1] : CI_Citation
- + code : CharacterString

**Where are identifiers?**
- aggregationInfo/aggregateDataSetIdentifier
- dataQualityInfo/report/measureIdentification
- extent/geographicElement/geographicIdentifier
- processStep/processingInformation/identifier
- acquisitionInformation/operation/identifier
- platform/identifier
- platform/instrument/identifier

**Where are Citations?**
- identificationInfo/citation
- applicationSchemaInfo.name
- dataQualityInfo/report/specification
- contentinfo/featureCatalogueCitation
- identificationInfo/aggregationInfo/aggregateDataSetName
- identificationInfo/descriptiveKeywords/thesaurusName
- dataQualityInfo/report/evaluationProcedure
- dataQualityInfo/lineage/source/sourceCitation
- dataQualityInfo/lineage/processStep/processingInformation/documentation
- dataQualityInfo/lineage/processStep/processingInformation/algorithm/citation
- acquisitionInformation/operation/citation
- acquisitionInformation/platform/citation
- acquisitionInformation/platform/instrument/citation

**Notes:**
This need for identifiers in metadata records was first recognized in the DIF and FGDC Remote Sensing Extensions. ISO 19115 extends this idea in several important ways. First, ISO uses a MD_Identifier object. This object includes a Citation that references the authority or namespace for the identifier along with the identifier itself. The identifier is guaranteed to be unique in that namespace.

Including identifiers in the ISO metadata records gives metadata creators a mechanism for uniquely identifying metadata records and many other items. This is becoming more important as metadata records evolve from single files into collections of related objects that can be harvested into repositories like Geospatial One-Stop along multiple paths. There is no reliable way to identify duplicate records without an identifier in the actual record.

Identifiers occur in many places in the ISO standard. The identifiers in Citations are particularly important because Citations also occur in many locations throughout the standard.

## Do you want to manage metadata using a relational or XML database?

- Unique IDs are critical for identifying items and creating relationships in relational databases.

**Notes:**
Including identifiers in the FGDC RSE and ISO metadata records gives the metadata creators a mechanism for uniquely identifying metadata records for the first time. This is becoming more important as metadata records can be harvested into repositories like Geospatial One-Stop along multiple paths. There is no reliable way to identify duplicate records without an identifier in the actual record.

## Do you want to serve metadata using a REST web service? 

- Number of ID's Included in Standard

- ISO 19115 and 19115-2 include identifiers for between 30 and 40 different items (see background slides for details)

- Key Principle for REST Services: Give every “thing” an ID

**Notes:**
The first principle of REST is “Give Everything an ID”. Given this principle, one measure of how “RESTful” a metadata standard might be is the number of id’s that are included in the standard. This compares the number of IDs included in four metadata standards. The original FGDC standard has none, the Directory Interchange Format (DIF) and the FGDC Remote Sensing Extensions (RSE) have one and the ISO 19115(-2) has 36.

These ids make it possible to reference many individual elements in an ISO metadata record directly, so the ISO standard is very compatible with a RESTful approach.

## Do you need to identify people in different roles?

**CI_ResponsibleParty**
- + individualName [0..1] : CharacterString
- + organisationName [0..1] : CharacterString
- + positionName [0..1] : CharacterString
- + contactInfo [0..1] : CI_Contact
- + role : CI_RoleCode

**Where Are Contacts?**
- MD_Metadata.contact
- MD_Identification.pointOfContact
- MD_Usage.userContactInfo
- LI_ProcessStep.processor
- MD_MaintenanceInformation.contact
- MD_Distributor.distributorContact
- MD_ExtendedElementInformation.source

**CI_RoleCode**
- + resourceProvider
- + custodian
- + owner
- + user
- + distributor
- + originator
- + pointOfContact
- + principalInvestigator
- + processor
- + publisher
- + author

## Do you need different documentation for different parts of your data?

**MD_ScopeCode**
- + attribute
- + attributeType
- + collectionHardware
- + collectionSession
- + dataset
- + series
- + nonGeographicDataset
- + dimensionGroup
- + feature
- + featureType
- + propertyType
- + fieldSession
- + software
- + service
- + model
- + tile

**ScopeCodes can apply to:**
- 1) MD_Metadata
- 2) DQ_DataQuality
- 3) MD_MaintenanceInformation

**MD_Metadata**
- + fileIdentifier [0..1] : CharacterString
- + language [0..1] : CharacterString
- + characterSet [0..1] : MD_CharacterSetCode = "utf8"
- + parentIdentifier [0..1] : CharacterString
- + hierarchyLevel [0..*] : MD_ScopeCode = "dataset"
- + hierarchyLevelName [0..*] : CharacterString
- + contact [1..*] : CI_ResponsibleParty
- + dateStamp : Date
- + metadataStandardName [0..1] : CharacterString
- + metadataStandardVersion [0..1] : CharacterString
- + dataSetURI [0..1] : CharacterString

## Do you need different documentation for different temporal and spatial subsets?

**Notes:**
ISO 19115 combines spatial and temporal extents into a single object, called an EX_Extent, that includes temporal, vertical, and geographic extents.

The temporal and vertical extents are simple descriptions of temporal or vertical ranges. 

The geographic extent can be described in three ways: 
The first option is a simple bounding box which works quite well in some simple situations. This is similar to the bounding box in the FGDC Metadata Standard, except that the FGDC box described the spatial extent of the entire dataset and this Extent object gives the spatial extent of a specific quality report.

The second option generalizes the bounding box to a bounding polygon. This covers a large number of cases which might involve natural shapes, such as watersheds, or political  shapes, like county forecast zones, rather than rectangles. In the FGDC Standard this is similar to the G-Ring option for the spatial extent, except that it applies only to a specific quality report.

The final option allows specification of an identifier of a geographic region. This identifier includes a namespace and a name and allows the use of named spatial objects, like counties, to describe spatial extents. This is similar to the spatial keyword element of the FGDC Metadata Standard.

## Do you need to reference On-Line Resources?

**CI_OnlineResource**
- + linkage : URL
- + protocol [0..1] : CharacterString
- + applicationProfile [0..1] : CharacterString
- + name [0..1] : CharacterString
- + description [0..1] : CharacterString
- + function [0..1] : CI_OnLineFunctionCode

**CI_OnLineFunctionCode**
- + download
- + information
- + offlineAccess
- + order
- + search

**Notes:**
As the World Wide Web has developed into a ubiquitous information source, links to on-line information and services have become a critical element in all metadata records. These links are treated very differently in the FGDC and ISO standards. In the FGDC, URLs are free text. In the ISO, they are CI_OnlineResource objects that include a rich set of fields for describing the on-line resource.

The ISO OnlineResource and the FunctionCodes make it possible to generate a classified list of links with titles and descriptions directly from the metadata. This is a big improvement over the simple text strings in the FGDC standard. 

## Do you need to describe many kinds of aggregations?

**MD_InitiativeTypeCode**
- + campaign
- + collection
- + exercise
- + experiment
- + investigation
- + mission
- + sensor
- + operation
- + platform
- + process
- + program
- + project
- + study
- + task
- + trial

**MD_AggregateInformation**
- + aggregateDataSetName [0..1] : CI_Citation
- + aggregateDataSetIdentifier [0..1] : MD_Identifier
- + associationType : DS_AssociationTypeCode
- + initiativeType [0..1] : DS_InitiativeTypeCode

**DS_AssociationTypeCode**
- + crossReference
- + largerWorkCitation
- + partOfSeamlessDatabase
- + source
- + stereoMate

**Notes:**
The ISO AggregateInformation object provides a powerful mechanism for grouping multiple metadata records into sets based on a number of relationships described by the AssociationType and InitiativeTypeCodes. Note that the sets are identified by MD_Identifiers which suggests that they could have metadata records of their own.

## Does data quality vary within the dataset?

**DQ_Scope**
- + level: MD_ScopeCode
- + extent [0..1] : EX_Extent
- + levelDescription [0..*] : MD_ScopeDescription
**EX_Extent**
- describes the spatial and temporal extent of the quality section.
**MD_ScopeDescription**
- + attributes : Set<GF_AttributeType>
- + features : Set<GF_FeatureType>
- + featureInstances : Set<GF_FeatureType>
- + attributeInstances : Set<GF_AttributeType>
- + dataset : CharacterString
- + other : CharacterString

**MD_ScopeCode**
- + attribute
- + attributeType
- + collectionHardware
- + collectionSession
- + dataset
- + series
- + nonGeographicDataset
- + dimensionGroup
- + feature
- + featureType
- + propertyType
- + fieldSession
- + software
- + service
- + model
- + tile

**Notes:**
ISO 19115 includes a Scope object that allows the specification of what elements in a dataset are described by some other object. This scope can be specified using the ScopeCode list, which is required, a spatial and temporal extent (optional), and a Description that provides more detail.

Where are ScopeCodes?
metadata/hierarchy_level
dataQualityInfo/scope
metadataMaintenance/updateScope

## Do you need to track compliance with standards?

**DQ_ConformanceResult**
- + specification : CI_Citation
- + explanation : CharacterString
- + pass : Boolean

**Notes:**
Standards are critical The ISO Standard include three types of data quality reports. The first, the DQ_ConformanceResult, describes how the dataset was tested for conformance to a published standard and whether the dataset passed the test. The second, the DQ_QuantitativeResult, provides a mechanism for describing the results of a quantitative quality evaluation. Finally, the QE_CoverageResult (from ISO 19115-2), allows a quality result to be expressed as a spatial object. For example, if a gridded dataset has an associated grid of quality flags, that quality grid could be described here. Note also that the coverage results have an associated file and format.

## Do you need to use spatial features to describe quality?

**QE_CoverageResult**
- + spatialRepresentationType: vector, grid, textTable, tin, stereoModel, video
- + resultFile [1]: MX_DataFile
- + resultFormat [1]: MD_format

**Notes:**
The ISO Standard include three types of data quality reports. The  QE_CoverageResult (from ISO 19115-2) is a new report type that allows a quality result to be expressed as a spatial object. For example, if a gridded dataset has an associated grid of quality flags, that quality grid could be described here. Note also that the coverage results have an associated file and format.

## Do you need to keep track of user problems?

**MD_Usage**
- + specificUsage : CharacterString
- + usageDateTime [0..1] : DateTime
- + userDeterminedLimitations [0..1] : CharacterString
- + userContactInfo [1..*] : CI_ResponsibleParty

**Notes:**
Tracking problems that users have with datasets and sharing that information with other users so that they can avoid repeating them is a new feature in ISO 19115. 

## Do you need to describe sources with different spatial and temporal extents?

**LI_Source**
- + description [0..1] : CharacterString
- + scaleDenominator [0..1] : MD_RepresentativeFraction
- + sourceReferenceSystem [0..1] : MD_ReferenceSystem
- + sourceCitation [0..1] : CI_Citation
- + sourceExtent [0..*] : EX_Extent

**Notes:**
Many datasets and products are created by bringing together data and information from multiple sources. Information about these sources is held in the Lineage section of ISO 19115. The Source object includes an extent which allows description of the temporal and spatial extent of each source.

## Do you need to describe requirements and objectives?

**MI_Requirement**
- + citation[0..1] : CI_Citation
- + identifier : MD_Identifier
- + requestor[1..*] : CI_ResponsibleParty
- + recipient[1..*] : CI_ResponsibleParty
- + priority : MI_PriorityCode
- + requestedDate : MI_RequestedDate
- + expiryDate : DateTimeMI

**MI_Objective**
- + identifier[1..*] : MD_Identifier
- + priority[0..1] : CharacterString
- + type[0..*] : MI_ObjectiveTypeCode
- + function[0..*] : CharacterString
- + extent[0..*] : EX_Extent

- 19115-2

**Notes:**
Keeping track of requirements and objectives associated with a mission or project and associating them with datasets is an on-going task for many project managers. The extensions to 

## Do you need to describe platforms and instruments?

**MI_Platform**
- + citation[0..*] : CI_Citation
- + identifier : MD_Identifier
- + description : CharacterString
- + sponsor[0..*] : CI_ResponsibleParty

**MI_Instrument**
- + citation[0..*] : CI_Citation
- + identifier : MD_Identifier
- + description : CharacterString
- + description[0..1] : CharacterString

- 19115-2

## Do you need to explain why you did things to the data?

**LI_ProcessStep**
- + description : CharacterString
- + rationale [0..1] : CharacterString
- + dateTime [0..1] : DateTime
- + processor [0..*] : CI_ResponsibleParty
- + source [0..*]: LI_Source

## Do need to share data with international partners?

**I**nternational
**S**tandards
**O**rganization

## Metadata Standard Selection Guide

- Do you need to unambiguously identify things using your own namespace?
- Do you want to manage metadata using a relational or XML database?
- Do you want to serve metadata using a REST web service?
- Do you need to identify people in different roles?
- Do you need different documentation for different parts of your data?
- Do you need different documentation for different temporal and spatial subsets?
- Do you need to reference On-Line Resources?
- Do you need to describe many kinds of aggregations?
- Does data quality vary within the dataset?
- Do you need to track compliance with standards?
- Do you need to use spatial features to describe quality, like grids of quality flags?
- Do you need to keep track of user problems?
- Do you need to explain why you did things to the data?
- Do need to share data with international partners?

**Yes**

## ISO 19115 is up for a five year review in 2008. This is the time for evolution.

You are the force that influences evolution.

Get Ingenious!

ted.habermann@noaa.gov

- You are the force that influences evolution.
- Get Ingenious!
- ted.habermann@noaa.gov

- The time is now.