# Presentation: 278825

## Challenges to Developing a Syntax to Capture Metadata Mapping within XMDR

**Challenges to Developing a Syntax to Capture Metadata Mapping within XMDR**

**(or: A Social Science Data Expert Meets Semantics)**
**Fredric Gey**
**UC Berkeley**
**XMDR Project Meeting**
**May, 2007**
**UC Berkeley Faculty Club**

- XMDR Mapping May 2007

**Notes:**
A new name on the marque/masthead/title page credits
Karlo Berket joined LBL’s XMDR Project group as a part time programmer in January
He comes to us from the Cybersecurity and Network :& Distributed Systems Research Group at LBNL,
which continues to occupy and pay for the other half of his working hours.
I might add that both Karlo and Kevin are serious telecommuters – Karlo from MA and Kevin from NY,
But they keep in close touch with us via email, weekly teleconferences, and our project wiki

## XMDR Mapping Capture (outline)

**In January 2007 proposed strawman syntax for XMDR to capture mapping between classification systems**
**Heavily influenced by the SIC NAICS content loading**
**Very naïve compared to real-world challenges**
**Conceptual framework, example problems**
**Drawn principally from ontology mapping research**
**Discussion points for future development**

- XMDR Mapping May 2007

**Notes:**

Tell story about painful upgrade from T23 to T43
Analogy to upgrading 11179 to include more formal semantic information
We’ve been hearing about inference from Frank for a number of years (Prolog, etc.)

Problem that had a silver lining
Karlo machine crashed, so he had to re-install—did it on xmdr server and revised recipe

## Metadata/Ontology Mapping

**Ontology mapping is now a substantial research area**
**Driven by need to integrate disparate ontologies produced by different organizations or different units of one organization**
**Commercial and research tools under development**
**Tools generally solve pieces of the mapping problem**
**Five workshops on the subject – **_**[OM-2006](http://oaei.ontologymatching.org/2006/)**_** was held in Georgia Nov 2006, OM-2007 will be in Busan, Korea this November**
**Evaluating mapping tools is also being researched**
**Can you compare apples and oranges (or rather apple processing equipment versus orange processing equipment)?**
**OM Ontology Alignment Initiative (OM 2004, 2005, **_**[2006](http://oaei.ontologymatching.org/2006/)**_**)**
**Elucidates problems and generates evaluation frameworks**
**Dimensions of mapping challenge pertinent to XMDR**

- XMDR Mapping May 2007

**Notes:**

Tell story about painful upgrade from T23 to T43
Analogy to upgrading 11179 to include more formal semantic information
We’ve been hearing about inference from Frank for a number of years (Prolog, etc.)

Problem that had a silver lining
Karlo machine crashed, so he had to re-install—did it on xmdr server and revised recipe

## Relationship to Schema Mapping

**Schema mapping is closely related to Ontology mapping **
**Schema mapping researched in the database community**
**Driven by need to integrate different databases produced by different organizations or different units of one organization**
**Essential for Federated databases where database administration is under local (distributed) control**
**Simple example “person name”**
**DB1: Name [type text]**
**DB2: Lastname, Firstname, Middleinitial, Title (e.g. Dr, Mr, Ms), Suffix (e.g. Jr, III) **

- XMDR Mapping May 2007

**Notes:**

Tell story about painful upgrade from T23 to T43
Analogy to upgrading 11179 to include more formal semantic information
We’ve been hearing about inference from Frank for a number of years (Prolog, etc.)

Problem that had a silver lining
Karlo machine crashed, so he had to re-install—did it on xmdr server and revised recipe

## Conceptual Context of Mapping

**Mapping (also known as ontology alignment) can consist of**
**Integration of multiple component ontologies into a unified ontology**
**Alignment between part of a global ontology to an external local ontology**
**Alignment between different parts of ‘global’ ontologies**
**Alignment between special subject areas having overlapping instances (e.g. medical literature to computational biology literature)**
**Mapping tools (Noy & Musen 2002) can be **
**Partial not global**
**Be instance-based or not (for common or non-overlapping instances)**
**Be lexical-based (i.e. utilize lexical clues to infer equivalences)**
**May only deal with class hierarchies (not instances)**
**May only produce ‘articulation’ rules between ontologies**

- XMDR Mapping May 2007

**Notes:**

Tell story about painful upgrade from T23 to T43
Analogy to upgrading 11179 to include more formal semantic information
We’ve been hearing about inference from Frank for a number of years (Prolog, etc.)

Problem that had a silver lining
Karlo machine crashed, so he had to re-install—did it on xmdr server and revised recipe

## Strawman Syntax for XMDR mapping from an Old Classification to a New Classification

- <lgRel:association association="**mapsTo**" forwardName="**mapsTo**"
- 	<reverseName="**mappedFrom**“>
- <targetCodingScheme="***coding_scheme_name***">
**-**<lgRel:sourceConcept sourceConcept="***source_concept_code***">
- _[-](file:///../sic-naics/Harold-SIC.xml)_<lgRel:targetConcept targetConcept="*** target_concept_code***** **">
-   	<lgRel:associationQualification associationQualifier="**exact | almost_exact | approx**" />
-                    <lgRel:associationQualification MappingType="**semantic | statistical**" />
-       	<lgRel:associationQualification MapsToDegree="***fraction***" />
-           	<lgRel:associationQualification MapsFromDegree="***fraction***" />
-          	<lgRel:associationQualification MapsToThreshold="***percent***" />
- </lgRel:targetConcept>
-  </lgRel:sourceConcept></lgRel:association>

- XMDR Mapping May 2007

## MappingType
Semantic – meaning of the source concept is aligned with the meaning of the target concept
Statistical – mapping is done using a common database indexed/classified by both source and target concept codes
<associationQualifier="exact | almost_exact | approx" />
exact -  mapping is 1-1
Almost_exact -  for statistical mapping, mapping is almost exact within some ε (epsilon) percent difference as a MapsToThreshold
Approx – mapping is inexact with overlaps between concepts
Degree of (statistical) mapping
<lgRel:associationQualification MapsToDegree="fraction1" />
<lgRel:associationQualification MapsFromDegree="fraction2" />
i.e. a fraction2 of the source concept is represented by fraction1 of the target concept

**MappingType**
**Semantic**** – meaning of the source concept is aligned with the meaning of the target concept**
**Statistical**** – mapping is done using a common database indexed/classified by both source and target concept codes**
- <associationQualifier="**exact | almost_exact | approx**" />
**exact ****-  mapping is 1-1**
**Almost_exact**** -  for statistical mapping, mapping is almost exact within some ****ε**** (epsilon) percent difference as a **_**MapsToThreshold**_
**Approx**** – mapping is inexact with overlaps between concepts**
**Degree of (statistical) mapping**
- <lgRel:associationQualification MapsToDegree="***fraction1***" />
- <lgRel:associationQualification MapsFromDegree="***fraction2***" />
- i.e. a ***fraction2*** of the source concept is represented by ***fraction1*** of the target concept

- XMDR Mapping May 2007

## MappingMode
Automatic – meaning of the source concept is automatically mapped between source and target concepts by some software
Manual – mapping is done with human editors
Observation: Semantic mapping could be automatic if done by NLP software which compares the textual definitions of source and concept codes
ReferenceDatabase  - the detailed database which has been utilized for the statistical mapping (e.g. 1997 Economic Census company establishment for SIC  and NAICS)
Your qualifier goes here

**MappingMode**
**Automatic**** – meaning of the source concept is automatically mapped between source and target concepts by some software**
**Manual**** – mapping is done with human editors**
**Observation: Semantic mapping could be automatic if done by NLP software which compares the textual definitions of source and concept codes**
**ReferenceDatabase****  - the detailed database which has been utilized for the statistical mapping (e.g. 1997 Economic Census company establishment for SIC  and NAICS)**
**Your qualifier goes here**

- XMDR Mapping May 2007

## SIC to NAICS Matching(example for Mineral Industries)

- 47

- XMDR Mapping May 2007

**Notes:**
Note more detailed explanation in NAICS.  Note partial matching

## NAICS to SIC Matching

**Important for historical data comparison and development of time series**

- XMDR Mapping May 2007

**Notes:**
Note more detailed explanation in NAICS.  Note partial matching

## NAICS to SIC Matching

**Mappings are:**
**One to one (rarely)**
**Many to one (sometimes)**
**Many to many (usually)**
**Census Bureau supplies**
**Comparable statistics for 1997 Economics Census **
**Downloadable code files with bridge**
**Open Issue**
**Can statistical allocation between code sets be captured?**
**I.e. NAICS****1**** ******** SIC****1**** (.35) ********SIC****2****(.65)**

- XMDR Mapping May 2007

**Notes:**
Note more detailed explanation in NAICS.  Note partial matching

## What are the Limitations/Assumptions of the Strawman Proposal?

**It assumes the mapping is ‘instance based’**
**Good enough for classification schemes like SIC and NAICS where the instances (reporting of individual business establishment statistics) are identical**
**It only provides for individual concept-to-concept equivalence**
**It assumes the structure of the source and target will provide context of relationships between (exact or amorphous) groups of concepts**
**It does not capture the possibility of large numbers of non-overlapping concepts when the conceptual overlap between two ontologies is small**
**It does not capture structural inconsistency**
**It does not capture temporal drift and evolution**
**It does not capture that corresponding concepts may be considerably more detailed (have more attributes) in a particular ontology **

- XMDR Mapping May 2007

**Notes:**

Tell story about painful upgrade from T23 to T43
Analogy to upgrading 11179 to include more formal semantic information
We’ve been hearing about inference from Frank for a number of years (Prolog, etc.)

Problem that had a silver lining
Karlo machine crashed, so he had to re-install—did it on xmdr server and revised recipe

## Ontology Integration

**Multiple component ontologies are integrated into a single global ontology**
**O****i ****∩ ****O****k ****= ****Ф**** (null set), or**
**O****i ****∩ ****O****k ****≤  ****ε**** **
**Does XMDR need to concern itself with this case?**

**O****global**

- XMDR Mapping May 2007

## Ontology Structural Inconsistency (thanks Mala)

**Ontology Structural Inconsistency (thanks Mala)**

**How do we capture that two different ontologies may have the identical concept at a different structural positions in their hierarchies/directed graphs?**

**O****1**

**O****2**

**C****1****m**

**C****1****m**

**C****1****1**

-        **Computers                                                                         Computing Machines****          /               \                                                                                    /        \****         /                  \                                                                                 /          \   ****    software          hardware                                                    Internet      Standalone****     /          \                       /         \                                              /       \                   /           \****Internet Standalone Internet Standalone               Software Hardware Software Hardware**

- XMDR Mapping May 2007

## Ontology Concept Subset Mapping

**How do we capture that subsets of concepts between two different ontologies are equivalent rather than individual concepts?**

**O****1**

**O****2**

**{C****1****m ****}  ≡  {C****2****s ****}**

**{C****1****k****}   **** ≡**** ****   {****C****2****r****}**

- XMDR Mapping May 2007

## Ontology Overlap/Non-overlap

**How do we capture that two different ontologies each have large numbers of concepts with no corresponding concept in the other ontology?**
**0(****O****1 ****∩ ****O****2****) << 0(****O****1 ****∩ ****O****2****)  **
**Is this important?**

**C****1****m**** = C****2****s**

**C****1****k****    ****≠**** **** ****   ****C****2****r**

**O****2**

- XMDR Mapping May 2007

## Ontology Granularity

**How do we capture that concepts in one ontology may have significantly greater detail than in another ontology**
**In terms of subconcepts**
**In terms of concept attributes/properties**
**Omega is light on geographic places compared to GNS**

**Omega**
**(geography)**

**GNS**
**(Geographic**
**Names **
**Server)**

- XMDR Mapping May 2007

## Ontology Granularity (Omega)

**Omega is light (and inconsistent) on geographic places compared to GNIS**

- XMDR Mapping May 2007

## Ontology Granularity (GNS)

**GeoNames Server search of populated places in Finland**
**Tampere, Finland is described by**
**Feature type PPL (populated place)**
**Latitude/Longitude**
**Name variants**
_**[http://en.wikipedia.org/wiki/Tampere](http://en.wikipedia.org/wiki/Tampere)**_** **

- XMDR Mapping May 2007

## Beyond individual concept mapping
Individual concept to set of concepts
Between sets or clusters of equivalent concepts
Capturing extent of non-overlap
Granularity differences between ’equivalent’ source and target concepts
Capturing structure in the mapping syntax
Dealing with structural mismatches
Can we borrow from the mathematical literature on topological mapping between directed graph structures?

**Beyond individual concept mapping**
**Individual concept to set of concepts**
**Between sets or clusters of equivalent concepts**
**Capturing extent of non-overlap**
**Granularity differences between ’equivalent’ source and target concepts**
**Capturing structure in the mapping syntax**
**Dealing with structural mismatches**
**Can we borrow from the mathematical literature on topological mapping between directed graph structures?**

- XMDR Mapping May 2007