# Presentation: 425269

## Metadata, Annotations, and Pedigree: Enabling Knowledge Discovery and Enhancing Collaboration in the CMCS

- Carmen M. Pancerella
- Sandia National Laboratories
- carmen@ca.sandia.gov
- April 30, 2003

## Data and metadata

- CH3OOH

**	**H°atomiz   (	) =

- ±

**	**kcal/mol

**: **value and uncertainty

**	**** **data

- units: kcal/mol

- quantity: enthalpy of atomization

- species: methylhydroperoxide, CAS# 3031-73-0

- temperature: 0 K

- 0

- calculated: G3//B3LYP
- creator: T. Windus using Ecce
- more info: http://avatar.emsl.pnl.gov:8080/Ecce/.../CH3OOH/.../GxEnergy

*[calculated, G3//B3LYP, T. Windus, more at http://...]*

## Metadata Converts Scientific Data into Knowledge

- Metadata provides identification and documentation to scientific data.
- Example: Attaching an owner, creation date, abstract, type to data.
- Example: Tracking data to program versions, and possibly bugs for that version.
- Metadata documents the context and value of the data.
- Example: The theoretical atomization energy of methylhydroperoxide (and its uncertainty) from Ecce (used as input to ATcT) contains information identifying the species and the quantity, units, the theoretical method used, vibrational frequencies and geometry, reference to source file, creator, etc.
- Metadata facilitates cross-scale transfer of data.
- Example: Can show a chain of inputs, including input parameters and configuration files, across scales.
- Example: Can retrieve literature references which describe this data.
- Metadata allows users to comment on the data and its quality.
- Example: CMCS infrastructure can be used for scientific peer review of data.
- Metadata is necessary for effective collaboration.
- Example: Scientific data becomes more usable to others when it is documented.

**Annotation is another term for metadata.  Annotations can be added by ****either the data owner or a third party. **

## Data Pedigree: A Special Kind of Metadata

- Data pedigree or data provenance is a relationshipwhich provides a “line of ancestors”.
- Pedigree allows for the categorization and tracing of the scientific data, and for the identification of the data’s ultimate origin, possibly across scales.
- Pedigree metadata is associated with CMCS resources, and is browsable and searchable from the CMCS portal.
- Pedigree includes the series of steps necessary to reproduce the data.
- Data is linked to projects, references, inputs, and outputs.

## Data/Metadata Management Standards

- WebDAV -- Web Digital Authoring and Versioning
- Extension to HTTP for file management and collaborative file sharing on remote web servers
- Files/collections and properties (data about data)
- Methods: GET, POST, PUT, COPY, MOVE, DELETE, LOCK/UNLOCK, MKCOL, PROPFIND, PROPPATCH
- DASL (DAV Searching and Locating) -- efficient search for DAV
- http://www.webdav.org
- XML -- Extensible Markup Language
- XML Schema -- self-describing file format
- XSLT (XML Stylesheet Language Transformation)
- Translate from XML to HTML
- Translate from XML documents  to other XML documents
- Progressive condensation of diverse XML formats into more widely adopted formats
- http://www.xml.org

## Data/Metadata Management Standards

- RDF -- Resource Description Framework
- Enables encoding, exchange, reuse, and hence, interoperability of structured metadata
- Machine understandable semantics, developed for Semantic Web
- What is difference between XML and RDF?
- RDF is more expressive -- triples
- RDF data model can be expressed in XML
- RDF is a framework to realize potential in XML
- XLink -- XML Linking Language
- Advanced, scalable, and maintainable hyperlinking and addressing functionality for XML
- Specifies constructs that may be inserted into XML resources to describe links between objects
- SAM -- Scientific Annotated Middleware
- Built on top of WebDAV, in particular Jakarta Slide
- Jim Myers, PNNL, P.I.        http://www.scidac.org/SAM

## CMCS Metadata Stored as WebDAV Properties

- DAV property is a keyword/value pair: namespace:tag, and a well-formed XML value.

## How Metadata is Populated in CMCS

- SAM Metadata Services Layer
- When data is put into WebDAV, SAM causes XSLTs to be executed to extract metadata from XML files, based on MIME type.
- Similarly, Binary File Descriptor (BFD) provides an interface to extract metadata from binary files.
- CMCS data management/pedigree API to facilitate insertion and modification of metadata, in the proper XML format.
- Java code which allows software developers and scientists to easily write programs to add/edit metadata.
- Scientists can use these APIs to integrate with existing or new chemical science applications.
- Uses open source DAV and XML libraries.
- Any WebDAV client application
- DAVExplorer: Java application
- CMCSExplorer: Integrated in the CMCS portal

## CMCS Metadata, Annotations, and Pedigree

- Using Dublin Core for some basic pedigree properties of electronic publication: creator, dates, publisher, is-referenced-by, references, etc.
- Digital library standard for metadata
- http://www.dublincore.org
- CMCS properties for Chemical Science to enable searching: species name, CAS, chemical properties, and chemical formula.
- CMCS properties for defining scientific data: inputs, outputs, and is-part-of-project.
- CMCS properties for scientific publication and peer review annotations: is-sanctioned-by.
- Currently defined 36 elements in the core CMCS pedigree.
- Flexible infrastructure for addition of new metadata.  As new metadata is added to infrastructure,current apps will not break!

**CMCS metadata is strongly encouraged, though not required, for all CMCS data, and CMCS metadata is highly extensible. **

## Metadata at Work in CMCS: Chemistry Search

- Searches CMCS defined metadata (keywords, chemical species, chemical property, creator)
- Chemical Species can be defined by Name, Formula, or Chemical Abstracts Service (CAS) number.
- NIST Species Dictionary Web Service
- De-references aliases
- NIST Kinetics Database (in progress)
- 10,000 literature references and
- 100000 reactions
-       mapped as searchable, annotatable
-       CMCS webDAV resources

## Metadata at Work : Support Peer Review of Data

## Pedigree Browser Returns         Project Information	

## Pedigree Browser Returns Literature References

## Pedigree Browser Shows Input and Output Relationships	

## Pedigree Graph in CMCS

**Pedigree graph provides a two-dimensional visualization of a data resource and all of its relationships.   Users can easily see the relationships without following all pedigree links.**

## CMCS Pedigree Graph Shows Relationships Between Resources

- SAM returns GXL (Graph eXchange Language) with pedigree relationships, configurable at SAM server.
- GXL is converted to SVG (Scalable Vector Graphics) for presentation.
- Open issues and desired features: implementation and presentation of pedigree graphs.
- Appropriate graph layout for all pedigree graphs
- Use of colors to show different relationships
- Managing stale links
- Spanning multiple DAV servers
- User-configurable (length of pedigree tree and properties returned)
- Handling versioning

## Data + Metadata = Knowledge

- Metadata allows the creation of revolutionary new types of collaborative chemical science projects, coordinating research efforts across chemical scales.
- Metadata dramatically increases accessibility of state-of-the-art research knowledge to industry and the public.
- CMCS data repository with pedigree tracking provides a new means for evaluating research claims.
- CMCS pedigree graphing provides a visual representation of data relationships.
- CMCS metadata supports allows scientific peer review of electronic data, which is important to groups like IUPAC.
- CMCS metadata is extensible.
- CMCS needs to explore guidelines for others to annotate data.

## Extra Slides Follow

## Grid Fabric

**Chemical**
**Mechanisms**

**Reacting**
**Flow**

***Grid Fabric***
**Storage   Security     Event Services     Directory Services**

**Quantum**
**Chemistry**

**Kineticist**

**Thermo-**
**Chemistry**

**Kinetics**

***Shared Data Repository***

**XML **
**Data Set**

**Binary **
**Data Set**

***Scientific Annotation Middleware***
**Parsers  Translators  Annotators  Notifications  WebDAV**

**XML **
**Data Set**

**Text **
**Data Set**

***Multi-scale Chemical Science Portal***

**Community**
**Tools**

**Knowledge**
**Management**
**Tools**

**Research Support**
**Tools**

**Thermochemist**

**Metadata**

**Metadata**

**Metadata**

## Metadata at Work: Data Viewer Registered With SAM

## Metadata at Work in the CMCS: Search

## Metadata Editing in CMCSExplorer