# Presentation: 215443

## Scientific Annotation Middleware (SAM)8/19/2003 Review

- Jim Myers, Alan Chappell, Matt Elder, Mike Peterson, Tara Talbott, Prasad Saripalli  - PNNL
- Al Geist, Jens Schwidder, David Jung - ORNL

- $500K/yr, 5 year project

## SAM Project Objectives

- Develop a lightweight, flexible middleware to support the creation and use of metadata and annotations
- owner, pedigree, classification, dependencies, ...
- Enable the sharing of annotations among portals and problem solving environments, software agents, scientific applications, and electronic notebooks
- Improve the completeness, accuracy, and availability of the scientific record.

## Approach

- Middleware design capable of integrating into multiple service architectures
- “Schema-less” store that accepts arbitrary content and metadata
- Dynamic Metadata/Data transformations to support evolving standards and lightweight integration
- Layered design to allow basic and advanced clients and interactions between them
- Support evolving science practice without introducing barriers...

## Architecture

## Collaboratory for Multiscale Chemical SciencePortal Demonstration

**Collaboratory for Multiscale Chemical Science****Portal Demonstration**

- Web Folder  SAM
- ELN
- Add content
- Annotate existing data
- View data in notebook
- Portal  View ELN links and information in Pedigree Browser

## SAM: General Features

- Extends Jakarta Slide, uses OpenJMS, Jaxen, JDOM, COG
- Supports WebDAV protocol standards for data/metadata access, search, ...
- Supports external authentication and authorization mechanisms
- Data Store interface allows data/metadata to be stored in files, relational DBs, DataGrids, ...
- Client-side protocol, API, and component-level interfaces

## Web Distributed Authoring and Versioning (WebDAV)

- An early web service
- Put/Get data with arbitrary properties (dynamic)
- Properties can be discovered and accessed independently
- DASL, Versioning, Transactions, ...
- Widely supported (MS Office, databases, file system drivers,...)

## Configurable Security

**Configurable Security**

- Authentication/coarse authorization via JAAS modules within Tomcat Realm or Request Filter
- Credentials handled as opaque token within SAM
- Authorization API – subject-action-resource sent to authn. module for decision
- Lower-level API for user/group/role management

- MyProxyserver

- SAM Server

- Client

- name/password

- Proxy
- Certificate

- Proxy certificate

- GridFTP
- server

- name/password

***Supports Grid/non-Grid use, and***
***Integration with multiple access control mechanisms***

- Authn

- JAAS

- Y/N

- Cred

- GridStore

## Jakarta Slide API for storing
data, 
metadata, 
structure, 
versions, 
locks, 
access control information
SAM XML Configurable JDBC Store
SAM GridFTP Store

- Jakarta Slide API for storing
- data,
- metadata,
- structure,
- versions,
- locks,
- access control information
- SAM XML Configurable JDBC Store
- SAM GridFTP Store

**Data Store Interface**

- Content

- Property

- Property

- Property

- type

- size

- path

- ACLs

- Lock

- Version

- URL

- Store Implementation(s)

- SAM
- Database(s)

- Third Party
- Database

- File System

- Web

- SAM/
- WebDAV

- GET/PUT

- PROPFIND/
- PROPPATCH

- MKCOL

- ACL

- Property

## Client Access 

- WebDAV
- Protocol
- Client libraries (SAM/Slide, CMCS DSI, PerlDAV, ...)
- Apps (DAVExplorer, Cadaver, ...)
- File System Drivers (WebDrive, DAVFs, ...)
- Web Form Protocol (PUT/POST) for MMS Configuration and NS Layer
- Web/Grid Services
- Notarization Services

## SAM Metadata Services Layer

**SAM Metadata Services Layer**

- Property Generation from binary/ASCII/XML files
- Dynamic Content Translation
- “hastranslations” property
- JMS Events for access and changes

- Fortran
- Application

- ‘Local
- Disk’

- DataGrid

- SAM

- DAV

- DAV+

- JMS

- ELN

## Binary Format Description (BFD) Language

- XML Language to describe data formats
- Based on XSIL
- Delimiters
- References
- If,For loop logic
- Generic Parser to extract and semantically tag data in files/streams

- <XSIL>
-   <Param Name=“units”>meters</Param>
-  <Param Name=“numColumns” Type="int“/>
-   <Vector 	Name=“orbitData”>
-     <Dim><XBFDvalue-of
-                    select="/XSIL/Param
-                    [@Name='numColumns']"/>
-     </Dim>
-     <Dim>4</Dim>
-   </Vector>
-   <Stream Type=“remote”
-                 XBFDStreamnumber=“0”
-                 Encoding = “biinary”/>
- </XSIL>

## SAM Semantic Services Layer

**SAM Semantic Services Layer**

- SAM Metadata Layer plus configurable:
- Translation of DAV Properties to RDF Triples
- RDF/GXL Pedigree Generation
- RDF-based Mapping of Properties
- RDF-based Data Translation
- ...

## SAM Notebook Services Layer

- SAM Metadata and Semantic Layers plus:
- Notebook Management, Page Display, ...
- Notarized Timestamps
- Notarization Proxy and Notarizer Web Services
- Digital Signatures
- Canonicalization
- Data/Signature Migration Capabilities
- Notebook API, Notebook Components

## Transitional Electronic Notebook

- Refactored DOE2000 ELN
- Encapsulation of all server dependencies
- SAM Client Libraries
- Use WebDAV and SAM NS
- Structure and metadata captured in WebDAV properties
- ELN Launch Portlet
- ELN URL Editor
- Allows inclusion/annotation of data available via a URL
- Annotation Applet/Portlet
- Sample SAM notebook component

## SAM Components and Notebook Interface

- Web components for interacting with SAM ala CMCS Explorer/Pedigree Browser at MMS, SS, and NS layers
- Redesigned Notebook Interface as a next-generation notebook system and demonstration of Components

## Project Management

- Weekly Phone Meetings
- Shared CVS Repository, ant-based builds
- Project Notebook
- Project Website with overview, plans, progress, papers, software, bug report/help desk form ...
- Open Source Licensing in progress

## Collaboratory for Multiscale Chemical Science (CMCS)
SAM as primary data system, pedigree, notebook 
CHEF Portal/ Development of Grid User Computing Environments (GUCE) Project
SAM as shared doc/data tool, alternative collaborative tool data store, notebook
Genomes-To-Life
SAM as annotation/metadata repository, notebook
Internal PNNL Projects
Concept Map Repository, Interface to Lustre, Biological Data Annotation
DOE2000 Notebook Community (1500+ email addresses)
Upgrades to DOE2K Notebooks
E.g. Columbia University Environmental Science Lab Notebooks
Other (Climate, Geo, CMCS/CHEF collaborations, ...)
Discussions, pilot testing, ...

- SAM as primary data system, pedigree, notebook
- CHEF Portal/ Development of Grid User Computing Environments (GUCE) Project
- SAM as shared doc/data tool, alternative collaborative tool data store, notebook
- Genomes-To-Life
- SAM as annotation/metadata repository, notebook
- Internal PNNL Projects
- Concept Map Repository, Interface to Lustre, Biological Data Annotation
- DOE2000 Notebook Community (1500+ email addresses)
- Upgrades to DOE2K Notebooks
- E.g. Columbia University Environmental Science Lab Notebooks
- Other (Climate, Geo, CMCS/CHEF collaborations, ...)
- Discussions, pilot testing, ...

**Collaborations**

## Community Interactions

- Java Content Repository (JSR 170) Expert Group
- Global Grid Forum
- Grid Information Retrieval
- Grid File System
- Data Format Description Language
- ...
- Science of Collaboratories Project
- Project/Peer Reviews, NEESGrid Advisory Board
- Presentations/Papers in Science, Information, and Records Communities

## Plans – Short Term (’03-’04)

- Migration to Slide 2 (DASL, active development)
- Version 1.5 MMS (web service transforms, hardening, security APIs, management interfaces)
- Semantic Service Layer (RDF-based scoping/ DASL queries)
- SAM Notebook Interface/Components (portlet-based)
- Continuing interactions with Collaborators, Community
- (third-party annotation, review, ...)

## Plans – Longer Term (’04-’06)

- Web/Grid Service Interfaces
- Jakarta Slide 3 Migration (JSR 170 Compliant)
- Metadata Management Services (relationship management agents, transaction-aware)
- Semantic Services (relationship inference, workflow-by-example)
- Notebook Services (records capabilities)
- Hardening...

## Acknowledgments

- U.S. Department of Energy
-  Pacific Northwest National Laboratory
- Pacific Northwest National Laboratory is a multiprogram national laboratory operated by Battelle Memorial Institute for the U.S. Department of Energy under Contract DE-AC06-76RL0 1830
- Oak Ridge National Laboratory
- Oak Ridge National Laboratory is a multiprogram national laboratory operated by UT-Battelle, LLC for the U.S. Department of Energy under Contract DE-AC05-00OR22725

## Scientific Annotation Middleware

- _[http://www.scidac.org/SAM/](http://www.scidac.org/SAM/)_
- Jim.Myers@pnl.gov