# Presentation: 182533

## Identifier Services FrameworkArchitecture/Design Overview,First Results & Next Steps

*caBIG Architecture/Vocabularies and Common Data Elements Workspaces*
*Ohio State University - July 12-14, 2006*
*Frank Siebenlist - franks@mcs.anl.gov*

## caGrid’s Identifiers - Content

- Identifier Service Framework Intro
- GGF’s ID&EPR resolution requirements
- GGF’s WS-Naming Specifications
- Handle System Leverage
- caBIO Integration effort
- Next Steps
- Acknowledgements

## caGrid’s Identifier Services Framework

- Identifier
- “Naming” of individual Data-Objects
- Globally Unique Name for each Data-Object
- Services
- Create/modify/delete name-object bindings
- Resolve name to data-object
- Framework
- Provide for Trust Fabric => Binding Integrity
- Policy-driven Administration => Curator Model
- Fully Integrated with caGrid’s Architecture and Implementation

## Why (Standardized) Data-Object Identifiers?

- Efficiency
- Passing by reference vs by value(Data-Object can be many Mbytes)
- Data-Object Equality test through String comparison(inequality test is no requirement...)
- Consistency
- Standardized way of referencing objects
- Standard identifier => data-object resolution mechanism
- Meta-data binding to standard object reference
- Well-known primary/foreign key for (distributed) JOINs
- Name for policy expression for data-object access
- Name for audit entries about data-object related activities
- ...
- Possible correlation of all of the above...

## Data-Object Identifier Properties

- Identifier is a String
- Identifier is a forever globally unique name for single Data-Object
- Identifier can be (globally) resolved to associated Data-Object
- Data-Objects are immutable, almost immutable or mutable...
- Identifier value “meaningless” opaque string for consumer
- Resolution information embedded in Identifier Name
- Only meaningful for resolution service related components
- Identifier is a Universal Resource Identifier (URI)
- URI-schema will be made completely transparent from Identifier producing applications and consumers.
- ”bigid:” - at least until we have learned more about its usage...(... and to avoid distracting schema-choice discussions)

## Identifier Usage Model

## Naming Authority, Identifier Curator, Data Owner and Identifier User

- Naming Authority (NA)
- Guards integrity of identifier namespace & bindings
- Maintains identifier to data-object’s endpoint mapping
- Conceptually equivalent to caDSR...
- Identifier Curator/Administrator
- Understands semantics/access of data owner’s objects
- Trusted by NA to administer binding for certain identifiers
- Administers identifier to data-object’s endpoint binding
- Data Owner
- Provides access to data-objects through “endpoint-references”
- Identifier User/Consumer
- Trusts an NA for certain identifier bindings
- Uses 2-step resolution to obtain data-object(identifier => endpoint => data-object)
- (In-)Directly trusts Data Owner for data-object integrity

## Identifier Services Framework Requirements

- Fully integrate with caGrid Architecture and Implementation
- WS-Interface specifications and implementations:
- Naming Authority, Identifier Curator and Data Owner Services
- In practice, co-location option of Curator/Data- or NA/Curator/Data Services makes sense
- Java APIs to accommodate co-located functionality
- Abstract as much as possible of framework intrinsics, resolution, and naming schema from identifier producers and consumers
- Ideally it should be a transparent infrastructure service
- Support (secure) Data-Object migration, replication, caching...
- All requirements for truly distributed deployment
- Solid Trust Fabric for Identifier Administration and Resolution
- Success stands or falls with integrity of the underlying framework...
- Leverage existing Identifier framework implementation
- where possible and where it makes sense (Handle System, LSID)

## GGF&OGSA’s WS-Naming RequirementsEPR Minter & Endpoint Identifiers

**Notes:**
This slides shows that GGF’s OGSA has identified a requirement for globally unique identifiers for resources.
In other words, the EPRs are not enough and URI-based identifiers are needed to bind policy, location, EPRs, attributes, audit, etc.
These requirements are very similar to caBIG’s, but come from a different angle.

## GGF&OGSA’s WS-Naming Requirements EPR & Identifier Consumer

## GGF&OGSA’s WS-Naming Requirements EPR, EPI and Message

## GGF’s WS-Naming Requirements EPR Resolution Svcs (all)

## GGF’s WS-Naming Requirements EPR Resolution Svcs (from EndPoint Identifier)

## Identifier & Data Object Model

## caBIG-IRI Naming Convention

- Or a “random” suffix without semantics:
**bigid://1.2.2456/****MRTU4PDCC4HC6MQ4WSEZ2WZOARVRKPEM**
- Identifiers are opaque to applications - they shouldn’t care!!! (implementation choice based on deployment considerations)

## Identifier & Data-Service

## Identifier Consumer

## Identifier Consumer First Step

## Data Object Versioning...

- Complicated...
- Should it be reflected in the Identifier?
- NO
- Versioning should be part of Data Modeling
- “version” part of primary key
- Use cases determine how the versions are used
- Consumer needs interfaces to reflect usage
- Hide consumer from implementation

## Handle System Integration

- CNRI’s Handle System leveraged for the following:
- Global name prefix assignment(similar to dns-ip-name/ip-address registration)
- Global resolution infrastructure(how to find the resolution svcs)
- Identifier’s meta-data repository(context, identification, creation, ..., type, etc.)
- Integrated security model(trust fabric for Naming Authorities, ACL-based admin)
- The open source Handle server code is enhanced to accommodate pluggable co-location with DataSvc(caBIO has >200million data-objects regenerated every 2 weeks...)

## caBIO & Identifiers Requirements (1)

- caBIO creates/regenerates 20-200 million data-object every 2 weeks
- data used from many different sources
- 24 hour regeneration process
- Every (re-)generated data-object should be (re-)assigned an identifier
- Without affecting the regeneration process “too much”
- Same regenerated data-object should be assigned the same identifier as before
- Requires us to bind some data-object identification to the identifier to match-up regenerated data-objects with their previously assigned IDs

## caBIO & Identifiers Requirements (2)

- Anticipate that over their life-time, some data-objects will move to other servers
- To different administrative domain or organization
- Most probably based on “type” or ownership of data-objects
- Some data-objects will not be regenerated
- End of their life-cycle...
- But associated identifiers will live forever
- Existing caBIO query tools should work as before
- But researcher should be able to query specifically for the identifiers
- Given a identifier, a caGrid-client should be able to resolve this ID to the associated data-object
- Global resolution
- Transparent, simple retrieval mechanism

## caBIO & Identifiers Implementation (1)

- Identifiers part of the data-object’s data-model
- Full-fledged attribute with standard name/type
- Existing query tools continue to work
- Application must specify a “data object context”
- Needed at identifier creation time
- administrative “grouping” of IDs for potential moving of data-objects
- Applications must specify “data-object identification info”
- Needed at identifier creation time
- Allows IdSvc-runtime to reassign same ID to same data-object
- Given a identifier, application can ask for associated “data object context” and “data-object identification info”
- Helper function to aide application to locate associated data-object

## caBIO & Identifiers Implementation (2)

- Identifier Service Naming Authority co-located
- Co-located in same JVM & uses same (Oracle) database for ID metadata
- Essential to meet the performance goal of not affecting the re-generation process “too much”
- WS-Naming resolution service implementation
- Allows clients to “find” the data-objects through an identifier
- Based on “emerging” GGF WS-Naming specification
- WS-Transfer GET implementation
- Simple data-object retrieval mechanism
- Based on “emerging” W3C WS-Transfer specification
- Resolution and transfer services implemented through caCore SDK
- Essentially proxied to the caBIO application
- Lightweight registration/call-back pattern used between (caBIO-)application and resolution/transfer implementation
- Minimizes dependencies and improves modularity

## caBIO & Identifiers Integration Results

- Small part of caBIO application has been modified to create IDs
- Data-model has been extended for Gene Domain Object
- IdSvc interfaces used to create/get IDs
- Resolution/transfer functions implemented
- Identifier were created and added to caBIO’s database tables
- Client resolved data-objects through the identifiers
- (results were achieved last Monday&Tuesday...)

## caBIO & Identifiers Integration Next Steps

- caBIO-IdSvc Implementation Guide
- Identification of all the unique keys in each of the caBIO data tables
- Improving performance of identifier creation
- Deployment/packaging of the grid identifier framework
- Improving of JavaDocs and development guide
- Global referral/resolution protocol implementation & standardization
- Not fully implemented yet
- GGF is looking at this caBIG effort for “guidance”

## Identifier Service’s Next Victim: Workflow

- Addresses the use case where the Naming Authority is not co-located with the data-objects
- More “conventional” usage pattern
- Requires webservices interface for identifier creation
- Requires webservice administrative interface for identifier-location binding
- Requires access/admin policy enforcement
- Co-location made this easy
- caBIO and Workflow are expected to provide the basic usage patterns for most of caBIG’s Identifier deployment

## Identifier Services Framework: Next Steps

- High Level Architecture and Design Document (80%)
- Implementation Design Document - (in progress)
- Implementation of WS-Applications, Java APIs & Libraries (80%)
- Documentation & Tutorials (in progress)
- caBIO Integration
- Taking it from prototype to complete integration by 1Q07
- Workflow Integration
- Much “easier” than caBIO from engineering point of view
- Should be able to use IdSvc facilities by Sep/Oct

## Acknowledgements (non-complete...)

- Rachana Ananthakrishnan and Raj Kettimuthu from ANL for the resolution/transfer services
- Lars Olson (UIUC/CNRI) and Sam Sun (CNRI) for the identifier service runtime components
- George Komatsoulis, Doug Mason,  Manav Kher, Vinay Kumar, and the rest of the caBIO team for the integration work
- Our caGrid colleagues for advise and suggestions
- Avinash and Arumani for keeping us on-track...
- Finally... Scott Oster for giving this presentation!
- (and note that we only just started ;-) )