# Presentation: 051082

## caGrid Version 0.5 DeploymentRegistrationcaBIG Architecture Workspace Face to FaceGeorgetown UniversityAugust 16, 2005

- Scott Oster
- Ohio State University
- oster@bmi.osu.edu

## Outline

- Overview
- Creating Service Data
- Registering Services
- Aggregating Service Data
- Notifying Updates to Service Data

## Basic Processes

**Service Data Creation**
- Grid Services describe their metadata as Service Data Elements (SDEs)
- Clients can query a given service’s SDEs, and even request to be notified of changes to them (subscribe)
**Grid Service Handle Registration**
- Upon startup, Grid Services notify a registry (Index Service) of their existence and caBIG-relevant SDEs
**Service Data Aggregation**
- The Index Service then creates an entry in its list of services and subscribes to the SDEs the service told it about
**Notification of Service Data Updates**
- As Grid Services modify their SDEs (if ever), their new values are sent to anyone subscribed to them
- Before a Grid Service shuts down, it attempts to notify the registry which will remove it from its list of services

## Service Metadata Registration and Aggregation Review

- All services register their GSH and SDE subscription information to an Index Service
- The Index Service subscribes to the standardized SDEs and aggregates their contents

- Clients can discover services using a discovery API which facilitates inspection of data types
- Leveraging semantic information in EVS (from which SDEs are drawn), services can be discovered by the semantics of their data types

## Service Data Creation

- Service Data Elements (SDEs) represent a service’s dynamic state or metadata.
- SDEs are described by XML Schemas and represented to clients as Objects or XML documents
- Can be statically defined in a service’s GWSDL or dynamically created

## Creating an SDE in a Grid Service

**Creating an SDE in a Grid Service**

-  public void postCreate(GridContext context) throws GridServiceException    {
- 	super.postCreate(context);
-       // Create Service Data Element (published under MathData name)
-       **mathDataSDE** = this.getServiceDataSet().**create**("MathData");
-       // Create a MathDataType instance and set initial values
-       mathDataValue = new MathDataType(); // the Auto-Generated Java Bean representation
-       mathDataValue.setLastOp("NONE");
-       mathDataValue.setNumOps(0);
-       mathDataValue.setValue(0);
-       // Set the value of the SDE to the MathDataType instance
-       **mathDataSDE.setValue(mathDataValue);**
-       // Add SDE to Service Data Set
-       **this.getServiceDataSet().add(mathDataSDE);**
-     }

## Creating an SDE with caGrid

- caGrid provides standard XML schemas (and Java Beans) for standardized metadata
- caGrid deployment tools provide ability to create standard SDEs simply by selecting a file containing an XML representation
- Leverages Globus’s Monitoring and Discovery System (MDS)
- MDS Service Data Providers create caGrid standard SDEs

## Service Data Provider Configuration

- <service ...>
-   ...
- <parameter name="operationProviders“ value="**org.globus.ogsa.impl.base.providers.servicedata.ServiceDataProviderManager**"/>
- <parameter name="**serviceConfig**" value="etc/caGrid-Analytical-SDE-config.xml" />
- ...
- </service>

- <serviceConfiguration ...>
- ...
- <executedProviders>
-     <pe:ServiceDataProviderExecution>
-         <pe:serviceDataProviderName>AsyncDocument</pe:serviceDataProviderName>
-         <pe:serviceDataProviderImpl>org.globus.ogsa.impl.base.providers.servicedata.impl.AsyncDocumentProvider</pe:serviceDataProviderImpl>
-         <pe:serviceDataProviderArgs>-f etc/**commonSDE.xml**</pe:serviceDataProviderArgs>
-         <pe:serviceDataName xmlns:com="http://cagrid.nci.nih.gov/1/CommonServiceMetadata">
-             **com:CommonServiceMetadata**</pe:serviceDataName>
-         <pe:refreshFrequency>60</pe:refreshFrequency>
-         <pe:async>true</pe:async>
-     </pe:ServiceDataProviderExecution>
-     ...
- </executedProviders>
- </serviceConfiguration>

- Service’s wsdd loads the Service Data Provider Manager and configures it with a specified configuration file

- Service's Deployment Descriptor (wsdd)

- Service Data Provider Manger Configuration File

## Grid Service Handle Registration

- Registration is a core Globus (and OGSI) concept
- Globus Service Group Registration
- org.gridforum.ogsi.ServiceGroupRegistration
- Generic Interface, generally extended by specialized registries
- Provides basic registration (add) and deregistration (remove)

- TerminationTimeType  **add**(LocatorTypeHolder serviceLocator, ExtensibilityType content, TerminationTimeType terminationTime)

** **void  **remove**(ExtensibilityType matchExpression)** **

**Add** specifies the registering Service’s “address”, the duration of registration, and an extensible group of registry-specific content
**Remove** species a group of registry-specific content used to specify the registration to delete

## Programmatic Registration

- // locate the registry
- ServiceGroupRegistration **registry** = ...;
- TerminationTimeType **time** = new TerminationTimeType();
- ExtendedDateTimeType dateTime = new ExtendedDateTimeType();
- Calendar cal = Calendar.getInstance();
- dateTime.setValue(InfinityType.infinity); // never expiring registration
- time.setAfter(dateTime);
- time.setBefore(dateTime);
- ExtensibilityType **contents** = new ExtensibilityType();
- // ... add registry-specific contents here ...
- // locate the registering service
- LocatorTypeHolder **member** = ...;
- // register the service
**registry.add(member, contents, time);**

## Registering with caGrid

- caGrid deployment tools provide ability to register to a registry simply by selecting an appropriate registry GSH
- Leverages Globus’s Registry Publish Provider

## Registry Publish Provider Configuration

- <?xml version="1.0" encoding="UTF-8" ?>
- <serviceConfiguration>
-   <registrations>
-     <registration registry="**http://cagrid-registry.nci.nih.gov:8080/ogsa/services/base/index/IndexService**“
-         keepalive="true" lifetime="1200" remove="true">
-     </registration>
- </registrations>
- </serviceConfiguration>

- <service ...>
-   ...
- <parameter name="operationProviders“ value="**org.globus.ogsa.impl.core.registry.RegistryPublishProvider**"/>
- <parameter name="**registrationConfig**" value="etc/caGrid-Analytical-SDE-registration.xml" />
- ...
- </service>

- Service's Deployment Descriptor (wsdd)

- Registry Publish Provider Configuration File

- Service’s wsdd loads the Registry Publish Provider and configures it with a specified configuration file

## Service Data Aggregation

- In order for discovery queries to work, services’ metadata must aggregated
**MDS Aggregator API** leverages lower-level Globus capabilities to create a coupling between common data collection mechanism and common processing mechanisms
**DataAggregationSource**
- Produces data for aggregation
- Example: *SubscriptionAggregationSource* uses OGSI subscriptions
**DataAggregationSink**
- Consumes aggregated data
- Example: *ServiceDataAggregationSink* receives service data and replaces or appends aggregated service data values

## Globus Index Service

- One of the GT3.2 Base Services (WS Information Services)
- Composes Globus Registry components with MDS Aggregator and Service Data Provider APIs
- Collects, aggregates, and queries Service Data
- Creates aggregate Service Data dynamically on demand

## Globus Index Service cont.

- Index Service specializes the extensible content of the registry interface to allow specification of Data Aggregation information
- Index Service uses some additional attributes of the SDEs it aggregates
**gt:originator**: Specifies the GSH of the service from which the SDE originated (as SDEs may be aggregated up a hierarchy)
**ogsi:goodUntil**: Specifies the DateTime after which the SDE should be considered invalid (Time To Live)
**ogsi:goodFrom**: Specifies the DateTime after which the SDE should be considered valid
**ogsi:availableUntil**: Specifies the DateTime until which the SDE should be available

## caGrid Service Data Aggregation

- Using dynamic keep-alive indexing
- Registered services periodically renew subscriptions
- Services automatically reregister in the face of a registry failure
- Services attempt to deregister during a “clean” shutdown
- Index Service drops old entries
- “Dead” services are pruned from the registry
- Registry Publish Provider configuration is extended to include SDE subscription information
- Configured automatically based on the type of service being deployed
- The additional SDE attributes needed by the Index Service are automatically added by the Service Data Providers

## Registry Publish Provider Configuration for Aggregation

- Configuration file now specifies details of subscriptions to SDEs in the registering service which the Index Service should make

- <?xml version="1.0" encoding="UTF-8" ?>
- <serviceConfiguration xmlns:ogsi=“http://www.gridforum.org/namespaces/2003/03/OGSI”
-     xmlns:aggr=“http://www.globus.org/namespaces/2003/09/data_aggregator”>
-   <registrations>
-     <registration
-        registry="**http://cagrid-registry.nci.nih.gov:8080/ogsa/services/base/index/IndexService**"
-        **keepalive**="true" **lifetime**="1200" **remove**="true">
-       <aggr:DataAggregation>
-         <ogsi:params>
-           <**aggr:AggregationSubscription**>
-             <ogsi:serviceDataNames>
-               <ogsi:name xmlns:a="http://cagrid.nci.nih.gov/1/AnalyticalServiceMetadata">a:**AnalyticalServiceMetadata**</ogsi:name>
-               <ogsi:name xmlns:c="http://cagrid.nci.nih.gov/1/CommonServiceMetadata">c:**CommonServiceMetadata**</ogsi:name>
-             </ogsi:serviceDataNames>
-             <aggr:**lifetime**>60000</aggr:lifetime>
-           </aggr:AggregationSubscription>
-         </ogsi:params>
-       </aggr:DataAggregation>
-     </registration>
- </registrations>
- </serviceConfiguration>

## Notification of Service Data Updates

- Globus provides basic publish/subscribe mechanism for extensible content (operation providers and standard port types)
- osgi:NotificationSource
- ogsi:NotificationSink
- NotificationSinkManager is a Class simplifying client side sink setup and notification subscriptions.

- void** subscribe**(ExtensibilityType subscriptionExpression, LocatorType sink, ExtendedDateTimeType expirationTime, LocatorTypeHolder subscriptionInstanceLocator, TerminationTimeTypeHolder currentTerminationTime)** **

- void **deliverNotification**(ExtensibilityType message)

## caGrid Notification

- caGrid Data services use a custom factory (CaGridDataServiceFactoryPortType) which extends OGSA-DAI’s GridDataServiceFactoryPortType and Globus’s NotificationSource
- caGrid Analytical services extend Globus’s NotificationSource
- All caGrid services leverage Globus’s Notification Source Provider to implement the necessary notification capabilities
- Service Data Providers periodically refresh SDEs which create notification of the new values to the Index Service

## Review of Components

## Summary of caGrid Registration

**Service Data Creation**
- Deployer handles automatically (developer is prompted for necessary information)
- ServiceDataProviderManager configured to use AsynDocumentProviders
**Grid Service Handle Registration**
- Deployer handles automatically (developer is prompted for Index Service GSH)
- RegistryPublishProvider configured to register to Index Service
**Service Data Aggregation**
- Deployer handles automatically
- RegistryPublishProvider configured with appropriate DataAggregation information for each service type
**Notification of Service Data Updates**
- caGrid Services automatically support subscription and notification
- Data Services use a factory which extends OGSA-DAI’s GDSF as well as ogsi:NotificationSource
- Analytical Services extend ogsi:NoticationSource

## Acknowledgements