# Presentation: 458973

## Grid Information Service(Meta Directory Service)

- Globus ToolkitTM Developer Tutorial
- The Globus ProjectTM
- Argonne National LaboratoryUSC Information Sciences Instutute
- http://www.globus.org/

## Section Overview

- MDS review
- Information model
- Client tools and APIs
- Configuring servers and adding service providers

- Globus ToolkitTM Developer Tutorial: MDS

## Globus MDS Review

- Meta Directory Service (MDS)
- Globus Toolkit implementation of a Grid Info Service
- System information is critical to operation of the grid and construction of applications
- A basis for configuration and adaptation in heterogeneous, dynamic environments
- Requirements and characteristics
- Uniform, flexible access to information
- Scalable, efficient access to dynamic data
- Access to multiple information sources
- Decentralized maintenance

- Globus ToolkitTM Developer Tutorial: MDS

## “Classic” MDS Architecture (MDS-1)

- Resources push information into a central organization server via regular updates (globus-gram-reporter), where it can be retrieved by clients.
- Regular updates don’t scale as the number of resources grow rapidly.  Commercial LDAP servers are optimized for “read” requests, and can’t handle frequent “write” requests.
- If organization server is unavailable, no information is available.

- Globus ToolkitTM Developer Tutorial: MDS

## “Standard” MDS Architecture (MDS-2)

- Resources run a standard information service (GRIS) which speaks LDAP and provides information about the resource (no searching).
- GIIS provides a “caching” service much like a web search engine.  Resources register with GIIS and GIIS pulls information from them when requested by a client and the cache as expired.
- GIIS provides the collective-level indexing/searching function.

**GIIS**
- Cache contains info from A and B

- GIIS requests information from
- GRIS services as needed.

- Client 1

- Client 2

- Client 3

- Clients 1 and 2 request infodirectly from resources.

- Client 3 uses GIIS for searching
- collective information.

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
GIIS is an index node. Index nodes can be designed and optimized for various requirements

## MDS-2 Service Architecture

- Dynamic Registration via Reg. Protocol (GRRP)
- Resource Inquiry via Info. Protocol (GRIP)
- Co-located with resource on network
- Resource Discovery (via GRIP or other)
- Using GRIP allows resource/directory hierarchy

*discovery (GRIP?)*

*lookup (GRIP)*

*registration (GRRP)*

*VO-specific Aggregate Directories*

*standard Resource Description services*

**R**

**R**

**R**

**R**

**A**

**A**

**?**

- Globus ToolkitTM Developer Tutorial: MDS

## Distributed Services

- Service scales with Grid growth
- Loose consistency model tolerates failures
- Interoperability by GRIP/GRRP protocols

**D**

**D**

**D**

**D**

- Globus ToolkitTM Developer Tutorial: MDS

## Soft-state Registration

- Periodic notification
- “Service/resource is available”
- Expected-frequency metadata
- Automatic directory construction
- Add new resources to directory
- Invite resources to join new directory
- Self-cleaning
- Reduce occurrence of “dead” references

- Globus ToolkitTM Developer Tutorial: MDS

## MDS-2 Implementation

- Grid Information Service (GRIS)
- Provides resource description
- Modular content gateway
- Grid Index Information Service (GIIS)
- Provides aggregate directory
- Hierarchical groups of resources
- Lightweight Dir. Access Protocol (LDAP)
- Standard with many client implementations
- Used for GRIP (and GRRP currently)

- Globus ToolkitTM Developer Tutorial: MDS

## New MDS-2.1 Features

- Security Mechanisms
- GSI mutual-authentication
- Fine-grained access control by GSI name
- Performance Enhancements
- Better query speeds
- Less stale information
- New Information Model (schema)
- Better representation of computers
- Cleaner namespace management

- Globus ToolkitTM Developer Tutorial: MDS

## MDS-2.1 External Software Stack

- OpenLDAP 2.0.x (.14)
- Implements LDAPv3 protocol
- Client and server components
- Cyrus-SASL
- Generic security
- We provide loadable SASL/GSS plugin
- Globus GSI
- Provides GSS-API interface to PKI
- Shared library used by our SASL plugin

- Globus ToolkitTM Developer Tutorial: MDS

## MDS 2.1 Security

- PKI authentication
- Static authorization
- Class, attribute, object name rules
- “Self” authorization
- Semi-dynamic rule
- Requires “owner” attribute on objects
- Dynamic authorization
- Directory-based group lists (or future CAS)
- Per-object access rule attributes
- LDAP dynamic authorization (beta?)

- Globus ToolkitTM Developer Tutorial: MDS

## MDS 2.1 Information Model

- Structural information
- Resource hierarchy maps to objects
- Named positions in LDAP DIT
- Merged information
- Some parents “join” child data
- Simplifies common query patterns
- Auxiliary information
- Uniform representation of leaf/parent data
- Uses LDAP auxiliary objectclasses

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
Auxiliary is a kind of ‘type’. (structural, auxiliary, possibly others). An obj can have 1 structural type only – 
Way of representing a collection of related attributes -  thus can be combined in ways that strucs cannot be
Obj must have exactly 1 structural type, but may have 0 or more aux types also.

Note: this slide is a lead in for the next, which is the picture

## GRIS Host Objects

*OS *

*OS *

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
Dashed boxes are structural types, colors are aux types
Note there are more than 1 aux types for each category (color)

Host, made up of these collections of devices,

Why didn’t we derive more structural types ... no bare structural
Redundant schemas and code to write to make more oopy
Tagged aux type vs sub typed structural type (or algebraic type)
  potentially unbounded number of types at runtime
  we can glue together any combination of types we want at runtime

## GRIS Object Hierarchy

- Mds-Host-name=*hostname*
- Mds-Software-Deployment=operating system
- Mds-Device-Group-name=processors
- Mds-Device-name=cpu 0
- Mds-Device-Group-name=memory
- Mds-Device-name=physical memory
- Mds-Device-name=virtual memory
- Mds-Device-Group-name=filesystems
- Mds-Device-name=/scratch1
- Mds-Device-name=/scratch2
- Mds-Device-Group-name=networks
- Mds-Device-name=eth0

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
Textual representation of the tree. Same as previous slide

## GRIS Structural Class Hierarchy

- Mds
- Attr: Mds-validfrom  (like createtime)
- Attr: Mds-validto (accuracy metadata)
- Attr: Mds-keepto (discard metadata)
- MdsHost
- MdsDevice
- MdsDeviceGroup
- MdsSoftwareDeployment
- Every MDS object: name, time metadata

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
List of structural types we have
MdsHost inherits from Mds

Each struct type has an attr called name example Mds-Host

Just a bag of attributes. Name does NOT identify an attribute instance, but a set of attributes.

Attribute type, class, class name, 
Human readable alias for OID

## GRIS Auxiliary Class Examples

- MdsCpu
- Attr: Mds-Cpu-vendor
- Attr: Mds-Cpu-model
- Attr: Mds-Cpu-speedMHz
- MdsCpuCache
- Attr: Mds-Cpu-Cache-L1kB
- MdsCpuSmp
- Attr: Mds-Cpu-Smp-size
- MdsCpuTotal
- Attr: Mds-Cpu-Total-count

- Once per CPU
- Once per CPU
- Once per SMP
- Once per MPP

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
Mds-Cpu is the class name, vendor is the attr name
LDAP consider the whole string 

## GRIS Auxiliary Class Examples

- MdsCpuFree  (once per SMP)
- Attr: Mds-Cpu-Free-1minX100
- Attr: Mds-Cpu-Free-5minX100
- Attr: Mds-Cpu-Free-15minX100
- MdsCpuTotalFree  (once per MPP)
- Attr: Mds-Cpu-Total-Free-1minX100
- Attr: Mds-Cpu -Total-Free-5minX100
- Attr: Mds-Cpu -Total-Free-15minX100

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
1,5,15 minute averages – normalized by cpu count (fast uptime)
LDAP does not support values with units, so we embed it into the attr type

Attr type, object classes, attr syntax
Human device, what does it mean to you - semantics

Actual representation of the fields – data typing

Fixed precision fractional value (1/2 represented as 50, 1 is 100). Cause there is no floating point attr syntax in LDAP.

## Client Tools

- Globus Toolkit includes 2 command line client tools for querying MDS services
- grid-info-search: General purpose client
- grid-info-search –h <host> -p <port> -b <base> \    -T <timeout> [<filter>] [<attributes>]
- -x: Anonymous access
- grid-info-host-search: Same as grid-info-search, but defaults to GRIS standard port
- E.g. grid-info-host-search –h localhost
- Both clients can search for specific system information and filter results.

- Globus ToolkitTM Developer Tutorial: MDS

## LDAP Client C API

- RFC 1823 defines an IETF draft standard C client API for accessing LDAP databases
- Connect to server
- Pose query which returns data structures contains sets of object classes and attributes
- Functions to walk these data structures
- Globus Toolkit uses OpenLDAP client library

- Globus ToolkitTM Developer Tutorial: MDS

## LDAP Client API: Other Languages

- Java
- JNDI is a standard Java package for accessing LDAP directories
- Used by Java CoG
- Python
- Has libraries that wrap the OpenLDAP client
- Used with PyGlobus
- Perl
- Has modules for LDAP access

- Globus ToolkitTM Developer Tutorial: MDS

## LDAP exercises

- Go to the “ldap” subdirectory
- Documentation
- RFC 1823: The LDAP Application Programming Interface
- Howes and Smith, LDAP: Programming Directory-Enabled Applications with Lightweight Directory Access Protocol, MacMillan 1997ISBN 1-57870-000-0
- Follow instructions in the file README

- Globus ToolkitTM Developer Tutorial: MDS

## Configuring Servers and Adding Service Provider

- Globus ToolkitTM Developer Tutorial: MDS

## Configuration files for Registration

**dc-n1.isi.edu**
- grid-info-site-policy.conf
- grid-info-resource-register.conf
- grid-info-resource-ldif.conf

**dc-n2.isi.edu**
- grid-info-resource-register.conf
- grid-info-resource-ldif.conf

- GRIS

- GIIS

- GRIS

- Globus ToolkitTM Developer Tutorial: MDS

## Configuration files for Registration

- A resource that is hosting a GIIS
- grid-info-site-policy.conf
- Determines whether to accept incoming registrations
- Accept everything, or only registrations from the resources explicitly defined in this conf file (can use wildcards)
- Default policy is to only accept registrations from self, and from port 2135
- In a hierarchical GIIS environment, this file *must* be modified from the default

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
Note that these default policy settings for a GIIS cause a lot of confusion for people first starting to create an indexing structure.

## grid-info-site-policy.conf

- objectclass: MdsRegistrationPolicy
- policydata: (&(Mds-Service-hn=mako.isi.edu)(Mds-Service-port=2135))
**Change ‘policydata’ to this for completely open policy:**
- (Mds-Service-hn=*)
**Change ‘policydata’ to this to restrict to 2 specific hosts:**
- (&(|(Mds-Service-hn=dc-n2.isi.edu)(Mds-Service-hn=dc-n3.isi.edu))(Mds-Service-port=2135))

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
Note on 2nd example: it is best to remove the port restriction all together vs. using an * wildcard. This is one of the first things to check if you are getting no results from a query to a GIIS that is getting registrations (and other data) from other hosts.
Note on 3rd example: dc-n2 and dc-n3 are the only resources allowed to register with this GIIS, and they must do so from port 2135
Note on Wildcards: can also do things like: (Mds-Service-hn=dc-n*.isi.edu) or (Mds-Service-hn=*.isi.edu)

## Configuration files for Registration

- A resource registering GRIS information with a GIIS
- grid-info-resource-register.conf
- Which GIIS’s this GRIS should register to, and how
- GIIS could be on the same machine, but may not be
- grid-info-resource-ldif.conf
- Determines which GRIS providers are active and available to send data to the GIIS’s that this GRIS is registering with

- Globus ToolkitTM Developer Tutorial: MDS

## Registering a GRIS with a GIIS

- dn: <LDAP add object DN>
- regtype: mdsreg2
- reghn: <host to send reg to>
- regport: <port to send reg to>
- regperiod: <how often to send reg (seconds)>
- [service attribute/value]...
- where service attribute entries depend on the type of LDAP object being published

- $GLOBUS_LOCATION/etc/grid-info-resource-register.conf

- Globus ToolkitTM Developer Tutorial: MDS

## ExampleGRIS on dc-n2 registers with GIIS on dc-n1

- dn: Mds-Vo-Op-name=register, Mds-Vo-name=site, o=grid
- regtype: mdsreg2
- reghn: dc-n1.isi.edu
- regport: 2135
- regperiod: 600
- type: ldap
- hn: dc-n2.isi.edu
- port: 2135
- rootdn: Mds-Vo-name=local, o=grid
- ttl: 1200
- timeout: 20
- mode: cachedump
- cachettl: 30

- $GLOBUS_LOCATION/etc/grid-info-resource-register.conf

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
Note that the policy file on dc-n1 must be modified from a default installation to accept this registration

## Registration Control Parameters

- regperiod
- How often this GRIS will send a message to the GIIS noting its existence
- ttl
- How long the recieving GIIS should keep the registration information before assuming that the GRIS is no longer available
- In general ttl should be:  ttl = 2 x (regperiod)
- cachettl
- Recommendation to the GIIS about how long to maintain in cache, the GRIS information provided by this resource

- $GLOBUS_LOCATION/etc/grid-info-resource-register.conf

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
Note that if ttl < regperiod, the data will appear to be intermittently non existent in the GIIS, or could never be returned from the GIIS

## MDS-2.1 GRIS Providers

- grid-info-cpu reports CPU/load info
- grid-info-fs reports filesystem info
- grid-info-mem reports RAM/VM info
- grid-info-net reports NIC/net info
- grid-info-os reports OS info
- grid-info-platform reports arch. info
- grid-info-merged merges all host info*
- Extensible for other sources, e.g. GRAM

- Globus ToolkitTM Developer Tutorial: MDS

## GRIS Dispatch Logic

- For each provider:
- Could search intersect provider? No, then skip.
- Is provider cache stale? Yes, then refill.
- Apply search filter to cache data.
- Combine all intersecting providers’ results

- Globus ToolkitTM Developer Tutorial: MDS

## GRIS Response Issues

- MDS 2.0 and 2.1 are lazy
- Probes are not issued unless queried
- Some system probes are slow
- “Best” probe may take several seconds
- How to avoid stale data?
- Clients set time-out per query
- GRIS/GIIS define time-out per source
- Fresh data found before timeout is returned
- Cache fill continues after client time-out

- Globus ToolkitTM Developer Tutorial: MDS

## 2.1 GRIS Provider Times

| OS | RH 6.2 | RH7.1 | Irix | Solaris |
| --- | --- | --- | --- | --- |
| Platform | 0.04 | 0.03 |  |  |
| Os | 0.06 | 0.04 |  |  |
| Cpu | 0.28 | 0.15 |  |  |
| Mem | 0.12 | 0.10 |  |  |
| Fs | 0.14 | 0.09 |  |  |
| Net | 0.45 | 0.17 |  |  |
| Merge | 0.80 | 0.37 |  |  |
| total | 1.89 | 0.95 |  |  |
| OLD | 10.01 | N/A | 8.29 | 18.27 |

- Globus ToolkitTM Developer Tutorial: MDS

## New Information Providers

- Decide what information to publish into MDS
- Create a program that implements the IO interface requirements of a GRIS Information Provider
- Enable the information provider for an MDS installation

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
Note that an info provider will execute with the same security context as the MDS slapd process since this is the process that will fork and exec the info provider. Important because some people want to create info providers that require access to resources that the slapd process will typically not have access to.
The LDIF data objects that are returned by the info provider must match the schema loaded into MDS. MDS will suppress any results that do not match the schema.

## New Information Providers

- Decide what information to publish into MDS
- OID: conflict avoidance
- OID Registered with IANA Private Enterprise Numbers
- http://www.iana.org/cgi-bin/enterprise.pl
- 1.3.6.1.4.1.3536.* Globus OID subspace
- 1.3.6.1.4.1.3536.2.* Globus Info Services OID subspace
- 1.3.6.1.4.1.3536.2.6.* MDS OID subspace
- ISI will delegate sub-trees of the Globus Info Srvcs OID space
- mailto:mds-oid-registrar@globus.org
**OR**, register your own with IANA
- Object Naming: conflict avoidance
- ISI will coordinate prefix naming. Request a prefix from:
- mailto:mds-oid-registrar@globus.org
- Schema
- Data Modeling problem
- LDAP schema syntax problem
- $GLOBUS_LOCATION/etc/grid-info-resource.schema

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
IANA = Internet Assigned Numbers Authority. For those who do not know IANA, it is the org that handles things like registering new media types for the MIME protocol; registering Internet standard tcp ports; software object id’s; etc.
1.3.6.1.4.1 = IANA Private Enterprise Numbers space. 3536 = Globus registered subspace. Note that 1.3.6.1.4.1.3536.2.*  implies that there are other Globus Information Services (besides MDS).
Object Naming – LDAP standards suggest that all object names be prefixed. All MDS objects are prefixed with mds-*. For example, tacc-* would be a good choice for all schema objects defined by the Texas Advanced Computing Center at UT Austin. HIGHLY recommend that all objects are prefixed to avoid naming conflicts – note that a number of LDAP standard things are not prefixed, so a collision is more likely than one would expect.
At some point, ISI would like to transition the coordination of naming and OID’s to another group, such as GGF. We believe that these issues are grid community issues, but are providing the coordination in order to facilitate rapid/coordinated creation of info providers and evolving monitoring schema.

## New Information Providers

- Create your program
- Any language. The only requirements are the I/O interface of your program:
- Must be callable by fork and exec from the  slapd process
- You can pass in arguments to your program
- Must return data in LDIF format
- Data returned must match the LDAP schema
- RFC 2849

- Globus ToolkitTM Developer Tutorial: MDS

## New Information Providers

- Enable your new information provider by adding a configuration block
- $GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf
- # generate memory info every minute
- dn: Mds-Device-Group-name=memory, Mds-Host-hn=mako.isi.edu, Mds-Vo-name=local, o=grid
- objectclass: GlobusTop
- objectclass: GlobusActiveObject
- objectclass: GlobusActiveSearch
- type: exec
- path: /globus/libexec
- base: grid-info-mem-linux
- args: -devclassobj -devobjs -dn Mds-Host-hn=mako.isi.edu,Mds-Vo-name=local,o=grid -validto-secs 60 -keepto-secs 60
- cachetime: 60
- timelimit: 10
- sizelimit: 3

- Globus ToolkitTM Developer Tutorial: MDS

## New Information Providers

- dn
- Where the object lives in the DIT
- These lines must be included:
- objectclass: GlobusTop
- objectclass: GlobusActiveObject
- objectclass: GlobusActiveSearch
- type: exec
- path
- Path to the information provider program
- base
- Name of the information provider program
- args
- Arguments to be passed to the information provider program
- cachetime
- In seconds, how long GRIS will consider the data to not be stale
- timelimit
- In seconds, how long the GRIS should wait for the information provider to return data before giving up on it
- sizelimit
- Max number of LDIF objects to be read from the output of the information provider

- $GLOBUS_LOCATION/etc/grid-info-resource-ldif.conf

- Globus ToolkitTM Developer Tutorial: MDS

**Notes:**
Objectclass statements are required, but should always be these values for now. May be used for other things in the future.