# Presentation: 551110

## Grid Computing and the Globus Toolkit

- Jennifer M. Schopf
- Argonne National Lab
- National eScience Centre

## What is a Grid?

- Resource sharing
- Computers, storage, sensors, networks, ...
- Sharing always conditional: issues of trust, policy, negotiation, payment, ...
- Coordinated problem solving
- Beyond client-server: distributed data analysis, computation, collaboration, ...
- Dynamic, multi-institutional virtual orgs
- Community overlays on classic org structures
- Large or small, static or dynamic

## Why Is this Hard or Different?

- Lack of central control
- Where things run
- When they run
- Shared resources
- Contention, variability
- Communication
- Different sites implies different sys admins, users, institutional goals, and often “strong personalities”

## So Why Do It?

- Computations that need to be done with a time limit
- Data that can’t fit on one site
- Data owned by multiple sites
- Applications that need to be run bigger, faster, more

## What Kinds of Applications?

- Computation intensive
- Interactive simulation (climate modeling)
- Large-scale simulation and analysis (galaxy formation, gravity waves, event simulation)
- Engineering (parameter studies, linked models)
- Data intensive
- Experimental data analysis (e.g., physics)
- Image & sensor analysis (astronomy, climate)
- Distributed collaboration
- Online instrumentation (microscopes, x-ray) Remote visualization (climate studies, biology)
- Engineering (large-scale structural testing)

## Key Common Feature

-    The size and/or complexity of the problem requires that people in several organizations collaborate and share computing resources, data, instruments

## The Globus Approach

## The Role of the Globus Toolkit

- A collection of solutions to problems that come up frequently when building collaborative distributed applications
- Heterogeneity
- A focus, in particular, on overcoming heterogeneity for application developers
- Standards
- We capitalize on and encourage use of existing standards (IETF, W3C, OASIS, GGF)
- GT also includes reference implementations of new/proposed standards in these organizations

**Notes:**
Heterogenity: 

Trying to support heterogenity:  no preconditions:
	- different hardware (types of machines, different configurations, different “data producers”, etc.)
	- different software (different applications, different languages, different operating systems, etc.)

Standards:
	Standards enable interoperability.  An example:
		- If the community agrees to standardize certain patterns for interacting with a type of service, then services
can interact with each other
		- WSRF

## Globus is an Hour Glass

- Local sites have an their own policies, installs – heterogeneity!
- Queuing systems, monitors, network protocols, etc
- Globus unifies
- Build on Web services
- Use WS-RF, WS-Notification to represent/access state
- Common management abstractions & interfaces

- Local heterogeneity

- Higher-Level Services
- and Users

- Standard GT4
- Interfaces

## On April 29, 2005 the Globus Alliance releasedthe finest version of the Globus Toolkit to date!

**On April 29, 2005 the ****Globus Alliance released****the finest version of the Globus Toolkit to date!**

- Don’t take our word for it!
- Read the **UK eScience Evaluation of GT4**
- www.nesc.ac.uk/technical_papers/UKeS-2005-03.pdf
- (Reachable from www.globus.org, under “News”)

## How it Really Happens

- Implementations are provided by a mix of
- Application-specific code
- “Off the shelf” tools and services
- Tools and services from the Globus Toolkit
- Tools and services from the Grid community (compatible with GT)
- Glued together by...
- Application development
- System integration

## A Typical eScience Use of Globus:Network for Earthquake Eng. Simulation

-  Links instruments, data,
- computers, people* *

## Without the Globus Toolkit

- WebBrowser

- ComputeServer

- DataCatalog

- DataViewerTool

- Certificate
- authority

- ChatTool

- CredentialRepository

- WebPortal

- ComputeServer

*Resources implement standard access & management interfaces*

*Collective services aggregate &/or virtualize resources*

*Users work with client applications*

*Application services organize VOs & enable access to other services*

- Database
- service

- Database
- service

- Database
- service

- SimulationTool

- Camera

- Camera

- TelepresenceMonitor

- RegistrationService

- A

- B

- C

- D

- E

| Application Developer | 10 |
| --- | --- |
| Off the Shelf | 12 |
| Globus Toolkit | 0 |
| Grid Community | 0 |

**Notes:**
Example application

This user needs access to particular resources (computers, cameras and databases)

In this case the application developer has a lot of work to do 

## With the Globus Toolkit

**With the Globus Toolkit**

- WebBrowser

- ComputeServer

- GlobusMCS/RLS

- DataViewerTool

- CertificateAuthority

- CHEF ChatTeamlet

- MyProxy

- CHEF

- ComputeServer

*Resources implement standard access & management interfaces*

*Collective services aggregate &/or virtualize resources*

*Users work with client applications*

*Application services organize VOs & enable access to other services*

- Database
- service

- Database
- service

- Database
- service

- SimulationTool

- Camera

- Camera

- TelepresenceMonitor

- Globus IndexService

- GlobusGRAM

- GlobusGRAM

- GlobusDAI

- GlobusDAI

- GlobusDAI

| Application Developer | 2 |
| --- | --- |
| Off the Shelf | 9 |
| Globus Toolkit | 4 |
| Grid Community | 4 |

**Notes:**
CHEF is a portal environment from ????

Key points:

	reuse rather than reinvent!  The application developer can take a long vacation instead of working so hard implementing grid functionality

	interoperability: 

## Globus is Grid Infrastructure

- Software for Grid infrastructure
- Service enable new & existing resources
- E.g., GRAM on computer, GridFTP on storage system, custom application service
- Uniform abstractions & mechanisms
- Tools to build applications that exploit Grid infrastructure
- Registries, security, data management, ...
- Open source & open standards
- Each empowers the other
- Enabler of a rich tool & service ecosystem

## The Globus Toolkit:“Standard Plumbing” for the Grid

*Not* turnkey solutions, but *building blocks* & *tools* for application developers & system integrators
- Some components (e.g., file transfer) go farther than others (e.g., remote job submission) toward end-user relevance
- Easier to reuse than to reinvent
- Compatibility with other Grid systems comes for free
- Today the majority of the GT public interfaces are usable by application developers and system integrators
- Relatively few end-user interfaces
- In general, not intended for direct use by end users (scientists, engineers, marketing specialists)

## Globus is a Building Block

- Basic components for grid functionality
- Highest-level services are often application specific, we let applications concentrate there
- Easier to reuse than to reinvent
- Compatibility with other Grid systems comes for free
- We provide basic infrastructure to get you one step closer

## Standards

## Leveraging Existingand Proposed Standards

- WSRF and WS-N (GGF, OASIS)
- WS-Agreement, WSDL 2.0, WSDM
- GridFTP v1.0 (GGF)
- OGSI v1.0 (GGF)
- SSL/TLS v1 (from OpenSSL) (IETF)
- X.509 Proxy Certificates (IETF)
- SAML, XACML

## GT Protocols

- Web service protocols
- WSDL, SOAP
- WS Addressing, WSRF, WSN
- WS Security, SAML, XACML
- WS-Interoperability profile
- Non Web service protocols
- Standards-based, such as GridFTP
- Custom

## WSRF & WS-Notification

- Naming and bindings (basis for virtualization)
- Every resource can be _uniquely referenced_, and has one or more _associated services_ for interacting with it
- Lifecycle (basis for fault resilient state management)
- Resources created by services following _factory_ pattern
- Resources destroyed _immediately_ or _scheduled_
- Information model (basis for monitoring & discovery)
- _Resource properties_ associated with resources
- Operations for _querying_ and _setting_ this info
- Asynchronous _notification_ of changes to properties
- Service Groups (basis for registries & collective svcs)
- Group membership rules & membership management
- Base Fault type

## WS Core Enables Frameworks:E.g., Resource Management

- Web services(WSDL, SOAP, WS-Security, WS-ReliableMessaging, ...)

- WS-Resource Framework & WS-Notification  (*)
- (Resource identity, lifetime, inspection, subscription, ...)

- WS-Agreement(Agreement negotiation)

- WS Distributed Management(Lifecycle, monitoring, ...)

- Applications of the framework(Compute, network, storage provisioning,job reservation & submission, data management,application service QoS, ...)

- * An evolution of Open Grid Services Infrastructure (OGSI)

**Notes:**
Service-Oriented Frameworks

General functionality can be used, extended, specialized for specific purposes
Both specifications and implementations
Captures commonality that can be applied across resources/services
Avoid least common denominator solutions
Allows service consumers to operate at different levels of abstraction
General operations against any service that implements the frameworks
Specific operations against services that extend/specialize the frameworks

## WSRF vs XML/SOAP

- The definition of WSRF means that the Grid and Web services communities can move forward on a common base
- Why Not Just Use XML/SOAP?
- WSRF and WS-N *are *just XML and SOAP
- WSRF and WS-N are just Web services
- Benefits of following the specs:
- These patterns represent best practices that have been learned in many Grid applications
- There is a community behind them
- Why reinvent the wheel?
- Standards facilitate interoperability

## Globus is a Tool

- A Grid development environment
- Develop new OGSA-compliant Web Services
- Develop applications using Java or C/C++ Grid APIs
- Secure applications using basic security mechanisms
- A set of basic Grid functionality
- Services and clients
- Libraries
- Development tools and examples
- The prerequisites for many Grid community tools

## GT Domain Areas

- Core runtime
- Infrastructure for building new services
- Security
- Apply uniform policy across distinct systems
- Execution management
- Provision, deploy, & manage services
- Data management
- Discover, transfer, & access large data
- Monitoring
- Discover & monitor dynamic services

## Globus Toolkit: Open Source Grid Infrastructure

**Data Mgmt**

**Security**

**Common****Runtime**

**Execution Mgmt**

**Info Services**

- GridFTP

- Authentication
- Authorization

- Reliable
- File
- Transfer

- Data Access
- & Integration

- Grid Resource
- Allocation &
-  Management

- Index

- CommunityAuthorization

- Data
- Replication

- Community
- Scheduling
- Framework

- Delegation

- Replica
- Location

- Trigger

- Java Runtime

- C Runtime

- Python Runtime

- WebMDS

- Workspace
- Management

- Grid Telecontrol Protocol

**Globus Toolkit v4**
**www.globus.org**

- CredentialMgmt

**Globus Toolkit:**** Open Source Grid Infrastructure**

## Our Goals for GT4

- Usability, reliability, scalability, ...
- Web service components have quality equal or superior to pre-WS components
- Documentation at acceptable quality level
- Consistency with latest standards (WS-*, WSRF, WS-N, etc.) and Apache platform
- WS-I Basic Profile compliant
- WS-I Basic Security Profile compliant
- New components, platforms, languages
- And links to larger Globus ecosystem

## WSRF vs XML/SOAP

- The definition of WSRF means that the Grid and Web services communities can move forward on a common base
- Why Not Just Use XML/SOAP?
- WSRF and WS-N *are *just XML and SOAP
- WSRF and WS-N are just Web services
- Benefits of following the specs:
- These patterns represent best practices that have been learned in many Grid applications
- There is a community behind them
- Why reinvent the wheel?
- Standards facilitate interoperability

## GT2 Evolution To GT4

**ALL** of GT2 functionality is in GT4
- What happened to the GT2 key protocols?
- Security: Adapted X.509 proxy certs to integrate with emerging WS standards
- GRAM: Took ad hoc protocols away and now use WS-RF standards (ManagedJobFactory and related service definitions)
- GridFTP: Using GridFTP standard from GGF, Reliable File Transfer (RFT) supplies the WSRF-compliant interface
- MDS/LDAP: Replaced LDAP extensions with WSRF standards for notification, subscription, and registration

## GT2 vs GT4

- Pre-WS Globus is in GT4 release
- Both WS and pre-WS components (ala 2.4.3) are shipped
- These do NOT interact, but both can run on the same resource independently
- Basic functionality is the same
- Run a job
- Transfer a file
- Monitoring
- Security
- Code base is completely different

## Why Use GT4?

- Performance and reliability
- Literally millions of tests and queries run against GT4 services
- Scalability
- Many lessons learned from GT2 have been addressed in GT4
- Support
- This is our active code base, much more attention
- Additional functionality
- New features are here
- Additional GRAM interfaces to schedulers, MDS Trigger service, GridFTP protocol interfaces, etc
- Easier to contribute to

## 4.0 is not a typical “.0” release,but the culmination of months of testing 

- 3.0.0

- 3.2.0

- 3.9.5

- 4.0.0

- 3.9.4

- 3.9.3

- 3.9.2

- 3.9.1

- 3.9.0

- 3.3.0

- 3.2.1

- 3.0.1

- 3.0.2

- CVS trunk

- 4.0.1

- Stable release branch

- Development release

- Stable release

**Notes:**
4.0 was actually the 7th release of the WSRF-based code:  Abstract toolkit, difficult to test in isolation

Better testing and documentation
	- 4.0.0 was actually the 6th release of GT4
	- Documentation and code was tested at each release by an engaged community

The recent 4.0.1 release included:
Important bug fixes, including the WS MDS exploit and C message-level security fixes
Improved support for Tomcat container
A version of OGSA-DAI with a complete set of public interfaces
MPICH-G2 support
Updates of many contributions and tech previews
Full Apache license

## Testing Overview

- Nightly builds and tests
- TestGrid at USC/ISI
- Stand up services for several weeks
- Perform stress tests
- TestGrid at LBNL
- Focus on WS Core performance and interoperability tests
- Performance and reliability testing is a major focus
- Component-specific approaches mostly
- Calls for Community Testing near release time - we welcome new testing help!

## Tested Platforms

- Debian
- Fedora Core
- FreeBSD
- HP/UX
- IBM AIX
- Red Hat
- Sun Solaris

- SGI Altix (IA64 running Red Hat)
- SuSE Linux
- Tru64 Unix
- Apple MacOS X (no binaries)
- Windows – Java components only

- List of binaries and known platform-specific install bugs at
- http://www.globus.org/toolkit/docs/4.0/admin/ docbook/ ch03.html

## Documentation Overview

**Documentation Overview**

- Current document significantly more detailed than earlier versions
- _[http://www.globus.org/toolkit/docs/4.0/](http://www.globus.org/toolkit/docs/4.0/)_
- Tutorials available for those of you building a new service
- _[http://www-unix.globus.org/toolkit/tutorials/BAS/](http://www-unix.globus.org/toolkit/tutorials/BAS/)_

- Globus® Toolkit 4: Programming Java Services (The Morgan Kaufmann Series in Networking), by Borja Sotomayor, Lisa Childers (Available through Amazon, £19.99  or $20)

## Grid Packaging Technology (GPT)

- Collection of XML-based packaging tools
- Straight forward definition of complex dependency and compatibility relationships between packages
- Way for developers to define the packaging data and include it as part of their source code distribution
- Automatic generation of binary packages
- Developer tools
- Convert a source distribution into a GPT package
- Patch-n-build capability similar to RPM spec files so you can retain their own build system if needed
- User Tools
- Enable collections of packages to be built and/or installed
- Package manager for those systems that don't have one
- Developed at NCSA

## Virtual Data Toolkit (VDT)

- Grid middleware distribution focused on ease of use (and installation)
- Contents
- Globus Toolkit & Condor, Condor-G
- Virtual Data Tools (Chimera, Pegasus, RLS)
- Utilities (GSI-OpenSSH, MyProxy, MonaLisa, NetLogger, KX.509, etc.)
- GriPhyN Virtual Data System (containing Chimera and Pegasus)
- Uses PACMAN for distribution, install, configuration.
- Used by GriPhyN, iVDGL, Open Science Grid, LCG, UK NGS, and others

## Installation in a nutshell

- Quickstart guide is very useful
- 	http://www.globus.org/toolkit/docs/4.0/ admin/docbook/quickstart.html
- Verify your prereqs!
- Security – check spellings and permissions
- Globus is system software – plan accordingly

## Now that you’vedone your installation... Lets talk about what you get!

## Globus Toolkit: Open Source Grid Infrastructure

**Data Mgmt**

**Security**

**Common****Runtime**

**Execution Mgmt**

**Info Services**

- GridFTP

- Authentication
- Authorization

- Reliable
- File
- Transfer

- Data Access
- & Integration

- Grid Resource
- Allocation &
-  Management

- Index

- CommunityAuthorization

- Data
- Replication

- Community
- Scheduling
- Framework

- Delegation

- Replica
- Location

- Trigger

- Java Runtime

- C Runtime

- Python Runtime

- WebMDS

- Workspace
- Management

- Grid Telecontrol Protocol

**Globus Toolkit v4**
**www.globus.org**

- CredentialMgmt

**Globus Toolkit:**** Open Source Grid Infrastructure**

## GT4 Web Services Runtime

- Supports both GT (GRAM, RFT, Delegation, etc.) & user-developed services
- Redesign to enhance scalability, modularity, performance, usability
- Leverages existing WS standards
- WS-I Basic Profile: WSDL, SOAP, etc.
- WS-Security, WS-Addressing
- Adds support for emerging WS standards
- WS-Resource Framework, WS-Notification
- Java, Python, & C hosting environments
- Java is standard Apache

## What does Core give you?

- Reference implementation of WSRF and WS-N functions
- Naming and bindings (basis for virtualization)
- Every resource can be _uniquely referenced_ and has one or more _associated services_ for interacting
- Lifecycle (basis for resilient state management)
- Resources created by svcs following a _factory_ pattern
- Resource destroyed _immediately_ or _scheduled_
- Information model (basis for monitoring & discovery)
- _Resource properties_ associated with resources
- Operations for _querying_ and _setting_ this info
- Asynchronous _notification_ of changes to properties
- Service groups (basis for registries & collective svcs)
- Group membership rules and membership management
- Base fault type

## Apache Axis Web Services Container

- Good news for Java WS developers: GT4.0 works with standard Axis* and Tomcat*
- GT provides Axis-loadable libraries, handlers
- Includes useful behaviors such as inspection, notification, lifetime mgmt (WSRF)
- Others implement GRAM, etc.
- Major Globus contributions to Apache
- ~50% of WS-Addressing code
- ~15% of WS-Security code
- Many bug fixes
- WSRF code a possible next contribution

** Modulo Axis and Tomcat release cycle issues*

- Axis

- Security
- Addressing

- GT
- bits

- App
- bits

## GT4 Web Services Runtime

**GT4 Web Services Runtime**

## WS Core Enables Frameworks:E.g., Resource Management

- Web services(WSDL, SOAP, WS-Security, WS-ReliableMessaging, ...)

- WS-Resource Framework & WS-Notification
- (Resource identity, lifetime, inspection, subscription, ...)

- WS-Agreement(Agreement negotiation)

- WS Distributed Management(Lifecycle, monitoring, ...)

- Applications of the framework(Compute, network, storage provisioning,job reservation & submission, data management,application service QoS, ...)

**Notes:**
Service-Oriented Frameworks

General functionality can be used, extended, specialized for specific purposes
Both specifications and implementations
Captures commonality that can be applied across resources/services
Avoid least common denominator solutions
Allows service consumers to operate at different levels of abstraction
General operations against any service that implements the frameworks
Specific operations against services that extend/specialize the frameworks

## WSRF/WSNs Compared(Humphrey et al, HPDC 2005)

|  | GT4-Java | GT4-C | pyGridWare | WSRF::Lite | WSRF.NET |
| --- | --- | --- | --- | --- | --- |
| Languages supported | Java | C | Python | Perl | C#/C++/VBasic, etc. |
| WS-Security password profile | Yes | No | In progress | In progress | Yes |
| WS-Security X.509 profile | Yes | In progress | Yes | In progress | Yes |
| WS-SecureConversation | Yes | No | Yes | No | Yes |
| TLS/SSL | Yes | Yes | Yes | Yes | Yes |
| Authorization | Multiple | Multiple | Callout | None |  |
| Persistence of WS-Resources | Yes | Not default | Yes | Yes | Yes |
| Memory Footprint | JVM + 10M | 22 KB | 12 MB | 12 MB | Depends |
| Memory size per WS-Resource | Depends on resource state | 70B | Depends on resource state | 0 (file/DB) or 10B (process) | Depends on resource state |
| Unmodified hosting environment | Yes | No | Yes | Yes (Apache) | Yes |
| Compliance with WS-I Basic Profile | Yes | Yes | Yes | In progress | Yes |
| Compliance with
WS-I Basic Security Profile | Yes | Yes | Yes | No | Yes |
| Logging | Log4J | Yes | Yes | Yes | WSE diagnostics |
| WS-ResourceLifetime | Yes | Yes | Yes | Yes | Yes |
| WS-ResourceProperties | Yes | Yes | Yes | Yes | Yes |
| WS-ServiceGroup | Yes | Yes | Yes | Yes | Yes |
| WS-BaseFaults | Yes | Yes | Yes | Yes | Yes |
| WS-BaseNotification | Yes | Consumer | Yes | No | Yes |
| WS-BrokeredNotification | Partial | No | No | No | Yes |
| WS-Topics | Partial | Partial | Partial | No | Partial |

## GetRP Test

- Distributed client and service on same LAN
- (times in milliseconds)

**No Security**

**X509 Signing**

**HTTPS**

- 10.05

- 2.34

- 25.57

- 17.1

- 8.23

- 181.96

- 14.8

- 140.5

- 81.39

- N/A

- 11.46

- 2.85

- 12.91

- 55.6

- 149.67

**Notes:**
This is Marty Humphrey’s slide

GT4-C is fastest
WSRF.NET and GT4-Java are comparable for no security and HTTPS
WSRF.NET is fastest (except GT4-C) for X509
Message layer security is an order of magnitude slower than transport layer security

Largely, we see these patterns repeat across all tests
Mention that pyG does not implement https connection caching and that’s why their HTTPS perf is slow (but they are implementing it)

## GT4 WS Core Performance

|  | GT4 Java | GT4 C | GT4 Python | WSRF.NET |
| --- | --- | --- | --- | --- |
| GetRP | 181.96 | 14.77 | 140.50 | 81.39 |
| SetRP | 182.04 | 14.99 | 142.21 | 82.48 |
| CreateR | 188.46 | 14.98 | 132.26 | 96.22 |
| DestroyR | 182.03 | 15.76 | 136.12 | 86.89 |
| Notify | 219.51 | N/A | 244.93 | 101.57 |

|  | GT4 Java | GT4 C | GT4 Python | WSRF.NET |
| --- | --- | --- | --- | --- |
| getRP | 11.46 | 2.85 | 149.67 | 12.91 |
| setRP | 11.47 | 2.86 | 150.79 | 12.3 |
| createR | 18.00 | 2.82 | 132.60 | 20.84 |
| destroyR | 14.92 | 2.71 | 149.21 | 16.05 |
| Notify | 29.26 | 9.67 | 169.07 | 45.0 |

- (1) Message-level security (times in milliseconds)

- (2) Transport-level security (times in milliseconds)

- “WSRF/WSNs Compared,” HPDC 2005.

## Globus Toolkit: Open Source Grid Infrastructure

**Data Mgmt**

**Security**

**Common****Runtime**

**Execution Mgmt**

**Info Services**

- GridFTP

- Authentication
- Authorization

- Reliable
- File
- Transfer

- Data Access
- & Integration

- Grid Resource
- Allocation &
-  Management

- Index

- CommunityAuthorization

- Data
- Replication

- Community
- Scheduling
- Framework

- Delegation

- Replica
- Location

- Trigger

- Java Runtime

- C Runtime

- Python Runtime

- WebMDS

- Workspace
- Management

- Grid Telecontrol Protocol

**Globus Toolkit v4**
**www.globus.org**

- CredentialMgmt

**Globus Toolkit:**** Open Source Grid Infrastructure**

## Globus Security

- Control access to shared services
- Address autonomous management, e.g., different policy in different work-groups
- Support multi-user collaborations
- Federate through mutually trusted services
- Local policy authorities rule
- Allow users and application communities to set up dynamic trust domains
- Personal/VO collection of resources working together based on trust of user/VO

## Virtual Organization (VO) Concept

**Virtual Organization (VO) Concept**

- VO for each application or workload
- Carve out and configure resources for a particular use and set of users

## GT4 Security

- Users

## GT4 Security

- Public-key-based authentication
- Transport- and message-level authentication
- Extensible authorization framework based on Web services standards
- SAML-based authorization callout
- Integrated policy decision engine
- XACML policy language, per-operation policies, pluggable
- Credential management service
- MyProxy (One time password support)
- Community Authorization Service
- Standalone delegation service
- Ability to map between Grid and local identity

## Security Tools

- Basic Grid Security Mechanisms
- Certificate Generation Tools
- Certificate Management Tools
- Getting users “registered” to use a Grid
- Getting Grid credentials to wherever they’re needed in the system
- Authorization/Access Control Tools
- Storing and providing access to system-wide authorization information

## Other Security Services Include ...

- MyProxy
- Simplified credential management
- Web portal integration
- Single-sign-on support
- KCA & kx.509
- Bridging into/out-of Kerberos domains
- SimpleCA
- Online credential generation
- PERMIS
- Authorization service callout

## A Cautionary Note

- Grid security mechanisms are tedious to set up
- If exposed to users, hand-holding is usually required
- These mechanisms can be *hidden entirely* from end users, but still used behind the scenes
- These mechanisms exist for good reasons.
- Many useful things can be done without Grid  security
- It is unlikely that an ambitious project could go into production operation without security like this
- Most successful projects end up using Grid security, but using it in ways that end users don’t see much

## GT4’s Use of Security Standards 

-  Supported,       Supported,** **        **Fastest, **
-   but slow         but insecure      **so default**

## GT-XACML Integration

- eXtensible Access Control Markup Language
- OASIS standard, open source implementations
- XACML: sophisticated policy language
- Globus Toolkit ships with XACML runtime
- Included in every client and server built on GT
- Turned-on through configuration
- ... that can be called transparently from runtime and/or explicitly from application ...
- ... and we use the XACML-”model” for our Authz Processing Framework

## GT Authorization Framework

## Globus Certificate Service

- An online service that issues low-quality GSI certificates
- Intended for people who want to experiment with Grid components that require certificates but do not have any other means of acquiring certificates.
- These certificates are *not* to be used on production systems.
- Not a true Certificate Authority (CA)
- No revoking or reissuing certificates
- No verification of identities
- The service itself is not especially secure.

## Simple CA

- A convenient method of setting up a certificate authority (CA).
- The Certificate Authority can then be used to issue certificates for users and services that work with GSI and WS-Security.
- Simple CA is intended for operators of small Grid testing environments and users who are not part of a larger Grid.
- Most production Grids will not accept certificates that are not signed by a well-known CA, so the certificates generated by Simple CA will usually not be sufficient to gain access to production services.

## MyProxy

- MyProxy is a remote service that stores user credentials.
- Users can request proxies for local use on any system on the network.
- Web Portals can request user proxies for use with back-end Grid services.
- Grid administrators can pre-load credentials in the server for users to retrieve when needed.
- Greatly simplifies certificate management!

## CAS: Community Authorization Service

- CAS allows resource providers to specify course-grained access control policies in terms of communities as a whole.
- Fine-grained access control is delegated to the community.
- Resource providers maintain ultimate authority over their resources (including per-user control and auditing) but are spared most day-to-day policy administration tasks.

## VOMS

- A community-level group membership system
- Database of user roles
- Administrative tools
- Client interface
- voms-proxy-init
- Uses client interface to produce an attribute certificate (instead of proxy) that includes roles & capabilities signed by VOMS server
- Works with non-VOMS services, but gives more info to VOMS-aware services
- Allows VOs to centrally manage user roles

## Globus Toolkit: Open Source Grid Infrastructure

**Data Mgmt**

**Security**

**Common****Runtime**

**Execution Mgmt**

**Info Services**

- GridFTP

- Authentication
- Authorization

- Reliable
- File
- Transfer

- Data Access
- & Integration

- Grid Resource
- Allocation &
-  Management

- Index

- CommunityAuthorization

- Data
- Replication

- Community
- Scheduling
- Framework

- Delegation

- Replica
- Location

- Trigger

- Java Runtime

- C Runtime

- Python Runtime

- WebMDS

- Workspace
- Management

- Grid Telecontrol Protocol

**Globus Toolkit v4**
**www.globus.org**

- CredentialMgmt

**Globus Toolkit:**** Open Source Grid Infrastructure**

## Execution Management (GRAM)

- Common WS interface to schedulers
- Unix, Condor, LSF, PBS, SGE, ...
- More generally: interface for process execution management
- Lay down execution environment
- Stage data
- Monitor & manage lifecycle
- Kill it, clean up
- A basis for application-driven provisioning

## GRAM - Basic Job Submission and Control Service

- A uniform service interface for remote job submission and control
- Includes file staging and I/O management
- Includes reliability features
- Supports basic Grid security mechanisms
- Available in Pre-WS and WS
- GRAM is *not* a scheduler.
- No scheduling
- No metascheduling/brokering
- Often used as a front-end to schedulers, and often used to simplify metaschedulers/brokers

## GT4 WS GRAM

- 2nd-generation WS implementation optimized for performance, flexibility, stability, scalability
- Streamlined critical path
- Use only what you need
- Flexible credential management
- Credential cache & delegation service
- GridFTP & RFT used for data operations
- Data staging & streaming output
- Eliminates redundant GASS code

## GRAM

- Intended for jobs where arbitrary programs, stateful monitoring, credential management, and file staging are important
- If the application is lightweight, with modest input/output, may be a better candidate for hosting directly as a WSRF service

## GT4 WS GRAM Architecture

- GRAM
- services

- GT4 Java Container

- GRAM
- services

- Delegation

- RFT File
- Transfer

*Transfer*
*request*

- GridFTP

- Remote
- storage
- element(s)

- Local
- scheduler

- User
- job

- Compute element

- GridFTP

- sudo

- GRAM
- adapter

*FTP*
*control*

*Local job control*

*Delegate*

*FTP data*

**Client**

*Job functions*

*Delegate*

- Service host(s) and compute element(s)

**GT4 WS GRAM Architecture**

- SEG

*Job events*

## GT4 WS GRAM Architecture

- GRAM
- services

- GT4 Java Container

- GRAM
- services

- Delegation

- RFT File
- Transfer

*Transfer*
*request*

- GridFTP

- Remote
- storage
- element(s)

- Local
- scheduler

- User
- job

- Compute element

- GridFTP

- sudo

- GRAM
- adapter

*FTP*
*control*

*Local job control*

*Delegate*

*FTP data*

**Client**

*Job functions*

*Delegate*

- Service host(s) and compute element(s)

**GT4 WS GRAM Architecture**

- SEG

*Job events*

- Delegated credential can be:
- Made available to the application

## GT4 WS GRAM Architecture

- GRAM
- services

- GT4 Java Container

- GRAM
- services

- Delegation

- RFT File
- Transfer

*Transfer*
*request*

- GridFTP

- Remote
- storage
- element(s)

- Local
- scheduler

- User
- job

- Compute element

- GridFTP

- sudo

- GRAM
- adapter

*FTP*
*control*

*Local job control*

*Delegate*

*FTP data*

**Client**

*Job functions*

*Delegate*

- Service host(s) and compute element(s)

**GT4 WS GRAM Architecture**

- SEG

*Job events*

- Delegated credential can be:
- Used to authenticate with RFT

## GT4 WS GRAM Architecture

- GRAM
- services

- GT4 Java Container

- GRAM
- services

- Delegation

- RFT File
- Transfer

*Transfer*
*request*

- GridFTP

- Remote
- storage
- element(s)

- Local
- scheduler

- User
- job

- Compute element

- GridFTP

- sudo

- GRAM
- adapter

*FTP*
*control*

*Local job control*

*Delegate*

*FTP data*

**Client**

*Job functions*

*Delegate*

- Service host(s) and compute element(s)

**GT4 WS GRAM Architecture**

- SEG

*Job events*

- Delegated credential can be:
- Used to authenticate with GridFTP

## Submitting a Sample Job

- Specify a remote host with –F
- globusrun-ws –submit –F host2 –c /bin/true
- The return code will be the job’s exit code if supported by the scheduler

## Data Staging and Streaming

- Simplest stage-in/stage-out example is stdout/stderr
- globusrun-ws –S –s –c /bin/date
- -S is short for “-submit”
- -s is short for –streaming
- The output will be sent back to the terminal, control will not return until the job is done

## Resource Specification Language

- For more complicated jobs, we’ll use RSL to specify the job
- <job>
- <executable>/bin/echo</executable>
- <argument>this is an example_string </argument>
- <argument>Globus was here</argument>
- <stdout>${GLOBUS_USER_HOME}/stdout</stdout>
-  <stderr>${GLOBUS_USER_HOME}/stderr</stderr>
- </job>

## Resource Specification Language

- <job>
- <executable>/bin/echo</executable> <directory>/tmp</directory> <argument>12</argument>
- <environment><name>PI</name> <value>3.141</value></environment>
- <stdin>/dev/null</stdin>
- <stdout>stdout</stdout>
- <stderr>stderr</stderr>
- </job>

## Resource Specification Language

- <job>
- <executable>/bin/echo</executable> <directory>/tmp</directory> <argument>12</argument>
- <environment><name>PI</name> <value>3.141</value></environment>
- <stdin>/dev/null</stdin>
- <stdout>stdout</stdout>
- <stderr>stderr</stderr>
- </job>

## Submitting Using XML

- Create the file containing the RSL
- You may validate the RSL ahead of time
- globusrun-ws –validate –f rslfile.xml
- If the file validates, submit using          -submit

## At Most Once Submission

- You may specify a UUID with your job submission
- If you’re not sure the submission worked, you may submit the job again with the same UUID
- If the job has already been submitted, the new submission will have no effect
- If you do not specify a UUID, one will be generated for you

## Staging Data

- GRAM’s RSL allows many fileStageIn/fileStageOut directives
- The transfers will be executed by RFT
- May specify additional RFT options using the RFTOptions tag
- There is no GASS cache staging option anymore

## Staging Data – Stage In

- GRAM’s RSL allows many fileStageIn/fileStageOut directives
- <fileStageIn> <transfer>
- <sourceUrl>
- gsiftp://job.submitting.host:2811/bin/echo</sourceUrl>
- <destinationUrl>file:///${GLOBUS_USER_HOME}/my_echo</destinationUrl>
- </transfer> </fileStageIn>

## Staging Data – Stage Out

- <fileStageOut>
- <transfer>
- <sourceUrl>file://${GLOBUS_USER_HOME}/stdout
- </sourceUrl>
- <destinationUrl>gsiftp://job.submitting.host:2811/tmp/stdout</destinationUrl>
- </transfer>
- </fileStageOut>

## Staging Data - Cleanup

- <fileCleanUp>
- <deletion>
- <file>file://${GLOBUS_USER_HOME}/my_echo</file>
- </deletion>
- </fileCleanUp>

## Staging Data - Credentials

- The GridFTP servers you’re using may require different credentials than the GRAM service you’re submitting to
- The RSL allows you to specify separate credentials for the executable and staging components of the job

## RSL Substitutions

- GRAM will perform some variable substitutions for you
- GLOBUS_USER_HOME
- GLOBUS_USER_NAME
- GLOBUS_SCRATCH_DIR
- GLOBUS_LOCATION
- SCRATCH_DIR will be a compute-node local high-speed storage if defined, or GLOBUS_USER_HOME if not

## Batch Submission

- Your client does not have to stay attached to the execution of the job
- -batch will disconnect from the job and output an EPR
- You may redirect the EPR to a file with –o
- Use the EPR file with –monitor or -status
- You may also kill the job using -kill

## Specifying Scheduler Options

- RSL lets you specify various scheduler options
- what queue to submit to
- which project to select for accounting
- max CPU and wallclock time to spend
- min/max memory required
- All defined online under the schema document for GRAM

## Choosing User Accounts

- You may be authorized to use more than one account at the remote site
- By default, the first listed in the grid-mapfile will be used
- You may request a specific user account using the <localUserId> element

## Multijobs

- You may specify more than one <job> element in a <multijob>
- At that point, you want to specify the <factoryEndpoint> in the RSL rather than the commandline
- Will be used by MPICH-G to support MPI jobs

## WS GRAM Performance

- Time to submit a basic GRAM job
- Pre-WS GRAM: < 1 second
- WS GRAM: 2 seconds
- Concurrent jobs
- Pre-WS GRAM: 300 jobs
- WS GRAM: 32,000 jobs

## CondorG

**CondorG**

- The Condor project has produced a “helper front-end” to GRAM
- Managing sets of subtasks
- Reliable front-end to GRAM to manage computational resources
- Note: this is not Condor which promotes high-throughput computing, and use of idle resources

## Chimera “Virtual Data”

- Captures both logical and physical steps in a data analysis process.
- Transformations (logical)
- Derivations (physical)
- Builds a catalog.
- Results can be used to “replay” analysis.
- Generation of DAG (via Pegasus)
- Execution on Grid
- Catalog allows introspection of analysis process.

- Galaxy cluster
- size distribution

- Sloan Survey Data

## Pegasus Workflow Transformation

- Converts Abstract Workflow (AW) into Concrete Workflow (CW).
- Uses Metadata to convert user request to logical data sources
- Obtains AW from Chimera
- Uses replication data to locate physical files
- Delivers CW to DAGman
- Executes using Condor
- Publishes new replication and derivation data in RLS and Chimera (optional)

- ChimeraVirtual DataCatalog

- ReplicaLocationService

- MetadataCatalog

- StorageSystem

- StorageSystem

- StorageSystem

- ComputeServer

- ComputeServer

- ComputeServer

- ComputeServer

- DAGman

- Condor

*  *

*t*

## Globus Toolkit: Open Source Grid Infrastructure

**Data Mgmt**

**Security**

**Common****Runtime**

**Execution Mgmt**

**Info Services**

- GridFTP

- Authentication
- Authorization

- Reliable
- File
- Transfer

- Data Access
- & Integration

- Grid Resource
- Allocation &
-  Management

- Index

- CommunityAuthorization

- Data
- Replication

- Community
- Scheduling
- Framework

- Delegation

- Replica
- Location

- Trigger

- Java Runtime

- C Runtime

- Python Runtime

- WebMDS

- Workspace
- Management

- Grid Telecontrol Protocol

**Globus Toolkit v4**
**www.globus.org**

- CredentialMgmt

**Globus Toolkit:**** Open Source Grid Infrastructure**

## GT4 Data Management

**Stage/move** large data to/from nodes
- GridFTP, Reliable File Transfer (RFT)
- Alone, and integrated with GRAM
**Locate** data of interest
- Replica Location Service (RLS)
**Replicate** data for performance/reliability
- Distributed Replication Service (DRS)
- Provide **access** to diverse data sources
- File systems, parallel file systems, hierarchical storage: GridFTP
- Databases: OGSA DAI

## GridFTP

- A high-performance, secure, reliable data transfer protocol optimized  for high-bandwidth wide-area networks
- FTP with well-defined extensions
- Uses basic Grid security (control and data channels)
- Multiple data channels for parallel transfers
- Partial file transfers
- Third-party (direct server-to-server) transfers
- Reusable data channels
- Command pipelining
- GGF recommendation GFD.20

## Striped GridFTP Service

- A distributed GridFTP service that runs on a storage cluster
- Every node of the cluster is used to transfer data into/out of the cluster
- Head node coordinates transfers
- Multiple NICs/internal busses lead to very high performance
- Maximizes use of Gbit+ WANs

## GridFTP in GT4

- 100% Globus code
- No licensing issues
- Stable, extensible
- IPv6 Support
- XIO for different transports
- Striping  multi-Gb/sec wide area transport
- Pluggable
- Front-end: e.g., future WS control channel
- Back-end: e.g., HPSS, cluster file systems
- Transfer: e.g., UDP, NetBLT transport

- Disk-to-disk onTeraGrid

**Notes:**
Future: integration of provisioning
E.g., storage reservation

## Copying Files (in a nutshell)

- globus-url-copy [options] srcURL dstURL
- guc gsiftp://localhost/foo file:///bar
- Client/server, using FTP stream mode
- guc –vb –dbg –tcp-bs 1048576 –p 8 gsiftp://localhost/foo gsiftp://localhost/bar
- 3rd party transfer, MODE E
- guc https://host.domain.edu/foo ftp://host.domain.gov/bar
- from secure http to ftp server

## The Options: Improving Performance

- -p (parallelism or number of streams)
- rule of thumb 4-8, start with 4
- -tcp-bs (TCP buffer size)
- use either ping or traceroute to determine the RTT between hosts
- buffer size = BW (Mbs) * RTT (ms) *1000/8/<(parallelism value – 1)>
- If that is still too complicated use 2MB
- -vb if you want performance feedback
- -dbg if you have trouble

## Tuning GridFTP

- Many ways you can tune the performance
- Two sources of data are
- _[http://www.globus.org/toolkit/docs/4.0/data/gridftp/rn01re01.html](http://www.globus.org/toolkit/docs/4.0/data/gridftp/rn01re01.html)_
- _[http://www.nsf-middleware.org/OnTheGrid](http://www.nsf-middleware.org/OnTheGrid/2004-09-MaxGridFTP.pdf)__[/ 2004-09-MaxGridFTP.pdf](http://www.nsf-middleware.org/OnTheGrid/2004-09-MaxGridFTP.pdf)_

## Exercise: Simple File Movement

- grid-proxy-init
- echo test > /tmp/test
- look at servers started for each:
- guc gsiftp://hostname/tmp/test file:///tmp/test2
- get (from server to client)
- guc file:///tmp/test2 gsiftp://hostname/tmp/test3
- put (from client to server)
- guc gsiftp://hostname1/tmp/test3 gsiftp://hostname2/tmp/test4
- Third party transfer (between two servers)
- guc –dcpriv gsiftp://localhost/dev/zero gsiftp://localhost/dev/null
- transfer with encryption on data channel

## Troubleshooting

- Can I get connected?
- telnet to the port: telnet hostname port
- 2811 is the default port
- You should get something like this:
- 220 GridFTP Server gridftp.mcs.anl.gov 0.17 (gcc32dbg, 1108765962-1) ready. ** Development Release **
- If not, you have firewall problems, or xinetd config problems.  You are never even starting the server.

## Troubleshooting

- no proxy
- grid-proxy-destroy
- guc gsiftp://localhost/dev/zero file:///dev/null
- add –dbg
- grid-proxy-init
- guc gsiftp://localhost/dev/zero file:///dev/null
- add –dbg

## Troubleshooting

- Bad source file
- grid-proxy-init
- guc gsiftp://localhost:2812/tmp/junk file:///tmp/empty
- junk does not exist
- Note that an empty file named empty is created
- We need to fix this in globus-url-copy, but for now it is there

## RFT - File Transfer Queuing

- A WSRF service for queuing file transfer requests
- Server-to-server transfers
- Checkpointing for restarts
- Database back-end for failovers
- Allows clients to requests transfers and then “disappear”
- No need to manage the transfer
- Status monitoring available if desired

## Reliable File Transfer:Third Party Transfer

**Reliable File Transfer:****Third Party Transfer**

- RFT Service

- RFT Client

- SOAP Messages

- Notifications(Optional)

- GridFTP Server

- GridFTP Server

- Fire-and-forget transfer
- Web services interface
- Many files & directories
- Integrated failure recovery
- Has transferred 900K files

## Replica Location Service

- Identify location of files via logical to physical name map
- Distributed indexing of names, fault tolerant update protocols
- GT4 version scalable & stable
- Managing ~40 million files across ~10 sites

- Index

- Index

| Local DB | Update send (secs) | Bloom filter  (secs)  | Bloom filter (bits) |
| --- | --- | --- | --- |
| 10K | <1 | 2 | 1 M |
| 1 M | 2 | 24 | 10 M |
| 5 M | 7 | 175 | 50 M |

## Reliable Wide Area Data Replication

**Birmingham****•**

**Reliable Wide Area Data Replication**

- Replicating >1 Terabyte/day to 8 sites
- >30 million replicas so far
- MTBF = 1 month

- LIGO Gravitational Wave Observatory

- www.globus.org/solutions

## OGSA-DAI

- Grid Interfaces to Databases
- Data access
- Relational & XML Databases, semi-structured files
- Data integration
- Multiple data delivery mechanisms, data translation
- Extensible & Efficient framework
- Request documents contain multiple tasks
- A task = execution of an activity
- Group work to enable efficient operation
- Extensible set of activities
- > 30 predefined, framework for writing your own
- Moves computation to data
- Pipelined and streaming evaluation
- Concurrent task evaluation

## OGSA-DAI

- Provide service-based access to structured data resources as part of Globus
- Specify a selection of interfaces tailored to various styles of data access—starting with relational and XML

## The OGSA-DAI Framework

**OGSA-DAI service**

**Engine**

**SQLQuery**

**JDBC**

***Data ***
***Resources***

***Activities***

**DB2**

**The OGSA-DAI Framework**

**GZip**

**GridFTP**

**XPath**

**XMLDB**

**XIndice**

**readFile**

**File**

**SWISS**
**PROT**

**XSLT**

**SQL**
**Server**

**Data-**
**bases**

**Application**

**Client Toolkit**

**Notes:**
Data resources used to be called connection managers

## Extensibility Example

**OGSA-DAI service**

**Engine**

**SQLQuery**

**JDBC**

***Multiple***
***SQL GDS***

***SQLQuery***

**Extensibility Example**

**Notes:**
Data resources used to be called connection managers

## OGSA-DAI: A Framework for Building Applications

- Supports data access, insert and update
- Relational: MySQL, Oracle, DB2, SQL Server, Postgres
- XML: Xindice, eXist
- Files – CSV, BinX, EMBL, OMIM, SWISSPROT,...
- Supports data delivery
- SOAP over HTTP
- FTP; GridFTP
- E-mail
- Inter-service
- Supports data transformation
- XSLT
- ZIP; GZIP
- Supports security
- X.509 certificate based security

## OGSA-DAI: Other Features

- A framework for building data clients
- Client toolkit library for application developers
- A framework for developing functionality
- Extend existing activities, or implement your own
- Mix and match activities to provide functionality you need
- Highly extensible
- Customise our out-of-the-box product
- Provide your own services, client-side support, and data-related functionality

## Data Replication Service (tech preview)

- Pull “missing” files to local site

- Site B

- List of required Files

*Reliable File Transfer**Service*

*Data Replication Service*

*GridFTP*

*Local*
*Replica**Catalog*

*Replica*
*Location**Index*

- Site A

*Data Replication Service*

*Reliable File Transfer Service*

*Local*
*Replica*
*Catalog*

*Replica**Location**Index*

*GridFTP*

## MCS - Metadata Catalog Service

- A stand-alone metadata catalog service
- WSRF service interface
- Stores system-defined and user-defined attributes for logical files/objects
- Supports manipulation and query
- Integrated with OGSA-DAI
- OGSA-DAI provides metadata storage
- When run with OGSA-DAI, basic Grid authentication mechanisms are available

## Globus Toolkit: Open Source Grid Infrastructure

**Data Mgmt**

**Security**

**Common****Runtime**

**Execution Mgmt**

**Info Services**

- GridFTP

- Authentication
- Authorization

- Reliable
- File
- Transfer

- Data Access
- & Integration

- Grid Resource
- Allocation &
-  Management

- Index

- CommunityAuthorization

- Data
- Replication

- Community
- Scheduling
- Framework

- Delegation

- Replica
- Location

- Trigger

- Java Runtime

- C Runtime

- Python Runtime

- WebMDS

- Workspace
- Management

- Grid Telecontrol Protocol

**Globus Toolkit v4**
**www.globus.org**

- CredentialMgmt

**Globus Toolkit:**** Open Source Grid Infrastructure**

## Monitoring and Discovery System(MDS4)

**Monitoring and Discovery System****(MDS4)**

- Grid-level monitoring system used most often for resource selection
- Aid user/agent to identify host(s) on which to run an application
- Uses standard interfaces to provide publishing of data, discovery, and data access, including subscription/notification
- WS-ResourceProperties, WS-BaseNotification, WS-ServiceGroup
- Functions as an hourglass to provide a common interface to lower-level monitoring tools

## GLUE Schema Attributes

- (cluster info,
- queue info, FS info)

- Information Users :
- Schedulers, Portals, etc.

- WS standard interfaces for subscription, registration, notification

## Monitoring and Discovery

- “Every service should be monitorable and discoverable using common mechanisms”
- WSRF/WSN provides those mechanisms
- A common aggregator framework for collecting information from services, thus:
- Index Service: Registry supporting Xpath queries, with caching
- Trigger Service: perform action on condition
- Deep integration with Globus containers & services: every GT4 service is discoverable
- GRAM, RFT, GridFTP, CAS, ...

## GT4 Monitoring & Discovery

**GT4 ****Monitoring & Discovery**

- GRAM

- User

- Index

- GT4 Cont.

- RFT

- Index

- GT4 Container

- Index

- GridFTP

- adapter

- Registration &
- WSRF/WSN Access

- Custom protocols
- for non-WSRF entities

- Clients
- (e.g., WebMDS)

- Automated
- registration
- in container

- WS-ServiceGroup

**Notes:**
WebMDS: configurable via XSLT style sheets

## MDS4 Extensibility

- Aggregator framework provides
- Registration management
- Collection of information from Grid Resources
- Plug in interface for data access, collection ,query, ...
- WebMDS framework provides for customized display
- XSLT transformations

## With a standard deployment,a project can...

**With a standard deployment,****a project can...**

- _Discover needed data_ from services in order to make job submission or replica selection decisions by querying the VO-wide Index
- _Evaluate the status_ of Grid services by looking at the VO-wide WebMDS setup
- _Be notified_ when disks are full or other error conditions happen by being on the list of administrators
- Individual projects can examine the state of the resources and services of interest to them

## Slide 127

## Slide 128

## Scalability Experiments

- MDS index
- Dual 2.4GHz Xeon processors, 3.5 GB RAM
- Sizes: 1, 10, 25, 50, 100
- Clients
- 20 nodes also dual 2.6 GHz Xeon, 3.5 GB RAM
- 1, 2, 3, 4, 5, 6, 7, 8, 16, 32, 64, 128, 256, 384, 512, 640, 768, 800
- Nodes connected via 1Gb/s network
- Each data point is average of 8 minutes
- Ran for 10 mins but first 2 spent getting clients up and running
- Error bars are SD over 8 mins
- Experiments by Ioan Raicu, U of Chicago, using DiPerf

## Slide 130

## Versioning and Support

- Versioning
- Evens are production (4.0.x, 4.2.x),
- Odds are development (4.1.x)
- We support this version and the one previous
- Currently we’re at 4.0.1 so we support3.2 and 4.0

## Several Possible Next Versions

- 4.0.2 – stable release
- 100% same interfaces, bug fixes only
- Expected in 4-8 weeks?
- 4.1 – development release
- New functionality
- Expected in 6-10 weeks?
- 4.2 - stable release
- When 4.1 has “enough” new functionality, and is stable
- 5.0 – substantial code base change
- With any luck, not for years :)

## Globus Toolkit: Open Source Grid Infrastructure

**Data Mgmt**

**Security**

**Common****Runtime**

**Execution Mgmt**

**Info Services**

- GridFTP

- Authentication
- Authorization

- Reliable
- File
- Transfer

- Data Access
- & Integration

- Grid Resource
- Allocation &
-  Management

- Index

- CommunityAuthorization

- Data
- Replication

- Community
- Scheduling
- Framework

- Delegation

- Replica
- Location

- Trigger

- Java Runtime

- C Runtime

- Python Runtime

- WebMDS

- Workspace
- Management

- Grid Telecontrol Protocol

**Globus Toolkit v4**
**www.globus.org**

- CredentialMgmt

**Globus Toolkit:**** Open Source Grid Infrastructure**

## The Globus Ecosystem

**Globus components** address core issues relating to resource access, monitoring, discovery, security, data movement, etc.
- GT4 being the latest version
- A larger **Globus ecosystem** of open source and proprietary components provide complementary components
- A growing list of components
- These components can be combined to produce **solutions** to Grid problems
- We’re building a list of such solutions

## Many Tools Build on, or Can Contribute to, GT4-Based Grids 

- Condor-G, DAGman
- MPICH-G2
- GRMS
- Nimrod-G
- Ninf-G
- Open Grid Computing Env.
- Commodity Grid Toolkit
- GriPhyN Virtual Data System
- Virtual Data Toolkit
- GridXpert Synergy
- Platform Globus Toolkit

- VOMS
- PERMIS
- GT4IDE
- Sun Grid Engine
- PBS scheduler
- LSF scheduler
- GridBus
- TeraGrid CTSS
- NEES
- IBM Grid Toolbox
- ...

## GlobalCommunity

**Global****Community**

## Example Solutions

- Portal-based User Reg. System (PURSE)
- VO Management Registration Service
- Service Monitoring Service
- TeraGrid TGCP Tool
- Lightweight Data Replicator
- GriPhyN Virtual Data System

## DocumentingThe Grid Ecosystem

**The Grid Ecosystem: Software Components for Grid Systems**
**And Applications**

- www.grids-center.org

## Example Solutions

- Portal-based User Reg. System (PURSE)
- VO Management Registration Service
- Service Monitoring Service
- TeraGrid TGCP Tool
- Lightweight Data Replicator
- GriPhyN Virtual Data System

## Condor-G

- The Condor Project @ U Wisconsin Madison develops software for high-throughput computing on collections of distributed compute resources
- Condor-G is an interface to GRAM created by the Condor team that allows users to submit jobs to GRAM servers

## GridShib

- Allows the use of Shibboleth-transported attributes for authorization in GT4 deployments
- And, more generally, SAML support
- 2 year project started December 1, 2004
- Participants
- Von Welch, UIUC/NCSA (PI)
- Kate Keahey, UChicago/Argonne (PI)
- Frank Siebenlist, Argonne
- Tom Barton, UChicago
- Beta software released September 16, 2005

**Notes:**
GridShib Objectives:
Priority 1: Pull mode operation
Globus services contact Shibboleth to obtain attributes about identified user
Support both GT4.x Web Services and pre-WS code
Priority 2: Push mode operation
User obtains Shib attributes and push to service
Allows role selection
Priority 3: Online CAs
Pseudonymous operation
Integration with local authentication services

(This slide from a presentation by Tom Barton)

## Handle System

- The Handle System from CNRI (http://www.handle.net) is a general-purpose global name service enabling secure name resolution over the internet
- The Handle System-GT Integration Project leverages the Handle System for identifier and resolution services through tight integration with GT4’s Web services protocols

## MPICH-G2

- MPICH-G2, developed at Northern Illinois University and Argonne National Lab, is a grid-enabled implementation of the MPI v1.1 standard
- MPICH-G2 is implemented using the pre-WS GRAM component in GT4; integration with GT4 WS GRAM is expected in the near future

## Nimrod/G

- Nimrod is a specialized parametric modeling system from Monash University
- Nimrod/G uses a simple declarative parametric modeling language to express parameter sweep experiments.  Based on GT4 WS services, Nimrod/G enables the formulation, execution and monitoring of multiple individual parametric experiments

## Ninf-G4

- Ninf-G4, from AIST, is a reference implementation of the GGF standard GridRPC API
- Ninf-G4 is provides higher-level programming APIs for the development and execution of parallel applications on the Grid

## PERMIS

- PERMIS is an EU-funded Privilege Management service that implements Role-Based Access Control
- Thanks to the work of the UK Grid Engineering Task Force, services running in a Java WS Core container can use PERMIS via GT4’s SAML authorization callouts

## SRB

- SRB is a package from SDSC providing a uniform interface for connecting to network-based heterogeneous data resources
- GT4’s GridFTP includes an interface to SRB data sources, and vice versa

## Sun Grid Engine

- Sun Grid Engine is an open source distributed resource management system from Sun Microsystems
- In a collaboration between the London e-Science Centre, Gridwise and MCNC, the Sun Grid Engine has been integrated with GT4

## Tells Us About YourGrid Tools & Solutions 

- We list links to related projects on the “Related Software” of the Globus Toolkit web **www.globus.org/toolkit/tools/**
- “Solutions” are documented on the Globus web **www.globus.org/solutions/**
- If we’ve got details wrong or you have a GT4-related tool to list on our website, please send mail to info@globus.org

## The Globus Commitment to Open Source

- Globus was first established as an open source project in 1996
- The Globus Toolkit is open source to:
- allow for inspection
- for consideration in standardization processes
- encourage adoption
- in pursuit of ubiquity and interoperability
- encourage contributions
- harness the expertise of the community
- The Globus Toolkit is distributed under the (BSD-style) Apache License version 2

## The Future:Structure

- NSF Community Driven Improvement of Globus Software (CDIGS) project
- 5 years of funding for GT enhancement
- Regular Globus **roadmaps** outlining plans
- GlobDev			http://dev.globus.org
- Apache-like community development site
- Community governance of components
- “Globus Toolkit” & other related software
- Open for business early 2006
- “Globus Alliance” = “GlobDev committers”

## GlobDev 

- The current set of Globus components will be organized into several “Globus Projects”
- Projects release products
- Each project will have its own group of “Committers”
- Committers are responsible for governance on matters relating to their products
- The “Globus Management Committee” will
- provide overall guidance and conflict resolution
- approve the creation of new Globus Projects

## The Future:Content

- We now have a solid and extremely powerful Web services base
- Next, we will build an expanded open source Grid infrastructure
- Virtualization
- New services for provisioning, data management, security, VO management
- End-user tools for application development
- Etc., etc.
- And of course responding to user requests for other short-term needs

## Upcoming Tutorials

- International Summer School for Grid Computing
- Summer 2006
- Last year was at Vico Equense (Naples)
- Globus World
- Joint with GGF
- Sept 11-15, DC
- Possibly a EU Globus week?

## General Globus Help and Support

- Globus-discuss list
- discuss@globus.org
- _[http://globus.org/about/contacts.html](http://globus.org/about/contacts.html)_
- GT4 specific list
- gt4-friends@globus.org
- Bugzilla
- Bugzilla.globus.org

## Opportunities for Collaboration

- Use of Globus software
- Feedback & involvement in design
- Development of new Globus components
- E.g., new information providers to enable use of GT to manage an entire Grid
- Examples and documentation
- Globalization and localization of software
- New applications and tools
- E.g., Grid operations, emergency response, ecogrid, bioinformatics, ...

## What’s This Abouta Globus Company?

- Univa was announces yesterday (Dec 13, 2004)
- _[http://biz.yahoo.com/prnews/041213/nym040_1.html](http://biz.yahoo.com/prnews/041213/nym040_1.html)_
- _[http://www.univa.com](http://www.univa.com/)_
- Steve Tuecke is CEO
- Both Carl Kesselman and Ian Foster are in advisory role
- Basic concept: “Redhat Linux for Globus”
- This will _NOT_ affect the GT open source policy
- This _WILL_ allow greater industrial involvement and investment in Grids

## Credits

- Globus Toolkit v4 is the work of many talented Globus Alliance members, at
- Argonne Natl. Lab & U.Chicago
- USC Information Sciences Corporation
- National Center for Supercomputing Applns
- U. Edinburgh
- Swedish PDC
- Univa Corporation
- Other contributors at other institutions
- Supported by DOE, NSF, UK EPSRC, and other sources

## Some Useful Pointers

- UK ETF GT4 report
- http://www.nesc.ac.uk/technical_papers/ UKeS-2005-03.pdf
- GRAM command line man page
- http://www.globus.org/toolkit/docs/4.0/ execution/wsgram/rn01re01.html
- GridFTP command line man page
- http://www.globus.org/toolkit/docs/4.0/data/gridftp/rn01re01.html
- MDS4 TG site
- http://snipurl.com/j24r

## For More Information

- Jennifer Schopf
- _[jms@mcs.anl.gov](mailto:jms@mcs.anl.gov)_
- www.mcs.anl.gov/~jms
- Globus Alliance
- www.globus.org
**GlobusWORLD 2006**
**September 10-14**
**Washington, D.C. **

- 2nd Edition
- www.mkp.com/grid2

## Short-Term Priorities: Security

- Improve GSI error reporting & diagnostics
- Secure password, one-time password, Kerberos support for initial log on
- Trust roots, use of GridLogon
- Identity/attribute assertions in GT auth. callouts (e.g., Shib, PERMIS, VOMS, SAML)
- Extend CAS admin & policy support
- Security logging with management control for audit purposes

## Short-Term Priorities: Data Management

- Space & bandwidth management in GridFTP
- Concurrency in globus-url-copy
- Priorities in RFT
- Data replication service
- Enhance policy support in data services
- Physical file name creation service
- Scalable & distributed metadata manager

## Short-Term Priorities: Execution Management

- Implement GGF JSDL once finalized
- Advance reservation support
- Policy-driven restart of “persistent” jobs
- Improved information collection for jobs
- Improved management of job collections
- Credential refresh
- Development of workspace service
- Integration of virtual machines (Xen, VMware) and associated services
- Windows port of WS GRAM

## Short-Term Priorities: Information Services

- Many more information sources, including gateways to other systems
- Automated configuration of monitoring
- Specialized monitoring displays
- Performance optimization of registry
- Archiver service
- Helper tools to streamline integration of new information sources

## Short-Term Priorities: WS Core

- Streamlined container configuration
- Remote management interface
- Dynamic service deployment
- Service isolation: multiple service instances
- WS-Notification, subscription performance
- Full functionality in C WS Core
- Optimized WS-ServiceGroup support
- WS-SecureConversation support