# Presentation: 446865

## A Globus PrimerWhat is the Grid and How Do I Use It?

- A Globus PrimerWhat is the Grid and How Do I Use It?

- Lee LimingU. of Chicago and Argonne National LaboratoryThe Globus AllianceGRIDS Center/NSF Middleware Initiative

## Approach

- What is the Grid?
- How are Grids built and used (today)?
- What Grid software is available and what does it do?
- BREAK (somewhere during part III)
- How have others succeeded?

- Globus Primer

- GlobusWORLD 2005

## I. What is the Grid?

## History

- For many years, a few whacky computer scientists have been trying to help other scientists use distributed computing.
- Interactive simulation (climate modeling)
- Very large-scale simulation and analysis (galaxy formation, gravity waves, battlefield simulation)
- Engineering (parameter studies, linked component models)
- Experimental data analysis (high-energy physics)
- Image and sensor analysis (astronomy, climate study, ecology)
- Online instrumentation (microscopes, x-ray devices, etc.)
- Remote visualization (climate studies, biology)
- Engineering (large-scale structural testing, chemical engineering)
- In these cases, the scientific problems are big enough that they required people in several organization to collaborate and share computing resources, data, instruments.

- Globus Primer

- GlobusWORLD 2005

## What Types of Problems?

- Your system administrators can’t agree on a uniform authentication system, but you have to allow your users to authenticate once (using a single password) then use services on all systems, with per-user accounting.
- You need to be able to offload work during peak times to systems at other companies, but the volume of work they’ll accept changes from day-to-day.

- Globus Primer

- GlobusWORLD 2005

## What Types of Problems?

- You and your colleagues have 6000 datasets from the past 50 years of studies that you want to start sharing, but no one is willing to submit the data to a centrally-managed storage system or database.
- You need to run 24 experiments that each use six large-scale physical experimental facilities operating together in real time.

- Globus Primer

- GlobusWORLD 2005

## Some Core Problems

- Too hard to keep track of authentication data (ID/password) across institutions
- Too hard to monitor system and application status across institutions
- Too many ways to submit jobs
- Too many ways to store & access files and data
- Too many ways to keep track of data
- Too easy to leave “dangling” resources lying around (robustness)

- Globus Primer

- GlobusWORLD 2005

## Challenging Applications

- The applications that Grid technology is aimed at are *not* easy applications!
- The reason these things haven’t been done before is because people believed it was too hard to bother trying.
- If you’re trying to do these things, you’d better be prepared for it to be challenging.
- Grid technologies are aimed at helping to overcome the challenges.
- They solve some of the most common problems
- They encourage standard solutions that make future interoperability easier
- They were developed as parts of real projects
- In many cases, they benefit from years of lessons from multiple applications
- Ever-improving documentation, installation, configuration, training

- Globus Primer

- GlobusWORLD 2005

## Earth System Grid

-    _Goal_: address technical obstacles to the sharing & analysis of high-volume data from advanced earth system models

- Globus Primer

- GlobusWORLD 2005

## Collaborative Engineering:NEESgrid

- Collaborative Engineering:NEESgrid

- U.Nevada Reno

- www.neesgrid.org

- Globus Primer

- GlobusWORLD 2005

## Enterprise Workload Management: SAP AG R/3

- _Goal_: Lower cost of operation and improve performance of sales and pricing tools.
- Applied to R/3 products (CRM, SCM)
- Adapts to changing demand & resources
- Discovers and allocates resources as needed to meet specified response time goals

- SAP AG R/3 Internet Pricing & Configurator (IPC)

- Globus Primer

- GlobusWORLD 2005

**Notes:**
In October 2004, SAP AG publicly demonstrated at its TechEd conference in San Diego, California, three applications from its flagship R/3 product line that had been modified to use a Globus-based Grid. The objective of these demonstrations was to show cost-effective use of an enterprise Grid to adapt to changing workload demands. Internet Pricing and Configurator (IPC) and Workforce Management (WFM) are commercial offerings in SAP’s Customer Relationship Management (CRM) application suite, while the third, Advanced Planner and Optimizer (APO), is offered as an application in their Supply Chain Management (SCM) suite. Each of these three applications was modified so that it could (1) dynamically adjust the number of worker process that could be employed to meet computational demands, and (2) use Globus components to discover and reserve available resources, and execute, monitor, and remove the worker processes on those resources.

Near-real-time Provisioning. SAP’s Internet Pricing and Configurator (IPC) application is used by many e-Commerce Web sites to compute prices when customers are shopping on the Web. Like APO, the IPC application currently cannot adapt to a changing work load, thus requiring the SAP customer to over-provision clusters to run IPC, and/or have unacceptable response time when demand is high due to many people simultaneously shopping on the Web.

For this demonstration, SAP modified IPC to monitor its response time, and when that response time reaches a customer-defined threshold, to use Globus technology to discover, reserve, and run additional worker processes on Globus-enabled computers. Within a matter of seconds, these worker processes come online and bring the Web site response time back to an acceptable level. Similarly, when the load on the system drops below a customer-defined threshold, Globus is used to shut down worker processes and remove them from the computers, thus allowing other similarly equipped applications to make use of those computers.

## Forget Homogeneity!

- Trying to force homogeneity on users is futile. Everyone has their own preferences, sometimes even *dogma*.
- The Internet provides the model...

- Globus Primer

- GlobusWORLD 2005

## Evolution of the Grid

- Increased functionality,
- standardization

- Time

- Globus Toolkit

- Open Grid
- Services Arch

- GGF: OGSI, WSRF, ...
- (leveraging OASIS, W3C, IETF)
- Multiple implementations,
- including Globus Toolkit

- Defacto standards
- GGF: GridFTP, GSI
- (leveraging IETF)

- Custom
- solutions

- X.509,
- LDAP,
- FTP, ...

- Web services

- App-specific
- Services

- Globus Primer

- GlobusWORLD 2005

## Open Grid Services Architecture

- Define a service-oriented architecture...
- the key to effective virtualization
- ...to address vital Grid requirements
- AKA utility, on-demand, system management, collaborative computing, etc.
- ...building on Web service standards.
- extending those standards when needed

- Globus Primer

- GlobusWORLD 2005

## WSRF & WS-Notification

- Naming and bindings (basis for virtualization)
- Every resource can be _uniquely referenced_, and has one or more _associated services_ for interacting with it
- Lifecycle (basis for fault resilient state management)
- Resources created by services following _factory_ pattern
- Resources destroyed _immediately_ or _scheduled_
- Information model (basis for monitoring & discovery)
- _Resource properties_ associated with resources
- Operations for _querying_ and _setting_ this info
- Asynchronous _notification_ of changes to properties
- Service Groups (basis for registries & collective svcs)
- Group membership rules & membership management
- Base Fault type

- Globus Primer

- GlobusWORLD 2005

## Grid and Web Services Convergence

- The definition of WSRF means that the Grid and Web services communities can move forward on a common base.

- Globus Primer

- GlobusWORLD 2005

## An “Ecosystem” of Grid Software

- There isn’t a Grid software kit for everybody (yet).
- Varying requirements
- Experimentation and learning
- Reluctance to invest in a “static” solution
- There _are_ many tools that work well together.
- Results of successful projects
- Reusable solutions
- Implication: Integrate it yourself (for now).
- Provides considerable flexibility
- Requires expertise and effort
- Reminder: *These are ambitious applications!*

- Globus Primer

- GlobusWORLD 2005

## II. How are Grids Built and Used (Today)?

## Methodology

- Building a Grid system or application is _currently_ an exercise in software *integration*.
- Define user requirements
- Derive system requirements or features
- Survey existing components
- Identify useful components
- Develop components to fit into the gaps
- Integrate the system
- Deploy and test the system
- Maintain the system during its operation
- This should be done iteratively, with many loops and eddies in the flow.

- Globus Primer

- GlobusWORLD 2005

## A Problem-Driven, Collaborative R&D Methodology

- A Problem-Driven, Collaborative R&D Methodology

- Design

- Deploy

- Build

- Apply

- Analyze

- Apply

- Apply

- Deploy

- Apply

- Globus Primer

- GlobusWORLD 2005

## Who Is the Grid For?

- Any Grid (distributed/collaborative) application or system will involve several “classes” of people.
- “End users” (e.g., Scientists, Engineers, Customers)
- Application/Product Developers
- System Administrators
- System Architects and Integrators
- Each user class has unique skills and unique requirements.
- The user class whose needs are met varies from tool to tool (even within the Globus Toolkit).

- Globus Primer

- GlobusWORLD 2005

## What End Users Need

- Secure, reliable, on-demand access to data,
- software, people, and other resources
- (ideally all via a Web Browser!)

- Globus Primer

- GlobusWORLD 2005

## How it Really Happens

- WebBrowser

- ComputeServer

- DataCatalog

- DataViewerTool

- Certificate
- authority

- ChatTool

- CredentialRepository

- WebPortal

- ComputeServer

*Resources implement standard access & management interfaces*

*Collective services aggregate &/or virtualize resources*

*Users work with client applications*

*Application services organize VOs & enable access to other services*

- Database
- service

- Database
- service

- Database
- service

- SimulationTool

- Camera

- Camera

- TelepresenceMonitor

- RegistrationService

- Globus Primer

- GlobusWORLD 2005

## How it Really Happens

- Implementations are provided by a mix of
- Application-specific code
- “Off the shelf” tools and services
- Tools and services from the Globus Toolkit
- Tools and services from the Grid community (compatible with GT)
- Glued together by...
- Application development
- System integration

- Globus Primer

- GlobusWORLD 2005

## How it Really Happens(with Grid community software)

- How it *Really* Happens(with Grid community software)

- WebBrowser

- ComputeServer

- GlobusMCS/RLS

- DataViewerTool

- CertificateAuthority

- CHEF ChatTeamlet

- MyProxy

- CHEF

- ComputeServer

*Resources implement standard access & management interfaces*

*Collective services aggregate &/or virtualize resources*

*Users work with client applications*

*Application services organize VOs & enable access to other services*

- Database
- service

- Database
- service

- Database
- service

- SimulationTool

- Camera

- Camera

- TelepresenceMonitor

- Globus IndexService

- GlobusGRAM

- GlobusGRAM

- GlobusDAI

- GlobusDAI

- GlobusDAI

| Application Developer | 2 |
| --- | --- |
| Off the Shelf | 8 |
| Grid Community | 9 |

- Globus Primer

- GlobusWORLD 2005

## Planning Tasks

- Identify project/application goals
- Identify specific functional requirements
- Identify/develop social policies, procedures, processes
- Identify components to meet requirements
- Form an integration plan and identify functional gaps
- Implement the product
- Deploy the product
- Provide O&M for the product, assess status and iterate

- Globus Primer

- GlobusWORLD 2005

## Application “Vision”

- Never underestimate the amount of work that can be wasted for lack of a specific target.
- Desired end user capabilities (accomplishments?)
- ...leads to: specific end user goals, milestones
- ...leads to: specific system *requirements*
- ...leads to: specific system *features*
- ...leads to: specific system* components*
- A specific target application/system will _focus_ the work dramatically. For example:
- Enable engineers to run *many* more simulation runs.
- Enable realtime remote observation of MRI scanning.
- Provide broad, easy-to-use access to satellite data.
- Encourage scientists to share/use experimental data.

- Globus Primer

- GlobusWORLD 2005

## Social Policies/Procedures

- How will people use the system?
- Who will set up access control?
- Who creates the data?
- How will computational resources be added to the system?
- How will simulation capabilities be used?
- What will accounting data be used for?
- Not all problems are solved by technology!
- Understanding how the system will be used is important for narrowing the requirements.

- Globus Primer

- GlobusWORLD 2005

## Continuous Deployment

-  Involve “real users” as early as possible
- You’ll learn a lot and be able to “course correct”
- You will establish a set of happy users to help down the road
- Pick early adopters carefully.
- Aggressive users, technologically skilled, representative of the target user base.
- Set expectations carefully.
- Be wary of over-investment.
- Deployment is a significant chunk of your effort.
- Separate team?
- Make sure it’s linked to the development activity.
- Demonstrate results early and often, and work with new users to get an “ownership” of the code and features

- Globus Primer

- GlobusWORLD 2005

## The Importance of Community

- All Grid technology is evolving rapidly.
- Web services standards
- Grid interfaces
- Grid implementations
- Grid hosting services (ASP, SSP, etc.)
- Community is important!
- Best practices (GGF, OASIS, etc.)
- Open source (Linux, Axis, Globus, etc.)
- Application of community standards is *vital*.
- Increases leverage
- Mitigates (a bit) effects of rapid evolution
- Paves the way for future integration/partnership

- Globus Primer

- GlobusWORLD 2005

## III. What Grid Software is Available and What Does It Do?

## An “Ecosystem” of Grid Software

- The role of the Globus Toolkit
- A tour of today’s ecosystem
- Security
- Monitoring and Discovery
- Computation/Processing
- Data
- Collaboration
- Packaging/Distribution

- Globus Primer

- GlobusWORLD 2005

## What Is the Globus® Toolkit?

- The Globus Toolkit is a collection of solutions to problems that frequently come up when trying to build collaborative distributed applications.
- Heterogeneity
- To date (v1.0 - v4.0), the Toolkit has focused on *simplifying heterogenity* for application developers.
- We aspire to include more “vertical solutions” in future versions.
- Standards
- Our goal has been to capitalize on and encourage use of existing standards (IETF, W3C, OASIS, GGF).
- The Toolkit also includes reference implementations of new/proposed standards in these organizations.

- Globus Primer

- GlobusWORLD 2005

## “Standard Plumbing” for the Grid

*Not* turnkey solutions, but *building blocks* and *tools* for application developers and system integrators.
- Some components (e.g., file transfer) go farther than others (e.g., remote job submission) toward end-user relevance.
- Since these solutions exist and others are already using them (and they’re free), it’s easier to reuse than to reinvent.
- And compatibility with other Grid systems comes for free!

- Globus Primer

- GlobusWORLD 2005

## Leveraging Existingand Proposed Standards

- SSL/TLS v1 (from OpenSSL) (IETF)
- LDAP v3 (from OpenLDAP) (IETF)
- X.509 Proxy Certificates (IETF)
- GridFTP v1.0 (GGF)
- OGSI v1.0 (GGF)
- And others on the road to standardization:
- WSRF (GGF, OASIS), DAI, WS-Agreement, WSDL 2.0, WSDM, SAML, XACML

- Globus Primer

- GlobusWORLD 2005

## What Does the Globus Toolkit Cover?

- Globus Primer

- GlobusWORLD 2005

## Areas of Competence

- “Connectivity Layer” Solutions
- Service Management (WSRF)
- Monitoring/Discovery (WSRF and MDS)
- Security (GSI and WS-Security)
- Communication (XIO)
- “Resource Layer” Solutions
- Computing / Processing Power (GRAM)
- Data Access/Movement (GridFTP, OGSA-DAI)
- In development: Telecontrol (NTCP/GTCP)
- “Collective Layer” Solutions
- Data Management (RLS, MCS, OGSA-DAI)
- Monitoring/Discovery (MDS)
- Security (CAS)

- Globus Primer

- GlobusWORLD 2005

## What Is the Globus Toolkit?

- A Grid development environment
- Develop new OGSA-compliant Web Services
- Develop applications using Java or C/C++ Grid APIs
- Secure applications using basic security mechanisms
- A set of basic Grid services
- Job submission/management
- File transfer (individual, queued)
- Database access
- Data management (replication, metadata)
- Monitoring/Indexing system information
- Tools and Examples
- The prerequisites for many Grid community tools

- Globus Primer

- GlobusWORLD 2005

## Globus Toolkit Components

- Globus Primer

- GlobusWORLD 2005

## How To Use the Globus Toolkit

- By itself, the Toolkit has surprisingly limited *end user* value.
- There’s very little user interface material there.
- You can’t just give it to end users (scientists, engineers, marketing specialists) and tell them to do something useful!
- The Globus Toolkit is useful to *application developers* and *system integrators*.
- You’ll need to have a specific *application or system* in mind.
- You’ll need to have the right *expertise*.
- You’ll need to set up prerequisite *hardware/software*.
- You’ll need to have a *plan*.

- Globus Primer

- GlobusWORLD 2005

## Security Tools

- Basic Grid Security Mechanisms
- Certificate Generation Tools
- Certificate Management Tools
- Getting users “registered” to use a Grid
- Getting Grid credentials to wherever they’re needed in the system
- Authorization/Access Control Tools
- Storing and providing access to system-wide authorization information

- Globus Primer

- GlobusWORLD 2005

## Basic Grid Security Mechanisms

- Basic Grid authentication and authorization mechanisms come in two flavors.
- Pre-Web services
- Web services
- Both are included in the Globus Toolkit, and both provide vital security features.
- Grid-wide identities implemented as PKI certificates
- Transport-level and message-level authentication
- Ability to delegate credentials to agents
- Ability to map between Grid & local identities
- Local security administration & enforcement
- Single sign-on support implemented as “proxies”
- A “plug in” framework for authorization decisions

- Globus Primer

- GlobusWORLD 2005

## Basic Grid Security Mechanisms

- Basic security mechanisms are provided as libraries/classes and APIs.
- Integrated with other GT tools and services
- Integrated with many Grid community tools and services (and applications & systems)
- A few stand-alone tools are also included.

- Globus Primer

- GlobusWORLD 2005

## A Cautionary Note

- Grid security mechanisms are tedious to set up.
- If exposed to users, hand-holding is usually required.
- These mechanisms can be *hidden entirely* from end users, but still used behind the scenes.
- These mechanisms exist for good reasons.
- Many useful things can be done without Grid security.
- It is unlikely that an ambitious project could go into production operation without security like this.
- Most successful projects end up using Grid security, but using it in ways that end users don’t see much.

- Globus Primer

- GlobusWORLD 2005

## Globus Certificate Service

- An online service that issues low-quality GSI certificates
- Intended for people who want to experiment with Grid components that require certificates but do not have any other means of acquiring certificates.
- These certificates are *not* to be used on production systems.
- Not a true Certificate Authority (CA)
- No revoking or reissuing certificates
- No verification of identities
- The service itself is not especially secure.

- Globus Primer

- GlobusWORLD 2005

## Simple CA

- A convenient method of setting up a certificate authority (CA).
- The Certificate Authority can then be used to issue certificates for users and services that work with GSI and WS-Security.
- Simple CA is intended for operators of small Grid testing environments and users who are not part of a larger Grid.
- Most production Grids will not accept certificates that are not signed by a well-known CA, so the certificates generated by Simple CA will usually not be sufficient to gain access to production services.

- Globus Primer

- GlobusWORLD 2005

## MyProxy

- MyProxy is a remote service that stores user credentials.
- Users can request proxies for local use on any system on the network.
- Web Portals can request user proxies for use with back-end Grid services.
- Grid administrators can pre-load credentials in the server for users to retrieve when needed.
- Greatly simplifies certificate management!

- Globus Primer

- GlobusWORLD 2005

## KX.509 and KCA

- Institutions that already have a Kerberos realm can use KX.509 and KCA to provide local users with Grid proxy certificates without using a Certificate Authority.
- When users authenticate with Kerberos, they may obtain proxy certificates in addition to their Kerberos tickets.
- KCA is a Kerberized certification service, and KX.509 is a Kerberized client that generates and stores proxy certificates.
- Unlike MyProxy, KX.509 and KCA create credentials for users, so remote sites must be configured to trust the local KCA service’s certification authority.

- Globus Primer

- GlobusWORLD 2005

## PKINIT

- PKINIT is a service that allows users to use Grid certificates to authenticate to a Kerberos realm.
- For sites that use Kerberized services (like AFS), this allows remote Grid users to obtain the necessary Kerberos tickets to use the site’s local facilities properly.
- PKINIT replaces the Kerberos “klog” command and uses the user’s Grid certificate to eliminate the need for a Kerberos passphrase.

- Globus Primer

- GlobusWORLD 2005

## CAS: Community Authorization Service

- CAS allows resource providers to specify course-grained access control policies in terms of communities as a whole.
- Fine-grained access control is delegated to the community.
- Resource providers maintain ultimate authority over their resources (including per-user control and auditing) but are spared most day-to-day policy administration tasks.

- Globus Primer

- GlobusWORLD 2005

## VOMS

- A community-level group membership system
- Database of user roles
- Administrative tools
- Client interface
- voms-proxy-init
- Uses client interface to produce an attribute certificate (instead of proxy) that includes roles & capabilities signed by VOMS server
- Works with non-VOMS services, but gives more info to VOMS-aware services
- Allows VOs to centrally manage user roles

- Globus Primer

- GlobusWORLD 2005

## Shibboleth

- Federated authorization and authentication management for Web-based services*
- Home institutions handle user authentication and role (“attribute”) information
- Resource providers use role info for authorization
- Allows for user privacy (resource provider sees a “handle” from the home institution, which can protect identity)

**** There is ongoing work to extend Shibboleth capabilities to non-browser applications, including Grid systems*

- Globus Primer

- GlobusWORLD 2005

**Notes:**
It is not uncommon for organizations (companies, Universities, libraries, etc.) to obtain access to remote services for use by members of their organization. For example, a library may establish a subscription to a commercial information service for use by library users. Shibboleth is aimed at freeing resource providers from having to maintain lists of all authorized users from all of the institutions that they serve. The diagram shows how a user gets access to a “new” resource. The web server doesn’t recognize the user, so it redirects him/her to a “Where are you from?” service that identifies the organization they belong to by querying the user. The organization manages Shibboleth handle and attribute authority services, which use local authentication services and user preferences to verify the user’s identity and generate a handle that the resource provider can use to identify the user. The resource can query the attribute authority for roles that the user should have (permission to use resources as a member of the organization) to determine whether to give the user access or not. Note that the “handle” does not have to include any personal identification information, so the organization can enforce anonymity if they are required to do so. (Universities, for example, are sometimes prohibited by law from sharing student information with resource providers.) Shibboleth has been clearly demonstrated for use with web browser-based services, and some people are using it with other (non-browser) services. 

## Solution Example: PURSE

- _P_ortal-based _U_ser _R_egistration _Se_rvice
- Portal extensions (CGI scripts) that automate user registration requests.
- Solicits basic data from user.
- Generates cert request from ESG CA (implemented with “simple CA” from GT).
- Admin interface allows CA admin to accept/reject request.
- Generates a certificate and stores in MyProxy service.
- Gives user ID/password for MyProxy.
- Benefits
- Users never have to deal with certificates.
- Portal can get user cert from MyProxy when needed.
- Database is populated with user data.

- Globus Primer

- GlobusWORLD 2005

## Solution Example: VOX and VOMRS

- Extends VOMS to include an ESG-like registration service
- Web registration interface
- Builds user database with extended fields
- Populates VOMS server

- Globus Primer

- GlobusWORLD 2005

## Break

- Next up:III. More of the Grid EcosystemIV. How Others Succeeded

## Monitoring/Discovery Tools

- Basic OGSA Infrastructure Components
- Specialized Monitoring/Discovery Components
- Specialized collection/monitoring agents
- Viewing and display tools for showing system information for a variety of specialized purposes

- Globus Primer

- GlobusWORLD 2005

## OGSA Infrastructure Elements

- WS Core Monitoring Features
- GT 3.x implements OGSI ServiceData
- GT 4.x implement WSRF ResourceProperties
- In either case, the capability provides:
- a standard interface for obtaining status and configuration information
- a standard interface for clients to subscribe to particular information (i.e., notification)
- the basis for registration and “grouping” of related services

- Globus Primer

- GlobusWORLD 2005

## Globus Index Service

- Provides a registry capability
- Services register with the index service to make their presence known to other Grid components
- Index service can also pro-actively subscribe to specific services based on configuration
- Data, datatype, data provider information
- Provides a caching capability
- Caches resource property values from registered services
- Indexes can be set up for a variety of uses, projects

- Globus Primer

- GlobusWORLD 2005

## Globus Trigger Service

- Triggers email alerts when pre-defined conditions are met
- Especially useful for system status alerts in “production” operation situations
- Monitoring behavior
- Subscribes to a set of resource properties
- Runs a set of pre-configured tests on the resulting data streams to evaluate trigger conditions
- When a trigger condition matches, sends email to the pre-configured address(es)

- Globus Primer

- GlobusWORLD 2005

## Archive Service

- Logs the values of resource property information over time
- Especially useful for auditing uptime/downtime for service agreement and/or diagnostic purposes
- Consumes resource property data
- Subscribes to a set of resource properties
- Stores resulting data in a database (Xindice)
- Provides access to stored data
- Other consumers can contact database archive interface

- Globus Primer

- GlobusWORLD 2005

## WebMDS

- Web browser interface to resource properties and index services
- Collects monitoring information from pluggable sources
- By default, by using WSRF Resource Properties poll operations
- MDS archiver plug in development to display historical information
- Formats monitoring data (in XML) into HTML output using pluggable XSLT style sheets
- Runs in Tomcat

- Globus Primer

- GlobusWORLD 2005

## Ganglia Cluster Toolkit

- Ganglia is a toolkit for monitoring clusters and aggregations of clusters (hierarchically).
- Ganglia collects system status information and makes it available via a web interface.
- Ganglia status can be subscribed to and aggregated across multiple systems.
- Integrating Ganglia with MDS services results in status information provided in the proposed standard GLUE schema, popular in international Grid collaborations.

- Globus Primer

- GlobusWORLD 2005

## TeraGrid’s Inca

- Hierarchical Status Monitoring
- Groups tests into logical sets
- Supports many levels of detail and summarization
- Flexible, scalable architecture
- Very simple reporter API
- Can use existing test scripts (unit tests, status tools)
- Hierarchical controllers
- Several query/display tools

- Globus Primer

- GlobusWORLD 2005

## MonALISA

- Supports system-wide, distributed monitoring with triggers for alerts
- Java/JINI and Web services
- Integration with Ganglia, queuing systems, etc.
- Client interfaces include Web and WAP
- Flexible registration and proxy mechanisms support look-ups and firewalls

- Globus Primer

- GlobusWORLD 2005

## Solution Example:Earth System Grid

- Monitors current status and history of ESG RLS, OGSA-DAI, and Web Portal services.
- Based on GT Index Service, with additions:
- Archiver Service
- Trigger Service
- Web-based Service Data Viewer
- Email alerts
- Python test scripts
- Everything can be reused!

- Globus Primer

- GlobusWORLD 2005

## Computing/Processing Tools

- Workflow Managers
- Organize and coordinate task execution within a complicated application
- Often coordinates data movement and task execution
- Metaschedulers
- Optimize use of distributed compute pools
- Virtual Data Tools
- Manage the trade-off between data storage and processing power

- Globus Primer

- GlobusWORLD 2005

## GRAM - Basic Job Submission and Control Service

- A uniform service interface for remote job submission and control
- Includes file staging and I/O management
- Includes reliability features
- Supports basic Grid security mechanisms
- Available in Pre-WS and WS
- GRAM is *not* a scheduler.
- No scheduling
- No metascheduling/brokering
- Often used as a front-end to schedulers, and often used to simplify metaschedulers/brokers

- Globus Primer

- GlobusWORLD 2005

## Grid-enabled Schedulers

- Scheduling systems that are easily integrated with GRAM via plug-ins
- Wide variety of capabilities (supported models)
- Wide variety of support (commercial vs. open source)
- Note that GRAM can be used as *either* an interface to a scheduler *or* the interface that a scheduler uses to submit a job to a resource.

- Condor
- OpenPBS
- Torque
- PBSPro
- Sun Grid Engine
- Platform LSF

- Globus Primer

- GlobusWORLD 2005

**Notes:**
Condor has many interesting scheduling features, including a scavenger mode, the ability to form “pools” of otherwise unrelated systems, and a matchmaking model for matching requests to available services. OpenPBS, Torque, and PBSPro are all variants of the PBS Portable Batch System. (Torque and PBSPro are commercial products.)  Sun Grid Engine is available as a free open source tool or as a commercial product. Platform LSF is a commercial product line that scales across a wide range of needs.

## Platform CSF

- An open source implemenation of OGSA-based metascheduler for VOs.
- Supports emerging WS-Agreement spec
- Supports GT GRAM
- Uses GT Index Service
- Fills in gaps in existing resource management picture
- Integrated with Platform LSF and Platform Multicluster
- Anticipated for inclusion in GT 4.0 release

- Globus Primer

- GlobusWORLD 2005

## Condor-G, DAGman

- Condor-G and DAGman address many workflow challenges for Grid applications.
- Managing sets of subtasks
- Getting the tasks done reliably and efficiently
- Submitting to Grid resources via GRAM
- Checkpointing and Migration

- Globus Primer

- GlobusWORLD 2005

## MPICH-G2

- A Grid-enabled implementation of the MPI v1.1 framework
- For applications that have concurrent jobs that communicate with each other
- Chooses communication method (shared memory, local high-performance interconnect, IP, etc.) based on location of source & destination
- Uses Grid for security, job submission, WAN communication, etc.

**MPICH-G2 automatically chooses the best communication method for messages based on the location of each subjob and other factors.**

- Globus Primer

- GlobusWORLD 2005

**Notes:**
The MPI model is especially useful for parallel applications that use many concurrent processes communicating with each other to solve a problem.  Many physical modeling applications (structural and mechanical engineering, fluid dynamics, climate modeling) fall into this category. MPI provides programmers with an easy-to-use framework for building their programs without having to worry about how messages are passes between processes or how the processes are executed.  MPICH-G2 is an MPI implementation that uses Grid capabilities for job submission (starting processes on remote Grid systems), security (authenticating to remote systems and authenticating message channels between distributed systems), and message passing between distributed systems. One of the best features of MPICH-G2 is that it is “smart” about choosing the best communication method for messages between processes: high-speed local interconnects for processes on the same system vs. IP (TCP or UDP) messages for processes on different systems.

## Ninf-G

- Grid-enabled RPC implementation
- Allows existing libraries to be used as RPC calls rather than local calls
- Implements GGF’s GridRPC specification

**Without Ninf:**** Library linked **
**directly to application**

**With Ninf:**** Library linked to Ninf server code and application linked to Ninf client stubs**

- Globus Primer

- GlobusWORLD 2005

**Notes:**
RPC is a programming model that allows library routines to be run remotely without the calling application seeing any difference in the API. A layer of software is interposed between the calling program and the library that mimics the library’s API. This is especially useful if the library offers routines that are computationally intensive and that can be run on different systems in parallel. The application can use threads or multiprocessing to invoke multiple library calls without blocking on their return and each call is processed on a different remote system. Note that the source code for the library is not needed: RPC can be used with binary libraries for which source is not available. Ninf-G uses the Globus GRAM service for remote job submission, GSI security, and (optionally) the Globus MDS Index service to discover remote job submission hosts. (Without an Index service, a local configuration file is used to identify remote resources.)

## Chimera “Virtual Data”

- Captures both logical and physical steps in a data analysis process.
- Transformations (logical)
- Derivations (physical)
- Builds a catalog.
- Results can be used to “replay” analysis.
- Generation of DAG (via Pegasus)
- Execution on Grid
- Catalog allows introspection of analysis process.

- Galaxy cluster
- size distribution

- Sloan Survey Data

- Globus Primer

- GlobusWORLD 2005

## Data Tools

- Virtual Data Tools
- Manage the trade-off between data storage and processing power (already covered)
- Movement/Transfer Tools
- Interfaces that meet specialized application or user needs
- “Last mile” integration to specialized storage systems
- Optimization Tools
- Help optimize the use of storage systems for specialized user communities

- Globus Primer

- GlobusWORLD 2005

## GridFTP

- A high-performance, secure, reliable data transfer service optimized  for high-bandwidth wide-area networks
- FTP with well-defined extensions
- Uses basic Grid security (control and data channels)
- Multiple data channels for parallel transfers
- Partial file transfers
- Third-party (direct server-to-server) transfers
- Reusable data channels
- Command pipelining
- GGF recommendation GFD.20

- Globus Primer

- GlobusWORLD 2005

## globus-url-copy

- Command-line client for GridFTP servers
- Text interface
- No “interactive shell” (single command per invocation)
- Many features
- Grid security, including data channel(s)
- HTTP, FTP, GridFTP
- Server-to-server transfers
- Subdirectory transfers and lists of transfers
- Multiple parallel data channels
- TCP tuning parameters
- Retry parameters
- Transfer status output

- Globus Primer

- GlobusWORLD 2005

## UberFTP

- UberFTP is an interactive (text prompt) client for GridFTP.
- Supports more features than NCFTP
- Parallelism
- Server-to-server transfers

- Globus Primer

- GlobusWORLD 2005

## GSI-SCP/SFTP

- GSI-OpenSSH is a version of OpenSSH that supports Grid authentication.
- Remote terminal (gsi-ssh)
- Remote Copy (gsi-scp)
- Secure FTP (gsi-sftp)
- More familiar to many users than GridFTP.
- Doesn’t take advantage of GridFTP’s capabilities (parallelism, partial files, third-party transfers, etc.)

- Globus Primer

- GlobusWORLD 2005

## RFT - File Transfer Queuing

- An OGSA service for queuing file transfer requests
- Server-to-server transfers
- Checkpointing for restarts
- Database back-end for failovers
- Allows clients to requests transfers and then “disappear”
- No need to manage the transfer
- Status monitoring available if desired

- Globus Primer

- GlobusWORLD 2005

## OGSA-DAI

-       Figure courtesy of Malcolm Atkinson and Rob Baxter, UK eScience Center

- OGSA interface for accessing XML and relational data stores
- Implements the GGF DAIS WG standard (in progress)

- Globus Primer

- GlobusWORLD 2005

## MCS - Metadata Catalog Service

- A stand-alone metadata catalog service
- OGSA service interface
- Stores system-defined and user-defined attributes for logical files/objects
- Supports manipulation and query
- Integrated with OGSA-DAI
- OGSA-DAI provides metadata storage
- When run with OGSA-DAI, basic Grid authentication mechanisms are available

- Globus Primer

- GlobusWORLD 2005

## RLS - Replica Location Service

- A distributed system for tracking replicated data
- Consistent local state maintained in Local Replica  Catalogs (LRCs)
- Collective state with relaxed consistency maintained  in Replica Location Indices (RLIs)
- Performance features
- Soft state maintenance of RLI state
- Compression of state updates
- Membership and partitioning information maintenance
- Note:
- RLS (developed by Globus Alliance and the DataGrid Project) replaces earlier components in the Globus Toolkit 2.x.

- Globus Primer

- GlobusWORLD 2005

## NeST

- Grid-optimized Storage Appliance
- Multiprotocol (pluggable) access mechanisms
- Pluggable storage system layer (flexible)
- Supports reservations
- Issues ClassAds (for use with Condor matchmaking)
- Can be run at user level
- Can manage users/groups
- “No Futz” installation

- Supports creation of I/O communities
- Combine the jobs-to-data and data-to-jobs strategies
- Users can build their own NeSTs
- Works very well with Condor and Condor applications

- Globus Primer

- GlobusWORLD 2005

## DataCutter

- Provides segmenting and processing capabilities for large (shared) datasets
- Filters - Services that can be “placed” (executed) on computational resources near storage system to transform the data in some way between endpoints
- Streams - Filters input and output data via streams, which can be unidirectional or bidirectional (pipe)
- Integrated with GSI (to secure streams), GRAM (to place filters), and MDS (to monitor and discover resources for filter placement)

- Globus Primer

- GlobusWORLD 2005

## Solution Example:Striped GridFTP Service

- A distributed GridFTP service that runs on a storage cluster
- Every node of the cluster is used to transfer data into/out of the cluster
- Head node coordinates transfers
- Multiple NICs/internal busses lead to very high performance
- Maximizes use of Gbit+ WANs

- Globus Primer

- GlobusWORLD 2005

## Solution Example: Lightweight Data Replicator

- Goal: Maintain large datasets replicated at several sites
- For sites: 2 - 100
- For data: 50Gb - 1Pb
- Motivation: LIGO gravitational wave observatory in U.S. and Europe

- Built on:
- GSI
- GridFTP
- Globus RLS
- PyGlobus
- Python daemons and client tools to bring it all together

- Globus Primer

- GlobusWORLD 2005

## Solution Example: Virtual Data System (VDS)

- Workflow implemented using virtual data and Grid processing capabilities
- Uses Metadata to convert user request to logical data sources
- Obtains abstract workflow info from a VDC (catalog)
- Uses replication data to locate physical files
- Delivers concrete workflow instructions to DAGman
- Executes using Condor
- Publishes new replication and derivation data in RLS and VDC

- ChimeraVirtual DataCatalog

- ReplicaLocationService

- MetadataCatalog

- StorageSystem

- StorageSystem

- StorageSystem

- ComputeServer

- ComputeServer

- ComputeServer

- ComputeServer

- DAGman

- Condor

*  *

*t*

- Globus Primer

- GlobusWORLD 2005

## Collaboration Tools

- Web Portals
- Tools for building web interfaces that provide access to system/application capabilities
- High-function Virtual Meeting Spaces
- Tools for facilitating collaborative work in realtime with lots of gadgets

- Globus Primer

- GlobusWORLD 2005

## Sakai

- Flexible web-based system for supporting distributed learning and collaborative work
- Allows users and groups to establish shared web spaces and add content via a browser interface
- Emphasizes JSR-168 portlet specification and localization
- Highly extensible, with support for JetSpeed, Velocity, and other portal interfaces

- Globus Primer

- GlobusWORLD 2005

## Open Grid Computing Environment (OGCE)

- Provides standard compliant portlet components for building Web portals.  E.g.,
- MyProxy
- GridPort
- GT services (GRAM, GridFTP, MDS, etc.)
- Java CoG
- Provides a “quick start” for building Grid-enabled portals.

- Globus Primer

- GlobusWORLD 2005

## Access Grid

- Provides seamless group-to-group collaboration spaces for groups that are not co-located
- Immersive audio (sounds like “everyone is here”)
- Multiple multicast video streams (multipoint “everyone sees everyone”)
- Can integrate other video sources into the space
- Display walls common, but not required
- “Virtual spaces” (like channels) allow people to find each other
- Ideally, groups can work together without thinking about the technology.
- Increasingly Grid-enabled

- Globus Primer

- GlobusWORLD 2005

## System Packaging/Distribution

- Software Distribution and Packaging Tools
- Getting software distributed and installed uniformly throughout a broad collaboration
- Tools that help create integrated distributions that work on a wide variety of systems
- NOT COVERED (in the interest of time)
- See: PACMAN
- See: Grid Packaging Technology (GPT)

- Globus Primer

- GlobusWORLD 2005

## IV. How Others Have Succeeded

## Success Stories

- Earth System Grid
- Data Intensive
- Climate Studies
- Management of datasets generated by massive computation
- Focus is on access to distributed data

- NEESgrid
- Distributed Collaboration
- Earthquake Engineering
- Physical & Simulation capabilities combined
- Focus on experimentation and community data

- Globus Primer

- GlobusWORLD 2005

## Data-Intensive Science:the Earth System Grid

- Data-Intensive Science:the Earth System Grid

- Globus Primer

- GlobusWORLD 2005

## ESG Project Goals

- Improve productivity/capability for the simulation and data management team (data producers).
- Improve productivity/capability for the research community in analyzing and visualizing results (data consumers).
- Enable broad multidisciplinary communities to access simulation results (end users).
- The community needs an integrated “cyberinfrastructure” to enable smooth *workflow* for *knowledge development*: compute platforms,  collaboration & collaboratories, data management, access, distribution, and analysis.

- Globus Primer

- GlobusWORLD 2005

## Earth System Grid

-    _Goal_: address technical obstacles to the sharing & analysis of high-volume data from advanced earth system models

- Globus Primer

- GlobusWORLD 2005

## ESG Requirements

- Move data a minimal amount, keep it close to computational point of origin when possible.
- When we must move data, do it fast and with a minimum amount of human intervention.
- Keep track of what we have, particularly what’s on deep storage.
- Make use of the facilities available at a number of sites. (Centralization is not an option.)
- Data must be easy to find and access using standard Web browsers.

- Globus Primer

- GlobusWORLD 2005

## Major ESG Components

- Grid Services
- GRAM
- GridFTP (+striped GridFTP server)
- MDS (+WebSDV, +Trigger Service, +Archiver)
- MyProxy
- SimpleCA
- RLS
- MCS

- Other Services
- OpenDAPg
- HPSS
- SRM
- Apache, Tomcat
- ESG-specific services
- Workflow Manager
- Registration Service

- Globus Primer

- GlobusWORLD 2005

## Under the Covers of ESG

- Globus Primer

- GlobusWORLD 2005

## ESG Deployment

- Four data centers (LBNL, LLNL, NCAR, ORNL)
- User registration and authorization established
- Two major datasets are available, with associated metadata
- Work underway to add IPCC datasets as they are produced

- Globus Primer

- GlobusWORLD 2005

## ESG Interesting Points

- A lot of effort has been needed to build acceptable metadata models.
- Ease of use (simple interfaces, like registration service) is critical!
- Users shouldn’t have to see anything other than web interface and the data they ask for.
- Don’t bother giving certificates to users as long as they’re using the portal for everything.
- Specific goals (e.g., providing access to specific datasets) will dramatically focus work.

- Globus Primer

- GlobusWORLD 2005

## Collaborative Engineering:NEESgrid

- Collaborative Engineering:NEESgrid

- U.Nevada Reno

- www.neesgrid.org

- Globus Primer

- GlobusWORLD 2005

## NSF’s Goals for NEESgrid

- Encourage collaboration among earthquake engineering researchers and practitioners.
- Provide remote access to large-scale NSF earthquake engineering facilities.
- Provide distributed collaboration tools.
- Provide easy-to-use simulation capabilities.
- Allow integration of physical and simulation capabilities.
- Provide a community data repository for sharing data generated by use of the system.
- Create a *cyberinfrastructure* for earthquake engineering.
- Define and implement Grid-based integration points for system components.

- Globus Primer

- GlobusWORLD 2005

## NEESgrid Core Capabilities

- Tele-control and tele-observation of experiments
- Data cataloging and sharing
- Remote collaboration and visualization tools and services
- Simulation execution and integration

- Globus Primer

- GlobusWORLD 2005

## NEESgrid Requirements

- Single sign-on with Grid credentials
- Web interfaces for end users
- Collaboration services (chat, video, documents, calendars, notebooks, etc.)
- Telepresence services (video feeds)
- Telecontrol (in limited instances)
- Data viewing, data browsing and searching
- Simulation capabilities
- Uniform interfaces for major system capabilities
- Control
- Data acquisition
- Data streams
- Data repository services

- Globus Primer

- GlobusWORLD 2005

## More NEESgrid Requirements

- System security
- Protect facilities from misuse
- Physical safety!
- Distributed collaboration during realtime experiments
- Automated (pre-programmed) control of distributed experiments (physical *and* simulation)
- Simplify effects of heterogeneity at facilities

- Globus Primer

- GlobusWORLD 2005

## NEESgrid High-level Structure

**Certificate Authority****MyProxy****Account Mgmt Tools**
**Index Service**
**Monitoring Tools**
**NEESgrid Website**
**Bugzilla**
**Mailing Lists**

- Globus Primer

- GlobusWORLD 2005

## Architecture ofNEESgrid Equipment Site 

- Globus Primer

- GlobusWORLD 2005

## Major NEESgrid Components

- OGSA Services
- NTCP - Uniform Telecontrol Interface
- NMDS - Metadata Repository Management
- NFMS - File Repository Management
- Creare Data Turbine - Data & Video
- CHEF - Web Portal, Collaboration Tools
- NEESgrid Simulation Portal - Simulation Tools
- OpenSEES, FedeasLab - Simulation Frameworks
- Other Grid Services
- MyProxy - Authentication
- GridFTP - File Movement
- GRAM - Job Submission/Management
- MDS, Big Brother - System Monitoring
- GSI-OpenSSH - Administrative Logins
- GPT - Software Packaging

- Globus Primer

- GlobusWORLD 2005

## Tele-Control Services

- Transaction-based protocol and service (NTCP) to control physical experiments and computational simulations.
- OGSI-based implementation (GT3.2)
- Plug-ins to interface the NTCP service
- A computational simulation written in Matlab
- Shore Western control hardware
- MTS control hardware (via Matlab and xPC)
- LabView control software
- Still-image camera control
- DAQ triggering
- Security architecture, including GSI authentication and a flexible, plug-in-based authorization model.

- Globus Primer

- GlobusWORLD 2005

## Tele-Control Services

- Globus Primer

- GlobusWORLD 2005

## NEESgrid Deployment

- NEES-POPs installed at 16 facilities
- Experiment-based Deployment (EBD)
- Sites propose experiments
- SI and sites cooperatively run experiment using NEESgrid (deployment)
- Tests architecture and components, identifying new requirements
- October 2004 transition to M&O team (SDSC)
- First round of research proposals also begin in October 2004
- Grand Opening in November 2004

- Globus Primer

- GlobusWORLD 2005

## NEESgrid Interesting Points

- Requirements are hard to define when a community is unused to collaboration.
- Early deployment and genuine use is critical for focusing work.
- Iterative design is useful in this situation.
- Considerable effort has been needed for data modeling (still unproven).
- Plug-in interfaces (“drivers”) are much more useful than originally imagined.
- Real users don’t want to deal with WSDL. They need user-level APIs.

- Globus Primer

- GlobusWORLD 2005

## Conclusions

## Lessons Learned

- The Globus Toolkit has useful stuff in it.
- To do anything significant, a lot more is needed.
- The Grid community (collectively) has many useful  tools that can be reused!
- System integration expertise is mandatory.
- OGSA and community standards (GGF, OASIS, W3C, IETF) are extremely important in getting all of this to work together.
- There’s much more to be done!

- Globus Primer

- GlobusWORLD 2005

## Continue Learning

- Visit the Globus Alliance website at: www.globus.org
- Read the book: *The Grid: Blueprint for a New Computing Infrastructure (2**nd** edition)*
- Talk to others who are using the Toolkit:discuss@globus.org   (subscribe first)
- Participate in standards organizations:GGF, OASIS, W3C, IETF
- Attend GlobusWORLD, an annual event

- Globus Primer

- GlobusWORLD 2005