# Presentation: VIDRJW6O3QVGGX6R5ZQD6UWQ4FLR3CS6

## Grid Applications Today and Tomorrow

- Ed Seidel
- Director, Center for
- Computation &
- Technology, LSU
- Co-Chair,
- Applications Research Group, GGF
- TWG-Apps, EU Gridstart Project

- PHYSICS TODAY

## The GridHot Area in Computational Science

- Computational Devices
- Scattered Across the World
- Compute servers (double 18 months)
- Handhelds
- Networks (double each 9 months)
- Playstations, cell phones
- Sensors
- How to take advantage of this for
- science, engineering, business, art?
- Harness multiple sites and devices
- Make it accessible to scientists,
- engineers, artists
- Integrate communities and
- resources

## Current Grid App Types

- Community Driven
- Distributed communities share resources
- Video Conferencing
- Virtual Collaborative Environments
- Data Driven
- Remote access of huge data, data mining
- Weather Information systems
- Particle Physics
- Process/Simulation Driven
- Demanding Simulations of Science and Engineering
- Task farming, resource brokering, distributed computations, workflow
- Interactive
- Remote, steered, etc...

## What do we want to do on Grids?

- Collaboration
- Distributed communities share resources
- “Standard Things”
- Task Farming, Resource Brokering, Distributed jobs...
- “New Scenarios”
- Apps abstracted, become services
- Dynamic apps decide their future, find their services (data, resources, people)
- Apps distributed, spawned, task farmed, controlled, monitored with other apps/people

## From Telephone Conference Calls to Access Grid Intern’l Video Meetings

**Access Grid Lead-Argonne**
**NSF STARTAP Lead-UIC’s Elec. Vis. Lab**

**Creating a Virtual Global Research Lab**

**Source: Smarr**

## High Capacity Computing:  Want to Compute What Happens in Nature!

## Grand Challenge Collaborations

- NASA Neutron Star Grand Challenge
- • 5 US Institutions
- • Attack colliding neutron star problem
- EU Astrophysics Network
- • 10 EU Institutions
- • 3 years
- • Continue these problems

- NSF Black Hole Grand Challenge
- 8 US Institutions
- 5 years
- Attack colliding black hole problem

- Examples of Future of Science & Engineering
- • Require Large Scale Simulations, beyond reach of any machine
- • Require Large Geo-distributed Cross-Disciplinary Collaborations
- • Require Grid Technologies, but not yet using them!

## Numerical Relativity ApplicationsStudy sources of Grav. Waves

- Desired BH, NS collisions/collapse
- 10^20 Flops, multiple orbits, adaptive meshes, hydrodynamics, 1 day on 50TF machine
- Real time scheduling across multiple resources for distributed computing
- Lambda provisioning on demand to enable, spawning (for analysis, steering tasks), migration, interactive viz from distributed collaborations
- Output
- Minimal 25-100TB for full analysis of waves, event horizon structure, hydrodynamics (multiple orbits)
- May be physically distributed due to Grid activities above (spawning , migration, etc)

## Issues for LIGO, GEO, LISA Gravitational Wave Astronomy

- Ensembles of simulations
- Each simulation must be varied for accuracy, sensitivity analysis to gauge parameters, resolution, algorithms
- Dozen simulations per physical scenario
- Parameter space!
- Huge variations in physical scenarios for waveform catalogs
- In total:  103 - 106 simulations needed, potentially generating 25TB each, stored in federated repositories

## Storage, Network Issues

- Simulation data stored across geographically distributed spaces
- Organization, access, mining issues
- Need Lambda provisioning for
- Grid computing scenarios
- Remote viz and monitoring of simulations by distributed collaborations
- Analysis of federated data sets by virtual organizations
- Data analysis of LIGO, GEO, LISA signals
- Interacting with simulation data
- Managing parameter space/signal analysis

## Software/Toolkit Issues

- Major problem to enable community
- Require toolkits for
- Sharing/developing of community codes
- AMR, remote viz, analysis
- Logging, robust fault tolerance in grid scenarios, QoS, monitoring
- Lambda provisioning
- Data replication and management
- These problems are common to all!

## Future view: much of it here now

- Scale of computations much larger: Grids!
- Complexity approaching that of Nature
- Simulations of the Universe and its constituents
- Black holes, neutron stars, supernovae
- Human genome, human behavior
- Teams of researchers working together
- Must support efficient, high level problem description
- Must support collaborative computational science
- Must support all different languages
- Ubiquitous Grid Computing
- Resources, applications, replaced by abstraction: Grid Services
- Very dynamic simulations, even deciding their own future
- Apps may find services themselves:  distributed, spawned, etc...
- Must be tolerant of dynamic infrastructure
- Monitored, viz’ed, controlled from anywhere, with colleagues elsewhere
- Applications:  SCOOP, UCoMS, Hurricanes, others

## Many Components for Grid Computing

- Resources:  Apps Research Group
- Worldwide “Virtual Organization”
-  for Grid Computing
- Over 2 dozen sites, 4 continents:
- Alliance, NPACI, UK, Europe, Canada, Australia
- Infrastructure:  Globus, others
- Develops fundamental technologies needed
- to build computational grids.
- Security:  logins, data transfer
- Communication
- Information (GRIS, GIIS)
- More generally:  Grid Services
- Communities to use it!  Virtual Organizations
- Collaborations of groups with different expertise:  no single group

## A Missing Component for Grid Computing

- Grid Aware Applications
- Most apps developed before Grid
- Have no “awareness”
- Take limited advantage
- Develop new generation of Grid Applications
- Not just task farming/resource selection, totally new apps
- Apps find Grid services, or even provide them
-  Ease of Use:  automatically find resources, given need!
-  Distributed simulations:  use as many machines as needed!
-  Remote Viz and Steering, tracking:  watch what happens!
- Grid Enabled Modular Toolkits/Libraries for Grid Computation:  Provide to Scientist/Engineer
- Plug your Science/Eng. Applications in!

## Grid Applications so far...

- Typical scenario
- Find remote resource
- Task farm
- Distribute
-  Launch job(s)
- Static, possibly coupled
- Visualize or collect results
- Can go far beyond this
- Make it much, much easier
- Portals, standards
- Make it much more dynamic, adaptive, fault tolerant
- Migrate this technology to general user

***Metacomputing the Einstein Equations: Connecting T3E’s in Berlin, Garching, San Diego***

## Distributed Computation:Harnessing Multiple Computers

- Why would anyone want to do this?
- Capacity: computers can’t keep up with needs
- Throughput
- Issues
- Bandwidth (increasing faster than computation)
- Latency
- Communication needs, Topology
- Communication/computation
- Techniques to be developed
- Overlapping communication/computation
- Extra ghost zones to reduce latency
- Compression
- Algorithms to do this for scientist

## Remote Viz/Steering watch/control simulation live...

**Remote Viz data**

**HTTP**

**Streaming HDF5**
**Autodownsample**

**Amira**

**Any Viz Client:**
**LCA Vision, OpenDX**

**Changing any steerable parameter**
***•   ******Parameters***
***•   ******Physics, algorithms***
***•   ******Performance***

**Remote Viz data**

**Notification**

## Dynamic Adaptive Distributed Computation(T.Dramlitsch, with Argonne/U.Chicago)

- NCSA Origin Array
- 256+128+128
- 5x12x(4+2+2) =480

- OC-12 line
- (But only 2.5MB/sec)

- 2

- 12

- 5

- 2

- These experiments:
-  Einstein Equations (but could be any Cactus application)
- Achieved:
-  First runs: 15% scaling
-  With new techniques: 70-85% scaling, ~ 250GF

- Won
- “Gordon Bell Prize” (Supercomputing 2001, Denver)

- Dynamic Adaptation: Number of ghostzones, compression, ...

## Many New Ideas
Consider:  the Grid IS your computer
Networks, machines, devices come and go
Dynamic codes, aware of their environment, seek out resources
Distributed and Grid-based thread parallelism
Begin to change the way you think about problems:  think global, solve much bigger problems
Many old ideas
1960’s all over again
How to deal with dynamic processes
Processor management 
Memory hierarchies, etc

- Consider:  the Grid IS your computer
- Networks, machines, devices come and go
- Dynamic codes, aware of their environment, seek out resources
- Distributed and Grid-based thread parallelism
- Begin to change the way you think about problems:  think global, solve much bigger problems
- Many old ideas
- 1960’s all over again
- How to deal with dynamic processes
- Processor management
- Memory hierarchies, etc

- Dynamic Distributed Computing*Want Apps to be able to Adapt to Changing Environment*

## What do we want to do on Grids?

- Collaboration
- Distributed communities share resources
- “Standard Things”
- Task Farming, Resource Brokering, ...
- “New Scenarios”
- Apps abstracted, become services
- Dynamic apps decide their future, find their services (data, resources, people)
- Apps distributed, spawned, task farmed, controlled, monitored with other apps/people

## GridLab Projectwww.gridlab.org

- Funded by the EU (5+ M€), January 2002 – April 2005
- Application and Testbed oriented
- Cactus Code, Triana Workflow, all the other applications that want to be Grid-enabled
- Gravitational Wave Astro (LIGO/GEO) and Numerical Relativity
- Main goal: develop a Grid Application Toolkit (GAT) and set of grid services/tools...:
- resource management (GRMS),
- data management,
- monitoring,
- adaptive components,
- mobile user support,
- security services,
- portals,
- ... and test on real testbeds/applications

## GridLabwww.gridlab.org

- Resources, applications, etc replaced by abstract notion: Grid Services
- Grid is very dynamic, services come and go
- Develop protection against this
- Use to advantage:  matches nature of many applications
- Very dynamic simulations, even deciding their own future
- Apps may find the services themselves:  distributed, spawned, etc...
- Must be tolerant of dynamic infrastructure
- Monitored, viz’ed, controlled from anywhere, with colleagues elsewhere
- Developing tools/services to create this world
- Grid Application Toolkit (GAT)
- Need to become standardized

## New Paradigms for Dynamic Grids

- Code/User/Infrastructure should be aware of environment
- What Grid Sevices are available??
- Resources available NOW, and what is their current state?
- What is my allocation?
- What is the bandwidth/latency between sites?
- Where do I get my data from sensor network?
- Code/User/Infrastructure should be able to make decisions
- A slow part of simulation runs asynchronously...spawn it off!
- New, more powerful resources available...migrate there!
- Machine went down...reconfigure and recover!
- Need more memory (or less!): get it by adding(dropping) machines!
- Code/User/Infrastructure should be able to publish to central server for tracking, monitoring, steering...
- Unexpected event...notify users!
- Collaborators from anywhere all connect, examine simulation.

## New Grid Applications

- Intelligent Parameter Surveys, Monte Carlos
- May control other simulations!
- Dynamic Staging: move to faster/cheaper/bigger machine (“Grid Worm”)
- Multiple Universe:  clone to investigate steered parameter (“Gird Virus”)
- Automatic Component Loading
- Needs change, discover/load/execute component
- Automatic Convergence Testing
- Look Ahead
- Spawn off and run coarser resolution to predict likely future
- Spawn Independent/Asynchronous Tasks
- Dynamic Load Balancing:  inhomogeneous loads, multiple grids
- Inject dynamically aqcuired data

## Future Dynamic Grid Computing

**Free CPUs!! **

**Add more **
**resources **

**Clone job with **
**steered parameter **

**Queue time over, **
**find new machine **

## What can be done now?

- Some Current Examples, work NOW as prototypes:  Building blocks for the future
- Dynamic, Adaptive Distributed Computing
- Increase scaling from 15 - 70%
- Migration:  Cactus Worm
- Spawning
- Task Farm/Steering Combination
- If these can be done now, think what you could do tomorrow
- We are developing tools to enable such scenarios for any applications

## GridLab Members

- PSNC (Poznan) - coordination
- AEI (Potsdam)/LSU
- ZIB (Berlin)
- Univ. of Lecce
- Cardiff University
- Vrije Univ. (Amsterdam)
- SZTAKI (Budapest)
- Masaryk Univ. (Brno)
- NTUA (Athens)
- Sun Microsystems
- Compaq (HP)
- ANL (Chicago, I. Foster)
- ISI (LA, C.Kesselman)
- UoWisconsin (M. Livny)

- Collaborating with:  Users!
- SCOOP
- UCoMS
- EU Astrophysics Network
- DFN TiKSL/GriKSL
- NSF ASC Project
- other Grid projects
- Globus, Condor,
- GrADS,
- PROGRESS,
- GriPhyn/iVDGL,
- Most of the other European Grid Projects (GRIDSTART)
- Gravitational Wave European Network

## Spawning on ARG Testbed

- Main Cactus BH Simulation starts here

- All analysis tasks spawned automatically to free resources worldwide

- User only has to invoke “Spawner” thorn...

## Global Grid Testbed Collaboration

- 5 continents
- North America, Europe, Asia, Africa, Australia
- Over 14 countries, including China, Japan, Singapore, S. Korea, Egypt, Australia, Canada, Germany, UK, Netherlands, Czech, Hungary, Poland, USA
- About 70 machines, 7500+ processors
- Many hardware types, including PS2, IA32, IA64, MIPS,
- Many OSs, including Linux, Irix, AIX, OSF, True64, Solaris, Hitachi
- Many organizations
- DOE, NSF, MPG, Universities, Vendors
- And yet:  all run same Grid infrastructure, and many can be used for applications...
- 4 different applications

## Task Farm/Steering Combo 

- Main Cactus BH Simulation starts in Berkeley

- Dozens of smal. jobs sent out to test parameters

- Data returned for main job

- Huge job generates remote data
- to be visualized in Baltimore

## Notification

- Local monitoring:
- Clients on machines provide information

- Portal stores users
- details and
- notification
- preferences

## Task Farm/Remote Viz/Steer Demo

- Big BH  Sim
- (LBL, NCSA, PSC, ...)

-  Task Farm Manager Status in portal...

## Grid2003: An Operational Grid

**Grid2003: An Operational Grid**
-  **28 sites (2100-2800 CPUs)**
-  **400-1300 concurrent jobs**
-  **10 applications**
-  **Running since October 2003**

**Korea**

**http://www.ivdgl.org/grid2003**

## Grid2003: A Necessary Step

- Learning how to operate a Grid
- Add sites, recover from errors, provide info, update, test, etc.
- Need tools, services, procedures, documentation, organization
- Need reliable, intelligent, skilled people
- Learning how to cope with “large” scale
- “Interesting” failure modes as scale increases
- Increasing scale must not overwhelm human resources
- Learning how to delegate responsibilities
- Multiple levels: Project, Virtual Org., service, site, application
- Essential for future growth

-  **Grid2003 experience critical for building “useful” Grids**
-  **Frank discussion in “Grid2003 Project Lessons” doc**

## DØ Southern Analysis Region (DØSAR)

**DØ ****S****outhern ****A****nalysis ****R****egion (DØSAR)**

**Consortium coordinating activities to maximize computing and analysis resources**
**UTA, OU, LTU, LU,**** ****SPRACE, Tata,**** ****KSU, KU, ****UMiss, CSF**
**MC farm clusters – dedicated and multi-purpose, rack mounted and desktop, 10’s-100’s CPU’s**
**http://www-hep.uta.edu/d0-sar/d0-sar.html**

## Conclusions

*                                                                                                                                                      *

**Conclusions**

- DØSAR has considerable resources used in an  effective manner
- Generated and delivered a total of 18.9M MC events
- TMB Fixing in progress at UTA-RAC
- Plan to participate in re-processing
- DØSAR is moving to SG/runjob
- McFarm Þ SG/McFarm Þ SG/runjob
- Migrate DØSAR monitoring tools to SG/runjob
- Enhance brokering capabilities
- Need enhanced brokering for seamless job creation to execution to data storage
- Will work with FNAL SG Team for smooth integration and implementation of D0Grid
- DØSAR demonstrates that “Grid” empowers
- 4th DØSAR Workshop at KSU in Sep., 2004.
- We should organize and collaborate on Grid applications for High Energy Physics and Astrophysics in Louisiana!

## Confusion: GGF products, home grown products, other products...

- Few complete specifications
- Few mature, compliant implementations
- Need to wait for more finished product

- All grid products

- GGF standard

- Relevant to VObs

- Mature implementation

- Now

## TWG: “Grid Applications and Application Interfaces” 

- Peter Kacsuk, SZTAKI, Budapest
- Jochen Fingberg, NEC Europe
- Guy Lonsdale, NEC Europe
- Ed Seidel, AEI, Potsdam and LSU, USA

- Goals
- Define both common and specific requirements of applications for Grid environments in terms of
- application classes and domains
- particular Grid capabilities
- Issues
- Grid Application Programming Interface (A P I)
- Tools that would facilitate new application development,
- Legacy application integration within  Grids
- More uniform and efficient development and deployment

**Notes:**
ADVISE ON NAME CHANGE IN PROCESS
	LSU CAPITAL
	CENTER FOR COMPUTATION & TECHNOLOGY

PRESENTING ON BEHALF OF ALL LOUISIANA’S RESEARCH INSTITUTIONS

## How to build Grid Apps? 

**Monitoring**

**Resource **
**Management**

**Information**

**Security**

**Data**
**Management**

**GLOBUS (v1, 2, 3, 4, ...)**

**Application**
**Manager**

**Logging**

**Notification**

**Migration**

**Profiling**

**SOAP**

**WSDL**

**CORBA**

**OGSA**

**Other**

**Other Grid**
**Infrastructure?**

**“****Is there a better resource I could be using?”**

**UNICORE**

**“****Where can I get my data?”**

- How to build Grid Apps?

## Grid Application Toolkit 

- “Is there a better resource I could be using?”

- GAT_FindResource(  )

## GAT Adaptor

- Interface between GAT Engine and one or more capabilities
- Translates user requests to appropriate interface syntax for a capability provider
- Active adaptors change dynamically
- Includes “security context”
- Return appropriate error codes
- Examples
- OGSA adaptor (provides many capabilities)
- Globus adaptor (directly talk to gatekeepers)
- Adaptors for each GridLab service provider
- “Local” adaptors (GAT_MoveFile => “cp”, GATFindResource => “localhost”)

## SCOOP: Southeastern Coastal Ocean Observing Program
Involving LSU, FLorida, W&M
Sensor networks deployed across Gulf
Linked to computer models
UCoMS
Petroleum Engineering, CS, ULL
Deploy sensor networks across Gulf
Data collected to provide input to simulations, task farmed out
Results collected (transmitted back)

- Involving LSU, FLorida, W&M
- Sensor networks deployed across Gulf
- Linked to computer models
- UCoMS
- Petroleum Engineering, CS, ULL
- Deploy sensor networks across Gulf
- Data collected to provide input to simulations, task farmed out
- Results collected (transmitted back)

- Grid Applications

## Philosophy

- Application makes GAT API calls for operations which may be Grid-related.
- Application links against the GAT Engine
- Application runs irrespective of actual underlying infrastructure deployment
- Engine loads adaptors which are valid in the environment extant when the application starts
- Adaptors try to do Grid operations on request, on failure another adaptor provided function may be called.
- Application can thus be compiled, linked and tested without any Grid services
- Same application executable can run in a full Grid enviroment.

## SAGA GGF Working Group

- Simple API for Grid Applications
- Aims to define standard APIs for commonly needed Grid operations
- Various implementations to be created
- GridLab GAT with GAT Engine and adaptors
- Others
- Leaders
- Keith Jackson, LBNL
- Tom Goodale, LSU
- Stephen Pickles, Manchester

## Data coming in from sensors all over Gulf Data Grid
Feeds in to models on SuperMike and other Grid sites
Indication of strengthening storm changing direction:  Must Act!
Multiple simulations, expertise, and data connected together:  FL, LA, notification, steering sensors
Early warning, preparation:
Insurance industry, petroleum industry, local economy all save millions, lives saved

- Feeds in to models on SuperMike and other Grid sites
- Indication of strengthening storm changing direction:  Must Act!
- Multiple simulations, expertise, and data connected together:  FL, LA, notification, steering sensors
- Early warning, preparation:
- Insurance industry, petroleum industry, local economy all save millions, lives saved

- SCOOP:  Hurricanes on Grid

- Source:  Wilhelmson, Jewett, Shaw

## Summary

- Grids are coming
- A lot of hype, promise, but definitely coming
- Will be critical for exploiting resources, integrating communities
- New applications must be created to take advantage
- New paradigms, New Tools
- Think creatively!
- We are in the early phases
- Similar to parallel computing decade ago
- MPI, HPF, etc ---> GAT, other Grid App development tools
- Need to move towards standards:  SAGA, GAT:  you can help!