# Presentation: 190059

## 10 Actions When SuperScheduling:A Grid Scheduling Architecture 

- Jennifer Schopf
- Argonne National Laboratory
- jms@mcs.anl.gov
- Scheduling Architecture Workshop
- GGF7, Tokyo
- March 5, 2003

**Notes:**
Will show many slides quickly – some to give you a chance to refer back to them.

## 10 Actions History

- 10 Actions for Superscheduling
- June 2000-July 2001
- How does a user schedule on the grid?
- Resulted in GGF document GWD-CI.5
- Late 2002 this was updated, sent to special issue
- Includes examples from current approaches
- www.mcs.anl.gov/~jms/Pubs

## Context

- Grid scheduling defined as the process of making scheduling decisions involving resources over multiple administrative domains
- User is currently the most common “Grid Scheduler”
- The steps defined are currently performed by other Grid-level schedulers, but none of them by any one approach

## Context, cont.

- Grid schedulers aren’t Local Resource Managers (LRMS)
- no ownership or control over resources
- jobs get submitted to LRMS as user
- Grid scheduler doesn’t have control or often even info about job submitted at this level
- We did not consider errors

## In a nutshell...

## Ordering is approximate(from GGF doc)

- 	We use the word “step” and a numbering system for easy reference.  This does not imply that these actions are actually performed in this order, or that they all MUST occur in every system. In general, don’t pay too much attention to the numbering. Some of the steps may be interactive, recursive, repeated, or just plain ignored by a given system.

## Phase One: Resource Discovery

- Phase One-Resource Discovery

## 1. Authorization Filtering

- Where do you have an account?
- User:
- List in a drawer
- Ideally:
- A “wallet” of credentials, smart enough to remember my username at different sites as well
- Today’s systems:
- EDG and GridLab scheduler use MDS for this info

## 2. Application Definition

- User:
- Generally user defined
- Often inaccurate, incomplete
- Ideally:
- Smart compilers or other tools to automatically generate information about application requirements and runtimes
- Today’s systems:
- User defined at the command line info or Condor ClassAds

## 3. Minimum Requirement Filtering

- Use static data to limit the search space
- Used to cut down dynamic queries needed
- Can be combined with dynamic search (4)
- User:
- I know I need Linux, I don’t consider others
- Ideally:
- Automatic, part of dynamic search
- Today’s systems:
- Part of dynamic search (4)

## Phase One: Resource Discovery

- Phase One-Resource Discovery

- Phase Two - System Selection

## 4. Information Gathering

- Dynamic searches to match resources with application requirements
- User:
- Might use the Globus MDS or a portal information service like HotPage, might just know
- Ideally:
- Seamless interface to global monitoring
- Today’s systems
- EDG interacts with MDS, PBS has it’s own internally

## 5. System Selection

- Matching between resources and application information
- Users:
- Best estimate
- Ideally:
- Perfect matches based on current information, using variance information and other predictions
- Today’s systems:
- Condor - matchmaking
- PBS - heuristic algorithms
- Maui/Silver - submit to local sites, evaluate

## Phase One: Resource Discovery

- Phase One-Resource Discovery

- Phase Three- Job Execution

## 6. Advance Reservation (Optional)

- Reserve resources in a guaranteed way
- Users:
- Call up sys admins and friends (call, on the phone...)
- Ideally:
- Automatically done when you submit a job based on user requirements
- Current systems
- Enabled in PBSPro and Maui

## 7. Job Submission

- Run the job on the resources selected
- User
- Qsub
- Ideally
- “Make it so”
- Current systems
- Each has it’s own API

## 8. Preparation tasks(11. Clean-up tasks)

- File transfers, directory set ups
- Users
- Scp, ftp, mkdir
- Ideally
- Automatically done as part of job submission
- Current systems
- Condor/DagMan can do file staging

## 9. Monitoring Progress

- How is my job doing?
- Should I move it somewhere else?
- Users:
- qstat
- Moving is hard to do, so generally not done
- Ideally:
- System takes care of it based on intuitive knowledge of user requirements, and good prediction techniques
- Current Systems:
- Every LRMS has a stat command

## Summary

## For more information

- Jennifer Schopf
- jms@mcs.anl.gov
- Current document
- 	www.mcs.anl.gov/~jms/Pubs/sched.arch.2002.pdf