# Presentation: 366768

## Transitioning to Gold

- Scott Jackson
- Pacific Northwest National Laboratory
- 10 DEC 2003

## High Level differences

| Feature | QBank | Gold |
| --- | --- | --- |
| Dynamic charging | Yes | Yes |
| Reservations | Yes | Yes |
| Expiration Cycles | Yes | Yes |
| Tracks composite resources (Processors, Memory, Disk, ...) | Yes | Yes |
| Applies other charge factors (QOS, Nodetype, etc.) | Yes | Yes |

## High Level differences

| Feature | QBank | Gold |
| --- | --- | --- |
| Default Projects | Yes | Yes |
| Credit and Debit 
allocation types (& overdraft) | Yes | Yes |
| Charge Quotes | Yes – cheating possible | Yes -- guaranteed |
| Preserves Transaction History | Yes | Yes |
| Preserves Historical State | Periodically via Checkpoints | Yes via Journaling |

## High Level differences

| Feature | QBank | Gold |
| --- | --- | --- |
| Undelete objects | No | Yes |
| Extensible and dynamic objects (custom accounting) | No | Yes |
| Strong Authentication and Integrity Checking | Yes – secret key checksum | Yes – HMAC-SHA1 or password (for web GUI) |
| Strong Encryption | No | Yes – 3DES or password (for web GUI) |
| Authorization | Multiple levels (None, User, Admin, Bank) plus custom. | Instance-level RBAC w/ special instance types (SELF, ANY, MEMBER, ADMIN) plus custom. |

## High Level differences

| Feature | QBank | Gold |
| --- | --- | --- |
| Web Interface | No | Yes – in development |
| Hierarchical Nested Projects | No | Yes |
| Multi-Site Peer Exchange | Experimental support | In future plan |
| Documentation | Self-documented clients. Some existing documentation {Getting Started Guide, Technical Paper}. | Self documented clients. Other docs {User Guide, Admin Guide, Technical Paper} in future plan. |

## High Level differences

| Feature | QBank | Gold |
| --- | --- | --- |
| Projects per allocation | Tied to 1 specified project  | Shared by 1 or more specified projects or NONE, DEFINED, SPECIFIED, ANY with negation. |
| Users per allocation | Distributed among 1 or more specified users or KITTY, RESERVE or ANY. Amounts are not shared between individually specified users.  | Shared by 1 or more specified users or NONE, MEMBER, DEFINED, SPECIFIED, ANY with negation. |
| Machines per allocation | Shared by 1 or more specified machines or ANY  | Shared by 1 or more specified machines or NONE, MEMBER, DEFINED, SPECIFIED, ANY with negation. |

## Allocations

## Gold Interfaces

- Web GUI
- Management view (in development)
- Object view (mostly complete)
- Explosion view (in future plan)
- Command line
- Perl clients
- Gold – interactive perl low-level object interface
- Goldc – interactive java low-level object interface
- Application Programming Interfaces
- Java API
- Perl API

## Topics for Training

- What features do I want to highlight?
- New allocation paradigm
- Hierarchical nested projects
- Credit and Debit allocations
- Security
- Developing web gui
- Command line clients
- Dynamic nature of objects
- Journaling (time travel, undo)
- Query language – show how complex actions can be performed
- Roles

## Topics for Training

- Under-used existing features?
- Deposit shares
- Default accounts
- Charge factors (composite, QOS, NodeType, Machine, etc.)
- Quotes

## Topics for Training

- What can you do with Gold you can’t do with QBank?
- What can you do with QBank you can’t do with Gold?
- The divisions between users is more visible and comprehensible.
- Discuss difference of how allocations are constructed and the implications on deployment strategies.
- Discuss what’s going to happen during deposit ...
- How to get help – command line usage, method usage, support queue

## The following slides are included as an appendix of reference material

## Introduction

- In a nutshell, Gold is:
- A resource bank (allocation management system)
- tracks and manages resource usage. Much like a bank, it associates a cost to computing resources and allows resource credits to be allocated to users and projects. As jobs complete or as resources are utilized, projects are dynamically charged and resource usage recorded.
- An accounting system
- Can be dynamically customized to record any type of accounting data – pacct, sar, node availability, etc.
- An information service
- Also functions as a powerful generalized information service useful in a variety of means, such as providing mappings for meta-scheduling mappings of machines to resources, applications, accounts, users, etc.

## But First, A Little Background...

- Scalable Systems Software Center
- Research, develop and support an integrated suite of systems software and tools for the effective management and utilization of the highest scale computational resources.
- SciDAC
- Scientific Discovery through Advanced Computing – A DOE initiative to improve the impact of scientific computing
- QBank
- A dynamic allocation management system developed at PNNL and in use at about a dozen sites.

## Motivation

- Show return on investment
- Funding sources have invested heavily in a supercomputer and require a means to show that it is being utilized efficiently.
- Fairness
- Management needs a means to fairly distribute the underlying computing resources (processors, memory, disk) to the various users and projects.
- Capacity Planning
- Accurate allocation and usage information is needed to effectively make decisions on resource commitments and new procurements.
- Centralized Access Control
- Many organizations want centralized control over which users and projects have access to what machines and for how long.

## Motivation (from a meta-scheduling slant)

- Meta-computing
- Trust issues
- Guarantees
- Distributed Accounting
- Local Control
- Equitable trade agreement
- Security
- Different userids, accounts, execution environments on different resources

## Nonfunctional Requirements

- Scalable
- Targeting systems with tens of thousands of processors and thousands of simultaneous jobs.
- Secure
- Will use strong authentication (no clear-text passwords) to prevent unauthorized access and data encryption to prevent sensitive information from being intercepted (XML-DSIG and XML-ENC).
- Fault Tolerant
- Database performs automatic rollbacks on failed transactions. A distributed design that includes data replication will be researched.

## Nonfunctional Requirements (cont.)

- Open Source
- Allows free distribution, allows sites to make local modifications, and derived works, and promotes sharing of patches, ports and enhancements from user community.
- Portable
- Written in Java – initially tested to a reference Linux platform and expanded to include architectures used at the largest DOE computing facilities.
- Easy to Use
- Web accessible GUI (based on PHP and Javascript) will help managers, users and admins gain the access they need from their own PC’s.

## Operational Characteristics

- Supports familiar bank operations
- Deposits, withdrawals, transfers, refunds, balance checks and bank statements
- Reservations
- Before a job runs, a reservation (or hold) is placed on the account based on the wallclock limit. This prevents overdrafts.
- Quotations
- In a meta-scheduling environment it is useful to know how much a job is going to “cost” so that you can make a decision on the best place to run your job.
- Hierarchical Accounts
- Projects can be nested (trickle down deposits, trickle up withdrawals)

## Dynamic Resource Management Interaction

-    Make Deposits, etc.
-    Submit Job
-    Balance Check
-    Make Reservation
-    Start Job
-    Job Completes
-    Remove Reservation & Make Withdrawal

## Dynamic Resource Management Interaction(with meta-scheduling)

**Resource Manager**
**(PBS, LL)**

**Allocation Manager**
**(Gold)**

- 0

- 2

- 1

- 5

- 3

- 3

- 7

-    Make Deposits, etc.
-    Submit Job
-    Locate Feasible Systems & Obtain Quote
-    Stage Job
-    Balance Check
-    Make Reservation
-    Start Job
-    Job Completes
-    Remove Reservation & Make Withdrawal

**Meta-Scheduler**
**(Silver)**

**Scheduler**
**(Maui)**

- 6

- 8

- 4

## Allocations

- An allocation is a collection of resource credits valid toward an arbitrary group of users, machines and projects and a timeframe for expenditure.
- Commonly associated with a single project (account), and a set of users and machines.
- Fine-grained control of who can use how much within a project can be achieved by multiple allocations in the same project.

## The dimensions of Allocation Management

_**Projects**_
- Grand Challenge
- Development
- Weather Modeling
- Chem101
- Navy
- SETI
- Viz
- ...

_**Users**_
- Tom
- Sheri
- Scientist
- Developer
- Admin
- Workshop
- Manager
- ...

- _Resources_

- _Time_

- DOE

- PNNL

- LLNL

- SDSC

- ANL

- MPP1

- Colony

- Jupiter

## Allocation-User Distribution Possibilities

## Allocation-Machine Distribution Possibilities

## Allocation Timeframes

## Allocation Timeframes

## Allocation Timeframes

## Allocation Timeframes

## Allocation Timeframes

## Journaling

- State Preservation
- Preserves indefinite historical state of all objects and records
- Bank Statements
- Journaling allows bank statements to show balances for any arbitrary time in the past
- Undo/Redo
- With a powerful querying/updating comes the potential for rampant administrative mistakes
- Time Travel
- You can run any command as if it were an arbitrary date in the past

## Flexible Charging Mechanism

- Besides CPU, a resource supplier can charge based on the amount of memory, disk, or any other consumable resource as well as quality of service, primetime, nodetype, class, etc.
- An external pricing engine interface will allow any sort of charging algorithm to be used such as dynamic price adjustment according to load or queue backlog, a query to an external information service or a cached second-price auction result.

## Traceback Mechanism

- Allows all parties of a transaction (resource requestor and provider) to have a record of the resource utilization and to have a say as to whether or not the job should be permitted to run, based on their independent policies and priorities. A job will only run if all parties are agreeable to the idea that the target resources can be used in the manner and amount requested.

- MetaLBNL

- runPNNL

**Meta**
**Account**

**PNNL**

**LBNL**

- Traceback debit

## Flexible and Extensible

- Powerful Querying/Updating Capabilities
- Create, query, modify, delete, undelete
- Support for operators (equals, less than, not equal, matching, etc.)
- Conjunctive expression combinations (and, or)
- Object joined queries
- Dynamically Extensible
- New object/record types and their fields can be dynamically created/modified through the regular query language (command line or GUI). This capability turns this system into a *generalized information service. *This capability is extremely powerful and can be used for meta-scheduling resource-mapping, an interface for persistence for other components, and all varieties of accounting possibilities!

## Schedule

- 4Q02	Requirements gathering completed and release initial Resource Management Interface Specs
- 2Q03	QBank bundled with SSS initial release
- 4Q03	Pre-alpha release of Gold
- 1Q04	Alpha release of Gold
- 3Q04	Beta release of Gold
- 4Q05	General release of Gold (includes support)

## Contact Information

- Scott Jackson
- Pacific Northwest National Laboratory
- _[Scott.Jackson@pnl.](mailto:Scott.Jackson@pnl.gov)__[gov](mailto:Scott.Jackson@pnl.gov)_
- (509) 376-2205
- Scalable Systems Software Center
- _[http://www.scidac.org/ScalableSystems](http://www.scidac.org/ScalableSystems)_
- SSS Resource Management and Accounting Working Group
- _[http://sss.scl.ameslab.gov/home.](http://sss.scl.ameslab.gov/home.shtml)__[shtml](http://sss.scl.ameslab.gov/home.shtml)_