# Presentation: 261106

## Gold Accounting Manager

- Scott Jackson
- Scalable Systems Software Center
- SC2002
- 20 NOV 2002

## Introduction

- In a nutshell, Gold is:
- A resource bank (allocation management system)
- tracks and manages resource usage. Much like a bank, it associates a cost to computing resources and allows resource credits to be allocated to users and projects. As jobs complete or as resources are utilized, projects are dynamically charged and resource usage recorded.
- An accounting system
- Can be dynamically customized to record any type of accounting data – pacct, sar, node availability, etc.
- An information service
- Also functions as a powerful generalized information service useful in a variety of means, such as providing mappings for meta-scheduling mappings of machines to resources, applications, accounts, users, etc.

## But First, A Little Background...

- Scalable Systems Software Center
- Research, develop and support an integrated suite of systems software and tools for the effective management and utilization of the highest scale computational resources.
- SciDAC
- Scientific Discovery through Advanced Computing – A DOE initiative to improve the impact of scientific computing
- QBank
- A dynamic allocation management system developed at PNNL and in use at about a dozen sites.

## Motivation

- Show return on investment
- Funding sources have invested heavily in a supercomputer and require a means to show that it is being utilized efficiently.
- Fairness
- Management needs a means to fairly distribute the underlying computing resources (processors, memory, disk) to the various users and projects.
- Capacity Planning
- Accurate allocation and usage information is needed to effectively make decisions on resource commitments and new procurements.
- Centralized Access Control
- Many organizations want centralized control over which users and projects have access to what machines and for how long.

## Motivation (from a meta-scheduling slant)

- Meta-computing
- Trust issues
- Guarantees
- Distributed Accounting
- Local Control
- Equitable trade agreement
- Security
- Different userids, accounts, execution environments on different resources

## Nonfunctional Requirements

- Scalable
- Targeting systems with tens of thousands of processors and thousands of simultaneous jobs.
- Secure
- Will use strong authentication (no clear-text passwords) to prevent unauthorized access and data encryption to prevent sensitive information from being intercepted (XML-DSIG and XML-ENC).
- Fault Tolerant
- Database performs automatic rollbacks on failed transactions. A distributed design that includes data replication will be researched.

## Nonfunctional Requirements (cont.)

- Open Source
- Allows free distribution, allows sites to make local modifications, and derived works, and promotes sharing of patches, ports and enhancements from user community.
- Portable
- Written in Java – initially tested to a reference Linux platform and expanded to include architectures used at the largest DOE computing facilities.
- Easy to Use
- Web accessible GUI (based on PHP and Javascript) will help managers, users and admins gain the access they need from their own PC’s.

## Operational Characteristics

- Supports familiar bank operations
- Deposits, withdrawals, transfers, refunds, balance checks and bank statements
- Reservations
- Before a job runs, a reservation (or hold) is placed on the account based on the wallclock limit. This prevents overdrafts.
- Quotations
- In a meta-scheduling environment it is useful to know how much a job is going to “cost” so that you can make a decision on the best place to run your job.
- Hierarchical Accounts
- Projects can be nested (trickle down deposits, trickle up withdrawals)

## Dynamic Resource Management Interaction

-    Make Deposits, etc.
-    Submit Job
-    Balance Check
-    Make Reservation
-    Start Job
-    Job Completes
-    Remove Reservation & Make Withdrawal

## Dynamic Resource Management Interaction(with meta-scheduling)

**Resource Manager**
**(PBS, LL)**

**Allocation Manager**
**(Gold)**

- 0

- 2

- 1

- 5

- 3

- 3

- 7

-    Make Deposits, etc.
-    Submit Job
-    Locate Feasible Systems & Obtain Quote
-    Stage Job
-    Balance Check
-    Make Reservation
-    Start Job
-    Job Completes
-    Remove Reservation & Make Withdrawal

**Meta-Scheduler**
**(Silver)**

**Scheduler**
**(Maui)**

- 6

- 8

- 4

## Allocations

- An allocation is a collection of resource credits valid toward an arbitrary group of users, machines and projects and a timeframe for expenditure.
- Commonly associated with a single project (account), and a set of users and machines.
- Fine-grained control of who can use how much within a project can be achieved by multiple allocations in the same project.

## The dimensions of Allocation Management

_**Projects**_
- Grand Challenge
- Development
- Weather Modeling
- Chem101
- Navy
- SETI
- Viz
- ...

_**Users**_
- Tom
- Sheri
- Scientist
- Developer
- Admin
- Workshop
- Manager
- ...

- _Resources_

- _Time_

- DOE

- PNNL

- LLNL

- SDSC

- ANL

- MPP1

- Colony

- Jupiter

## Allocation-User Distribution Possibilities

## Allocation-Machine Distribution Possibilities

## Allocation Timeframes

## Allocation Timeframes

## Allocation Timeframes

## Allocation Timeframes

## Allocation Timeframes

## Journaling

- State Preservation
- Preserves indefinite historical state of all objects and records
- Bank Statements
- Journaling allows bank statements to show balances for any arbitrary time in the past
- Undo/Redo
- With a powerful querying/updating comes the potential for rampant administrative mistakes
- Time Travel
- You can run any command as if it were an arbitrary date in the past

## Flexible Charging Mechanism

- Besides CPU, a resource supplier can charge based on the amount of memory, disk, or any other consumable resource as well as quality of service, primetime, nodetype, class, etc.
- An external pricing engine interface will allow any sort of charging algorithm to be used such as dynamic price adjustment according to load or queue backlog, a query to an external information service or a cached second-price auction result.

## Traceback Mechanism

- Allows all parties of a transaction (resource requestor and provider) to have a record of the resource utilization and to have a say as to whether or not the job should be permitted to run, based on their independent policies and priorities. A job will only run if all parties are agreeable to the idea that the target resources can be used in the manner and amount requested.

- MetaLBNL

- runPNNL

**Meta**
**Account**

**PNNL**

**LBNL**

- Traceback debit

## Flexible and Extensible

- Powerful Querying/Updating Capabilities
- Create, query, modify, delete, undelete
- Support for operators (equals, less than, not equal, matching, etc.)
- Conjunctive expression combinations (and, or)
- Object joined queries
- Dynamically Extensible
- New object/record types and their fields can be dynamically created/modified through the regular query language (command line or GUI). This capability turns this system into a *generalized information service. *This capability is extremely powerful and can be used for meta-scheduling resource-mapping, an interface for persistence for other components, and all varieties of accounting possibilities!

## Schedule

- 4Q02	Requirements gathering completed and release initial Resource Management Interface Specs
- 2Q03	QBank bundled with SSS initial release (possible alpha-testing on Gold)
- 2Q04	Beta release of Gold
- 4Q05	Production release of Gold (includes support)

## Contact Information

- Scott Jackson
- Pacific Northwest National Laboratory
- _[Scott.Jackson@pnl.](mailto:Scott.Jackson@pnl.gov)__[gov](mailto:Scott.Jackson@pnl.gov)_
- (509) 376-2205
- Scalable Systems Software Center
- _[http://www.scidac.org/ScalableSystems](http://www.scidac.org/ScalableSystems)_
- QBank documentation and download
- _[http://www.](http://www.emsl.pnl.gov:2080/docs/mscf/qbank)__[emsl](http://www.emsl.pnl.gov:2080/docs/mscf/qbank)__[.](http://www.emsl.pnl.gov:2080/docs/mscf/qbank)__[pnl](http://www.emsl.pnl.gov:2080/docs/mscf/qbank)__[.](http://www.emsl.pnl.gov:2080/docs/mscf/qbank)__[gov](http://www.emsl.pnl.gov:2080/docs/mscf/qbank)__[:2080/docs/](http://www.emsl.pnl.gov:2080/docs/mscf/qbank)__[mscf](http://www.emsl.pnl.gov:2080/docs/mscf/qbank)__[/](http://www.emsl.pnl.gov:2080/docs/mscf/qbank)__[qbank](http://www.emsl.pnl.gov:2080/docs/mscf/qbank)_