# Presentation: 474786

## STAR fileCatalog, tagDBand Grand Challenge Architecture

- A. Vaniachine
**presenting for the Grand Challenge Collaboration**
**(**_**[http:/www-](http://www-rnc.lbl.gov/GC/)**__**[rnc](http://www-rnc.lbl.gov/GC/)**__**[.](http://www-rnc.lbl.gov/GC/)**__**[lbl](http://www-rnc.lbl.gov/GC/)**__**[.](http://www-rnc.lbl.gov/GC/)**__**[gov](http://www-rnc.lbl.gov/GC/)**__**[/GC/](http://www-rnc.lbl.gov/GC/)**_**)**
**April 9, 2000**
**Joint ALICE-STAR Computing Meeting**

## Outline

**GCA Overview**
**STAR Interface:**
***fileCatalog***
***tagDB***
***StChallenger***
**Current Status**
**Conclusion**

## GCA: Grand Challenge Architecture

***An order-optimized prefetch architecture for data retrieval from multilevel storage in a multiuser environment***
**Queries select events and specific event components based upon tag attribute ranges**
**query estimates are provided prior to execution**
**collections as queries are also supported**
**Because event components are distributed over several files, processing an event requires delivery of a “bundle” of files **
**Events are delivered in an order that takes advantage of what is already on disk, and multiuser policy-based prefetching of further data from tertiary storage**
**GCA intercomponent communication is CORBA-based, but physicists are shielded from this layer**

## Participants

**NERSC/Berkeley Lab****	**
**L. Bernardo, A. Mueller, H. Nordberg, A.Shoshani, A. Sim, J. Wu**
**Argonne**
**D. Malon, E. May, G. Pandola**
**Brookhaven Lab**
**B. Gibbard, S. Johnson, J. Porter, T.Wenaus **
**Nuclear Science/Berkeley Lab**
**D. Olson, A. Vaniachine, J. Yang, D.Zimmerman**

## Problem

**There are several**
**Not all data fits on disk ($$)**
**Part of 1 year’s DST’s fit on disk**
**What about last year, 2 year’s ago?**
**What about hits, raw?**
**Available disk bandwidth means data read into memory must be efficiently used ($$)**
**don’t read unused portions of the event**
**Don’t read events you don’t need**
**Available tape bandwidth means files read from tape must be shared by many users, files should not contain unused bytes ($$$$)**
**Facility resources are sufficient only if used efficiently**
**Should operate steady-state (nearly) fully loaded**

## Bottleneks

- Keep recently accessed data on disk, but manage itso unused data does notwaste space.
- Try to arrangethat 90% of fileaccess is to diskand only 10%are retrievedfrom tape.

## Solution Components

**Split event into components across different files so that most bytes read are used**
**Raw, tracks, hits, tags, summary, trigger, ...**
**Optimize file size so tape bandwidth is not wasted**
**1GB files, ******** means different # of events in each file**
**Coordinate file usage so tape access is shared**
**Users select all files at once**
**System optimizes retrieval and order of processing**
**Use disk space & bandwidth efficiently**
**Operate disk as cache in front of tape**

## STAR Event Model

- T. Ullrich, Jan. 2000

## Analysis of Events

**1M events = 100GB – 1TB**
**100 – 1000 files (or more if not optimized)**
**Need to coordinate event associations across files**
**Probably have filtered some % of events**
**Suppose 25% failed cuts after trigger selection**
**Increase speed by not reading these 25%**
**Run several batch jobs for same analysis in parallel to increase throughput**
**Start processing with files already on disk without waiting for staging from HPSS**

## GCA System Overview

- GCA
- STACS

- Staged
- event
- files

*Event*
*Tags*

**(Other)**
**disk-resident**
**event data**

- Index

- HPSS

- pftp

*fileCatalog*

## STACS: STorage Access Coordination System

## Organization of Events in Files

**Organization of Events in Files**

- Files

## The Details 

**Range-query language, or query by event list**
**“****NLa>700 && run=101007”, **
**{e1,r27012;e3,r27014;e7;r27017 ...}**
**Select components: dst, geant, ...**
**Query estimation**
**# events, # files, # files on disk, how long, ...**
**Avoid executing incorrect queries**
**Order optimization**
**Order of events you get maximizes file sharing and minimizes reads from HPSS**
**Policies**
**# of pre-fetch, # queries/user, # active pftp connections, ...**
**Tune behavior & performance**
**Parallel processing**
**Submitting same query token in several jobs will cause each job to process part of that query**

## Interfacing GCA to STAR

**Interfacing GCA to STAR**

**GC System**

**StIOMaker**

**fileCatalog**

**tagDB**

**Query**
**Monitor**

**Cache**
**Manager**

**Query**
**Estimator**

**STAR Software**

**Index**
**Builder**

## Limiting Dependencies

**STAR-specific**

**IndexFeeder server**
**IndexFeeder read the “tag database” so that GCA “index builder” can create index **
**FileCatalog server**
**FileCatalog queries the “file catalog” database of the experiment to translate fileID to HPSS & disk path**

## Eliminating Dependencies

**StIOMaker**

**ROOT + STAR Software**

**CORBA + GCA software**

## STAR fileCatalog

**Database of information for files in experiment.****	****File information is added to DB as files are created.**
**Source of File information **
** ****for the experiment**
** ****for the GCA components (Index, gcaClient,...)**

## Cataloguing Analysis Workflow

**Cataloguing Analysis Workflow**

- fileCatalog

- Job configuration manager

## Transactionless Solution

**MySQL:**
**no views **
- reader has to join tables
**no transactions**
- db snapshot during the update may be inconsistent
- updates may be long (= no table locking for read)
- Experiment: few writers, hundreds of readers
- Compromise:
**use pre-calculated views (= extra tables)**
**fileCatalog data are duplicated**
**views update is quick (table can be locked)**
**clients read see consistent data at any time**

## Problem:	SELECT NLa>700

**ntuple**

## STAR Tag Structure Definition

**STAR Tag Structure Definition**

**Selections like**
******qxa2+qxb2 > 0.5**
**can not use index**

## STAR Tag Database Access

**STAR Tag Database Access**

## TagDB in ROOT Files

**Tag data are stored in ROOT TTree files**
**Branches (3 out of 6 requested) saved in the split mode**
- StrangeTag
- FlowTag
- ScaTag
**173 physics tags [int/float] out of 500 requested**
**Disk resident tag files name+path are stored in the MySQL *****fileCatalog *****database**
**Files are selected from database in the ROOT CINT macro through the ROOT-MySQL interface and are chained for further selections by user **

## MDC3 Index

**6 event components:**
**179 physics tags:**
**120K events**
**8K files**

**fzd**
**geant**
**dst**
**tags**
**runco**
**hist**

**StrangeTag**
**FlowTag**
**ScaTag**

## GCA MDC3 Integration Workshop 

**http://www-rnc.lbl.gov/GC/meetings/14mar00/default.htm**

**14-15 March 2000**

## User Query

**ROOT Session:**

## Conclusion

**GCA developed a system for optimized access to multi-component event data files stored in HPSS**
**General CORBA interfaces are defined for interfacing with the experiment**
**A client component encapsulates interaction with the servers and provides an ODMG-style iterator**
**Has been tested up to 10M events, 7 event components, 250 concurrent queries**
**Has been integrated with the STAR experiment ROOT-based I/O analysis system**