# Presentation: 374475

## The D0 Trigger Database Application

**Elizabeth Gallas**
- FNAL CD / Run 2 D0  C&A
**Margherita Vittone,Steve White,Eric Wicklund**
- FNAL CD / CEPA / Database Software (L.Lueking)

**D0 Trigger Database Review**
**May 18, 2005 1-6pm Racetrack**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## TDB Review Web Page Summary

**May 18, 2005 1-6pm Racetrack**
**Review web page summary:**
**www-d0.fnal.gov/~gallas/d0_private/trig/ TDB_Review_0505.html**
**D0 Trigger Database Project Plan:**
**http://d0db-dev.fnal.gov:8514/atrig/TDB_Plan.html**
**last edited September 2004**
**new version after production cut**
**D0 Trigger Database Documentation:**
**Click on Help! in the Report Interface**
**http://d0db-dev.fnal.gov:8514/atrig-dev/cgi/ tdb_report_element.py?function=helpme**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## TDB: Project  Drivers  & Scope

**Purpose:**
**Critical to D0 online operations**
**Create, Modify, Store and Report -- all Run 2 D0 Trigger Configurations**
**Generate configuration format: ‘xml’ -- for online and simulation**
**Document – **
**TDB is the first place collaborators look to get an understanding of the D0 Trigger System**
**Requirements**
**Technical**
**Generate a precise configuration for a complex and evolving ‘device’**
- COOR (Scott Snyder) – does online resource allocation
- many D0 Trigger/Online Groups
**Interface**
**Store/Retrieve configuration information from a Database for a User. **
- Functional, Usable, Documented

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Trigger System Design

**Fast, complex, high rate,...,multi-level **
**Level 1 - electronics and firmware**
**reduce 1 MHz  to 10 kHz by looking for interesting signatures (high Pt tracks, high Et energy deposition)**
**Level 2 - firmware and software**
**10 kHz to 1kHz by refining L1 objects, match objects found by different detectors**
**Level 3 - software**
**1kHz to 50 Hz - execute streamlined versions of offline reconstruction programs to select events.**
**Programmable !**
**through the ‘trigger configuration’ generated from the TrigDB**
**and online resource allocation by COOR**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

**Notes:**
Its not possible to talk about the trigger database without talking first describing the trigger itself

## Trigger Database Design

- _Design_**:**
**Three levels of decision making **
**Level 1 - hardware, firmware**
- NEOTERMs – “Terms in the And/Or Network”
**Level 2 - firmware, software**
- PREPROCESSORs, TOOLs and FILTERs
**Level 3 – software**
- TOOLS and FILTERs
**complexity is a reflection of the complexity of the trigger**
**symmetry/commonality is taken advantage of wherever possible**
**seemingly cryptic nomenclature reflective of trigger programming.  **

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## A Trigger is a Logical Condition

**Identified by a **_**Trigger Name**_
**w/Distinct criteria at Level 1, Level 2, and Level 3 (Script)**
**each of which is satisfied if all of its logical conditions (**_TERMS)_** is satisfied**
**Trigger is satisfied (true) for an event if all 3 Level Scripts are true for that event**

**2EM_HI_Z**

**CEM(2,10)**
**TTK(2,pt3)**
**Afastz**
**2EM(.9,10.,.trk)**

**ele(2,.95,glob,..)**
**InvMass(ele1,ele2,75,100....)**

_**Trigger Name**_

_**Script**_

_**Terms**_

**L1**

**L2**

**L3**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## A Trigger List

**identified by Trigger List Name/Version**
**contains one or more Triggers (Trigger Name)**
**like a tree with Triggers as branches**
**if any trigger is satisfied, the event is recorded and the trigger bit for that trigger name is set to TRUE in the event record**

**global_CMT-?.??**

**2EM_HI**

**EM_MAX**

**3JET_HT**

**MU_JET_HI**

**L1Script**
**L2Script**
**L3Script**

_**Example:**_

**...**
**logical**
**TERMS**
**(yes/no)**
**...**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Trigger Database Design

- Trigger List
- tl_name/tl_version

- Trigger Name
- tn_name/tn_version

- 1
- N

- L1 Script
- l1s_name/l1s_version

- L3 Script
- l3s_name/l3s_version

- L2 Script
- l2s_name/l2s_version

**NEOTERM**
- And/Or Term

- 1
- N1

- 1

- 1

- 1

- 1

- 1

- 1

- L2 Filter
- t_name/t_version

- 1
- N2

- L3 Filter
- t_name/t_version

- 1
- N3

- NAME/VERSION scheme is repeated throughout the design.
- The name is intended to reflect the conditions in that definition

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Trigger Database Design

- Trigger List
- tl_name/tl_version

- Trigger Name
- tn_name/tn_version

- 1
- N

- L1 Script
- l1s_name/l1s_version

- L3 Script
- l3s_name/l3s_version

- L2 Script
- l2s_name/l2s_version

**NEOTERM**
- And/Or Term

- 1
- N1

- 1

- 1

- 1

- 1

- 1

- 1

- L2 Filter ‘Term’
- t_name/t_version

- 1
- N2

- L2 Tool ‘Term’
- t_name/t_version

- 1

- 1

- N2f

- N2t

- 1

- N2ft

**L2 Object**
- O_name/O_version
- (set of parameters,
- types, defaults,
- min, max)

- Based on

- Based on

**‘****Release Version’**
- P15.02.00 or **pseudoversion**

- L2 Preprocessor
- pp_name/pp_version

- 1

- Npp

- L3: same as L2
- without preprocessors

- A ‘Term’ is a tool or filter with a distinct set of parameter = value pairs

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Trigger Database ER Diagram

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Trigger Database Application Components

- D0 offline production database instance (d0ofprd1)
- About 40 Tables in Oracle
- About 25 Views in Oracle
- Documentation
- Web based html
- TDB Report Interface.
_**Web based interfaces:**_
_**TDB Entry Interface                   (about 13 modules)  ****_
_**TDB Report Interface                 (about 14 modules)**_
- TDB MISWEB Interface
_**Trigger Db Server**_
_**Like the SAM db servers –           **_
_**a customized db server**_

- Command line interfaces:
- DeleteTemporaryElements.py**
- CheckStatCurr.py		  **
- CheckStatUsed.py 	 	  **
- clientDemo.py  		  **
- add_user.py
- rcpGen.py  	 	  **
- set_statUsed.py
_**xmlgen.py **_
- GetBitNames.py
- GetRuns.py
- GetStuff3.py
- l3tooltype.py
- streamXmlGen.py
- ** - interfaces using the TriggerDbServer

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Trigger Database Web Interfaces

**http://www-d0.fnal.gov/trigger_meister/private/trigdb/**

_**Trigger Database Web Interfaces**_

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Trigger Database Entry Interface

- Entry Interface:**  **
**Used by experts to enter data **
**Has URL links into the Reporting Interface**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Trigger Database Report Interface

**python_dcoracle**
**More flexible queries**
**Multi-dimensional queries**
**Queries by date**
**Queries by user**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

##  Simple TriggerList in ‘xml’

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Anyone can generate TriggerList ‘xml’

**Create a Trigger List in the database**
**specify Trigger List name / version**
**On any D0 unix or Linux system:**
**setup d0cvs**
**cvs co trigdb_xmlclient**
**cd trigdb_xmlclient**
**gmake**
**source bin/xmlclient_setup**
**Run the program with desired options**
**For help:**
**xmlgen.py  (no arguments for help)**
**global_CMT-12.20 for ONLINE:**
**xmlgen.py -tlname global_CMT -tlversion 12.20  -file -OneStream all**
**global_CMT-12.20 for Simulation:**
**xmlgen.py -tlname global_CMT -tlversion 12.20  -file -Sim**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## xmlgen.py -h

**MANDATORY INPUT (wildcard %):**
** ****xmlgen.py --tlist-name listname --tlist-version version**
**OPTIONAL SETTINGS (the first 2 are most often used for online lists):**
** ****--OneStream all (to write all events to one stream like 'all')**
** ****--file (writes the xml to file named listname-version.xml)**
** ****--debug 0 (0 for debug mode (all levels), 1 for L1, 2 for L2, 3 for L3)**** ****(put this argument first if you want other input arguments reported)**
** ****--Sim**
** ****Chooses set of options typicalin offline simulation: including **
- --OneStream all
- --GetCrates
-  --SRDirective useL1=no uses L3 tools script SRtools_sim/1 for L3 instantiation of the L3 error handling tool to write a logfile (port 0) called testfile1with typical simulation use file and stats thresholds does not include &smt_monitoring; (inserted for all online xml)
**--UniqueL1L2 (generate unique L1/L2 names for all triggers, even if they share L1/L2 conditions)**
**--PrescaleFile (writes a default prescale file named listname-version-default.prescales)**
**--realNames (SR parser cannot handle realNames so use this option for testing only)**
**--NumNodes (number of l3 nodes to be used, overriding value in database (usually 0))**
**--SRDirective useL1=yes,monitorinfo=10,sendmoninfo=yes (is the current default)(enter a comma separated list of directives needed at top of the <triglist>)**
**--GetCrates (will generate real cratelists rather than use allcrates_readout.xml)**
**--Database (default is 'd0ofprd1')**
**--help (this help)**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Project Deliverables Status

**IN USE since December 2001**
**all Global (Physics) and nearly all Special Runs **
**a number of Commissioning and Calibration configurations as functionality of trigger systems come online**
**Contains about 100,000 records**
**364 Trigger Lists (nearly 4000 Triggers defined)**
**single List can contain 1000s of parameters**
**>250 Trigger Lists used online in >20000 Physics Runs**
**All data is entered by people (10 certified)**
**Trigger Lists are a unique combination of conditions which are designed individually to configure a complex system**
**Nearly no duplicate records**
**Once a trigger list element is created, other trigger lists can use it**
**not large by database standards**
**but implementation is complex, reflective of programmability of the D0 Trigger System**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Project  Status

**TDB Application is critical to the online operation of D0**
**TDB meets some but not all of the  requirements for its current capabilities**
**The ‘NOT ALL’ part wastes a considerable amount of expert time**
**The system needs to be expanded**
**D0 Trigger Steering Committee**
**The system cannot be expanded until its deficiencies for current capabilities are addressed.**
**Why: same expert is involved; doing so will exacerbate the existing problem**
**Need help to bring the project up to the design specifications**
**Defined CD Project:**
- http://www-d0.fnal.gov/d0dist/dist/packages/trigdb_userweb/devel/www/TDB_Plan.html

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Effort Profile (Past and Current)

**People who have worked on this project (FNAL CD):**
**Elizabeth Gallas, Vicky White, Jeremy Simmons, Matt Vranicar, Rich Wellner, John Weigand, Vijay Murthi, Steve White, Adam Lyon.**
**People currently working on the project (FNAL CD):**
**Elizabeth Gallas**
**Margherita Vittone – web Entry Interface (client using db server)**
**Steve White – trigger db server and client interface to server**
**Eric Wickland – web Report Interface (client using dcoracle)**
**List of tasks (next  3 slides)**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## List of Tasks (1): Tasks to fix/implement Entry/Report client and DbServer

**Project Plan – A lot of progress **
**TriggerDbServer - convert from fnorb to omniorb, clean up**
**Steve While**
**Entry Client: trigdb_status, l1dialogs, neoterms**
**Margherita Vittone**
**Report Client: tdb_objects, terms, L1dialogs, neoterms**
**Eric Wicklund**
**Project Plan – Next items in the schedule**
**tdb_scripts, tn, tl, tldependency**
**trigdb_objects, terms, scripts, triggernames, triggerlists**
**trigdb_tltransformation (expert required)**
**t(rig)db_L2pp, t(rig)db_ed, dg**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## List of Tasks (2): Tasks for Maintenance, Integrity and New Functions

**As needed items:**
**Enforcing system constraints**
**more checks insure correct configuration**
**Performance Issues**
**needed**
**Machine Issues**
**on order for Luminosity Database**
**Clean up program**
**2-4 weeks (implementation dependent)**
**Changes to Level 1 Exposure Group Rules**
**2-4 weeks**
**Level 1 Pseudoterm Implementation**
**2-4 months**
**Trigger List to Release correspondence**
**4 weeks**
**Run 2b Changes**
**specification dependent (Vancouver)**
**Level 1 subdetector version tracking**
**specification dependent**
**Additional COOR functionality**
**specification dependent**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## List of Tasks (3): Some Requests since September 2004

**L2 Splitting**
**L2 ORing**
**L2 Calorimeter Calibration**
**L3 ScriptRunner Directives**
**Round Robin Streaming**
**L1 Calorimeter Region and Quadrant Terms**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Risks, Problems and Related Issues

**For Trigger DB:  Requests for changes are constant**
**many requests are valid and worth pursuing (physics)**
**many have been implemented**
**requires thoughtful analysis and specification of requests **
**assess extent of changes to existing TrigDB implementation and how it fits into the existing schedule, backward compatibility**
**assess extent of changes to other online systems, communication**
**many requests not mitigated by D0 management, TB (incoherent)**
**source of constant distraction**
**some guidelines for limiting the system should be in place**
**must understand the cost of implementation on all affected systems**
**Note: managing hundreds (thousands?) of triggers is not trivial**
**Not just for Trigger DB, but Online and Luminosity as well.**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Notes in Summary (1)

**Trigger DB is **
**a complex application**
**a working application**
**Note: At no point has anyone worked on this application full time and at times no one has been working on it **
**manpower shortfall caused us to fall behind the needs of the experiment**
**We are catching up thanks to help (CD CEPA Group) **
**Margherita, Steve and Eric – CEPA group (Lee Lueking)**
**kick start:  have contributed hundreds of lines of code**
**satisfying specified goals of the project plan**
**Making the production cut is top priority**
**Aside:  Catching up also helps in case of failure**
**more people knowledgeable about application**
**After cut: can move forward in Trigger Project Plan**
**some of the next tasks are more complex (before new features)**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## Notes in Summary (2)

**Challenge to D****Ø****  (falls upon THIS REVIEW) **
**understand the complexity of the overall system**
**COOR and online resources, TriggerDb, RunS DB, LumDB**
**For the Trigger DB**
**Organize the needed complexity into fixed columns in tables**
**Writing Entry interfaces which allow experts to enter the trigger programming needed in a reasonable time for hundreds of triggers**
- Prevent disasters (closed Entry system) -- accidentally modifying, deleting trigger lists in development or used online
**Writing  Report Interfaces which make the information needed available**
**For new features:**
**clearly and consistently state the needs and priorities**
**clearly specify the details needed for implementation (in all systems)**
- example: Round Robin Streaming
**New Manpower ?**
**organize, coordinate, WRITE, TEST, implement**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005

## What is the Trigger Db NOT?

- The Trigger Database is not designed to know about:
**Runs **
**Stores**
**Magnet settings**
**alignment**
**calibration**
**release version installed in L2 or L3**
**(but may know about release compatibility)**
**time**
- Why not ?
**These other aspects of a Run are recorded in other databases or using other methods**
**The trigger database is an offline database**
**including ‘real time’ information would be an expansion in scope of the project**

- D0 Trigger Database

- Elizabeth Gallas              May 18, 2005