# Presentation: 299356

## D0 Level 3 filters

- Dan Claes                      Moacyr Souza
*U. of Nebraska *             *Fermilab** / **Lafex - Rio de Janeiro*

*Dzero Collaboration Meeting*
*      **November 9th, 2001*

##  status of ScriptRunner

-  status of ScriptRunner
-  status of Filters/Tools
- status of L3 Monitoring

## Build/append/delete pieces of the execution tree, on 

-       demand.
-         status: working properly
- Do the Filter Dispatching
-        -Execution tree implemented as a *linked list pattern*
-       -Intended behavior : never hear about!
-          1) No logic involved
-          2) There is only *one* way to go : otherwise -> crash
-               This is how robustness was insured
-            status: No complaints!  (never heart about!)
- Gather status & statistics and send Monitoring data
-       to servers  : I will be taking about at the end

-  Digest the coor commands and
-    act according
-      status: working properly after some
-      re-arrangements with L3 Frame

## Slide 4

##  SR duties (other than make the filters dispatching) 

-  Fill the L3Chunk
-    For each filter, record
-        -  status :          passed/failed     (and called)
-        - Unbiased :    L3Unbiased
-                                  L2Unbiased
-                                  “forced_unbiased”
-        - L3prescaled
-        - others
-  Fill the Debug Chunk (on demand)
-       - can vary
-  Fill the Event_Header
-    For each event, record :
-         - L1 active && fired bit mask
-         - L2 active && fired bit mask
-         - L3 active && fired bit mask
-         - Streaming information
-         - run #, event #, Luminosity Block, data_size, ...

##  Send monitoring information at a given heart beat

-  Send monitoring information at a given heart beat
-    For each filter, record
-         - Rates :        # called ,   # passed
-         - Special events :        # L3Unbiased
-                                               # prescaled
-         - Others : Histogram of the overall filtering time/per event
-                            ...
-  and time to time,  send the Tools’ monitoring data (l3fstatmanager):
-             per Tool :        # of candidates
-                                      Timing
-                                      Correlations, others ...
-  Send monitoring information  per Luminosity block
-   For each L2 bit:
-       - # events received
-     - corresponding L2 bit number
-     - L3 bit names   (l3scripts)
-     - L3 prescale factors
-     - # events passed, per filter
-     - # events failed, per filter
-     - # events L3Unbiased

-   All this is already implemented in the SR side
-   All monitoring information are grouped in the
-                   L3MonChunk
-      and SR currently gather all those information and
-      insert them into it (L3MonChunk)
-  There is only one heart beat:
-     is that when the luminosity block index changes
-  Any other persistent data can be added!

## For the L3 Monitoring Server :

- In the SR / l3fchunk side: L3MonChunk methods
- (together they allow the implementation of the monitoring server)
-  L3MonChunk operator + (L3Monchunk &c1, L3MonChunk &c2)
-      implement  the sum of all elements belonging to the
-      L3MonChunk private members. This is, of course,  crucial to the
-       monitoring server implementation.
-  Void makeMonString( std::string &) :   information for the Daq_Monitor server
-        This method produces a string with all the monitoring information
-        in a format readable by the L3 monitor server.
-   void  makeString( std::stringt &) :    information for the Luminosity server
-        This method produces a string with all the monitoring information
-         per luminosity block and all other information, in a format
-        readable by the luminosity monitor server.

## L3 Monitor Server :

- Main ingredients:
-  Register nodes at start of run time
-  Receive data from the Linux nodes and store them on node’s ( or process ) boxes
-  Set up an alarm to time out nodes that overflow the
-       time slice allotted
-  Sum up information from all active nodes ( processes ) when either all nodes are done or if the alarm fires
-  Produce (string) messages with all the information assembled according, to be sent to
-          - Daq_Monitor_Display server
-          - Luminosity server
-  At the end of run time ( for all or for some nodes )
-    send information to the Run Summary server AND
-    clear all boxes for those nodes ( processes ) that
-    ended.

## L3 Monitoring

- L3 Monitor
- Server

- L3 Nodes

- L3 Nodes

- L3 Nodes

- L3 Nodes

- Run Summary
- Server

- DAQ Monitor
- Display

- Luminosity
- Server

**Information (L3MonChunk**
**as a d0omStream object)**
**is sent, as ITC messages,**
**each time that the luminosity**
** ****index changes**

**Here the information from**
**all actives (not timed out)**
**nodes are summed up**

- L3 Monitoring

**Rates, timing, ... **
**so far**

**Statistics, rates,...,**
**for this luminosity**
** ****index value**

- To be implemented

- Alarm

##               The L3 Monitoring Server

-           Progress/Problems in the last (NT) run
- Progress:
- During the past beam run with NT nodes, we were able to run the Monitor Server and send information to the Daq Monitor. Also we were able to receive the luminosity monitoring information.
- Normally, 4 to 8 NT nodes were active.
- Problems:
- We had several problems, mostly related to ITC messages framework.
- One major problem  was that:
-  A certain number of nodes (2 to 4) always timed out - different ones, thought.
-  That made the monitoring display hard to understand for  people not aware of this.
-   That made also the luminosity information more or less useless, because time outs must be very
-     exceptional to this (and any other) scheme make sense.
-  Hard to debug code in the SR side, because the others higher priorities during beam time

##  The L3 Monitoring Server:

-  Progress/Problems with Linux nodes
- We had, so far, none of the problems seen with NT nodes :
-      - no time outs
-      - no corrupted data
- We have implemented the feature that the server can deal
- with several processes per node. So far, we had no problems.
- But we have not yet tested the system at full load, with :
-   - Lots of nodes  >> 10  (expected  ~100 nodes, 2 processes per
-                                                     node)
-   - Lot of information travelling

- L3 Nodes

- L3 Nodes

- L3 Nodes

- L3 Nodes

- L3 Nodes

- L3 Nodes

- Taking advantage of the modular design, we may consider the possibilty of :

## L3MonChunk:

-   map<int l2bit , map< string l3name, Stat> > _map;     //  L3ChunkStatMap
-   vector<int > _time;                                                        //  Time Histogram
-   map<int run#, ClientData *> _clientData
-                                        // holds all client information per run #:
-                                        //L1<->L2 map. L2 active bits mask
-                                        // and L3 #<->name)
-   map<int l2bit, string name > _l2map;                    // L2 #<->name map (all)
-   map<string name, l3ToolStats *>    _tool;              // ToolStatMap
-   map[<string name , l3FilterStats *>_filter;             //FilterStatMap
-   map<int l2bit, map<string l3name, LumStat> >  _lum_old;           // L3LumStatMap
-   map<int l2bit, map<string l3name, LumStat> >  _lum_old_save;   // L3LumStatMap

- L3MonChunk:

## L3TEle

- diem_cal

- L3FJet
- (Jet10)

- (Ele_Loose)

- L3TJet
- (Jet_10)

- L3FMEt
- (calmet_)

- Ele1

- el_jet_met

- Execution tree

- L2 bits

- L3 scripts

- L3 filters

- Tools

- L3FEle
- (ele2)

- L3FInvMass
-    (inv_1)

- .
- .
- .

- .
- .
- .

- .
- .
- .

- (ele1)

- L3FEle
- (ele1)