# Presentation: 276692

## The LHC Trigger Challenge and The CMS Strategy

- Christos Leonidopoulos
- CERN-PH

**Wine and Cheese Seminar**
**18 January 2008 − FNAL**

## What are we trying to do?

- Find the most interesting physics signals at LHC
- Store them for off-line processing

## What do we expect to see?

| Process | σ (nb) | Production
rates (Hz) |
| --- | --- | --- |
| Inelastic | 108 | 109 |
|  | 5×105 | 5×106 |
|  | 15 | 100 |
|  |  2 |   20 |
|  |  1 | 10 |
|  |  0.05 |   0.1 |
|  |  0.05 |  0.1 |
|  |  0.05 |  0.1 |
|  |  10-3 | 10-2 |

## What is the problem?

- We don’t keep all these events → Selection
- Old Physics happens more often than New Physics
- New Physics buried under a ton of Old Physics

## We don’t keep all these events

- How many do we keep? About 150-300 Hz
- Why only so few? Not enough resources!
-  300 Hz at 1-2 MB/event → Up to 36 GB per minute
-  Up to 6 ́000 ́000 GB of storage needed per year
-  Plus: about 30 secs to reconstruct every event off-line
- “Interesting” physics occurs at ~10, 1 or < 1 Hz
-  We are only interested in a (tiny) fraction of all events
-  We *don’t* really want to keep all these events

## Old Physics: more likely than New Physics

| Process | σ (nb) | Production
rates (Hz) |
| --- | --- | --- |
| Inelastic | 108 | 109 |
|  | 5×105 | 5×106 |
|  | 15 | 100 |
|  |  2 |   20 |
|  |  1 | 10 |
|  |  0.05 |   0.1 |
|  |  0.05 |  0.1 |
|  |  0.05 |  0.1 |
|  |  10-3 | 10-2 |

- It is challenging (to say the least)
- to find these rare exciting events

## LHC reference numbers

- 25 ns or 7.5 m

## New Physics buried under Old Physics

-  Interaction rate:
-  Distance between bunch crossings:
-  Non-empty bunch crossings:
-  Average # of interactions per (non-empty) crossing:

- (*) Total inelastic cross section (±20%)

- (*)

## New Physics buried under Old Physics

- Pileup: serious problem at LHC at high luminosities

## Interactions every 25 ns ...
In 25 ns particles travel 7.5 m

- Interactions every 25 ns ...
- In 25 ns particles travel 7.5 m

- Cable length ~100 meters ...
- In 25 ns signals travel 5 m

- The 25 ns challenge

*Credit: Daniel Froidevaux*

## What are we trying to do? (v.2)

-  Select the most interesting physics signals at LHC
-  150-300 Hz out of ~ 1 GHz of “noise” (selection: 10−7)
-  In real time
-  Store them for off-line processing

## Background is a Disease

- Meet the Cure

## ATLAS and CMS triggers

- _ATLAS_
-  3 levels (traditional design)
-  L1: hardware, firmware
-  L2 + EvF: high-level software

- _CMS_
-  L2, L3: merged into HLT
-  L1: hardware, firmware
-  HLT: high-level software

- 40 MHz

- 100-200 Hz

- 2 kHz

- 100 kHz

## Particle-id at Level-1

## Why not use tracker info at Level-1?

- Thoughts of including
- tracker info at L1 for SLHC

## ATLAS High Level Trigger

-  L2 and L3 (Event Filter) form
-    High Level Trigger (HLT)
-  L2 (~500 CPUs) accesses ~10% of
-    event info (full granularity)
-    seeded by L1 objects
-  Event Filter (~2000 CPUs) accesses
-   full event using “off-line quality”
-   algorithms
-  Custom L2-steering system
-  L1: 2.5 μs, L2: 40 ms, L3: 4s

## CMS High Level Trigger

-  L2 and L3 merged into
-    High Level Trigger (HLT)
-  HLT (~2000 CPUs) accesses full event
-   info (full granularity) seeded by
-   L1 objects using “off-line quality”
-   algorithms
-  L1: 3.2 μs, HLT: 40 ms

## ATLAS vs. CMS Triggers

## ATLAS vs. CMS Triggers

- Overall:
-  Very similar performances
-  Trigger bandwidth determined by detectors
-    and physics programs, not trigger design
-  Systems still differ (two farms vs. single farm at HLT)
-    so: commissioning and debugging also different

## Trigger: A tricky business

- (*)  LHC upgrade: 1B CHF, CMS+ATLAS detectors: 1.2B CHF

- Which begs the question(*):
- Will _your_ favorite new physics signal
- be included in the small fraction
- of selected events? (unexpected signatures always a worry)

## What are we trying to do? (v.3)

-  Select the most interesting physics signals at LHC
-  150-300 Hz out of ~ 1 GHz of “noise” (selection: 10−7)
-  In real time
-  Store them for off-line processing
-  Don’t screw up

## What to avoid

-  Killing the interesting physics altogether
-  Biasing the selected event samples:
-  Uncertainties in topologies of rejected events
-  Introduction of large systematic errors

## How to build good triggers

## Ask old people

- Learn from previous experiments

## How to build good triggers

- General strategies
- As simple as possible
- As inclusive as possible
- Robustness
- Redundancy

-  No single silver bullet
-  Using common sense (and trigger studies)

## Simplicity

- General strategies
- As simple as possible
- As inclusive as possible
- Robustness
- Redundancy

-  Construct triggers with simple conditions
-  Simple triggers easier to
-  commission
-  debug
-  understand

## Be inclusive

-  Better to have one trigger covering similar analyses
-  Even better: covering other, unrelated analyses
-  Should be able to discover the unexpected as well

- Strong social aspects, often ignored
- Competition inside experiment
-  One (wo)man’s signal is another (wo)man’s background
-  It’s best for your analysis to rely on a popular trigger
- Inertia: people get used to “old” triggers
- Safety: people tend to ignore “new” triggers

- General strategies
- As simple as possible
- As inclusive as possible
- Robustness
- Redundancy

- Your favorite trigger should be
- deployed online as early as possible

## Robustness

- General strategies
- As simple as possible
- As inclusive as possible
- Redundancy

-  Make sure your trigger can run for *many* events
-  Including pathological events
-  Including events with x10 more hits than MC predicts
-  Make sure your trigger is immune
- To beam conditions, detector problems

- Missing *E****T*** : the popular trigger for
- SUSY particles
- Dark matter candidates
- But also: neutrinos (so: *W*s, Higgs, etc)

## Missing ET at DØ

- It takes a *long* time to
-  Commission the detector
-    for data-taking
-  Remove all problematic runs
-  Understand noisy environment
-  Discover (and remove)
-    problematic channels

- Missing *E****T*** :
- Not ideal for startup
- Typically the last trigger to be commissioned

## Redundancy

-  Make sure your signal can be selected
-    by more than one triggers
-  Helps to understand biases
-  Ensures that if a trigger has problems (rates too high
-     or instability) you still get your events

- General strategies
- As simple as possible
- As inclusive as possible
- Robustness

## How is the trigger different at LHC?

## Trigger trends

- Luminosity, rates, event sizes:
- all increased by ~an order of magnitude

## “If I have seen further it is by standing on the shoulders of Giants”

## Evolution in computing

- Advances in
-  Networking (Ethernet, Terabit/s networks)
-  PC industry (computing power and memory abundance)
-  Software standards (Linux, http, XML, C++, Java)
- offer affordable, modular, scalable, upgradable solutions

## CMS trigger: scalable

- The trigger at CMS evolves with luminosity
- Adjusts to increases in:
-  DAQ capacity (L1 rate)
-  CPU-power needed at HLT
-  By adding/upgrading PCs as necessary

## Building triggers

-  *pp* inelastic collisions: mainly hadrons at ~few GeV
-   Interesting physics: typically with larger *p**T*
-    Make sure we can still trigger on events
-    with many soft particles
-  Signatures (event topologies) compatible
-    with new (or old but still interesting) physics
-  Simple objects: leptons, jets, photons
* *More advanced objects: taus, b-jets
-  Trigger’s “sine qua non”:
-  High efficiency on signal events
-  Large reduction of background (SM) rates

## CMS Trigger Examples

- _Nota Bene:_
-  CMS focusing on early luminosity studies (1032 cm-2 s-1 and lower)
-  Summary of comprehensive trigger study carried out last summer

## The “HLT Exercise” at 1032 cm-2 s-1

-  Most extensive study of the High-Level Trigger algorithms, software, rates, efficiencies and technical requirements (June 2007)
- All algorithms developed, tested and run within the latest software fwk
- Detector geometry simulated reflecting the most up-to-date understanding of detector layout
- Reconstruction code based on offline code
- Assuming half of DAQ available, maximum L1 output: 50 kHz
-  For the first time:
- Actual RAW data format expected from the CMS readout simulated
-  Code for data-unpacking deployed, included in all timing studies
- Deployment of Level-1 trigger emulator
-  Realistic set of events input to HLT

- “What is the CPU performance of the HLT?”

## High Level Trigger Menu

-  μ: 50 Hz
-  eγ: 30 Hz
-  jets/MET/Ht: 30 Hz
-  τ: 7 Hz
-  b-jets: 10 Hz
-  x-channels: 20 Hz
-  prescaled: 15 Hz
-  Total: 150 Hz

-  Leptons: “bread & butter” triggers
-   for many physics analyses
-  Prescaled triggers should
-   accompany every physics trigger

- Significant contributions
- from US-CMS/LPC physicist

## CPU-performance of HLT

- _Conclusion:_ physics and CPU performance consistent with CMS physics program and resources.
- Running on CPU very close to what will be procured  (†Core 2 5160 Xeon  at 3.0 GHz)
-  43 ms/event accepted by Lvl-1  (DAQ TDR - Dec 2002: estimated 40ms)

- Average time needed to run *full *Trigger Menu on L1
- accepted events: 43 ms/event †

- CERN-LHCC 2007-021, LHCC-G-134
- CMS AN-2007/090

-  # of MC events used in study: ~40M
-  Time to generate MC: 2-3 months
-  Equivalent of real-data taking at 1E32:
-    A few secs...

## Do we trust the MC?

- We trust some aspects of it; for the rest we take a conservative approach
-  Safety factor of 3 in allocation of L1 bandwidth; only 17 kHz
-     allocated to simulated channels – to account for:
-  Uncertainty in maximum DAQ bandwidth (especially at startup)
-  Input cross sections (especially QCD; Tevatron shows factors of ~2)
-  All that we have not simulated:
-  beam conditions, noise spikes, other electronics correlations...
-  Safety factor of 2 in HLT accept rate; only 150 Hz
-     allocated to simulated channels – to account for
-  Uncertainties in cross sections (e.g. heavy-flavor cross section)
-  Uncertainties in simulation (e.g. rate for a jet faking an electron: experience from Tevatron experiments shows Monte Carlo reliable to within a factor 2)

- Optimization work continues: Additional improvements have been incorporated since last summer

## The Next Steps:Preparing for the Year Ahead

## Trigger performance & monitoring

- Evaluate the performance of the HLT algorithms (background rejection, CPU/memory cost, physics efficiency of signal events)

- LPC and FNAL: central role

## Trigger performance & monitoring

- Online monitoring:
- When: While running (with low statistics and non-final calibration and alignment)
- Decide: if we need to take immediate action to fix a problem
- Offline monitoring:
- When: Immediately after the run (with higher statistics almost final calibration/alignment
- Decide: if the trigger has expected behavior & if the run as taken is good for physics.
- Algorithm Performance:
- When: After a large multi-run sample is collected
- Decide: if modifications are needed to the trigger algorithm.

## Trigger performance & monitoring

- Evaluate the performance of the HLT algorithms (background rejection, CPU/memory cost, physics efficiency of signal events)
- Includes contacts from detector, object-id and physics groups to ensure usefulness of triggers at  reasonable bandwidth and CPU/memory cost
- Large ongoing effort to put together and validate code for trigger monitoring and performance evaluation

## Preparing the 2008 Trigger Menus

## LHC Startup

| Bunches | * | Ib | Luminosity | Event rate |
| --- | --- | --- | --- | --- |
| 1 x 1 | 18 | 1010 | 1027  | Low |
| 43 x 43 | 18 | 3  x 1010 | 3.8 x 1029 | 0.05 |
| 43 x 43 | 4 | 3  x 1010 | 1.7 x 1030 | 0.21 |
| 43 x 43 | 2 | 4  x 1010 | 6.1 x 1030 | 0.76 |
| 156 x 156  | 4 | 4  x 1010 | 1.1 x 1031 | 0.38 |
| 156 x 156 | 4 | 9  x 1010 |  5.6 x1031 | 1.9 |
| 156 x 156 | 2 | 9  x 1010 | 1.1 x1032 | 3.9 |

- Maximum luminosity expected in 2008: 1032 cms-2 s-1

## Luminosity effects

## CMS trigger at low luminosities

- Lower luminosities allow us to trigger
-  with lower thresholds, looser requirements
-  e.g. no isolation on leptons
-  on physics that we cannot trigger on later
-  e.g. *B* physics or other low-*p**T* physics

## The startup Trigger Menus

- Remove most requirements,
-   put thresholds at lowest possible value
- Calorimeter: low *E**T* electron ( 5 GeV) or Jet ( 10-15 GeV)
- Muons without any threshold
-    (effective kinematic limit: 3 GeV *p**T* cut)
-  Other candidate triggers for later running are active for diagnostic and study purposes (trigger efficiencies)

## The startup Trigger Menus #2

- Bandwidth investment in “technical” triggers
-  Calibration, alignment, monitoring
-  Zero-bias triggers (using LHC machine clock information)
-  Minimum bias triggers (looking for any detector activity indicating inelastic collisions)
-  Save small fraction of L1-accepted events irrespective of HLT decision (“Auto-accept” for CDF or “mark-n-pass” for DØ)
- Collect events that will aid in detector understanding
-    and physics commissioning:
-  Samples enriched in *J/**ψ*, *W*, *Z*, top signatures

## The startup Trigger Menus #3

- Inject more realism in our trigger studies
-  Expected uncertainties in detector alignment & calibration at startup
-  Imperfect detector (hot or dead channels)
-  Heavier detector occupancies
-  Can the algorithms cope?
- What happens to the CPU-performance?

-  The goal is *not* to predict what will happen when LHC starts
-  Put mechanism in place for dealing with *any* problem
-  Quick feedback from all CMS groups: evaluate, adjust and evolve!
-  While experts work on problems, the show must go on. Fast.

## Strategy

***The Unknowns***
- As we know,
- There are known knowns.
- There are things we know we know.
- We also know
- There are known unknowns.
- That is to say
- We know there are some things
- We do not know.
- But there are also unknown unknowns,
- The ones we don't know
- We don't know.

- Things we know and understand

- Problems we know we will face

- Where we can deal w/ any problem
- and deliver events that CMS needs

## Trigger Commissioning

## Magnet Test & Cosmic Challenge (MTCC)

## L1 commissioning: Emulator

- Bit-by-bit comparison of L1 trigger
- behavior in emulator and hardware

## HLT commissioning: cosmic muons

- Successful integration of several components:
-  Deployment of HLT algorithms in online environment
-  Loading of HLT parameters, calibration, alignment from online database
-  Real-time filtering with real (cosmic) data
-  Storing offline data tagged with trigger information
-  Online event display and trigger monitoring

## HLT commissioning

- “HLT-exercise” menu being run on online Filter Farm with MC events
- Online testing necessary commissioning step in order to:
- Ensure stability, robustness of HLT code
- Ensure (CPU-performance and) memory footprint requirements are under control
-  Evaluate different hardware options before placing purchase order

## The Bottom Line

## Fit everything into O(100) Hz

- How should the bandwidth be shared among
-    the large number of available triggers?
-    A difficult question – many things to consider:
-  Are triggers inclusive enough?
-  Which triggers are used by what physics analyses?
-  What are the experiment’s priorities?

- Example #1:
- “Experiment X has a stronger chance of discovering the Higgs first”

- Example #2:
- “Rumors are that experiment Y is seeing a bump on channel Z.
- We must increase bandwidth of corresponding trigger”

## Epilogue

- The trigger is a dynamic creature, made by human beings
- Bound to imperfections, common sense, inertia and strong personalities
- Must evolve with time, luminosity increases and better detector understanding
- It requires dedicated studies by analysis users
- But it remains the single most important item in hadron experiments: what makes the difference between discovering New Physics at LHC or not

## Epilogue

- Higgs, SUSY,
- Dark Matter,
- Exotica

## Backup

## Exploded View of CMS

**Minus Side**

**Plus Side**

## L1 Trigger Commissioning: CAL, MUO

***L1 Trigger Commissioning: CAL, MUO***

- Calorimeter Triggers:
- ECAL Barrel, HCAL, and RCT fully installed
- GCT e/ trigger fully installed
- Vertical pattern tests TPGRCTGCTGT run successfully, ever increasing scope (complexity & no. of channels) ...
- GCT jet trigger available in February
- Production of ECAL endcap TCCs in spring ‘08
- Muon Triggers:
- DT Track Finder crates operational (3/6)
- Installing remaining crates now, final connections to detector Jan’08
- CSC Track Finder is operational
- RPC central crates & 2/12 PAC crates operational
- RPC Trigger-Board production expected January
- GMT is operational

- 

- 

- 

- 

- ()

- ()

- 

## L1 Trigger Commissioning: Central Systems

***L1 Trigger Commissioning: Central Systems***

- GT
- Operational (incl. new TCS board!)
- FDL-readout fixes: January
- TTC:
- TTC system is operational
- Installation of final RF2TTC before Jan. Global Run
- Control Software - Trigger Supervisor
- Operational for the majority of L1 systems
- New TS release available, 				upgrade of all cells starting now ...
- Emulator & DQM
- Emulator operational offline
- Online integration planned for January
- Emulator configuration from DB ongoing
- Basic Trigger-DQM operational, continuous additions ...

- 

- 

- 

- ()

- ()

## HLT Physics performance: eγ, μ 

- HLT Physics performance: eγ, μ

- _Muon HLT efficiency for benchmark channels_

- _Egamma HLT efficiency for benchmark channels_

- Good *W/Z* efficiencies
- for muon, egamma HLT

## HLT Physics performance: tau

- _HLT efficiency:Higgs_

- _HLT efficiency: Z_