Evolution of the ALICE
computing model in Run 3
Tatsuya Chujo(for the ALICE collaboration)
WLCG workshop
April 12, 2015, OIST, Japan

ALICE data collection (current)
Nominal LHC beam crossing at 40 MHz
ALICE:
Multi-level trigger system needed:
40 MHz → a few kHz

Single Pb-Pb collision events(√sNN = 2.76 TeV)

Online:1) Reject background2) Select most interestinginteractions
3) Custom computer toreduce the total data
volume2Computing model (Run1 and Run2, -2018)slide by T. Sugitate

3

ALICE upgrade (2018-)
ALICE upgrade; high rate capability
GEM-TPC continuos high rate
readout
ITS Silicon high rate readout
DAQ (RCU etc.)
For LHC high luminosity upgrade, Pb-Pb@50kHzRecord all MB events, x100 statistics（Unique capability in ALICE）

Standard GEM
Pitch=140μm
Hole φ=70μm→Access to high precision
measurements and rare probes
Physics Goals:
Measure- heavy quarks, photons, lepton pairs
azimuthal anisotropy- Jet w/ PID hadron simultaneously
4ALICE Upgrade Plans: (2018-)• Current: reducing the event rate from 40 MHz to ~ 1 kHz– Select the most interesting particle interactions– Reduce the data volume to a manageable size

• After 2018:– Much more data (X 100) because:• Higher interaction rate
• More violent collisions → More particles → More data (1 TB/s)• Physics topics require measurements characterized by;– Very small signal/background ratio → large statistics– Large background → traditional triggering or filtering techniques very
inefficient for most physics channels• Read out all particle interactions (PbPb) at the anticipated interaction rateof 50 kHz• No more data selection– Continuous detector read-out– Read-out and process all interactions with a standard computer farm.
– ~1,500 nodes with the computing power expected by then➡ Total data throughput out of the detectors: 1 TB/s

5

Expected data bandwidth (after 2018-)
Detector

Input to Online
System (GB/s)

Peak Output toLocal Data
Storage (GB/s)

Average Output
to Computing
Center (GB/s)

TPC

1,000

508

TRD

81.5

10

1.6

ITS

40

10

1.6

Others

25

12.5

2

TOTAL

1,146.5

82.5

13.2

Note: LHC luminosity variation during fill and efficiency taken into accountfor average output to computing center6

The ALICE Online-Offline (O2) Project• From Detector Readout to Analysis:• What is the “optimal” computing architecture?
-Handle >1 T Byte /s detector input
Support for continuous readout
Online reconstruction to reduce data volume
Common hardware and software systemdeveloped by the DAQ, HLT, Offline teams
7

Functional Requirements of the O2 system✓ Data fully compressed before data storage.
✓ Reconstruction with calibrations of better quality.
✓ Grid capacity will evolve much slower than the ALICE
data volume.
✓ Data archival of reconstructed events of the current year
to keep Grid networking and data storage within ALICE
quota.
✓ Needs for local data storage higher than originally
anticipated8Basic idea of the O2 system
Detectors electronics
Continuous and triggered streams of raw data

1.2 TB/s

Readout, chopping, and aggregation
Pattern recognition and calibration
Local data compression
Compressed Sub-Timeframes
Data aggregation
Synchronous global reconstruction,calibration and data vol. reduction
Compressed Timeframes

60 GB/s

Data storage
and archival
Compressed Timeframes
Asynchronous and refined calibration,
reconstruction
Quality control - Event extraction

Reconstructed	  events

9

The ALICE O2 Hardware Architecture

10Computing model (Data flow)
AcronymDescriptionRAW

Raw data as it comes fromthe detector.

CTF

Compressed Time Frame
containing the history ofOM(100 ms) of detector
readout information in theform of identified clusters
that belong to identified
tracks.
Event Summary Data.

ESD
AOD

Analysis Object Data forphysics analysis.

HISTO The subset of AOD

information specific for agiven analysis.

MCMontecarlo simulation

by Pierre Vande Vyvre

11

The ALICE O2: Data Reduction (I)from	  First	  Level	  Processors	  (FLPs)

Par/ally	  compressedsub-­‐/meframes

EPNs

by Pierre Vande Vyvre (modified)

Timeframe	  building

O(1000)

Full	  /meframe

Global	  processing

Detector	  reconstruc+on

Calibra+on	  1	  	  on	  full	  detectors

e.g.	  track	  ﬁndinge.g.	  space	  charge	  distor/on

Data	  Reduc+on	  1

Condi4on	  	  &	  Calibra4on	  DatabaseSub-­‐/meframes	  compressed	  /meframes	  AOD	  etc.

QA
Compressed	  /meframesto	  Storage- Reduce data from > 1 TB/s to ~80 GB/s for storage- Only possible with online reconstruction

12

The ALICE O2: Data Reduction (II)slide by A. Uras (IC3INA 2013)13

The ALICE O2: Data Reduction (III)•Data compression factors ranging from 2 to 20 according to the detector•

TPC still accounts for 60% of the total event size
slide by A. Uras (IC3INA 2013)
14

The ALICE O2: Data Storage

by Pierre Vande Vyvre (modified)from	  Event	  Processing	  Nodes	  (EPNs)
Compressed	  /meframes

O2/T0/T1

Storage

T0/T1

Storage

Archiveto	  Reconstruc4on	  Pass•Data in “intermediate” formats (not directly usable for physics analysis):- 80 GByte/s peaks to be handled, distributed over ~1,250 nodes- Average load of 15 GByte/s- Local storage in O2 system- Permanent storage in computing center•Data in “final” formats (usable for physics analysis):- GRID storage, accessible by experiment's users

15O2/T0/T1
Storage

T0/T1
ArchiveStorage

O2 system (2)Compressed	  /meframes

Reconstruc4on	  passes	  	  and	  event	  extrac4on

Analysis

Asynchronous

O2/T0/T1

O(1)

AOD

Globalreconstruc+on

Calibra+on	  2Event	  extrac+on
TaggingAOD	  extrac+on

Sub-­‐/meframes
Timeframes
Compressed	  /meframes
AODQAAnalysis	  Object	  Data	  	  	  (AOD)
Analysis	  	  Facili4es
AnalysisO(1)Histograms,trees

T2

Reconstruc+on
Event	  building
AOD	  extrac+on

Compressed	  /meframes

Simula+on

QA	  data

Data
Quality	  Control

Storage
CCDB	  Objects

AOD

Simula4on

Asynchronous data
flow and processing

ESDO(10)

Condi+on	  	  &	  Calibra+on	  Database

Storage
QAby Pierre Vande Vyvre(modified)16

Computing model (O2 processing flow)
Facility

Function

O2 processing flow
O2 ALICE Online-Offline Facility at

LHC Point 2.
Online reconstruction during the run.
Provides data storage capacity. Afterdata taking: runs the calibration andreconstruction tasks.

T0CERN Computer Center facility
providing CPU, storage and archiving
resources.

T1

Grid site connected to T0 with highbandwidth network links (100+ Gb)
providing CPU, storage and archiving
resources. Reconstruction andcalibration tasks

T2

Regular grid site with good network
connectivity (10+ Gb); runningsimulation jobs.

AF

Dedicated Analysis Facility of HPC
type that collects and stores AODs
produced elsewhere and runs theorganized analysis activity.

by Pierre Vande Vyvre- Maintain the advantages of the Grid and theanalysis trains- Make it more open and more effective
17

Impact on the network• 2015-2017:– LHC Run2 data taking, x2 more heavy ion data.
– Data analysis on Run-2 (+Run-1)– Almost no change from Run-1 scheme. Due to data larger data volume,
network traffic will increase, at least x2.

• 2018- beyond:– LHC Run-3 data taking, x100 more data.
– Architecture change (O2) applied.
• O2, AF, and T0/T1/T2 scheme.
– Significant data reductions, reconstruction in O2 mainly, and analysis →reduce data volume.
1. Can keep the similar network traffic as Run-2?
2. Or if we have AF (using HPC), then it will need more network traffic thanthat in Run-2 → Accelerate local physics analysis.

18Summary and Outlook• ALICE computing upgrades on online-offline for thedata taking after 2018 is ongoing.
• Continuous minimum bias event readout at 50 kHz in Pb-Pb collisions.
• 1 TB/s raw data from detector, need a significant data reduction down to80 GB/s to storage, and make a physics outputs timely.

• O2 Scheme: Online reconstruction and calibration by O2 (near ALICE) &T0/T1, organized analysis at Analysis Farm (AF), and simulation at T2.

• Designing based on physics requirements is completed.
• Intensive works on modeling, technologies (processing
platform & network), O2 prototyping.
• Technical Design Report (TDR) is progressing. It will be
submitted to LHCC in April 2015.

19