# Presentation: 373039

## SIMULATION OF DAILY ACTIVITITIES AT 

**SIMULATION OF DAILY ACTIVITITIES AT **
**REGIONAL  CENTERS**

- MONARC Collaboration
**Alexander Nazarenko and Krzysztof Sliwa**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

## MONARC SIMULATION TOOLS

**The simulation tool developed within MONARC is based on Java technology which provides adequate tools for developing a flexible and distributed process oriented simulation. Java has a built-in support for multi-threaded objects and for concurrent processing, which can be used for simulation purposes provided a dedicated scheduling mechanism is developed (this “simulation engine” has been developed by Iosif Legrand).**
**Java also offers good support for graphics which can be easily interfaced with the simulation code. Proper graphics tools, and ways to analyse data interactively, are essential in any simulation project (Alex Nazarenko’s contributions were the greatest here)**
**MONARC simulation and modelling of distributed computing systems provides a realistic description of complex data, and data access patterns and of very large number of jobs running on large scale distributed systems and exchanging very large amount of data (Data Model developed by Krzysztof Sliwa and Iosif Legrand)**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

## Baseline Model for Daily Activities

**Baseline Model for Daily Activities**

**Event processing rate: ****1, 000, 000, 000 events/day**

- _[  ](file:///Slide2_PGA.htm)_

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

## Regional Center Model

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

## VALIDATION MEASUREMENTS

- “**atlobj02**”-local

-   **DB on local disk**

**2 CPUs x 300MHz**

**13.05 SI95/CPU**

- “**monarc01**”-local

-  **DB on local disk**

**4 CPUs x 400MHz**

**17.4 SI95/CPU**

**DB on AMS Server **

**server : “atlobj02”**

**client : “monarc01”**

**Local DB access**

**DB access via AMS**

**monarc01 is a 4 CPUs SMP machine**
**atlobj02    is a 2 CPUs SMP machine**

**Multiple jobs reading concurrently objects from a data base. **

**Object Size = 10.8 MB**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

##  Validation Measurements I   The AMS Data Access Case 

**Simulation**

**Measurements**

- Raw Data

- DB

**LAN**

**4 CPUs Client**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

##  Validation Measurements ISimulation Results

-  Validation Measurements ISimulation Results

**DB access via AMS**

**Local DB access**

**Simulation results for AMS & Local Data Access**

**1,2,4,8,16,32 parallel jobs  executed on 4 CPUs SMP system**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

##  Validation Measurements I

**Local DB access 32 jobs**

**The  Distribution of the jobs processing time**

**Simulation**
**mean**** ****109.5**

**Measurement**
**mean 114.3**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

##  Validation Measurements IMeasurements & Simulation 

-  Validation Measurements IMeasurements & Simulation

**Simulation**

**Measurements**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

##  Validation Measurements II

**1000BaseT**

**Server**

**Client**

**Test 1**
**CNAF**

**Sun Ultra5, 333 MHz**

**Sun Ultra5, 333 MHz**

**sunlab1**

**gsun**

**10BaseT**

**Test 2**
** ****PADOVA**

**Sun Ultra10, 333 MHz**

**Sun Enterprise 4X450 MHz**

**cmssun4**

**vlsi06**

**2 Mbps**

**Test 3**
** ****CNAF-CERN**

**Sun Ultra15, 333 MHz**

**sunlab1**

**monarc01**

**Sun Sparc20, 125 MHz**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

##  Validation Measurements IITest 1

**Gigabit  Ethernet Client - Server**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

##  Validation Measurements IITest 2

**Ethernet Client - Server**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

##  Validation Measurements IITest 3

**2Mbps WAN Client - Server**

**CPU usage**

**Data Traffic**

**12 3 4 5   6**

**12 3 4 5   6**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

**Similar data processing     jobs are performed in six RCs**
** ****CERN has **
**    ****RAW, ESD, AOD, TAG**
** ****CALTECH, INFN, Tufts,    KEK has**
**    ****ESD, AOD, TAG**
**“****Caltech2”  has **
**    ****AOD, TAG**

-  Physics Analysis Example

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

-     **Each group reads 100% TAG events and follows:**
**~10% to AOD**                 ~**1% to ESD **                ~**0.01% to RAW **

** **
**~20 ****Jobs/Day in total evenly spread among participating RCs   ****        **

- _[  ](file:///Slide4_2005.htm)_

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

-   **(Les Robertson's estimate of July 99)****           **

- _[  ](file:///Slide5_Jobs.htm)_

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

-   **Totally**** ****220 Independent Jobs****: 200 Physics Group Analysis and 20 Group Selection****        **

- _[  ](file:///Slide5a_Rec.htm)_

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

- _[  ](file:///Slide6_RCs.htm)_

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

-     **5 Tier 1 Regional Centers and one Tier 2 center**

**200 Analysis and 20 Selection ****Jobs are ****evenly**** spread among Tier1 RCs  ****       **

- _[  ](file:///Slide7_AMS.htm)_

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

**  ****      **
**           **** ****One RC (CERN) configured to run ****200 concurrent Physics Group Analysis**** Jobs and ****20 Selection**** jobs a day **

-   _[Model1](http://www.cern.ch/MONARC/sim_tool/Publish/publish/Model3/200PA_20x40PS_rAMS_bil_abn/index.html)__[ ](http://www.cern.ch/MONARC/sim_tool/Publish/publish/Model3/200PA_20x40PS_rAMS_bil_abn/index.html)__  (optimized AMS distribution)                   _ _[Model2](http://www.cern.ch/MONARC/sim_tool/Publish/publish/Model3/200PA_20x40PS_wAMS_bil_abn/index.html)__[ ](http://www.cern.ch/MONARC/sim_tool/Publish/publish/Model3/200PA_20x40PS_wAMS_bil_abn/index.html)__     _

- _[  ](file:///Slide8_5vc1RC.htm)_

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

**  **_[Model1](http://www.cern.ch/MONARC/sim_tool/Publish/publish/Model3/1RC_20PGS_bil_abn/index.html)__[ ](http://www.cern.ch/MONARC/sim_tool/Publish/publish/Model3/1RC_20PGS_bil_abn/index.html)__        _
**     ****One RC (CERN) minimally configured to run ****20 concurrent Physics Group Selection**** Jobs a day **

**Model2****       **
**Five Tier 1 Centers minimally configured to perform ****the same task**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

**  ****Model1****         **
**            ****One RC (CERN) minimally configured to run all the Jobs a day **

-   **  ****Model2****         **** **
**            ****Five Tier 1 Centers ****optimized ****to perform ****the same task with 30 MBps WAN**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

**            ****Five Tier 1 and one Tier 2 Centers ****optimized ****to perform ****the complete set with 30 MBps WAN ****and**** optimized LAN**

** **
**  ****Model1****    (fixed values)                       ****Model2****  (randomized data processing times and sizes)    **** **
** **

**  **** ****           **

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

## Slide 24

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

## Slide 25

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

## Slide 26

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000

## Replication job: partial dataset replication from CERN to Regional Centers

**Replication job: partial dataset replication from CERN to Regional Centers**
**Flexible data access:  data exchange between Regional Centers without getting data directly from CERN (depending on load, availability,...) **
**Imposing coherence on the concurrent jobs: if Reconstruction and/or Replication is taking place, Analysis/Selection jobs should be able to monitor new data availability if requested**** **
**Improving Cost function: adding cost of WAN, adding other hidden costs currently not accounted for**
**Optimization with respect to the parameter *****Time*Cost***** for a given task run on different architectures**

- K. Sliwa/ Tufts University  DOE/NSF ATLAS Review

- January 20, 2000