# Presentation: 323104

## Is a Grid cost-effective?

- Ralf Gruber, EPFL-SIC/FSTI-ISE-LIN, Lausanne

## HPC in Europe

- 176 in Europe, 12 have more than 1 Tflops/s Linpack
- First is CEA-DAM: No. 7
- Germany: 71, UK: 39, France: 22, Italy: 16, Others: 28
- Industry:
- 108, first (Telecom I) at No. 96
- BMW: 11, Daimler-Chrysler: 5, Car F: 6
- Not one big, but many smaller machines
- HPC Companies:
- Quadrics
- Scali, SCI-based clusters: No. 51
- SCS: see Toni’s presentation
- Beowulf production: Paralline, Dalco, ......

**HPC in Europe**

## Swiss-Tx project

- 1998: Prototype Swiss-T0 with 16 Alphas 21164
- 1999: Swiss-T1 (Baby) with 16 Alphas 21264
- 2000: Swiss-T1 with 70 Alphas 21264
- Know-how transfer to industry:
- 2001: GeneProt protein sequencing machine with 1420 Alphas 21264
-           Peak performance=1780Gflop/s
- In June 2001, would have been No. 12 in the Top500, 2nd in Europe and
- Was world number 1 of industrial computer installations
- Would be No. 48 (=C-Plant) in the Top500 list of November 2002 and
- Is still number 2 of industrial computer installations

**Swiss-Tx project**

## Is a grid cost-effective?

**Is a grid cost-effective?**

- Reasons:
- Since 25 years, we can use machines all over the world
- Those who needed good connections, installed it (HEPNET, Swissprot, ..)
- Using Java is against HPC

## Parallel machines at EPFL and CSCS

- SGI Origin3800 (500 MHz) 128 processors
- HP Alpha ES45/Quadrics (1.25 GHz) 100 processors
- Institutes
- PC clusters (CFD, Chemistry, Mathematics, Physics)
- IBM SP-2 (EFD)
- CSCS
- NEC SX-5 (16 processors)
- IBM Regatta (256 processors, 1.3 GHz)

**Parallel machines at EPFL and CSCS**

## Optimal grid scheduling

- 	. Single processor
- 	. Cluster
- 	. Application
- Application tailored Grid scheduling

**Optimal grid scheduling**

## Characteristic single processor parameters Va and ra

_**Examples**_
**SAXPY:** **y** = **y** + a * **x**
- Ops = 2
- LS = 3 (2 loads + 1 store)
**V****a**** = 2 / 3 **
**Matrix*matrix multiply and add: V****a**** = n / 2**
**r****a**** = min (R******** , R******** * V****a**** / V****m****) = min (R******** , M******** * V****a****)**

******r****a**** = 2/3 * M******** **

******r****a**** = R******** **

**Characteristic single processor parameters ****V****a ****and ****r****a**

## Results with MATMULT Va = 1 (double precision)

- Machine		P	R 	ra=M** **	**V****** r 	%
- NEC SX-5	1	8000	8000	   **1**
- Pentium 4 1.5/R	1	1500	  400	   **4**	229	57
- Alpha 21264	2	2000	  333	   **6**	200	60
- Pentium 4 1.7/S	1	1700	  133	 **12**	  92	69
- AMD 1.2/S	1	2400	  133	 **18**	  57	43
- r:	Performance mesurée
- %:	100*r/ ra
- /S: 	Slow SDRAM memory
- /R:	Fast Rambus or RDRAM memory

**Results with MATMULT ****V****a**** =**** ****1 (double precision)**

**R******** **[Mflop/s]** **= Theoretical peak performance
**M******** **[Mword/s] = Theoretical peak memory bandwidth

## Tailoring clusters to applications

**Tailoring clusters to applications**

## Tailoring clusters to applications

**Application:** ******a** **= O / S**
**Machine: ********m**** = r****a**** / b**

- O: Number of operations in Flops
- S: Number of words sent in Words
- ra : Theoretical peak performance of application in Mflops/s
- b: Peak network bandwidth per processor in Mwords/s

**Tailoring clusters to applications**

## Cluster characterisation

**Machine****	****   ****	****   P****	****  P*r****a**** ****	****  C ****	****<d>****	**** ********m**
**	****	****	****[Mflops/s]****	****[Mwords/s]**
**T1 (TNet)****	****  ****	****32*2****	****21333****	****  640 ****	****1.25****	****  40**
**T1 (Fast Ethernet) ****	****32*2****	****21333****	****    48****	****1****	****444**
**IELNX (P4+FE)****	****22****	**** 8800****	****    34****	****1****	****250**

******m**** = P * r****a**** **[Mflops/s]** * <d> / C **[Mwords/s]

******m**** = ****r****a**** / b**
**b = C / P <d>**** **

**Cluster characterisation**

## LAUTREC on Swiss-T1  + TNet

******m**** = 100**
**Water molecules:**
******a**** = 5*P*(0.65*N****orb****+4.24*log****2****V) / 3*(P-1)**
**P=8, N****orb****=128, log****2****V=20 **
******a**** = 330**
****** ****= 3.3 ****(3.6 measured)**
**-> 25% of overall time is due to communication**
**75% is due to computation**

**LAUTREC on Swiss-T1  + TNet**

## LAUTREC on Swiss-T1  + Fast Ethernet

******m**** = 1333**
**Water molecules:**
******a**** = 5*P*(0.65*N****orb****+4.24*log****2****V) / 3*(P-1)**
**P=8, N****orb****=128, log****2****V=20 **
******a**** = 330**
****** ****= 0.25 ****(0.25 measured)**
**-> 20% of overall time is due to computation**
**80% is due to communication**

**LAUTREC on Swiss-T1  + Fast Ethernet**

## LAUTREC : Effect of latency

**Break-even message length:  beml=L*b=1000B**
**Fast Ethernet: L=100 ********s MPI latency, b=10MB/s**
**Break-even message length:  beml=L*b=1000B**
**Average message length in Lautrec: aml= *********V/16*P****2**
**For test case (V=96**3, P=8): aml=40 kB>>beml**

**LAUTREC : Effect of latency**

## Point-to-point applications

- FE/FV:
- O Nb of volume nodes
- O Nb of variables per node square
- O Nb of non-zero matrix elements
- O Nb of operations per matrix element

- FE/FV:
- S Nb of surface nodes
- S Nb of variables per node

- FE/FV:
- a Nb of nodes in one direction
- a Nb of variables per node
- a Nb of non-zero matrix elements
- a Nb of operations per matrix element
- a Nb of surfaces

- a (NS/FV/100**3) **** 2000
- a (Poisson/FD/100**3) ****400
- Reminder (Beowulf+Fast Ethernet): m **** 250

**Point-to-point applications**

## Other quantities

- Price per 1h CPU time
- Engineering salary
- Energy consumption
- Maintenance/servicing/personnel costs
- User commodity

**Other quantities**

## Optimal Grid scheduling

- . Estimate machine and application parameters by counts
- . Measure machine and application parameters (PAPI, ...)
- . Build up a data base on these parameters
- . Find and submit to best suited Grid ressource (not always optimum)
- . Update the data base dynamically
- . Perform statistics on decisions and decision failures

**Optimal Grid scheduling**

## Optimal Grid scheduling

- . Match machine/application (MPI or not MPI)
- . Best price/performance ratio based on parameterisation
- . Availability of the ressources
- . Engineering costs
- . Energy consumption

**Optimal Grid scheduling**

## Optimal Grid scheduling

- . Detect too often demanded unavailable ressources
- . Detect real costs of an application
- . Detect applications that should be parallelised/optimised to reduce costs
- . Guide decision making for the next purchase
- . Guide decision on R&D money attribution

**Optimal Grid scheduling**

## Is a grid cost-effective?

**Is a grid cost-effective?**

- Minimise overall costs by application adapted job execution
- Purchase not available demanded low-cost ressources
- Parallelise cost-ineffective applications
- Reduce engineering and energy costs
- Note: “Cheap” ressources do not have to be used up during 90%
- Results in
- More computing ressources for the same price
- More rapid increase of application efficiencies
- Questions
- Do computer manufacturers play the game?
- Do application owners play the game?
- Can we change users, decision makers and computing centres?

## Reference

*Parameterisation to tailor commodity clusters to applications*,
- Future Generation Computer Systems **19 **(2003) 111-120
- see also:
- http://sawww.epfl.ch/SIC/SA/publications/SCR02/scr13e.html

**Reference**