The Next-Generation Supercomputing Symposium 2009, Tokyo, Japan

Creating a European
Supercomputing InfrastructureThomas Lippert, Forschungszentrum Jülich, Germany

Regards
fromProf. Dr.
Achim Bachem
Coordinatorof thePRACE Project
Chair ofThomas Lippert

Tokyo, 8.10.2009

Research Centre
2JülichOutline• A European SC RI: Why?
• What is PRACE?
• Where do we stand?
• What comes next?
• Questions
Thomas Lippert

Tokyo, 8.10.20093

Outline• A European SC RI: Why?
• What is PRACE?
• Where do we stand?
• What comes next?
• Questions
Thomas Lippert

Tokyo, 8.10.20094

Supercomputing Drives Basic Sciences
New Physics

Nuclear Physics

Plasma Physics
Geophysics

Thomas Lippert

Astrophysics
Astrophysik
Kosmologie
Cosmology

Particle Physics
Tokyo,Structureof 8.10.2009
MatterSolid State Physics
Chemistry5Supercomputing Drives Applied Science

Environment
Weather/ Climatology
Pollution / Ozone HoleThomas Lippert

Ageing Society
Medicine
Biology

Materials/ Inf. Tech
Spintronics
Nano-science

Tokyo, 8.10.2009

EnergyPlasma Physics
Fuel Cells

6Supercomputing Drives
Engineering and Business
Competitiveness• Reducing design costs byvirtual prototyping:Æ faster time to market• Allowing investigations
where economics or ethics
preclude experimentation
Æ imperative ofsupercomputing
Thomas Lippert
Tokyo, 8.10.20097

Top 500 list 11/07

Computational Advantage

Thomas Lippert

6 years

Tokyo, 8.10.20098

Performance per Country (TOP500 6/2009)

Thomas Lippert

Tokyo, 8.10.20099

Thomas Lippert

Tokyo, 8.10.200910

Numbers of Systems in Industry
TOP500 worldwide
Industry

TOP500 Europe (145)Rest

Industry

37%

46%54%63%

Thomas Lippert

Rest

Tokyo, 8.10.200911

Numbers of Systems in Industry from TOP100
TOP100 EU (34)TOP100 worldwide
Industry

Rest

Industry

11%12%89%

Thomas Lippert

Rest88%

Tokyo, 8.10.2009

12Outline• A European SC RI: Why?
• What is PRACE?
• Where dowe stand?
• What comes next?
• Questions
Thomas Lippert

Tokyo, 8.10.200913

Preparation (2004-2006): The Scientific Case
• Weather, Climatology, Earth Science• Astrophysics, Elementary particle physics,
Plasma physics• Material Science, Chemistry, Nanoscience• Life Science• Engineering
Airplane/helicopter simulation, biomedical flows,
gas turbines, combustion engines, forest fires,
virtual power plant, etc.
Thomas Lippert

Tokyo, 8.10.200914Vision• Achieve leadership in public and private research• Provide world-class HPC systems for word-class science… and Mission• Create a leading, persistent high-end HPC infrastructure– Deploy 3 – 5 systems of the highest performance level (tier-0)– Provide world-class support and training• Implement pan-European Peer-Review procedure
Thomas Lippert

Tokyo, 8.10.200915

JUGENE@FZJ: #3 worldwide, #1 in Europe
1st PRACEsystem
IBM Blue Gene/P
72 racks, 294912 cores
1 Petaflop/s peak
144 Tbyte memory
6 Pbyte disks
25 PByte tape capacity
Highest scalability
Thomas Lippert

Tokyo, 8.10.200916

Thomas Lippert

Tokyo, 8.10.200917

PRACE History
and first steps
HPCEUR2004

2005

HET

PRACE MoU PRACE Preparatory

2006

2007

2008

EU-Grant: INFSO-RI-211528, 10 Mio. €PRACE Implementation PhasePhase
2009

2010

2011

2012

PRACE Operation2013

Foreseen: PRACE Tier-0 centres providing
HPC-capability service in a legal entity
Thomas Lippert

Tokyo, 8.10.200918

The HPC-RI is an item on the ESFRI Roadmap
The European Roadmap for
Research Infrastructures is the firstcomprehensive definition at theEuropean level
Research Infrastructures are
one of the crucial pillars of theEuropean Research Area
A European HPC service – impact
foreseen: strategic competitiveness attractiveness for researchers
 supporting industrialdevelopmentThomas Lippert

Tokyo, 8.10.2009

19

The PRACE Initiative (MoU)•••••

2007: MoU by 15
European member states
2008: F, D, E, NL, UK
reconfirmed theircommitment for EU HPC
Research Infrastructure
2009: Italy became aPrincipal Partner
2009: 4 new European
member states havejoined the PRACE
initiative
Bulgaria and Czech
Republic joined
Thomas Lippert

Tokyo, 8.10.200920

The PRACE Project
EU is funding the PRACE
Preparatory Phase Project (Grant: INFSO-RI-211528)• Partners: 16 from 14 countries• Duration: 1/2008 – 12/2009• Budget: 20 M€, EC: 10 M€• Kickoff: January 29-30, 2008
Thomas Lippert

Tokyo, 8.10.200921

PRACE Project Organization
EU-Preparatory Project Organization– WP1 Management– WP2 Organizational concept– WP3 Dissemination, outreach and training– WP4 Distributed computing– WP5 Deployment of prototype systems– WP6 Software enabling for prototype systems– WP7 Petaflop/s systems for 2009/2010– WP8 Future Petaflop/s to Exaflop/s technologies
Thomas Lippert

Tokyo, 8.10.200922

Outline• A European SC RI: Why?
• What is PRACE?
• Where do we stand?
• What comes next?
• Questions
Thomas Lippert

Tokyo, 8.10.200923

After the First Successful Year:• PRACE is a collaborative achievement of over
250 persons at the 16 partner sites• Project review March 5-6, 2009, in Brussels“The project made very good progress“Thomas Lippert

Tokyo, 8.10.200924

Selected Results and Highlights• Applications• Systems/Architectures• Training and OutreachThomas Lippert

Tokyo, 8.10.200925

Categorisation of Applications• Benchmark applications based on
European HPC usage• Surveys of PRACE partners’ HPC
systems and major applications• 24 systems and 69 applications• Quantitative basis for selecting
representative applications• Disseminated as Technical Report

Other 5.8
Plasma Physics 3.3
Computational
Engineering 3.7

Particle Physics 23.5

Life Sciences 5.3

Astronomy &Cosmology 5.8

Earth & Climate 7.8

Thomas Lippert

Tokyo, 8.10.2009

Computational
Chemistry 22.1

CFD 8.6

Condensed Matter
Physics 14.2

26

Representative Benchmark Suite• Set of applications benchmarks to be used in the procurement
process for Petaflop/s systems• 12 core applications, plus 8 additional applications– Core: NAMD, VASP, QCD, CPMD, GADGET, Code_Saturne,TORB, ECHAM5, NEMO, CP2K, GROMACS, N3D– Additional: AVBP, HELIUM, TRIPOLI_4, PEPC, GPAW, ALYA,SIESTA, BSIT
• Each application will be ported to appropriate subset of prototypes• Synthetic benchmarks for architecture evaluation– Computation, mixed-mode, IO, bandwidth, OS, communication• Applications and Synthetic benchmarks integrated into JuBE
– Juelich Benchmark Environment
Thomas Lippert
Tokyo, 8.10.200927

Mapping
Applications
toArchitectures
Thomas Lippert

E = estimatedTokyo, 8.10.200928

Software Enabling for Petaflop/s Systems (WP6)
Categorisation ofApplications
Prototype Systems

Representativeapplications

Benchmarks

Application
RequirementsBenchmarks
Petascaled
applications

Optimised
applications

Analysis

Architectures

Petascaling

Thomas Lippert

Optimisation

Libraries&Models

Tokyo, 8.10.200929

Prototypes for Petaflop/s systems in 2009/2010

IBM BlueGene/P (FZJ)
01-2008 / 06-2009

IBM Power6 (SARA)
07-2008

Cray XT5 (CSC)
11-2008

IBM Cell/Power (BSC)
12-2008

NEC SX9, vector part (HLRS)
02-2009
Thomas Lippert

Intel Nehalem/Xeon (CEA/FZJ)
06-2009
Tokyo, 8.10.200930

Procurement Strategy, Cost Estimates• Analysis of European procurement procs completed• Work in progress:– Definition of general procurement process– Definition of selection and evaluation criteria– Evaluation process for offers by vendors• Market watch: what will it take to be in the Top 5 / Top 10 in2010 … 2011 ?

• TCO estimates based on market survey, vendor input
and partners’ experience, repeated annually– Overall goals are consistent with committed funding of 80 to120 M€/aThomas Lippert

Tokyo, 8.10.200931

Prototypes for Systems beyond 2010
Sites

Hardware/Software

Porting effort

CEA“GPU/CAPS”

1U Tesla Server T1070(CUDA, CAPS, DDT)Intel Harpertown nodes“Evaluate GPU accelerators andGPGPU programming models andmiddleware.” (e.g., pollutant
migration code (ray tracingalgorithm) to CUDA and HMPP)

CINES-LRZ“LRB/CS”

Hybrid SGI ICE2 /UV/Nehalem-EP&Nehalem-EX/ClearSpeed / Larrabee

Gadget,SPECFEM3D_GLOBE,
RaXml, Rinf, RandomAccess,
ApexMap, Intel MPI BM

CSCS“UPC/CAF”

Prototype PGAS language
compilers (CAF + UPC for
Cray XT systems)“The applications chosen for thisanalysis will include some of thosealready selected as benchmark
codes”EPCC“FPGA”

Maxwell – FPGA prototype(VHDL support &consultancy + software
licenses (e.g., Mitrion-C))“We wish to port several of the
PRACE benchmark codes to thesystem. The codes will be chosen
based on their suitability forexecution on such a system.”Thomas Lippert

Tokyo, 8.10.200932

Prototypes beyond 2010 (cont’d)
Sites

Hardware/Software

Porting effort

FZJ (BSC) eQPACE (PowerXCell cluster with
“Cell & FPGA special network processor)

Extend FPGA-based
interconnect beyond QCD
applications.

LRZ
RapidMind (Streaming Processing“RapidMind” Programming Paradym)
X86, GPGPU, Cell

ApexMap, Multigrid, FZJ(QCD), CINECA (linearalgebra kernels involved insolvers for ordinary differentialequations), SNICNCF
ClearSpeed CATS 700 units“ClearSpeed”

Astronomical many-bodysimulation, Iterative sparse
solvers with preconditioning,
finite element code,
cryomicrotome image analysis

CINECAI/O Subsystem (SSD, Lustre, pNFS)-KTH

AMD Istanbul Cluster (HP)

Energy effcient system withstandard hardwareinterconnect”

Thomas Lippert

Tokyo, 8.10.200933

Outreach and Education
Industry seminars:
1st Seminar Sept. 3, 2008 Amsterdam, Netherlands
2nd Seminar Sept. 7-8, 2009 Toulouse, France

Summer & winter schools:
Stockholm, Athens

PRACE website

PRACE booth at SC08PRACE booths:
ISC, ICT, SC,
1st Industry seminar, 3.9.2008

ICT 2008, PRACE-BoothPRACE Summer School Stockholm

Thomas Lippert

Tokyo, 8.10.2009

PRACE Winter Schoolat the OTE academy, Athens
26-29.8.200934

First industry seminar(August 31, 2008, Amsterdam)• Goals– Understanding industrial needs and expectations– Options for industry involvement in PRACE• Motivation– HPC is a key technology in the 21st century and science– Industry needs independent access to HPC in Europe– PRACE will foster industrial application on top-end– PRACE pushes HPC technology development in EU
Thomas Lippert

Tokyo, 8.10.200935

Thomas Lippert

Tokyo, 8.10.200936

Outline• A European SC RI: Why?
• What is PRACE?
• Where do we stand?
• What comes next?
• Questions
Thomas Lippert

Tokyo, 8.10.200937

Next Steps• Contracts for the legal entity are in final negotiation, signatureplanned in December 2009
• Temporary seat in Lisbon• PRACE Tier-0 Infrastructure will become operable during firsthalf of 2010
– Small HQ in charge of organisational, financial tasks and Peer Review– Tier-0 services for the Scientific Community provided by the “hostingmembers” of the legal entity.

• Implementation and further development will be major task in
the next years: co-funded by the EC• Æ 1st PRACE Implementation Phase Project in preparation
Thomas Lippert

Tokyo, 8.10.200938

Accessing the future PRACE RIDisclaimer: this is work in progress
Access Model• Based on peer-review: “Best systems for best science”• Three types of resource allocations– Test / evaluation access– Project access – for a specific project, grant period ~ 1 year– Programme access – resources managed by a community• Free-of-charge for European scientific communities
Thomas Lippert

Tokyo, 8.10.200939

Accessing the future PRACE RI cont.
Disclaimer: this is work in progress
Access Models for Commercial users ( < 10%)• Peer Review based, free of charge if results are madepublic (similar to US INCITE programme)• Paid AccessFunding of the Infrastructure• Mainly national funding through partner countries• European contribution• Monitor relation between contribution and usage
Thomas Lippert

Tokyo, 8.10.200940

How can users get involved ?
• EU country: Join PRACE Initiative!
• International: Cooperation of partnerorganizations is welcome!
• Porting of codes to the PRACE Prototypes– Prototypes are mainly be used project-internally, but– Prototypes are also made available to selected users fortesting/porting purposes using a light-weight peer-review process

Participation in the PRACE training events– 5 Code Porting Workshops so far– PRACE booth at SC’09
Thomas Lippert

Tokyo, 8.10.200941

Summary
In its first 18 months, the project
had …
– major achievements in all areas– raised significant awareness with all
stakeholders– reconfirmed the commitmentof the Governments

PRACE is well prepared tomaster the future challenges
towards a European RI
Thomas Lippert

Tokyo, 8.10.200942

PRACE
All-Hands meeting
February 2009

Thomas Lippert

Tokyo, 8.10.200943

Partnership for Advanced Computing inEurope: Creating a European HPC
Research Infrastructure
PRACE, the Partnership for Advanced Computing in Europe, ispreparing the creation of a persistent, pan-European Research
Infrastructure that will provide a world-class HPC service and thus
contribute to the advancement of European competitiveness inindustry and research. The objective of the preparatory two year
project is the completion of the necessary legal, administrative, andtechnical work that will allow the permanent Research Infrastructure
to commence operation in 2010. PRACE reached its first major
milestone by successfully completing the EU mid-term review inearly March. This presentation will provide an overview ofachievements of the first year and of the tasks and challenges thatare to come beyond 2009.

Thomas Lippert

Tokyo, 8.10.200944