THE EUROPEAN APPROACH FOR EXASCALE AGESTHE ROAD TOWARD SOVEREIGNTY

Jean-marc.denis@European-processor-initiative.eu
Chairman of the Board

THIS PROJECT HAS RECEIVED FUNDING FROM THE EUROPEAN UNION’S HORIZON 2020 RESEARCH AND INNOVATION PROGRAM UNDER GRANT
AGREEMENT NO 826647

Copyright © European Processor Initiative 2020.

2

AGENDA

EPI in European HPC strategy(post)-Exascale supercomputers specifications(Post) Exascale processor specifications
Wrap-Up
Copyright © European Processor Initiative 2020.

3

EPI IN EUROPEAN HPC STRATEGY

Copyright © European Processor Initiative 2020.

5

CONTEXT
European Commission President
Jean-Claude Juncker
Paris, 27 October 2015

« Our ambition is for Europe to become oneof the top 3 world leaders in highperformance computing by 2020 »Creation of the European processor
Initiative
23 members from 10 EU countriesàà

General Purpose processor in 2022
Accelerator IP

Brussels, 1 Dec 2017

Vice President
Andrus Ansip« I encourage even more EU countries to engage in thisambitious endeavour »Digital Day Rome, 23 March 2017

Ministers from seven MS (France, Germany, Italy,
Luxembourg, Netherlands, Portugal and Spain) sign adeclaration to support the next generation of computing
and data infrastructures

Copyright © European Processor Initiative 2019.

Ursula Von Der Lyen
State of the Union

Brussels – September, 16th, 2020
- Investment of 8 billion euros in the next
generation of supercomputers - cuttingedge technology made in Europe.
- The European industry will develop ourown next-generation microprocessor

6EUROHPC JOINT UNDERTAKING (JU) – CONTEXT¡ The European High Performance Computing Joint Undertaking (EuroHPC JU) is

pooling European resources to buy and deploy top-of-the-range supercomputersand develop innovative exascale supercomputing technologies and applications.

¡ It aims to improve quality of life, advance science, boost industrial
competitiveness, and ensure Europe’s technological autonomy.
¡ The JU is currently supporting two main activities (2020-2021):¡

Developing a pan-European supercomputing infrastructure: 3 pre-exascale supercomputers (aim to be among
the top 5 WW), and 5 petascale supercomputers. Benefit European private and public users, working in academia
and industry, everywhere in Europe.

¡

Supporting research and innovation activities: developing a European supercomputing ecosystem, stimulating atechnology supply industry (from low-power processors to software and middleware, and their integration into
supercomputing systems), and making supercomputing resources in many application areas available to a large
number of public and private users, including small and medium-sized enterprises.

Copyright © European Processor Initiative 2020.

7

RECENT NEWS FROM EUROHPC

Copyright © European Processor Initiative 2020.

>1,00038
TBD

PflopsPflopsEFlopsPFLOPSYear

Country

MachineSupplier

Finland

LUMI

HPE

550

2021

Italy

Leonardo

ATOS

248

2021

Spain(*)

MareNostrum5

TBD

>200

2021

Luxembourg

MeluXina

ATOS

10

2021

Portugal

Deucalion

ATOS
Fujitsu

102021

CZ Rep

IT4I (name tbd)

HPE

15,2

2021

Bulgaria

NCSA

ATOS

6

2021

Slovenia

Vega

ATOS

6,8

2021

TBD (DE?)

TBD(**)

TBD>1,0002023

TBD (FR?)TBD(**)

TBD>1,0002024(*) announced on Dec. 15th at the earliest(**) with EU processor based on EPI developments)

8

EPI OBJECTIVES¡ Overall: Develop a complete EU designed high-end

microprocessor, addressing Supercomputing and edge-HPC
segments. Short-term objective¡ supply the EU-designed microprocessor to empower the EU
Exascale machines¡ Long-term objective¡ Europe needs a sovereign (=not at risk of limitation or embargo by

non-EU countries) access to high-performance, low-power
microprocessors, from IP to products¡ EPI has been set to fulfil this objective¡ EPI has to cover all Technical Readiness levels (TRL)¡ TRL 1-3 are for long-term objectives (EU IP)*and*¡ TRL 4-9 are for short to mid-term objectives (decade) with products designedin EU

Copyright © European Processor Initiative 2020.

9

27 PARTNERS FROM 10 EU COUNTRIES

Copyright © European Processor Initiative 2020.

10

FROM IPR
FROM EPI¡ SIPEARL is

TO PRODUCTS
TO SIPEARL¡ Incorporated in EU (France)¡ the industrial and business ‘hand’ of EPI¡ the Fabless company¡ licence of IPs from the partners¡ develop own IPs around it¡ licence the missing components from the market¡ Raise in equity the missing budget (~100M€)¡ generate revenue from both the HPC, IA, server and eHPC

markets¡ integrate, market, support & sell the chip¡ work on the next generations
Copyright © European Processor Initiative 2020.

11

OVERALL ROADMAP
Dec. 20182022

2019 à 2020 à 2021 à
Petascale
Pre-exascale

Short term
objective

>2023

Exa

Exa

General purpose processor(Gen1)ARM ZEUS Core

GPP (Gen2)ARM >ZEUS Core

TRL >=6

CRONOS

RHEA
Common Platform

mid term
objective

2023

EPAC IP development& Test chip (22nm)

7nm

GA

Gen2 development

GA

EPAC based
Accelerator

TRL <4

Copyright © European Processor Initiative 2020.

12

(POST)-EXASCALE SUPERCOMPUTERSSPECIFICATIONS

Copyright © European Processor Initiative 2020.

13

HPC BEFORE ARTIFICIAL INTELLIGENCE

Theoretical model

Copyright © European Processor Initiative 2020.

HPC Application

Results

14

HPC WITH ARTIFICIAL INTELLIGENCE
Theoretical model

HPC Application

Results

Results

Big Data

Copyright © European Processor Initiative 2020.

AI

15

HPC & AI AT EXASCALE: IT’S ALL ABOUT WORKFLOWS (1/3)
Edge
Computing

Edge
Computing

X86
Public Clouds

Realtime dataprocessing

Realtime dataprocessing

Theoritical model

SUMMITX86

HPC Application

FUGAKUsunway taihulight
Results

Results

Big Data

Copyright © European Processor Initiative 2020.

AI

16

HPC & AI AT EXASCALE: IT’S ALL ABOUT WORKFLOWS (2/3)
Edge
Computing

Edge
ComputingRealtime dataprocessing

Europeanapproach?

Legacy X86
Copyright © European Processor Initiative 2020.

Public Clouds

Realtime dataprocessing

ü EU Architectureü EU HWü EU SWü Handle all complexity the others don’t know how to

RHEA (ARM)

RHEA +ACC.
(ARM+Risc-V)

RHEA + GPU(ARM+Nvidia/AMD)

RHEA + ***(ARM+Common Platform)
17

HPC & AI AT EXASCALE: IT’S ALL ABOUT WORKFLOWS (3/3)
Edge
Computing

Edge
ComputingPublic Clouds

Realtime dataprocessing

Realtime dataprocessing

Europeanapproachü EU Architectureü EU HWü EU SWü Handle all complexity in a unified environment

Unified environment: Compilers / programming languages / libraries
CPU

CPU

CPU

Legacy X86

RHEA (ARM)

ACC. A

ACC. BRHEA +ACC(ARM+Risc-V)

Copyright © European Processor Initiative 2020.

ACC. CCPURHEA + GPU(ARM+Nvidia/AMD)

ACC. DCPURHEA + ***(ARM+Common Platform)

ACC. E

18

TAKEWAY #1
Exascale and post-Exascale
superpercomputers will modular.
They’ll have massively nonhomogenous architectures, combining
one general purpose processor withseveral different accelerator kinds

Copyright © European Processor Initiative 2019.

19

TAKEWAY #2
Software will play an even more
important role as a unification layer
between all technologies, between all
modulesà Opensource and standardization

are more important than ever

à Proprietary SW stacks, especiallyfor specialized HW will become
problematic

Copyright © European Processor Initiative 2019.

20

(POST) EXASCALE PROCESSORSPECIFICATIONS

Copyright © European Processor Initiative 2020.

21

TAKE AWAYS #1 & #2 CONSEQUENCES
General Purpose Processors
have to be (much) more open

The race to FLOPS is now in theaccelerators area

GPU A

...

FPGA

GPP
Quantum

Neuro

CryptoCopyright © European Processor Initiative 2020.

GPU B

22

TAKEWAY #3
SPECIFICATION FOR EXASCALE GENERAL PURPOSE PROCESSORS (PART 1)¡ Need extreme flexibility and performances on

external links¡

Composable architectures, from 1 to >=4 sockets (CCIX
and/or CXL)¡HBM 2e / 3¡*and* DDR 5¡*and* PCIe G5¡*and* CXL

¡ Transparent integration in end-to-end dataflow : IoT

ßà Edge ßà Datacenter ßà Cloud¡Easy to port / optimize¡

Opensource tools¡

Unified development tools

Copyright © European Processor Initiative 2019.

23

THE R_PEAK CHALLENGE – CPU VS. GPU
CPU and GPU

0,70

European Processor

à CPU cannot compete against GPU in the race to R_PEAK(meaning R_MAX)à Future CPU have to be designed for maximizing the efficiency

0,50

0,40

Question: why does Byte/Flop mean CPU efficiency?

CPU
area

Bytes per Flops – higher is better

0,60

0,30

0,20

GPU area

0,10

0,00
0

10 000

20 00030 00040 00050 00060 00070 00080 000

R_Peak (GFLOPS) – higher is better
Copyright © European Processor Initiative 2019. Event/Recipient/Place/Date

24

0,60

THE R_PEAK CHALLENGE – FOCUS ON CPU
Unbalanced
Too much memory BW for available
performance2011 - Fujitsu - SPARC64 VIIIfx 8C…
2010 - Intel - westmere

Bytes per Flops – higher is better

0,502009 - Intel - nehalem
0,402012 - Intel - KNC
2017 - AMD - Naples AVX2
2018 - Cavium - ARM ThunderX22017 - AMD - Naples TDP
2012 - Intel - ivy bridge

2011 - Intel - sandy bridge
0,30

Balanced system
2019 - Fujitsu - ARM - A64FX

0,20

2015 - Intel - broadwell2013 - Intel - haswell

0,10Unbalanced
Too much performance for available memory BW
2017 - Intel - skylake AVX512
2017 - Intel - skylake TDP

2019 - AMD - Rome2016 - Intel - KNL AVX

2016 - Intel - KNL TDP

0,00
0

500

Copyright © European Processor Initiative 2020.

1 000

1 500

2 000

R_Peak (GFLOPS) – higher is better

2 500

3 000

3 500
25

(Higher is better)

7,00%

6,00%

HPGC Efficiency vs. R_Peak2011 - K
5,00%2019 - Astra - Apollo 70

4,00%

3,00%

2019 - Trinity – Cray XC40
2020 - Fugaku
2015 - Shaheen II - Cray XC40

2,00%

1,00%

2012 - SuperMUC - iDataPlex DX360M4

2012 - Sequoia – BlueGene/Q

2016 - Oakforest-PACS - PRIMERGYCX1640 M1
2012 - Titan - Cray XK72018 - SuperMUC-NG - ThinkSystem
SD650
2016 - Sunway TaihuLight - Sunway MPP

0,00%

0,00000,10000,20000,30000,4000

Bytes per Flops

0,50000,60000,7000(Higher is better)

(Higher is better)

7,00%

Why mem bandwidth and Bytes/Flops matter?
JUST GET MORE FROM YOUR EXTRA EXPENSIVE SUPERCOMPUTER
6,00%

HPGC Efficiency vs. R_Peak2011 - K
5,00%2019 - Astra - Apollo 70

4,00%

3,00%

2019 - Trinity – Cray XC40
2020 - Fugaku
2015 - Shaheen II - Cray XC40

2,00%

1,00%

2012 - SuperMUC - iDataPlex DX360M4

2012 - Sequoia – BlueGene/Q

2016 - Oakforest-PACS - PRIMERGYCX1640 M1
2012 - Titan - Cray XK72018 - SuperMUC-NG - ThinkSystem
SD650
2016 - Sunway TaihuLight - Sunway MPP

0,00%

0,00000,10000,20000,30000,4000

Bytes per Flops

0,50000,60000,7000(Higher is better)

(Higher is better)

7,00%

Why mem bandwidth and Bytes/Flops matter?
JUST GET MORE FROM YOUR EXTRA EXPENSIVE SUPERCOMPUTER
6,00%

2023 - European Supercomputer …

HPGC Efficiency vs. R_Peak2011 - K
5,00%2019 - Astra - Apollo 70

4,00%

3,00%

2019 - Trinity – Cray XC40
2020 - Fugaku
2015 - Shaheen II - Cray XC40

2,00%

1,00%

2012 - SuperMUC - iDataPlex DX360M4

2012 - Sequoia – BlueGene/Q

2016 - Oakforest-PACS - PRIMERGYCX1640 M1
2012 - Titan - Cray XK72018 - SuperMUC-NG - ThinkSystem
SD650
2016 - Sunway TaihuLight - Sunway MPP

0,00%

0,00000,10000,20000,30000,4000

Bytes per Flops

0,50000,60000,7000(Higher is better)SO… WHAT TO EXPECT FROM AN EXASCALE GENERAL PURPOSE PROCESSOR? (PART 2)¡ World class manufacturing process (7nm or better)¡ Need extreme flexibility and performances on external links¡

Composable architectures, from 1 to >=4 sockets (CCIX and/or CXL) HBM 2e / 3¡*and* DDR 5¡*and* PCIe G5¡*and* CXL

¡ Transparent integration in end-to-end dataflow : IoT ßà Edge ßà Datacenter ßà Cloud¡Easy to port / optimize¡

Opensource tools¡

Unified development tools¡ Need “good enough” FP64 performances.

No need to compete with specialized devices like GPUs

¡ Change the metric. Need much better byte/Flop ratio than today

Copyright © European Processor Initiative 2020.

29

GPP ARCHITECTURE CONCEPT
PCIe gen5
links

HSL
links

D2D links
to adjacent chiplets
ZEUS

ZEUS

ZEUS

MPPA

ZEUS

ZEUS

eFPGA

EPAC

FPGA

FPGA

HBM
memories

DDR
memories

EPAC
Copyright © European Processor Initiative 2020.

¡

ARM-ZEUS GPP core¡

EPAC - EPI Accelerator (TITAN)¡

MPPA - Multi-Purpose Processing Array¡eFPGA or FPGA - embedded FPGA¡

Cryptographic ASIC (EU Sovereignty)

30

CONCEPT OF COMMON PLATFORM : INTERPOSER & MULTI-CHIPLET

HBM

HBM

HBM

HBM

Chiplet #2
Chiplet #1

Silicon interposer (passive)

Package substrate

Copyright © European Processor Initiative 2020. 2nd R-CCS Int'l workshop/Kobe/2020-02-18

31

CONCEPT OF COMMON PLATFORM : INTERPOSER & MULTI-CHIPLET

CPU
Accelerator(s)

HBM

HBM
HBM

Chiplet #1

Chiplet #2

HBM

Silicon interposer (passive)

Package substrate

Copyright © European Processor Initiative 2020. 2nd R-CCS Int'l workshop/Kobe/2020-02-18

32

HETEROGENEOUS INTEGRATION
I/O coherent
Accelerators¡ Integrating customized

functions at different levels

Traditional GP-GPU, interconnect
and to the storage

PCIe links¡ EPI accelerator IPs today are

Package substrate

DDR
memories

HBM
memorires

integrated in Rhea designinter-chiplet links
over substrate

EST Common
Platform Design

Coherent
chiplet

CCIX links via PCB

Coherent
accelerator
Copyright © European Processor Initiative 2020. 2nd R-CCS Int'l workshop/Kobe/2020-02-18

Possible location
for User functions
Commerical FPGA chips
or CCIX enabled companions chips(accelerator, NIC or other EST chips)
33

SO… WHAT TO EXPECT FROM AN EXASCALE GENERAL PURPOSE PROCESSOR? (PART 3)¡ World class manufacturing process (7nm or better)¡ Need extreme flexibility and performances on external links¡

Composable architectures, from 1 to >=4 sockets (CCIX and/or CXL) HBM 2e / 3¡*and* DDR 5¡*and* PCIe G5¡*and* CXL

¡ Transparent integration in end-to-end dataflow : IoT ßà Edge ßà Datacenter ßà Cloud¡Easy to port / optimize¡

Opensource tools¡

Unified development tools¡ Need “good enough” but excellent FP64 performances.

No need to compete with specialized devices like GPUs

¡ Need much better byte/Flop ratio than today¡ Based on multi-die and IO-DIE building blocks, combined at package level
Copyright © European Processor Initiative 2020.

34

WRAP-UP

Copyright © European Processor Initiative 2020.

35

FINAL TAKEAWAYS
Dec. 20182019 à 2020 à 2021 à

2022

Petascale
Pre-exascale

Short term
objective

>2023

Exa

Exa

General purpose processor(Gen1)ARM ZEUS Core

GPP (Gen2)ARM >ZEUS Core

TRL >=6

CRONOS

RHEA
7nm

Common Platform

mid term
objective

2023

GA

Gen2 development

GA

EPAC IP development& Test chip (22nm)

EPAC based
Accelerator

TRL <4CPUCPU
RHEA (ARM)

ACC. A

ACC. BCPURHEA +ACC(ARM+Risc-V)

ACC. C

Exascale and post-Exascale
supercomputers will modular.
They’ll have massively nonhomogenous architectures,combining one general purpose
processor with several different
accelerator kinds

Copyright © European Processor Initiative 2020.

CPU

RHEA + GPU(ARM+Nvidia/AMD)

ACC. D¡ Need extreme flexibility and performances on external links¡

Composable architectures, from 1 to >=4 sockets (CCIX and/or CXL) HBM 2e / 3¡*and* DDR 5¡*and* PCIe G5¡*and* CXL

¡ Transparent integration in end-to-end dataflow

Unified environment: Compilers / programming languages / libraries
Legacy X86¡ World class manufacturing process (7nm or better)CPU

RHEA + ***(ARM+Common Platform)

ACC. E

Software will play an even more
important role as a unification
layer between all technologies,
between all modules
à Opensource andstandardization are more
important than ever
à Proprietary SW stacks,
especially for specialized
HW will become
problematic

IoT ßà Edge ßà Datacenter ßà Cloud¡Easy to port / optimize¡

Opensource tools¡

Unified development tools¡ Need “good enough” but excellent FP64 performances.

No need to compete with specialized devices like GPUs

¡ Need much better byte/Flop ratio than today¡ Based on multi-die and IO-DIE building blocks, combined at package level38

THANK YOU FOR YOUR ATTENTION

w www.european-processor-initiative.eu@EuProcessor
European Processor Initiative
European Processor Initiative

Copyright © European Processor Initiative 2020.

43