AMD EPYC CPU PLATFORM FOR H
P
Cjames.chen@amd.com

 [AMD Public Use]







 [AMD Public Use]

Use of third-party marks / logos/ products is for informational purposes only and no endorsement of or by AMD is intended or implied. GD-83

 [AMD Public Use]

1.5 exaflops (ORNL in 2021)

2.0 exaflops (LLNL in 2023)

AMD EPYC™ CPUs + HBM-enabled Radeon Instinct™ GPUs

AMD EPYC™ CPUs + HBM-enabled Radeon Instinct™ GPUs

Reference: https://www.amd.com/en/products/exascale-era [AMD Public Use]

https://www.olcf.ornl.gov/wp-content/uploads/2019/05/frontier_specsheet_v4.pdf

 [AMD Public Use] University of Edinburgh’s supercomputing center (EPCC)

 High-Performance Computing Center Stuttgart (HLRS)

 28PF, EPYC 7H12

 26PF, EPYC 7742

 General-purpose machine for R&D in the UK

 General-purpose machine for R&D and industry.

References
https://www.nextplatform.com/2019/10/18/amd-cpus-will-power-uks-next-generation-archer2-supercomputer/https://www.hlrs.de/systems/hpe-apollo-hawk/ [AMD Public Use]

 Lumi in CSC’s data center in Kajaani, Finland (2021)
 552PF AMD EPYC™ CPUs + HBM-enabled Radeon Instinct™

 Phase 1 in Q3 2021; Full system in Q2 2022

References

https://www.lumi-supercomputer.eu/lumi-one-of-the-worlds-mightiest-supercomputers/
https://pawsey.org.au/powering-the-next-generation-of-australian-research-with-hpe/ [AMD Public Use]

SEE ENDNOTES MI100-03

 [AMD Public Use]

HPC
AI

SEE ENDNOTES, MI100-04, MI100-05,
MI100-16 [AMD Public Use]

Roadmap Subject to Change [AMD Public Use] [AMD Public Use]∞∞

Multi-die Multi-Chip Module

m

DD
R

DD
R

∞
∞

Monolithic die

∞
∞

7nm
CPU
die

DD
R

I/O

7nm
CPU
die

DD
R

14nm
I/O DIE

I/O

7nm
CPU
die

DD
R

∞
∞

7nm
CPU
die

DD
R

DD
R

Chiplet

DD
R

7nm
CPU
die
7nm
CPU
die
7nm
CPU
die
7nm
CPU
die

 [AMD Public Use] [AMD Public Use]COMPUTE
Up to 2X AMD “Zen” x86 cores(up to 64 cores/128 threads)

PERFORMANCE
8 Cores+ L3

8 Cores+ L3

8 Cores+ L3

8 Cores+ L3

Up to 4X shared L3 cache (256MB)
Up to 2X L3 cache per core
(16MB per 4 cores)
Reduced System Diameter(NUMA domain)

TDP range: 120W-280W*

INTEGRATED I/O – NO CHIPSET
128 lanes PCIe® 3 & 4*
DDR4
MemoryControllers

PCIe3/4
SATA3

ServerController
Hub

AMD Secure
Processor

MEMORY

2 DIMMs/channel capacity of 4TB/socket− Used for PCIe, SATA, andCoherent Interconnect− Up to 32 SATA or NVMe devices

SECURITY

8 channel DDR4 with ECC
up to 3200 MHz*RDIMM, LRDIMM, 3DS, NVDIMM~4x Peak TFLOPS/Socket~2X increased perf/socket

Dedicated Security Subsystem
8 Cores+ L3

8 Cores+ L3

8 Cores+ L3

8 Cores+ L3

Hardware Root-of-Trust

Additional Security Features

 SME, SEV, SEV-ES* A MOTHERBOARD DESIGNED FOR 2ND GEN EPYC PROCESSORS IS REQUIRED TO ENABLE ALL FUNCTIONALITY. SEE ENDNOTE
ROM-06SEE ENDNOTES ROM-04, 07, 342

 [AMD Public Use]

Dual-socket AMD 7742 CPUs per node
Rpeak = 2 x # of CORES x Base Freq x 16 (AVX2 DP FLOPS)
Rpeak = 2 x 64 x 2.25 GHz x 16 = 4,608 GFLOPS
Rmax = 4,000 GFLOPS (HPL 2.3 output)
Reference: https://www.amd.com/en/products/cpu/amd-epyc-7742#product-specs

 Is HPL a good metric?

 [AMD Public Use]

ADVANCED FEATURES CONSISTENT ACROSS SIMPLIFIED PRODUCT STACK
32
24

48/64 CORE

CORE

CORE

12/16
CORE

8 COREAMD EPYC™

AMD EPYC™
AMD EPYC™

All-in feature set includes:
8 Channels of DDR4-3200*
4TB memory capacity
128 lanes PCIe® 4*
SMT & Turbo boost
18G Infinity Fabric™
Secure Memory Encryption
Secure Encrypted
Virtualization

7F72

7F52

7F32



* A MOTHERBOARD DESIGNED FOR 2ND GEN EPYC PROCESSORS IS REQUIRED TO ENABLE ALL
FUNCTIONALITY. SEE ENDNOTE ROM-06

 [AMD Public Use]

•
•
•
•

Core-countL3 cache
8 memory channels per CPU
PCIe gen 4

NOTE: This guide is designed to provide a highlevel workload assessment based on AMD
engineering estimates. Performance varies
depending on system configuration.

 [AMD Public Use]

31
10** INCLUDES TIES WHICH ARE NOT COUNTED IN THE AMD WORLD RECORDS TOTAL NUMBER
World records as of June 9, 2020. See AMD.com/worldrecords for details

 [AMD Public Use]EPYC™ 7742 versus Xeon® 8280. SEE ENDNOTES ROM-63, ROM-113, ROM-56, ROM-49, ROM-42

 [AMD Public Use]

Exceptional scaling of this magnitude can be attributed to cache effects fitting more of the model into the processor cache.

Reference: https://www.amd.com/system/files/documents/EPYC-7002-Weather-Modeling-with-WRF.pdf

 [AMD Public Use]

The highest super-linear scalability for multi-node calculation is due to the benefit of cache memory advantage.

Reference: https://www.amd.com/system/files/documents/AMD-EPYC-7002-OpenFOAM.pdf

 [AMD Public Use]

Reference: https://www.amd.com/system/files/documents/AMD-EPYC-7002-AnsysFluent-Scalability-Performance.pdf [AMD Public Use]

AOCC (AMD Optimizing C/C++ Compiler)

CPU LibrariesµProf (Performance & Power Profiling Tools)

Use of third party marks / logos/ products is for informational purposes only and noendorsement of or by AMD is intended or implied.

 [AMD Public Use]AMD EPYC™ PROCESSOR COMPILER SUPPORTAN OPEN SOURCE STRATEGY

Performance
Compilers

AOCC

AMD Optimizing C/C++ Compiler – Built on LLVM and Optimized for
EPYC Processors
Java Compiler and Runtime - AMD contributes enablement and optimization to
OpenJDK; Oracle then incorporates in their releases
AMD is actively involved in the LLVM community and will upstream
EPYC processor optimizations implemented in AOCC

Platform
CompilersWidely used Open Source compiler - AMD upstreams processor
enablement and tuning for EPYC

Widely used Windows compiler – AMD contributes processor enablement andtuning for EPYC
Can utilize other popular compilers on AMD platforms.
https://www.aravision.com/products/pgi/pdf/pgirn.pdf
PGI, Intel & Others
https://www.amd.com/system/files/documents/compiler-options-guideamd-epyc-7xx1-series-processors.pdf
Use of third party marks / logos/ products is for informational purposes only and no

Ecosystem
Compilers

endorsement of or by AMD is intended or implied.

 [AMD Public Use]

 Read the HPC tuning guide https://developer.amd.com/wp-content/resources/56827-1-0.pdf

 Review the hardware specification
 Tune the BIOS parameters Compile the program with the right flags and check the settings of math libs

 Map the process with hardware architecture [AMD Public Use] OPENMP (1-node Dual-7742)
 export OMP_NUM_THREADS=32
 export OMP_BIND_PROC=true

 export GOMP_CPU_AFFINITY=0-127:4
 OPENMPI+OPENMP (1-node Dual-7742)
 export OMP_NUM_THREADS=4
 mpirun -np 32 --bind-to l3cache My_PROGRAM
 Reference: https://developer.amd.com/wp-content/resources/56827-1-0.pdf

 [AMD Public Use]









https://www.amd.com/en/processors/epyc-cpu-selector
https://www.amd.com/en/products/specifications/
https://developer.amd.com/resources/epyc-resources/
https://developer.amd.com/amd-aocc/https://developer.amd.com/amd-aocl/
https://www.amd.com/en/processors/server-tech-docs/
 https://prace-ri.eu/wp-content/uploads/Best-Practice-Guide_AMD.pdf
 https://www.dell.com/support/article/en-sg/sln319015/amd-rome-is-it-for-real-architecture-and-initialhpc-performance?lang=en
 https://hpcadvisorycouncil.atlassian.net/wiki/spaces/HPCWORKS/pages/1280442391/AMD+2nd+Gen+E
PYC+CPU+Tuning+Guide+for+InfiniBand+HPC

 [AMD Public Use] [AMD Public Use]

EPYC-07 - Based on June 8, 2018 AMD internal testing of same-architecture product ported from 14 to 7 nm technology with similar implementation flow/methodology, using performance from
SGEMM.



ROM-04 Based on standard calculation method for determining FLOPS. ROM-04

ROM-06 - Some supported features and functionality of 2nd Gen AMD EPYC™ processors require a BIOS update from your server manufacturer when used with a motherboard designed for the
1st Gen AMD EPYC series processor. A motherboard designed for 2nd Gen EPYC processors is required to enable all available functionality.

ROM-07 - Motherboards designed for 1st Gen EPYC processors may not be compatible with 2nd Gen AMD EPYC processors with a TDP greater than 200 watts. Contact the server manufacturer
to confirm compatibility.

ROM-3421-n, 2-socket 2nd Gen EPYC™ 7742 powered server (http://spec.org/cpu2017/results/res2019q4/cpu2017-20191125-20001.pdf) scoring 701 SPECrate®2017_int_base has up to 2.3x theSPECrate® 2017 Integer (Base) performance of the highest previous generation score of 304 SPECrate®2017_int_base by a 1-n, 2-socket 1st Gen EPYC™ 7601 powered server result(http://www.spec.org/cpu2017/results/res2019q2/cpu2017-20190411-11817.html ) as of 1/26/20. ROM-342



ROM-741: Best published SPECrate®2017_int_base performance as of 9/29/2020.
AMD EPYC 2P processor scores reflect highest scoring processor at each available core level (core count is per processor):
AMD EPYC 7742 (64C) score of 701 http://spec.org/cpu2017/results/res2019q4/cpu2017-20191125-20001.pdf
AMD EPYC 7642 (48C) score of 586 http://spec.org/cpu2017/results/res2020q3/cpu2017-20200622-23009.html
AMD EPYC 7542 (32C) score of 437 http://spec.org/cpu2017/results/res2020q3/cpu2017-20200706-23309.html
AMD EPYC 7302 (16C) score of 246 http://spec.org/cpu2017/results/res2020q1/cpu2017-20191220-20441.pdf
AMD EPYC 7262 (8C) score of 135 http://spec.org/cpu2017/results/res2020q2/cpu2017-20200608-22777.html
Intel® Xeon® Scalable 2P processors scores using the highest competitive part in the Bronze, Silver and Gold metal classes:
Intel Xeon Gold 6258R (28C) score of 397 http://spec.org/cpu2017/results/res2020q3/cpu2017-20200915-23981.html
Intel Xeon Gold 5220R (24C) score of 319 http://spec.org/cpu2017/results/res2020q3/cpu2017-20200608-22594.html
Intel Xeon Silver 4216 (16C) score of 193 http://spec.org/cpu2017/results/res2020q3/cpu2017-20200804-23653.html
Intel Xeon Bronze 3206R (8C) score of 60.1 http://spec.org/cpu2017/results/res2020q2/cpu2017-20200609-22881.html
SPEC®, SPECrate® and SPEC CPU® are registered trademarks of the Standard Performance Evaluation Corporation. See w ww.spec.org for more information ROM-741

 [AMD Public Use]

ROM-742: Both published and estimated SPECrate®2017_int_base performance as of March 2, 2020.
AMD EPYC 7742 (64C) score of 355 http://spec.org/cpu2017/results/res2020q1/cpu2017-20191223-20460.pdf
AMD EPYC 7642 (48C) score of 306 http://spec.org/cpu2017/results/res2020q3/cpu2017-20200827-23775.html
AMD EPYC 7542 (32C) score of 223 http://spec.org/cpu2017/results/res2019q4/cpu2017-20191028-19322.pdf
AMD EPYC 7402P (24C) score of 177 http://spec.org/cpu2017/results/res2019q4/cpu2017-20191028-19326.pdf
AMD EPYC 7302P (16C) score of 124 http://spec.org/cpu2017/results/res2019q4/cpu2017-20191125-20005.pdf
AMD EPYC 7262 (8C) score of 68.3 http://spec.org/cpu2017/results/res2020q1/cpu2017-20191220-20437.pdf
Intel® Xeon® Scalable 2P processors scores using the highest competitive part in the Bronze, Silver and Gold metal classes. Estimates are based on highest score 2P processor result and divided by 1.99x for conservative estimate:Intel Xeon Gold 6258R (28C) estimated score of 199 (derived from 2P score of 397/1.99=199) https://spec.org/cpu2017/results/res2020q3/cpu2017-20200915-23981.html
Intel Xeon Gold 5220R (24C) estimated score of 160 (derived from 2P score of 319/1.99=160) http://spec.org/cpu2017/results/res2020q3/cpu2017-20200608-22594.html
Intel Xeon Silver 4216 (16C) score of 87.3 http://spec.org/cpu2017/results/res2019q2/cpu2017-20190524-14372.pdf
Intel Xeon Bronze 3206R (8C) estimated score of 30.2 (derived from 2P score of 60.1/1.99= 30.2)
http://spec.org/cpu2017/results/res2020q2/cpu2017-20200609-22881.html
Results may vary with production system publications. The average of all published Intel results that have a 1P and 2P corresponding number is 2.08x with the maximum being 2.3x and the minimum being 1.95x so conservatively AMD
estimated using 1.99x scaling. AMD expects Intel published numbers in the near future will likely be higher. SPEC®, SPECrate® and SPEC CPU® are registered trademarks of the Standard Performance Evaluation Corporation. See
www.spec.org for more information. ROM-742



ROM-42 - Based on AMD internal testing of ANSYS FLUENT 19.1, lm6000_16m benchmark, as of July 17, 2019 of a 2P EPYC 7742 powered reference server versus a 2P Intel Xeon Platinum 8280
powered server. Results may vary.



ROM-49 - Based on AMD internal testing of LSTC LS-DYNA R9.3.0, neon benchmark, as of July 17, 2019 of a 2P EPYC 7742 powered reference server versus a 2P Xeon Platinum 8280 powered
server. Results may vary.



ROM-56 - Based on AMD internal testing of Altair RADIOSS 2018, T10M benchmark, as of July 17, 2019 using a 2P EPYC 7742 powered reference server versus a 2P Xeon Platinum 8280 powered
server. Results may vary.



ROM-63 - Based on AMD internal testing of ESI VPS 2018.0, NEON4m benchmark, as of July 17, 2019 using a 2P EPYC 7742 powered reference server versus a 2P Xeon Platinum 8280 powered
server. Results may vary.



ROM-113
AMD Internal testing as of 30July2019 of a 2P AMD EPYC 7742 powered reference platform versus a 2P Intel Platinum 8280 powered production server, on GROMACS version 2019.3 benchmark.
Results may vary. ROM-113

 [AMD Public Use] [AMD Public Use]

The information contained herein is for informational purposes only, and is subject to change without notice. Timelines, roadmaps, and/or product release dates shown in theseslides are plans only and subject to change. “Polaris”, “Vega”, “Radeon Vega”, “Navi”, “Zen”, “Naples”, and “Rome” are codenames for AMD architectures, and are not productnames.
While every precaution has been taken in the preparation of this document, it may contain technical inaccuracies, omissions and typographical errors, and AMD is under noobligation to update or otherwise correct this information. Advanced Micro Devices, Inc. makes no representations or warranties with respect to the accuracy or completeness ofthe contents of this document, and assumes no liability of any kind, including the implied warranties of noninfringement, merchantability or fitness for particular purposes, withrespect to the operation or use of AMD hardware, software or other products described herein. No license, including implied or arising by estoppel, to any intellectual property
rights is granted by this document. Terms and limitations applicable to the purchase or use of AMD’s products are as set forth in a signed agreement between the parties or in
AMD's Standard Terms and Conditions of Sale.
The information contained herein is for informational purposes only, and is subject to change without notice. While every precaution has been taken in the preparation of thisdocument, it may contain technical inaccuracies, omissions and typographical errors, and AMD is under no obligation to update or otherwise correct this information. Advanced
Micro Devices, Inc. makes no representations or warranties with respect to the accuracy or completeness of the contents of this document, and assumes no liability of any kind,including the implied warranties of noninfringement, merchantability or fitness for particular purposes, with respect to the operation or use of AMD hardware, software or otherproducts described herein. No license, including implied or arising by estoppel, to any intellectual property rights is granted by this document. Terms and limitations applicable tothe purchase or use of AMD’s products are as set forth in a signed agreement between the parties or in AMD's Standard Terms and Conditions of Sale. GD-18©2020 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo, AMD Instinct, EPYC, Radeon Instinct, ROCm, and combinations thereof are trademarks ofAdvanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies.