Aurora
Leading HPC into the Future
Dr. Robert W. Wisniewski

Chief Architect, HPC, Intel
Aurora Technical Lead and PI

Notices & DisclaimersIntel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending onsystem configuration.
No product or component can be absolutely secure.
Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. For
more complete information about performance and benchmark results, visit http://www.intel.com/benchmarks .
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark andMobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary.
You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product whencombined with other products. For more complete information visit http://www.intel.com/benchmarks .
Intel Advanced Vector Extensions (Intel AVX) provides higher throughput to certain processor operations. Due to varying processor power characteristics, utilizing AVX
instructions may cause a) some parts to operate at less than the rated frequency and b) some parts with Intel® Turbo Boost Technology 2.0 to not achieve any or maximum
turbo frequencies. Performance varies depending on hardware, software, and system configuration and you can learn more at http://www.intel.com/go/turbo.
Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. Theseoptimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any
optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certainoptimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more
information regarding the specific instruction sets covered by this notice.
Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs andprovide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.
Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether
referenced data are accurate.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the propertyof others.

Three pillars of the Exascale era
HPC Simulation
Model Drives Data

Data Analytics
Data Drives Insight

Data StoreVisualization
3

Artificial Intelligence
Model Inferred from Data

Artificial Intelligence/Deep Learning brings exciting newtechnology to accelerate progress"Predicting Disruptive Instabilities in Controlled Fusion Plasmas throughDeep Learning”
NATURE: (accepted for publication, Jan. 2019, published, April 17, 2019 – DOI: 10.1038/s41586-019-1116-4 )Princeton’s Fusion Recurrent Neural Network code (FRNN) uses convolutional & recurrent
neural network components to integrate both spatial and temporal information for predicting
disruptions in tokamak plasmas with unprecedented accuracy and speed on top supercomputers4

Converged workloads benefit from a tightly-coupled“Data Centric” architecture
Tomorrow
AI/ML

SIM

HPDA

(communication through thin linearized pipe to filesystem)

SIMToday(interactive workflows via tightlycoupled, high-bandwidth, active sharing of program
data objects)

DAOS, NVM,

HDD

5

AI/MLHPDA

New ArchitectureAurora at a Glance

6

Aurora at a Glance

7

Aurora System Architecture8

core system software HPC Components•mOS– Scalable operating system•Unified Control System– Unified, Productive (single pane of glass), Reliable•

MPI– Scalable, high performance, topology optimized

GEOPM

•PMIx
– Process management with “Instant On”•DAOS– Distributed Asynchronous Object Store
9SN

CN
CN
CN
CN

row/rack
controller1 A

SN
SN

Controlnetwork– Global Extensible Open Power Manager

HPCNetwork•

Operation Data

row/rack B
controller2.
.
row/rack.
Q

controllern

History Data

Data
NetworkSMC

Inventory Data

Configuration
Data
RAS Data
Service andDiagnostic DataDisaggregated High-Performance Storage Using DAOSDAOS Nodes (DNs)

Xeon® servers
Storage-class memory and NVMe attached storage
DAOS service

High-PerformanceFabric

System Service Nodes
Login Nodes

External Parallel File
System(s)Lustre, GPFS, …
Gateway Nodes (GNs)

Xeon servers with no local storage
IO forwarding service and data mover
10

DAOS: Distributed Asynchronous Object Storage
Scale-out object store built from the groundup for massively distributed NVM storage
3rd Party
Applications

Data Model Library▪ Ultra-fine grained I/OgRPCDAOS Storage EngineStorage
PlatformStorage
Media11OFIOpen Source Apache 2.0 License
PMDKIntel®
NVRAM
Optane™

SPDKNVMe
NVMeNVMe▪ Built over new user space
PMEM/NVMe software stack▪ High throughput/IOPS at arbitraryalignment/size

WorkflowRich DataModels

DAOS Benefits

HDD

Management
I/O▪ Scalable communication and I/Oover homogenous, shared-nothing
servers▪ Software-managed redundancy– Declustered replication anderasure code with self healing

OpenHPC
GNU
Linux

RRV
RRV

Contributors
include Intel,OEMs, ISVs,labs, academia

OEMCommunityUniversity
Stack

RRV
Parallel File system
Resource Manager RRVUpstream
UpstreamsourceUpstream
sourceCommunities
UpstreamsourceCommunities
sourceCommunities

Communities

RRVs

Facilitates a vibrantand efficient software
ecosystem•Eases HPC application
development•Simplifies systemadministration andmaintenance•Extends to newworkloads (AI and BD)•

Allows users toquickly takeadvantage ofhardware innovation

UniversityCommunity

OEMStack

Integrates andtests HPC stacks
and makes themavailable•

PROJECTBase
HPC StackCadence: ~quarterly

Continuous Integration Environment-Build Environment & Source Control-Bug Tracking-User & Dev Forums-Collaboration tools-Validation Environment“RRV” = Relevant and Reliable Version

Converged workloads benefit from a tightly-coupled“Data Centric” architecture
Tomorrow
AI/MLSIM

HPDA(communication through thin linearized pipe to filesystem)

SIMToday(interactive workflows via tightlycoupled, high-bandwidth, active sharing of program
data objects)

DAOS, NVM,

HDD

13

AI/ML

HPDA

New Architecture

Bringing Spark Analytics to Exascale
Applications / Workloads

1. Port workloads to Spark
2. Integrate with cluster resource management (in Spark Job Scheduler)
3. Support NUMA Aware Task Scheduling (in Spark Task Scheduler)

Spark

4. Support DAOS as intermediate data storage (in Spark BlockManager)

JVM

Cluster Resource
Management.

5. Support high performance fabric (in Spark Shuffle)
6. Support kernel offloading to new hardware (in Spark MLlib)

ComputeNetworkStorage

7. Support DAOS as input/output storage (in Spark DataSource)

Bring Spark analytics capability to Exascale
Leverage new hardware and high performance fabric to achieve great performance14

Summary of Heterogeneity Trends▪ Heterogeneity is all around us– Compute, memory, I/O, software ecosystem▪ New types of compute requirements– AI, Big Data, Edge
▪ AI and Cloud are large markets and thus primary drivers ofrequirements▪ HPC is more complex than ever and fundamental shifts are occurring

15Addressing Challenges Raised by Trends▪ System design methodology needed▪ Leverage massive investment by cloud and AI, but optimize for HPC– ex: GPUs
▪ Integrate heterogeneous components at the right level▪ Provide a programming model encompassing expanding compute– Scalar, Vector, Matrix, Spatial, Mixed Precision, and Edge → HPCmachines▪ Provide scalable software that supports new data models▪ Facilitate platforms for converged HPC, AI, and Big Data computing
16

AI/ML

HPDA

SIM

Arigato Gozaimashita