組み込みから HPC まで
ARM コアで実現するエコシステム
Shinnosuke Furuya, Ph.D., HPC Developer Relations, NVIDIA
2021/08/26

AGENDAAutomotive
NVIDIA Drive

HPC
NVIDIA Grace CPU

Network
NVIDIA BlueField DPU

2

“Best Places to Work in 2021”“Most Innovative Companies”

GLASSDOORFAST COMPANY“100 Best Companies to Work For”

“50 Smartest Companies”

FORTUNE

MIT TECH REVIEW“World’s Best Performing CEO”

“World’s Best CEOs”

HARVARD BUSINESS REVIEWBARRON’S

Founded in 1993

Jensen Huang, Founder & CEO

19,000 Employees$16.7B in FY213

AUTOMOTIVE
4NVIDIA ORIN
Advanced, Software-defined Platformfor Autonomous Machines
24.5 billion transistors
12 A78 (Hercules) ARM64 CPUs
254 INT8 TOPS - CUDA Tensor Core GPU + DLA
205 GB/s memory bandwidth
4 10Gbps ENET
8K 30 Dec | 4K 60 Enc – H264 / H265 / VP9
4 R52 Lock-step Pairs Integrated Safety Island ASIL-D
Secure key storage
FUSA ASIL-B Chip | ASIL-D Systematic
6

NVIDIA DRIVE ATLANTOPS is the New Horsepower

Fusing Next Generation AI and BlueField
Industry’s First 1,000 TOPS SoC
400 Gbps Networking with Secure Gateway
ASIL-D Safety Island7

HYPERION 8AV PLATFORM
State-of-the-Art Advances for Data Collection,
Development and Testing
2x Orin AV Computer
1x Orin IX Computer
4x Orin + 4x MLNX 3D GT Data RecorderSensor Suite: 8 Cameras [8MP], 4 Fisheyes [3MP], 3 InCabin, 9 Radar, 2 Lidar

8THE FUTURE CAR IS SOFTWARE DEFINED9NVIDIA DRIVE AV

10

HPC
11

GIANT MODELS PUSHING LIMITS OF EXISTING ARCHITECTURE
Requires a New Architecture
System Bandwidth Bottleneck

100 TRILLION PARAMETER MODELS BY 2023

HBM2eDDR4Model Size (Trillions of Parameters)

GPU

1000
100GPU

10x86

10.10.01
0.001
0.00010.000012018

GPUGPT-3 (175B)
Megatron-LM
Turing-NLG(8.3B)(17.2B)
GPT-2T5 (11B)(1.5B)
BERT-Large(340M)
ELMo (94M)

2019

2020

2021

GPU

2022

2023

GPU

8,000

GB/secCPU

200

GB/secPCIE Gen4 (Effective Per GPU)

16

GB/secMem-to-GPU

64

GB/sec12

ANNOUNCING NVIDIA GRACE
Breakthrough CPU Designed for Giant-Scale AI and HPC Applications

FASTEST INTERCONNECTS>900 GB/s Cache Coherent NVLink CPU To GPU (14x)>600GB/s CPU To CPU (2x)HIGHEST MEMORY BANDWIDTH>500GB/s LPDDR5x w/ ECC>2x Higher B/W
10x Higher Energy Efficiency

NEXT GENERATION ARM NEOVERSE CORES>300 SPECrate2017_int_base est.
Availability 2023

13TURBOCHARGED TERABYTE SCALE ACCELERATED COMPUTINGEvolving Architecture For New Workloads
CURRENT x86 ARCHITECTURE

INTEGRATED CPU-GPU ARCHITECTURE
HBM2e

DDR4

HBM2e

LPDDR5xGPU

GRACE

GPUGPUGRACEGPU

3 DAYS FROM 1 MONTH
Fine-Tune Training of 1T Modelx86
GPUGPUGRACEGPUGRACEGPUREAL-TIME INFERENCE
ON 0.5T MODEL

Interactive Single Node NLP Inference
GPU

8,000

GB/secGPU

8,000

GB/sec

CPU200GB/sec

CPU500GB/secPCIE Gen4(Effective Per GPU)

16GB/secNVLink500

GB/secMem-to-GPU

2000

GB/secMem-to-GPU

64

GB/secTransfer 2TB in 30 secs

Transfer 2TB in 1 secs

Bandwidth claims rounded to nearest hundred for illustration.
Performance results based on projections on these configurations Grace : 8xGrace and 8xA100 with 4th Gen NVIDIA NVLink Connection between CPU and GPU and x86: DGX A100.
Training: 1 Month of training is Fine-Tuning a 1T parameter model on a large custom data set on 64xGrace+64xA100 compared to 8xDGXA100 (16xX86+64xA100)
Inference: 530B Parameter model on 8xGrace+8xA100 compared to DGXA100.

14

ANNOUNCINGTHE WORLD’S FASTEST
SUPERCOMPUTER FOR AI
20 Exaflops of AI
Accelerated w/ NVIDIA Grace CPU and NVIDIA GPU
HPC and AI For Scientific and Commercial Apps
Advance Weather, Climate, and Material Science

15

NETWORK16INTRODUCING NVIDIA BLUEFIELD-3 DPU
First 400Gb/s Data Processing Unit

Offloads and Accelerates Data Center Infrastructure
Isolates Application from Control and Management Plane
Powerful CPU – 16x Arm A78 Cores
Datapath Accelerator – 16x Cores, 256 Threads
Process Networking, Storage, and Security at 400 Gbps

17

INTRODUCING NVIDIA BLUEFIELD-3 DPU
First 400Gb/s Data Processing Unit

DDR5 MEMORY INTERFACEDATA PATH ACCELERATOR

22 Billion Transistors

CONNECTX-7400Gb/s Ethernet & InfiniBand Connectivity

PCIe GEN 5.0

ARM CORES

400Gb/s Crypto Acceleration
18M IOP/s Elastic Block Storage

ACCELERATIONENGINES300 Equivalent x86 Cores

18

BLUEFIELD DPU GENERATIONSBlueField

BlueField-2

BlueField-3

Port speed

2 x 100Gb/s
InfiniBand and Ethernet

2 x 100Gb/s, 1 x 200Gb/s
InfiniBand and Ethernet

1 x 400Gb/s, 2x200Gb/s
InfiniBand and Ethernet

Performance

Bandwidth: 200Gb/s
DPDK Max Msg Rate:150Mpps
RDMA max msg rate: 200MppsBandwidth: 200Gb/s
DPDK Max Msg Rate: 215Mpps
RDMA max msg rate: 215Mpps

Bandwidth: 400Gb/s
DPDK max msg rate: 250Mpps
RDMA max msg rate: 330Mpps

Modulation

NRZ

NRZ & 50G PAM4

NRZ & 100G PAM4

DDR Channels

DDR4-2400MT/s Dual channels

DDR4-3200MT/s Single channel

2 x DDR5-5600 Interfaces

Max Arm Cores

16 x A72 Arm cores

8 x A72 Arm cores

16 x A78 Arm cores (Hercules)

Embedded ASIC

ConnectX-5

ConnectX-6 Dx

ConnectX-7

Gen3.0 x32 / Gen4.0 x16

Gen4.0 x16

Gen5.0 x32

PCIe19

PLATFORMINFRASTRUCTURECYBER
SECURITY

STORAGEEDGENVIDIA DOCA

Enabling Broad BlueField Partner Ecosystem
DOCA
Software Development Framework for BlueField DPUs

ORCHESTRATIONSECURITY

NETWORKINGSTORAGE

Offload, Accelerate, and Isolate Infrastructure Processing
Support for Hyperscale, Enterprise, Supercomputing andHyperconverged Infrastructure

MANAGEMENTACCELERATION LIBRARIESTELEMETRY

Software Compatibility for Generations of BlueField DPUs
DOCA is for DPUs what CUDA is for GPUs

20

BLUEFIELD-3 USE CASES
Unprecedented Innovation for Modern Data Centers

Cloud Computing

Cyber Security

HPC & AI

Bare-Metal I Virtualized I ContainerizedPrivate I Public I Hybrid Cloud

Distributed Security | NGFW I
Micro-segmentation

Cloud-Native Supercomputing |Accelerated DLRM

Telco & Edge

Data Storage

Media Streaming

Telco Cloud | CloudRAN |Edge Compute

HCI I Elastic Block Storage I
Instance Storage

Visual High Quality I
8K Video I CDN
21BLUEFIELD ENABLES
CLOUD-NATIVE
SUPERCOMPUTING
Multi-Tenancy with Zero-Trust Security
Collective offload with UCC accelerator
Smart MPI progression
User-defined algorithms
1.4X higher application performance

22

NVIDIA DPU ROADMAP
Exponential Growth in Data Center Infrastructure Processing
BlueField-4

64B Transistors
160 SPECint
1000 TOPS
800 Gbps

100X

BlueField-3

22B Transistors
42 SPECint
1.5 TOPS
400 Gbps

10X

BlueField-2

7B Transistors
9 SPECint
0.7 TOPS
200 Gbps

1X

DOCA — ONE ARCHITECTURE

2020

2022
* BlueField-4 product to include opt-in GPU and non-GPU configurations

2024
23

SUMMARY24

3 CHIPS. YEARLY LEAPS. ONE ARCHITECTURE.

25

NVIDIA ECOSYSTEM PLATFORM

GEFORCERIVA

MAXINEMEGATRONMERLIN

cuQUANTUM

CLARA

METROPOLIS

MORPHEUSAERIAL

ISAAC

DRIVE

APPLICATION FRAMEWORKS

PLATFORM SOFTWAREVGPURTX

OMNIVERSEDGXAI ENTERPRISEEGX

HGXFLEET COMMAND

AGX

CHIPS & SYSTEMSGPUCPUDPU

26

SUMMARY•

Tegra SoC has a long history, and that experience has been applied to current Xavier, the next generation Orin, andthe next generation Atlan•

The future car is software defined, and NVIDIA provide whole ecosystem such as DRIVE Hyperion and DGX Systems•

Grace CPU is designed for giant-scale AI and HPC applications•BlueField-3 DPU is the first 400 Gb/s data processing unit

•DOCA enables broad BlueField ecosystem•GPU, CPU and DPU chips make a yearly leaps in one architecture27