FPGAs in HighPerformance Computing
from Data Acquisition to the Cloud
Andrew Putnam
May 25, 2022

Introduction© Microsoft Corporation

Slick new hardware© Microsoft Corporation

Useful hardware© Microsoft Corporation

What makes a public cloud company successful?

© Microsoft Corporation

Innovation in Software vs. Hardware• SW is flexible, but is also SO much bigger• You can’t lead from the bottom• Just look at AMD GPUs vs. nVidia• x86 and Windows aren’t the leaders because they’ve always been the best

• Nobody wants to do throw-away work
• Work needs to (plausibly) span multiple generations© Microsoft Corporation

HPC with the Cloud?
•The idea sounds great•

Pay for compute only when you use it•

When it breaks, it’s someone
else’s problem•

No need to call the realtor /utility company when you
want a bigger machine•

New hardware just shows up.
No retrofits needed.

Why hasn’t Supercomputing moved to the Cloud?
CPUs look largely the same, but…
❑ Top 500 often include specialized accelerators (especially GPUs)❑ Networks are highly specialized, tuned for low-latency, high bandwidth❑ Won’t running virtual machines kill performance?© Microsoft Corporation

Why hasn’t Supercomputing moved to the Cloud?
CPUs look largely the same, but…
❑ Top 500 often include specialized accelerators (especially GPUs)❑ Networks are highly specialized, tuned for low-latency, high bandwidth❑ Won’t running virtual machines kill performance?

Accelerator Integration

FPGAs

CPUs

© Microsoft Corporation

CPUs

FPGAs

CPUs

FPGAs

FPGAs

CPUs

Bump-in-the-wire Architecture

ML
BrainWave

Hardware as a Service
Network Acceleration
Compute AccelerationNIC

40G Ethernet

AccelNet
PCIe Gen 3© Microsoft Corporation

CPUPCIe Gen 3

Global-Scale FPGA

2 FPGAs Involved

7 FPGAs Involved© Microsoft Corporation

Network Latencies
6x8 Torus Latency

25

LTL L2

LTL Average Latency
LTL 99.9th Percentile

Round-Trip Latency (us)2015
Example L1 latency histogram

LTL L1

10
Example L0 latency histogram

LTL L0 (same TOR)

5

Catapult Gen1 Torus(can reach up to 48 FPGAs)

Examples of L2 latency histograms for different pairs of FPGAs

01

10100

1000

Number of Reachable Hosts/FPGAs© Microsoft Corporation

10K
10000

100K

100000250K1000000

Why hasn’t Supercomputing moved to the Cloud?
CPUs look largely the same, but…
❑ Top 500 often include specialized accelerators (especially GPUs)❑ Networks are highly specialized, tuned for low-latency, high bandwidth❑ Won’t running virtual machines kill performance?

Network Acceleration – Azure Accelerated Networking
VM / Container(10.1.1.1)

VM / Container(10.1.1.2)

Network Acceleration – Azure Accelerated Networking
VM / Container(10.1.1.1)

VM / Container(10.1.1.2)

OSOS

NIC

NIC

Virtualization Overhead – Standard Virtual Machines
VM(10.1.1.1)
Guest OSHypervisor
SDN StackNIC

SDN Stack Functions
SLB: Software Load Balancers
ACL: Access Control Lists
NAT: Network Address Translation
VNET: Virtual Networks
Metering

Virtualization Overhead – SRIOV NICs

VM

VM

VM

VM

VM

Guest OS

Guest
OS

Guest
OS

Guest
OS

Guest
OS

Hypervisor

SDN

SR-IOV NIC

Virtualization Overhead – Azure SmartNIC w/ FPGAs

VM

VM

VM

VM

VM

Guest OS

Guest
OS

Guest
OS

Guest
OS

Guest
OS

Hypervisor

SDN

SR-IOV NIC

GFT

SDN in FPGA

Virtualization Overhead – Azure SmartNIC w/ FPGAs & DPDK

VM

VM

VM

VM

VM

DPDK

DPDK

DPDK

DPDK

DPDK

Hypervisor

SDN

SR-IOV NIC

GFT

SDN in FPGA

AccelNet PerformanceWhy hasn’t Supercomputing moved to the Cloud?
CPUs look largely the same, but…
❑ Top 500 often include specialized accelerators (especially GPUs)❑ Networks are highly specialized, tuned for low-latency, high bandwidth❑ Won’t running virtual machines kill performance?

Developer Experience•

Focus on the Customer•

In Supercomputing, developers are often the customer•

Traditional HPC machines require long, in-advance reservations•

Cloud allows for gradual scaling, 24/7/365 availability•Enabling physicists / chemists / biologists / etc.. to experiment is far more
important to impact than peak performanceWhy is the FPGA a good choice as an accelerator?
• Greater Performance and Efficiency than CPU, more general purpose than ASIC
• Many applications aren’t about throughput or double-precision floating point• AI/ML, Bioinformatics, text processing, financial services…
• Exploits different forms of parallelism than other accelerators© Microsoft Corporation

FE: Feature Extraction
Query: “FPGA Configuration”{Query,Document}
Document

L2
Score
Score

NumberOfOccurrences_0 = 7

NumberOfOccurrences_1 = 4NumberOfTuples_0_1 = 1

Feature Extraction Accelerator

FPGAs in Cosmology

EOR Science can be done with a paperclip
and a supercomputer-- Don C. Backer

Cosmologists often refer to their telescopes as“software telescopes”

FPGAs in Physics Applications

ASIC vs. FPGA
Performance Range

ASIC

Performance

Performance

Performance

FPGA

First UseFirst Use

ASIC Superiority© Microsoft CorporationTimeTime
FPGA Superiority
Under heavy change

Planning

Development

Deployment

Requirement lock to decommission is over a decade(as long as Azure itself has existed)
A lot changes over a decadeSuperiority Depends

Resource Functionality Over Time for 40Gbps GenerationDate© Microsoft Corporation

Pkts/sec

Description

8/9/2017

22.5M PPS

7/27/2018

22.5M PPSPFC Added

12/15/2018

22.5M PPS

Fast Offload, new Lookup

6/11/2020

22.5M PPS

PdParser, multi-tenancy, Flow
Scaling to 4Million+

9/24/2020

100M PPS

GFT-V2, 100MPPS, Shell Update

10/1/2020

100M PPS

PCAP-V3, Filtering

Conclusion• Software is more important than hardware when you want to make an impact

on the world• The Cloud will replace dedicated supercomputers• In large part due to developer experience• Think of FPGAs as a *complement* to GPGPUs, not just a competitor• FPGAs play a role in all parts of the HPC stack• High Flexibility enables a much longer lifetime, especially in new areas© Microsoft Corporation© Copyright Microsoft Corporation. All rights reserved.