Mike Strickland, Data Center Solution ArchitectIntel Programmable Solutions Group
December 2018

notices and disclaimersIntel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration.
No product or component can be absolutely secure.
Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. For more complete information
about performance and benchmark results, visit http://www.intel.com/benchmarks .
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using
specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests
to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information visit
http://www.intel.com/benchmarks .
Intel® Advanced Vector Extensions (Intel® AVX)* provides higher throughput to certain processor operations. Due to varying processor power characteristics, utilizing AVX instructions may cause a) someparts to operate at less than the rated frequency and b) some parts with Intel® Turbo Boost Technology 2.0 to not achieve any or maximum turbo frequencies. Performance varies depending on hardware,
software, and system configuration and you can learn more at http://www.intel.com/go/turbo.
Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and
SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessordependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer tothe applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.
Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide costsavings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.
Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate.
Intel, the Intel logo, and Intel Xeon are trademarks of Intel Corporation in the U.S. and/or other countries.
*Other names and brands may be claimed as property of others.
OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos© 2018 Intel Corporation.

Scale Your Innovation

2

Agenda Intro: Scale Out and Scale Up with FPGA
 Programmer Tool Chain– Financial Library API Example
 Libraries & Turnkey Solutions

Scale Your Innovation3

Microsoft Exa-op with FPGAs(Ignite Sept 2016)

Translate every Wikipedia English page
to another language in the blink of an eye
Link to video (~ minute 55)

Scale Your Innovation

4

MSFT Single FPGA Algorithm, Networking, & Data Access Acceleration

Microsoft Scale Out FPGA Multi-Function Accelerator“Diversity of cloud workloads and … rapid … change” (weekly or monthly)– Search, SmartNIC, Machine Learning, Encrypt, Compress, Big Data Analytics,…
Bing Search: 2X server level perf, 29% latency reduction, 10% increase in power (1)Networking Virtualization: 10X latency improvement, 2X perf many db and OLTP workloads (2)Machine Learning: Stratix 10 capable of 90 TFLOPs 8 bit floating point (3)Scale Your Innovation

For details: see footnotes page in backup

5

What FPGA Does Well
 Custom Processing Pipelines Multiple Workloads

 Variable Precision Arithmetic

 High-bandwidth data caching Heterogenous Dataflows

 Look-aside AND In-line Acceleration

 Diverse Memory Hierarchy Parallel processing

Low Latency

Inherently Parallel

High Performance

Variable precision

Reprogrammable

Energy Efficient

Scale Your Innovation

6

Applications Acceleration Overview
Framework or API’s with OpenCL underneath option
Intel® Openvino™
 With FPGA acceleration option

Data analytics
 Open Relational D/B– Data Warehouse, Real Time Cassandra NoSQL, ElasticSearch Hadoop/Spark

HpC: programmer api
 Genomics, Financial
 Government pattern matching Video transcode Emerging: oil & gas
Scale Your Innovation

7

Infrastructure Acceleration Overview
Networking & Data access In-line advantage over look-aside Compression, Encryption, Dedupe Virtualization or complete network stack Torus & inline acceleration

NVME over roce with accelerators
 Attala cpu offload & inline acceleration

Scale Your Innovation

8

Scale Out: novo-g 3D torus
Developed and deployed at CHREC
 Most powerful reconfigurable computer in
research community (2009-present)–

448 (soon 512) high-end Altera FPGAs with 3.5TB(soon 4.5TB) of FPGA-attached SDRAM

 2012 Alexander Schwarzkopf Prize forTechnology Innovation @ NSF

App acceleration
 Computer vision, finance, bioinformatics,
molecular dynamics, crypto, et al.

Hardware emulation
 Behavioral emulation of future apps
and systems, up to Exascale

2009: 96 Altera Stratix-III E260 FPGAs, each with
4.25GB SDRAM, on Gidel ProcStar-III cards
2010: 96 Altera Stratix-III E260 FPGAs, each with
4.25GB SDRAM, on Gidel ProcStar-III cards
2011: 96 Altera Stratix-IV E530 FPGAs, each with
8.50GB SDRAM, on Gidel ProcStar-IV cards
2012: 96 Altera Stratix-IV E530 FPGAs, each with
8.50GB SDRAM, on Gidel ProcStar-IV cards
2014: 32 Altera Stratix-V D8 FPGAs, each with 16GB
SDRAM and 3D torus i/f, on Gidel ProcE-V
card
2015: 32 Altera Stratix-V D8 FPGAs, each with 16GB
SDRAM and 3D torus i/f, on Gidel ProcE-V
card
3D torus interconnect

2015 - 2016
 64 Altera Stratix-V D8 FPGAs––

On Gidel ProcE-V PCIe boards
Additional 64 in development

 3D torus interconnect (4x4x4)–

6 links per Stratix-V (40 Gb/s per link)

Made possible by support from UF, Altera, Gidel, NSF, and DOE

Scale Your Innovation

9

520N

Scale Out: stratix 10 2D torus

PCIe FPGA Board
Intel Stratix 10 FPGA GX2800 F1760 NF43

On-board System
Manager (USB and
FPGA access)On-board USB
Blaster II and UARTto-USB

 -1 or -2 SerDes and FPGA speed
grades

PCIe ¾ Length, Dual Width Gen3 x16, standard height, ¾lengthFour 100G QSFPs
 Range of line rates include 4x
40/100G or 16x 10/25G

4x DDR4 Banks 2400MT/s, up to 32GB total

OpenCL HPC BSPFPGA configuration functions incl. JTAG,
remote flash upgrade, reconfiguration,and fallback mechanism

Scale Your Innovation

10

Scale Out: 3D-Torus roadmap concept

520N
High performance computenode with optical IO for
creation of directly-coupled,
dense FPGA clusters(4) QSFP28 Cages supporting 1G  100G line rates
 Upgrade to (6) 100G network ports using QSFP-DD
 Enables 3D-Torus network connectivity

Scale Your Innovation

11

Intel® Embedded multi-die interconnect bridge (EMIB) technology

Scale Your Innovation

12

Scale Up: Stratix ® 10 MX
Development Kit ContentsIntel.com page: (link)

Development board–

Stratix ® 10 MX FPGA, 2.1M LE, 8GB HBM2

2x QSFP28 cages

2GB DDR4 onboard



Hi-Lo and DIMM Connectors for DDR4



PCIe Gen3x16–

Endpoint via edge connector–

Rootport via slot

Scale Your Innovation

13

CONTINUING PRODUCT LEADERSHIP Built on Intel Custom Foundry 10 nm platform 2nd Generation Intel® HyperFlex™ Architecture
 2nd Generation EMIB-based heterogenous SiP

FalconMesa

 Next Generation HBM Support
 Up to 112 Gbps Transceiver Rates
 PCI-Express Gen4 x16 Support

10nm FPGAs Built on
World’s Most Advanced FinFET Process

Delivering Industry Leading Performance and Power
14

Removing The Barriers of Adoption
SoftwareStacksOpenVINO™
Tool Kit

Libraries

DLA

Primitives

ParallelCompilers
HLS Compiler
DSP Builder

Boost
Productivity

OpenCL™
LLVM Compiler

High-Level Design Backend Compiler
Boost
PerformanceIntel® Quartus® Prime Design Suite
DCP Platform / Acceleration Framework
Hardware Developers

Scale Your Innovation

Software Developers

16

Common Heterogenous Programming Environment
Common environment for heterogenous
programmingIntel® Software
Development Community Plugin to Intel® System Studio and Intel Parallel
Studio CPU, GPU, FPGA, …

Easy path to FPGA
 Already familiar environment

Heterogenous SDK

 Intel® Developer Zone
FPGA

Scale Your Innovation

GPU

17

OpenCL™ “Programmer Friendly” Acceleration
Software Programmers
Need Logic and Data Management


By writing lines of code

OpenCL™ Compiler BenefitsEase of use

Scalable

HeterogeneousLeverage existing libraries



Vendor choice w/open standards

Foundation for HLS & SPIR

Context
CompileCode

Kernel  KernelExternal IO  KernelMix ‘n Match HDL & Kernels

Scale Your Innovation

ExecuteDDRx Global Memory Buffer

Channels/Pipe ExtensionCreate Data
and Arguments

I/O

KernelKernelKernelI/O

18

Leveraging Software Development Environment
Modify kernel.cl

Hardwareperformancemet?

x86 Emulator (sec)Functional Bugs?

Optimization Report (sec)

Stall-free pipeline?
Memory coalesced?

Profiler (hours)

DONE!
Scale Your Innovation

19

API Call + OpenCL Option
FinLib Example••••

High level C++ APIs
OpenCL implementation“Quants” can use high level APIs
Similar approach used for PairHMM

Scale Your Innovation

20

User Application
Data Framework (e.g. Apache Spark*)
Scalable Functions (e.g. PDE Solver)
OEM Qualification

Ecosystem Partners & Integrators

Ingredients needed to make FPGA in computing “mainstream”

Library Primitives (Mathematics, Statistics)
Developer SDKs (e.g. OpenCLTM)
Acceleration Stack, Drivers, BSPs & Interface IP
Boards and Platforms
FPGA Silicon & FPGA Design Tools

Scale Your Innovation22

acceleration Stack ingredients: Overview
Developed by User
(Domain Expert)

Application

Intel®
Xeon®

User, Intel, and 3rd Party(Tuning Expert)

Libraries

PCIe* Drivers
Provided by IntelFPGA Interface Manager
Provided by Intel

Drivers

Accelerator
Functional
Unit (AFU)

SoftwareOpen Programmable
Acceleration Engine (OPAE)
Provided by IntelFPGA
HardwareUser, Intel, or 3rd-Party IP
Plugs into AFU Slot(tuning Expert)

Signal Bridge and Management
FPGA Platforms(Programmable Acceleration Cards)

Scale Your Innovation

Qualified and Validated forvolume deployment
Provided by OEMs
23

Open Programmable Acceleration Engine (OPAE) Technology
Simplified FPGA Programming Model for Application Developers

Consistent API across product generations and platforms• Abstraction for hardware-specific FPGA resource details

Applications, Frameworks, Intel® Acceleration Libraries

Designed for minimal software overhead and latency• Lightweight user-space library (libfpga)

FPGA API(C) (Enumeration, Management, Access)Open ecosystem for industry and developer community

FPGA Driver• License: FPGA API (BSD), FPGA driver (GPLv2)

FPGA driver being upstreamed into Linux* kernel
Supports both virtual machines and bare metal platforms
Faster development and debugging of Accelerator Functions
with the included AFU Simulation Environment (ASE)**(Common – AFU, Local Memory)FPGA Driver

FPGA Driver(Physical Function – PF)(Virtual Function - VF)

OSHypervisor

FPGA Hardware + Interface Manager
Bare Metal

Virtual Machine

Includes guides, command-line utilities and sample code

Start developing for Intel® FPGAs with OPAE today: http://github.com/OPAE

Scale Your Innovation**ASE requires Acceleration Functions written in RTL and a properly installed RTL simulator:
Synopsys* VCS-MX, Mentor Graphics* ModelSim-SE*/QuestaSim
Some names pending final approval and may change in the future.
Supports: Red Hat Enterprise Linux* 7.3 w/ kernel 4.7, Intel® Xeon® Processors v4 or newer24

Arria® 10 GX FPGA [10AX115N2F40E2LG]
High-perf, multi-gigabit SerDes transceivers up to 15 Gbps
1150K logic elements available (-2L speed grade)
53 Mb of embedded memory

On-board Memory
8 Gbytes DDR4 Memory Banks with ECC (2 banks), 2400 Mbps
1Gb Mbit (128 MB) FlashInterfaces & Dimensions

Board Management Controller
Voltage, current, temperature monitoring
Power sequencing and reset
Platform Level Data Model (PLDM)

Power
70W TDP, 45W FPGA
650 LFM at Tla 55°C – Passively Cooled

PCIe x8 Gen3 electrical, x16 mechanical *USB 2.0 interface for debug and prog FPGA and Flash
1x QSFP with 4x 10GbE or 40GbE support½ Length, ½ Height, 1RU

SoftwareAcceleration Stack for Intel® Xeon® CPU with FPGAs
FPGA Interface Manager Installed

Specifications preliminary and are subject to change

Scale Your Innovation

25

Intel® Programmable Acceleration Card

1H19

with Intel® Stratix® 10 FPGA GX (aka “Darby Creek”2)

Versatile Workload Acceleration•

Customizable Hardware Architecture using Intel® Stratix® 10 FPGA GXHigh Performance with Intel® Stratix® 10 FPGA GX••

2.8M logic elements available with 229Mb of embedded memory
32GB DDR4 Memory with ECC (4 banks), 2400 Mbps

High Data Ingestion and Lower Latency••

PCIe* x16 v3 with SRIOV support
2x QSFP with 100GbE support

PCIe* Form Factor Compliant•••

Dual slot, three-fourths length, full height
225W TDP – Passively CooledIntel Max® 10 based Board Management Controller•

Configuration, telemetry, and remote update
1 Rush Creek: 1.1M LE A10, PCIe Gen 3 x8, 8G DDR3, 40GbE

Darby Creek offers 1,2 : 2.5X

more logic elements

 2X

higher PCIe bandwidth

 4X

more system memory density

 2.5X

faster network interface

2 Darby Creek: 2.8M LE S10, PCIe Gen 3 x16, 32G DDR4, 100GbE

Scale Your Innovation

26

PDE Solver – Double No-Touch (DNT) Option Pricing Engine
How much should the bank charge the Investor as Option Price?
DNT Option Terms:••••

Currencies / Asset
Start/Expiry Date
Barrier levels
Payout price

Option Pricing
EnginePDE
SolverShould I
buy?

OptionPrice
Investor

Bank’s financial data andsecret sauces
DNT = Double No-Touch
PDE = Partial Differential Equation

Scale Your Innovation

28

Creating a PDE Solver in FPGAFPGA used to provide a solver for a particularly computationally challenging workload
 Intent: Improve time to results–

More results (Present Value of Options (PVs)) in a given amount of time or compute resource Starting point: C-model implementation of PDE Solver created (880 lines of C code) End point: Optimised OpenCL™ implementation (920 lines of OpenCL)

Task

Dev. Time

ResultConvert C model to OpenCL

2 weeks

142 PV/s**

Optimise pipeline

1 week

174 PV/s**

New C Code + Open CL optimisations

2 weeks

387 PV/s**

Scale infrastructure (4 x FPGA Cards)

1 week1511 PV/s**

Scale Your Innovation** See appendix for server configuration (8)29Throughput (PV/s)

2500

Throughput Averaged Over 50 Consecutive Batches(1940 PDEs/batch)
Measured

2000

2278

Projected

1500

1898
1511

1000

1136

500
0

767
387
1

2

3

# of PACs

4

5

6

Not STAC Benchmarks

Throughput scales linearly to the number of FPGAs

Scale Your Innovation** See appendix for server configuration (8)30inference

Mainstream AI

Flexible
Acceleration

Scale Your Innovation

Data Center/
Workstation
Data Center/
Workstation

AI

Gateway/edge

Generaltraining

Intel AI For Compute
Deep
Learning
Mainstream
Training

Intensive
Training

Mainstream
Inference

Intensive
InferenceStreaming Inference

Intel
GNA

(IP)

Mainstream
Inference

Higher Inference Vision
1-20W
Throughput

Speech/Audio Autonomous
1-100+mW
driving

CustomInference32

Why FPGAs WIN In Deep Learning
Low Latency memory
constrained Workloads

First to market to
Accelerate evolving AI
Workloads
 Adversarial Networks
 Reinforcement Learning Neuromorphic computing

Falcon Mesa





RNN
LSTM
Speech WL

Enabling real time AI in a wide range ofembedded, edge, and data center applications

Flexible system level functionality with
Deterministic latencyScale Your Innovation

AI+I/O ingest
AI+Networking
AI+Security
AI+Pre/Post processing

RNN – Recurrent Neural Network
LSTM – Long Short-Term Memory33

Public Intel FPGA Machine Learning Success
Microsoft: Microsoft has revealed that Intel FPGAs have been installed acrossevery Azure cloud server, creating what Microsoft is calling the world’s first AI
supercomputer.

NEC: To create the NeoFace Accelerator, the engine software IP was integrated into
an Intel Arria 10 FPGA, which operate in Xeon processor–based servers.

JD.COM: Arria®10 FPGA can achieves significant improvement in the performanceof LSTM accelerator card compared to GPU.

Inspur/iFlytech: Server vendor Inspur Group and Intel launched a speech
recognition acceleration solution based on Intel's Arria® 10 FPGAs and DNN algorithm
from iFLYTEK.

Scale Your Innovation

34

Evolving deep learning Requirements
2017

2018

Convolutional Neural Network (CNN)

Recurrent Neural Network (RNN)

Floating Point

Floating Point

FP32

Scale Your Innovation

FP16

FP11

FP9

BFLOAT

35

CaffeUser
Visibility
via
OpenVINO™
toolkitSupport at DLA
API level isextremely costly

TensorFlow

Model
Optimizer

Standard ML Frameworks

Intel Deep Learning
Deployment Toolkit• Supports common software frameworks (Caffe, TensorFlow)• Intel DL software stack provides network optimizations• Intel FPGA Deep Learning Acceleration Suite provides turnkey or customized CNN acceleration for common networks
Optimized Acceleration Engine

Inference
EngineIntel
PreTrained
ModelsPre-compiled Network Architectures
Intel DLA
Network
Bitstreams

DLA SWAPI

GoogleNet Optimized Template
ResNet Optimized Template
SqueezeNet Optimized Template
VGG Optimized Template
Additional, Generic CNN TemplatesIntel®
Xeon®
Scalable
Processor

Scale Your Innovation

Heterogenous
CPU/FPGA
Deployment

DDR

Feature Map CacheIntel
Arria®
FPGA

Conv
PE Array

MemoryReader/Writer

DDR
DDR
DDR

CrossbarConfigEngineHardware Customization Supported ** Encrypted DLA source code license required, sold separately

36CPU + FPGA accelerate ai inference

VGG-16

Scale Your Innovation

SqueezeNet* 1.1

GoogLeNet (32)

VGG-16* (32)SqueezeNet* 1.1

37

Ai futureproofing with bfloat16 support
8 bit exponent

FP32
5 bit exponent

5

FP16

1
0

8 bit exponent

BFLOAT16

Scale Your Innovation

1
0

38

FinLib is in intel ® Quartus ® 17.0!
 Option pricing– European, American, Equities, Average rate,
Spread normal, Spread lognormal
 Statistical functions– Norm_std(), norm_cdf(), norm_icdf()….
 Working Demo accessible from the Intel labs

Scale Your Innovation

In 17.0
release39

Genomics – GATK Acceleration
FPGA Acceleration in GATK Targets PairHMM full integration

Latest Intel Benchmark
Configuration

PairHMM

CPU
Cores
Used

Peak Perf(GCUPS)

Average Perf
(GCUPS)

2 Socket Intel® Xeon® Processor E5
v4 (Note 7)

AVX

1

0.699

0.676

2 Socket Intel® Xeon® Processor E5
v4 (Note 7)

AVX

44

22.0

21.2

2 Socket Intel® Xeon® Processor E5
v4 + Intel® Arria® 10 FPGA (Note 7)

OpenCL

1

44.1

32.4

Scale Your Innovation

Performance results are based on testing as of March 2018 and may not reflect all publicly available security updates. Seeconfiguration disclosure for details. No product can be absolutely secure. System Configuration: (Click for footnotes)

40Falcon Accelerated Genomics Pipeline
Adaptive cross-hardware platform to provide a path to efficient & cost-effective genome analysis
Lower is Better> 10X Speed-up on GATK4 &GATK3.x

Available on public & private clouds or on-premise

Falcon Accelerated Genomics PipelinesEnd-to-end solution with performance optimization
Accelerates GATK best practices; No proprietary pipelines
Supports multiple GATK versions (3.8 & 4.0)
Both germline and somatic best practices

ONPREM“

Falcon solution is so fast! What had taken me over a week to do
on my computer cluster, I was able to do with the Falconaccelerated Genomics pipeline in a few hours”

Amy Cummings, MD, UCLA Medicine

42

Scale Your Innovation

42

FPGAs Offer Unique Value for Analytics/Streaming
Single Multi-function Accelerator
Moderate Acceleration is common
 PCIe lookaside acceleration (two data copies)
Network
Streaming

Data

Data

Offloads algorithm, networking,and data access processing

Scale Your Innovation

Significant Acceleration requires FPGA Multifunction and inline w/single FPGA Relational: 2.3X TPC-H w/Swarm644• PostgreSQL, MariaDB, MySQL, …

 NoSQL: 4X Cassandra5 w/rENIAC (80/20 R/W)
 Hadoop/Spark: 3X+ data streaming6 w/Bigstream, Megh

43

Different Data Store Approaches
Structured Data/ Relational

Semi-structured Data/NoSQL

Unstructured Data

Partners

Scale Your Innovation

44

Swarm64 Relational Database Acceleration
Two Workloads: Traditional Data Warehousing, Real Time Data Analytics
Database accelerate with a plugin
Swarm64 SDA: Seamless Plug-In

Acceleration Overview

Application– 20X+ single table inserts/s for real time data analytics

PostgreSQL - MySQL – MariaDB - Oracle*

S64 Tables– With modest tuning, 15M PostgreSQL INSERT/s– 2.3X* TPC-H data warehousing on Arria 10

Native Tables– 3X+ TPC-H for many CSP hosting configs

Your Choice of Storage Architecture* planned2© Swarm64 AS, 20172

– Data & tables managed by Swarm64

Closing the Performance Gap
13.8x– 3X+ storage compression

Capture data (in real-time)
up to 14 times faster11.1x

Note: this is SQL to relational d/b, not SQL to semi/unstructured data.
3.4x
2.9x
Cost of Storage &Network up to 13
times lower

13.2x

Analyse up to
21 times faster4

Scale Your Innovation

20.9x© Swarm64 AS, 2017

Note *: TPC-H SF1000, Dual Intel® Xeon® Gold 6130, 2.10 GHz, (12) 32GB DDR4-2166,(4) 960GB SSD RAID0 HPE MK000960GWJPPIntel Confidential for NDA Use Only

Source: Swarm64
45

NoSQL: System & IO Acceleration Opportunity
Source: rENIAC CEO

Scale Your Innovation

Intel Confidential for NDA Use Only

46

rENIAC Distributed Data Engine/Switch (rDS)
4X+ Cassandra (80% R/20% W), POCs of 2X, going to 4X by feb
Overview– No customer application change– Plug-in card with10GbE– Proxy tier or on database server– Distributed cache, proxy for reads andwrites– Predictable latency for SLAs
– Roadmap for storage compaction

Significant Acceleration√ Networking/CQL acceleration√ Data access acceleration√ Compression√ Hashing
Scale Your Innovation

Intel Confidential for NDA Use Only47

Spark*: several Acceleration Areas Ingest/Apache Kafka*: Extract, transform, load and filtering (BigStream, Megh) SQL over Apache Spark (BigStream) BigDL: Deep learning acceleration (Megh) Machine learning MLlib: e.g. ALS (Megh) Hadoop/Spark: Shuffle phase (A3Cube)
Scale Your Innovation48

Bigstream Hyper-acceleration
DatastreamingPOCs now
 Frictionless acceleration: Arria 10 and Stratix 10– Zero code changes

BIG DATAPLATFORMS– Cross platform: Spark, Kafka, TensorFlow– Cloud or on-prem Intelligent and adaptive

Dataflow Adaptation
Layer– Automatic profiling and partitioning of computation–

Bigstream Dataflow

Between CPU and FPGA

– Overlay dataflow execution on FPGAHYPER-ACCELERATIONLAYER

Bigstream Hypervisor

 Kafka consumer speedup up to 13X
 Spark SQL TPC-DS results– 4X average speedup for 26 of the queries with Arria 10

 Industry targets: FinServ/FinTech, AdTech, Healthcare
 Use cases: Spark SQL analytics, ingest/ETL, EDW

Scale Your Innovation

Intel Confidential for NDA Use Only

CPU

FPGA

Other

https://bigstream.co/wp-content/uploads/2017/03/Bigstream-whitepaper-v1.4.pdf

http://bigstream.co/resources/video-strata

49

REAL TIME ANALYTICS STACK OPTIMIZED FOR
HETEROGENEOUS CPU+FPGA PLATFORM
Application
Data
Sources

ETL

Internal
Data
Sources

Admin

Data Processing

Streams

ML

DL

Connector
In-lineProcessing

1

Off-load
Processing

2

FPGA
External
Data
Sources

PacketRX

Accelerated Functions

BI
PacketTX

Data Stores

CPU+FPGA platform for 1 in-line processing of streaming analytics and 2off-load processing of ML and DL
to deliver >5x performance efficiency and provide actionable operational insights.

Scale Your Innovation

50Intel Apache YARN Submission: https://issues.apache.org/jira/browse/YARN-5983

Scale Your Innovation

51

Summary FPGAs are ready for scale out and scale up
 Intel ® Acceleration Stack: driver, FPGA Interfaces, virtualization, security, etc.
 Variety of Interfaces: OpenCL™, library call, framework level AI, Genomics, and Financial acceleration options
 Data Analytics acceleration with no change to application required– Relational DB, NoSQL, SPARK* shuffle phase, Kafka* streaming, BigDL (deep learning) FPGA advantage of multiple concurrent functions and inline acceleration

Scale Your Innovation

53

Footnotes(1): A Reconfigurable Fabric for Accelerating Large-Scale Datacenter Services, sec 2.3
https://www.microsoft.com/en-us/research/publication/a-reconfigurable-fabric-for-accelerating-large-scale-datacenter-services/(2) : Microsoft's Production Configurable Cloud (Mark Russinovich) (Slide 26)
https://www.slideshare.net/ChrisGenazzio/microsofts-configurable-cloud(3) : Accelerating Persistent Neural Networks at Datacenter Scale
https://www.hotchips.org/wp-content/uploads/hc_archives/hc29/HC29.22-Tuesday-Pub/HC29.22.60-NeuralNet1-Pub/HC29.22,622-Brainwave-DatacenterChung-Microsoft-2017_08_11_2017.pdf(4)

TPC-H SF1000, Dual Intel® Xeon® Gold 6130, 2.10 GHz, (12) 32GB DDR4-2166, (4) 960GB SSD RAID0 HPE MK000960GWJPP, CentOS 7.4.1708, Kernel
3.10.0-693.21.1.e17.x86_64, Docker 18.03.0.ce, Swarm64 DB 1.4.1-PREVIEW, PostgreSQL 10.3

(5)


Cassandra Stress Test (80% R/20% W)“Dual Xeon E52670, 2.6 Ghz, 32 cores total, 64GB ram, 1 TB NVMe, Centos 7.4”(6) https://bigstream.co/wp-content/uploads/2017/03/Bigstream-whitepaper-v1.4.pdf

Scale Your Innovation

55

(7) Configuration Details for ‘PairHMM Comparison - Xeon/FPGA

1Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as
SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors
may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases,
including the performance of that product when combined with other products. For more complete information visit www.intel.com/benchmarks.

INTEL
CPU

Intel® Xeon CPU E5-2699, v4, 2.20 GHz
Intel Arria® 10 GX, 10AX115S2F45I1SG2

FPGA

Systolic ArrayFPGAResource
Usage

Scale Your Innovation

ALM

427,200

Memory53.0 Mb

DSP Blocks
1,518
208 Processing Elements (PEs)
Logic

55%

Memory50%

DSP Blocks

99%

Frequency

230 MHz

Input Data

Chromosome 21 from 30x WGS NA12878
56

(8) **System configuration for performance testing
Server configuration:
Dell PowerEdge R740
2 x Intel® Xeon® Gold 6132 @ 2.6 GHz
192GB (12 x 16GB) RDIMM, 2666MT/s, Dual Rank
Operating System:
Red Hat Enterprise Linux: Release 7.5 with Linux kernel 3.10.0-862.el7.x86_64
FPGA:Intel Programmable Acceleration Card with Intel Arria® 10 GX FPGA, Acceleration Stack version 1.0
Test performed during August 2018.
OpenCL code was developed within Intel® Programmable Solutions Group.
Functional correctness was verified by comparison with single-precision floating point results from
CPU, using the “==” operator in C/C++
Tests were performed with pre-production, proof-of-concept code.
Not all capabilities are part of shipping products.

Scale Your Innovation

57