ABOUT US…
TechnologyProducts

Investors

Processors and software
solutions designed for AI

IPU-Processor PCIe Cards andPoplar® software stack>$310m infunding

2

BRISTOL,LONDON,PALO ALTOAUSTIN, SEATTLEOSLO, BEIJING

200+ TEAM &GROWINGFAST3OUR IPU LETS INNOVATORS CREATE THE NEXT
BREAKTHROUGHS IN MACHINE INTELLIGENCE

4

A COMPLETELY NEW WORKLOAD
Intelligence processing is highly parallel
probabilistic optimization on a graph structure

TRAINING DATA /SENSOR DATA
Learn parameters(and model) fromdata

TRAINING &INFERENCE
PROCESSORRESULT
Infer outputs usingknowledge model

KNOWLEDGE
MODEL5TODAY’S PARALLEL MACHINES ARE INFLEXIBLE
Instruction operates on long data vector(typically 32x32 bits or 64x16 bits)

All units performthe same Instruction
on each program step

Data vector
inSingle Instruction
Multiple Data/Thread(SIMD) Architecture

Small number ofprocessors (~80)
each with awide vector unit

Data vector
out

Only efficient for large blocks of dense data / large mini-batches
6

MEMORY BANDWIDTH IS
LIMITING PERFORMANCE

PROCESSORMEMORY INTERFACE

If 10x faster…

…10x faster how?

MEMORY7A NEW PROCESSOR IS REQUIREDApps and Web
ScalarGraphics and HPC
VectorIPU

Machine Intelligence
Graph8

IPU ADVANTAGEALEXNET network visualization from POPLARTMMASSIVE PERFORMANCE LEAP
• up to 100x faster on training and inference• model held inside the processor• 100x memory bandwidth

MUCH MORE FLEXIBLE• every network type supported efficiently• latency reduced by over 10x

EASIER TO USE• seamless ML framework support• Poplar® software stack9

COLOSSUS GC2

The world's most complex processor chip with 23.6 billion transistors
10x IPU-LINKSTM

320GB/s chip-to-chip bandwidth

300MB IN-PROCESSOR-MEMORYTM

1,216 independent IPU-CORESTM
each with IN-PROCESSOR-MEMORYTM tile
> 100GFLOPS per IPU-CORETM> 7,000 programs executing in parallel

45TB/s memory bandwidth per chip
the whole model held inside the processor

8TB/s all to all IPU-EXCHANGETM
PCIe Gen4 x16

non-blocking, any communication pattern

64GB/s bi-directional hostcommunication bandwidth10

BRIDGING PARALLEL HARDWARE TO SOFTWAREIPU is the world first BSP processor
Bulk Synchronous Parallel (BSP)
compute | synchronize | exchange
Easy to program
no live locks or dead locks
Widely used in compute clusters:
Google | Facebook | …

1. Compute

2. Sync
3. Exchange

4. Repeat…

11

POPLAR®
Software stack

12CONFIDENTIALPOPLAR®
Software stack

SEAMLESS INTERFACE TO INDUSTRYSTANDARD ML FRAMEWORKS

OPTIMIZED GRAPH MAPPING AND
CODE COMPILER BUILT USING LLVM

DRIVERS, UTILITIES AND
GRAPH ENGINE FOR EXECUTION

POPLAR GRAPH COMPILERIPU

POPLAR® GRAPH PROGRAMMINGFRAMEWORK (C++ & PYTHON)
AND OPEN SOURCE ML LIBRARIESCPUSIMADVANCED
VISUALIZATION AND DEBUG TOOLSCOMPREHENSIVE USERDOCUMENTATION, EXAMPLES,APPLICATION NOTES AND TUTORIALS

13

DEVELOP YOUR MODEL USINGINDUSTRY STANDARD ML FRAMEWORKS

Optimized support for
inference and training
14

POPLIBSTM
Highly optimized open source libraries partition work and data efficiently across IPU devices
C / C++ and Python language bindings

poputil

popopspoplinpoprandom

popnn

Utility functions forbuilding graphs

Pointwise andreduction operators

Matrix multiply andconvolution functions

Random number
generation

Neural network
functions (activation
fns, pooling, loss)

POPLAR®

github.com/graphcore/poplibs

15

POPLAR® C++ / PYTHON, GRAPH FRAMEWORK LETS YOU
MODIFY OR CREATE YOUR OWN LIBRARY ELEMENTS
codelet:gGraph g(device);g.addCodelets("codelets.cpp");
Tensor t1 = g.addTensor(“float”, {4, 5});
Tensor t2 = g.addTensor(“float”, {4});ComputeSet cs = g.addComputeSet(“myComputeSet”)

5.6

VertexRef v1 = g.addVertex(cs, “AdderVertex”);
VertexRef v2 = g.addVertex(cs, “AdderVertex”);

0.3

3.22

44.5

3.13

6.49

24.3

9.2

0.01

0.23

953.1

0.22

123.2

3.2

5.67

55.3

99.8

7.228.66

22.1

t1cs

g.connect(t1[1][1], v1["x"]);g.connect(t1.slice({3, 1}, {4, 3}), v1["y"]);g.connect(t2[0], v1["z"]);g.connect(t1[0][3], v2["x"]);g.connect(t1.slice({2, 2}, {3, 4}), v2["y"]);g.connect(t2[3], v2["z"]);v2

v1

g.setTileMapping(t1.slice({0, 0}, {4, 2}), 0);g.setTileMapping(t1.slice({0, 2}, {4, 5}), 1);g.setTileMapping(t2, 2);
9.3

g.setTileMapping(v1, 0);g.setTileMapping(v2, 1);

2.95

103.2

0.001

t216

DeepVoice GRAPH VISUALIZATION FROM POPLAR®

POPLIBSTM and POPLAR®
expand ML Framework
output to a full computegraph.

17

POPLAR® MAPS AND COMPILES GRAPH TO IPUs

POPLAR® GRAPH COMPILER:
Load balances code across processor cores
Allocates data to ‘In-Processor-Memory’
Orchestrates data exchanges

POPLAR® GRAPH ENGINE:
executes graph under BSP on IPU or multiple IPUs

18

ADVANCED VISUALIZATION AND DEBUG TOOLSBSP superstep

1 – IPU
Processor CoreIPU Processor Corestime
COMPUTEEXCHANGE

WAITING FOR SYNC

SYNC

19

EXAMPLE BSP TRACE : RESNET-50 TRAININGCOMPUTEEXCHANGE

WAITING FOR SYNC

SYNC
20

DRIVERS AND UTILITIESGraphcore device accesslibrary provides host softwareinterface to IPU hardware
platforms

IPU hardware abstraction layer
allows provision of IPU devices
via multiple host interfaces

PCI Express driver providing
device management andaccess to IPU resources fromhostHardware configuration,
installation and test managed
from host utility applications

21

COMPREHENSIVE DOCUMENTATIONDOCUMENTATION, CODELET EXAMPLES, APPLICATION NOTES, AND TUTORIALS including VIDEOSCONFIDENTIAL22PRODUCTSC2 IPU-Processor PCIe Card23

C2 IPU-PROCESSOR PCIe CARDDOUBLE WIDTH PCIE CARD WITH 2 – COLOSSUS GC2 IPU PROCESSORSCARD-TO-CARD IPU-LINKSTM (320GB/s)
250 TERA-FLOP MIXED PRECISION IPU COMPUTE @ 300W
24

C2 IPU-PROCESSOR PCIe CARD
IPU-LINKSTM

GC2IPU PROCESSOR

GC2IPU PROCESSOR

PCIe Gen4.0 x16 Interface

25

ResNet50 network visualization from POPLARTMWE HAVE DEVELOPED A NEW KIND OF HARDWARETHAT WILL LET INNOVATORS CREATE THENEXT GENERATION OF MACHINE INTELLIGENCE26THANK YOU

27