The 2nd R-CCS Symposium,Future Co-design Session 13:30 to 15:30
Day 2, Feb 18, 2020, Kobe, Japan

Synchoricity as the basis for going
BeyondMoore
Not a Typo
Ahmed Hemani
Professor, Dept. Of Electronics, School of EECS, KTH,Stockholm Sweden
Email: hemani@kth.se1

Going Beyond Moore !
Solutions to go beyond Moore

Make it easy to use the solution

1. Squeeze more out of CMOS
a. ASICs like custom functional hardwareb. Delivers 2-4 orders better energy-delay product compared toGPUs, FPGAs and Multi-cores

2. Complement CMOS with emerging technologies
a. 2.5D and 3D Integration (DRAM)
b. Computation in memory using Memristorsc. Plasmonics“Science makes progress, not when you find a solution,
but when you make it easy to use the solution”-- Venki Ramakrishnan, Nobel Laureate2© Ahmed HemaniSynchronicity

Synchoricity

Time is discretized using clock ticks

Space is discretized using a virtual grid

D Q
clk1

D Q
clk2

Can be temporally composed If
clk1 = clk2
&The two clocks are skew aligned

Can be spatially composed If
If the number of grid cells
in each dimension are equal&Their interconnect edges are abuttable

3

SiLago (Silicon Lego) Blocks
SiLago Blocks are the new standard cells
RTL & Coarse Grain Reconfigurable
4-5 orders larger than Standard Cells
A SiLago BlockCharacterized with postlayout data
Empowers Synthesis from Higher Abstractions
Inter SiLago Block Wires bought to peripheryat right place and right metal layer to enable
compositoin by abutment

4(c) Ahmed Hemani

VLSI Designs are Composed by Abutting SiLago Blocks

All Wires – functional and infrastructural (reset, clocks and power grid) arecreated as a result of abutment
Cost-Metrics of the composite design becomes known with post layout
accuracy5(c) Ahmed Hemani

Inspiration from Construction Industry

6

An Analogy

7(c) Ahmed HemaniWe shifted to pre-fabricated wall segments

1. Productivity gain did not solely come from thelarge size of the pre-fabricated wall segments
2. Productivity gain came from physical design
discipline that enables composition by abutment
3. IPs in VLSI Design lack this discipline andcomposition by abutment8(c) Ahmed Hemani

Lego Kits The Berkeley Dwarfs SiLago Regions Types
The Berkeley Dwarfs
1

Dense Linear Algebra

2

Sparse Linera Algebra

3

Spectral Methods

4

N Body Methods

5

Structured Grids

6

Unstructured Grids

7

MapReduce

8

Combinational Logic

9

Region Types – SiLago Block Types
Functional

Infrastructural

Graph Theory

NOCs

Outer Modem

Scratch Pad Memory

Inner Modem

PLL + CGU

Protocol Processing

Power Management

Spectral Methods

Memory Controller

Dense Linear Algebra

FIFO, FIFO Controller

Graph Traversal

Sparse Linear Algebra

RISC Processors – RISC-V

10

Dynamic Programming

Dynamic Programming

DMA

11

Back-track and Branch n Bound

State Machines

Memory Consistency

12Graphical Models

13

Finite State Machine© Ahmed Hemani

9Hardware Centric vs. Software CentricAccelerators vs. Flexilators
Software Centric Platform Based Design

By default
Functionalities are
mapped as software

SoftwareAccelerators
HardwareOnly power and performancecritical functionalities are
mapped as hardware accelerators

Hardware Centric Synchoros VLSI Design

By default
Functionalities are
mapped as custom
functional hardware

Custom Hardware
Hyper-CISC
Instructions
Flexilators
e.g. RISC-V

Only flexibility critical, dynamic and non-deterministic functionalities
are mapped toSiLago Flexilators: RISC-V, FSMs, FIFOs, Arbiters, Schedulers,
NOCs etc.
Lego Flexilators

10© Ahmed Hemani

Why does Synchoros VLSI Design Work ?

~300 MUSD

HPC LIB Impl

SiLago Blocks
Automated

BooleanPhysical Level

Std. Cells

Standard Cells

(CV)
(FV)

(CV)

(FV)

ManualConstraints Verification (CV)ManualRTL

Function Verification (FV)

Algorithm

HLS

Automated

Application

Synchoros VLSI Design Style
O(100 million Gates)

Standard Cells
O(10 million Gates)

Standard Cells

Automated
One Time Engineering

Full Custom Mead-Conway
O(10K Gates)
SystemPhysicalGDSII© Ahmed HemaniLog (# of Solutions)11HPC Application
L Algorithms

Sampling Rate,
Total Latencyc

One Time Engineering Effort

SiLago Application Level Synthesis

HPCLIBs

HLS

1M
HPC LibImplementations

1. Select Optimal Solution from ML solutions
2. Global Interconnect, buffers and control
3. Floorplanning

HLS tools do not automate this.
Manually refined

Number and types of SiLago blocks + Mapping

SiLago Blocks
Characterization Data
Hardened Blocks
Scripts

Compose Ready-to-manufacture Chip
Mask Patterns(c) Ahmed Hemani

Reports

12

SiLago Design Instances =  Region Instances
Protocol Processing Region

Scratchpad

NOCs
Type,Number, Size and Position

TSVs
3D Memory
Control

Dense Linear Algebra

NOCs

Buffered/Pipelined
NOC SiLago Blocks

ofSiLago Region Instances
Decidedby Synthesis Scratchpad
Tools
Scratchpad
Based on
Functionality
AndConstraints
Dense Linear Algebra

NOC Switch
SiLago Blocks

Conceptual
Does Not Exist

Interrupt Ctrl

PLL/CGUPMC
Ethernet
Data
MemoryRegion Specific
Network
Interface Units

SystemController

Program
Memory13

SiLago can also potentially reduce the manufacturing cost
Flash CTRLAll SiLago
designs are composedof Storagea finite number DRAMof CTRL
Streaming
Protocol
Processing
Ethernet
SiLago block Types
PLL/CGUPMC

All SiLago blocks can only have a finite types ofneighbors

DataStorage

Each SiLago blocks’s mask depending on the neighbor
System Ctrl
Inner
Outer
Flexilators
Inner
Outer
types canbe saved asModema component mask
Modem
Modem

Modem

The DFT Cost canalso be factored out
The DFT can bemade much more
efficient reducing
time spent on ATE

The entire design mask can be composed from suchProgram
Storagecomponent masks
14(c) Ahmed Hemani

What becomes possible
600 $Jetson TX1(GPU)

667 $
126 $
Keystone II
Parellela(DSP)
Multicore+NoC

1600 $
ZC706
FPGA

196 $
SiLago22 nm(volume of 10k)

Energy per frame (mJ)

106
103
101
10-1
10 M

1G100 M
Operations of CNN

10 G

Data on GPU, DSP, Parallela and FPGA adapted fromG. Hegde, S. Siddhartha, and N. Kapre, “CaffePresso: Accelerating convolutional networks on embedded SoCs,” ACM Transactions on Embedded Computing
System, vol. 17, 2017.
15© Ahmed Hemani

Going Beyond Moore !
Solutions to go beyond Moore

Make it easy to use the solution

1. Squeeze more out of CMOS
a. ASICs like custom functional hardwareb. Delivers 2-4 orders better energy-delay product compared toGPUs, FPGAs and Multi-cores

2. Complement CMOS with emerging technologies
a. 2.5D and 3D Integration (DRAM)
b. Computation in memory using Memristorsc. Plasmonics“Science makes progress, not when you find a solution,
but when you make it easy to use the solution”-- Venki Ramakrishnan
16© Ahmed Hemani

BCPNN
Bayesian Confidence Propagation Neural Network
Professor Anders Lansner

17

BCPNN Requirements
Functional Requirements: Human Scale - Realtime
1. Realtime simulation
2. 2 Million HCUs – non-deterministically concurrent
3. 170 TFlops/s – BCPNN Computation
4. 50 TBs – Synaptic Weight Storage
5. 200 TBs / s – Bandwidth for synaptic storage
6. 250 GBs / s – Spiking Bandwidth

Infrastructural Requirements
18

The BCPNN Computation Model
Human Scale Cortex Dimensions

HCU = S MCUs

High Level of
Temporal Locality

10 000 Spikes/s

10 000 Connections

MCU
State Vector

MCU Row
HCU
SynapticMemory(25 MB)
100 MCUs

Column updates
are more
expensive

Input Spike
Computation
Output Spike
ComputationDelay 100 × 100
Buffer Spikes/s

SupportComputation
100 / s19

Infrastructural Operations are Significant

170 TFlops
Input Computation
Unit R1 SP FPUs

Output Computation
Unit R2 SP FPUs

Input Computation
FSM

Output Computation
FSM

oSDIN
outgoing Spike Distribution Interconnect

Scratchpad Memories
Output Computation

Outgoing Spikes Queue & Controller

Scratchpad Memories
Input Computation

Output Computation Controller

Input Computation Controller

Incoming Spikes Queue & Controller

iSDIN
incoming Spike Distribution Interconnect

Infrastructural Operations

Delay Buffers & Controller for fanout spikes

HCU State Storage Memory Interface

ms Timer20© Ahmed Hemani

The Silicon Lego Bricks for Method Applied to BCPNNA Structured Physical Design Scheme to enable System-level synthesis
TSVs + Controller
NOCInterface
Ques +Controller

SRAMs

H- TileH- TileFPUs

H- TileNOC Corridor

H- TileH- TileNOC Corridor

H-TileController

H- TileH- TileH- TileH- Tile21

1489mm

240mm

Bank 0: 128 Mb

H-Cube8 layers of DRAM
1 Bank per layer
2 Banks / HCU
TSV Micro Channel
4 HCUs + Control

HCU0HCU2

BCU: Brain Computation Unit
1.529 X 1.729 mm232 H-Cubes
32 Micro ChannelsHCU1HCU3RIBBCPNN Implementation

1079 mm

Column

200 mmTSV Area

250 mmIn Collaboration withProf. Nobert When andDr. Christian WiessTU Kaiserslautern

22

BCPNN: ASIC vs GPUs(b) Energy Breakdown GPUs(a) Energy Breakdown ASIC

Infrastructure
3%

Computation
15%

SRAM
9%

Infrastructure20%

ASIC: 3.0 kW
GPUs: 2.6 MW

1.52

DRAM
73%

Energy Delay Product: 3.06 kJ  s

SpiNNaker-2
comparable
to GPUs

1320
Computation
20%# of GK210 cores
Energy1s Realtime
Energy Delay ProductMemory60%: 5000: 563.1 kJ: 4.69 s simulated time
: 2642 kJ  s23© Ahmed Hemani

The Impact of Column Access Elimination + Exploiting
Temporal Locality
Baseline

Infrastructure
3%

Column Access EliminatedComputation
27%

Computation
15%

SRAM
9%

Column Access Eliminated +Temporal Locality

Computation
63%

Infrastructure
6%

1.52

SRAM
8%

DRAM
14%

DRAM
59%

DRAM
73%

3 kW

0.40

0.

1.88 kW

50 GFLOPs / Watt
28Infrastructure
nm bulk CMOS
SRAM
10%
13%
Sustained– not peak
800 W

Such optimizations can be automaticallyinferred from Simulations© Ahmed Hemani

24Interconnect and Storage are Expensive
3.2 pJ = 32 bit Data 1 mm ~= 32-bit FLOP > accessing 1 bit in 3D integrated DRAM

6.3 pJ

P. Kogge and J. Shalf, “Exascale computing trends: Adjusting to the ‘new normal’ for computer architecture,” Comput. Sci. Eng., vol. 15, no. 6, 2013.
© Ahmed Hemani

25Computation in Memory using Memristors
Benefits:
1. Single cycle dot product2. Can be extended to do addition,multiplication, element wise multiplication,
matrix inversion
3. No need to fetch, decode and execute
instructions  addresses wire problem
4. In some application instances, initialization ofmatrix would be a one-time event

DACsADCsReminiscent of Analog Computation

Challenges
1. Large matrices will need to be fragmented
resulting in movement of data. Needcomplimentary control circuitry
2. ADC’s consume significant power and injectlatency
3. Accuracy
4. Experimental solutions reported. Not part ofmainstream design flow

Source of Diagram Above: Chenchen Liu, Qing Yang, Bonan Yan, Xiaocong Du, Hai (Helen) Li, “A Memristor Crossbar Based Computing Engine Optimized for High Speed and Accuracy”, ISVLSI 2016
© Ahmed Hemani26

Memristor based CIM in the SiLago FrameworkRegion Types – SiLago Block Types
Functional

Infrastructural

Graph Theory

NOCsOuter Modem

Scratch Pad Memory

Inner Modem

PLL + CGU

Protocol Processing

Power Management

Spectral Methods

Memory Controller

Dense Linear Algebra

FIFO, FIFO Controller

Sparse Linear Algebra

RISC Processors – RISC-V

Dynamic Programming

NVM

State Machines

DRAM Vaults

Memristor CIM

1. A Memristor CIM in a range of dimensions
2. Characterized with post-layout data and circuit
level simulations and validated with test chips
3. Exports, functional matrix operations andinfrastructural operations like initializing crossbar,
NIU operations, reg file operations etc.
4. Higher abstraction synthesis tools can refine interms of CIM SiLago blocks and know its
performance, energy and area.
27© Ahmed Hemani

25 Watt Biologically Plausible Human Scale Brain
Single Precision Floating Point
Move to 16/32 bit Integer Arithmetic

Caveat:Based on best effort estimates andnot on actual implementation

1. Synaptic Storage/Access will reduce by ~50%
2. Computation Energy will reduce by ~75%
~ 2 TOPs/watt
28 nm bulk CMOS

~800 Watts~250 Watts

ReRAM
Computation in Memory~25 Watts
28© Ahmed HemaniWave Based Computing using Plasmons
1. Logic values encoded as phase of the waves
2. Interference of waves interpreted as majority
gate computation

29© Ahmed Hemani

Plasmonics + CMOS Computing using SiLago blocks
SiLago micro-architecture block (full adder)

Plasmon
Detector

SiLagoMajority Gate

Demodulator

OutputCoutA
Cin

CMOS Logic
Drivers

SUM

B

SiLago Interconnect
waveguide

Plasmon detector

Plasmonwaveguides© Ahmed Hemani

High Speed
Plasmon Logic Circuit

PlasmonSources

2N112233NN

Plasmonic
Wave-computingInputs
THz
Phase modulators

THz CMOS
Driver and Control

Electrically driven
Plasmon sources

Digital CMOS
Control & Memory

1PhaseModulatros

PlasmonExa – Plasmonics based Exa-scale computing design.
Synthesized in terms of Silicon Lego (SiLago) blocks.

30Impact

1. 1000 X Power Density
2. More Affordable

Software Centric / GPU +Based Computing© Ahmed Hemani

1. 1000 X Power Density
2. More Affordable

Hardware Centric
SiLago Based Computing
31