ポストムーア時代の通信と計算のバランスとアクセラレータクラウド構想
工藤知宏
東京大学 情報基盤センター
産業技術総合研究所 情報技術研究部門
本発表で紹介する研究の一部はJSPS科研費16H02793の助成を受けたものです1Computation and Communication• Computation and communication have been two key componentsof IT infrastructure for a long time
– Integration of C and C is becoming more important• Data Intensive / AI / IoT applications• Heterogeneous computing for post-Moore era• C&C (Computer and Communication) has been a slogan (orsymbol) of NEC corporation since 1977
– At that time, the Internet was at the very early stage. For example, TCP/IP
did not exist. The word “Communication” was used for broadcast andonline computers.
– Now the term C&C usually stands for Command and Control servers,
which are used attackers to control malware affected computers…
2

Wide area network trend
Global Consumer Internet Traffic

Petabyte per Month

90000

60000

Energy Efficiency Trend in IP routers

Internet video
File sharing
Web, email, and data
Online gaming

10～12%/yr.

CAGR 21%3000002013

2014

2015
2016
Year

2017

2018

Source: Cisco VNI Global Forecast 2013-2018

Traffic Growth Rate

Source: C. Lange, et. al., IEEE JSTQE, vol. 17, no.2,
pp. 285-295, March/April, 2011, Fig. 10

Energy Efficiency
Improvement Rate

Concern about energy consumption incommunication networks is increasing

Ever-increasing video related traffic• Increasing users due to YouTube + SmartPhone• Higher definition: HD→４K→８K• Demands to real-time share of video→uncompressed
CAGR

Global Consumer Internet Traffic
120,0000%

100,000

22%

80,00060,00040,00033%

20,0000

2014
Internet video

2015

2016

Web, email, and data

2017
File sharing

2018

2019

Online gaming

Source: Cisco Visual Networking Index:Forecast and Methodology, 2014–20194T. Asami and S. Namiki, ECOC 2008

Japan’s Case

100,000
10,000100,000
Japan’s Total
Electricity Generation
In 2005

10,000

1,000

1,000

100

Need forEnergy
Savings
By 3-4 Digits

10100
101.0

10.1

0.1
2050

2000

2010

2020

2030

2040

Total Internet Traffic （Tbps)

Annual energy consumption of IP Routers（TWh）

The Fundamental Problem of the Internet• The current technologies can’t scale to the increasing traffic in future.
• 3-4 digit energy saving is necessary, which means we need a new paradigm.
5

Network extends to Cloud• Networks (especially organizational networks such ascompanies and universities campus networks) provides
many functions– Routing, Firewall, Access control, Redundant services etc.

• As the available bandwidth of the internet increases,functions can be moved to cloud– Use of network function virtualization (NFV)

6

Network Service Cloud (NSC)• Network functions (router,firewall etc.) implemented as
VNF (Virtual NetworkFunction) and built up in Cloud• Can be connected fromanywhere (incl. outside ofcampuses)• Campus network can be verysimple• Just provide physicalnetwork connectivity

Network Service Cloud
Firewall
ProxySD-WAN

Campus A
7

RouterCampus B

Advantages of NSC• Cost efficient– Cloud infrastructure shared by many organizations– Allocate resources according to demands– Centralized operation reduces cost for 7D/24H• Reliable– Failed resources can be replaced with new one dynamically• Position independent– Can be accessed from anywhere– Disaster prevention by using data centers at distant places8Challenges for NSC• Cloud for “network”– Current clouds are not designed for network– Bandwidth hungry applications require stable guaranteedperformance– Coordination of “physical” and “virtual” is an issue• Including connections between clouds and campuses• Interoperation with AAA/policy– Need to support different kind of members who have different
rights/duties• Manageability
9– Network managers of different organizations should be able tomanage NFV functions according to their needs/policies

Cloud extends to network• Recently, IoT is gathering wide attention– Computing is done at the edge, middle of the network andclouds• Network is not just pipes anymore. It is a manageable
resource– Software Defined Network (SDN)• Cloud (esp. IaaS) extends to network and edge

10

5G, IoT, Cloud11

http://www.chinadaily.com.cn/business/tech/2016-02/23/content_23608976.htm

IoT: Edge, Cloud, Fog

12https://leanbi.ch/wp-content/uploads/2018/02/Fog_Achitecture.png

5G impact: requirements of 5G

http://www.3glteinfo.com/when-5g-coming/13https://ars.els-cdn.com/content/image/1-s2.0-S2352864817301335-gr3.jpg
14

Wide area network challenges• Coordination of many different kinds of resources– Network (SDN)– Edge devices, Fog devices (cf. MEC (Mobile Edge Computing) Servers)– Cloud resources, VNF– Services– IoT support• Huge number of edge devices• In and outside campuses• Computing and communication should be more tightly coupled

15

Data Centers : Current• General purpose CPUs and storages are connected by anetwork such as Ethernet– Almost homogeneous.
– Recently, GPUs and FPGAs are being introduced as coprocessors of servers.

• Expect to stay in the main stream for the next 5 years• Cost of data movement between servers is high– Near data processing: avoid moving data as much as possible

16

Data Center Challenges after 2023
• End of Moore’s law– No more performance advances by shrinking transistors– Need new architecture for data centers• Emerging new applications– Data processing, esp. machine learning– More operations per clock/transistor• Neuromorphic, analog, quantum computing etc.
• Need to move more data to/from processing engines

17

Control flow to Data flow centric• Data flow of emerging applications (machine learning, bigdata) are rather simple• Circuit switched network will be sufficient for suchapplications

18

Ideal Heterogeneous System
OS / “Control Plane”

DPUGPUGPUGPUGPUGPUDPUDPUGPUDPUFPGAFPGAOptical Network

Storage class memory

19FPGA

1. For resource utilization efficiency, cloud(shared pool of resources) is advantageousfor heterogeneous systems

FPGA

2. To provide performance guaranteed slices tousers, bare-metal type slice is required

3. Make use of ultra-wide bandwidthinterconnection
Curtesy of R. Takano (AIST)Flow-in-Cloud (FiC): Disaggregated accelerator cloud

Slice

Slice

Computeservers
FiC Accelerator NetworkFlowOS
meta
accelerator

FPGA20GPU

SCM

Resource pool of “engines”

Configuring “meta-accelerators”
according to the application requirement
Curtesy of R. Takano (AIST)

Accelerator bare-metal cloud
FPGAFPGASensors

FiC Network

Data

Server Network

Data
FPGAFPGAGPUGPUGPUGPU
MEM
MEM
MEM
MEM21

• For resource
utilization
efficiency, sharedpool of resources isadvantageous forheterogeneoussystems• To provide
performance
guaranteed slices
to users, baremetal type slice isrequiredAccelerator bare-metal cloud

DataSensors

Server Network

Data

GPU
GPU

MEM
MEM

GPU
FPGAGPUGPUGPU

22

FPGAFPGA• For resource
utilization
efficiency, sharedpool of resources isadvantageous forheterogeneous
systems• To provide
performance
guaranteed slices
to users, baremetal type slice isrequired

FiC Switch and FPGA Board

8Gbps full dup x 8• A prototype of FiC switch andFPGA board is working.
• FiC switch provides circuit
switching with time-divisionmultiplexing.

DDR4 PC4-2130016GBFPGAController

Xilinx Kintex
Ultrascale XCKU095

Raspberry Pi 3 model B

DDR4 PC4-2130016GB
23

Data Movement is Critical• Data movement is unavoidable on a heterogeneous
system and the bottleneck to achieve higher performance• E.g., ABCI@AIST– 12 GFlops/W ⇔ 83 pJ/Flop

Data Movement Energy• GFlops/W = GFlop/second / Joule/second Access to SRAM
Access to DRAM
= GFlop/Joule– Energy per bit budget: 0.42 pJ/bit

O(10 fJ/bit)
O(1 pJ/bit)

Movement to HBM

O(10 pJ/bit)

Movement to DDR3 off-chip

O(100 pJ/bit)• 200 bits/Flop• Scaling performance is getting harder under tight energybudget
24

Curtesy of R. Takano (AIST)System in a Package (SiP)

NVIDIA Tesla P100 https://www.nvidia.co.jp/object/tesla-p100-jp.html• SoC and HBM2 memory are put on top of a siliconinterposer• Avoiding pin bottleneck of the package
25

HBM and GPU integration on an interposer• Silicon interposer
is used to mitigate
the bin-bottleneckproblem ofpackages.
• Intel announced
EMIB which uses
smaller silicon tointerconnect dies
https://japan.cnet.com/article/35069635/
26

SiP introduced new “layer” in communication

Inter-DC / Wide area communication
Intra-DC communication (inter-rack)
Intra-rack communication (inter- chasis)
Intra-chasis communication (inter-board)
In-package
communication(inter-die)On-board
communication(inter-package)
Intra-die communication (inter-transistor)

27• Realizable bandwidth andpower consumption differs
layer by layer

Extremely wide bandwidth can be a breakthrough• Take advantage ofheterogeneous computing to
improve performance after
the end of the Moore’s law– Use of “engines” suitable for
special purposes• At function processing– Data have to be moved
between engines• Bandwidth is the key
Communication bandwidth larger than DRAM’swill change the whole scenario of computing
28

Inter-Server interconnect BW trends• Ethernet is widelyused• InifiniBand for HPC– Mellanox Oligopoly• Intel OmniPath

InfiniBand Roadmap (http://www.inifnibandta.org)

29

Trends of electrical links• 50Gbpsis used inInfiniBand(HDR)• 6~7times /10 years
improvement• Short range only forwide bandwidth
Trends of copper-based electrical links fromrecent 10-year ISSCC papers.
Jeong, G.-S.; Bae, W.; Jeong, D.-K. Review of CMOS
Integrated Circuit Technologies for High-Speed Photo-Detection.
Sensors 2017, 17, 1962. より引用

30

– Mellanox’s InfiniBand
HDR copper AOC: upto 3 meters)

Limit of electrical I/O and possibility of Optical I/O• Limits of bandwidth of electrical I/O– Currently, 50Gbps/ch, slow performance improvement– Not all pins of a package can be used for high speed I/O• New packaging technologies :MCP/2.5D/3D– SiP (System in Package)• Multiple chips are integrated to a single package• Bandwidth between chips can be improved• Limitation in size. For example, integratable memory size is limited.

31

Requirements for future DC networks• Moving bulk-data is important– In the “At-Function Processing”, data should be moved
between processing “engines”• Data movement performance and cost comparable toDRAM access should be realized– >5Tbps end-to-end bandwidth for bulk data transfer• DRAM bandwidth :HBM2 (3D stack DRAM): 1TBytes/s (4 stacks)– <10pJ/bit end-to-end power consumption• DRAM power consumption : DRAM: 5pJ/bit or more32

More bandwidth for inter-acc. network• OCS (Optical Circuit Switching)– Electrical switching requires O/E/O conversion of all the data →Switching of multiple 10s of Tbps links is unrealistic• Hybrid use of EPS and F-OCS (Fast-OCS)– EPS for short and low bandwidth flows– F-OCS for long lasting wide bandwidth data flows• Take advantage of future data centers’ traffic profile– Machine learning dominates the data center load
– Control flow is combined with data flow
• Relatively static bulk data communication patterns
33

Electrical packet switching and Optical circuit switching~ 10 nJ/bit
IP: Process every packet for routing

Suitable for small granularity data for different destinations~ W/port
Optical Path: Set up end-to-end paths

Suitable for bulk-data transfer with QoS guarantee
34Complementary aspects → The hybrid use!
• EPS

• F-OCS– Low energy efficiency– High energy efficiency– Good affinity with computing– Good for mice flows– Control plane needed– Good for elephant flows

Hybrid network for highly energy efficient, flexible switching for all granularity

35

Current status of optics for interconnects• OBO(On-Board Optics)– COBO (Consortium for OBO):
Microsoft, Juniper, Cisco,Broadcom etc.

• Inter Package I/O– Integrate OEO devices in a SiP– Remove pin BW bottleneck of apackage

36

Samtec FireFly

Rockley OpticsDirect

Optical integration in a package

https://www.3dincites.com/2014/02/411-cea-letis-interposer-roadmap/37

Optical Switch Technologies
MEMS or Piezoelectricbeam-steering

Space optics

http://www.glimmerglass.com/

http://www.polatis.com/index.asp

J Port count: ~ 384x384
L Physical size: can be large
J Low insertion loss and crosstalk
L Switching speed: 10 ms ~Waveguide

PLC or Silicon Photonics

AWG router J Port count: ~ 2000x2000
L Tunable laser required
K Tradeoff between
the port count andthe per port bandwidth
K. Ueda, et al., OFC2015, W3D.1
LCOS or MEMS
K Port count: ~ 1x20
L Switching speed:
10s ~ 100s ms
K Insertion loss: ~6dB
https://jp.finisar.com/
J Wavelength routing

WSS

Fast switch
http://www.nttelectronics.com/index.htm

38K. Tanizawa, et al., OFC2015, M2B.5

K Port count: ~ 32x32
J Physical size: can be very small
L Insertion loss, crosstalk, and PDL
J Switching speed: 3 ms, 30 us

PLZT, SOA, LN, DLP WSS…
J Switching speed: 10 ns ~L Port count: ~ 4x4
L Insertion loss, crosstalk, and PDL
Curtesy of K. Ishii (AIST)

Ultra-wide Bandwidth Optical Datacenter Network•

Wavelength Bank (WB):

 Single DWDM light source in a system: Distributed tocomputing nodes via optical amplifiers
 No light source is required at each computing node: lowcost, low power• Silicon photonics optical circuit at each node

 De-multiplex, modulate, multiplex and transmit
 Enables hybrid implementation with electronics

WDM Light Source Rack

Optical frequency
comb is distributed toeach compute node

Silicon Photonics Optical Switch Racks
Server Racks

Transmitter Details

39

MUX

DEMUX

Modulator array

Advantages:- No hermetic seal
- No thermoelectric cooler- High frequency accuracy

Optical transmitter unit can be monolithicallyimplemented with Si-ph technologies

Target in 2025
Bandwidth: 10 Tbps / fiber
Energy: 1 ～ 10pJ/bit# of wavelength# of levels / chBandwidth / fiber

1120 Gbps

48640 Gbps

3285.12 Tbps

Courtesy of S. Namiki (AIST)

Conclusion• Integration of computing and communication is becoming
more and more important• We have to consider power consumption andperformance balance of computing and communication• There are many layers in communication, andperformance and power consumption differ layer by layer• Optical communication can bring a breakthrough in IT
infrastructure40