Extreme Big Data - Tools and Library
Next Generation Big Data Infrastructure Technologies
Towards Yottabyte / Year

Extreme Big Data (EBD) Overview
Our project , called EBD, aims to achieve theconvergence of extreme supercomputing and big
data in order to cope with explosion of data frommultiple sources such as massive numbers of sensors whose resolution is increasing exponentially,high resolution simulations generating huge dataresults, as well as evolution of social infrastructures that allow for “opening up of data silos”, i.e.,
data sources being confined within an institution,much as how scientific data are being handled in
the modern era as common asset openly accessible within and across disciplines. Our primarytarget proxy applications include metagenomics,social simulation, climate simulation with real-timedata assimilation, and machine learning. Based onthese EBD co-design applications, we define
future EBD convergent SW/HW architecture andsystem. We have several on-going collaboration
work with RIKEN AICS, ORNL, LLNL, ETH andJST Graph CREST / Univ. Kyushu.

This Research is Supported by JST, CREST[1]SpGEMM on GPU

Increasing GPU Occupancy

We have devised new Sparse General Matrix-Matrix Multiplication
algorithm on GPU, which achieves further speedups and reduces
memory usage so that various matrix data can be applied byutilizing GPU’ s on-chip shared memory and appropriate assigningof GPU resources.
Two Phases Algorithm : 1st phasecounts the number of non-zero(1) Count the number of intermediate products of each row
elements of output matrix, and 2nd(2) Group rows by the number of intermediate products
phase calculates the output matrix(3) Count the number of non-zero elements of each row of outputmatrix for all groups with CUDA kernels and streams
Reduce memory usage(4) Set row pointers of output matrix to store in CSR by Thrust scanGrouping rows (2, 4)
Better utilization of GPU resources (5) Group rows by the number of non-zero elements
Two ways threads assignments(6) Calculate the output matrix for all groups with CUDA kernels
and streams
Improve the load-balancea. Calculate values and column indices on hash tableb. Shrink table to hold only non-zero elements
Hash table on fast shared memory c. Sort by column index in ascending orderAccelerate (3)counting part and(6)calculation partcuSPARSE

BHSPARSECUSPPROPOSAL

25320

2.515cuSPARSE

105

BHSPARSE

Fig.1: Job 1 and Job 2 cannot run concurrently
as Job 2 wants two unoccupied GPUs on thesame node.

F ig .2 : T h e ar c h i tec ture o f mrC U D A , o u r
middleware for handling remote GPU migration
on top of rCUDA.

PROPOSAL

1.5

0.5
0CUSP
109876543210

BHSPARSEPROPOSAL

10.3% reduction onaverage from cuSPARSE

Economics

CircuitEpidemiology

webbase[1] Dalton et al., “CUSP: Generic parallel algorithms for sparse
matrix and graph computations ver.0.5.1”

makespanis reducedby ~30%

10Maximum Memory Allocation Ratio
compared to cuSPARSE

Multi-GPU batch-queue systems usually experience large numberof idle GPUs due to the scattered idle-GPU problem (Fig.1). We
addressed this problem by allowing jobs to utilize remote GPUs andmigrating execution on a remote GPU back to a local GPU as soon
as one becomes available. This method enables the systems toserve more GPU jobs concurrently while minimizing execution timepenalty caused by remote GPU communication.

Speedup is up
to x4.0

2GFLOPSGFLOPSCUSP[1] S. Matsuoka, et al. “Extreme Big Data (EBD): Next generation big data infrastructure technologies
towards yottabyte/year”, Supercomputing frontiers and innovations, 2014.

[2] NVIDIA, “Nvidia cuda sparse matrix library (cuSPARSE)”[3] Liu et al., “An Efficient GPU General Sparse Matrix-Matrix
Multiplication for Irregular Data” , IPDPS2014

For more details, please visit SC Regular Poster“Fast Sparse General Matrix-Matrix Multiplication
on GPU with Low Memory Usage”
Acknowledgments. This research was supported by JST, CREST (Research Area: Advanced Core Technologies for Big Data Integration).

Fig.3: GPU occupancy patternswhen using
FCFS (top) and our method (bottom).

Fig.4: Distribution of jobs’ lifetime (waiting +execution time) decrease when using ourmethod compared with FCFS.

Reference: P.Markthub, A.Nomura, and S.Matsuoka, “Serving More GPU Jobs, with Low Penalty, using Remote GPU
Execution and Migration,” IEEE Cluster 2016.
Acknowledgments. This research was supported by JST, CREST (Research Area: Advanced Core Technologies for Big Data Integration).

http://www.gsic.titech.ac.jp/sc16