Dealing with Deeper Memory Hierarchy
in Post-Petascale Era
Project Overview

Target Architecture

On Exa-scale supercomputers, the “MemoryWall” problem will become even more severe, which
prevents the realization of Extremely Fast&Big Simulations.
This project promotes research towards this problem via co-design approach among application algorithms, system software, architecture.

We consider near future HPC systems where each node has deeper
memory hierarchy that consists of heterogeneous memory including
NVM. We do not exclude usage of shared layer such as burst buffers.

Comm/BW reducingalgorithms+High Bandwidth Memory (HBM):DRAM chips are 3D-stacked with TSV technology.
This has advantage in bandwidth over traditional DDR,
but capacity tends to be limited.

System software formem hierarchy mgmt

+HPC Architecture withhybrid memory devices
HMC, HBM

Next-gen NVM O(GB/s) Flash

Target: Realizing Extremely Fast&Big simulations ofO(100PB/s) & O(PB) in Exa-scale era

Next-Gen Non-volatile Memory (NVM):
Several kinds of NVM such as STT-MRAM, ReRAM,FeRAM, 3D Xpoint will be available in near future.
They have different properties in BW and capacity.
NAND Flash devices:
Not only SSDs with traditional SATA/SAS interfaces,
but recent PCIe/m.2 SSDs with O(GB/s) bandwidth
are already available.

HHRT: System Software for Swap Integration with App Algorithm
Motivation
Towards achieving fast&big simulations, we need to exploit highspeed of upper memory layer (e.g. GDDR/HBM on GPUs) and large
capacity of lower memory layer (e.g. NAND Flash).
However, programming with consideration of memory hierarchy istroublesome.
Overview
To make memory hierarchy programming easier, we implemented
system software, named HHRT (hybrid hierarchical runtime).
● HHRT automatically supports data swapping between GPUmemory and lower layer memory (host memory, Flash SSD)
● HHRT supports “process-wise” swapping, not “page-wise” like OS.
● HHRT is used by user programs written in MPI and CUDA. It is awrapper library for MPI/CUDA.
● Programmers still have responsibility to improve locality for better
performance
Execution Model

Device memory

cudaMemcpy
Process’s data

MPI comm●●●Host memory

With HHRTNodeDevice memoryMPI commProcess’s data

GPU mem
capacity

Host mem
capacity

Integration with Real Simulation Application

w/o HHRT (typically)
Node

Temporal blocking for stencil computation
HHRT enables “larger” execution then upper memory, but app execution suffers from larger swapping cost, especially for stencil computation that has worse memory access locality.
A well known technique, “Temporal Blocking” (TB), which improves locality of stencil, achieves reasonable performance on
HHRT.
7-pointstencilon
Temporal Blocking
a K40 GPU & 950pro m.2 SSD●●●

Host memory

https://github.com/toshioendo/hhrt

We integrated our techniques withstencil-based city airflow simulation.

Airflow performanceon a K20X GPU

Original code on
MPI+CUDA was developed by N. Onodera and T. Aoki.
We integrated TB into
it and executed on
HHRT.

8x larger
simulation thandevice memory!

T. Endo, G. Jin: Software Technologies Coping with Memory Hierarchy of GPGPU Clusters for Stencil Computations. IEEE Cluster 2014.
T. Endo, Y. Takasaki, S. Matsuoka: Realizing Extremely Large-Scale Stencil Applications on GPU Supercomputers. IEEE ICPADS 2015.
T. Endo: Realizing Out-of-Core Stencil Computations using Multi-Tier Memory Hierarchy on GPGPU Clusters. IEEE Cluster 2016.

PI: Toshio Endo (Tokyo Tech, endo@is.titech.ac.jp), supported by JST-CREST

http://www.gsic.titech.ac.jp/sc16