A Case for Co-scheduling for Hybrid Memory
Based Systems
1,2
2
Eishi Arima , and Carsten Trinitis

1The University of Tokyo, 2Technical University of Munich

1. Background

2. Motivation and Goal

Recent technology trends:

ü Manycore processors (e.g., A64FX, KNL, GPUs) for higher arithmetic throughput
ü 3D stacked DRAMs (e.g., wide-I/O, HMC, and HBM[1]) for higher memory bandwidthü Emerging NVRAMs (e.g., Optane DIMM/SSD [2,3]) for larger memory capacity

Promising node architecture (our target):

Manycore processor + hybrid memory system

Problem: increasing resource wastes within a node

1. Waste of processor core resources when memory intensive
2. Waste of memory bandwidth resources when CPU intensive
ü The wastes become even worse when these resources are scaled

Promising approach: co-scheduling [6]Manycore processor

ü Co-running multiple jobs on one single node at the same timeü Mixing CPU/memory intensive jobs is effective

Goal: to understand the impact of co-scheduling on the
recently emerging hybrid memory based systems

Hybrid main memory
Knights Landing[5]Memory pool with IMDT[3] Future exa-scale machine[4]Fig. 1: Target systems (extracted figures)ü Note: the combination of co-scheduling and hybrid memory system is notwell studied in prior studies
ü To do so, we utilized the system summarized in the table below

3. Summary

TABLE1: Evaluation settings

Our new insight: problem size awareness is quite
important when co-scheduling multiple applications on
the hybrid memory based nodeü The followings highly depend on the problem sizes:
1) Optimal selections of co-run application pairs2) Optimal resource allocations to them

CPU Package

XeonPhi 7210, 64cores, 1.3GHz, quadrant mode, x1 socket

MemorySystemMCDRAM(1st memory): 16GB 450GB/s, DDR4(2nd memory): 96GB
90GB/s, Data management: hardware cache mode

OSCent OS 7

CompilerIntel C++/Fortran Compiler 19, Options: -O3, -qopenmp, -xMIC-AVX512

Workloads

Streaming + CORAL benchmark[7] (AMG, LULESH, MCB, miniFE, SNAP)

4. Experiment
Optimal† core allocations when co-runningü The CPU resource requirement (= thread-level scalability) depends on the problem size
– The 2nd memory (large but slow) is more frequently accessed when the problem size isscaled, which changes the bottleneck

ü The optimal settings also depend on the problem sizes due to the scalability changes

10

constant**

16 is enough0

0 8 16 24 32 40 48 56 64# of threads
4GiB

40

*

20
*

10

48 is
enough0

20

32 is enough0 8 16 24 32 40 48 56 64# of threads

0 8 16 24 32 40 48 56 64# of threads

4GiB

*

80GiB

48 is
enough

10

0

40

*

20

*

10

80GiB

80GiB

30

30

MiniFE

30

*

4GiB

40

64
56
48
40
32
24
16
8
0

(4,4) (80,4)(4,80)(4,4) (80,4)(4,80)
Streaming-Streaming
Streaming-Amg

0

4GiB

80GiB

20

20
*

10

24 is enough0
0 8 16 24 32 40 48 56 64# of threads

10*

32 is enough0

0 8 16 24 32 40 48 56 64# of threads

Contact Info:
Eishi Arima, The University of Tokyo
Email: arima@cc.u-tokyo.ac.jp

Acknowledgement:This Poster &Extended Abst.

Application Pair: Appi-Appj

We thank Prof. Martin Schulz and the members of his research group at TU Munich forvaluable discussions. This work is partly supported by JSPS Grant-in-Aid for Early-Career
Scientists (JP18K18021), and Research on Processor Architecture, Power Management,
System Software and Numerical Libraries for the Post K Computer System of RIKEN.

60%
40%
20%

-20%

(4,4)(80,4)(4,80)(4,4)(80,4)(4,80)
Streaming-Streaming Streaming-Amg

Data Size of (Appi, Appj) [GiB](4,4)(80,4)(4,80)(4,4)(80,4)(4,80)
Streaming-Lulesh
Streaming-MCB(4,4)(80,4)(4,80)(4,4)(80,4)(4,80)
Streaming-miniFE Streaming-Snap

Application Pairs: Appi-Appj

Fig. 4: Performance improvement over time sharing‡

Impact of problem size awareness on co-run performanceü A non-problem-size-aware resource allocation policy will not work well
– We should rethink the policy too
Time Sharing‡
Relative
Performance1) Developing problem size aware co-schedulingmethodology/algorithm
2) Extending a conventional co-scheduling framework
to support our approach3) Evaluating our strategy using various hybrid memory
systems(e.g.,DRAM DIMM + Optane DIMM/SSD[2,3])(4,4) (80,4)(4,80)
Streaming-Snap* Core settings are optimal†ü Speed-up highly depends on the problem sizes– We should revisit the pair selection policy in co-scheduling to consider this80%0%Fig. 2: Thread-level scalability vs data size

5. Future Challenges(4,4) (80,4)(4,80)
Streaming-miniFE

Performance improvement (co-run opportunities)

30

30(4,4) (80,4)(4,80)
Streaming-MCB

Fig. 3: Core allocation breakdowns

0 8 16 24 32 40 48 56 64# of threads

40# of Cores (Appj)(4,4) (80,4)(4,80)
Streaming-Lulesh

Data Size of (Appi, Appj) [GiB]

Snap

*

# of Cores (Appi)

Speed-up over
Time Sharing‡

20

Mcb

Relative Performance

le

s ca

80GiB

Relative Performance

30

4GiB

40

LuleshRelative Performance

40

80GiB

Relative Performance

4GiB

Amg

Relative Performance

Relative Performance

Streaming# of Cores

CPU resource requirement analysis (solo-run)

1.4
1.2
1
0.8
0.6
0.4

Co-run: optimized core settings for (80,4)Co-run: optimized core settings for (4,80) - correct

Large
Gap

Streaming-Streaming

Streaming-Amg

Streaming-Lulesh

Streaming-MCB

Streaming-miniFE

Fig. 5: Performance comparison at (4,80) [GiB]†Optimal: total execution time is minimized

Streaming-Snap‡Time sharing: conventional exclusive solo-runs

References:
1. JEDEC Standard. “High Bandwidth Memory (HBM) DRAM,” JESD235 (2013).
2. J. Izraelevitz et al. “Basic Performance Measurements of the Intel Optane DC Persistent Memory Module,”
arXiv preprint arXiv:1903.05714 (2019).
3. Intel. “Intel® Memory Drive Technology, Set Up and Configuration Guide,” 2019.
4. T. Vijayaraghavan et al. “Design and Analysis of an APU for Exascale Computing,” In HPCA, pp. 85–96, 2017.
5. J. Jeffers et al. “Intel Xeon Phi Processor High Performance Programming: Knights Landing Edition,” Morgan
Kaufmann Publishers Inc. 2016.
6. M. Bhadauria et al. “An Approach to Resource-aware Co-scheduling for CMPs,” In ICS, 189–199, 2010.
7. CORAL Benchmark Codes. https://asc.llnl.gov/CORAL-benchmarks/.