A Case for Co-scheduling for Hybrid Memory Based Systems[Extended Abstract]
Eishi Arima

Carsten Trinitis

The University of Tokyo
Tokyo, Japan
arima@cc.u-tokyo.ac.jp

Technical University of Munich
Munich, Germany
Carsten.Trinitis@in.tum.de1

INTRODUCTION&!)*+%)*+%)*+

$!
#!

"!

$!
#!

"!
!

! & "' #% $# %! %& (' '%0(,+(%1*"$23[1] Streaming%)*+%)*+!"#$%&'"()"*+,*-$./""!
!

! & "' #% $# %! %& (' '%0(,+(%1*"$23[3] Lulesh%)*+&!)*+&!)*+

%!

$!
#!

"!

$!
#!

"!
!

!

! & "' #% $# %! %& (' '%0(,+(%1*"$23"!
!

%!

#!

#!

[2] Amg&!)*+$!

$!

! & "' #% $# %! %& (' '%0(,+(%1*"$23%!

&!)*+%!

!"#$%&'"()"*+,*-$./"!"#$%&'"()"*+,*-$./"!"#$%&'"()"*+,*-$./"&!)*+%!

!

!"#$%&'"()"*+,*-$./"

Manycore architectures and hybrid memory designs using emerging memory technologies (e.g., 3D stacked DRAM and Non-VolatileRAM) have been adopted in recent HPC systems, and the total system throughput/bandwidth have been significantly improved bythem. However, at the same time, these technology trends incur
more significant resource wastes within a node as CPU/memory intensive applications do not effectively utilize memories/CPUs. To
mitigate the resource wastes, prior researches have shown that coscheduling, i.e., launching multiple applications on one node, is apromising approach, but they mostly did not consider the combination of co-scheduling and hybrid memory systems. In this work,
we explicitly target the combination and make a case for it. Our
preliminary evaluation clarifies the following key insight: the problem size of each application plays an important role in optimizing
co-scheduling strategies for hybrid memory based systems.

%)*+%!

!"#$%&'"()"*+,*-$./"

ABSTRACT! & "' #% $# %! %& (' '%0(,+(%1*"$23! & "' #% $# %! %& (' '%0(,+(%1*"$23[3] Snap[1] Mcb[2] MiniFE
HPC systems will continue to face the severe memory bandwidth/Figure 1: Thread-level scalability v.s. problem size
capacity limitations, and hybrid memory systems, which arecomposed of multiple different memory technologies, are promising to combat the problem. On the one hand, 3D stacking memohas not been well-studied yet. In this work, we explicitly target
ries, such as HBM [4], are effective to increase the memory bandthe combination and make a case for it by deploying a prelimiwidth, but they suffer from serious capacity limitations [5]. On thenary evaluation. In particular, we demonstrate the following key
other hand, Non-Volatile RAMs, such as 3D Xpoint memories [2, 3],insight: the problem size of each application plays a key role inare promising to considerably scale memory capacity, however,
optimizing co-scheduling for hybrid memory based systems.
their performance is much lower than that of conventional DRAMs.
Due to the trade-off in bandwidth and capacity, hybrid memory ar2 EVALUATION ENVIRONMENT
chitectures are promising design choices [5, 8, 9].
We target systems with emerging hybrid memories, which are hetMeanwhile, to improve the processor performance, the induserogeneous in terms of bandwidth and capacity (e.g., MCDRAMtry has gradually increased the core count in the last decade. As+ DDR4 in Knights Landing [5] or systems using Optane DIMa consequence, many-core processors have appeared on the marM/SSD [2, 3] as DRAM extensions). In this work, we use a Knights
ket [5] and are used in various HPC systems. However, at the same
Landing based system as an example whose settings are listed intime, it has become more and more difficult to efficiently convert
Table 1. Note that our observations shown later are caused by thehigh core counts into performance due to the thread-level scalabilbandwidth/capacity heterogeneity in the memories and thus will
ity limitations caused by sequential code portions, shared resource
happen on any of our target systems. In the evaluation, we utilize
contentions, and intensive memory accesses.
the hardware-based approach (cache mode) for the data manageCo-scheduling, i.e., launching multiple applications on one sinment on the different memories. As for the workloads, we chosegle node, is a well-known approach to mitigate such resource wastes [1,Streaming as well as several mini applications chosen from CORAL
6]. On the one hand, memory intensive applications need less probenchmark suite [7].
cessor cores as performance is restricted by the memory system(e.g., bandwidth). On the other hand, CPU intensive applications
3 KEY OBSERVATION: THREAD-LEVEL
need more cores but consume less memory bandwidth. Therefore,
SCALABILITY V.S. PROBLEM SIZE
the resources can be more efficiently utilized by simultaneously
Figure 1 demonstrates how thread-level scalability depends on theexecuting different kinds of applications on one single node.
Although prior studies have shown the effectiveness of co-scheduling, problem (or data) size. As shown in the figures, applications generally scale worse when the problem size is larger on our system.
the combination of co-scheduling and hybrid memory systems

# of Cores# of Cores (Appi)

64
56
48
40
32
24
16
80(4,4) (80,4) (4,80)(4,4) (80,4) (4,80)

Streaming-Streaming
Data Sizes of (Appi, Appj) [GiB]# of Cores (Appj)(4,4) (80,4) (4,80)

Streaming-Amg(4,4) (80,4) (4,80)

Streaming-LuleshStreaming-MCB
Application Pairs: Appi-Appj(4,4) (80,4) (4,80)(4,4) (80,4) (4,80)

Streaming-miniFE

Streaming-Snap(4,4) (80,4) (4,80)(4,4) (80,4) (4,80)

Speed-up overtime sharing

Figure 2: Optimal core resource allocations
100%
80%
60%
40%
20%
0%-20%(4,4) (80,4) (4,80)(4,4) (80,4) (4,80)

Streaming-Streaming
Streaming-Amg
Data Sizes of (Appi, Appj) [GiB](4,4) (80,4) (4,80)(4,4) (80,4) (4,80)

Streaming-Lulesh
Streaming-MCB
Application Pairs: Appi-Appj

Streaming-miniFEStreaming-Snap

Figure 3: Performance improvement brought by co-scheduling

RelativePerformance

Time Sharing

Co-scheduling: optimized for (80,4)

Co-scheduling: optimized for (4,80) - correct

1.4
1.2
10.8
0.60.4
Streaming-Streaming

Streaming-Amg

Streaming-Lulesh
Streaming-MCB
Application Pairs: Appi-Appj

Streaming-miniFE

Streaming-Snap

Figure 4: Performance comparison at (4,80) (= data sizes of (Appi, Appj)[GiB])
NameCPU
MemoryOS
Compiler

5 CONCLUSION AND FUTURE CHALLENGES

Remarks
XeonPhi 7210, 64cores, 1.3GHz
MCDRAM (Fast Mem.): 16GiB, 450GiB/s
DDR4 (Large Mem.): 96GiB, 90GiB/s
CentOS 7
ICC 19.0.1.144

In this work, we made a case for co-scheduling on a hybrid memory based system and demonstrated that the optimal co-schedulingpair selections and resource allocations highly depend on the problem sizes of applications. In our future work, we are going to develop a problem size aware co-scheduling algorithm and extend a
conventional co-scheduling framework to support it.

Table 1: System configuration
This is because when the data size is increased, the large but slow
memory is more frequently accessed, which shifts the performance bottleneck from the CPU or the fast memory to it. Asfor SNAP, the scalability is limited by the other reasons (e.g., interference among threads).

4

ACKNOWLEDGEMENTS
We thank Prof. Martin Schulz and the members of his researchgroup at TU Munich. This work is partly supported by JSPS Grantin-Aid for Early-Career Scientists (JP18K18021), and Research on
Processor Architecture, Power Management, System Software andNumerical Libraries for the Post K Computer System of RIKEN.

IMPACT ON CO-SCHEDULING

Figure 2 shows the optimal1 core allocations to two applications
when we execute them at the same time. As shown in the figure,this highly depends on the problem sizes of applications as thecore resource requirements are strongly related to their threadlevel scalability (see Figure 1). Then next, Figure 3 shows the speedup over Time Sharing 2 when the core allocations are optimized.
This figure suggests that the optimal co-scheduling pair choices
highly depend on the problem sizes. Finally, Figure 4 describes
the performance comparison, which suggests that a non-problemsize-aware core allocation approach can considerably degrade
performance that can be even worse than Time Sharing.

REFERENCES[1] Bhadauria, M., et al. An Approach to Resource-aware Co-scheduling for CMPs.
In ICS (2010), pp. 189–199.
[2] Intel. Intel® Memory Drive Technology, Set Up and Configuration Guide, 2019.
[3] Izraelevitz, J., et al. Basic Performance Measurements of the Intel Optane DC
Persistent Memory Module. arXiv preprint arXiv:1903.05714 (2019).
[4] JEDEC Standard. High Bandwidth Memory (HBM) DRAM. JESD235 (2013).
[5] Jeffers, J., et al. Intel Xeon Phi Processor High Performance Programming: Knights
Landing Edition. Morgan Kaufmann Publishers Inc., 2016.
[6] Jiang, Y., et al. Analysis and Approximation of Optimal Co-Scheduling on Chip
Multiprocessors. In PACT (2008), pp. 220–229.
[7] ORNL, ANL, and LLNL. CORAL Benchmark Codes. https://asc.llnl.gov/
CORAL-benchmarks/, 2013.
[8] Vetter, J. S., et al. Opportunities for Nonvolatile Memory Systems in ExtremeScale High-Performance Computing. Computing in Science Engineering 17, 2(2015), 73–82.
[9] Vijayaraghavan, T., et al. Design and Analysis of an APU for Exascale Computing. In HPCA (2017), pp. 85–96.

1 Optimal: minimizing total execution time2 Time Sharing: conventional exclusive solo-runs

2