(3+3&2,/3)" !$
-03.13%#   2018 511

'(*.

1••••••

HPC/#1 '",/0'$%.1 /#1 '%0")+-1#
*1&!(1%")+-1#2TraceRP3


2/,.36'!
•&4+6(/')145/,.36'
••/,.36'!" HPC)*-0!33%*/,.36'!
••24'! ! $# !%/,.36'!

*Kogge, P. M.: Architectural Challenges at the Exascale Fron- tier, Simulating the Future: Using One Million Cores and Beyond (invited talk) (2008)3 On/Off
Activeモード

低電力モードPdynamicモード遷移

Pbase

消費電力

タイムアウト モード遷移

Activeモード

パケット処理完了時刻

時間

パケット パケット処理
到着時刻
開始時刻

4$)A0B+5*4-9;?B0"
•

524@B*$••

&(%'#-9;?B0($-9;?B0••• CFBigSim*ENsim**D
/ CFDimemas***D$•/-9;?B0$

HPC-/3:$#! 'C8>1.<6D(,7B4On/Off=A*"8>1.<6(,7B4 '-9;?B0

*Zheng, G., et al. : BigSim: a parallel simulator for performance prediction of extremely large parallel machines, 18th International Parallel and Distributed Processing
Symposium, 2004. Proceedings. (2004).
**Hideki, M., et al. : NSIM: An Interconnection Network Simulator for Extreme-Scale Parallel Computers, IEICE Transactions on Information and Systems, Vol. E94.D, No. 12,
pp. 2298–2308 (2011).
***Saravanan, K., et al. : Power/Performance Evaluation of Energy Efficient Ethernet (EEE) for High Performance Computing, Proceedings of the 2013 IEEE International
Symposium on Performance Analysis of Systems and Software, pp. 205–214 (2013).

5TraceRP•••••• ),  #&+ '*" ($&-#On/Off),&-#•'*" ($TraceR%-!6

TraceR
ProgramBigSimTrace

Configuration(Topology etc.)

User

TraceR

Performance

CODES

ROSS

Jain, N., Bhatele, A., White, S., Gamblin, T. and Kale, L. V.: Evaluating HPC Networks via Simulation of Parallel Work- loads, Proceedings of theInternational Conference for High Performance Computing, Networking, Storage and Analysis, pp. 14:1–14:12 (2016).

7

TraceR
Program
Trace

BigSimConfiguration(Topology etc.)

TraceR

Performance

CODES

UserROSS•BigSim*•••$15';)39:17&420,9#
CHARM++ #"($17+*# /-.8;%.6;* # !

*Zheng, G., Kakulapa1, G. and Kale ́, L. V.: BigSim: a parallel simulator for performance predic1on of extremely large parallel machines, 18th Interna+onal
Parallel and Distributed Processing Symposium, 2004. Proceedings. (2004).

8

TraceR
Program
TraceBigSim

Configuration(Topology etc.)

TraceR

Performance

CODES

UserROSS•

ROSS*•• ')+.#%,"! %,"! #(!#-% &-$! ).*

*Carothers, C. D., Bauer, D. and Pearce, S.: ROSS: a high- performance, low memory, modular time warp system, Proceedings Fourteenth Workshop
on Parallel and Distributed Simulation, pp. 53–60 (2000).

9TraceR
Program
Trace

BigSimConfiguration(Topology etc.)

TraceR

Performance

CODES

UserROSS•CODES*••

HPC  '*! (* 
$%'*
Model-net! (*#&)+')
" 

*Mubarak, M., Carothers, C. D., Ross, R. B. and Carns, P.: Enabling Parallel Simulation of Large-Scale HPC Network Systems, IEEE Trans. Parallel Distrib.
Syst., Vol. 28, No. 1, pp. 87–100 (2017).

10

TraceR
Program
Trace

BigSimConfiguration(Topology etc.)

TraceR

Performance

CODES

UserROSS•

TraceR*••"!$#%   %

*Jain, N., Bhatele, A., White, S., Gamblin, T. and Kale, L. V.: Evaluating HPC Networks via Simulation of Parallel Work- loads, Proceedings of theInternational Conference for High Performance Computing, Networking, Storage and Analysis, pp. 14:1–14:12 (2016).

11

TraceRP
Program
Trace

BigSimConfiguration(Topology etc.)

TraceR

PowerUser

CODES

 

Performance ROSS

12

TraceRP

Activeモード


低電力モード

TSTW

Pdynamicモード遷移

Pbase

消費電力

タイムアウト モード遷移

Activeモード

パケット処理完了時刻

時間パケット パケット処理到着時刻
開始時刻

13

TraceRP:#7$
•

•
•••

#••••••

ING* ;ActiveING:, ?%1=7
Up/DownJM@2>3>6On/OffJM@?
On/OffJM@ &:, ; <8CODES:DNCANG?
FHLBPFat-treeOTorusODragonfly
Ts7Tw; Off:+?
, ING9(/1=7Tw:)?

$•

"!9.-50>4IEK7'$

14,'.#



Reviriego!On/Off-/$#,'.*Power per port Nport

*%&(min(1, D · ⇢i )Tf
D=Twake + Tsleep + TfActive,0)
" Pbase / port

,0)
"  LoadTransfer

i=1Pdynamic

Wake

P = Pbase + Pdynamic

XSleep+0(Transfer•Tsleep

Twake Tf

*P. Reviriego et al., "An energy consumption model for Energy Efficient Ethernet switches," 2012 International Conference on High
Performance Computing & Simulation (HPCS), Madrid, 2012, pp. 98-104.

time

15


•



 •5-7••••ping-pong!

Ts#••($+* 2!)<3/7# 19&64•8$,;' 2<."!# 0+.:<%
••0+.:<%
••($+* 2!)<3/7# 19&64•ping-pong
8$,;'!

16

&(•••

MPI3;,96*8,1923;/.%$)#&*

4<05=+3;,96•••••

near-neighbor>  71-83;,96
permutation>&3;,96qbox>  '!")MPI stentcil4d>.2<-:3;,96a2a>All-to-All

17   ! &#

Fat-tree

Torus

Dragonfly

&

288

256

342

%%

323232 &

24612 &12

13

$&

3-level3D torsu
4x8x8

"& 619"&

18On/Off
600[us]17[us]

TSTW

Pdynamic

1.36[W]

Pbase

消費電力

タイムアウト モード遷移

2.08[W]

モード遷移時間

パケット処理完了時刻

パケット パケット処理
到着時刻
開始時刻

10BASE-T

Energy Efficient Ether19   , Fat-tree

正規化消費電力

正規化実行時間

1.2

1.2

↓27%

0.8

1

↑1.2%

0.8

0.6

0.6

0.4

0.4

0.2

0.2

00

near-neighbor permutation

qbox

プログラム

stencil4d

a2a

正規化実行時間

正規化消費電力1

20   , Torus

正規化消費電力

正規化実行時間

1.2

1.2

↑4.0%

1

↓33%

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

00

near-neighbor permutation

qbox

プログラム

stencil4d

a2a

正規化実行時間

正規化消費電力

121, Dragonfly  
正規化消費電力



正規化消費電力

11.2

↑0.6%

1

↓26%

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

00

near-neighbor permutation

qbox

プログラム

stencil4da2a

正規化実行時間

1.2

正規化実行時間

22•


•

•

Ts

Fat-treeTs 



•
•
Torus

Tw=17
Ts=10, 50, 250, 1250
Dragonfly



23



 Ts!
Fat-tree•••Ts 


•
Torus

Tw=0Dragonfly

24



Tw


Fat-tree•••Tw 


•
Torus

Ts=0Dragonfly

25
51   8,192
正規化消費電力

正規化消費電力

1.2

正規化実行時間

1.2

11

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

00

Fat-treeTorus

トポロジ

Dragonfly

正規化実行時間••26/,0,.
•/,0•
•
•

><=KM5.&-*2 &NOn/OffHL5O$.#&,3!
126CEJM:(
6CEJM:NTraceRPO3%
••••Fat-treePTorusPDragonfly-BI;7F@-On/OffHL5. ?GDM:3 

.
•
•

><=KM5"3 )+ '4MAL9M8,)+6CEJM:3%