ǐǲǋķÅǱǅǡǧǭǲǅǩǰĬOǇǲǗǲǂǰǙǧǲǋ ReedbushƜƂƇƭǇǑǭǲǆǅǇǏǢ
Ähp `_øǉǰǋǲ
ǇǲǗǲǂǰǙǧǲǏƵǰƿûĈŔŚb ±GõįƞÌĳ• ĥ½• ReedbushǅǇǏǢƞė• ǇǑǭǲǆǅǇǏǢ• ǝǰǍǠǲƾƜƪƭļ!
• STREAM, HPL, HPCG
• Intel MPI Benchmark• IOR• ƥƙƧ

2016/10/18¹ÂLustreǨǲǄ20162Äh`_øǉǰǋǲƞǇǗǂǰ
2_ƞh]ǅǇǏǢǶ6ǃƶƾǬFY
08 09 10 11 12 13 14 15 16 17 18 19 20 21 22
Hitachi SR11K/J2
IBM Power-5+

18.8TFLOPS, 16.4TB

Yayoi: Hitachi SR16000/M1
IBM Power-7
54.9 TFLOPS, 11.2 TB

Hitachi HA8000 (T2K)
AMD Opteron
140TFLOPS, 31.3TB

Oakleaf-FX: Fujitsu PRIMEHPC
FX10, SPARC64 IXfx
1.13 PFLOPS, 150 TB

Oakbridge-FX

136.2 TFLOPS, 18.4 TB

Peta

2016/10/18

K

K computer¹ÂLustreǨǲǄ2016

Post-K3

ǐǲǋķÅǱǅǡǧǭǲǅǩǰĬO
ǇǲǗǲǂǰǙǧǲǋǅǇǏǢǻ{0ƞĥ½• ǟǇǑT2K‒ čÚhƙ3PƘFǅǇǏǢƱ{0¾/ċ3PHPC_ø·Ļ (Joint Center for Advanced HPC:JCAHPC)‒ 201612¿őíśl r• ÄhƞƦƘ{0‒ Oakleaf/Oakbridge-FX (FX10) ÞŢĜU‒2015\Ƅ80%Ń

‒ ǟǇǑT2KĂĭƞƑƧƞ ŜöƛǅǇǏǢ‒ Post FX10ǳ2018þą*śl rǴƞǗƶǮǎǑǅǇǏǢü ǐǲǋķÅǶDeep LearningƛƚŷµĴ;í8ŗś©
2016/10/18¹ÂLustreǨǲǄ20164

0
2012-04
05
06
07
08
09
10
11
12
2013-01
02
03
2013-04
05
06
07
08
09
1011
12
2014-01
02
03
2014-04
05
06
07
08
09
101112
2015-01
02
03
2015-04
05
06
07
08
09
101112
2016-010203;íé
10090

2016/10/1880+% Average
Oakleaf-FX + Oakbridge-FX

Oakleaf-FX

Oakbridge-FX

Yayoi

807060% 5040302010¹ÂLustreǨǲǄ201655ÄhǉǰǋǲƞǇǗǂǰ
2_ƞh]ǅǇǏǢǶ6ǃƶƾǬ
FY08 09 10 11 12 13 14 15 16 17 18 19 20 21 22
Hitachi SR11K/J2
IBM Power-5+

18.8TFLOPS, 16.4TB

Yayoi: Hitachi SR16000/M1
IBM Power-7
54.9 TFLOPS, 11.2 TB

Hitachi HA8000 (T2K)
AMD Opteron

Oakforest-PACS
Fujitsu, Intel KNL

25PFLOPS, 919.3TB

140TFLOPS, 31.3TB

Oakleaf-FX: Fujitsu PRIMEHPC
FX10, SPARC64 IXfx
1.13 PFLOPS, 150 TB

Post T2K
JCAHPC:čÚhǱÄhPost FX1050+ PFLOPS (?)

Oakbridge-FX

136.2 TFLOPS, 18.4 TBǐǲǋķÅǱǅǡǧǭǲǅǩǰĬOǇǲǗǲǂǰǙǧǲǋPeta

2016/10/18

K

Reedbush, SGI
Broadwell + Pascal
1.80-1.93 PFLOPS

K computer¹ÂLustreǨǲǄ2016

Post-K6

7

ǟǇǑT2KǻOakforest-PACS
http://www.cc.u-tokyo.ac.jp/system/ofp/• 201612¿1¹ą*śl• 8,208 Intel Xeon/Phi (KNL)ǶǙǲƾĦ25PFLOPS• xdŋƄÍđ

• ¾/ċ3PHPC _ø·Ļ(JCAHPC: Joint Center
for Advanced High Performance Computing)• čÚhpĹĎÿpûĈǉǰǋǲ• Ähp`_øǉǰǋǲ• ÄhpÈƽǥǰǗǇƞÄhp`_øǉǰǋǲ4Ɯŷ Ñŝƞ²ģVƄ ƙƛƔƗĻĹƍƭǇǲǗǲǂǰǙǧǲǋǅǇǏǢƱĻĞ
ƋǶ¾/ċƞhĴÐŰĦĹĎ_øƱÍđǱőXƍƭƑƧƞĘĝ• http://jcahpc.jp

2016/10/18¹ÂLustreǨǲǄ20167GPUƞ{0ƈƮƥƘǻ• CUDAƜƪƭǜǮƿǪǡǰƿǻĲŢ• 2,000ƱŃƁƭǨǲǄǲ
• CUDAƱ@ƋƗƨƫƀƞƟhe

• OpenACC• OpenMPƙůƋƑƶǰǋǚƸǲǇǻſƩƍſƄĦƃƔƑ»ƞĦRǶCUDAƙƐƮƣƚhƅƛƄƛƆƛƔƑ• ǐǲǋÿpǶÝpĠǳDeep LearningǴ• ÃƞĹĎÿpǶĹĎp8ŗƙƟñƛƔƑ8ŗƞµĴǨǲǄǲś©
ƄB• ÄhpǁǕǢEÿpûĈÑÍ• Ähpóş
• Eôï+6êơƞÝpĠœí¹ÂLustreǨǲǄ2016

2016/10/18

8

ReedbushǅǇǏǢ (1/2)• ǅǇǏǢÍ£Ǳőíǻ¹ÂSGI => HPE?
• Reedbush-UǃǛǅǇǏǢǻCPUƞƦ
• Intel Xeon (Broadwell-EP) x 2ǊǀǎǑ
• 256GB DDR4-2400 ǣǤǫ• 420 Ǖǲǒ, 508.0 TF• 20167¿ą*śl

• Reedbush-HǃǛǅǇǏǢǻGPU¯Ň• CPUǱǣǤǫǻReedbush-U ƙPÎ• NVIDIA Tesla P100 (Pascal GPU)• 120 Ǖǲǒ, 145.2 TF(CPU)+ 1.15~1.27 PF(GPU)=
1.30~1.42 PF• 20173¿ą*śl
2016/10/18¹ÂLustreǨǲǄ20169ReedbushǅǇǏǢ (2/2)• ǇǑǭǲǆ/ǚƳƶǬǅǇǏǢ•9ǚƳƶǬǅǇǏǢ (Lustre)• 5.04 PB, 145.2 GB/sec• ŰŌǚƳƶǬƽǥǎǅǧǅǇǏǢ: Burst Buffer (DDN IME(Infinite Memory Engine))• SSD: 209.5 TB, 385.2 GB/sec• ƶǰǋǲǂǔƾǑ: 100G bps / Ǖǲǒ• Reedbush-UŷǇǑǭǲǆ: InfiniBand EDR (100G)• Reedbush-H: InfiniBand FDR (56G) x2• Ť=, 5H, ĻĞũć• ĉ5, 368.4 kW (5HŠƆ)• < 90 m2
2016/10/18¹ÂLustreǨǲǄ201610

ReedbushǅǇǏǢgĵ

2016/10/18¹ÂLustreǨǲǄ201611

“Reedbush”ƙƟ
• L'homme est un roseau
pensant.
• Man is a thinking reed.
• ŜƟġƁƭīƘžƭ
Pensées (Blaise Pascal)
Blaise Pascal(1623-1662)

2016/10/18¹ÂLustreǨǲǄ201612ĹĎǕǲǒ: 1.795-1.926 PFlops

Reedbush-H (w/Accelerators)
1287.4-1418.2 TFlops

Reedbush-U (CPU only) 508.03 TFlops
CPU: Intel Xeon E5-2695 v4 x 2 socket(Broadwell-EP 2.1 GHz 18 core,
45 MB L3-cache)
Mem: 256GB (DDR4-2400, 153.6 GB/sec)

CPU: Intel Xeon E5-2695 v4 x 2 socket
Mem: 256 GB (DDR4-2400, 153.6 GB/sec)
Ų420 GPU: NVIDIA Tesla P100 x 2
Ų120(Pascal, SXM2, 4.8-5.3 TF,
Mem: 16 GB, 720 GB/sec, PCIe Gen3 x16,
NVLink (for GPU) 20 GB/sec x 2 brick )SGI Rackable
C2112-4GP3

SGI Rackable C1102-PL1

InfiniBand EDR 4x
100 Gbps /nodeDual-port InfiniBand FDR 4x56 Gbps x2 /node

InfiniBand EDR 4x, Full-bisection Fat-tree
145.2 GB/s

9ǚƳƶǬ
ǅǇǏǢ
5.04 PB
Lustre Filesystem
DDN SFA14KE x3

Mellanox CS7500
634 port +SB7800/7890 36
port x 14385.2 GB/sŰŌǚƳƶǬƽǥǎǅǧǅǇǏǢ
209 TB
DDN IME14K x6

ďêǃǲǖǲ
ğ
Login
Login
Login
Login
Login
Login
node
node
node
node
node
node
ǮƿƶǰǕǲǒ x6

UTnet

ǨǲǄReedbush-HǕǲǒƞǛǮǎƾZǣǤǫ
128GB

DDR4
DDR4
DDR4DDR4Intel Xeon
E5-2695 v4(Broadwell-EP)

QPI
76.8GB/s

15.7 GB/s15.7 GB/s

PCIe sw

NVIDIA
Pascal

DDR4
DDR4
DDR4
DDR4ǣǤǫ
128GB

76.8GB/s

PCIe sw
20 GB/s
NVLinK
NVLinK
20 GB/s

G3x16

G3x16

IB FDR
HCA

G3x16

G3x16

76.8GB/s

76.8GB/s
QPIIntel Xeon
E5-2695 v4(BroadwellEP)

NVIDIA
PascalIB FDRHCA

EDR switch
EDR

2016/10/18¹ÂLustreǨǲǄ2016

14

1Í£
Ŭù

ǅǇǏǢ1

(ĹĎǕǲǒ)ěêŀáĎĦěǕǲǒ´ě

ĺ¢uŘǔǎǑǯǲƾǑǟǮǆǲ
9ǚƳƶǬǅ
ǇǏǢŰŌǚƳƶǬ
ƽǥǎǅǧǅǇ
ǏǢ

Reedbush-U

Reedbush-H

508.03 TFlops

1297.151417.15
TFlops420

120

105 TByte

30 TByte

Full-bisection BW Fat-TreeǅǇǏǢQ

LustreǚƳƶǬǅǇǏǢǃǲǖ(OSS)

DDN SFA14KE

ǃǲǖ(OSS)´

3ǇǑǭǲǆuŘ

5.04 PB^

145.2 GB/secǃǲǖ

DDN IME14Kǃǲǖ´6uŘ

209 TByte^

385.2 GB/secŬùǠǅǰQCPUǜǮǉǎǃQ

Reedbush-U

Reedbush-H

SGI Rackable C2112-4GP3

SGI Rackable C1100ǅǫǲǈ(śõ )Intel Xeon E5-2695v4 (Broadwell-EP)
2 (36)

ǜǮǉǎǃ´(ǂƴ´)
TÚ´

2.1 GHz (Turbo boost ¼¾h 3.3 GHz)
1209.6 GFlops

êŀáĎĦ
Memory256 GB

uŘ

153.6 GB/secǣǤǫ^
GPUǜǮǉǎǃQNVIDIA Tesla P100 (Pascal)

ǂƴ´ǳFǴ56 (SM)ǣǤǫuŘǳFǴ

16 GBǣǤǫ^ǳFǴ720 GB/sec

êŀáĎĦǳFǴ

None4.85.3 TFlops2¯Ň´CPU-GPUŜ­Ě

PCI Express Gen3 x16ǭǲǰ
(16 GB/sec)GPUŜ­Ě

NVLink 2 brick (20 GB/sec x2)

ƶǰǋǲǂǔƾǑInfiniBand EDR 4x(100 Gbps)

InfiniBand FDR 4x 2ǫǰƾ(56 Gbps x2)

9ǚƳƶǬǅǇǏǢInfiniBand EDR• DDN SFA14KE 3ǉǎǑ• OSSƟǂǰǑǮǲǪ ƞǠǅǰƙƋƗA

Active/StandbyOSS• 1ǉǎǑ 2ǂǰǑǮǲǪ

• HDD: 360 x 6TB NLSAS(35 RAID6 (8D2P) +10 ǇǞƴ)• ǂǰǑǮǲǪƑƬ
IB-EDR x 4ǟǲǑ(Active-Standby x 2 Ę)
• uŘ: 5.04 PB

SAS 12Gx 10

OSSCtrl0…

OSSCtrl0…

SAS 12Gx 10

Enclosure 0• 6TB * 8Â * 35RAIDƿǬǲǜ * 3ǉǎǑ• ǖǰǒǻ145.2 GB/sec• (EDR:100G *2ǟǲǑ)* (64b66bƹǰǂǲǐƵǰƿ)* 2 ǂǰǑǮǲǪ
* 3ǉǎǑ

2016/10/18

OSS¹ÂLustreǨǲǄ2016

Enclosure 1
Enclosure 2
Enclosure 3
Enclosure 417

ǚƳƶǬƽǥǎǅǧǅǇǏǢ
• DDN IME14K 6ǉǎǑ (12Ǖǲǒ)• 1ǉǎǑ 2Ǖǲǒ• SSD: 48 x 800GB NVMe

• ǕǲǒƑƬ
IB-EDR x 3ǟǲǑ
ǳƀƓ1ǟǲǑƟGen3 x8ǟǲǑƞ
ƑƧ 64GbpsƘŌ)• uŘ: ē209 TB• 1ǉǎǑƑƬ 38.4 TB, erasurecoding 10D1PƘ®ĎInfiniBand EDR

Ǖǲǒ0

Ǖǲǒ1

• ǖǰǒǻ385.2 GB/sec• ((EDR:100G *2ǟǲǑ)* (64b66bƹǰǂǲǐƵǰƿ)+ (PCIe Gen3 x8: 64G * 1ǟǲǑ)* (128b130bƹǰǂǲǐƵǰƿ))* 2 Ǖǲǒ
* 6ǉǎǑ

2016/10/18¹ÂLustreǨǲǄ201618

ǇǑǭǲǆǪǎƾƞÍ£ǳ3ǪǎƾǴ
• 2 x IME, ǂǰǑǮǲǪ(SFA14KE)+ 5 x ƹǰƾǮǲǆǥ

2016/10/18¹ÂLustreǨǲǄ2016

19

SFA14KEĥũ

2016/10/18¹ÂLustreǨǲǄ201620

IME14K ĥũIME
1M8ǳ2ǕǲǒǴ

2016/10/18¹ÂLustreǨǲǄ201621

ļ!ƜíſƑǊǚǑƷƸƴëc
• ǂǰǗƶǪ
• Intel C/Fortran Compiler 16.0.3• ´&ǪƶǛǪǫ
• Intel Math Kernel Library 11.3.3• MPI• Intel MPI 5.1.3• SGI MPT 2.14
• Mellanox HPC-X 1.6.392• Open MPI 1.10.2• MVAPICH2 2.2rc1
2016/10/18¹ÂLustreǨǲǄ201622ǝǰǍǠǲƾļ!
YƟ Reedbush-U ƞƦ• STREAM• HPL : ǕǲǒFŷ1Ē
• HPCG: ǕǲǒFŷ1Ē
• Intel MPI Benchmark• (mdtest)• IOR

2016/10/18¹ÂLustreǨǲǄ201623

STREAM• ǂǰǗƶǬƺǜǅǩǰǻ -xCORE-AVX2 -O3-qopt-streaming-stores always-DSTREAM_ARRAY_SIZE=400000000• KMP_AFFINITY=scatter
140120

Bandwidth (GB/sec)• FX10 1Ǖǲǒ(65GB/s)= BDW 1ǊǀǎǑ
• 130GB/s: êŀǙǲƾ 85%

100

Copy

80

Scale

AddTriad

135

60

125

40

115

20

1020

1520
# of Cores

30

010
2016/10/18¹ÂLustreǨǲǄ2016

25

30

3524

Broadwell-EPƞÍ£

Core
#0

L
1

L
2L3

Core
#5

L
1

L
2

Core
#1

L
1L
2L3

Core
#6L
1L
2Core
#2

L
1L
2L3

Core
#7

L
1

L
2

Core
#3L
1L
2L3Core#4L
1L
2L3DDR4DIMM

ǂƴƑƬL1ǐǲǋ: 2KB, L2: 256KB, L3:2.5MB(3À) => L3 Ɵ1Ƙ45MB

PCIe

QPI x2Core#8L
1L
2Memory

Memory16GB Ų2Æ

16GB Ų2ÆǺǊǀǎǑƞƦƱZüL3Core#9L
1L
2L3Core#14L
1L
2L3L3Core#10L
1L
2L3Core#15L
1L
2L3L3Core#11L
1L
2L3Core#16L
1L
2

L3L3Core#12L
1L
2L3Core#17L
1L
2L3Core#13L
1L
2L3Memory

Memory16GB Ų2Æ16GB Ų2Æ76.8 GB/ĀǊǀǎǑƑƬǣǤǫŘǻ16GBŲ8Ǽ128GB
2016/10/18¹ÂLustreǨǲǄ2016=(8ByteŲ2400MHzŲ4 channel)
25

HPL• Intel MKL 11.3.3 ƜSƥƮƗſƭǖƶǓǫ
xhpl_intel64_dynamic Ʊí• ǕǲǒF• 2MPI x 18Ǉǭǎǒ
• N=98112, NB=192• P=1,Q=2• ęÇǻ 1149.6 GF(ǙǲƾÖ 95.0%)
2016/10/18• 1Ē 420Ǖǲǒ
• (1MPI x 36Ǉǭǎǒ) x 420• N=1538688,NB=192• P=20,Q=21• ęÇǻ 446.3 TF(ǙǲƾÖ 87.8%)¹ÂLustreǨǲǄ201626

HPCG 3.0
• Intel MKL 11.3.3 ƜSƥƮƗſƭǖƶǓǫxhpcg_avx2 Ʊí• ǕǲǒF• 2MPI x 18Ǉǭǎǒ
• nx=ny=nz=144• ęÇǻ 21.9 GF(ǙǲƾÖ 3.6 %)2016/10/18• 1Ē 420Ǖǲǒ
• (2MPI x 18Ǉǭǎǒ) x 420• nx=ny=nz=144• Intel MPIƘǂǰǗƶǬ=> SGI MPTƱƀƑƧƞǪǎǗƱ;í• ęÇǻ 8457.78 TF(ǙǲƾÖ 1.66%)¹ÂLustreǨǲǄ201627

Intel MPI Benchmark• MPIsİƱÖņ

• Intel MPI 5.1.3• SGI MPT 2.14
• Mellanox HPC-X 1.6.392 … Open MPI 1.10.3rc4• (Open MPI 1.10.2)• MVAPICH2 2.2rc1• Mellanox CS7500 (Switch-IB2¯Ň)ƞªƕCollective
OffloadingÑĦ
• FCA (Fabric Collective Accelerator) 3.5 … SGI MPT, HPC-X• SHArP (Scalable Hierarchical Aggregation Protocol) …
HPC-X• ŅŘǜǮǑǂǬ OpenUCX (United CommunicationX) … HPCX
2016/10/18¹ÂLustreǨǲǄ201628

Pingpong űǭƶǏǰǅǴ• ¾|: HPCXƜƂſƗUCXƱƔƑaO: 1.1 us

4Intel MPI
MPT
MPT+FCA
HPCX
HPCX+UCXHPCX+FCA
MVAPICH2

3.5

Latency (usec)

32.5
2
1.5
10.5
0
1864

5124096

Message size (byte)
2016/10/18¹ÂLustreǨǲǄ201629

Pingpong űǖǰǒǴ
12000
Intel MPI
10000

Bandwidth (MB/sec)• ¾h: MVAPICH2
11.56 GB/sec• 128KBƟ
HPCX+UCXƄǝǇǑ
• SGI MPTƄſǽMPT
MPT+FCA8000

HPCX
HPCX+UCX

6000

HPCX+FCA
MVAPICH2

4000
2000
01

16256

4096

65536

1048576

Message size (byte)

2016/10/18¹ÂLustreǨǲǄ201630

Barrier
Communication time (usec)• FCAƜƪƬ3%(SGI MPT), 1.86%(HPCX)ŰŌ
• SHArPƜƪƬ3.2%ŰŌ
• 420ǕǲǒƘƨ
4.6usƋƃƃƃƫ
ƛſ(!)

25

Intel MPI
SGI MPT+FCA
HPCX+UCXHPCX+SHArP20

SGI MPT
HPCX
HPCX+FCA
MVAPICH21510502

2016/10/18

8¹ÂLustreǨǲǄ201632# of Nodes

128

512

31

Allreduce
20

Communication Time(usec)• 64byteƥƘƟ
SHArPƘƟ6usă

100001000 100141664

100Intel MPI
SGI MPT+FCA
HPCX

10

HPCX+FCAHPCX+SHArP

11864

512409632768 262144

Message size (byte)
2016/10/18¹ÂLustreǨǲǄ2016

32

IOR ű$:ǚƳƶǬǴ
• ǜǮǉǇƉƙƜñƛƭǚƳƶǬƜƴƾǉǇ• Intel MPI, -b 1m -t 1m -s 12000 (Lustre) -s 6000 (IME)

9ǚƳƶ
ǬǅǇǏǢ
(Lustre)ŰŌǚƳƶǬƽǥǎ
ǅǧ (IME)

2016/10/18

I/O

Rƅ

MPIIO

Write

76519

76194

77962

Read

77484

78568

79715

Write

74308

77653

76774

Read

79585

79944

80092

Write

217422220005Read196172204259

Write

218212220852

Read

196849208592

POSIX
MPIIO
POSIX

420Ǖǲǒ
420Ǖǲǒ
420Ǖǲǒ
x1 ǜǮǉǇ x2 ǜǮǉǇ x4 ǜǮǉǇ

Lustre: êŀǙǲƾĦ 145.2 GB/sec ƞ 50~55%
IME: êŀǙǲƾĦ 385.2 GB/sec ƞ 51~57%¹ÂLustreǨǲǄ201633

IOR űFǚƳƶǬǴ
• FǚƳƶǬ ƞ:ƺǚǉǎǑƜƴƾǉǇ
• Intel MPI, -b 1m -t 1m -s 7000 (Lustre) -s 12000 (IME)

9ǚƳƶ
ǬǅǇǏǢ
(Lustre)ŰŌǚƳƶǬƽǥǎ
ǅǧ (IME)

I/O

Rƅ

MPIIO

Write

20456

25549

Read

27225

35997

Write

19775

20211

22419

Read

28796

43939

33505

Write

180496173619129778Read

247192252049245318

Write

149260146783133707

Read

240619243813246661

POSIX
MPIIO
POSIX

420Ǖǲǒ
420Ǖǲǒ
420Ǖǲǒ
x1 ǜǮǉǇ x2 ǜǮǉǇ x4 ǜǮǉǇ

Lustre: êŀǙǲƾĦ 145.2 GB/sec ƞ 13~30%, $:ǚƳƶǬƜÖƢĦ
IME: êŀǙǲƾĦ 385.2 GB/sec ƞ 33~65%, $:ǚƳƶǬƙPČƃƐƮ
2016/10/18¹ÂLustreǨǲǄ2016

34

Burst BufferƜÁƍƭƈƙ• hŘƞ3ÀǐǲǋƞǳÖņöǴŰŌƛIä• ǁǕǢ6êƜƂƇƭǐǲǋǝǲǇ• hĴÐƛwĭ9ǐǲǋ•…

• hĴÐƴǜǫŎ£ƞCoupler• [Ŧǵæ• h×ǵÜÛ•…

• (Oakforest-PACSƞǏǇǑǝǎǒǴ

2016/10/18¹ÂLustreǨǲǄ2016

35

36

¹ÂLustreǨǲǄ2016ƥƙƧ
• Reedbush– Reedbush-U: CPUƞƦ20167¿ą*ślŷ9¿Óőíśl– Reedbush-H: GPU (NVIDIA Tesla P100)¯Ň20173¿ą*ślŷ4¿ÓőíślǇǑǭǲǆČƟU, HƘ3À• ¾µƞǂǤǐƵǏƵ§ĮƱ{0ƍƭƈƙƘŷŰſĦƱƭƙƙƨƜúÁŜƘqrą*– Broadwell-EP, InfiniBand EDR, Burst Buffer (IME)– (NVIDIA Tesla P100)– ¹ÂSGI, DDN Japanƞ÷ÎƜ Ł (_o_)• Burst BufferƜƕſƗƟƥƒĦļ!Ŋ
2016/10/18