Low Power Consumption andGreen Operation

Power performance in Linpack benchmark : 3068.71 (MFLOPS/W)
Peak power consumption of system equipment : 1620 (KW)
Average power consumption of system equipment* : 698 (KW)
Idle power consumption for system equipment : 470 (KW)
Yearly average PUE : 1.285

Global Scientific Information and Computing Center, Tokyo Institute of Technology[*] Average power consumption is an annual average of TSUBAME2.0's.

Cooling : Modular Cooling System
Front

Green Operation : Monitoring of Environment
Temperature, power consumption, etc., are observed in real-time
not only in the computer room but also to compute nodes and toeach rack.

Rear

Collection of sensor information
Fan
modules

Water
group
AC transfer

Heatexchanger
modules

Water
group
controller

Fan
modules

Environmental monitoring system

Power andnetwork
connectors

Environmental
Monitoring DB

Distribution
board

Distribution
board

Storage rack
Conversion
processing

Network rack

Water inlet

operation

Water outlet

The rack-contained water-cooling system with a built-in heatexchanger is employed, allowing high-density cooling up to 35kW
per rack, which is the top class in the world. Homogeneous cooling
air is provided through the inlet of the server with automatic
open/close doors where a humidifier is unnecessary. Powerconsumption is minimized with a completely automatedtemperature control to enable heat removal from 95% to 97% bywater cooling. Moreover, polycarbonate doors contribute to a great
noise reduction.

ComputerAdministration
consoleadministrator

Distributionboard

Air conditioner
Distribution
board

Small space installation

2F

2F

1F

1F

Peak power consumption of
air-conditioning equipment : 460 (KW)
Average power consumption of
air-conditioning equipment* : 204 (KW)

Despite the fact the performance boost is more than 70 times
compared to TSUBAME1.2, the space required for installation has
narrowed down.

HARDWARE AND SOFTWARE SPECIFICATIONS■ System Software "Dynamic provisioning" dynamically switched between Windows and LinuxThe job management system and the cluster management system areworking together to manage user environment as well as distributing
computational resources to the insufficient part by taking from thenode pool. Both batch schedulers for Linux and Windows manage todynamically increase or reduce the compute nodes. The job scheduling also manages to support the execution of a virtual machine.

Large-Scale GPU-Equipped High-Performance Compute Nodes
OS

High-Speed Network Interconnect
High-Speed and Highly Reliable Storage Systems

Batch System

PBS Professional

Low Power Consumption and Green Operation
System and Application Software■ ISV (commercial ) Software( * GPU full or partial support )(As of November, 2013)

Compilers, Debuggers and Libraries

Applications

Intel Compiler ( C/C++/Fortran )PGI Compiler*( C/C++/Fortran, OpenACC, CUDA Fortran )Total View Debugger*
CAPS Compiler* ( HMPP, OpenACC )CULA* ( Numerical Libraries for CUDA )ANSYS Fluent*, Workbench*
MSC Nastran*
LS-DYNA
Gaussian, Gauss View
MolproScigress
MATLAB*
AVS/Express, AVS/Express PCE■：The license for all users

SUSE Linux Enterprise Server 11 SP1Windows HPC Server 2008 R2■：The license for Tokyo Tech users

ABAQUS*, ABAQUS CAEPatranCST STUDIO SUITE* (MW-Studio*)
AMBER*
Materials Studio, Discovery Studio
Mathematica*
Maple*
EnSight
System ( 58 Racks )■：The license for industrial users

Published by Global Scientific Information and Computing Center, Tokyo Institute of Technology

Rack ( 30 nodes )2-12-1 Ookayama, Meguro-ku, Tokyo 152-8550, JAPAN TEL : +81-3-5734-2087 FAX : +81-3-5734-3198 E-mail : tsubame@gsic.titech.ac.jp

http://www.gsic.titech.ac.jp/
2 01311

Compute NodeN
( 2 CPUs)
CPU , 3GPUs
3
4.08 TFLOPS.
58.0 GB (CPU ) + 18 GB (GPU )122 TFLOPS
2.28 TB

1442nodes
2952 CPU sockets :224.7 TFLOPS
Turbo boost
4360GPUs:
5.562 PFLOPS
Total:
5.787 PFLOPS
Memory:116 TBLarge -Scale GPU - Equipped High-Performance Compute Nodes

High-Speed Network Interconnect

Compute nodes consist of three types of nodes: Thin, Medium and Fat nodes. Thin nodes, which provide most of the overall
compute performance, are equipped with two CPUs and three Kepler core GPUs in a compact design 17/2 inches in width and 2U sizein height. In addition, two QDR InfiniBand HCAs are connected to dedicated PCI Express buses to secure the communication bandwidth.
Power supply units are organized with 3+1 redundancy, improving the node reliability significantly.

Compute nodes of TSUBAME2.5 interconnected with Dual-Rail QDR InfiniBand networks of Fat-Tree type full bi-section
bandwidth achieve 200Tbps. End-to-End latency between the compute nodes is extremely low in microsecond-order time,
therefore resulting in high-speed performance and high-speed connection to highly reliable storages.
This network is linked by more than 3000 optical fiber cables in a total length of 100km.

Thin Node 1408 nodes

Medium Node 24 nodes

Fat Node 10 nodes

Thin Node × 1408 (MCS racks : 1260 +others :148)

Medium Node × 24

Fat Node × 10

Med Node

Fat Node

1 MCS Rack (Thin Node × 30)
15 nodes15 nodesodes

×42
Thin N
Node
Th
dThin NodeThi

HP ProLiant SL390s G7
CPU:Intel Xeon X5670 (Westmere-EP, 2.93GHz) ×2 sockets
six-core per socket, total 12 cores per node.

GPU:

NVIDIA Tesla K20X (GK110) ×3. 1.31TFLOPS and VRAM 6GB per GPU

Memory: 58GB DDR3 1333MHz, partly 103GB
SSD:

120GB(60GB×2), partly 240GB(120GB×2)

Network: 4X QDR InfiniBand ×2

HP ProLiant DL580 G7

HP ProLiant DL580 G7

CPU: Intel Xeon X7550(Nehalem-EX )2.0 GHz ×4 sockets(32cores/node)
GPU: NVIDIA Tesla S1070(NVIDIA Tesla C1060×4) or
NextIO vCORE Express 2070
(NVIDIA Tesla M2070×4)
Memory: 137 GB (DDR3 1066MHz )SSD: 120GB ×4 (480GB/node )Network: 4X QDR InfiniBand

CPU: Intel Xeon X7550(Nehalem-EX )2.0 GHz ×4 sockets(32cores/node )GPU: NVIDIA Tesla S1070(NVIDIA Tesla C1060×4)
Memory: 274 GB (8 nodes ) ,548 GB (2 nodes )DDR3 1066MHz
SSD: 120GB ×5 (600GB/node )Network: 4X QDR InfiniBand

Edge Switch #1

Edge Switch #2

Edge Switch #3

Edge Switch ×2

Edge Switch #4

Inter-Node Connection Network

DIMM 1D

DIMM 2A

DIMM 3E

DIMM 4B

DIMM 5F

DIMM 6C

Block Diagram of Thin Node

DDR3Intel Xeon
X5670 ×2

QPI

2
1
1 CPU0
0
0

0
1

ESI

1
2
1 CPU1
0
0

1,2

×1

Mellanox ConnectX× 16 Gen2

× 16
×24 PCIE Riser Conn

× 8 Gen2
× 4 Gen2

Intelkawela
82576

QPI-Riser

DIMM 1D

DIMM 2A

DIMM 3E

DIMM 4B

DIMM 6C

DIMM 5F

7,8,9,10
Tylersburg5,636D

×4

QDR InfiniBand
HCA

× 8 Gen2

QPI

0
1

ESI

10
9
8
Tylersburg- 7
36D
3
4
5
6

×8

NIC1
NIC2

× 16

InfiniBand QDR Network for LNET and Other Services

QSFP
SFP+
PCI-E SLOTPCI-E SLOT

GPU

RJ45

QDR InfiniBand
HCA

RJ45

Mellanox ConnectX

PCI-E SLOTGPUInternal
Micro SD

× 16

DVI
P6

PCI

SPI
LVAD

RN50/ES1000

PCI-E SLOT

GPU

SMX

SMX

SMX

OSSs
MDSs
FC

SMX

SMX

SMX

L2 Cache
SMX

SMX

SMX

SMX

Peak performance : 1.31 TFLOPS (double precision)
3.95 TFLOPS (single precision)
Shader clock : 732 MHz
Number of CUDA cores (SP) : 2,688 cores
Streaming Multiprocessor (SMX) : 14 SMX
Writable L2 cache: 1.5MB
Memory bandwidth : 250GB / sec
Memory clock : 2.6GHz (GDDR5)
ECC support:internal and external memory
On-board memory: 6GB

SMX

HOME

SFA12k #1

SFA10k #1

SFA10k #2

SFA10k #3

SFA10k #4

SFA10k #5/hpci0/data0/data1/work0/work1/gscr0

Systemapplication

SCSI“Global WorkSpace” #2“Global WorkSpace” #3“Scratch”“cNFS/Clusterd Sumba w/
GPFS”“NFS/CIFS/iSCSI
by BlueARC”

HPCI“Global Work Space” #1

SFA10k #6

GPFS with HSM

LustreHome

SROM

Gfarm: 〜 600TB

GPFS 2.4 PB

Lustre 3.6 PB

Home 1.2 PB

NVRAM

gsfd server:
HP ProLiant DL360p Gen8 × 2
Intel XeonE5 2640 × 2,
64 GB Mem,QDR IB (4x) × 2

NSD server: HP ProLiant DL380 G6 × 4
Intel Westmere EP × 2,
48GB Mem, QDR IB (4x) × 2

MDS :HP ProLiant DL360 G6 × 4
Intel Westmere-EP × 2,
48GB Mem, QDR IB（4x ）× 2

HP ProLiant DL360 G6 × 4
Intel Westmere EP × 2,
24GB Mem,QDR IB (4x) × 2

OSS :HP ProLiant DL360 G6 × 16
Intel Westmere-EP × 2,
24GB Mem,QDR IB（4x ）× 2

cNFS（GridScaler）/Clustered Sumba w/ GPFS:
HP ProLiant DL380 G6 × 4
Intel Westmere EP × 2,
48GB Mem, QDR IB（4x ）× 2

DDN SFA10k × 2
2TB SATA HDD × 1180 disks
600GB SAS HDD × 20 disks

Storage: DDN SFA 10k × 3,
2TB SATA HDD × 1770 disks,
600GB SAS HDD × 30 disks

Video

Storage:
DDN SFA12k × 1
4TB SAS HDD × 155 disks

Register File (65,536 × 32bit)

SMX

GPFS#1 GPFS#2 GPFS#3 GPFS#4

MDSs

gfsd servers

Storage:

NFS/CIFS/iSCSI:BlueArc Mercury 100 × 2
10Gbps × 2
Storage: DDN SFA 10k × 1
2TB SATA HDD × 600 disks

MemoryController

MemoryController

SMX

10GbE × 2

GROM

Details of SM (Streaming Multiprocessor)

PCI Express Host Interface

QDR IBI （4x ）× 16

RJ45

Details of GPU
K20X Architecture (Kepler GK110 Core)I （4x ）× 20
QDR IB

OSSs

PCI-E Riser

Serial（RJ45）USB

QDR IIB（4x ）× 20

HOME

10/100PHY

GXE

2 × USBICH10SMX

QDR IBI （4x ）× 16“Gfarm for HPCI”

6 × SATASMXQDR IBI （4x ）×4

NSD servers

SSD
SSD

Core Switch
Voltaire GridDirector 4700

Core switch : Voltaire Grid Director 4700×12
Edge switch : Voltaire Grid Director 4036×179, 4036e ×6

64KB Shared Memory / L1 Cache
48KB Read-Only Data Cache
CUDA core (SP) / SMX : 192 cores
DP unit / SMX : 64 SFU / SM : 32 units
Warp scheduler / SMX : 4 units
Shared memory / SMX : 16KB or 32KB or 48KB
Writable L1 cache / SMX : 48KB or 32KB or 16KB
Read-only data cache / SMX: 48KBCoreDP Unit
LDST
SFU

High-Speed and Highly Reliable Storage Systems
TSUBAME2.5 provides 11PB of massive storage volumes to serve various purposes, including about 190TB of SSDs embedded
in compute nodes for scratch I/O, 5.9PB of parallel file systems such as Lustre and GPFS for high speed parallel I/O,
1.2 PB of home storage volumes for providing campus cloud storage services,and over 4PB of tape libraries for hierarchical storage management handled with GPFS.

Large -Scale GPU - Equipped High-Performance Compute Nodes

High-Speed Network Interconnect

Compute nodes consist of three types of nodes: Thin, Medium and Fat nodes. Thin nodes, which provide most of the overall
compute performance, are equipped with two CPUs and three Kepler core GPUs in a compact design 17/2 inches in width and 2U sizein height. In addition, two QDR InfiniBand HCAs are connected to dedicated PCI Express buses to secure the communication bandwidth.
Power supply units are organized with 3+1 redundancy, improving the node reliability significantly.

Compute nodes of TSUBAME2.5 interconnected with Dual-Rail QDR InfiniBand networks of Fat-Tree type full bi-section
bandwidth achieve 200Tbps. End-to-End latency between the compute nodes is extremely low in microsecond-order time,
therefore resulting in high-speed performance and high-speed connection to highly reliable storages.
This network is linked by more than 3000 optical fiber cables in a total length of 100km.

Thin Node 1408 nodes

Medium Node 24 nodes

Fat Node 10 nodes

Thin Node × 1408 (MCS racks : 1260 +others :148)

Medium Node × 24

Fat Node × 10

Med Node

Fat Node

1 MCS Rack (Thin Node × 30)
15 nodes15 nodesodes

×42
Thin N
Node
Th
dThin NodeThi

HP ProLiant SL390s G7
CPU:Intel Xeon X5670 (Westmere-EP, 2.93GHz) ×2 sockets
six-core per socket, total 12 cores per node.

GPU:

NVIDIA Tesla K20X (GK110) ×3. 1.31TFLOPS and VRAM 6GB per GPU

Memory: 58GB DDR3 1333MHz, partly 103GB
SSD:

120GB(60GB×2), partly 240GB(120GB×2)

Network: 4X QDR InfiniBand ×2

HP ProLiant DL580 G7

HP ProLiant DL580 G7

CPU: Intel Xeon X7550(Nehalem-EX )2.0 GHz ×4 sockets(32cores/node)
GPU: NVIDIA Tesla S1070(NVIDIA Tesla C1060×4) or
NextIO vCORE Express 2070
(NVIDIA Tesla M2070×4)
Memory: 137 GB (DDR3 1066MHz )SSD: 120GB ×4 (480GB/node )Network: 4X QDR InfiniBand

CPU: Intel Xeon X7550(Nehalem-EX )2.0 GHz ×4 sockets(32cores/node )GPU: NVIDIA Tesla S1070(NVIDIA Tesla C1060×4)
Memory: 274 GB (8 nodes ) ,548 GB (2 nodes )DDR3 1066MHz
SSD: 120GB ×5 (600GB/node )Network: 4X QDR InfiniBand

Edge Switch #1

Edge Switch #2

Edge Switch #3

Edge Switch ×2

Edge Switch #4

Inter-Node Connection Network

DIMM 1D

DIMM 2A

DIMM 3E

DIMM 4B

DIMM 5F

DIMM 6C

Block Diagram of Thin Node

DDR3Intel Xeon
X5670 ×2

QPI

2
1
1 CPU0
0
0

0
1

ESI

1
2
1 CPU1
0
0

1,2

×1

Mellanox ConnectX× 16 Gen2

× 16
×24 PCIE Riser Conn

× 8 Gen2
× 4 Gen2

Intelkawela
82576

QPI-Riser

DIMM 1D

DIMM 2A

DIMM 3E

DIMM 4B

DIMM 6C

DIMM 5F

7,8,9,10
Tylersburg5,636D

×4

QDR InfiniBand
HCA

× 8 Gen2

QPI

0
1

ESI

10
9
8
Tylersburg- 7
36D
3
4
5
6

×8

NIC1
NIC2

× 16

InfiniBand QDR Network for LNET and Other Services

QSFP
SFP+
PCI-E SLOTPCI-E SLOT

GPU

RJ45

QDR InfiniBand
HCA

RJ45

Mellanox ConnectX

PCI-E SLOTGPUInternal
Micro SD

× 16

DVI
P6

PCI

SPI
LVAD

RN50/ES1000

PCI-E SLOT

GPU

SMX

SMX

SMX

OSSs
MDSs
FC

SMX

SMX

SMX

L2 Cache
SMX

SMX

SMX

SMX

Peak performance : 1.31 TFLOPS (double precision)
3.95 TFLOPS (single precision)
Shader clock : 732 MHz
Number of CUDA cores (SP) : 2,688 cores
Streaming Multiprocessor (SMX) : 14 SMX
Writable L2 cache: 1.5MB
Memory bandwidth : 250GB / sec
Memory clock : 2.6GHz (GDDR5)
ECC support:internal and external memory
On-board memory: 6GB

SMX

HOME

SFA12k #1

SFA10k #1

SFA10k #2

SFA10k #3

SFA10k #4

SFA10k #5/hpci0/data0/data1/work0/work1/gscr0

Systemapplication

SCSI“Global WorkSpace” #2“Global WorkSpace” #3“Scratch”“cNFS/Clusterd Sumba w/
GPFS”“NFS/CIFS/iSCSI
by BlueARC”

HPCI“Global Work Space” #1

SFA10k #6

GPFS with HSM

LustreHome

SROM

Gfarm: 〜 600TB

GPFS 2.4 PB

Lustre 3.6 PB

Home 1.2 PB

NVRAM

gsfd server:
HP ProLiant DL360p Gen8 × 2
Intel XeonE5 2640 × 2,
64 GB Mem,QDR IB (4x) × 2

NSD server: HP ProLiant DL380 G6 × 4
Intel Westmere EP × 2,
48GB Mem, QDR IB (4x) × 2

MDS :HP ProLiant DL360 G6 × 4
Intel Westmere-EP × 2,
48GB Mem, QDR IB（4x ）× 2

HP ProLiant DL360 G6 × 4
Intel Westmere EP × 2,
24GB Mem,QDR IB (4x) × 2

OSS :HP ProLiant DL360 G6 × 16
Intel Westmere-EP × 2,
24GB Mem,QDR IB（4x ）× 2

cNFS（GridScaler）/Clustered Sumba w/ GPFS:
HP ProLiant DL380 G6 × 4
Intel Westmere EP × 2,
48GB Mem, QDR IB（4x ）× 2

DDN SFA10k × 2
2TB SATA HDD × 1180 disks
600GB SAS HDD × 20 disks

Storage: DDN SFA 10k × 3,
2TB SATA HDD × 1770 disks,
600GB SAS HDD × 30 disks

Video

Storage:
DDN SFA12k × 1
4TB SAS HDD × 155 disks

Register File (65,536 × 32bit)

SMX

GPFS#1 GPFS#2 GPFS#3 GPFS#4

MDSs

gfsd servers

Storage:

NFS/CIFS/iSCSI:BlueArc Mercury 100 × 2
10Gbps × 2
Storage: DDN SFA 10k × 1
2TB SATA HDD × 600 disks

MemoryController

MemoryController

SMX

10GbE × 2

GROM

Details of SM (Streaming Multiprocessor)

PCI Express Host Interface

QDR IBI （4x ）× 16

RJ45

Details of GPU
K20X Architecture (Kepler GK110 Core)I （4x ）× 20
QDR IB

OSSs

PCI-E Riser

Serial（RJ45）USB

QDR IIB（4x ）× 20

HOME

10/100PHY

GXE

2 × USBICH10SMX

QDR IBI （4x ）× 16“Gfarm for HPCI”

6 × SATASMXQDR IBI （4x ）×4

NSD servers

SSD
SSD

Core Switch
Voltaire GridDirector 4700

Core switch : Voltaire Grid Director 4700×12
Edge switch : Voltaire Grid Director 4036×179, 4036e ×6

64KB Shared Memory / L1 Cache
48KB Read-Only Data Cache
CUDA core (SP) / SMX : 192 cores
DP unit / SMX : 64 SFU / SM : 32 units
Warp scheduler / SMX : 4 units
Shared memory / SMX : 16KB or 32KB or 48KB
Writable L1 cache / SMX : 48KB or 32KB or 16KB
Read-only data cache / SMX: 48KBCoreDP Unit
LDST
SFU

High-Speed and Highly Reliable Storage Systems
TSUBAME2.5 provides 11PB of massive storage volumes to serve various purposes, including about 190TB of SSDs embedded
in compute nodes for scratch I/O, 5.9PB of parallel file systems such as Lustre and GPFS for high speed parallel I/O,
1.2 PB of home storage volumes for providing campus cloud storage services,and over 4PB of tape libraries for hierarchical storage management handled with GPFS.

Low Power Consumption andGreen Operation

Power performance in Linpack benchmark : 3068.71 (MFLOPS/W)
Peak power consumption of system equipment : 1620 (KW)
Average power consumption of system equipment* : 698 (KW)
Idle power consumption for system equipment : 470 (KW)
Yearly average PUE : 1.285

Global Scientific Information and Computing Center, Tokyo Institute of Technology[*] Average power consumption is an annual average of TSUBAME2.0's.

Cooling : Modular Cooling System
Front

Green Operation : Monitoring of Environment
Temperature, power consumption, etc., are observed in real-time
not only in the computer room but also to compute nodes and toeach rack.

Rear

Collection of sensor information
Fan
modules

Water
group
AC transfer

Heatexchanger
modules

Water
group
controller

Fan
modules

Environmental monitoring system

Power andnetwork
connectors

Environmental
Monitoring DB

Distribution
board

Distribution
board

Storage rack
Conversion
processing

Network rack

Water inlet

operation

Water outlet

The rack-contained water-cooling system with a built-in heatexchanger is employed, allowing high-density cooling up to 35kW
per rack, which is the top class in the world. Homogeneous cooling
air is provided through the inlet of the server with automatic
open/close doors where a humidifier is unnecessary. Powerconsumption is minimized with a completely automatedtemperature control to enable heat removal from 95% to 97% bywater cooling. Moreover, polycarbonate doors contribute to a great
noise reduction.

ComputerAdministration
consoleadministrator

Distributionboard

Air conditioner
Distribution
board

Small space installation

2F

2F

1F

1F

Peak power consumption of
air-conditioning equipment : 460 (KW)
Average power consumption of
air-conditioning equipment* : 204 (KW)

Despite the fact the performance boost is more than 70 times
compared to TSUBAME1.2, the space required for installation has
narrowed down.

HARDWARE AND SOFTWARE SPECIFICATIONS■ System Software "Dynamic provisioning" dynamically switched between Windows and LinuxThe job management system and the cluster management system areworking together to manage user environment as well as distributing
computational resources to the insufficient part by taking from thenode pool. Both batch schedulers for Linux and Windows manage todynamically increase or reduce the compute nodes. The job scheduling also manages to support the execution of a virtual machine.

Large-Scale GPU-Equipped High-Performance Compute Nodes
OS

High-Speed Network Interconnect
High-Speed and Highly Reliable Storage Systems

Batch System

PBS Professional

Low Power Consumption and Green Operation
System and Application Software■ ISV (commercial ) Software( * GPU full or partial support )(As of November, 2013)

Compilers, Debuggers and Libraries

Applications

Intel Compiler ( C/C++/Fortran )PGI Compiler*( C/C++/Fortran, OpenACC, CUDA Fortran )Total View Debugger*
CAPS Compiler* ( HMPP, OpenACC )CULA* ( Numerical Libraries for CUDA )ANSYS Fluent*, Workbench*
MSC Nastran*
LS-DYNA
Gaussian, Gauss View
MolproScigress
MATLAB*
AVS/Express, AVS/Express PCE■：The license for all users

SUSE Linux Enterprise Server 11 SP1Windows HPC Server 2008 R2■：The license for Tokyo Tech users

ABAQUS*, ABAQUS CAEPatranCST STUDIO SUITE* (MW-Studio*)
AMBER*
Materials Studio, Discovery Studio
Mathematica*
Maple*
EnSight
System ( 58 Racks )■：The license for industrial users

Published by Global Scientific Information and Computing Center, Tokyo Institute of Technology

Rack ( 30 nodes )2-12-1 Ookayama, Meguro-ku, Tokyo 152-8550, JAPAN TEL : +81-3-5734-2087 FAX : +81-3-5734-3198 E-mail : tsubame@gsic.titech.ac.jp

http://www.gsic.titech.ac.jp/
2 01311

Compute NodeN
( 2 CPUs)
CPU , 3GPUs
3
4.08 TFLOPS.
58.0 GB (CPU ) + 18 GB (GPU )122 TFLOPS
2.28 TB

1442nodes
2952 CPU sockets :224.7 TFLOPS
Turbo boost
4360GPUs:
5.562 PFLOPS
Total:
5.787 PFLOPS
Memory:116 TB