New Supercomputer System SX-ACE at the Cybermedia Center
Cybermedia Center, Osaka University, Japan

New Supercomputer System SX-ACE at the Cybermedia Center

System Overview
The SX-ACE, newly introduced by the Cybermedia Center (CMC) isa “clusterized” vector-typed supercomputer, composed of 3 clusters,
each of which is composed of 512 nodes. Each node is equippedwith 4-core multi-core CPU and a 64 GB main memory. These 512
nodes are interconnected on a dedicated and specialized network
switch, called IXS (Internode Crossbar Switch) and forms a cluster.
Note that IXS interconnects 512 nodes with a single lane of 2-layer
fat-tree structure and as a result exhibits 4 GB/s for each direction ofinput and output between nodes. In the CMC, 2 Peta-byte storageis managed on NEC Scalable Technology File System (ScateFS), NECdeveloped fast distributed and parallel file system, so that it can beaccessed from the large-scale computing systems including theSX-ACE at the CMC

Node Performance
As a single SX-ACE node has a multi-core vector-typed processor composedof 4 cores, each of which exhibits 64 GFlops vector performance, and a 64
GB main memory, the vector performance per node becomes 256 GFlops.
On the other hand, the maximum transfer between processor and main
memory is performed with 256 GB/s. This fact indicates that a single SX-ACE
node achieves high memory-bandwidth performance of 1 Byte/Flops, taking
higher CPU performance into consideration.
Internode communication enables 4GB/s x 2 (bi-directional) high-bandwidth
data communication with a specialized internode communication control
unit named RCU connected to IXS. Also, communication between node anddisk storage enables 10 Gbps data communication with I/O control unitcalled IOP.

IXS

Disk
4 GB/s x2（bidirectional）

10 GbpsIOP

IOPCPU
core

RCU
IOP

RCU

RCU

IOP
RCUCPUCPUCPUCPU
core CPU
core CPU
core CPU
IOP
core CPU
core CPU
core CPU
core CPUCPU
core
core CPU
core CPU
core CPU
256
GB/scorecore
core
256 GB/s core
CPUCPU
256 GB/s CPU
core
core
core
256 GB/s
X-bar switch
X-bar switchi
X-bar switchi
X-bar switchi

RCU
CPU
core

256 GB/s

X-bar switchi
Main memory
Main memory
Main memory
Main memory
Main memory

System Performance
SX-ACE
Per-node

1 cluster

3 cluster# of CPU

15121536# of core

42048

6144

Performance

276 GFLOPS

141 TFLOPS

423 TFLOPS

Vector performance

256 GFLOPS

131 TFLOPS

393 TFLOPSMain memory

64 GB

32 TB

96 TB

Storage

2 PB

The SX-ACE which the CMC has introduced is composed of 3 clusters (1536 nodes). Theoretical peak performance of the SX-ACE at
the CMC is derived as the left table indicates.
Importantly, note that performance is the sum of vector-typed processor and scalar processor on SX-ACE. SX-ACE has a 4-core
multi-core vector-typed processor and a single scalar processor.

IT Core AnnexIT Core Annex is a new datacenter that aims to aggregate andaccommodate computer systems and supercomputer systems on
campus for energy-efficient administration and management. It wasdesigned and built based on the careful consideration on air flow
and circulation for efficient cooling. The SX-ACE at the CMC is set
up at the IT Core Annex.
A remarkable feature of IT Core Annex has two indirect evaporative
air cooling systems to improve PUE for more energy-efficient drivingof the newly introduced supercomputer system SX-ACE at the CMC.

Contact : system@cmc.osaka-u.ac.jp
http://www.hpc.cmc.osaka-u.ac.jp/