Osaka University, Japan
SC22 BOOTH 1613

Large-scale Computing Systems at the Cybermedia Center
Overview of High-Performance Computing Environment at the CMC
USER

Supercomputer
NetworkLogin & Front End
NodeInternet

Cloud

Large-scale computing systems ( OCTOPUS and SQUID ) and data aggregate
infrastructure ( ONION ) are deployed on CMC-Supercomputer network, a.k.a
CMC-SCinet, a low-latency and wide-bandwidth network. This architectural design
allows users to access to large-scale storage systems, perform large-scale
high-performance computation and analysis on our large-scale computingsystems.

Large-scale Computing System

OCTOPUSSQUID

OCTOPUS is short for Osaka university Cybermedia cenTer
Over-Petascale Universal Supercomputer. OCTOPUS is a cluster
system being operated since December 2017. This system iscomposed of General purpose CPU nodes, GPU nodes, Large-scale
shared-memory nodes, and Xeon Phi nodes, total 319 nodes.
These nodes and large-scale storage EXAScaler ( Lustre 3.1PB ) are interconnected on InfiniBand EDR ( 100 Gbps ) and forma cluster.

SQUID is short for Supercomputer for Quest to UnsolvedInterdisciplinary Datascience. SQUID is a new cluster systembeing operated since May 2021. This system is composed ofGeneral purpose CPU nodes, GPU nodes, and Vector nodes,
total 1,598 nodes. These nodes and large-scale storage EXAScaler ( Lustre 21.2 PB ) are interconnected on InfiniBand HDR( 200 Gbps ) and form a cluster.
Table 2 Data Sheet of SQUID

Table 1 Data Sheet of OCTOPUS
GPULarge–scale
shared-memory

Xeon PhiIntel Xeon Skylake( 2.6 GHz, 12 cores ) x 2

Intel Xeon Skylake( 2.0 GHz, 16 cores ) x 8Intel Xeon Phi KNL( 1.3 GHz, 64 cores )General purpose
CPUType of nodesCPU

Type of nodes
CPU

RHEL 7.3

OSGeneral purpose
CPUGPUVector

Intel Xeon Icelake ( 2.4 GHz, 38 cores ) x 2

AMD EPYC Rome
( 2.8 GHz, 24 cores )CentOS 8.4

OS# of nodes (total)

236

37244

# of nodes (total)

1,520

4236

# of cores (total)

5,664

888

256

2,816

# of cores (total)

115,520

3,192

864

# of memory (total)

45

7128

# of memory (total)

389 TB

22 TB

5 TB

Peak performance

471.2 TFLOPS

16.4 TFLOPS

858.3 TFLOPS

117.1 TFLOPS

Peak performance

8.871 PFLOPS

6.797 PFLOPS

0.922 PFLOPSNVIDIA
A100 x 336

NEC SX-Aurora TSUBASA
Type20A x 288

NVIDIA Tesla
P100 x 148

Accelerator

Accelerator

Data Aggregation Infrastructure

ONION
Primary storage

Primary storage

Externalstorage

Cloud

S3
S3

S3SFTP
S3DAV

S3Cameras /Sensors

Cameras /Sensors

S3Lustre

EXAScaler

objectS3

For archives

S3SFTPS3

For high-speed
analysis

SCP/SFTPSQUID users
HTTPS
For data manipulation

HTTPS

HTTPS

WWWPC

Browser

Mobilefile

ONION stands for Osaka university Next-generation Infrastructure for Open researchand open innovatioN. ONION is a new data aggregation infrastructure that is linked toSQUID. ONION consists of ONION-object ( AWS S3 compatible object storage ) ,ONION-file ( storage service using Nextcloud ) , and EXAScaler ( a parallel file system
based on Lustre ) .
ONION makes it easy for users to data between your PC and large-scale computingsystem. In addition, ONION can be used in a variety of ways, such as immediate sharingof calculation results with those who do not have a SQUID or OCTOPUS account andmanipulating data from a smartphone. Of course, it can also be used to store and shareresearch data in the laboratory.
Table 3 EXAScaler ( on SQUID )Table 4 ONION-object

Effective capacity (HDD)

20 PB

Effective capacity (NVMe)

1.2 PBMax number of inodes

Approx. 8.8 Billion

Max expected effective throughput (HDD)

Over 160 GB/s

Max expected effective throughput (NVMe)

Write : Over 160 GB/s Read : Over 180 GB/s

Contact : sc22@ais.cmc.osaka-u.ac.jp

Effective capacity

Data protection method

950 TiB* We plan toexpanse sequentially
Erasure Coding(Data chunk:4 +Parity chunk:2)

http://www.hpc.cmc.osaka-u.ac.jp/en/