TSUBAME 2.5

Architecture & Services
Hardware
Storage
Parallel file system area: 5.93 PB

Users’ home space: 1.2 PB

Archive Storage: 4 PB

Server- HP DL380 G6 x4(NFS, CIFS)- BlueArc Mercury 100 x2(NFS, CIFS, iSCSI)Storage- DDN SFA 10000 x1(10 enclosures x1)

Server- HP DL380 G6 x30
Storage- DDN SFA 10000 x5(10 enclosure x5)
Luster (5 File Systems)- OSS: 20, OST: 6.9PB- MDS: 10, MDT: 30TB

Server- StorageTek
SL8500 x2

Interconnect: Full bisection / Non-blocking
Core Switch

Edge Switch

Voltaire Grid Director 4036
- 179 switches- InfiniBand QDR 36 ports

HPCI Global Shared Storage:
600 TB
Server- HP DL360p G8 x2(Gfarm File System)
Storage- DDN SFA12K x1
Gfarm File System- Storage Service x2

HPCI Login NodeHP ProLiant SL390s G7

Edge Switch (w/10GbE)

GSI, Shibboleth, Gfarm, ...

Voltaire Grid Director 4700
- 12 switches- InfiniBand QDR 324 ports

GSI-SSH, Gfarm Client

Voltaire Grid Director 4036E- 6 switches- InfiniBand QDR 32 ports- 10 GbE 2 ports

Dual Rail Connection

Thin compute node HP ProLiant SL390s G71,408 nodes

CPU:Intel Xeon X5670 x2
2.93 / 3.2 GHz,
6 cores, 76.8 GFLOPS
GPU: NVIDIA Tesla K20X x3
1.31 TFLOPS, 6 GB VRAM
Memory: 58 GB or 103 GB
SSD:
120 GB, 220 GB or 240 GB
Read 230MB/sec, Write 180MB/sec

Medium / Fat compute node
HP ProLiant DL580 G7
Medium : 24 nodes
Fat :10 nodesCPU:Intel Xeon X7550 x4
2.0 / 2.4 GHz
8 cores, 76.8 GFLOPS
GPU: NVIDIA Tesla S1070 / M2070 x4
Memory: 137 GB (M), 274 / 548 GB (F)SSD:480GB (M), 600 GB (F)Read 230MB/sec, Write 180MB/sec100 Gbps

SINET5Compute nodes: 5.76 PFLOPS (CPU + GPU) / 225 TFLOPS (CPU)Software
System Software- OS:
SUSE Linux Enterprise Server 11 SP3,- Compiler:Intel Compiler, PGI CDK,
GNU Compiler Collection- MPI:
OpenMPI, MVAPICH2- Job Scheduler: PBS Professional
Software for GPU Computing- Computing Architecture: CUDA 7.5- Automatic Parallelization: PGI Accelerator Compiler- Linear Algebra Library: CULA

HPCI Confederation
HPCI : High Performance Computing Infrastructure- National grid infrastructure for HPC researchResources- 11 supercomputers in Japan, including TSUBAME 2.5- 100PB global shared storage to share dataServices- One-stop sign up to all resources- Single sign on to all resources using Shibboleth & GSI
Status- 28 projects and 198 users use TSUBAME 2.5 for the HPCI projecton FY2016
For more details, Please go to booth #737
“Research Organization for Information Science & Technology (RIST)”

Validated Applications

ABAQUS, AMBER, ANSYS Fluent / Mechanical, AVS Express,
CST STUDIO SUITE, Discovery Studio, GAMESS, Gaussian,GROMACS, LS-DYNA, Maple, Materials Studio, Mathematica,MATLAB, MSC Nastran / Marc, POV-Ray

Optional Applications

Desmond MD, LAMMPS, myPresto/psygene-G, NAMD, NWChem,PHASE, Quantum Espresso, Remcom XFdtd, STAR-CCM+, VASP,Caffe, Theano, Chainer, TensorFlow

Debug & Performance Analysis Tools
PAPI, Scalasca, Score-P, TotalView, Vampir(Red: GPU Ready, Blue: GPU Experimental Support)

Hokkaido University

GSI-SSH
Tohoku University

Single Sign On
GSI-SSHWeb Access(Shibboleth)

Kyushu University

HPCI Portal(HPCI Office)File Access

10PB
Storage

Global Shared Storage
Gfarm File System

13.5PB
Storage

400TB
Storage
Kyoto University

RIKEN, AICSHPCI CA(NII)

Osaka University

University of Tsukuba
The University of Tokyo
Shared Storage / East Hub
The University of Tokyo (Kashiwa)

ISM
Tokyo Institute of Technology
JAMSTEC

Nagoya University

Shared Storage / Tokyo Tech Hub
Tokyo Institute of Technology (Tokyo)

Shared Storage / West Hub
RIKEN, AICS (Kobe)

http://www.gsic.titech.ac.jp/sc16