Inter-Cloud Infrastructure

Cloud Infrastructure for Big Data Analysis
Inter-Cloud Infrastructure for Big Data Analysis
Building a Testbed Infrastructure on Overlay Cloud
Using SINET5 network infrastructure (100Gbps Network)
Cooperation with Colud and Supercomputer
Providing the Science Data Repository
Testbed: Peta Bytes class object storage
Real System: Exa Bytes class object storage
Cloud and Supercomputer Federaton
Development of collaboration technology between cloud andsupercomputer using TSUBAME 3.0 as a test caseAcademic Cloud@ Hokkaido Univ.
etc.
Public Cloud Resource@ SINET Inter-Cloud

Exabyte DB on
Global Object Storage

High Bandwidth& Secure Network
by SINET5

Tight connection of Supercomputerand Inter-Cloud resources via SINET L2VPN

Apply cloud container technology
to Supercomputer for resourcecooperation with the Inter-Cloud

Cloud Resource@ NII

Cloud Platform@ Kyushu Univ.

DockerConta iner

TSUBAME3.0@Tokyo Tech.

@National Institute of Genetics
TSUBAME3 Bigdata Storage

HuronFS: New HPC Storage Hierachy for HPC & Cloud
Computenode X

Parallel File System/ Shared Cloud Storage
Consistent hash

Master 0IOnode

Master Y

IOnodeMaster M-1

IOnode

IOnode

IOnode

IOnode

IOnode

IOnode

IOnode

IOnode

IOnode

IOnodeIOnode

IOnode

IOnodeSHFS 0

SHFS YSHFS M-1

HuronFS

Multiple Client Sequential I/O

Metadata Performance

3500
write

3000

read2500
2000

Ops/s

Throughput (MiB/s)

Background
HPC
HPC users can cause cngestion on PFS
Not all the HPC systems have fast PFS
Cloud
Cloud storage cannot provide enough I/O throughput for data intensive applications.
Loose consistency model in cloud storage
SystemHuronFS (Hierarchical, UseR-level and ON-demand File system)
as a new tier in torage hierarchy.
Several dedicated nodes/instances as burst buffers to accelerateaccesses by buffering intermediate data.
Implemented on CCI : Utilizing the high performance network in
HPC & High portability
Implemented with FUSE : Supporting POSIX & No code modification required

1500
1000
500
0
1248Nodes (8 processes per node)

18000
16000
14000
12000
10000
8000
6000
4000
2000
016

124Nodes816

Code available at:https://github.com/EBD-CREST/HuronFS

Evaluation of HPC-Big Data Applications
Execution time [sec]0.2024816# of nodes(2 MPI ranks, 12 OpenMP threads per node)Acknowledgments. These researches are supported by CREST, JST CRESTGrant Numbers JPMJCR1303 and JPMJCR 1501.
(Research Area: Advanced Core Technologies for Big Data Integration).

Execution time [sec]Execution time [sec]Execution time [sec]

NICAM-DC-MINI: A compute-intensive miniapp from Fiber miniapp suite
Backgroud
TSUBAME-KFC
AWS EC2 c4.8xlarge instances
Cloud platforms exhibit elasticity, flexibility, usability and scalability, which
Computation
Communication
Stall
Computation
Communication
Stallhave been attracting users to use these environments as a cost effective
90,00090,00080,00080,00070,00070,000measure to run their applications or businesses. However, the feasibility of60,000
60,00050,00050,00040,00040,000running high performance computing applications on clouds has always
30,00030,00020,00020,000been a concern mainly due to virtualization overheads and high-latency in10,000
10,00000
1416
1416
terconnection network.
# of nodes (10 MPI ranks per node)# of instance (10 MPI ranks per instance)
Goals
To investigate the potential role of these virtual machines in addressing Graph500 Benchmark at Scale 26TSUBAME-KFC
AWS EC2 c4.8xlarge instances
the needs of HPC and data-intensive workloads
App Time
MPI Time
App Time
MPI Time
Experimentation
1.6
1.6
1.4
1.4
1.2
1.2
Performance evaluation of applications on AWS C4 instances against the110.8
0.8
0.6
0.6
baseline results of a supercomputer, TSUABME-KFC
0.4
0.4
0.2
02

4
816
# of instance(8 MPI ranks, 12 OpenMP threads per instance)* 2 MPI ranks and 12 openmp threads per node achieved the best performance on TSUBAME-KFC. In case of AWS EC2, itwas 8 MPI ranks and 12 openmp threads per instance.

TSUBAME-KFC: Intel Xeon E5-2620v2 x2, InfiniBand FDR
Amazon EC2 c4.8xlarge instance: Intel Xeon E5-2666v3, 10Gbps Ethernet

http://www.gsic.titech.ac.jp/sc17