Inter-Cloud Infrastructure

Cloud Infrastructure for Big Data Analysis
Inter-Cloud Infrastructure for Big Data Analysis
Building a Testbed Infrastructure on Overlay Cloud
Using SINET5 network infrastructure (100Gbps Network)
Cooperation with Colud and Supercomputer
Providing the Science Data Repository
Testbed: Peta Bytes class object storage
Real System: Exa Bytes class object storage
Cloud and Supercomputer Federaton
Development of collaboration technology between cloud andsupercomputer using TSUBAME 3.0 as a test caseAcademic Cloud@ Hokkaido Univ.
etc.
Public Cloud Resource@ SINET Inter-Cloud

Exabyte DB on
Global Object Storage

High Bandwidth& Secure Network
by SINET5

Tight connection of Supercomputerand Inter-Cloud resources via SINET L2VPN

Apply cloud container technology
to Supercomputer for resourcecooperation with the Inter-Cloud

Cloud Resource@ NII

Cloud Platform@ Kyushu Univ.

DockerConta iner

TSUBAME3.0@Tokyo Tech.

@National Institute of Genetics
TSUBAME3 Bigdata Storage

HuronFS: New HPC Storage Hierachy for HPC & Cloud
Computenode X

Parallel File System/ Shared Cloud Storage
Consistent hash

Master 0IOnode

Master Y

IOnodeMaster M-1

IOnode

IOnode

IOnode

IOnode

IOnode

IOnode

IOnode

IOnode

IOnode

IOnodeIOnode

IOnode

IOnodeSHFS 0

SHFS YSHFS M-1

HuronFS

Multiple Client Sequential I/O Performance
TSUBAME2

AMAZON Web Service

3500
write

14000

readThroughput (MiB/sec)

3000
Throughput (MiB/s)

Background
HPC
HPC users can cause congestion on PFS
Not all the HPC systems have fast PFS
Cloud
Cloud storage cannot provide enough I/O throughput for data intensive applications.
Loose consistency model in cloud storage
SystemHuronFS (Hierarchical, UseR-level and ON-demand File system)
as a new tier in torage hierarchy.
Several dedicated nodes/instances as burst buffers to accelerateaccesses by buffering intermediate data.
Implemented on CCI : Utilizing the high performance network in
HPC & High portability
Implemented with FUSE : Supporting POSIX & No code modification required

2500
2000
1500
1000
500

12000
10000
8000

CloudBB (read)
CloudBB (write)
Amazon S3 (read)Amazon S3 (write)

6000
4000
2000
00
1

248Nodes (8 processes per node)

1
2
48
1632
64
The number of compute nodes ( = The number of IONs)

16

System: Amazon EC2 Tokyo Region, Instance Type: m3.xlarge, Cloud Storage: Amazon S3,Mount Method: s3f, Chunk Size: 5MB, Client Local Buﬀer Size: 100MB

Code available at:https://github.com/EBD-CREST/HuronFS

Evaluation of HPC-Big Data Applications
Execution time [sec]0.2024816# of nodes(2 MPI ranks, 12 OpenMP threads per node)Acknowledgments. These researches are supported by CREST, JST CREST
Grant Numbers JPMJCR1303 and JPMJCR 1501.
(Research Area: Advanced Core Technologies for Big Data Integration).

Execution time [sec]Execution time [sec]Execution time [sec]

NICAM-DC-MINI: A compute-intensive miniapp from Fiber miniapp suite
Backgroud
TSUBAME-KFC
AWS EC2 c4.8xlarge instances
Cloud platforms exhibit elasticity, flexibility, usability and scalability, which
Computation
Communication
Stall
Computation
Communication
Stallhave been attracting users to use these environments as a cost effective
90,00090,00080,00080,00070,00070,000measure to run their applications or businesses. However, the feasibility of
60,00060,00050,00050,00040,00040,000running high performance computing applications on clouds has always
30,00030,00020,00020,000been a concern mainly due to virtualization overheads and high-latency in10,000
10,000001
4161416terconnection network.
# of nodes (10 MPI ranks per node)# of instance (10 MPI ranks per instance)
Goals
To investigate the potential role of these virtual machines in addressing Graph500 Benchmark at Scale 26
TSUBAME-KFC
AWS EC2 c4.8xlarge instances
the needs of HPC and data-intensive workloads
App Time
MPI Time
App Time
MPI Time
Experimentation
1.6
1.6
1.4
1.4
1.2
1.2
Performance evaluation of applications on AWS C4 instances against the110.8
0.8
0.6
0.6
baseline results of a supercomputer, TSUABME-KFC
0.40.40.2024816# of instance(8 MPI ranks, 12 OpenMP threads per instance)* 2 MPI ranks and 12 openmp threads per node achieved the best performance on TSUBAME-KFC. In case of AWS EC2, it
was 8 MPI ranks and 12 openmp threads per instance.

TSUBAME-KFC: Intel Xeon E5-2620v2 x2, InfiniBand FDR
Amazon EC2 c4.8xlarge instance: Intel Xeon E5-2666v3, 10Gbps Ethernet

http://www.gsic.titech.ac.jp/sc18