Osaka University, Japan
SC16 BOOTH 1437

Hi-IaaS : High Performance Computing Infrastructure as a Service
Hi-IaaS

Back Ground

Software defined HPC platform covering applicationsof IoT/BigData and Science
Providing diversity of HPC Platform dynamically
Hardware (GPU/FPGA, Cluster) Software (SPARK, MPI)

Features

Proposal : Open/Dynamic

Today : Special/Static

Reasonable, scale up, high utility

Expensive, not fit, low utility

Special Protocol

OpenCLJob Scheduler

OpenCLSpecial Topology

GPU/OpenCL

Modify selected node list alongwith the management policy andutilization of compute / network

Database

node12.domain
node07.domain
node18.domain
node02.domain…

Job / Accelerator Managementwith UCLA and Falcon Computing Solutions

Routing nodecommunication path as aFlow entry

NMM

Selected node list

MapReduce
YARN / Mesos(2) Network information(3) Recommended Nodes

Brain(4) Assigned Nodes(5) Flow Entry list

JobJobJobJob(6) Network Flow
ControlServerServer

RDStoreRDStoreRDStoreRDStoreMemoryMemoryMemoryMemoryInterconnect Network (ExpEther)

PCI-e NVMe Device PCI-e NVMe Device

accaccCeilometer(Telemetry)

1. Data base (KVS)
technology・Decrease data access tothe disaggregated storagedevices by using hash database technology
・Decrease usage of memory
by making server statelessacc

Modify Ironic to control each device

・Proposal:

Server

Node Acc Manager

Heat(Orchestration)

Ironic(BM Mng.)
Fabricator

abstraction

CommandExpEther
Driver12

Host3

HostEnd User

Horizon(GUI)

Host4

IaaS
Controller

Nova(Scheduler)

ExpEtherManagerHost

3. New Non Volatile
Memory・I m p l e m e n t c o m p a r e a n dwrite func tion along withNVMe Rev1.1 specification.
・Decrease exclusive communications between servers

acc

Modify Ironic (Bare metal control) to device level.

Scalable Key-Value store (NOSQL) on Resource Disaggregated Architecture

・Communication among servers degrades performance such as storage capsyty
and access latency.

ServerNodeNode Acc Manager

OpenStack based Resource Management of
Disaggregated Platform

Resource Disaggregated data Store (RDStore) with NEC・Light-weight software defined storage system with technologies
enhancing data access and management
・Sharing NVMe storage devices among server nodes atinterconnection level (ExpEther)

Global Acc Manager

NodeCompute Node

2. Data distribution andredundancy technology

MPI

Distributed File System (HDFS)

OpenFlow
Controller(Trema)Interconnect

・Apply distributed systemtechnology to the resource
disaggregated architecture
・Enhancing performance,scalability and availability

SparkNetwork Control(1) Selected Node/Mngmnt Information(6) Computing
Resource
Assignment

MPI

Global Accelerator Manager : Global resource allocation within acluster to optimize system throughput and accelerator utilization
Node Accelerator Manager : Accelerator sharing/isolation acrossapplications using accelerator-as-a-service

System Administrator

Resource
Assignment
PolicySpark

Reconfigurable H/W

Cluster/MPI

SDN-Enhanced Job Management Systemwith Nara Institute of Science and Technology (NAIST)JMS(OGS/GE)

Resource
Manager

S-Defined Subsystem

Fat TreeGPU

Job Script

Job/Resource
Cross
Management

Scale Out/SparkSpecial Device#!/bin/csh#$ -q QUEUE#$ -pe ompi 4#$ -l netprio=policy_namempirun -np 4 ./a.out

MPISpark

RDMAMust Be Reserveduser

Job-resource cross management system
Job management over Accelerators(GPU/FPGA)
Shared NVM+PCIe fabric based scalable High-speed storage
PCIe device level reconfiguration

HPC computing platform going into cloud. however
HPC computing platform needs application specificconfiguration
Ex. #GPU, #FPGA, InfiniBand, FatTree

Ethernet

12313Fabric

12A B C D E F G H I

413JContact : system@cmc.osaka-u.ac.jp Visit us at www.cmc.osaka-u.ac.jp

GUI on modified Horizon