Toward efficient and flexible resources provisioning
on SDN-enhaned Job Management System Framework
Cybermedia Center, Osaka University, JapanNowadays, usage patterns of users' computations on high-performance cluster system have been diversified for large-scale simulations and analyses in the various sciencefields. Since an HPC cluster system needs to accommodate multiple jobs concurrently, efficient and flexible resource management is essential for providing high performancecomputing capabilities for multiple users. However, most Job Management Systems (JMSs) available today, which are deployed on HPC cluster system for computational
workload distribution and balancing purposes, determine resource allocation to each jobs only based on computational resources such as CPU and memory. In thisresearch, we realize a novel JMS framework for handling various kinds of resources such as network resources and virtualized computational resources.

Proposed SDN-enhanced Job Management System Framework
We have been studying and developing a novel network-aware JMS integrated Software-Defined Networking (SDN) concept, which can dynamically control an entire
network in a centralized manner, into a traditional JMS [1]. The mechanisms to manage the network resources are designed and implemented as Network Management
module (NMM) leveraging OpenFlow, which is an implementation of the SDN concept. The SDN-enhanced JMS can allocate both computational and network resources
to each job according to the resource usage on a cluster system and Resource Assignment Policy defined by administrator.
Moreover, we have also been developing a mechanism for deploying job's processes to virtual machines (VMs) on computing nodes, and guaranteeing available bandwidth on communication paths allocated to a job by using QoS functions of Open vSwitches (OVSs) connected with VMs.

SDN-enhanced JMS Framework

NMM configures Flow Entries on OpenFlow switches
for assigning communication paths to the job.
OpenFlow network

Network Management Module(NMM)
Administrator can create resource assignment
policy for computational and network
programmablly through a ruby script.

ResourceAssignment
PolicyDatabase

Resource assignment
policy class module

Administrator#!/bin/csh#$ -q QUEUE#$ -pe orte 4#$ -l netprio=bandwidth#$ -l qos=1000#$ -l vm=my-vm

0.6/1.0

NetworkControlJobscript

0.5/1.0

1.0/1.0

0.9/1.0

OpenFlow
Controller(Trema)Brain1.0/1.0

0.8/1.0

Flowentry

Job

Job

Brain decides computational andnetwork resource to allocate tothe job according to resourceassignment policy.

Job

Job

Job

Job

Job

Job

Job

Job

Job

Job

Job

Job

Open vSwitch
VMJob

Job

Queueqmaster
JMS(OGS/GE)… run program …

“wrapper” configures parameters
for QoS on Open vSwitch, andboots up the VM with information
to execute the job.

shepherdVM
imagesUserwrapper

User can request resource assignment policy,bandwidth and VM image in a job script.

execd“execd” calls “wrapper”
instead of “shepherd”.

Computing node

Evaluation
To assess the efficiency of explicitly allocating communication paths to each jobs
by the SDN-enhanced JMS, we conducted an experiment to compare the average
execution time of jobs on a fat-tree cluster system. Note that processes of jobs
were allocated to real computational resources and a single computing nodeaccommodated only one process. In the experiment, we submitted multiple jobs,
each of which ran NAS Parallel Benchmarks with class B and generated 4 processes. As a result, our proposed SDN-enhanced JMS succeeded to reduce theaverage execution time of jobs.

SDN-enhanced JMS

1008060

SW2SW1OGS/GE120Time (sec)

Toward efficient and flexible resources provisioning on SDN-enhaned Job Management System Framework

Motivation and Objectives

40SW3SW4SW5SW620

02-tier fat-tree cluster system:
The number of computing nodes is 28 and every link is connected on a Gigabit Ethernet LAN.

BTCGEPFTISLUMGSPAverage execution time of NAS Parallel Benchmarks

Acknowledgments
This research was supported in part by the collaborative research of the National Institute of Information and Communications Technology (NICT) and Osaka University (Research on High Functional Network Platform Technology forLarge-scale Distributed Computing).

[1] Y. Watashiba, Y. Kido, S. Date, H. Abe, K. Ichikawa, H. Yamanaka, E. Kawai, H. Takemura, "Prototyping and Evaluation of a Network-aware Job Management System on a Cluster System Leveraging
OpenFlow", The 19th IEEE International Conference On Networks (ICON 2013), Dec. 2013.

Contact : Yasuhiro Watashiba, E-mail : watashiba-y@cmc.osaka-u.ac.jp
Web : http://hpc-sdn.ime.cmc.osaka-u.ac.jp/sdnjms/