Osaka University, Japan
SC16 BOOTH 1437

SDN-enhanced MPI: Towards Dynamic and Application-aware Interconnect Architecture

1. Software-Defined Networking (SDN)
Software-Defined Networking (SDN) is a new concept of networkarchitecture that decouples conventional networking function into a
programmable control plane (responsible for deciding how to control
the packets) and a data plane (responsible for the actual packetdelivery). Currently, OpenFlow is the most common implementationof SDN, which enables to dynamically control the for warding
functionality of network from a centralized controller.

3. SDN-enhanced MPI Communication PrimitivesController
Duplication

Network Controller

2. Centralized Control
over the Network
1. Programmable

3. Dynamic Control

Network Device

Network Device

SDNController(3) Networkpreparation request(1) Topology discovery(4) Route calculation(5) Route installation

Network Device(2) MPI_Allreduce invoked

Network Device

SDN_MPI_Bcast is an SDN-enhanced version of MPI _ Bcast,which is the broadcasting funct i o n i n M PI . SD N _ M PI _ B c a s t
offloads packet duplication operations during the broadcast onto
SDN switches. As a result,
SDN_MPI_Bcast has successfully decreased the number of communications and communication
latency of MPI_Bcast.

B. SDN_MPI_Allreduce [2]SDN _ MPI _ Allreduce is anSDN-enhanced version of MPI_Allreduce. Since it requires multiple simultaneous communication
between nodes, congestion may
happen on an oversubscribedinterconnect. We employ areal-time traffic load balancing
method to solve this problem.

4. Coordination Mechanism of Communicationand Computation

2. Aim of SDN-enhanced MPI
We have been developing SDN-enhanced MPI based on the ideathat a mechanism that configures and controls the network of acluster system depending on the requirement of each application isessential. The key concept of SDN-enhanced MPI is to utilize theunderlying network of a computer cluster to its maximum capacity
by leveraging the flexible network controllability of SDN.

A. SDN_MPI_Bcast [1]We propose an integrated framework [3] to combine SDN-MPI
components that we have developed in our previous works. In thisframework, MPI packets are tagged with MPI-layer information which
are used by the SDN switches to determine how to control thepackets.

SDN Controller(3) Packet Control based on tags

(1) Pre-install rules

Dynamic Interconnect Reconfiguration

eddMggge

SDN SwitchtaMPg
TaUnCommunication
Scheduling

PIIPacket

MPI Packet

MPI Application
Communication PatternTagging Module
MPI Library
MPI Application(2) Packets aretagged with application level info

Interconnect Topology and Usage

Publications[1] K. Dashdavaa et al., Architecture of a High-Speed MPI_Bcast Leveraging
Software-Defined Network, in: UCHPC2013 6th Work. Unconv. High Perform.
Comput., 2014: pp. 885–894.
[2] K. Takahashi et al., Performance Evaluation of SDN-enhanced MPI_Allreduceon a Cluster System with Fat-tree Interconnect, in: Proc. Int. Conf. High
Perform. Comput. Simul. - HPCS 2014, 2014: pp. 784–792.
[3] K. Takahashi et al., Concept and Design of SDN-Enhanced MPI Framework,in: Proc. Fourth Eur. Work. Softw. Defin. Networks - EWSDN 2015, 2015: pp.
109–110.

Takahashi : takahashi.keichi@ais.cmc.osaka-u.ac.jp
Contact Keichi
Khureltulga Dashdavaa : huchka@ais.cmc.osaka-u.ac.jp

MPI Packet

MPI Library
MPISocket API

MPI Packet

NetworkStackTo NIC
MPI Packet

System CallUser MPI
Application

Tagging
ModuleUser Space

Kernel Space

Visit us at www.cmc.osaka-u.ac.jp