The Workshop on Establishing Resilient Life-Space in the Cyber-Physical Integrated
Society, March. 17, 2015, Sendai, Japan

Disaster-Resilient Backbone andAccess Networks
Shigeki Yamada(shigeki@nii.ac.jp)

Principles of Informatics Research Division,
National Institute of Informatics (NII), Tokyo, Japan1

Introduction and Background• This presentation is a summary of two-year Resilient Network
Research Project promoted under JSPS Resilient Life Space Umbrella
Project.
• When natural disasters such as earthquakes, and tsunami occur, theymay cause network breakdowns due to link and node failures,
resulting in network service disruptions.
• The network should quickly recover and keep operating after thedisasters.
• Resilience: the ability of network to provide an acceptable level ofservice in the face of various faults and challenges to normaloperations.
• Resilient technologies for two types of network (the backbonenetwork and access network) are investigated to make networks
more resilient.
2National Institute of Informatics

Backbone Networks and Access Networks
Core Nodes
Core Links(OpticalWired links)

Backbone
Networks

Access
Node

Edge Nodes
Access NodeAccess Node
Wired

Wireless

AccessNetworks

Wired

Wireless

National Institute of Informatics

Proposed Approaches for Backbone Networks& Access Networks• Backbone networks– Abundant and redundant network resources (links androuters/switches) for large bandwidth and high reliability.
– A part of backbone networks may continue to survive even if alarge scale link/node failure occurs due to a large disaster.
– Utilizing still available network resources (links, nodes) could
enable the network to continue providing acceptable services forquick recovery.
• Access networks– located close to users, and usually not redundant: once a disasterbreaks down access networks, it may be very difficult to quickly
repair them.
– Rather than repairing the destructed access network, network
users in the disaster area could construct their access network,4
using any available devices more quickly andmoreeasily.
National Institute of Informatics

Requirements to Resilient Backbone Networks• Network resilience could be measured by the Network Recovery Timewith two major time components.
1. Failure Detection Time: detection of alarms and alerts to locate
network faults
2. Switchover Time: disables a failed port, enables another, reroutes
traffic around a failed switch or router• For failure detection, existing detection technologies like BFD(Bidirectional Forwarding Detection) could be utilized.
• For switchover, we apply SDN (Software Defined Networking)/OpenFlow technology because SDN/OpenFlow has a potential
capability to provide more programmability and flexibility to respondfaster to network situational changes than existing technologies (likeMPLS).
• For Network Recovery Time, at most 50 ms is considered tolerable to5complete path restoration, in the provider networks.
National Institute of Informatics

SDN/OpenFlow for Backbone Networks

as applications.

by separating the control plane and the data plane.
6National Institute of Informatics

Goal of Resilient Backbone Network• To provide non stoppable end-to-end services in the various criticalenvironments, including link/path and node failures
Users

OpenFlow
SwitchesBackbone
NetworkSDN/OpenFlow
Controller

LinkFailure
Fast Switchover

Server Virtualization(SDN/OpenFlow)

Cloud Data Center

Realtime Migration

Cloud Data Center
7
National Institute of Informatics

Three Research Issues for Fast Network Recovery
Using SDN/OpenFlow Technology (1)1.

Switchover mechanism from a faulty link to a normal link Switchover time: the time from failure detection to path restoration on an
end-to-end basis.
 OpenFlow supports a wide variety of switchover mechanisms not available inexisting network recovery mechanism.
 We investigate some of the OpenFlow–specific and OpenFlow-integratedswitchover mechanisms and evaluate their switchover performanceImplementation.
 OpenFlow–specific switchover mechanisms:
 FAILOVER GROUP TABLE-based implementations
 SELECT GROUP TABLE-based implementations
 Both utilize local states of OpenFlow switches without involvement of
remotely located controllers
 OpenFlow-integrated switchover mechanisms: OpenFlow with Multipath
TCP (MTCP) in the TCP layer
8National Institute of Informatics

Three Research Issues for Fast Network Recovery
Using SDN/OpenFlow Technology (2)2.

Communication delay (propagation delay) between SDN controllers and switches 2/3 of light speed: 5 ms delay for 1000 km distance Analysis of the communication delays under a realistic network topology
3. Global view of the network Necessary to find any available network resources (paths, links) to restore all
the end-to-end paths.
 In the conventional IP network, a global view is maintained by the IP routing
protocols like OSPF and BGP: Slow convergence time In the SDN/OpenFlow network, a global view could be maintained andupdated by multiple SDN controllers to keep the network scalable: severalsolutions have already been proposed.
 We assume that in our implementation and evaluation, a global view ismaintained among the SDN controllers either by the existing IP routing
protocols or any new routing protocols for SDN/OpenFlow.
 Focus on the end-to-end network level evaluations of network recovery
performance under realistic scenarios.
9National Institute of Informatics

1st Issue: Fast Local Switchover Mechanism (1):
FAST FAILOVER GROUP TABLE• FAST FAILOVER GROUP TABLE allows a fast switchover from theactive output port to the standby output port. (in theactive/standby mode) without the involvement of controller.

10National Institute of Informatics

Fast Local Switchover Mechanism (2):
SELECT GROUP TABLE•••

SELECT GROUP TABLE allows a single data flow to be divided into multiple subflows, eachwith a different path (output port) in a weighted round robin manner (in the active/activemode)
When link/port failures occur, the switch recalculates the weighted values, eliminates thefailed ports and reallocates the traffic to active output ports.
SELECT GROUP TABLE achieves a better resource allocation and less packet loss than FAST
FAILOVER GROUP

11National Institute of Informatics

Implementation and Evaluation of Fast Local Switchover
Mechanisms on Two Different Platforms• Software switch: Open vSwitch (OVS) on a Linux PC to support the FAST
FAILOVER GROUP TABLE• Hardware switch: Open vSwitch (OVS) mode on the hardware switch (Pica8P3295) to support the FAST FAILOVER TABLE• Average network recovery time (Disruption time) : 21.1 ms for software
switch and 39.5 ms for hardware switch

12National Institute of Informatics

Fast Local Switchover Mechanism (3):Multipath TCP (MPTCP) Integrated with OpenFlow• MPTCP is standardized by IETF (InternetEngineering Task Force), does not needto modify existing applications• MPTCP creates and maintains multiple
active paths for an end–to-end
connection• Divides the TCP flow into multiple active
TCP subflows• Each subflow may go through a different
path to achieve better resilience• OpenFlow achieves fast switchover
among multiple active paths when someof the paths fails
13
National Institute of Informatics

Implementation and Evaluation of MPTCP on WiFi
Network Environment• Two subflows each with a different path through a different
WiFi access point• When the path1 fails, path2 keeps transferring the added
path1 traffic to achieve seamless handover from Path1 toPath2

14
National Institute of Informatics

2nd Issue: Communication Delay between Controllers
and Switches

Communication Delay

National Institute of Informatics15

Analysis of Communication Delay between Controllers
and Switches• SINET3 topology is used toevaluate communication latencies
between controllers and switches.
• SINET3: the previous version ofcurrent SINET4, a Japanesenational research and education
network.
• Two latency metrics under 2/3
propagation delay of light speed :– Average Latency for allocationof controllers– Worst-Case latency: themaximum propagation delay
between nodes and controllers
16

National Institute of Informatics

Evaluation of Average and Worst-Case Latencies• The more controllers, the lower latencies• Should carefully choose the location of controller

17
National Institute of Informatics

Optimal Values of Average and Worst-Case Latencies• These latencies are much smaller, compared with thegeneral requirement of 50ms network recovery time18
National Institute of Informatics

3rd Issue: Network Recovery Performance Evaluations• Mininet 2.0, the virtual network
simulator is used to investigate theoverall behaviors of link failure
recovery under a realistic scenario
and SINET3 topology.
• The worst case latencies between
the controller and switches areassumed• When a link failure occurs on themain path, the controller software(POX) should install new rules to theswitches to switchover the traffic
flow from the faulty path to thebackup path.
19
National Institute of Informatics

Network Recovery Simulation Result• When a link failure occurs at 10th and 30th seconds, the traffic flow
is effectively turned from the faulty path to a new path thanks to
the POX controller’s global view of the network

20
National Institute of Informatics

Implementation and Evaluation of Network Recovery
by SDN integrated with IP Routing• Network recovery that integrates SDN with conventional IP
routing is implemented and evaluated on both the Mininet
simulator and a real testbed with physical OpenFlow switches.

21
National Institute of Informatics

Network Recovery Time Comparison under
OSPF protocol• Scen1: Mininet simulation running Route Flow (for OSPF)• Scen2: Pica8 switches running Route Flow (for OSPF)• Scen3: Pica8 switches running conventional IP routing protocol (L2/L3OSPF)• All results are close to the dead-interval of 4 seconds.

3.88 sec.
3.78 sec.

3.71 sec.

22
National Institute of Informatics

Effect of Communication Delays on Network Recovery Time• The larger the communication delay, the longer the network
recovery time23
National Institute of Informatics

Evaluation of Network Recovery Time under Multiple
Link Failures• A more complex network topology with 8 switches, assuming
multiple link failures• 5 redundant paths from source to destination• Mininet simulation running Route Flow (for OSPF)

24
National Institute of Informatics

Evaluation Result of Network Recovery Time under
Multiple Link Failures• The network recovery time of multiple link failures is roughly
several tens % larger than that of a single link failure

25
National Institute of Informatics

Network Recovery Time Comparison underRIPv2 protocol• Scen1: Mininet simulation running Route Flow (for RIPv2)• Scen2: Pica8 switches running Route Flow (for RIPv2)• Scen3: Pica8 switches running conventional IP routing protocol(L2/L3 RIPv2)• All Scen1 and Scen2 results are close to the timeout-timer of 15
seconds or 60 seconds.

15.49 s
(Untriggeredupdates)

16.09 s
(Untriggeredupdates)

9.43 s(Triggeredupdates)
26

National Institute of Informatics

Summary of Resilient Backbone Network Evaluation• SDN/OpenFlow technologies are technically feasible.
– It can offer a wide variety of switchovermechanism with fast switchover time of 20 to 40
ms under current implementation technologies– The overall network recovery time ranges from20 ms to 60 seconds, largely depending on theemployed routing protocol and its timer values toupdate the global view of network– Slow convergence problem remains when IP
routing is integrated.
27
National Institute of Informatics

Resilient Access Network• We propose to apply WiFi multihop access network
technology, using commodity mobile devices to provide
internet access services.
Users
LinkFaults

AccessNetwork

Smartphones/PCs

Fast and Onsite
Construction

WiFi MultihopNetwork

Backbone Network/ Internet

28

National Institute of Informatics

Multihop Communication Abstraction• A conventional
multihop accessnetwork requires eachnode to implement a
traditional ad hocrouting protocol andmaintain the routing
information for allnodes.
• The proposed multihop
communication
abstraction allows achain of single hop
WiFi network and does
not maintain multihop
routing tables
29
National Institute of Informatics

Tree Structure of Multihop Wireless AccessNetwork• A network auto-configuration software (NAS) is downloaded to transform eachnode into the WiFi virtual access point (VAP) and the WiFi station (STA), finallyforming a tree-structured multihop network

30
National Institute of Informatics

Network Reconfiguration Support:
Connectivity Status Table (CST)• Each node manages the CST containing– the status (“Connected”/”Disconnected”) of all the upward links over the pathfrom the Internet gateway (IGW) to its own node.
– the Hop_count that represents the hop distance from IGW• Each CST is automatically updated and propagated downward when the linkstatus changes

31
National Institute of Informatics

Network Auto-Configuration Software(NAS) Components in Each Node
• WiFi Abstraction for
multiple logical WiFiinterfaces• VAP abstraction to act
as Virtual accesspoint (VAP)• Reconfiguration
support forConnectivity Status
Table (CST)• NAS-downloading
trigger to download
the NAS• Only NAS is necessary
to construct the WiFi
multihop network
32
National Institute of Informatics

Field Experiments at Iwate Prefectual Universityand Ishinomaki Senshu University

33
National Institute of Informatics

Network Set-up Time
• All the tandem connected networks were set up by the university
students• The network setup time is less than 154 seconds in 8 hop network:
quick enough for emergency response

34
National Institute of Informatics

Experiment of Indoor Tandem-Connected Network with50m Distance• The experiment was made from the 1st floor to the 4th floor inside
the building of Iwate Prefectural University.

35
National Institute of Informatics

Round Tip Time (RTT) and Packet Loss of Indoor
Tandem-Connected Network with 50m Distance• 20% Packet loss and 200ms round trip time (RTT) in 12 hops were still
acceptable for ordinary Internet applications (Web browsing and Skype)• 50 m distance up to 12 hops: 600 m distance coverage

36
National Institute of Informatics

Round Tip Time (RTT) and Throughput of Outdoor
Tandem-Connected Network• 15 m Distance up to 20 hops
• (300 m distance coverage)• 30 m Distance up to 16 hops
• (480 m distance coverage)

37
National Institute of Informatics

Packet Loss of Outdoor Tandem-Connected
Network• The largest packet loss is only 5% at Node 15 with 30m hop distance: acceptableenough for VoIP services and web browsing

38
National Institute of Informatics

Throughput of Outdoor Tree-Structured Network when
Leaf Nodes concurrently transmit the packets• The throughput are different at different nodes• The lowest throughput of around 100Kbps was acceptable for Web browsing

39
National Institute of Informatics

Summary of Resilient Access Network andFinal Remarks• Resilient Access Networks: WiFi multihop access network is feasible
for real deployments.
– It can cover a large area of 500 m to 600 m in radius (more thanone kilometer in diameter) in the indoor and outdoor
environments for internet access in the disaster area
– It allows ordinary people or volunteers in the disaster area to set
up the network easily by themselves, using available commodity
mobile devices.
• Integrating SDN/OpenFlow based backbone network with WiFi
multihop access network could allow the whole network to become
more resilient to provide end-to-end seamless non-stoppable
services.

40National Institute of Informatics