Ethernet-based Software Defined
Network (SDN)

Cloud Computing Research Center for
Mobile Applications (CCMA), ITRI
雲端運算行動應用研究中心

11

SDN Introduction• Decoupling of control plane from data plane– Breaks the existing vertical integration model in network equipment
industries– Enables quick development and deployment of innovative third-party
control plane applications• Centralization of control plane– Management vs. Control vs. Data plane– Facilitates fast deployment and upgrade of control protocols– Enables global allocation of a network’s resources• Software in software-defined network– User applications– Control protocols2SDN Software Architecture
UserApplications
••••
•
•

Data plane
Controller
Control plane applications
Southbound AP
Northbound API
NorthNorthbound API

Management
Software

Northnorthbound API
Control
Plane
Application 1

ControlPlane
Application 2

Control
Plane
Application 3

Northbound API
Controller
Southbound API
VirtualSwitch

OpenFlow
SwitchEthernet
SwicthOpenFlow Architecture• OpenFlow switch: A data plane that implements a
set of flow rules specified in terms of the OpenFlow
instruction set• OpenFlow controller: A control plane that sets up
the flow rules in the flow tables of OpenFlowswitches• OpenFlow protocol: A secure protocol for an
OpenFlow controller to set up the flow tables in
OpenFlow switches
4OpenFlow Controller
OpenFlow Protocol (SSL/TCP)

Control Path

Flow Table

Data Path (Hardware)
5OpenFlow Basics
RuleAction

Stats

Packet + byte counters
1. Forward packet to zero or more ports
2. Encapsulate and forward to controller
3. Send to normal processing pipeline
4. Modify Fields
5. Any extensions you add!
Switch VLANPortIDVLAN MAC
pcp src

MAC
dst

EthtypeIPSrc

IPDst

IP
L4
IP
ToS Prot sport

L4dport+ mask what fields to match6Examples
Switching
Switch MAC
Portsrc
*

*

MAC
dst

Ethtype

00:1f:.. *

VLAN
ID

IP
Src

IP
Dst

IP
Prot

*

*

*

*

VLAN
ID

IP
Src

IP
Dst

IP
Prot

TCP
TCP
sport dport**

Actionport6

Flow Switching
Switch MAC
Portsrc

MAC
dst

Ethtype

port3 00:20..

00:1f.. 0800 vlan1 1.2.3.4 5.6.7.8

TCP
TCP
sport dport

4 17264

80

Actionport6

FirewallSwitch MAC
Port
src
*

*

MAC
dst

Ethtype

VLAN
ID

IP
Src

IP
Dst

IP
Prot***

*

*

*

TCP
TCP
sport dport*22Action
drop7

ExamplesRouting
Switch MAC
Portsrc
*

*MAC
dst

Ethtype

VLAN
ID

IPSrc

IP
Dst

IPProt****5.6.7.8*MAC
dst

Ethtype

VLAN
ID

IPSrc

IP
Dst

IPProtTCP
TCP
sport dport**

Action
port6

VLAN Switching
Switch MAC
Portsrc**

00:1f..

*vlan1

*

*

*TCP
TCP
sport dport**

Actionport6,port7,port98

SDN  OpenFlow• Can we apply SDN to Ethernet
switches, especially in cloud data
center space? Peregrine

9Cloud Data Center Architecture
Physical Server
VM0

VM1

VMn

Layer-3
BorderRoutersData Center
Network Fabrics
IEL
Load Balancing
Traffic Shaping
Intrusion Detection
NAT/VPN

Compute Server
RackStorageServer10Cloud Data Center Network• Cloud data centers are Big and Shared• Scalable and available data center fabrics– Not all links are used
– No load-sensitive routing– Fail-over latency is high (> 5 seconds)• Network virtualization: Each virtual data center (VDC) gets todefine its own network– VMs in a VDC belong to one or multiple subnets (broadcast domains)– Each VDC has its own private IP address space– Each VDC has a set of public IP addresses for service entry points and for VPN
connections with external sites– Each VDC has its Internet traffic shaping policy, intra-VDC and inter-VDC
firewalling policy, and server load balancing policy

11What’s Wrong with Ethernet?
• Spanning tree-based– Not all physical links are used
– No load-sensitive dynamic routing– Fail-over latency is high ( > 5 seconds)• Cannot scale to a large number of VMs (e.g. 1M)– Forwarding table is too small: 16K to 64K• Does not support VM migration and visibility• Does not support network virtualization

12Peregrine• A unified Layer-2-only data center network for LAN and SAN traffic• A SDN architecture using only commodity Ethernet switches:
centralized control plane and distributed data plane• Turn off Ethernet’s control protocol: spanning tree, source learning,flooding of unknown-destination-MAC packets, broadcast of ARP andDHCP– VLAN is optional because VLAN ID space is limited• Centralized load-balancing routing using real-time traffic matrix• Fast fail-over using pre-computed primary/backup routes• Native support for network virtualization– Private IP address space reuse– Multiple subnets per virtual network
13Peregrine Software Architecture
RouteAnalytics
ServerDirectoryServerVM2

VM3

Peregrine Agent
Ethernet Switch-based
Network Fabrics

PhysicalServer 1

PhysicalServer 4

Hypervisor

HardwarePhysicalServer 714

Dynamic Traffic Engineering• Periodic collection of real-time traffic matrix– Traffic volume between each pair of VMs– Traffic volume between each pair of PMs• Load balancing routing algorithm– Loads on the physical links– Number of hops
– Forwarding table entries– Prioritization• Computed routes are programmatically installed on
the forwarding tables of Ethernet switches
15

Fast Failure Recovery• For a given node N, proactively computes two disjoint paths(primary and backup) from every other node to N, and installs
them on the associated switches• All primary (secondary) paths to a given node N form aspanning tree
• Each node has two MAC addresses, which are installed on theswitches of its primary and backup trees, respectively• When a primary path from S to D goes down, S is notified thatit should use D’s backup MAC address to reach D• RAS and DS support fail-over16

Fast Failure Recovery Example

17

Network Virtualization• Multiple virtual networks running on a single physical network• The network of each virtual data center (VDC) consists of– VMs’ MAC addresses are pre-assigned– A single layer-2 network– A complete private IP address space, organized into multiple subnets eachwith its own broadcast domain– A set of public IP addresses– Its own copy of the DHCP and DNS service– Security: Intra-VDC and inter-VDC firewall policy– SLA: Traffic shaping policy– Scalability: Server load balancing policy

Private IP Address Space Reuse• Requirement: Every VDC has a VDC ID and its own full 24-bit
private IP address space (10.x.x.x), even though multiple VDCs
run on top of the same data center network• Two approaches:– Ethernet over TCP/UDP:• Every Ethernet packet is encapsulated inside an TCP/UDP packet orTCP/UDP connection as an Ethernet link
• Needs to implement in software such Ethernet switch functions as source
learning, flooding, VLAN, etc.
• Can work with arbitrary IP networks– Multi-tenancy-aware IP-MAC mapping: our approach• VDC ID + private IP address  MAC address• Runs directly on L2 networks, no need for Ethernet switch emulation• Inter-virtual-data-center isolation
2013/6/2

19

Hybrid Cloud Support
A2
A1, A3, A4VPN2

On-premise
physical data center

VPN2
A2, A3, A4 VPN1

A7

A9A1

A1, A2, A4VPN3

VPN1

Internet

VPN3

A3

A1
VPN4

Cloud-based
virtual data center

2013/6/2

A1, A2, A3VPN4A420

Cross-Site Global Load Balancing
Client<D2,S><S,D1><S,D2>SLBDNS
Server<S,D2>VPN<D1,S>SLBInternet

VPN<S,D1>VM

VM

Cloud-based
virtual data center
2013/6/2

On-premise
physical data center
21Peregrine in SDN Framework• Data plane: Ethernet switches• Southbound API: SNMP and CLI• Controller: (1) Physical network resource set-up, (2) Physical topology record
keeping, (3) SNMP trap processing, (4) Ethernet switch configuration, including
forwarding table programming, and (5) Traffic load information• Northbound API:––––

Failure/congestion notification, including SNMP trap packet delivery
ARP request packet deliveryForwarding table programming
Physical topology/traffic load querying• Control plane applications:– Dynamic traffic engineering– Fast fail-over for data/control plane failures– Network virtualization
22

Wish-List Ethernet Switch Features• Ability to turn off flooding of unicast packets withunknown destination MAC address• Ability to turn off source MAC address checkenables asymmetric routing• Bulk uploading of forwarding table• Fast link/switch failure detection• Link traffic load counting• Interception and redirection of packets of selectedtype23

Comparisons• Scalable and available data center fabrics– IEEE 802.1aq: Shortest Path Bridging– IETF TRILL– Competitors: Cisco, Juniper, Brocade– Differences: commodity switches, centralized load balancing routing andproactive backup route deployment• Network virtualization– OpenFlow protocol: between OpenFlow controller and OpenFlow switches– Competitors: Nicira, NEC and NTT– Generality carries a steep performance price• Every virtual network link is a tunnel– Differences: Simpler and more efficient because it runs on L2 switches
directly• Flow table state management and flow table lookup performance overheadCurrent Status• A fully operational Peregrine prototype that works on a 10switch and 100-server test-bed– Start-up and shut-down without out-of-band control network– Fast fail-over for both data plane and control plane failures– Dynamic traffic engineering• On-going work:– An agentless implementation of Peregrine– Encapsulate Peregrine with Internet Edge Logic functionalities as aQuantum plug-in implementation– Port dynamic traffic engineering, fast fail-over, and network
virtualization to an OpenFlow platform, i.e. OF controller + OFswitches
25Summary• Peregrine is a network system technology, not a network
device technology, and consists of– A hypervisor agent running on every compute node
• L7/Web application firewall and traffic shaping– A centralized route server and directory server– A VDC-aware Internet Edge Logic cluster• Server load balancing, VPN, NAT, and L4 firewall• Network virtualization technology that does not use tunnelingor VLAN
• A software defined network (SDN) architecture that runs oncommodity Ethernet switches, and is able to manage bothlegacy Ethernet and OpenFlow switches
26

Thank You!
For More Information:
Y.F. Juan / 阮耀飛Deputy Director
Strategy & Business Development
Cloud and Mobile Computing CenterIndustrial Technology Research Institutet: +886 (0) 3.591.6173
m: +886 (0) 975.876.919
e: yf.juan@itri.org.tw27