The Evolution of Software-Defined
Networking: From Academic Roots to Industry
Revolution
Marc Bruyere @ IIJ Lab© Internet Initiative Japan Inc.

1

Agendan Nothing is created ex-nihilo "何も虚無から創造されることはない"• An historical overview• Protocol Independent Switch Architecture and P4 Langn A short P4 tutorial• PISA – P4 Key elements• L3 routing short introduction• Learning by yourself

n A couple of projects from IIJ lab• The IIJ Lab testbed• P4 HolistIX : IXP switching fabric with a zero-touch approach• UmbrellaDC : Data-Center switching fabric fast flow completion - no overhead• Gandalf : Packet amplification and nano-sec latency timestampingn Outcomes and conclusion2• Year: 1995

OpenSignaling(OPENSIG)• Initiated to make networks (ATM, Internet, mobile)
more open and programmable.
• Aimed for separation between communication
hardware and control software.
• Led to the development of the General Switch
Management Protocol (GSMP), concluded in June2002.
• From : ITU-T

3

• Mid-1990s

ActiveNetworking• Proposed programmable network infrastructure forcustomized services.
• Two main approaches: user-programmable switches
and capsules (program fragments in messages).
• Despite considerable activity, it faced challenges inwidespread use and deployment.

Towards an Active Network Architecture (1996),David L. Tennenhouse, et al.,Computer Communication Review ACM
4• Early 2000s• An approach parallel to SDN, focusing on separation
within network devices.
• Key concepts for Separation of IP Control andForwarding are described in RFC 3654 & 5810

ForCES• IETF WG : Forwarding and Control Element Separation• Ongoing standardization with various publisheddocuments including architectural framework andcommunication protocols.

Real adoption of the concepts
But no vendor have adopted ForCes RFCs5

• Year: 2005
• Advocated a clean slate design emphasizing
separation between routing decision logic and network
interaction protocols.

4D Project• Influenced the thinking towards decoupled control anddata planes in networking.
• Inspired later works like NOX, proposing an "operating
system for networks" for OpenFlow-enabled networks.
• Research on a radical design project aimed at shaping
industry practices.
Albert Greenberg, Gisli Hjalmtysson, David A. Maltz, AndyMyers, Jennifer Rexford, Geoffrey Xie, Hong Yan, Jibin Zhan,and Hui Zhang. 2005. A clean slate 4D approach to network
control and management. SIGCOMM Comput. Commun. Rev.
35, 5 (October 2005), 41–54.
https://doi.org/10.1145/1096536.1096541

6• Year: 2007
• Predecessor to OpenFlow, designed for enterprisenetwork architecture.

Ethane• Focused on centralized control for managing policy
and security.
• Consisted of two components: a controller and an
Ethane switch with a flow table and secure channel.
• Implemented on HW NetFPGA and NEC Ethernet
switch with Broadcom ASIC and deployed atWilliam Gates Building @ Stanford University.

Martin Casado, Michael J. Freedman, Justin Pettit, Jianying Luo, Nick
McKeown, and Scott Shenker. 2007. Ethane: taking control of theenterprise. SIGCOMM Comput. Commun. Rev. 37, 4 (October 2007),
1–12. https://doi.org/10.1145/1282427.1282382

7

The OpenNetworkingFoundation• Year: 2011
• A full networking ecosystem and growing community.
• Google adoption of OpenFlow.
• Various start-up created ( Nicira Networks ).
• 60 members in less than a year

8

P4Programming
ProtocolIndependent
PacketProcessors• Year: 2014
• Enhances SDN flexibility ( no fixed fields as in OF ).
• A Protocol-Independent Switch Architecture• Introduce a domain specific language with a compiler
for programmable parsers and control flowmechanisms.
• Hardware target-independent

P. Bosshart et al., “P4: programming protocol-independent
packet processors,” SIGCOMM Comput. Commun. Rev.,
vol. 44, no. 3, pp. 87–95, Jul. 2014, doi:10.1145/2656877.2656890.
9

• P4-LANG versions.

P4 ShortTutorial• How to program a simple L3 routing.
• Do yourself the full tutorial to learn how to implement.
many others features and invent new ones.
• Thanks for the nice tutorial works from ETH Zurich.

10

11

IP forwarding
in a traditional router

1.2.3.4

1.2.3.5

1.2.3.254

5.6.7.1

5.6.7.2...

5.6.7.200
...

LAN 1

LAN 2
routerrouterrouter

1.2.3.0/24
5.6.7.0/24…

forwarding table

12

IP forwarding
in P4

1.2.3.4

1.2.3.5

1.2.3.254

5.6.7.1

5.6.7.2...

5.6.7.200
...

LAN 1

LAN 2
routerrouterrouter

1.2.3.0/24
5.6.7.0/24…

forwarding table

13

When forwarding an IP packet,
an IP router performs four actions:
Lookup the next hop to use
Update the M A C addresses
Decrement the TTL
Forward the packet to the output port
Each of them should be in implemented in P4

14

A P4 program consists of three basic parts

15

ParserMatch-Action Pipeline

Deparser

16

ParserMatch-Action Pipeline

Deparser

Specifies the headers to extractfrom the incoming packet

17

ParserMatch-Action Pipeline

Deparser

Specifies the processing logic to apply on
the extracted headers

18

ParserMatch-Action Pipeline

Deparser

Specifies how
the output packet
will look on the wire19

ParserMatch-Action Pipeline

DeparserV1Switch(MyParser(),
MyVerifyChecksum(),MyIngress(),MyEgress(),
MyComputeChecksum(),
MyDeparser()) main;20

#include <core.p4>#include <v1model.p4>
const bit<16> TYPE_IPV4 = 0x800;
typedef bit<32> ip4Addr_t;
header ipv4_t {…}struct headers {…}parser MyParser(…) {state start {…}state parse_ethernet {…}state parse_ipv4 {…}}Libraries
Declarations

Headers extraction

control MyIngress(…) {action ipv4_forward(…) {…}table ipv4_lpm {…}}Processing logic

apply {if (…) {…}}

control MyDeparser(…) {…}V1Switch(MyParser(),
MyVerifyChecksum(),MyIngress(),MyEgress(),
MyComputeChecksum(),
MyDeparser()) main;Output packet
assembly“main()”

21

Parser

22

The parser relies on a programmable state machine
which maps packets into headers and metadata

Packeta:b:c:d → 1:2:3:4
1.2.3.4 → 5.6.7.8

Headers and metadata
meta {ingress_port: 1, …}
ethernet {srcAddr: a:b:c:d, …}
ipv4 {srcAddr: 1.2.3.4, …}

1234 → 56789tcp {srcPort: 12345, …}

Payload23

The state machine has three predefined states:
start, accept and reject
initialstate

start

PacketHeaders and metadata
meta {ingress_port: 1, …}

a:b:c:d → 1:2:3:4

ethernet {srcAddr: a:b:c:d, …}

1.2.3.4 → 5.6.7.8

ipv4 {srcAddr: 1.2.3.4, …}
1234 → 56789
tcp {srcPort: 12345, …}

acceptrejectparsing
success

parsing
failurefinal states24

parser MyParser(…) {state start {transition parse_ethernet;}start

parse_ethernet

parse_ipv4parse_tcp

acceptparse_udp

rejectstate parse_ethernet {packet.extract(hdr.ethernet);transition select(hdr.ethernet.etherType) {0x800: parse_ipv4;
default: accept;}}
state parse_ipv4 {packet.extract(hdr.ipv4);
transition select(hdr.ipv4.protocol) {6: parse_tcp;
17: parse_udp;
default: accept;}}
state parse_tcp {packet.extract(hdr.tcp);transition accept;}
state parse_udp {packet.extract(hdr.udp);transition accept;}}

25

Match-Action Pipeline26

In the match-action pipeline,control blocks manipulate headers and metadata
A control block is an imperative program
which describes how to process packets

27

A control block is an imperative program
which describes how to process packets
Headers and metadata from parser
control MyIngress(inout headers hdr,
inout metadata meta,
inout standard_metadata_t std_meta) {bit<9> port;

Variable declaration

apply {
port = 1
std_meta.egress_spec = port;
hdr.ethernet.srcAddr = hdr.ethernet.dstAddr;
hdr.ethernet.dstAddr = 0x2;
hdr.ipv4.ttl = hdr.ipv4.ttl - 1;}}

Control flow

28

Actions allow to re-use codein a block or globally, they are similar to C functions
control MyIngress(inout headers hdr,
inout metadata meta,
inout standard_metadata_t std_meta) {action ipv4_forward(macAddr_t dstAddr,egressSpec_t port) {std_meta.egress_spec = port;
hdr.ethernet.srcAddr = hdr.ethernet.dstAddr;
hdr.ethernet.dstAddr = dstAddr;
hdr.ipv4.ttl = hdr.ipv4.ttl - 1;}
a pply {}ipv4_forward(0x123, 1);}

29Control blocks often rely on tables which map
specific headers values to specific actions

30

Control Plane

KeyMatchKeyAction
ID
DataHeaders andMetadataHitIDHeaders& Meta

DefaultAction
CodeDataHeaders& Meta
31

Table nametable
key = {Field(s) to match{:Match type;}
actions = {Possible actions}size =;

default_action =Max. # entries in table;

Default action}

32

Example: IP
forwarding table
1.2.3.4

1.2.3.5

1.2.3.254

5.6.7.1

5.6.7.2...

5.6.7.200
...

LAN 1

LAN 2
routerrouterrouter

1.2.3.0/24
5.6.7.0/24…

forwarding table

33

Control Plane
Destination IP address
KeyMatchKey

ActionID
Data

Headers andMetadata
HitID

ActionCodeHeaders& MetaDefault

DataHeaders& Meta34

Control Plane
Longest prefix match
Key

Match
Key

ActionID
Data

Headers andMetadata
Hit

ID

ActionCodeHeaders& MetaDefault

DataHeaders& Meta35

1:
ipv4_forward(mac,port)
ControlPlane
2: drop()Key

MatchKey

ActionID
Data

Headers andMetadata
HitID

Headers& MetaDefault

ActionCodeData

Headers& Meta36

Table nametable ipv4_lpm {key = {
hdr.ipv4.dstAddr: lpm;}
actions = {ipv4_forward;drop;}
size = 1024;
default_action = drop();

Destination IP address
Longest prefix match

Possible actions

Max. # entries in table
Default action}

37

Example: IP
forwarding table
1.2.3.4

1.2.3.5

1.2.3.254

5.6.7.1

5.6.7.2...

LAN 1
router
01:01:01:01:01:01

5.6.7.200...

1router2

LAN 2
router
02:02:02:02:02:02

action ipv4_forward(macAddr_t dstAddr, egressSpec_t port) {hdr.ethernet.srcAddr = hdr.ethernet.dstAddr;
hdr.ethernet.dstAddr = dstAddr;
hdr.ipv4.ttl = hdr.ipv4.ttl - 1;
standard_metadata.egress_spec = port;}

38

Example: IP
forwardingtable
1.2.3.4

1.2.3.5

1.2.3.254

5.6.7.1

5.6.7.2

...

LAN 1

...

1LAN 2

2

router
01:01:01:01:01:01

5.6.7.200

router

1.2.3.0/24

router
02:02:02:02:02:02

5.6.7.0/24…

forwarding table

39

Control Plane
Headers andtable_add ipv4_lpm ipv4_forward 1.2.3.0/24 => 01:01:01:01:01:01
1KeyMatchAction
2
table_add ipv4_lpm ipv4_forward 5.6.7.0/24 => 02:02:02:02:02:02
Metadata
KeyIDData

1.2.3.0/24

101:…, 1

5.6.7.0/24

102:…, 2

Hit

IDHeaders& MetaDefault

2

ActionCode

Data

Headers& Meta40

Deparser

41

The Deparser assembles the headers backinto a well-formed packet

Headers

Deparser

Packetethernet {srcAddr: a:b:c:d, …}
ipv4 {srcAddr: 1.2.3.4, …}
tcp {srcPort: 12345, …}

42

Headers
ethernet {srcAddr: a:b:c:d, …}

DeparserPacket
a:b:c:d → 1:2:3:4

ipv4 {srcAddr: 1.2.3.4, …}
tcp {srcPort: 12345, …}

control MyDeparser(packet_out packet, in headers hdr) {apply {
packet.emit(hdr.ethernet);

}
}

43

Headers
ethernet {srcAddr: a:b:c:d, …}
ipv4 {srcAddr: 1.2.3.4, …}

Deparser

Packeta:b:c:d → 1:2:3:4
1.2.3.4 → 5.6.7.8

tcp {srcPort: 12345, …}

control MyDeparser(packet_out packet, in headers hdr) {apply {
packet.emit(hdr.ethernet);packet.emit(hdr.ipv4);}}

44

Headers
ethernet {srcAddr: a:b:c:d, …}
ipv4 {srcAddr: 1.2.3.4, …}
tcp {srcPort: 12345, …}

Deparser

Packeta:b:c:d → 1:2:3:4
1.2.3.4 → 5.6.7.8
1234 → 56789

control MyDeparser(packet_out packet, in headers hdr) {apply {
packet.emit(hdr.ethernet);packet.emit(hdr.ipv4);
packet.emit(hdr.tcp);}}

45

Headers
ethernet {srcAddr: a:b:c:d, …}
ipv4 {srcAddr: 1.2.3.4, …}
tcp {srcPort: 12345, …}

DeparserPacket
a:b:c:d → 1:2:3:4
1.2.3.4 → 5.6.7.8
1234 → 56789

PayloadWe have implemented a simple L3 routing in a P4 pipeline46

47

Learning by yourself – Continue the experience

https://github.com/nsg-ethz/p4-learning

https://p4.org/learn

48

IIJ Lab – P4 SDN projects

49

IIJ Lab – P4 Projects – The Testbed in the WorkShop Room

• 5 x Tofino1 switch of 32 x 100G QSFP28 ports• 2 x Server with 10Gs and 100G Interfaces

50

IIJ Lab – P4 Projects – HolistIX• Introduce automation from the top down for IXPs
• Plan > Verify > Deploy• Based on the Software Designed NetworkUmbrella switching fabric implemented in P4• Change broadcast packets to unicast ones
• No more quarantine time – fast deployement• No overhead like EVPN – MPLS based51

IIJ Lab – P4 Projects – UmbrellaDC• Focus on Optimizing flow completion time for RPC
based protocols [ FlowLet load-balancing ]supporting TCP and Homa
• Improved latency with no Overhead and minimalpipeline steps (Homa focus on l4-l7 and not on l2-l3)• Today DC minimal properties:• Seamless migration (No @IPs change) > L2 Flat
• Top-to-bottom network management (e.g HolistIXapproach) - Single Source of Truth centralized management• Any end host should be able to communicate with any othervia all available physical paths

52

IIJ Lab – P4 Projects – GANDaLF
Gateway for Advanced Network Duplication and Latency Feedback• Nano-second measurement• 100Gps from 64B to Jumbo• No cost and hardware generated

53

IIJ Lab – P4 Projects – GANDaLF
Gateway for Advanced Network Duplication and Latency Feedback54

Outcomes, Thoughts and Conclusion

SDN - Research and Industry combined evolutionn P4-Lang is powerful language to describe match-action
data-plane pipeline !

n Even if Intel Barefoot as stopped to invest in Tofino3 ASIC,
various organization are using P4-Lang as their reference
model language.

n Others hardware chipset may support it in the future.
n P4 was made for high-performance hardware, does SmartNIC orsoft switch could keep P4 community active ?

n Self-learning is key for innovation – Have funn I am pleased to share more in-depth tech details !

55

Thanks !
Questions – Discussions© Internet Initiative Japan Inc.

56