/dev/stdpkt: A Service Chaining Architecture with
Pipelined Operating System Instances in a Unix Shell
Motomu Utsumi

The University of Tokyo, Japan
motomu@hongo.wide.ad.jp

Hajime Tazaki

IIJ Research Laboratory, Japan
tazaki@iij.ad.jp

ABSTRACT

Network Function Virtualization is being recognized as animportant solution to satisfy a wide spectrum of user requirements, such as cost-e�ective and �exible service provisioning. Service Function Chaining is a key function in Network
Function Virtualization to achieve rich and �exible pipelined
network services with multiple virtual network functions.
We have investigated the implementation of function chain
using a shell pipeline of Unix processes, which is the basetechnology contributing the practical success of the Unix
philosophy.
In this paper, we evaluate the applicability of this idea in
the function chaining scenario by conducting benchmarks inpossible use cases. The evaluation is based on the prototype
system which extends an existing userspace network stack,Linux Kernel Library, because it can use various mature network functions and has various performance optimization
techniques. The evaluation con�rms that, although the prototype system needs some performance improvements, thecomplex packet processing provided by functional chains ofvirtual nodes can be completely functional. Furthermore, we
demonstrate up to 25% improvement on application goodput.

CCS CONCEPTS• Networks → Middle boxes / network appliances; Network management;

KEYWORDS

Service Chaining; Network Function Virtualization; Middlebox; Unix pipe
Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies are notmade or distributed for pro�t or commercial advantage and that copies bear
this notice and the full citation on the �rst page. Copyrights for componentsof this work owned by others than ACM must be honored. Abstracting withcredit is permitted. To copy otherwise, or republish, to post on servers or toredistribute to lists, requires prior speci�c permission and/or a fee. Request
permissions from permissions@acm.org.
AINTEC ’17, November 20–22, 2017, Bangkok, Thailand© 2017 Association for Computing Machinery.
ACM ISBN 978-1-4503-5551-3/17/11. . . $15.00
https://doi.org/10.1145/3154970.3154972

Hiroshi Esaki

The University of Tokyo, Japanhiroshi@wide.ad.jp

ACM Reference Format:
Motomu Utsumi, Hajime Tazaki, and Hiroshi Esaki. 2017. /dev/stdpkt:
A Service Chaining Architecture with Pipelined Operating System
Instances in a Unix Shell. In AINTEC ’17: AINTEC ’17: Asian Internet
Engineering Conference , November 20–22, 2017, Bangkok, Thailand.
ACM, New York, NY, USA, 8 pages. https://doi.org/10.1145/3154970.
3154972

1

INTRODUCTION

By fully utilizing the power of virtualization, Network Function Virtualization has the potential to solve not only theoriginal motivation of replacing hard-to-upgrade facilities of
network functions based on hardware appliances, but also tobring the �exibility of composing those functions by chaining/concatenating multiple functions. We believe that the�exibility of full recomposablility is a key to making thetechnology useful in practice.
We implemented function chaining using Unix pipelines
in a standard Unix shell. Unix pipelines, �rst championed
by McIlroy [6], and later expanded to network resources by
Plan9 from Bell Labs [10], concatenates multiple programs
into a stream to process data. Following the Unix philosophy, composing a simple program (or function) which doesa simple job well, then chaining them to do larger jobs, we
implemented Service Function Chaining in a Unix shell, providing modularity, simplicity, robustness, etc [12], making
a well-working system. Prior work such as EtherPIPE [5]showed that this idea is useful for simple network packet
processing. However, it did not function well and was di�cult to implement, if the processing was complicated such as
network address translation (NAT) or load balancing withstateful connection tracking.
While Unix pipeline permits the bene�ts mentioned above,
there are multiple concerns with applying the idea to Service Function Chaining–lack of feature-rich functionalityof network protocols in which a handcrafted Unix applications have, the performance shortcomings by repurposing
the Unix pipe as a network channel, and the half-duplex
nature of Unix pipe.
This paper explores the applicability of Unix pipelines forservice function chaining and evaluates the idea with our
prototype implementation, called /dev/stdpkt by extending
Linux Kernel Library (LKL) [11]. The evaluation consists

AINTEC ’17, November 20–22, 2017, Bangkok, Thailandof a set of application goodput benchmarks and clari�es
its bene�ts for creating function chains with rich network
functions. Our contributions in this paper include:• The design and implementation of /dev/stdpkt toprovide network function platform with Unix pipes
and feature-rich userspace network stack.
• The benchmarks with TCP/UDP goodput measurements in practical scenarios (§ 4.2) and the durationof instantiation of network functions (§ 4.3).
• We also envision the possible use cases of function
chain under the Unix pipeline framework (§ 4.4).

2

REQUIREMENTS

Service function chaining can be divided into two components, network function and its interconnect. In a simple
case, we can use an application running on a commodity OS
over a virtual machine as a network function, and network
devices such as tap and bridge as an interconnect since we
do not face serious performance issue with this degree ofcomplexity. However, when deploying a large number of
network functions in a single physical node, there are new
set of requirements that must be achieved for decent service.
Using commodity OSes for the function chaining presents
various bene�ts of reusability of an existing application, arich set of supported network protocols, as well as the available options for development facilities such as tools andlanguages. However, those bene�ts become an obstacle (orgenerality tax [14]) when it comes to specializing the software stack such as high-performance network functions.
This was pointed out by prior work (ClickOS [9], OSv [3],MirageOS [7]) on designing of network function with the factthat the packet processing overhead of commodity OSes andboot latency are large. On the other hand, several proposals
on alternate interconnect such as VALE [13] also addressed
the issue of general-purpose operating systems and proposed
methods to completely bypass the kernel-based network
stack in order to avoid low-speed code path of packet processing. These approaches are trying to achieve high-speed
packet processing and quick boot, but at the cost of functional degradation. For instance, those proposals lack various
network functionalities such as full TCP stack with various
congestion avoidance algorithms, packet �ltering and modi�cation framework1 , packet encapsulation protocols and soon.
Our motivation is to �ll the gaps of the drawbacks of priorwork in order to satisfy the requirements of various users andservices. The following lists the set of requirements toward
our goal.
R1: Reusability of existing applications.
R2: Lightweight and quick instantiation.
1 net�lter (http://www.net�lter.org/) in Linux for instance.

Motomu Utsumi, Hajime Tazaki, and Hiroshi Esaki
Appl
i
cat
i
on
LKL
LKLs
y
s
t
em cal
l
Li
nuxk
er
nel
FS
AnonymouspipeNamedpi
pe
Binary mode
ASCII mode

NET

MM

DRI
VERS
VI
RTI
O
pi
per
x

pi
pet
x

AnonymouspipeNamedpi
pe
Binary mode
ASCII mode

Figure 1: LKL application components and its extended modules (hatched area): pipe as a network
channel.

R3: Reliable and feature-rich network functions.
R4: Development �exibility.
R5: Reasonable performance in function chaining.
The next section describes how we address the above
requirements with our prototype implementation.

3

DESIGN AND IMPLEMENTATION

We design and implement a service function chaining architecture by extending LKL. Our prototype implementation,
named /dev/stdpkt, uses LKL as a network function platform and uses a Unix pipe as an interconnect. LKL allows us
to run unmodi�ed Linux applications over Linux network
stack (originally implemented for a kernel-space program)so that features such as packet �ltering by net�lter can beeasily accomplished. This also gives us development �exibility since plenty of Linux applications can be used as-is.
Furthermore, the boot duration is reasonably quick (detailedin § 4.3) since an instantiation only involves a process invocation, which is often faster than instantiating an OS via a
generic hypervisor such as QEMU.
In this section, we describe how our design addresses requirements (§ 2) and the challenges to the design for complex
packet processing in a shell pipeline.

3.1

Challenges

Feature rich network function: To provide feature-rich
network function, we initially thought that we could extend
Linux virtual machines to run in a Unix shell and connect
each virtual machine via a pipe. However, a Linux virtual machine is heavy and takes a time to boot (four seconds in our
measurements § 4.3). This latency will spoil the shell usability and prevent us from providing multiple network function
on demand. Additionally, since the required function in a
Unix shell is only processing the packet, we do not need anyother feature such as isolation of multiple processes or user

/dev/stdpkt: Pipelined Operating System Instances in a Unix Shell AINTEC ’17, November 20–22, 2017, Bangkok, Thailand◆[DST_MAC] [SRC_MAC] [Eth_Type] XX XX XX ...
[DST_MAC] [SRC_MAC] [Eth_Type] XX XX XX ...
✓⇣◆# IF0= tap0 IF1= named pipe1|/dev/stdout \LD_PRELOAD=liblkl-hijack.so\⌘./nat-config.sh | \ # iptables -t nat
IF0= /dev/stdin|named pipe1 IF1= tap1 \Figure 2: Packet format in ASCII mode.
LD_PRELOAD=liblkl-hijack.so\./firewall-config.sh # iptables -A DROP✓
and kernel space separation. Some prior work proposed light-

weight operating systems for achieving quick boot and highperformance [9] [3]. Though using these operating systems
seems to meet our requirements (R2 and R5), these operating
systems typically provide fewer functionalities than Linux.
Function chain in a shell pipeline: Unix pipeline isprocess chaining for a data stream, each process emits data
to standard output which directs to the standard input of thenext process via a pipe character "|". The output of a program
can be used as the input of another program to trim, edit,or �lter data. And the output can pass to other commands.
While the pipeline in a shell is usually unidirectional, datagoes through bidirectionally in our situation since packet
�ow is usually bidirectional.

3.2

DesignOur design choice, to overcome the challenges, is extending
a userspace network stack derived from LKL, to harmonize
with Unix shell pipeline. Figure 1 illustrates the detail of one
LKL application component. An application linked with LKL
works as a network function while an anonymous pipe (standard input, output) and a named pipe connected between
other processes work as interconnects.
The interconnect is implemented as a typical virtio network device as other virtual network devices available in LKL.
Therefore, the extension is relatively small2 while retaining
full functionalities such as hardware o�oad or con�gurationinterfaces for LKL. The pipe-based virtual network device
has two di�erent modes of packet representation, binarymode and ASCII mode: the binary mode writes and reads thestream of raw packets to the device while the ASCII modetranslates the raw data into the text-based strings, which theformat is illustrated in Figure 2 (we use the same format as
proposed in EtherPIPE [5]). This ASCII mode is useful if we
use Unix commands such as sed, awk, or grep together withpipes.
Using a shell and pipes are enough to describe a network
topology. For instance, we can connect two LKL network
applications by a pipe, and can connect a group of LKL instances via a branch of a pipe by using tee command3 .
# LKLapp1 | tee named pipe1 | LKLapp2⇣⌘

Figure 3: Command: NAT and Firewall chaining.
LD_PRELOAD used in this example is a way to dynamically link a library at runtime.
# LKLapp3 < named pipe1
3.2.1 Feature rich network stack. Using LKL as a featurerich network stack brings other bene�ts. We can reuse various Linux applications if LKL supports application programming interface (API) used by the applications. In addition
to that, the mature development environment such as tools
and languages are also available as-is, thanks to the compatible interface of LKL with typical Linux system. Finally,
since an instantiation of network functions only involves asingle process invocation, boot time is very quick comparedto hypervisor-based virtual machines.

3.2.2 Multiple pipes for bidirectional communication. To
achieve bidirectional communication in a shell, we use anadditional named pipe since anonymous pipes are unidirectional. By writing bytes to standard output, an LKL application can transmit packets via pipes, then the next process
receives the packets at the other end of pipes from standard
input. Based on the LKL framework of network interface
backends (i.e., channels)4 , we extended LKL 1) to utilize pipes
as a network channel and 2) to support multiple network
interfaces in a single LKL application. As a result, an LKL
application can be used as a single command to cooperatewith other programs in a shell via pipes.

3.3

Usage

2 The modi�cation is around 500 LoC.

A set of commands in Figure 3 is an example service function chain with our proposal. A combination of iptables
commands allows us to create a function chain of NAT and�rewall, with the help of net�lter framework within LKL.
The command line generates a network topology illustrated
in Figure 5a: the (nat-config.sh) script contains iptables
command to con�gure a source address translation at theoutgoing interface (stdout of the �rst command) and thefirewall-config.sh script, also described with iptables
command, �lters packets received from stdin based on thecon�gured rules.

standard output.

4 Currently LKL supports tap devices, raw sockets, and Intel DPDK devices.

3 tee is the command to copy/duplicate the data from standard input toAINTEC ’17, November 20–22, 2017, Bangkok, Thailand

Motomu Utsumi, Hajime Tazaki, and Hiroshi Esaki

pe net
net
per
f pi
ser
v
er
LKL

named
pi
pe

LKL

Hos
t

(a) Topology of directly
connected
measurement.
(netperf | netserver)(b) UDP application goodput.

(c) TCP application goodput.

Figure 4: Application goodput in function of payload size of transmitted packets with the standard deviation from50 replications.

4

EVALUATIONThe evaluations consist of three parts: a packet processing
speed measurement of application goodput (§ 4.2), a boottime benchmark (§ 4.3), and a use case demonstration (§ 4.4).
The goal of our evaluation is to understand the performanceof our prototype both as network function and interconnect. We compared our implementation with other network
function implementations, OSv [3] and typical Linux virtual
machine over KVM, as well as Docker container. As for alternatives of interconnect to our pipe-based implementation,
we used Linux bridge with tap devices, and VALE [13].
VALE is a Virtual Local Ethernet which focuses on quickly
switching the packets. VALE is not suitable for our research
purpose because applications have to use netmap API toconnect to VALE. However, VALE is one of the candidates
for realizing high-performance interconnect. Thus we extended LKL to use VALE as a network channel to compare
the performance with Unix Pipe.

4.1

Experimental Setup

4.2

Packet Processing Speed

Our tests in this section were conducted on the same machineusing a single-CPU Intel i7-3770K server with four coresat 3.50GHz and 32 GB RAM. In all cases, we used Ubuntu
16.04, Linux kernel 4.4.0 for host and guest OS, netperf 2.7.05 ,iptables 1.6.1, QEMU 2.9.93, Docker 1.12.6, VALE6 , OSv 0.24425-g77b1f05, and our extended LKL7 .
Directly connected LKL applications: We measured theapplication goodput between two network function instances
directly connected via a network channel as illustrated in
Figure 4a. We used TCP_STREAM and UDP_STREAM modeof netperf to measure how fast the network function instance sends and receives packets. We changed the payload
size of netperf and measured 50 times for each payload size.
5 https://github.com/HewlettPackard/netperf

6 https://github.com/luigirizzo/netmap code downloaded at Sep 22, 20177 https://github.com/libos-nuse/lkl-linux-pipe

Table 1: Combination of instance and interconnect.
instance
LKL
LKL
LKL
Linux
OSv

interconnect
Unix pipes
bridge
VALE
VALE
VALE

legend
LKL-pipe
LKL-br
LKL-valeLinux-valeOSv-valeIn this measurement, to maximize the application goodput,
we con�gured network function instances as follows. In themeasurement with LKL and Linux, we used MTU size 60000
bytes so as not to fragment the packets, enabled checksum of�oad, TCP Segmentation O�oad (TSO), and allowed merging
receive bu�ers to reduce CPU overhead. In the measurementwith OSv, we can not change MTU size so we used 1500
bytes. Pipe bu�er size was 65536 bytes which is default sizein Linux kernel 4.4.0. We used taskset command for CPU
pinning to LKL application. We did not use the CPU pinningto Linux and OSv virtual machine because it did not improve
the goodput while LKL did. Table 1 shows the combinationof network function and interconnect we measured.
Figure 4b and 4c show the measured UDP and TCP application goodput in function of the payload size. The goodput
increases along with the increase of payload size at most of
the payload size, and Linux and OSv results outperform thatof LKL in most of the cases. Note that the reason why the
result of TCP is better than that of UDP is because TCP stack
has a segmentation-o�oad feature while UDP does not.
We observed that the choice of interconnect with LKL does
not present any signi�cant di�erences while OSv and Linuxdo. This suggests the bottleneck during packet processing
is inside LKL, not in an interconnect, thus using VALE with
LKL, in this case, does contribute to the overall performanceof LKL.
Connected via multi-hop nodes: Considering more practical use cases, we measured the network performance ofthe two network functions chaining. Figure 5a illustrates
the topology of NAT and �rewall LKL application chaining

/dev/stdpkt: Pipelined Operating System Instances in a Unix Shell AINTEC ’17, November 20–22, 2017, Bangkok, Thailand
NAT piewallpe fir
(
i
pt
abl
es
)
(
i
pt
abl
es)
net
per
f
veth0
br
0

LKL

named
pi
pe

tap0

Hos
t

LKL
tap1
br
1

net
ser
v
er
veth1

(a) Topology of chaining use case measurement. (netperf | iptables | iptables | netserver)(b) UDP application goodput.

(c) TCP application goodput.

Figure 5: Application goodput in function of payload size of transmitted packets with the standard deviation from50 replications.
Table 2: Combination of instance, interconnect, andnetwork function.
instance
LKL
LKL
LKL
LKL
Linux
Linux

interconnect
Unix pipe
Unix pipe
bridge tap
bridge tap
bridge tap
bridge tap

network function
routing x 2
NAT, �rewall
routing x 2
NAT, �rewall
routing x 2
NAT, �rewall

legend
LKL-pipe-route
LKL-pipe-nat
LKL-br-route
LKL-br-nat
Linux-br-route
Linux-br-nat

with Unix pipe network channel. We employed IP routing,
NAT, and �rewall as network functions in a chain. In thismeasurement, NAT implemented by iptables converts the
IP address of the outgoing interface. Firewall implemented
by iptables with 500 DROP rules which do not match any
packets between netperf and netserver. Other con�gurations
are the same as the previous experiment. Table 2 shows
the combination of the instance, interconnect, and network
function we measured.
Figure 5b and 5c plot the UDP and TCP application goodput in function of the payload size. The UDP application
goodput increased according to the increase of payload size.
At most of the payload sizes, chaining of two routing obtained higher goodput than chaining of NAT and �rewall:
this is as expected since the packet processing of NAT and�rewall is clearly heavier than pure IP routing.
It is worth noting that unlike the result of previous experiment (Figure 4), some of the goodput results with LKL(LKL-br-route/nat) is higher than one of Linux(Linux-br-route/nat) in both TCP and UDP: for instance,
the TCP and UDP goodput of 32K bytes packet with LKL(LKL-br-route/nat) are 5% - 40% faster than with Linux(Linux-br-route/nat). This result suggests that LKL has
a potential to process packets faster than Linux virtual machine if it works as a middlebox, not as an endpoint of communication. Although the result of Linux-pipe-route/nat
does not perform as well as Linux-br-route/nat does, ourpipe implementation can be updated with our private versionof anonymous/named pipe.

Figure 6: Boot time of each instance: the error bar indicates the standard deviation from 50 replications ofexperiment.

4.3

Boot time

In order to measure boot time, we collected elapsed time
between the start of the commands and the reception of the�rst packet from an instance at the host. We measured thistime with LKL, as well as with Docker container, Linux virtual machine, and OSv on KVM. We also measured how thenumbers of network function instances on a single machinea�ects the boot time. For this measurement, we booted alarge number of network function instances (100 instances)in sequence. We conducted this experiment for 50 times. In
this measurement, LKL, Docker, and OSv called the sleep
command after booting while Linux did not do anything
after a boot completion.
Figure 6 shows the boot time of each network function
instance with the instance ID, which we assigned to each
instance. The boot time of LKL instance was 50 - 60 msec for
all instance ID. This is the fastest type of instance–the Docker
takes 240 msec at the fastest (5 times slower than LKL), OSvboots 400 msec (8 times slower), and Linux takes four seconds(80 times slower). Note that we also observed the boot timeof OSv instances varies as the number of booted instances isincreasing. This is due to the concurrent processing load of
instances with OSv. In our measurement, single OSv instanceconsumes more CPU time than other instances at around 50

AINTEC ’17, November 20–22, 2017, Bangkok, Thailand
instances, and total CPU usage reached around 100% earlier
than the others.
Past research also conducted the similar experiment: ClickOS
boots in about 30 msec [9]. Jitsu unikernel, which is basedon MirageOS [7], takes 20 msec on x86 and 350 msec on
ARM [8]. OSv with memcached takes 600 msec to start serving requests [3]. In our measurement, the boot time of the�rst OSv instance was 400 msec. Though the boot time canvary depending on how “boot time” is de�ned and measured,
our results are comparable with the result of [3] and con�rm that LKL boots as quickly as other past research. OSv
developer [3] mentioned this boot time can be optimized
further using ramfs instead of ZFS. This optimization is also
applicable to LKL since we can change the con�guration of
LKL to support other features available in Linux.

4.4

Use case

In this section, we will demonstrate a couple of example
use cases of our proposal. A command line in Figure 7 represents port mirroring example by tee command to investigate
packet �ows. The tee command captures packets at the middle, duplicating them to the named pipe2. pktparser is ouroriginal tiny packet parser, which reads the Ethernet packet
binary stream from standard input and generates the hex
dump to standard output. By using existing commands suchas text2pcap8 and tcpdump command9 to �lter and decode
packets, it can easily inspect packets generated by applications.

◆

Motomu Utsumi, Hajime Tazaki, and Hiroshi Esaki# IF0= named pipe1|stdout
IFTYPE0= ascii
LD_PRELOAD=liblkl-hijack.so./app1-config.sh |grep --line-buffered -v 001122334455
IF0= stdin|named pipe1
IFTYPE0= ascii
LD_PRELOAD=liblkl-hijack.so./app2-config.sh✓Figure 8: Command: packet �ltering.

\
\
\
\
|\
\
\
\

⇣

⌘

Moreover, as we showed in § 4.2, LKL can provide NAT
and �rewall implemented by iptables. Naturally, we canuse any combination of network function above.

5

DISCUSSION

So far, we have focused on the functionality and its applicability of Unix pipeline to Service Function Chaining. However,through our prototype implementation and benchmarks, we
can see a couple of possible bottlenecks. The rest of thissection describes the detail of possible bottlenecks, unsuitable applications, and suitable applications of our current
prototype implementation.
Bottleneck The �rst one is LKL. As shown in § 4.2, in themeasurement of directly connected LKL applications, choice◆⇣ of interconnect with LKL has a small impact on application
goodput. However, OSv and Linux with VALE obtained better# IF0= named pipe1|stdout\
goodput than LKL with VALE. This suggests the bottleneck
LD_PRELOAD=liblkl-hijack.so\
is not the interconnect but LKL.
./app1-config.sh |\In contrast, LKL with bridge obtained better goodput
tee named pipe2 |\thanLinux with bridge in the measurement of multi-hop
IF0= stdin|named pipe1\
nodes,
Although we have not found the exact reason for this,LD_PRELOAD=liblkl-hijack.so\onepossiblereason is position of LKL instances in packet./app2-config.sh;communication.
In the measurement of directly connected# ./pktparser < named pipe2 |\LKL
applications,
we executed userland application which
text2pcap - - |\sends/receives
packets
and LKL instances work as an endtcpdump -r -;point,
whileinthemeasurementof multi-hop nodes, packet✓⌘
handling was done in kernel space and LKL instances work
Figure 7: Command: port mirroring.
as a middlebox. Thus overhead of packet handling in LKL
would depend on the function of LKL instance. As future
work, we would pro�le more deeply and explore the possiA command line in Figure 8 conducts packet �ltering
bility of better performance with LKL.
by grep command. In this case, grep command blocks theUnix pipes are the next bottleneck. We are using pipes forpacket whose source or destination mac address isinterconnects
without any modi�cations, thus any packets‘00:11:22:33:44:55’.
passedthrough
LKL via pipes are copied at least twice: when
8 text2pcap is the command to convert ASCII hex dump to packet capture
the
left
LKL
application
sends a packet, it copies packet data
format.
9 tcpdump is the command to read packet capture format and print out afrom user space to kernel space during a write(2) systemcall of the pipe, and then copied again at the right application
description of the contents of packet.

/dev/stdpkt: Pipelined Operating System Instances in a Unix Shell AINTEC ’17, November 20–22, 2017, Bangkok, Thailand
Table 3: Characteristics of various Service Function Chaining architectures.
R1:reusability
EtherPIPE [5]OpenNetVM [1]ClickOS [9]Jitsu [8]OSv [3]/dev/stdpkt (this paper)

++
+
+
++

R2: boot
latency
+
+
+
+
+
+

during a read(2) system call. Those memory copies can be
avoided if multiple processes share the memory resource byimplementing a private pipe which reduces the number ofdata copies.
Another possible bottleneck is process scheduling. Pipeand named pipe have a writer and the reader processes, andevery pipe has a bu�er, so if the writer process writes fasterthan the reader, the writer cannot write to the pipe until thereader reads the packet. This bottleneck would be solved byintroducing a process scheduler which consider the bu�eroccupancy. We plan to analyze this internal behavior moredeeply in future work.
Unsuitable application. The current implementation ofpipes is not dynamically con�gurable, we have to design theconnectivity in advance and then build the chains. Therefore, it is di�cult to apply our proposal when chains are
dynamically updated or chaining path depends on each �ow.
Extending the current prototype implementation to accept
new channel creation would address this.
Suitable application. LKL application boots quickly (lessthan 70 msec). Therefore, this idea is suitable for a short-lived
network function, we can deploy and destroy LKL network
functions in an instant on-demand. This enables operators
to use resources e�ciently.

6

RELATED WORK

We have explored both network function and interconnect
in previous work. However, this work does not satisfy therequirements we presented in § 2. Table 3 shows the characteristics of prior work. The rest of this section describes thedetail of related work.
EtherPIPE [5] proposes a character device driver for a
network interface card to utilize Unix commands for packet
processing. From the context of Service Function Chaining,
this research uses Unix commands as network functions andUnix pipes as interconnect. Their motivation came from thefact that Unix commands (cat, grep, sed, awk) are su�cientfor simple packet processing (R1). The technique of processing packets on UNIX shell is similar to our research. However,the network functions o�ered by EtherPIPE are limited to

R3: featurerichness–++++

R4: development R5: chaining�exibility
performance
++
++
+
N/A
+
N/A
++
+

simple packet processing (R3), while we often need more
complicated packet processing at the middlebox such as connection tracking and NAT with a stateful inspection. Theirachieved goodput was upto 1.5 Gbps with �ve MAC address�ltering rules at 64 byte frame size. Their implementatin
was a proof of concept and performance was future workfor their research (R5).
NetVM [1] is a platform for running network function on
KVM. OpenNetVM [15] is based on NetVM architecture andrun the network function as a native process or a process inside Docker containers to realize lightweight, quick boot, andisolation (R2). In their proposal, network function is actually
the callback function written in C language. Interconnect isshared memory with Intel’s Data Plane Development Kit [2].
Network function manager provides zero-copy packets routing between network functions. As a result, OpenNetVM
achieved 40 Gbps with 5 network function chaining (R5).
However, we cannot run existing Linux application on theirplatform (R1). If we want to implement the complicated network application such as stateful �rewall or proxy which
terminates a TCP connection, it requires a reasonable amountof re-implementation e�ort and it is hard to provide equalstability of commodity OS (R3).
ClickOS [9] is a platform for middlebox processing basedon Xen hypervisor. They choose Click modular router [4] asa programming abstraction of middlebox packet processing(R1). In their proposal, network function is minimal operating system optimized for running Click and interconnect
is ClickOS Switch which is based on VALE [13] and sharedmemory with Xen Memory Grant. They achieved quick boot(about 30 msec) and 21.7 Gbps throughput with two directly
connected network function (R2, R5). However, if we needa new module, we have to develop within Click framework(R4). Moreover, unlike LKL which has full Linux kernel functionality, ClickOS does not have a complete TCP stack (R3).
Jitsu [8] proposed architecture for providing an application on demand with MirageOS [7], type-safe unikernel. MirageOS can be used as a network function and Jitsu showed
the possibility of provisioning a required application withquick boot operating system on demand, indicating boot

AINTEC ’17, November 20–22, 2017, Bangkok, Thailand
latencies of 350ms on ARM and 30ms on X86 (R2). However,MirageOS focuses on running OCaml application. Therefore,
we cannot use existing applications and other languages todevelop (R1, R4).
OSv [3] proposed new operating system designed especially for virtual machine environment. They redesigned
many functions including network stack and achieved 24%25% higher performance of netperf TCP STREAM (singlestream throughput) than Linux virtual machine at the expense of functionalities [3]. Furthermore, OSv achieved fastboot, 600 ms (R2). Unlike other research and similar to our
research, one of their goals is running existing Linux executables by allowing application to call Linux ABI (R1). However,
LKL can support many more applications which are written for Linux because LKL provides equivalent functionality
such as net�lter (R3).

7

CONCLUSION AND FUTURE WORK8ACKNOWLEDGMENTS

This paper is the �rst attempt to design a Service Function
Chaining by applying Unix pipeline in a standard Unix shell.
We have proven the applicability of the Unix pipeline forService Function Chaining by showing performance benchmarks and use cases. We have extended LKL to use Unix pipes
as network channel by taking several challenges to overcome poor features of packet processing in Unix shell, andhalf-duplex nature of Unix pipe. Though optimizations are
required, directly connected LKL applications reach 8Gbps
goodput with UDP and 6.5 Gbps goodput with TCP at most.
Furthermore, NAT and �rewall LKL application chains obtain 10.5 Gbps goodput with UDP and 5.6 Gbps goodput
with TCP at most. The quick boot time of LKL application(-70 msec) allows us on-demand instantiation for resourcee�cient operations.
A couple of future directions are possible: 1) service chaining �ows in our proposal are �xed thus not able to dynamically update upon each network �ow request. This shallbe addressed by extending the current prototype to accept
new channel creation. 2) the base performance shown in thispaper suggests optimization to several components of our
prototype implementation. Our observations on the possible
bottleneck are listed and will be analyzed to provide moredetailed performance pro�ling in our future work.
The authors thank to Matthieu Coudron for his commentson this work and Zachary S. Bischof for his proofreading on
this paper.

REFERENCES[1] H����, J., R�����������, K. K., ��� W���, T. NetVM: High performance and �exible networking using virtualization on commodity
platforms. In USENIX NSDI’14 (Berkeley, CA, USA, 2014), pp. 445–458.

Motomu Utsumi, Hajime Tazaki, and Hiroshi Esaki[2] I����. Intel DPDK: Data Plane Development Kit. http://dpdk.org.
(Accessed Sep 26th 2017).
[3] K�����, A., �� ��. OSv—Optimizing the Operating System for Virtual
Machines. In USENIX ATC ’14 (June 2014), pp. 61–72.
[4] K�����, E., M�����, R., C���, B., J�������, J., ��� K�������, M. F.
The click modular router. ACM Trans. Comput. Syst. 18, 3 (Aug. 2000),
263–297.
[5] K���, Y., M������, T., H�������, H., C��, K., M����, R. V., ���N�������, O. A packet I/O architecture for shell script-based packet
processing. China Communications 11, 2 (Feb 2014), 1–11.
[6] L�����������, B. THE UNIX ORAL HISTORY PROJECT . http://www.
princeton.edu/~hos/Mahoney/expotape.htm. (Accessed Jun 2nd 2017).
[7] M�����������, A., �� ��. Unikernels: Library operating systems forthe cloud. In ACM ASPLOS ’13 (New York, NY, USA, 2013), pp. 461–472.
[8] M�����������, A., �� ��. Jitsu: Just-in-time summoning of unikernels. In USENIX NSDI’15 (Oakland, CA, May 2015), pp. 559–573.
[9] M������, J., A����, M., R�����, C., O������, V., H����, M., B������,R., ��� H����, F. Clickos and the art of network function virtualization.
In USENIX NSDI’14 (Berkeley, CA, USA, 2014), pp. 459–473.
[10] P�������, D. L., ��� W�����������, P. The Organization of Networks in Plan 9. In USENIX Winter. (1993).
[11] P������, O., G�������, L. A., ��� T����, N. R. I. C. R. . �. LKL: The
Linux kernel library. In RoEduNet, 2010 (2010), pp. 328–333.
[12] R������, E. S. The art of Unix programming. Addison-Wesley Professional, 2003, ch. Basics of the Unix Philosophy.
[13] R����, L., ��� L�������, G. VALE, a Switched Ethernet for Virtual
Machines. In ACM CoNEXT ’12 (New York, NY, USA, 2012), pp. 61–72.
[14] W�������, D., ��� K�����, R. Unikernel monitors: Extending minimalism outside of the box. In USENIX HotCloud ’16 (2016).
[15] Z����, W., �� ��. OpenNetVM: A Platform for High Performance
Network Service Chains. In ACM HotMIddlebox ’16 (2016), pp. 26–31.