datapath: Introducing 'ovdk_pipeline'

'ovdk_pipeline' implements an adapter for 'rte_pipeline'. 'rte_pipeline' is
an object in the DPDK packet framework:

    http://www.dpdk.org/ml/archives/dev/2014-June/003059.html

Each 'rte_pipeline' is wrapped by an 'ovdk_pipeline'. We use one
'rte_pipeline' per core in order to process packets for that core.

ovdk_pipeline_run() is routinely called in order to
process packets received by that pipeline.

The use of 'rte_pipeline' will:

1. Allow the vswitch to quickly take advantage of features developed
by DPDK for the packet framework

2. Allow easier customization of the dataplane

3. Enable hardware offload of some functionality to improve performance

Change-Id: I1b303eb466f7c187262c365df8a3d65844420caa
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Introducing 'ovdk_flow'

'ovdk_flow.c' provides the functionality required to extract
a 'struct ovdk_flow_key' from a packet and other related
helper functions.

Change-Id: I9d0df80262c4e3bf930f1a6ba7915ed29b83a04c
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Introducing 'ovdk_action'

'ovdk_action.c' contains any functions required in order
to execute actions on packets. For example, push a
vlan tag to a packet or modify the ethernet header.

For the most part, these functions use functionality
provided by the vswitchd. However a conversion between
the type 'struct rte_mbuf' and the type 'struct ofpbuf'
is required which is achieved using 'ofpbuf_helper.c'

Change-Id: I7b2c5a12f96c762248d5c191154f4bd2a6122d59
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Introducing 'ovdk_hash'

'ovdk_hash.c' provides the hash implementation that
is used by the packet framework when doing a lookup.

At the moment, the CRC32 instruction is used via the
rte_hash_crc() function.

Change-Id: I8cf75d996c2337241f94cf8bc0caabd0f80800db
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Introducing 'ovdk_mempools'

'ovdk_mempools' sets up the buffer pools used by the
vswitch. These buffer pools are also used by guest
applications via free and alloc rings when using the
ivshmem guest access method.

There are now two buffer types: Control and Packet.

Packet buffers contain actual packet data whereas control
buffers are used to send control messages to the datapath.
For example, they are used to set up flows and ports. These
control buffers should not be handled by an 'rte_pipeline'
but should be terminated in 'ovdk_datapath'.

Seperation of these pools avoid some nasty errors caused
by misinterpretation of the mbuf type.

Change-Id: I1a990a74a95aae3e186c7b899ba7c099a460983f
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Introducing 'ovdk_datapath'

'ovdk_datapath' handles messages received from the vswitchd. Each  message is
contained in an mbuf.

Communication between the vswitchd and 'ovdk_datapath' is achieved through
four rings:

 - request ring: Messages sent from the vswitchd to the datapath
 - reply ring:   Messages sent from the datapath to the vswitchd
 - free ring:    Mbufs sent by the vswitchd to be freed by the datapath
 - alloc ring:   Mbufs allocated by the datapath to be used by the vswitchd

'ovdk_datapath' has the responsibility of parsing each message received from
the request ring. When the message is parsed, 'ovdk_datapath' will call the
relevant functions in 'ovdk_pipeline' in order to configure the 'ovdk_pipeline'
in response to the message received.

'ovdk_datapath' also has the responsibility of allocating new mbufs from the
pool and sending those mbufs to the vswitchd. The vswitchd can then dequeue
these mbufs when it needs mbufs to send to the datapath. 'ovdk_datapath' has
the responsibility of freeing mbufs received from the free ring. This is
required as the vswitchd does not use DPDK threads. This avoids the situation
whereby when using mbuf free and alloc functions in a standard pthread,
memory corruption can occur as pthreads may be sharing mbuf caches.

Users of 'ovdk_datapath' must call ovdk_datapath_handle_vswitchd_cmd()
periodically in order to handle messages from vswitchd.

Change-Id: I7c30d90d1bc143b54b82d0bd00aa6a4260b80582
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Introducing 'ovdk_stats'

'ovdk_stats' provides functions to keep track of
vport statistics in a performant manner.

Change-Id: I4485f7db43edba4b947807d1c8aa2a0feebbe029
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Add infrastructure for different port types

Each port associated with the datapath is called a vport and
has an individual unique index called the 'vportid'. Currently,
this is equivalent to the concept of an Open vSwitch odp port
number.

Unlike previous versions of the vswitch, the ranges of these
numbers can be arbitrarily defined at compile time.
'ovdk_vport_types.h' defines these ranges and as such, needs to
be shared with the vswitchd.

At initialization time, resources for all vports must be
allocated. Then, as the vswitchd adds ports these vports
get added to the rte_pipeline. Each vport is described
in a memzone which can be shared to a guest if required. The
data in this memzone is described in 'ovdk_vport_info.h' and
can be accessed by ovs_vport.

If a port works with the DPDK packet framework, it should be
reasonably easy to enable this port with the vswitch.

Change-Id: Ie45378238cc401343ac850d05f9b8eb13e87adda
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Implement packet framework ethdev ports

The packet framework provides an abstract interface
'rte_port' that can be implemented in order to interface
an 'rte_pipeline' with a network device. This network device
can be a physical or virtual device.

An implementation of this interface for ethdev ports
is provided in DPDK. This patch allows for these ethdev
ports to be added to an 'ovdk_pipeline'. It is mainly
concerned with initialization of the ethernet devices
and population of structures required by 'ovdk_pipeline'

Change-Id: I801e60fb06ee3c76d39838648dd28613a284b345
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Implement packet framework IVSHMEM client ports

The packet framework provides an abstract interface
'rte_port' that can be implemented in order to interface
an 'rte_pipeline' with a network device. This network device
can be a physical or virtual device.

This patch introduces a new implementation of 'rte_port'
for IVSHMEM devices. This patch also enables these
ports to be added to an 'ovdk_pipeline'.

This patch initializes all the resources required for the
ports at start up. It also populates structures required
by 'ovdk_pipeline'

Change-Id: I4b9abd608bc31ba19d498e9870d01881d2e3c89d
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Implement packet framework userspace vhost ports

The packet framework provides an abstract interface
'rte_port' that can be implemented in order to interface
an 'rte_pipeline' with a network device. This network device
can be a physical or virtual device.

This patch introduces a new implementation of 'rte_port'
for user-space vhost devices. This patch also enables these
ports to be added to an 'ovdk_pipeline'.

This patch initializes all the resources required for the
ports at start up. It also populates structures required
by 'ovdk_pipeline'

Change-Id: I76f4f5a77ba0bd67ae82a0e5f49c9ceec6934742
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Implement packet framework bridge ports

Packet framework provides an abstract interface
'rte_port' that can be implemented in order to interface
an 'rte_pipeline' with a network device. This network device
can be a physical or virtual device.

An implementation of this interface for sink and source ports
is provided in DPDK.

This patch adds these sink/source ports as bridge ports to an
'ovdk_pipeline'. It is mainly concerned with population of
structures required by 'ovdk_pipeline'.

In standard Open vSwitch, bridge ports provide a standard
network interface on the host for each bridge port (e.g.
br0, br1). This interface works like any normal Linux
interface (i.e. can be viewed with ifconfig, can accept
'ping' requests, etc). This functionality is not available
with this implementation of a bridge port.

Change-Id: Iabf0dcc36409a18b85f33aab9ad0984f95ced1e3
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
datapath: Introducing 'ovs-dpdk'

'ovs-dpdk' is a user space DPDK application that implements
a fast dataplane using the Packet Framework. This dataplane
can be controlled using control messages sent via DPDK rings.

Each lcore runs an instantiation of an 'ovdk_datapath' and an
'ovdk_pipeline'.

Unlike previous versions of the vswitch, this implementation
allows the datapath to scale down to one DPDK lcore or up
to an arbitrary number of lcores by specifying a core mask
when starting the 'ovs-dpdk' application.

Work is distributed between lcores at the granularity of an
input port. Therefore, if you are using two ports and three
cores, one core is effectively not in use. Currently, when
adding ports to the switch, they are added to cores in a
round robin manner. This is different to the previous versions
of the vswitch as one core was dedicated to switching
virtual machine ports and 2 were dedicated to physical ports.
This created a bottleneck for some use cases.

Stats can be defined to start on an arbitrary port defined
at the command line.

Change-Id: I3e6fba9f9da96d7b8d17677c7545176ca39d9db0
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
dpif-dpdk: Introducing 'dpif-dpdk-flow-table'

'dpif-dpdk-flow-table.c' implements a flow table in
the vswitchd used to keep track of flows in the datapath.
Currently, this uses a simple array table. Future work
should modify this to improve look up performance.

As 'ovs-dpctl' does not use the vswitchd the state of this
table needs to be stored in a memory mapped file
so that it can be persistent across invocations of
'ovs-dpctl'.

Change-Id: I419f9c42ea45440022a35d7b6b9f5ae687a976d4
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
dpif-dpdk: Introducing 'dpif-dpdk-vport-table'

'dpif-dpdk-vport-table.c' implements a table in
the vswitchd used to keep track of vports in the datapath.
In particular, it needs to track to which lcore a
vport has been added to.

Currently, this uses a simple array table. Future work
should modify this to improve look up performance.

As 'ovs-dpctl' does not use the vswitchd the state of this
table needs to be stored in a memory mapped file
so that it can be persistent across invocations of
'ovs-dpctl'.

Change-Id: Ia8d34626bfe345d96690571d0296caddf8037799
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
guest: Remove 'kni_client'

As 'kni_client' is currenly not supported, remove
related code.

Change-Id: I6f9daa27e6c03155055d71128df04b2d980a56cd
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
OFTest: Remove oftest results

As host kni (vEth) ports are not currently supported,
remove the OFTest results file.

Change-Id: Iabb0ab8c9222aedd4abb3431b671d02b0f5fcf0b
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
vswitchd: Modify 'dpif-dpdk' to use packet framework enabled datapath

There is no easy way to do this and maintain a bisectable
code base as this change touches most aspects of the 'dpif-dpdk'
implementation. As a result, this patch is large but maintains
a bisectable history.

This removes the non packet framework datapath and
replaces it with the modified datapath using packet framework.
It also modifies 'dpif-dpdk' to use this modified datapath. In
summary, this patch enables 'ovs-vswitchd' to interface with
'ovs-dpdk'.

The main changes made to enable this are:

- Modify 'dpif-dpdk' to use the updated message structures for
communication with 'ovs-dpdk'
- Use the 'dpif-dpdk-vport-table' and 'dpif-dpdk-flow-table' to
track flows and vports added to ovs-dpdk
- Add logic to detect the number of lcores in the datapath
- Add code to distribute vports between lcores. A vport is
added as an in-port to one lcore but must be added as an out-port
to every core. This is because any lcore can output to any port.
- Update 'ovs-dpctl' to work with 'ovs-dpdk'
- Update the 'ovs_client' guest application
- Modify code to compile against DPDK 1.7.0
- Update all the unit tests to work with packet framework
enabled datapath.
- Remove any unit tests that are no longer relevant

Change-Id: I53b203113263de6d3bd765972641bea46ee62308
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
docs: Update documentation for Packet Framework changes

As the vswitch now uses the Packet Framework in the datapath,
this patch modifies the documentation accordingly.

Change-Id: I4ca1587a34b4d198d23fd35066ed952f84fcb391
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
Signed-off-by: Kevin Traynor <kevin.traynor@intel.com>
Signed-off-by: Eimear Morrissey <eimear.morrissey@intel.com>
Signed-off-by: Meghan Halton <meghan.halton@intel.com>
Signed-off-by: Mark B. Kavanagh <mark.b.kavanagh@intel.com>
Signed-off-by: Stephen Finucane <stephen.finucane@intel.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Signed-off-by: Mark D. Gray <mark.d.gray@intel.com>
ovs-dpdk: resolve stats_core bug

Currently, the 'stats_core' parameter is never parsed, as it shares a
common prefix with the 'stats' parameter - consequently stats
cannot be displayed. Change the name of the 'stats' parameter to
resolve this issue.

Change-Id: I9e731264738766ead6909ac925d35ea686e861f7
Signed-off-by: Mark Kavanagh <mark.b.kavanagh@intel.com>
Reviewed-By: Mark Gray <mark.d.gray@intel.com>
Reviewed-By: Ian Stokes <ian.stokes@intel.com>