CPqD/RouteFlow — pull request #39 by joestringer (2013-04-25 10:36:39+00:00)
Title: Vandervecken Synchronisation

Vandervecken
==========

There have been several branches of development by those involved in the Vandervecken project over the past six months, which have provided additional features such as ISL support, IPv6, MPLS, new controllers, new script for simplifying build/installation, and a whole host of bugfixes from the CARDIGAN pilot deployment. Read on for further details.

The below features have been tested in two environments: Ubuntu 11.10 and 12.04, under the POX, NOX (11.10-only) and Ryu controllers. NOX and Ryu were tested with Pica8 hardware as datapaths and interoperated using OF1.0 and OF1.2. POX was tested in the rftest environments using Open vSwitch instances as datapaths.

ISL
----

Inter-Switch Link (ISL) support allows multiple datapaths to be managed by a single VM. rfserver has a new parameter to specify how these datapaths connect to each other.
```
$ rfserver/rfserver.py -h
usage: rfserver.py [-h] [-i ISLCONFIG] configfile
...
-i ISLCONFIG, --islconfig ISLCONFIG
ISL mapping configuration file
...
```
This parameter is optional. Here is an example file:
```
vm_id,ct_id,dp_id,dp_port,eth_addr,rem_ct,rem_id,rem_port,rem_eth_addr
12A0A0A0A0A0,0,0x99,2,66:FD:B5:BE:39:77,0,0x9a,2,6E:71:60:A9:B9:8D
```
This associates datapaths 0x99 and 0x9A with VM 12A0A0A0A0A0, specifying that there is a connection between port 2 of DP0x99 and Port 2 of DP0x9A. Both are managed by Controller CTID=0. The ethernet addresses as specified are used for communication on the ISL. You will then need to connect these ports on the datapaths to allow this to function correctly.

build.sh
-----------

This is a script implemented as per the discussion at https://github.com/CPqD/RouteFlow/issues/19.

`build.sh` can be independently downloaded and used to fetch and build RouteFlow and dependencies. Its only requirement is `git`. It has support for:
- Fetch RouteFlow from git
- Fetch dependencies via apt-get
- MongoDB and Open vSwitch via git or source release on website
- Fetch nox/ryu rfproxy implementations from git (currently my repository, pending a shift to the RouteFlow organisation)
- Building all of this.
- Detects the version of Ubuntu that is running and attempts to make recommendations about software versions
- Ubuntu 12.04+ will not work with NOX
- Ryu should be used with OVS-1.10+.

FPM
-------
Support for a Forwarding Path Manager (FPM) is a new feature in Quagga. Vandervecken work includes support for the initial version of the FPM protocol that is part of Quagga `0.9.22`. This allows for IPv4 and IPv6 routes to be shared.

We also include experimental support for sharing Next-Hop Label-Forwarding Entries (NHLFE); This allows labels to be pushed, popped and swapped. There is not currently support for Forwarding Equivalence Class To NHLFE (FTN), but this should be reasonably straight forward to implement.

FPM works with either Quagga (for traditional routes), or with FIMSIM (https://github.com/ofisher/FIMSim) for traditional routes or label manipulation.

To build RouteFlow with FPM support, edit `rfclient/Makefile` and uncomment the following line:
```
# CFLAGS += -DFPM_ENABLED -I../../quagga-fpm/fpm/
```

Gateway Resolution Refactor
------------------------------------------

IPv4 and IPv6 Routes from FPM or NetLink are now cached before they are sent to datapaths. This allows RouteFlow to handle routes with valid gateways as they are received, and queue other routes until their gateways become resolvable. This refactor added thread-safety to many of the `FlowTable.cc` structures, however it is not comprehensive in this regard. Further analysis of thread-safety should be conducted.

The RTPolling thread now runs a tight loop of gathering NetLink messages, then performing minimal computation to queue the route for GWResolver.

GWResolver is a new thread that sits in the background and does the following:
1) Take a route off the queue
2) Attempt to resolve the destination gateway for the route (*)
3) If the gateway can be resolved, try to send the route to the datapath via RFServer. If it cannot be resolved, place at the end of the queue
4) If the route is sent, update the `FlowTable.cc` structures to reflect the new datapath state. If the route failed to send for some reason, re-queue.

(*) This involves initiating a connection, then adding the socket to a "pendingRoutes" list. It will remain on this list until the kernel sends a host notification message for the given gateway. At that point, it will be removed from pendingRoutes, the socket is closed, and the host is added to the hostTable.

IPv6
------
This is a work in progress; much of the work is done, with the major exception that FlowTable.cc does not allow multiple IP addresses per interface.

RFDB.py
-------------

`rfdb.py` is a simple script to query MongoDB for messages on a given channel -- `rfclient` for the rfclient<->rfserver channel, or `rfproxy` for the rfserver<->rfproxy channel. It prints these in a human-readable format.

Patchlist
-------------

- Remove FlowMod,RouteInfo,DatapathConfig and replace with RouteMod (6 patches)
- FlowTable: Separate gateway resolution from kernel pollers. This no longer blocks, and re-queues routes that have invalid or unreachable gateways. (6 patches)
- Threading and style fixes, including adding several mutexes to guard critical code sections. (11 patches)
- Simplify and refactor some of FlowTable's code to reduce code complexity. (6 patches)
- Make RFClient report local MAC addresses to RFServer. RFServer then determines how routes should match. This is groundwork for ISL. (2 patches)
- Tidy up route IP address handling and logging (7 patches)
- Add initial IPv6 support, only single-stack. (7 patches)
- Thread safety and code cleanups (2 patches)
- Add support for Quagga's FPMv1 protocol, and add NHLFE support; no FTN yet. (3 patches)
- Set route priority lower than control traffic priority---Fixes bugs with default routes. (1 patch)
- RFLib and RouteMod refactors and tidyups for initial Ryu support (4 patches)
- Add dependency fetch/build scripts (5 patches)
- Add database diagnostic tool (1 patch)
- Further code tidy-ups and script fixes (10 patches)
- Fix infrequent crashes (3 patches)
- Update documentation (3 patches)
- Add ISL support (1 patch)
- Add Mininet2 topology script

Caveats (Future work)
-------------------------------
- This patchset breaks RFWeb for listing IPC messages sent between RouteFlow components.
- FPM NHLFE entries are not cached like the NetLink routes are.