Package: ptf
PTF Packet Testing FrameworkIntroductionPTF is a Python based dataplane test framework. It is based on unittest, which
is included in the standard Python distribution.This document is meant to provide an introduction to the framework, discuss the
basics of running tests and to provide examples of how to add tests.Most of the code was taken from theOFTest
framework. However, PTF focuses on the
dataplane and is independent of OpenFlow. We also added severalnew features.ContributingBefore you start pushing new changes to this repository, you should notice
that the entiresrc/code is automatically formatted with Black.
Our GitHub Action pipeline will verify that code is correctly
formatted and fail if not.Two separate targets in makefile were prepared to make our work easier.
If you want to run a check, typemake format-check, but if you want to
reformat your code, please usemake format.Blackis listed in therequirements-dev.txt. To install it locally, you
can usemake set-devorpip install -r requirements-dev.txt.
More information about Black, you find atBlack's GitHub PageLonger StartDependenciesThe following software is required to run PTF:Python 3.xScapy 2.4.5 (unless you provide another packet manipulation module)pypcap (optional - VLAN tests will fail without this)tcpdump (optional - Scapy will complain if it's missing)Root/sudo privilege is required on the host, in order to runptf.The default packet manipulator tool forptfisScapy. To install it use:pip install scapy==2.4.5To enable VLAN tests, you need to installpypcap:pip install pypcapFor developer purpose, you should installrequirements-dev.txtwith:pip install -r requirements-dev.txtThetcpdumpis optional, but to install it use:# on CentOS
yum install tcpdump

# on Debian base
apt-get install tcpdumpRun PTFOnce you have written tests and your switch is running, you can run 'ptf'. Use--helpto see command line switches.For example:sudo ./ptf --test-dir mytests/ --pypath $PWD \
	 --interface 0@veth1 --interface 1@veth3 --interface 2@veth5 \
	 --interface 3@veth7 --interface 4@veth9 --interface 5@veth11 \
	 --interface 6@veth13 --interface 7@veth15 --interface 8@veth17This will run all the tests included in themytestsdirectory. The--pypathoption can be used to easily add directories to the Python PATH. This is useful
if you use a Python interface to configure your data plane (as part of your
tests). The--interfaceoption (or-i) can be used to specify the interfaces
on which to inject packets (along with the corresponding port number).Install PTFPTF can be installed withpip:# Install the latest versionpipinstallptf# Install specific versionpipinstallptf==0.9.1You can also install a local copy of PTF withpip install ..Writing tests for your switchTake a look at theexampledirectory. This is not a working example as it is
(the switch is not included), but it will show you how to write tests. This
directory contains the following:run_client.sh: a wrapper aroundptfswitch_sai_thrift: empty directory, this is where the Python bindings to
program the switch's control plane would be copiedmytests/sai_base_test.py: a wrapper Python class around PTF's BaseTest
class. It is the base class for all the tests we added tomytests/switch.pymytests/switch.py: some example testsRunning the exampleIf you want to run the example, you will need to obtainp4factory. For the following, I will
assume that you cloned the repository and installed the dependencies. I will
assume that environment variableP4FACTORYcontains the path to the cloned
repository.First, you need to create the required veths:cd $P4FACTORY/tools/
sudo ./veth_setup.shThe next step is to compile the target switch and to run it:cd $P4FACTORY/targets/switch/
make bm-switchsai
sudo ./behavioral-modelFinally, you can run the example tests:cd <ptf-dir>/example/
sudo ../ptf --test-dir mytests/ \
	 --pypath $P4FACTORY/targets/switch/tests/pd_thrift/
	 --interface 0@veth1 --interface 1@veth3 --interface 2@veth5 \
	 --interface 3@veth7 --interface 4@veth9 --interface 5@veth11 \
	 --interface 6@veth13 --interface 7@veth15 --interface 8@veth17New featuresWe added the following features to the base OFTest framework:FiltersThey can be used to discard some of the packets received from the switch. Take a
look atsai_base_test.pyfor an example. You
will see the following codetestutils.add_filter(testutils.not_ipv6_filter)which tells PTF to discard received IPv6 packets. You can add your own filters
(they have to be callable Python objects which take a Scapy packet as input).Ternary matchingA PTF test -just like an OFTest test- matches the received packets against
expected packets. This is an exact match. However, sometimes one does not care
about all the fields in the packets. PTF introduces the Mask class which lets
you specified some field you do not care about when performing the match. For
example:import mask
m = mask.Mask(expected_pkt)
m.set_do_not_care_scapy(IP, 'ttl')
verify_packets(<test>, m, <port list>)Test timeoutA timeout for test cases can be specified using the--test-case-timeoutcommand line option. This timeout must be expressed in seconds. A timeout of 0
is the same as no timeout (the default). If the timeout expires before the test
is done executing, an exception will be raised and the test counts as an
error. A timeout can also be specified for each individual test case, using the@testtimeoutdecorator, which needs to be imported fromptf.testutils. This
timeout takes precedence over the global timeout passed on the command line.Pluggable packet manipulation moduleBy default,ptfusesScapyas the packet manipulation module, but it can
also operate on a different one.Such modulemust define/implement the same symbols, as defined inScapyimplementation of packet. Most of them are just names of most common frame
headers (Ether, IP, TCP, UDP, ...).The default implementation can be found in file/src/ptf/packet_scapy.py. It can be used as a
reference when implementing your own version.To use another packet manipulation module, one needs to
provide it as argument-pmmor--packet-manipulation-modulewhen running theptfbinary.sudo ./ptf <other parameters> -pmm foo.packet_fooPlease make sure that this module is loaded into the runtime before running
any tests.ShardingYou can achieve parallelization by splitting tests into N groups and running them with separate PTF processes.
Each PTF instance will run disjoint subset of all selected tests.For example to run specific set of tests across 3 PTF instances:$ ssh mynode0 sudo ./ptf --test-dir mytests --num-shards 3 --shard-id 0 all ^other &
$ ssh mynode1 sudo ./ptf --test-dir mytests --num-shards 3 --shard-id 1 all ^other &
$ ssh mynode2 sudo ./ptf --test-dir mytests --num-shards 3 --shard-id 2 all ^other &Configuring PTFPlatformsThe "platform" is a configuration file (written in Python) that tells PTF how to
send packets to and receive packets from the dataplane of the switch.ethThe default platform,eth, uses Linux Ethernet interfaces and is configured
with the-ioption (or--interface). Pass the option as-i ofport@interface, for example-i 1@eth1. If no-ioptions are given the the
default configuration uses vEths.remoteAnother common platform,remote, provides support for testing of switches on a
different host. This can be useful for cases where interfaces are not available
on one host (i.e. they're not bound to a Linux interface driver) or where PTF
cannot run on the same host (unsupported OS, missing software, etc.).This can be enable by modifying theplatforms/remote.pyfile to point to 4
NICs on the host running PTF, like so:remote_port_map = {
    (0, 23) : "eth2", # port 23 of device 0 is connected to physical port on the server eth2
    (0, 24) : "eth3", # port 24 of device 0 is connected to physical port on the server eth3
    (0, 25) : "eth4",
    (0, 26) : "eth5"
}nnWe introduce a new platform,nn, which uses [nanomsg] (http://nanomsg.org/) to
send and receive packet to the switch. We support IPC and TCP nanomsg
sockets. When using this platform, you need to make sure that the Python package
[nnpy] (https://github.com/nanomsg/nnpy) is installed. Withnn, do not use--interface, instead use--device-socket. For each device, you need to
provide a list of enabled ports and a nanomsg socket address. For example:--device-socket 0-{1,2,5-8}@ipc:///tmp/bmv2_packets_1.ipcThis command will enable ports 1, 2, 5, 6, 7, 8 on device 0. Packets for device
0 will be captured and send on IPC socketipc:///tmp/bmv2_packets_1.ipc.Passing Parameters to TestsThere is a facility for passing test-specific parameters into tests that works as follows. On the command line, give the parameter--test-params="key1=17;key2=True"You can then access these parameters in your tests' Pyhton code using the
following code:import ptf.testutils as testutils
# Returns a dictionary which includes all your parameters
test_params = testutils.test_params_get()
# Returns the value of the parameter "param", or None if not found
param_value = testutils.test_param_get("param")Take a look atsai_base_test.pyfor an
example.Grouping Tests togetherIt is very easy to create groups of tests, using the providedgroupPython
decorator. Simply decorate your test with@group(<name of group>).Take a look atswitch.pyfor an example.One given test can belong to several groups. You can choose to run only the
tests belonging to a given group using a command like this one:sudo ./ptf --test-dir mytests/ --pypath $PWD <name of group>We also provide a convenientdisableddecorator for tests.Support for multidevice testsThe original OFTest was meant to unit test a single OF-compliant switch. With
PTF, we tried to add support for testing a network of several devices. If you do
not intend to use this multi-device feature, you do not need to worry about it,
it should not impact you. If you want to leverage this feature, here is what you
need to do:when adding interfaces, instead of writing<port_number>@<interface_name>,
you need to write<device_number>-<port_number>@<interface_name>when sending a packet, the port number becomes a tuple (device, port):send_packet(self, (<device_number>, <port_number>), pkt)theverify_*functions where also updated to include device information. For
example:verify_packets(self, pkt, device_number=<device_number>, ports=<port_list>). For more information, you can take a look at these
functions insrc/ptf/dataplane.py.