Wire-Speed Packet Capture and Replay at 200 Gbps

CESNET (Czech NREN) and Netcope Technologies are ready to demonstrate their NFB-200G2QL smart NICwith Virtex UltraScale+ FPGA specifically designed to push the achievable traffic processing throughput to 200Gbps in a single card. Unique high-speed DMA engines in the FPGA together with highly optimized Linux drivers enable to achieve 200Gbps data transfer through two PCIe Gen3 ×16 interfaces with minimal CPU overhead. Captured network traffic can be independently distributed among individual cores of two physical CPUs (NUMA nodes) without utilization of QPI. As a result, wire-speed packet capture to the host memory from two fully saturated 100Gbps Ethernet interfaces (QSFP28+) is achieved and various network monitoring applications can utilize the power of the latest FPGAs and CPUs for data processing. This is especially useful when traffic of both directions of a single 100GbE link needs to be processed. The proposed demonstration shows how the packets can be received from two 100Gbps Ethernet links at full speed and captured to the host memory at 200Gbps without any loss. The opposite direction of communication is also shown, i.e. how the packets can be transmitted from the host memory towards the two 100GbE network interfaces. Achieved speeds are demonstrated by counters and gauges showing generated, received/transmitted and captured packets. We also show detailed statistics of CPU load during the packet capture/replay for different packet lengths. ACM Reference Format: Lukáš Kekely and Viktor Puš. 2018. Wire-Speed Packet Capture and Replay at 200 Gbps. In SIGCOMM ’18: ACM Special Interest Group on Data Communication, August 20–25, 2018, Budapest, Hungary. ACM, New York, NY, USA, 2 pages. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for thirdparty components of this work must be honored. For all other uses, contact the owner/author(s). SIGCOMM ’18, August 20–25, 2018, Budapest, Hungary © 2018 Copyright held by the owner/author(s). 1 TECHNOLOGY OVERVIEW 100 Gigabit Ethernet (100GbE) was first defined by the IEEE 802.3ba standard [2] from 2010 and currently is the fastest deployed standard of Ethernet for computer networks. It enables for transmitting frames at a rate of 100Gbps, which translates up to nearly 150millions frames per second (new frame every 6.7 ns). We demonstrated COMBO-100G card back in 2014 [3] as the first PCI Express adapter card to support 100G Ethernet technology worldwide. The new low-profile NFB-200G2QL card is shown in Figure 1. It is world’s first PCI Express adapter designed to enable wire-speed processing of traffic at 200Gbps. This hardware-accelerated card with FPGA uses unique DMA modules that can sustain data transfers over PCI Express between the card and memory of the host computer at such high speeds. This feature makes the card ideal for deployment in the fastest backbone networks and in high-throughput data centers. Other main features of the card include: • two 100GbE QSFP28+ transceiver interfaces (cages), • powerful Virtex UltraScale+ VU7P FPGA chip, • three static QDR-IIIe memories (max. 288Mb each), • two PCI Express Gen3 interfaces with 16 lanes each, • PCI Express half-length and low-profile form factor, • external PPS input for precise timestamps. Figure 1: Front view of the NFB-200G2QL FPGA card. SIGCOMM ’18, August 20–25, 2018, Budapest, Hungary Lukáš Kekely and Viktor Puš As a base of our FPGA firmware, we have developed a platform for rapid development of hardware-accelerated applications. The platform includes a set of firmware IP cores, especially blocks for network interfaces and a unique highperformance DMA bus-master connection to the software layer via PCIe bus. The software layer consists of Linux device drivers, tools for card management, and libraries for high-speed data transfers between the card and the host memory (DPDK or proprietary SZE2). The framework also specifies a generic interface to optional traffic processing pipeline in FPGA that can be described using P4 language [4] to perform different operations on passing network data [1]. Our HaNIC solution is an example of such traffic processing in the FPGA firmware. It extends the functionality of a basic NIC by the support of packet parsing, filtering, and configurable hash-based distribution among CPU cores.