# A Forwarding Latency Optimization Method for Software Data Plane Based on Spin-Polling
## 2 Related Work
### Batch Processing Mechanism  
The batch processing mechanism are widely used on high-speed data plane processing frameworks, such as Netmap [32], PacketShader [31], DPDK [1], PF_RING [33], Open Onload [34], and PACKET_MMAP [35], and is usually combined with the spin-polling mechanism. The core of the batch processing mechanism is to amortize the fixed overhead over multiple packets. Batch processing has the advantages of improved instruction cache locality, prefetching effectiveness, and the prediction accuracy [22]. There is also a lot of previous work studying batch mechanisms. Ref. [30] proposes a performance evaluation model by analyzing software virtual switches based on batch mechanisms. Ref. [20] proposes a generic model that allows for detailed prediction of performance metrics related to NFV routers based on batch mechanisms. Ref. [19] focuses on a high-speed data plane processing framework for Rx batch size, and proposes a load-sensitive dynamic adjustment method of Rx batch size to reduce the packet latency.  
Unless otherwise specified, the length of the Rx/Tx queues in this paper refers to the number of packets in the Rx/Tx queues. In the software data plane, there is a batch size in both the Rx queue and the Tx queue, which we call the _Rx batch size_\(\theta_{Rx}\) and the _Tx batch size_\(\theta_{Tx}\), respectively. Specifically, the \(\theta_{Rx}\) is an \"upper limit\": it specifies the upper limit of the number of packets that can be received each time the CPU core checks one Rx queue. However, the \(\theta_{Tx}\) is a \"threshold\": if the length of a Tx queue reaches the threshold,  
Figure 2: Schematic diagram of a single CPU working based on the spin-polling mechanism.  
all the packets in the queue will be sent out, which we call _regular burst mode_; otherwise, the packets will wait in the queue. Note that the data plane cannot tolerate packets staying in the Tx queue for too long. When the network load is light, the length of the Tx queue often takes a long time to reach the threshold \(\theta_{Tx}\). In order to cope with the light network load, a timer or a counter can limit the waiting time of the packets in the Tx queues. In fact, maintaining a timer costs more than a counter. The latter sets a threshold \(N\) of polling times for a CPU core. When the CPU core polls \(N\) times, it will perform a Tx queue flushing: forwarding all packets in all Tx queues, which we call _timeout flush mode_.  
Many existing works [12; 13; 14; 15; 16] only focus on the benefits of fast notifications brought by the spin-polling mechanism, while ignoring the performance waste caused by the spinning cores. Ref. [6] raises the problem of performance waste under the spin-polling mechanism and explores the queue scalability under this mechanism, but they do not give performance optimization suggestions in combination with the batch processing mechanism. In terms of the batch processing mechanism, some works [20; 30] are limited to the performance model analysis under this mechanism. Ref. [19] notes that some adjustments to the batch processing mechanism could improve forwarding latency, but the study only focuses on Rx queues, and the analysis is limited. Our work focuses on the spin-polling mechanism, and by analyzing the characteristics of the data plane under different input traffic rates, we propose a batch processing mechanism adjustment method relying on real-time CPU utilization, which can effectively improve the forwarding latency of the data plane at different input traffic rates.