# Low-Cost Software Evaluation of Communication Delays in Networked Industrial Systems
## 5 Packet Generators
### Results and Discussion  
Table 2 summarizes the results of the comparison discussed in Section 5.1, listing the measured \(\bar{X}_{\omega}\), \(s_{\omega}\), and range (maximum minus minimum inter-packet delay). Figure 3 shows the inter-packet delay frequency distribution obtained in the experiments. Four groups of three plots are arranged in a square grid, and each group concerns a different value of \(\omega\). Within a group, each plot highlights with a darker color the frequency distribution of one tool with respect to the others, in the same order as they are listed in Table 2.  
The vertical range of each plot is the relative frequency, from 0 to 100%, while horizontal ranges are the same within each group and have been divided into 40 uniform histogram bins. Readability plots have also been hand-picked around the nominal \(\omega\), which has been highlighted with a vertical, dashed line. In the case of Iperf3, a fraction of the samples fell out of the selected horizontal range and is not displayed; this fraction is reported as HOR in the plots. As shown in Table 2, all tools were able to reach a \(\bar{X}_{\omega}\) very close to the target \(\omega\) (within less than 0.1% error) in all experiments. However, the \(s_{\omega}\) and range of Iperf3 are not ideal and worse than the other generators in all but one case. A closer analysis of Iperf3's frequency distributions plotted in Figure 3 shows they are often multimodal, with a peak close to the minimum possible \(\omega\) of 6.7 us. This corresponds to bursts of (almost) back-to-back packets in the packet stream. To compensate for those bursts, Iperf3's rate control mechanism also sends other packets with an inter-packet delay higher than the nominal \(\omega\). The anomaly becomes less evident at \(\omega\) = 10 us because the inter-packet interval during the burst is inherently closer to the nominal \(\omega\) and becomes less detrimental. Starting from Iperf3 version 3.20, a command-line argument allows users to set how often Iperf3 compares the actual generated bandwidth with respect to the nominal one, and adjusts the rate accordingly. However, despite setting this parameter to 10 us, which is the same as the minimum nominal \(\omega\), we still observed the bursts shown in the figure. As mentioned in Section 5.1, those bursts could affect the DUT by temporarily overloading it and led us to rule out Iperf3 from further use. \begin{table}
\begin{tabular}{c l c c c} \hline \hline  & \(\omega\) & **Tool** & \(\bar{X}_{\omega}\) (**us**) & \(s_{\omega}\) (**us**) & **Range (us**)** \\ \hline \multirow{4}{*}{100 us} & \multirow{2}{*}{100 us} & Iperf3 & 100.009 & 16.435 & 367.360 \\  & & MoonGen & 100.057 & 1.331 & 34.560 \\  & & Rtgen & 100.009 & 3.365 & 95.840 \\  & & Iperf3 & 50.005 & 15.278 & 371.680 \\  & & MoonGen & 50.028 & 28.006 & 99.606 \\  & & Rtgen & 50.012 & 4.061 & 90.160 \\  & & Iperf3 & 25.002 & 9.320 & 138.560 \\  & & MoonGen & 25.014 & 8.819 & 70.724 \\  & & Rtgen & 25.004 & 0.458 & 98.640 \\  & & Iperf3 & 10.001 & 0.603 & 16.480 \\  & & MoonGen & 10.006 & 6.387 & 23.207 \\  & & Rtgen & 10.006 & 0.463 & 20.240 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Generators’ performance data for \(m_{\omega}=10^{5}\) nominal samples; \(l=64\,\mathrm{B}\); \(\omega\) = 100, 50, 25, and 10 μs. Figure 3: Frequency distribution of \(\omega\) for the experiments shown in Table 2. The vertical range of each plot is from 0 to 100% relative frequency. The performance of MoonGen and Rtgen is similar, with a slight advantage in favor of MoonGen, in terms of both \(s_{\omega}\) and range, at \(\omega\) = 100 \(\upmu\)s. However, this is no longer the case for smaller \(\omega\), where Rtgen's \(s_{\omega}\) becomes one order of magnitude smaller than MoonGen's, although MoonGen's range is still marginally better.