# Rethinking Modbus-UDP for Real-Time IIoT Systems
## 4 Experimental Results
### Effect of Data Link Errors
Namely, although errors affect the mean round-trip time of Modbus-UDP and Modbus-TCP in similar ways in absolute terms, increasing it by an amount in the range \([113.55,194.32]\) us for Modbus-UDP and \([117.43,240.13]\) us for Modbus-TCP, the effect on standard deviation is notably different, increasing it by a ratio in the range \([4.09,5.39]\) for Modbus-UDP instead of the higher \([5.10,7.49]\) for Modbus-TCP. Denoting with \(\sigma_{\text{UDP}}\) the standard deviation of Modbus-UDP and with \(\sigma_{\text{TCP}}\) the one of Modbus-TCP, Table 6 presents the results of one-tailed F-tests for the equality of variances with significance level \(\alpha=0.001\), null hypothesis \(H_{0}:\sigma_{\text{UDP}}=\sigma_{\text{TCP}}\), and alternate hypothesis \(H_{1}:\sigma_{\text{UDP}}<\sigma_{\text{TCP}}\). The F-tests cover all the test cases being considered, and the alternate hypothesis was accepted in all cases, thus confirming the statistical significance of the difference between the standard deviations of Modbus-UDP and Modbus-TCP in the presence of data link errors. Even more importantly for RT Class 3 (soft real-time) and Class 2 (hard real-time) systems, according to the classification proposed in [44], is the fact that, in the case of Modbus-TCP, a non-negligible fraction of samples led to a round-trip time well in excess of \(10\,\text{ms}\) and up into the hundreds of milliseconds, as shown in the rightmost column of Table 5. For the sake of illustration, these samples are represented by the wider bars at the extreme right of Figure 8a,b. The fact that about 2% of the samples fell in this category, the same value as the expected number \(e\) of transactions affected by errors, leads to the observation that, whenever TCP segment retransmission is called into action, the round-trip time of Modbus-TCP transactions grows inordinately, to the point of exceeding the default transaction timeout of the Modbus master library used for the experiments. In fact, the timeout had to be adjusted to conduct these tests. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multicolumn{3}{c}{**Test Case**} \\
**R/W** & \(\mathbf{N}\) & **Conf. int. of \(\sigma_{\text{UDP}}\)/\(\sigma_{\text{TCP}}\)** & **Accept** \\ \hline R & 1 & \([0,0.4508]\) & \(H_{1}\) \\ R & 60 & \([0,0.4746]\) & \(H_{1}\) \\ R & 120 & \([0,0.5007]\) & \(H_{1}\) \\ W & 1 & \([0,0.4929]\) & \(H_{1}\) \\ W & 60 & \([0,0.4962]\) & \(H_{1}\) \\ W & 120 & \([0,0.4462]\) & \(H_{1}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 6: One-tailed F-tests for the equality of standard deviations, comparing \(\sigma_{\text{UDP}}\) and \(\sigma_{\text{TCP}}\) at significance level \(\alpha=0.001\). \(H_{0}:\sigma_{\text{UDP}}=\sigma_{\text{TCP}}\), \(H_{1}:\sigma_{\text{UDP}}<\sigma_{\text{TCP}}\). \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Protocol** & **R/W** & \(\mathbf{N}\) & \(\mathbf{\mu\) (\(\mathbf{\mu}\)s)} & \(\mathbf{\sigma\) (\(\mathbf{\mu}\)s)} & **Min (\(\mathbf{\mu}\)s)** & **Max (\(\mathbf{\mu}\)s)** \\ \hline Modbus-TCP & R & 1 & 1836.76 & 1253.06 & 526 & 487,738 \\ Modbus-TCP & R & 60 & 1839.95 & 1180.89 & 550 & 298,533 \\ Modbus-TCP & R & 120 & 1835.40 & 1172.27 & 557 & 443,298 \\ Modbus-TCP & W & 1 & 1819.59 & 1174.32 & 531 & 417,290 \\ Modbus-TCP & W & 60 & 1818.13 & 1147.78 & 612 & 416,851 \\ Modbus-TCP & W & 120 & 1855.41 & 1247.55 & 550 & 448,144 \\ \hline Modbus-UDP & R & 1 & 723.15 & 547.65 & 427 & 8242 \\ Modbus-UDP & R & 60 & 731.58 & 543.42 & 460 & 8039 \\ Modbus-UDP & R & 120 & 759.47 & 569.08 & 468 & 8411 \\ Modbus-UDP & W & 1 & 724.23 & 561.19 & 441 & 7874 \\ Modbus-UDP & W & 60 & 748.72 & 552.16 & 456 & 8297 \\ Modbus-UDP & W & 120 & 745.42 & 539.76 & 468 & 8026 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Summary statistics of the data shown in Figure 8.  
This phenomenon was not observed when using Modbus-UDP, whose worst-case round-trip time remained consistently below 10 ms across all experiments.