# A Performance Analysis of Internet of Things Networking Protocols: Evaluating MQTT, CoAP, OPC UA
## 6 Local Experiments Performance Evaluation
### Topology 1, Variable Message Size  
The second set of experiments considers topology 1 and messages of variable size. The message size has been randomly selected from an interval between 7 KB and 1000 KB, as explained before. Results for the 3 protocols are respectively provided in Tables 9-11.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline
**Test** & & **AVG** & **STDEV** & **MAX** & **MIN** & **MED** & **IC** \\ \hline
1 ms & TTC & 20.86 & 6.34 & 27.61 & 9.31 & 22.30 & 3.93 \\ \hline
10 ms & TTC & 18.81 & 13.00 & 38.20 & 6.75 & 11.85 & 8.06 \\ \hline
1000 ms & TTC & 84.62 & 17.48 & 110.81 & 57.40 & 89.25 & 10.83 \\ \hline \end{tabular}
\end{table}
Table 6: MQTT results for topology 1 and fixed message size.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline
**Test** & & **AVG** & **STDEV** & **MAX** & **MIN** & **MED** & **IC** \\ \hline
1 ms & TTC & 11.99 & 0.92 & 13.37 & 10.68 & 12.02 & 0.57 \\ \hline
10 ms & TTC & 14.88 & 1.23 & 16.76 & 13.06 & 14.92 & 0.76 \\ \hline
1000 ms & TTC & 1015.28 & 119.60 & 1349.20 & 950.60 & 977.30 & 74.13 \\ \hline \end{tabular}
\end{table}
Table 7: CoAP results for topology 1 and fixed message size.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline
**Test** & & **AVG** & **STDEV** & **MAX** & **MIN** & **MED** & **IC** \\ \hline
1 ms & TTC & 22.02 & 1.34 & 24.82 & 20.31 & 22.02 & 0.83 \\ \hline
10 ms & TTC & 30.92 & 3.76 & 39.90 & 27.05 & 29.54 & 2.33 \\ \hline
1000 ms & TTC & 47.42 & 34.54 & 125.23 & 26.56 & 32.43 & 21.41 \\ \hline \end{tabular}
\end{table}
Table 8: OPC UA results for topology 1 and fixed message size.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline
**Test** & & **AVG** & **STDEV** & **MAX** & **MIN** & **MED** & **IC** \\ \hline
1 ms & TTC & 138.62 & 171.90 & 485.18 & 22.55 & 47.37 & 106.54 \\ \hline
10 ms & TTC & 10.67 & 6.92 & 29.46 & 6.79 & 8.06 & 4.29 \\ \hline
1000 ms & TTC & 80.32 & 9.55 & 94.27 & 68.01 & 79.87 & 5.92 \\ \hline \end{tabular}
\end{table}
Table 9: MQTT results for topology 1 and variable message size.  
In terms of performance when considering fixed message sizes vs. variable message sizes, all protocols exhibit a similar performance, even though CoAP shows a spike when messages are less frequent (IAT of 1000 ms). This behavior was observed in all of the repeated experiments, thus implying that the setup mechanism of CoAP impacts significantly TTC.  
If we consider now the performance due to changes in the frequency of messages (IAT), then OPC-UA is the protocol that provides closer TTC results across all experiments, and CoAP the protocol that is more sensitive to the frequency of messages less frequent messages have more impact in the TTC.  
Overall, all protocols show good performance for this set of experiments (minimum TTC value: 10.67 ms, MQTT; maximum TTC value: 1051 ms, CoAP).  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Test** & **AVG** & **STDEV** & **MAX** & **MIN** & **MED** & **IC** \\ \hline
1 ms & TTC & 16.81 & 2.84 & 22.22 & 14.54 & 15.61 & 1.76 \\ \hline
10 ms & TTC & 12.52 & 2.01 & 16.11 & 9.70 & 12.21 & 1.24 \\ \hline
1000 ms & TTC & 74.09 & 9.09 & 92.50 & 59.20 & 74.10 & 5.64 \\ \hline \hline \end{tabular}
\end{table}
Table 10: CoAP results for topology 1 and variable message size.  
Figure 5: Topology 1 results.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Test** & & **AVG** & **STDEV** & **MAX** & **MIN** & **MED** & **IC** \\ \hline
1 ms & TTC & 59.41 & 0.76 & 67.10 & 59.41 & 65.61 & 0.47 \\ \hline
10 ms & TTC & 42.13 & 32.31 & 67.10 & 43.13 & 59.41 & 20.02 \\ \hline
1000 ms & TTC & 437.07 & 1279.65 & 4079.00 & 26.76 & 32.79 & 793.12 \\ \hline \hline \end{tabular}
\end{table}
Table 11: OPC UA results for topology 1 and variable message size.