## 4 Testbed Results
### GOOSE Results
Overall, Device 1 exhibited a wider range of response times, with an average response time of just below 15 ms for all non-5G scenarios, but it also exhibited regular responses timed at close to 20 ms. In contrast to these results, Device 2 achieved a response time closer to 6 ms, with nearly all samples concentrated below 10 ms, even in the TCP configurations. However, these differences in response variance also remained stable across the 5G results, as the measured delay deltas \(\delta_{1}\) and \(\delta_{2}\), which represent the difference between the median non-5G and median 5G scenario, had similar values. In these tests, \(\delta_{1}=3.945\) ms and \(\delta_{2}=3.934\) ms. One particularly interesting result from this test is the addition of outliers for Device 2 and the increase in outliers for Device 1. This implies that although the network configurations closely maintained overall device performance, even over 5G, adding 5G increased the possibility for outliers in end-to-end latency. This was observed for both devices. To explain the introduction of outliers in these results, the latency isolation outlined in the previous section was measured, as shown in Figure 10. In this figure, the GTP encapsulation step includes the duration from when a GOOSE packet is sent to the Core Network to the GTP-encapsulated packet being forwarded to the gNB. The GTP-to-GOOSE extraction measures the timing from the GTP packet sent to the gNB to the final forwarding of the extracted GOOSE Ethernet frame to the OT device, whereas the end-to-end latency is the sum of these two values. From these results, we can observe that the latency of each step has a large number of outliers, with GTP encapsulation ranging from below 1 ms to over 10 ms, and GTP-to-GOOSE extraction averaging 5 ms but also repeatedly exceeding 10 ms. From these results, we can see that not all packets were guaranteed to be transmitted within the required latency for GOOSE messaging within a substation. However, we observed that the latencies were sufficient for less stringent R-GOOSE applications in OT WAN applications [17]. Figure 8: Device latency comparison bar graph. The results shown in Figures 9-11 consistently show outliers, pointing to the fact that it may not be possible to guarantee latencies to be within a well-defined range, which, in turn, impacts the viability of directly supporting time-sensitive networking. Figure 9 outlines the difference in processing between Device 1 and Device 2, which we will address in our future research by studying the device's internal protocol parsing and message handling operations. The separate message preparation steps, as shown in Figure 10, display a large latency spread for the GTP-to-GOOSE extraction and the GOOSE end-to-end latency measurements, which indicates the possibility that 5G resource-block allocations, frame timing, and other air-interface considerations negatively impact the observed performance and introduce non-deterministic considerations that contribute to this large spread, such as the possibility for retransmissions due to bit errors. Additionally, we can also observe a large number of outliers in this figure. To reduce the number of outliers stemming from 5G network operations, we further optimized the configuration of Ubuntu's low-latency kernel on both the gNB and Core Network workstations. The results from this test can be seen in Figure 11.  
Figure 10: Network latency comparison [26]. Figure 9: Device latency comparison box plot. in Figure 11. While there are still a number of outliers within this latency test, no outliers exceed 10 ms latency. We attribute the latency spread and remaining outliers to scheduling within the OT devices and their protocol processing, within the 5G network, and other 5G-related non-deterministic factors not covered by our performed optimizations. Therefore, we aim to study these issues further in our future research, including the investigation of additional optimization strategies and their benefits to latency and the observed end-to-end latency jitter.