# Deep Reinforcement Learning-Based Joint Scheduling of 5G and TSN in Industrial Networks
## 6 Results
### Results Analysis and Discussion
For DDPG with a potential function reward and without a potential function reward, we use the bootstrap method to estimate the delay and throughput of 95% confidence interval. The specific practice is to generate multiple samples from the forecast result data with resamples, and then calculate the 95% confidence interval indicators based on these resamples. As indicated in Figure 6, EDF achieves the best performance for time-triggered applications, because the EMLR of video is quite a lot more than that of time-triggered applications; then, the latency of time-triggered applications is closer to the deadline, so time-triggered applications can achieve the highest priority for scheduling. However, the PF algorithm is concerned more with the achieved data rate, but does not pay more attention to latency guarantee. Hence, the 5G system latency with PF increases with an increase in terminal number (\(|\mathbf{F}_{it}|:|\mathbf{F}_{vi}|=1:1\)), and the 5G system latency even exceeds the EMLR 6 ms when the terminal number reaches 16. The time-triggered application latency of DDPG without potential function reward can remain stable at around 4.5 ms, because it can achieve a larger reward value when the queueing delay is 4.5 ms. However, the time-triggered application latency of DDPG with potential function reward almost remains stable at around 4 ms, because the potential function not only considers the queueing time as an impact factor on the reward value, but also takes the gating state of the time-triggered queue in DS-TT into account. Unlike the time-triggered application, video is more concerned with the throughput. Figure 7 shows system throughput performance under the multi-application coexisting scenario. EDF prefers to schedule time-triggered applications, which means fewer available resources for video flow. Therefore, the throughput with EDF declines with an increase in terminal number. However, for DDPG without potential function reward, DDPG with potential function reward and PF-based scheduling algorithms, the throughput rises with an increase in terminal number. Because of the designed reward function, the DDPG-based scheduling algorithm provides more schedule opportunities for video flows with relaxed latency requirements for time-triggered applications, so the DDPG potential function reward achieves the best performance and improves by nearly 0.05% throughput compared with PF and by 0.01% throughput compared with DDPG without potential function reward. Generally speaking, the proposed algorithm is more versatile and can achieve better balance between various QoS guarantees in multi-application scenarios compared to EDF and PF. In addition, in order to fully consider the impact of different GCL setting schemes in DS-TT on 5G scheduling, three different GCL setting schemes in DS-TT are evaluated. The first GCL setting scheme for time-triggered queues remains \"ON\" in one TTI and changes to \"CLOSE\" in next TTI; the time during the \"ON\" state for the time-triggered queue is one TTI, and the first GCL setting scheme is represented as \"0101\". As for the second GCL setting scheme, the GCL state of the time-triggered queue changes every two TTIs, which is represented as \"0011\". Furthermore, in the third GCL setting scheme, the GCL state of \"ON\" for the time-triggered queue is selected with probability 0.5, which is represented as \"random\". Figures 8 and 9 show the latency and throughput performance of time-triggered and video flows within three GCL setting schemes in DS-TT, respectively. The bootstrap method is used to estimate the delay and throughput of the 95% confidence interval. As shown in Figure 8, it indicates that various GCL setting schemes can achieve nearly similar throughput performance, because the transmission opportunities of video in all three GCL setting schemes are the same; meanwhile, the proposed DDPG algorithm can guarantee throughput for video applications within various GCL setting schemes. However, the different GCL setting schemes have various impacts on 5G latency.