# Deep Reinforcement Learning-Based Joint Scheduling of 5G and TSN in Industrial Networks
## 7 Conclusions and Future Works  
In this article, a joint resource scheduling problem within 5G-TSN collaboration architecture in a multi-application scenario is formulated and discussed. In order to solve the formulated problem efficiently and intelligently, a DDPG-based joint scheduling algorithm is proposed, which aims to guarantee both latency requirements for time-triggered applications and throughput requirements for video applications. The proposed reinforcement learning model combines latency requirement, throughput requirement and GCL state in DS-TT to design a reward function, which can encourage the agent to learn an optimized action during model training. Finally, the simulation results indicate that the proposed reinforcement learning model is convergent and efficient. Meanwhile, compared with EDF and PF scheduling algorithms at three different gate settings, the proposed DDPG-based algorithm can achieve a stable latency guarantee for time-triggered applications, as well  
Figure 8: Throughput comparison in three GCL settings in DS-TT. Figure 9: Latency comparison in three GCL settings in DS-TT. as improving the overall throughput for video. Furthermore, several different GCL setting methods are proposed to verify the latency and performance within the proposed DDPG-based joint scheduling algorithm; the more discrete GCL settings are more useful for time-triggered applications. Generally speaking, the proposed DDPG-based joint scheduling algorithm can provide solid QoS guarantee for multiple applications and achieve better multi-application carrying capability in 5G-TSN collaboration architecture. However, several key aspects are still not considered in this article. Firstly, the scheduling problems of single base stations are mainly considered in this article, but in reality there are more scenarios of multiple base stations; secondly, we only consider that the radio channel quality affects the bits carried in one RB, but ignore the fact that a poor radio channel also leads to packet loss, so no retransmission procedure is considered in this algorithm. Finally, the proposed algorithm can only improve the throughput of video applications, but cannot maximize the throughput of the whole system. Therefore, in order to solve those problems, there are several aspects to be investigated in the future. First, a 5G-TSN scheduling algorithm based on the multi-agent reinforcement learning model needs to be studied for multiple base station scenarios; meanwhile, the action space should be designed to be more complex. Second, the impact of a retransmission scheme on resource allocation should be considered. Third, for current 5G-TSN joint scheduling, we mainly investigate 5G scheduling considering TSN states, but in the future, how 5G scheduling affects GCL settings in the TSN domain should be studied. Conceptualization, Y.Z. and L.S. ; methodology, L.S., Y.Z. and J.W. ; software, Y.Z. ; validation, Y.Z. and L.S. ; formal analysis, L.S., Y.Z. and J.W. ; investigation, Y.Z., L.S. and R.H.; resources, Y.Z. and L.S. ; data curation, Y.Z. ; writing--original draft preparation, Y.Z. and L.S. ; writing--review and editing L.S., J.W. and X.J. ; visualization, Y.Z. ; supervision, J.W. ; project administration, J.W. and L.S. ; funding acquisition, J.W. and L.S. All authors have read and agreed to the published version of the manuscript. This work was supported in part by the National Key Research and Development Program of China with Grant No. 2020 YFB1708800 from Ministry of Science and Technology and in part by the Guangdong Provincial Key R&D Program with Grant No. 2020B0101130007 from the Administration of Science and Technology, Guangdong Province, China. This paper studies the scheduling algorithm based on reinforcement learning. There is no data set directly available for this kind of scheduling algorithm, and its model training is based on various formulas (such as channel state generated by channel model, etc.) and intermediate data generated by numerical simulation.