# Deep Reinforcement Learning-Based Joint Scheduling of 5G and TSN in Industrial Networks
## 2 Related Work
### Related Research on 5G-TSN  
Scheduling time-critical applications in a 5G or TSN network is challenging, and it becomes even more difficult in the complicated scenarios of 5G-TSN integrated networks. Figure 2: 5G system as a TSN bridge. Currently, a few works mainly pay more attention to how to meet latency demands of time-critical applications using uRLLC in 5G wireless networks. Particularly to satisfy the strict latency requirements of time-critical applications, uRLLC has specified a mini-slot with durations of 0.125-0.25 ms to reduce scheduling delay for those applications. This is different from standard slot level access to radio resources for eMBB applications which has slot durations of 1 ms or more [8]. Reference [9] discusses QoS requirements for uRLLC applications and also proposes various methods to share radio resources among uRLLC and other application types. Reference [10] studies resource allocation schemes to maximize admissible uRLLC loads with associated latency constraints and the influence of retransmission schemes on uRLLC capacity. Furthermore, there are a few other works addressing how to meet QoS constraints of uRLLC via preemptive puncturing of resources assigned for eMBB flows in multiple-application coexistence scenarios; preemption-based methods allow time-critical applications to interrupt ongoing non-time-critical transmissions, which is appropriate for transmitting urgent frames. Reference [11] studies a linear rate loss model for uRLLC preemptive overlap to maximize the utility of eMBB applications while immediately satisfying latency demands of uRLLC. Reference [12] proposes an RAN slicing method to meet the delay demand of deterministic applications through resource reservation and preemption. Currently, reinforcement learning has been widely used for wireless resource scheduling problems [13]. In [14], a deep reinforcement learning algorithm is proposed to solve the channel allocation problem of a multi-beam satellite system, which effectively improved the spectrum resource efficiency and system capacity. Furthermore, some studies have paid attention to frequency resource allocation with reinforcement learning. In [15], an algorithm based on DDPG is proposed for 5G frequency allocation, which can guarantee a low bit-error rate and low latency for packet transmission, but this literature only considers a single application type, which makes the optimization problem easier and is not appropriate for actual network operation. In [16], a joint scheduling method based on DDPG is proposed to achieve a long-term QoS tradeoff between eMBB and URLLC services. However, all the above works only focus on dynamic scheduling schemes in 5 GSs to provide low latency and reliable transmission guarantees for time-critical applications in 5G, and never take the influences of TSN scheduling or TSN features into consideration. Reference [17] introduces 5G-TSN developments in 3 GPP standardization and identifies open issues in future research, such as time synchronization, session continuity and scheduling. In [18], a simulation model based on OMNet++ is addressed, including transmission procedures in 5G-TSN and analysis of several simulation results to prove that 5G-TSN can provide bounded latency for time-sensitive applications, but no joint scheduling scheme is proposed. Reference [19] takes both TSN shaping and 5G radio resource scheduling into consideration; the network constraints in the TSN and 5G domains are described, respectively, and the semi-persistent scheduling mechanism is adopted in 5 GSs, which means that the 5G resources in each TTI for time-triggered applications are predefined. The problem in this work is formulated as multi-objects optimization to minimize the resource loss rate (the ratio of unused and total resources in 5G and TSN, respectively), subject to the deadline constraints of time-triggered applications and resources constraints in both 5G and TSN, and finally constraint programming is applied to determine feasible scheduling solutions.