# Adaptive Traffic Signal Control Method Based on Offline Reinforcement Learning
## 2 Related Work  
In the field of traffic signal control, researchers have made significant contributions, from early fixed-time control methods to actuated control systems and rule-based adaptive methods. Each generation of approaches has introduced new perspectives and tools for addressing traffic problems. As early as the 20 th century, the max pressure (MP) algorithm was proposed and initially applied to routing and scheduling in wireless networks. [PERSON] et al. [2] first introduced the MP algorithm to traffic signal control in 2013, proposing an MP control model suitable for traffic intersection networks. Since then, various MP-based models have evolved, such as PressLight [3], which combines the max pressure method with a reward design aimed at maximizing network throughput. In the field of reinforcement learning, Deep Q-Network (DQN) [4] has been widely applied to optimize complex control tasks in recent years. For instance, in the study by [PERSON] et al. [5], deep reinforcement learning was utilized for resource management in 5G and satellite-integrated networks. Many advanced reinforcement learning algorithms, such as Q-learning and actor-critic (AC), also demonstrate their effectiveness in complex scenarios. In a similar context, we can consider applying deep reinforcement learning methods to traffic signal control to optimize resource utilization and enhance traffic management efficiency. For example, the following are some traffic signal control methods based on reinforcement learning: FRAP [6] improves efficiency and adaptability in complex traffic scenarios by simulating phase competition principles. IntelliLight [7] optimizes signal switching strategies using deep reinforcement learning and emphasizes interpretability. Colight [8] utilizes graph attention networks to facilitate communication between intersections, learning dynamic communication patterns and enabling index-free modeling of neighboring intersections. This approach has been successfully applied to optimize signal control in large-scale traffic networks. Similarly, AttentionLight [9] leverages the self-attention mechanism to automatically learn correlations between signal phases, eliminating the need for manually designing phase competition relationships, thereby significantly enhancing control accuracy and adaptability. DynamicLight [10] adopts a two-stage framework, where reinforcement learning is used to determine the phase, followed by selecting the phase duration from a set of predefined discrete values, thereby refining control precision. Despite the significant progress made by traditional traffic signal control methods and online reinforcement learning strategies in enhancing the intelligence of traffic management, they still exhibit limitations. Traditional control methods often struggle to adapt flexibly to complex and rapidly changing traffic conditions, making it difficult to maintain adaptability in dynamic environments. While online reinforcement learning can continually optimize strategies through a real-time interaction with the environment, its application typically requires extensive online data and computational resources. Moreover, online reinforcement learning faces challenges related to safety and stability during real-world deployment, particularly when quick responses are needed in complex traffic scenarios, potentially introducing risks. Offline reinforcement learning, an emerging learning paradigm, offers an alternative by enabling policy learning from historical data, thus avoiding the need for a real-time interaction with the environment [11]. Compared to online learning, this approach not only significantly reduces experimental costs but also mitigates safety risks during deployment. Offline RL can also integrate vast amounts of expert data to rapidly optimize control strategies. For instance, models like DataLight [12], which utilize offline RL for traffic signal control, demonstrate the potential of this method to advance the field. Similarly, [PERSON] al.