# Adaptive Traffic Signal Control Method Based on Offline Reinforcement Learning
## 5 Experiments  
### Dataset  
This study uses offline datasets constructed from seven sets of real-world traffic flow data [20], covering intersections from the cities of Jinan and Hangzhou in China and New York in the USA. Each dataset reflects the unique topology of the intersections in each city, as detailed below:  
* Jinan Dataset: The road network consists of a 3 \(\times\) 4 grid structure, with 12 intersections. Each intersection has four entry and exit ramps, comprising two 400 m east-west roads and two 800 m north-south roads. Each ramp has three lanes (left-turn, straight, and right-turn lanes). The traffic flow data are based on camera data from Jinan.
* Hangzhou Dataset: The road network consists of a 4 \(\times\) 4 grid, with 16 intersections. Each intersection has four entry and exit ramps, comprising two 800 m east-west roads and two 600 m north-south roads. Each ramp also has three lanes (left-turn, straight, and right-turn lanes). The traffic flow data are derived from cameras in Hangzhou.
* New York Dataset: This dataset simulates a network of 28 \(\times\) 7 intersections in the Manhattan area of New York, with each intersection connecting four 300 m road sections, including two east-west and two north-south roads.  
The offline datasets used for training were constructed based on publicly available road network files and traffic flow data for each of the three cities. Specifically, an offline dataset was generated for each city, where 20 rounds of simulation were conducted for each set of traffic flow data, recording the state, action, and reward information for all intersections in each simulation round. The state includes the queue length on the incoming lanes, the number of vehicles currently in motion, and the number of vehicles in each road segment. The action consists of the current traffic signal phase and the phase duration, while the reward is defined as the negative queue length to reflect traffic congestion levels. After the simulations, the data for each city were aggregated and randomly shuffled.  
To improve the diversity and generalization capabilities of the datasets, the selection of phase actions was designed to combine fixed-cycle and random strategies. Specifically, the next phase was chosen sequentially or maintained at the current phase with a certain probability, while there was also a probability of selecting a random phase as the next one. The phase duration was randomly generated within a preset range. This design ensures  
Figure 5: Network design of SD3-Light.  
that the dataset follows some conservative fixed rules while incorporating randomness, enhancing the model's adaptability to different traffic patterns.  
The offline datasets generated for the three cities each contain approximately 94,167 to 118,792 state-action-state transition \((\mathbf{s}_{i},\mathbf{a}_{i},\mathbf{s}_{i}^{\prime},\mathbf{r}_{i})\) tuples, ensuring the richness and representativeness of the training data. The diversity and broad coverage of the datasets effectively support the training of the proposed method and enhance its robustness in real-world applications.