## 2 Related Works  
Traditional traffic signal control methods have been widely deployed in urban traffic systems. Fixed-time control strategies, such as the Webster method [5], use pre-determined signal plans and are easy to implement but lack adaptability to real-time conditions. Actuated control improves upon this by using detectors to trigger phase changes based on vehicle presence [6], yet its scope remains locally limited. Dynamic control frameworks like TRANSYT and TASS bridge this gap partially by enabling time-of-day-based schedule adaptation and local logic adjustments [7; 8]. More complex adaptive control methods, such as SCOOT and SCATS [9; 10], allow for continuous real-time signal updates. However, they often require centralized infrastructure and suffer from scalability issues when deployed across large-scale city networks [11]. Reinforcement learning (RL) has emerged as a viable solution to overcome these limitations. Early works demonstrated that Q-learning-based agents could outperform fixed strategies in simple intersections [12]. With advances in Deep RL, models like DQN [13] and PPO [14] have been adapted to urban traffic control, enabling agents to learn state-action value functions and stochastic policies. Yet, these models struggle with scalability due to the exponential growth of the action space in multi-intersection environments. Recent methods attempt to address this by using spatio-temporal representations and multi-agent settings. For instance, STMARL [15] introduces a spatio-temporal attention mechanism over agent neighborhoods to improve coordination. HeteroLight [16] addresses the heterogeneity of intersection types using a general policy structure that adapts across zones. MARL-DSTAN [17] constructs temporal graphs between agents to support long-range dependency modeling in irregular traffic networks. These works motivate our choice to combine spatio-temporal modeling and hierarchical multi-agent structures to achieve better generalization and scalability in realistic networks. In addition to reinforcement learning, graph neural networks (GNNs) have been explored as a way to capture spatial dependencies in traffic networks. Graph-based approaches model road networks as graphs, where intersections represent nodes and roads represent edges. GAT (Graph Attention Network) [18] has been applied in traffic signal control to improve information aggregation across intersections. [PERSON]. [19] utilized GNNs to enhance the representation of traffic states in reinforcement learning models, demonstrating improved coordination in multi-agent systems. However, the integration of graph neural networks with multi-agent reinforcement learning for large-scale traffic control is still an evolving research area. Multi-agent reinforcement leafigureming (MARL) is another promising direction for traffic signal control, where multiple agents collaborate to optimize traffic flow across a network of intersections. Ref. [20] applied MARL to decentralized traffic signal control, showing that independent agents can learn effective policies but struggle with global coordination. To address this limitation. Ref. [21] introduced a centralized training and decentralized execution (CTDE) approach, enabling agents to share information during training while maintaining scalability in real-time control. However, MARL approaches often suffer from non-stationarity. Ref. [22] where agents continuously adapt their policies, leading to instability in learning. Addressing this issue requires improved training frameworks, such as parameter sharing and hierarchical reinforcement learning. Despite these advancements, several challenges remain in applying reinforcement learning to large-scale traffic signal control [23]. One major challenge is the trade-off between local and global optimization [24]. While centralized methods achieve better global coordination, they are computationally expensive and difficult to scale.