## 1 Introduction
Consequently, these approaches might not fully account for the unpredictable nature of real-world traffic scenarios. The advancement of artificial intelligence has led scholars to apply relevant theories and algorithms in analyzing traffic flow characteristics and managing vehicle operations [18], thereby significantly reducing computational complexity. Methods based on intelligent algorithms are dedicated to exploring dynamic and optimal driving strategies for vehicles. Cutting-edge technologies, such as reinforcement learning (RL) and deep reinforcement learning (DRL), facilitate the development of optimized driving strategies for efficient vehicle control at intersections. RL and DRL techniques are harnessed to develop various car-following models and control strategies for CAVs. These innovations aim to optimize trajectories, reduce energy consumption, enhance traffic efficiency, and bolster driving safety. A recent study combined energy-efficient driving with adaptive traffic signal control using RL. It achieved significant fuel savings, ranging from 31.73% to 45.90%, with varying degrees of mobility sacrifice [19]. Additionally, DRL was employed for longitudinal trajectory control in CAVs, ensuring fuel efficiency and safety at signalized intersections [20]. Eco-driving applications for semi-actuated intersections effectively reduced fuel consumption by 29.2% and noise by 21.9%, enhancing sustainability [21]. A parameterized RL approach was proposed to improve energy efficiency without disrupting other vehicles, offering promising results [22]. Moreover, RL-based control minimized energy consumption at signalized intersections while maintaining mobility, demonstrating the potential for sustainable traffic management [23]. Hybrid DRL-based eco-driving algorithms were proposed for low-level CAVs along signalized corridors, demonstrating substantial reductions in fuel consumption with minimal travel time impacts [24]. Some studies have proposed RL models for e-CAVs to mitigate traffic oscillations and improve energy efficiency. Thesemodels exhibited self-learning capabilities and showed the potential to enhance travel efficiency while reducing energy consumption [25; 26]. Additionally, previous studies have explored the application of RL-based methods to enhance traffic efficiency and driving safety. A framework using convolutional neural networks for prediction of time consumption at intersections was proposed, enabling optimal passing order and continuous control for connected vehicles [27]. The impact of leading autonomous vehicles on urban networks was investigated, showcasing potential congestion mitigation benefits [28]. A DRL-based reference speed-planning strategy was introduced for hybrid electric vehicles, with the goal of optimizing fuel economy and enhancing driving safety [29]. Utilizing deep neural networks and multi-agent reinforcement learning, traffic light controllers were effectively coordinated, resulting in substantial reductions in traffic congestion [30; 31]. Through efficient reward functions, controllers adapted to varying traffic demands and diverse traffic light cycles, thereby enhancing intersection safety [32]. Furthermore, an attention mechanism was incorporated to foster successful cooperation among mixed traffic streams and prevent intersection collisions [33]. However, these investigations often adopt discrete action spaces and simple reward functions to streamline the training process, considering relatively few influencing factors. Considering the limitations of current research, this paper proposes an advanced RL-based control method for navigating CAVs at multiple intersections. The approach integrates various factors, including the environment of signalized intersections and the specific driving characteristics of CAVs. The main contributions of our study are summarized as follows:  
1.