## 1 Introduction
By acquiring the status of traffic lights, the controlled CVs know if it should speed up or slow down, judging whether the vehicle could reach the junction within the duration of the green light. In fact, the works mentioned above are applicable to pre-timed or semi-actuated signal control environments. The traffic signal states are tractable for the CV to make decisions in these cases. Besides, some researches focus on situations with unknown traffic signal timing. For example, [PERSON] and [PERSON] [25] developed a simple phase prediction module that uses historically-averaged data and real-time phase data. Then, the ego vehicle is controlled with probabilistic traffic signal circumstances by DP. [PERSON] et al. [26], who formulated a random variable called effective red-light duration and appended a chance constraint to the optimal control problem. The control law is also determined by [PERSON] in their work. Nevertheless, the above-mentioned methods assume that the traffic light operates cyclically with a fixed cycle and duration. The situation with adaptive traffic signal control (ATSC) is ignored by scholars. Meanwhile, the traditional traffic signal control methods will be gradually eliminated in some urban areas with the development and construction of smart cities. ATSC is becoming a promising way to alleviate mounting traffic congestion. As an important part of an intelligent traffic system (ITS), ATSC approaches change the signal state in real-time according to the traffic flow with the intent to reduce vehicle delay [27]. In particular, the reinforcement learning (RL) technique is one of the most up-and-coming approaches to implementing ATSC in terms of its learning-based mechanism. By combining with the deep learning method, deep reinforcement learning (DRL) shows great potential to solve decision-making problems like ATSC [28; 29]. As far as the application is concerned, the DRL-based traffic signal control method is accomplished in the real world by fusing multi-source data. The application of the ATSC in Shenzhen, China, brings about more smooth traffic and less vehicle delay [30]. ATSC methods represented by DRL algorithms have two characteristics: (1) the change of the signal phase is determined by the current traffic state, where the state can be the average waiting time of vehicles, queue length, or some other numerical features; (2) the traffic lights controlled by ATSC methods do not have fixed cycles and phase plans. In this case, the standard SPaT information cannot be produced and obtained by CV, and this leads to more uncertainty regarding the control of the vehicle. More specifically, if the intelligent driving system of the vehicle cannot know when the phase would change, it is difficult to control the vehicle to cross the intersection during the green light, and this may raise the possibility that the vehicle performs a stop-and-go motion with increased energy consumption. Concurrently, considering the surrounding cars and the dynamics of the ego vehicle itself, the eco-driving task in an ATSC environment becomes a complex nonlinear optimal control problem, which is hard to be solved numerically. Therefore, the previous studies can no longer be applied to such situations. However, considering that the ATSC will replace the position of a traditional fixed timing signal control, the eco-driving strategy in such a situation is needed. While model-based control strategies like DP and MPC are time-consuming, the model-free DRL algorithms are capable of controlling the vehicle in a real-time way with a \"trial-and-error\" mechanism. In DRL theory, the agent can learn a policy and carry out the action produced by the policy according to the observed state. Thus, the DRL algorithms are applied to the eco-driving domain to realize adaptive control of vehicles. In this case, [PERSON] et al. [31] utilized Deep Q Network (DQN) to build an eco-driving system, but the action of the agent of the set is a discrete acceleration rate, since DQN cannot deal with continuous action space.