## 3 Problem Formulation  
The eco-driving problem can be formulated as an optimal control problem, whether in a fixed timing signal environment or an ATSC environment. The general scenario for the ego vehicle crossing a single intersection is shown in Figure 4.  
Some assumptions are made in this paper for simplification without losing generality:  
1. The ego vehicle knows its position on the lane, speed, and acceleration;
2. The data packets can be transmitted by the traffic light to the ego vehicle through V2I communication without delay or packet loss;
3. The acceleration/deceleration of the ego vehicle can be controlled accurately with the onboard control module;
4. The ego vehicle will not change the lane, which means that only longitudinal movement is considered. Figure 4: The driving scenario for the ego vehicle in a single intersection. Figure 3: The training process of the traffic lights with RAINBOW. With the intent to describe the problem, we also formulate the optimal control problem form for the task, while the control item is the acceleration of the ego vehicle. Firstly, the main goal of the problem is to reduce the energy consumption in the whole process:  
\[J_{1}=\sum_{j=1}^{m}\int_{t_{0}^{j}}^{t_{f}^{j}}\phi(v(t),a(t))dt, \tag{4}\]  
where \(m\) is the total number of intersections; \(i_{0}^{j}\) is the beginning time for the ego vehicle to enter the lane of intersection \(j\), while \(t_{f}^{j}\) refers to the time that the ego vehicle arrives at the stop line of the junction; \(\phi\) is the energy function that can produce instantaneous energy consumption of the vehicle. In this paper, the energy model for electric vehicles is embedded in SUMO with an energy brake-recovery mechanism, and the details can be found in [43]. Moreover, \(v(t)\) and \(a(t)\) represent the speed and acceleration of the vehicle in time \(t\). Secondly, mobility should also be considered, and we measure that by total travel time:  
\[J_{2}=\sum_{j=1}^{m}\frac{v_{max}\left(t_{f}^{j}-t_{0}^{j}\right)}{L_{j}}\, \tag{5}\]  
where \(v_{max}\) denotes the maximal road speed limit. Finally, a soft constraint is imposed on the objective function, and the objective function is denoted as:  
\[J(a)=\omega_{1}J_{1}+\omega_{2}J_{2}+\sum_{j=1}^{m}c\left(j,t_{f}^{j}\right) \frac{1}{\epsilon+p\left(j,t_{f}^{j}\right)}, \tag{6}\]  
where \(\omega_{1}\) and \(\omega_{2}\) are the weighting parameters; \(c\left(j,t_{f}^{j}\right)\) is a boolean value: the value is set to 0 when the traffic light is green; the value is set to 1 if the vehicle is not allowed to cross the junction owing to the spatiotemporal interval produced by a traffic signal. \(\epsilon\) is a small number to serve as a penalty item and \(p\left(j,t_{f}^{j}\right)\) measures the green probability for traffic light \(j\) in time \(t_{f}^{j}\). This soft constraint encourages the vehicle to pass the road crossing without halting in front of the stop line because of the red light, and the energy can be saved in this case. Therefore, let \(d_{j}(t)\) be the lane position of the vehicle approaching traffic light \(j\), and \(x(t)=\left[d_{j}(t),\ v(t)\right]^{T}\). The whole control problem is formulated as:  
\[minJ(a). \tag{7}\]  
Subject to:  
\[\dot{x}(t)=\left[v(t),\ a(t)\right]^{T},\ t\in\left[t_{0}^{1},t_{f}^{m}\right], \tag{8}\]  
\[0\leq v(t)\leq v_{max}\,\ t\in\left[t_{0}^{1},t_{f}^{m}\right], \tag{9}\]  
\[d_{max}\leq a(t)\leq a_{max}\,\ t\in\left[t_{0}^{1},t_{f}^{m}\right], \tag{10}\]  
Figure 4:0  
Figure 4:1  
Figure 4:2  
Equations (9) and (10) denote the dynamics constraints of the ego vehicle. (11) denotes the safety consideration, while \(a^{CF}(t)\) is the value of acceleration generated by the car following model. The acceleration of the ego vehicle generated by the control model cannot go beyond the car-following model-based value to ensure that the ego vehicle will not collide with the potential leader cars. (12) denotes the start state and the end state. In fact, the end speed can be any value.