## 2 Model Construction
### Reward Function  
Deep reinforcement learning aims to maximize rewards. The agent learns a strategy based on the rewards obtained from each action taken to make better action decisions. The model's reward function is established with a focus on reducing CO\({}_{2}\) emissions and improving intersection traffic efficiency. (1) Traffic efficiency  
\[R_{1}=b_{1}D_{t-1}-D_{t} \tag{5}\]  
where \(D_{t}\) and \(D_{t-1}\) represent the total waiting time at the intersection at time \(t\) and \(t-1\), respectively. Multiple experiments have shown that setting the weighting coefficient \(b_{1}\) to 0.9 helps the agent better learn and optimize strategies. (2) Carbon emissions  
\[R_{2}=b_{2}E_{t-1}-E_{t} \tag{6}\]  
where \(E_{t}\) and \(E_{t-1}\) represent the total CO\({}_{2}\) emissions at the intersection at time \(t\) and \(t-1\), respectively, and \(b_{2}\) is set to 0.9. To make more accurate decisions based on vehicle CO\({}_{2}\) emissions, real-time speed and acceleration information are obtained using roadside sensors. The instantaneous carbon emissions of fuel vehicles are computed through the Vehicle Specific Power method (\(VSP\)), while the instantaneous carbon emissions of electric vehicles are computed from the perspective of energy conversion efficiency. 1 Instantaneous CO\({}_{2}\) emissions of conventional gasoline vehicles  
Instantaneous energy consumption of gasoline vehicles is determined using the \(VSP\) method, known for its simplicity and wide applicability. As noted in the literature [44], the formula for \(VSP\) is as follows:  
\[VSP=v(1.1a+9.81 grade+0.132)+0.000302v^{3} \tag{7}\] \begin{table}
\begin{tabular}{c c c} \hline \hline
**Phase Number** & **Vehicle Travel Direction** & **Phase Illustration** \\ \hline
1 & \(NSG\) & \\
2 & \(NSLG\) & \\
3 & \(EWG\) & \\
4 & \(EWLG\) & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Phase action selection. where \(v\) and \(a\) denote the instantaneous speed and acceleration of the vehicle, respectively, and \(grade\) denotes the road gradient. Given that the study scenario involves urban intersections, we set \(grade\) = 0, simplifying the calculation formula as follows:  
\[VSP=v(1.1a+0.132)+0.000302v^{3} \tag{8}\]  
According to the simplified formula, the calculation of \(VSP\) only requires the vehicle's instantaneous speed and instantaneous acceleration. The accurate division of specific power (\(VSP\)) intervals is crucial for the precision of vehicle exhaust emission measurements. [PERSON] [45] divided the \(VSP\) into 14 operational condition intervals, with the emission rate for each interval representing the average instantaneous emission rate of various pollutants. The instantaneous CO\({}_{2}\) emissions of a fuel-powered vehicle can be determined based on the CO\({}_{2}\) emission rates corresponding to different specific power intervals, as shown in Table 2.  
2. Instantaneous CO\({}_{2}\) emissions of new energy electric-powered vehicles  
From the perspective of energy conversion, the instantaneous electricity consumption of electric vehicles is converted into instantaneous CO\({}_{2}\) emissions using the national grid's CO\({}_{2}\) emission factor. The formula for calculating the electricity consumed to charge electric vehicle batteries at time \(t\) is as follows:  
\[W_{t}=\frac{W_{e}}{\eta_{gb}} \tag{9}\]  
where \(W_{e}\) represents the instantaneous total electricity consumption of electric vehicles; \(\eta_{gb}\) represents the charging loss rate, set at 0.97; and \(W_{t}\) represents the actual power consumption at the time \(t\). As per the Ministry of Environmental Protection Climate Directive [2023] No.43 [46], the CO\({}_{2}\) emission factor is specified as 0.5703 tCO\({}_{2}\)/MWh. The formula for calculating carbon emissions from electric vehicles is as follows:  
\[W_{c}=W_{t}\text{-0.5703} \tag{10}\]  
where \(W_{e}\) represents the actual carbon emissions per second of the vehicle. (3). Comprehensive reward function