## 3 Construction of Integrated Control Problems Based on Heterogeneous FACMAC
### FACMAC-Based Combined VSL and LCC Controller
#### 3.3.3 Joint Action Space  
At each time step \(t\), the actions performed by different agent classes based on the current environment are not consistent. For an integrated ramp control group, the joint action \(u_{t}=\left\{a_{1,t}^{vsl},a_{1,t}^{loc},a_{2,t}^{vsl},a_{2,t}^{loc}...,a_{ramp _{i},t}^{vsl},a_{ramp_{i},t}^{dec}\right\}\) represents the set of actions taken by the VSL agent \(a_{ramp_{i},t}^{vsl}\) and the LCC agent \(a_{ramp_{i},t}^{loc}\) for each ramp \(ramp_{i}\). The action of the VSL agent for each ramp \(a_{r,t}^{vsl}\) is represented by \(l+1\) continuous variables (where \(l\) is the number of lanes) as  
\[a_{ramp_{i},t}^{vsl}=\left\{v_{1},v_{2},\ldots,v_{l},pos_{ramp_{i},vsl} \right\}, \tag{5}\]  
where \(pos_{ramp_{i},vsl}\) in Equation (5) represents the position of the VSL dynamic control area, serving as a unified decision variable for the control lengths of all lanes before the ramp. To maintain the consistency of continuous decision variables, all variables are set within the range of \([-1,1]\). Considering the efficiency and safety of highway traffic, VSLs are generally used to issue advisory speeds to vehicles in different lanes. The speed limit is defined as follows:  
1. To prevent the speed limit from being too low and causing vehicles to fail to move smoothly, the speed limit should not be lower than the minimum highway speed, \(Vsl\geq V_{min}\). 2. To ensure the effectiveness of speed limit control, the speed limit should not exceed the free flow speed, \(Vsl\leq V_{free}\). 3. To maintain traffic flow smoothness and continuity, the speed limit change between consecutive control periods should be within a certain range, i.e., \(\left|a_{ramp_{i},t}^{vsl}-a_{ramp_{i},t-1}^{vsl}\right|\leq m\). This avoids drastic changes in traffic speed due to speed limit control, thereby reducing congestion and accident risks. 4. The speed limit should be a multiple of 5, i.e., the speed limit value should be adjusted to the nearest multiple of 5. Based on the above constraints, the lane speed limit can be calculated using Equation (6):  
\[V_{ramp_{i},t}^{sl}=clip[a_{ramp_{i},t}^{vsl}*m,v_{\min},v_{\max}], \tag{6}\]The \(pos_{ramp_{i},nsl}\) in Equation (5) is converted to the corresponding length range in CTM by Equation (7):  
\[pos_{ramp_{i},nsl}=(pos_{ramp_{i},nsl}+1)/2*(pos_{\max}^{nsl}-pos_{\min}^{nsl})+1, \tag{7}\]  
where \(pos_{max}^{nsl}\) and \(pos_{min}^{nsl}\) represent the farthest and nearest cell positions in the VSL control area from the merging zone, respectively. The difference \(pos_{max}^{ssl}-pos_{min}^{nsl}\) ensures that the VSL dynamic control area remains within a limited range. Finally, the length of \(pos_{ramp_{i},nsl}\) is increased by 1 to ensure that the minimum control range of the dynamic speed limit area is one cell length. Figure 3 illustrates the variation of dynamic control areas on a single entry ramp. It is worth noting that the range of dynamic control areas for VSL and LCC at different entry ramps is influenced by their respective strategies. For the LCC agent action \(a_{ramp_{i},t}^{lcc}\), considering the multi-lane CTM model where the transferable vehicle number \(q_{i,j}^{trans\_stat}\) is primarily determined by the density difference between two adjacent lane cells, we use the lane transfer coefficient \(Ts_{ramp_{i},j\rightarrow\tilde{j}}\). This controls the proportion of vehicle flow transfer between lanes based on density rather than judging from a high-density lane to a low-density lane. This approach effectively manages the flow distribution between lanes. Therefore, the LCC agent action \(a_{ramp_{i},t}^{lcc}\) at each ramp is expressed as Equation (8):  
\[a_{ramp_{i},t}^{lcc}=\Big{\{}Ts_{ramp_{i},1\to 2},Ts_{ramp_{i},2 \to 1},Ts_{ramp_{i},2\to 3},Ts_{ramp_{i},3\to 2},pos_{ramp_{i},lcc} \Big{\}}, \tag{8}\]  
For a three-lane highway, within the LCC control area, vehicles on the inner and outer lanes can only transfer to one adjacent lane, while the middle lane has the ability to transfer vehicles to both sides.