## 3 Construction of Integrated Control Problems Based on Heterogeneous FACMAC
### FACMAC-Based Combined VSL and LCC Controller
#### 3.3.2 Joint State Space  
In macroscopic traffic flow theory, traffic conditions are usually described by a series of macroscopic variables such as traffic speed \(v\), density \(\rho\), and flow \(q\). According to the fundamental diagram (FD), these variables exhibit significant nonlinear characteristics. This is mainly because for a given traffic flow value, there can be two distinctly different traffic states: one is a stable free-flow state below the critical density \(\rho^{cr}\), and the other is an unstable flowing state at low speeds above \(\rho^{cr}\). This phenomenon indicates that simply using the flow \(q\) to represent the traffic conditions perceived by agents can easily lead to perceptual confusion. Therefore, when constructing the Dec-POMDP, both the density \(\rho\) and flow \(q\) are comprehensively considered as macroscopic traffic states.  
For an integrated ramp control group \(i\), there is a local observation \(o_{i}^{val}\) for the VSL agent and a local observation \(o_{i}^{lcc}\) for the LCC agent. Considering that \(o_{i}^{val}\) and \(o_{i}^{lcc}\) belong to the same integrated control group, information sharing exists during the training process. Specifically, at ramp \(i\), the local observation considers the following traffic characteristics:  
1. Bottleneck density \(\rho_{i}^{BN}\) and flow \(q_{i}^{BN}\): Typically, when traffic congestion occurs at a highway ramp merging bottleneck, there is significant traffic pressure and high traffic density at the bottleneck location due to inconsistent flow rates caused by vehicles merging into lanes. According to the definition, the bottleneck traffic state is represented by the lane cells at the entrance ramp.
2. Upstream traffic density \(\rho_{i}^{upstream}\) and flow \(q_{i}^{upstream}\): Due to the strong state correlation between upstream and downstream highway traffic, understanding the upstream traffic state is essential for controlling ramp traffic bottlenecks. Considering that too many state variables increase the difficulty of training, traffic information within 1.5 km upstream is used as the current observation, with this information averaged as \(\bar{\rho}_{i}^{upstream}\) and \(\bar{q}_{i}^{upstream}\).  
3. Ramp traffic density \(\rho_{i}^{r}\) and flow \(q_{i}^{r}\): Similarly, it is crucial to understand the traffic information related to the ramp, which is especially critical for the LCC agent, as it optimizes the lane density distribution based on these differences.
4. LCC control area lane transfer flow \(q_{i}^{trans\_stat}\): Here, we define an information coefficient \(q_{i,j,\bar{j}}^{trans\_stat}\) that reflects the amount of traffic that can be shifted from a lane to a neighbouring lane when a lateral lane change is performed, where \(q_{i,j,\bar{j}}^{trans\_stat}=max\{Nveh_{i,j},l*\rho_{i,j}/T\}\), and, in most cases, \(q_{i,j,\bar{j}}^{trans\_stat}\) is determined by the shifted traffic \(Nveh_{i,j}\).  
Based on the above state information, the VSL agent's observation \(\rho_{i}^{vsl}\) in the integrated ramp control group \(i\) is defined as \(\rho_{i}^{vsl}\in\rho_{i}^{BN}+q_{i}^{BN}+\bar{\rho}_{i}^{upstream}+\bar{q}_{ i}^{upstream}+\rho_{i}^{r}+q_{i}^{r}\), which is acquired through sensors such as roadside detectors to capture the current traffic state at ramp \(i\). The LCC agent's observation \(\rho_{i}^{loc}\) includes lane transfer flow information and is defined as \(\rho_{i}^{loc}\in\rho_{i}^{BN}+q_{i}^{BN}+\bar{\rho}_{i}^{upstream}+\bar{q}_{ i}^{upstream}+\rho_{i}^{r}+q_{i}^{r}+q_{i,\bar{j}}^{trans\_stat}\).