## 3 Beam Allocation Algorithm Based on MAPPO and Stackelberg Game  
### Problem Formulation  
The beam direction \(\bar{\theta}_{k,n}=\big{\{}\bar{\theta}_{1,n},\cdots,\bar{\theta}_{K^{\prime},n}\big{\}}\) of vehicle k is calculated based on the relative change \(\mathbf{I}_{k,n}=\big{\{}I_{1,n},\cdots,I_{K^{\prime},n}\big{\}}\) as follows:  
\[\begin{split}\bar{\theta}_{k,n}=&\big{\{}\hat{ \theta}_{1,n-1}+I_{1,n}\Delta\theta,\cdots,\hat{\theta}_{k^{\prime},n-1}+I_{k^{ \prime},n}\Delta\theta,\\ &\cdots,\hat{\theta}_{K^{\prime},n-1}+I_{k^{\prime},n}\Delta \theta\big{\}},\end{split} \tag{10}\]where \(\hat{\theta}_{k^{\prime},n-1}\) represents the beam direction corrected by sensing in the previous time slot; \(\Delta\theta\) denotes the angle interval between two adjacent indices in the code book. \(\mathbf{p}_{k,n}=\left\{p_{1,n},\cdots,p_{K^{\prime},n}\right\}\) indicates the transmit power of each beam for vehicle \(k\); \(c_{k,n}\in\left\{0,1,\cdots,M-1\right\}=C\) represents the frequency band index used by vehicle \(k\), with each V2V vehicle only able to select one frequency band, while a frequency band can be used by multiple vehicles.  
According to the above signal model and the definitions of related variables, the original beam allocation problem can be expressed as follows:  
\[\underset{\mathbf{p}_{k,n},\mathbf{k}_{k,n},\mathbf{\omega}_{k,n},\mathbf{\eta}_{k,n},\forall k,m}{\text{maximize}} \frac{1}{N}\sum_{n=1}^{N}\left(\sum_{k=1}^{K}\left(E_{k,n}+R_{k,n}^{r} \right)+R_{n}^{R}\right)\] (11) s.t. \[p_{\min}\leq p_{k^{\prime},n}\leq p_{\max},\forall k^{\prime} \in\mathbf{k}^{\prime}, \tag{11a}\] \[\sum_{k^{\prime}=1}^{K}p_{k^{\prime},n}\leq P_{\max},\] (11b) \[p_{m,n}\leq p_{\max}^{R},\forall m\in M,\] (11c) \[E_{k,n}\geq\Gamma_{E},\forall k\in K,\] (11d) \[R_{n}^{R}\geq\Gamma_{K}^{r},\] (11e) \[R_{k,n}^{r}\geq\Gamma_{V}^{r},\forall k\in K,\] (11f) \[I_{k^{\prime},n}\in\left\{-1,0,1\right\},\forall k^{\prime}\in \mathbf{k}^{\prime},\] (11g) \[c_{k,n}\in\left\{0,1,\cdots,M-1\right\},\forall k\in K,\] (11h) \[\sum_{c\in C}\psi_{k,c}=1,\forall k\in K,\] (11i) \[\sum_{k\in K}\psi_{k,c}\leq K,\forall c\in C,\] (11j) \[\psi_{i,j}\in\left\{0,1\right\}, \tag{11k}\]  
where \(E_{k,n}=\sum\limits_{k^{\prime}=1}^{K^{\prime}}E_{k^{\prime},n}=\sum\limits_{ k^{\prime}=1}^{K^{\prime}}\frac{R_{k^{\prime},n}^{r}}{p_{k^{\prime},n}}\) represents the total energy efficiency of vehicle \(k\); for ease of solution, we replace the sensing SINR with \(R_{k,n}^{r}=\sum\limits_{k^{\prime}=1}^{K^{\prime}}R_{k^{\prime},n}^{r}=\sum \limits_{k^{\prime}=1}^{K^{\prime}}\log_{2}\left(1+\omega_{k^{\prime},n}^{r} \right)\), which represents the total sensing rate of vehicle \(k\). Constraints (11a) to (11c) are related to transmit power, including the transmit power of each beam and the total transmit power. Constraint (11d) is related to the energy efficiency of vehicle \(k\). Constraint (11e) is related to the total achievable rate of I2V. Constraint (11f) is related to the sensing rate of vehicle \(k\). Constraint (11g) specifies the range of the relative change in beam index. Constraint (11h) defines the range of the frequency band index. Constraints (11i) to (11k) specify that a vehicle can select only one frequency band, but a frequency band can be selected by multiple vehicles.  
The original beam allocation problem is an MINLP problem, and since it requires the sensing information of target vehicles for assistance, it is also a sequential decision problem. Additionally, beamforming concentrates energy in the specified direction, effectively reducing interference in other directions. Therefore, maximizing overall sensing-communication performance can be approximated by maximizing the individual sensing-communication performance of each vehicle. To simplify the problem-solving process, and considering that the optimization objective is to maximize the energy efficiency, the total I2V reachable efficiency, and the sensing rate of the vehicle, we relax constraints (11d) to (11f) and temporarily disregard their effects.