## 3 Beam Allocation Algorithm Based on MAPPO and Stackelberg Game
### Beam Allocation Algorithm Based on MAPPO
This iterative process continues until the change in \(|\lambda_{k^{\prime}}(j)-\lambda_{k^{\prime}}(j-1)|\leq\Delta_{1},\forall k^ {\prime}\in k^{\prime}\) between two consecutive iterations meets a certain threshold, marking the end of the algorithm. Therefore, based on the Dinkelbach algorithm, problem (20) is transformed into

\[r_{n}=\sum_{k=1}^{K}\left(w_{c}E_{k,n}+w_{r}R_{k,n}^{r}\right)+w_{R}R_{n}^{R}, \tag{14}\]2 \[r_{n}=\sum_{k=1}^{K}\left(w_{c}E_{k,n}+w_{r}R_{k,n}^{r}\right)+w_{R}R_{n}^{R}, \tag{14}\]3 (21) s.t. \[r_{n}=\sum_{k=1}^{K}\left(w_{c}E_{k,n}+w_{r}R_{k,n}^{r}\right)+w_{R}R_{n}^{R}, \tag{14}\]4 \[r_{n}=\sum_{k=1}^{K}\left(w_{c}E_{k,n}+w_{r}R_{k,n}^{r}\right)+w_{R}R_{n}^{R}, \tag{14}\]5

However, the aforementioned problem is not a concave function with respect to \(p_{k^{\prime},n}\), necessitating further transformation. For the achievable rate \(R_{k^{\prime},n}^{c}\) and sensing rate \(R_{k^{\prime},n^{\prime}}^{r}\), we can decompose them into a log(numerator) minus log(denominator) form, where the first part is concave in \(p_{k^{\prime},n}\) but the second part is not. To address the non-concavity issue, this chapter employs the DC approximation technique to transform the non-concave part into a linear function. Specifically, the non-concave part is linearized using the first order Taylor expansion, making the linear function both convex and concave. Therefore, problem (21) can be further transformed into

\[r_{n}=\sum_{k=1}^{K}\left(w_{c}E_{k,n}+w_{r}R_{k,n}^{r}\right)+w_{R}R_{n}^{R}, \tag{14}\]6 \[r_{n}=\sum_{k=1}^{K}\left(w_{c}E_{k,n}+w_{r}R_{k,n}^{r}\right)+w_{R}R_{n}^{R}, \tag{14}\]7 \[r_{n}=\sum_{k=1}^{K}\left(w_{c}E_{k,n}+w_{r}R_{k,n}^{r}\right)+w_{R}R_{n}^{R}, \tag{14}\]8 \[r_{n}=\sum_{k=1}^{K}\left(w_{c}E_{k,n}+w_{r}R_{k,n}^{r}\right)+w_{R}R_{n}^{R}, \tag{14}\]9 \[L(w_{act})=-L^{clip}(w_{act})-S[\pi_{w_{act}}], \tag{15}\]0 \[L(w_{act})=-L^{clip}(w_{act})-S[\pi_{w_{act}}], \tag{15}\]1 (23) \[L(w_{act})=-L^{clip}(w_{act})-S[\pi_{w_{act}}], \tag{15}\]2

In this context, \(\mathcal{Q}\left(I_{k^{\prime},n}(j)\right)\approx\mathcal{Q}\left(I_{k^{ \prime},n}(j-1)\right)+\mathcal{Q}^{T}\left(I_{k^{\prime},n}(j-1)\right)\left( I_{k^{\prime},n}(j)-I_{k^{\prime},n}(j-1)\right)\) represents the first-order Taylor expansion approximation of the non-concave part, where \(I_{k^{\prime},n}(j-1)\) is the result from the previous iteration and can be considered a constant. \(I_{k^{\prime},n}^{c}(j),I_{k^{\prime},n}^{c}(j)\) are linear functions of \(p_{k^{\prime},n}(j)\), representing the interference power during downlink communication and echo sensing, respectively. The algorithm terminates when the outer iteration meets the criterion for \(\left|p_{k^{\prime},n}^{*}(i)-p_{k^{\prime},n}^{*}(i-1)\right|\leq\Delta_{2}, \forall k^{\prime}\in k^{\prime}\), outputting the joint power allocation for I2V and V2V. Note that the iteration index \(i\) represents the outer iteration, where each outer iteration involves solving the leader and follower problems, while the iteration index \(j\) represents the inner iteration, where each inner iteration involves executing the Dinkelbach algorithm. These iterations form a nested loop structure. Additionally, the optimal solution of this joint power allocation algorithm is deterministic, ensuring the stability of the environment when integrated into a multi-agent reinforcement learning framework. The CVX toolbox typically uses the interior-point method to solve nonlinear programming problems, so the computational complexity can be expressed as \(\Theta\left(n_{out}n_{in}l^{3.5}\log(1/\varepsilon)\right)\), where \(n_{out}\) represents the number of outer iterations, \(n_{in}\) represents the number of inner iterations, \(l\) represents the number of optimization variables, and \(\varepsilon\) represents the target accuracy. In summary, the joint power allocation algorithm based on the Stackelberg game proceeds as described in Algorithm 1. ### Target Vehicle Switching Based on Sensing Domain Sharing

Sensing information can serve as state to guide agent decisions.