## 2 System Model and Problem Formulations  
On a V2X network, each V2V link can independently select a different channel to maximize the transmission rate. However, the global performance is poor due to interference between different V2V links. On the other side, considering the V2X scenarios, BS has enough computing and storage resources and can achieve the efficient allocation of resources. With the help of reinforcement learning, this paper uses BS as an agent to interact with the unknown vehicle-mounted network environment. Suppose there are CU and D2D users co-existing in a vehicle-mounted communication network, where each device is equipped with a single antenna. In this paper, the sets of CU and D2D users are, respectively, expressed as \(I=\{1,2,\ldots,i,\ldots,I\}\) and \(J=\{1,2,\ldots,j,\ldots,J\}\). Each CU establishes V2I links with BS to support high-quality services, and each D2D user pair transmits information by establishing V2V links. In order to ensure high quality V2I link communication, it is assumed that each V2I link has been pre-assigned different orthogonal spectral subcarriers to eliminate the interference between V2I links in the network. Without sacrificing performance, V2V links and V2I links share the same spectrum resources. To improve the communication quality of V2V links, each V2V link needs to select its occupied spectrum subcarriers and transmitted power. The channel power gain of the V2I link established between the \(i\)-th CU user and BS on authorized channel \(i\) is defined as \(G_{i}[i]\). \(H_{j,B}[i]\) represents the interference channel gain from the vehicle transmitter of the \(i\)-th V2I link to the vehicle receiver of the \(j\)-th V2V link occupying the \(i\)-th subcarrier. \(P_{i}^{c}\) and \(P_{j}^{d}\) denote the transmitting power of the \(i\)-th V2I link vehicle transmitter and the \(j\)-th V2V link vehicle transmitter, respectively; \(\sigma^{2}\) represents the noise power. \(\rho_{j}^{i}=\{0,1\}\) represents the spectrum allocation scheme, if the \(j\)-th V2V link chooses the \(i\)-th channel, \(\rho_{j}^{i}=1\); otherwise \(\rho_{j}^{i}=0\). In this case, the reception signal-to-noise ratio (SINR) of V2I link \(i\) using the \(i\)-th subcarrier can be expressed as follows:  
\[\gamma_{i}^{c}[i]=\frac{P_{i}^{c}G_{i}[i]}{\sum_{j=1}^{J}\rho_{j}[i]P_{j}^{d}H _{j,B}[i]+\sigma^{2}}. \tag{1}\]  
Assume that each V2V link occupies only one channel. According to the Shannon formula, the transmission rate of the \(i\)-th V2I link using the \(i\)-th channel can be expressed as:  
\[C_{i}^{c}[i]=W\log_{2}(1+\gamma_{i}^{c}[i]), \tag{2}\]  
where \(W\) is the bandwidth of each channel. Similarly, \(H_{j}[i]\) represents the interference channel gain of the \(j\)-th V2V link occupying the \(i\)-th subcarrier. \(H_{k,j}[i]\) denotes the interference channel gain from the \(j\)-th V2V link vehicle transmitter to the \(j\)-th V2V link vehicle receiver on the \(i\)-th V2I link vehicle transmitter to the \(j\)-th V2V link receiver on the \(i\)-th subcarrier. In summary, the SINR of the V2V link \(j\) occupying the \(i\)-th subchannel can be represented as:  
\[\gamma_{j}^{d}[i]=\frac{\rho_{j}[i]P_{j}^{d}H_{j}[i]}{I_{j}[i]+\sigma^{2}}, \tag{3}\]  
where \(I_{j}[i]\) denotes the interference power received by the \(j\)-th V2V link from other V2V links and from all V2I links:  
\[I_{j}[i]=\sum_{k\
eq j}^{J}\rho_{k}[i]P_{k}^{d}H_{k,j}[i]+P_{i}^{c}G_{i,j}[i], \tag{4}\]  
\(\sum_{k\
eq j}^{J}\rho_{k}[i]P_{k}^{d}H_{k,j}[i]\) indicates the interference power between V2V links; \(P_{i}^{c}G_{i,j}[i]\) represents V2I link interference power. Therefore, the transmission rate of the \(j\)-th V2V link on the \(i\)-th channel can be expressed as:  
\[C_{j}^{d}[i]=W\log_{2}(1+\gamma_{j}^{d}[i]). \tag{5}\]  
To account for overall link performance and high-priority link interference requirements, we maximize an objective function that is a weighted sum of two terms and subtract one term.