# A Hybrid Multi-Agent Reinforcement Learning Approach for Spectrum Sharing in Vehicular Networks
## 3 System Model  
Consider a cellular-based vehicular communication network illustrated in Figure 1. The vehicular network comprises \(M\) V2I links and \(K\) V2V links, thus offering support for high-bandwidth entertainment services and dependable regular safety message exchange for advanced driving applications. We note that all the vehicles are capable of doing both V2I and V2V communication equipped with different multiple radios. The V2I links in our vehicular communication network employ cellular interfaces, thus establishing connections between vehicles and the Base Station (BS) to facilitate high-speed data transmission services. Conversely, V2V links utilize sidelink interfaces, thereby enabling localized D2D communications for periodically disseminating generated safety messages. The set represents V2I links in the network \(\mathcal{M}\) = {1,2,\(\ldots\),\(\mathcal{M}\)}, while the V2V links are represented by \(\mathcal{K}\) = {1,2,\(\ldots\),\(\mathcal{K}\)}. This setup enables efficient data exchange and seamless communication, thereby ensuring dependable safety message sharing and high-data-rate entertainment services among vehicles. The interference to the V2I links consists of two parts: the background noise and the signal from the V2V links sharing the same sub-band. The Signal-to-Interference-plus-Noise Ratio (SINR) for the \(m\)th V2I is given by Equation (1) as follows. \[\gamma_{m}=\frac{P_{m}g_{m,B}}{\sigma^{2}+\sum_{k\in\mathcal{K}}\rho_{k}P_{k}g _{k,B}}, \tag{1}\]  
where \(P_{m}\) is the transmission power of the V2I link, and \(P_{k}\) is the transmission power of the V2V link. \(\sigma^{2}\) in Equation (1) is the noise power, and \(g_{m,B}\) is the power gain of the channel corresponding to the \(m\)th V2I link, where \(B\) refers to the BS (Base Station), and \(g_{k,B}\) is the interference power gain of the \(k\)th V2V link, which reflects the impact of interference from the V2V link on the BS. \(\rho_{k}\) is the spectrum indicator, with 1 when the \(k\)th V2V user reuses the spectrum and 0 otherwise. The capacity of the \(m\)th V2I link can be expressed as given in Equation (2). \[C_{m}=W\log(1+\gamma_{m}), \tag{2}\]  
where \(W\) in Equation (2) signifies the bandwidth associated with each spectrum sub-band. Similarly, we can formulate the SINR for the \(k\)th V2V link, which might utilize the shared spectrum as given in Equations (3)-(5). \[\gamma_{k}=\frac{P_{k}g_{k}}{\sigma^{2}+G_{k}+G_{k}}, \tag{3}\]  
Figure 1: An illustrative structure of vehicular communication networks. \(B\) is the base station, \(m\) is V2I link, \(k\) is the V2V link, \(k^{\prime}\) is another V2V link, \(g_{m,B}\) is the power gain of V2I link, and \(g_{k,B}\) is the interference power gain of V2I link. with  
\[G_{k}=P_{m}g_{m,k}, \tag{4}\]  
and  
\[G_{k}=\sum_{k^{\prime}\
eq k}\rho_{k^{\prime}}P_{k^{\prime}}g_{k^{\prime}}g_{k^{ \prime}}g_{k^{\prime}}, \tag{5}\]  
where \(g_{k}\) is the gain of the \(k\)th V2V link; \(g_{m,k}\) is the interference from the V2I link; \(\rho k^{\prime}\) is the spectrum indicator, with 1 when \(k^{\prime}\) the V2V user reuses the spectrum and 0 otherwise; \(k^{\prime}\) is the other V2V user sharing the spectrum with \(k\)th V2V user; and \(g_{k^{\prime},k}\) and \(g_{k^{\prime}}\) are the interference from another \(k^{\prime}\)th V2V links using the same resource block. The capacity of the \(k\)th V2V can be expressed as given in Equation (6). \[C_{k}=W\log(1+\gamma_{k}^{d}). \tag{6}\]  
This paper aims to optimize the V2I link capacity for high-quality entertainment services, thus meeting low-latency and high-reliability requirements for V2V links to provide realistic and dependable information to vehicle users in road traffic. To achieve the first objective, the sum rate of the V2I links must be maximized. For the second objective, V2V users need to successfully transmit packets of size \(D\) within a finite time \(T_{\text{max}}\), as modeled in Equation (7).