# Energy-Efficient Resource Allocation Based on Deep Q-Network in V2V Communications
## 3 System Model  
We consider a V2X network consisting of a V2I network and a V2V network as shown in Figure 1. We focus on the uplink in the V2I network, where there are \(L\) V2I links denoted by \(\mathcal{L}=\{1,2,\ldots,L\}\). In the V2V network, there are \(K\) V2V links denoted by \(\mathcal{K}=\{1,2,\ldots,K\}\). In the V2I network, the spectrum is orthogonally allocated to the vehicles, where the number of orthogonal RBs is \(N_{\text{RB}}\). However, the V2V links share the resources, \(N_{\text{RB}}\), of the V2I network.  
Figure 1: A system model.  
In the V2I network, the received SINR and capacity of the \(l\)th V2I link are represented as follows:  
\[\text{SINR}_{l} = \frac{P_{l}^{\text{V2I}}h_{l}}{\sigma^{2}+\sum_{k\in\mathcal{K}}{{{ \it{{\it{{\it{{\it{{\it{H}}}}}}}}}_{k}}}_{l}p_{k}h_{k}}} \tag{1}\] \[C_{l} = W\log{(1+\text{SINR}_{l})}\ \ \text{[bits/second]}, \tag{2}\]  
where \(P_{l}^{\text{V2I}}\) is the transmit power of the vehicle and \(h_{l}\) is the channel power gain in the \(l\)th V2I link. Additionally, \(p_{k}\) is the transmit power of the \(k\)th V2V link, \(h_{k}\) is the channel power gain from the transmitter of the \(k\) V2V link to the base station, \(\sigma^{2}\) is the noise power, and \(W\) is the bandwidth. The indicator function, \(\mu_{k,l}\), denotes whether the resource is shared between the \(k\)th V2V link and the \(l\)th V2I link. That is, if the \(k\)th V2V link shares the RB of the \(l\)th V2I link, \(\mu_{k,l}=1\); otherwise, \(\mu_{k,l}=0\).  
In the V2V network, the received SINR and capacity of the \(k\)th V2V link are represented as follows:  
\[\text{SINR}_{k} = \frac{p_{k}g_{k}}{\sigma^{2}+I_{k}^{\text{V2V}}+I_{k}^{\text{V2I}}} \tag{3}\] \[I_{k}^{\text{V2I}} = \sum_{l\in\mathcal{L}}{{{\it{{\it{{\it{H}}}}}}_{k}}P_{l}^{\text{ V2I}}g_{l,k}}\] (4) \[I_{k}^{\text{V2V}} = \sum_{l\in\mathcal{L}}{\sum_{j\in\mathcal{K},j\
eq k}{{{\it{{\it{ {\it{H}}}}}}_{k}}_{l}}\mu_{j,l}p_{j}{{{\it{{\it{S}}}}}_{j,k}}}\] (5) \[C_{k} =W\log{(1+\text{SINR}_{k})}\ \ \ \text{[bits/second]}, \tag{6}\]  
where \(p_{k}\) is the transmit power of the vehicle and \(g_{k}\) is the channel power gain in the \(k\)th V2V link. Additionally, \(I_{k}^{\text{V2I}}\) is the interference from the V2I link sharing the RB of the \(k\)th V2V link, \(I_{k}^{\text{V2V}}\) is the interference from the V2V link sharing the RB of the \(k\)th V2V link, and \(g_{l,k}\) is the channel power gain from the transmitter of the \(l\)th V2I link to the receiver of the \(k\)th V2V link. The indicator function, \(\mu_{j,l}\), denotes whether the resource is shared between the \(j\)th V2V link and the \(l\)th V2I link. That is, if the \(j\)th V2V link shares the RB of the \(l\)th V2I link, \(\mu_{j,l}=1\); otherwise, \(\mu_{j,l}=0\).  
In order for the BS to know the channel state of the V2V links, each receiver of the V2V link reports its CSI to the BS, which results in a large signaling overhead. Hence, we assume that the BS does not know the CSI of the V2V links. The BS independently controls the resource allocation of the V2I links without considering the channel state of the V2V links. Consequently, vehicles on the V2V link individually select the RB and determine the transmit power based on the locally observed channel information. Here, the locally observed channel information in the V2V link consists of the following: the CSI of the V2I link, the interference power observed in the previous time slot, the instantaneous CSI of the V2V link, and the information on the RB selected by nearby vehicles.  
Our objective is to maximize the sum rate of the V2I links while increasing the probability of meeting the latency constraint of the V2V links by controlling the selection of RB and the transmit power of each V2V link. However, finding the optimal allocations of the RB and transmit power is an NP-hard problem. Hence, we propose a DQN-based approach to solve the resource allocation problem.