# Energy-Efficient Resource Allocation Based on Deep Q-Network in V2V Communications
## 5 Simulation Results
Because the interference increases according to the increase in the number of V2V links, the transmit power of vehicles increases in order to overcome the interference, in the proposed scheme and the conventional RL-based scheme. The proposed scheme significantly reduces the power consumption of V2V links in comparison with the conventional RL-based scheme because of the penalty function of the transmit power in the reward. In the random allocation scheme, because the transmitter randomly selects the transmit power, the transmit power of the V2V link is fixed on average, regardless of the amount of interference caused by other V2V links. In the simulation environment of this paper, the random allocation scheme shows a low power consumption due to the low transmit power but shows an outage probability that is too high. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Number of neurons in the input layer & 82 \\ Number of neurons in each hidden layer & 500, 250, 120 \\ Number of neurons in the output layer & 60 \\ Reward discount factor & 0.99 \\ Hidden layer activation function & ReLU \\ Optimizer & RMSProp \\ Learning rate \(\alpha\) & 0.001 \\ Values of \(\lambda_{\text{V2I}}\), \(\lambda_{\text{V2V}}\), \(\lambda_{\text{Latency}}\), and \(\lambda_{\text{pwr}}\) & 0.1, 0.9, 1, and 0.2, respectively \\ \hline \hline \end{tabular}
\end{table} 
Table 2: DQN framework parameters. Figure 5: Average transmit power of V2V links. Figure 6 shows the sum rate of V2V links as the number of V2V links increases. As the number of V2V links increases, the sum rate of V2V links increases in all the resource allocation schemes. The sum rate of the conventional RL-based scheme is slightly higher than that of the proposed scheme because the proposed scheme suppresses the transmit power of vehicles for the purpose of energy efficiency. The sum rate of the random allocation scheme is the worst because it randomly selects the RBs regardless of the interference to others. Figure 7 shows the sum rate of V2I links as the number of V2V links increases. Because the number of V2I links is fixed at 20, the interference from the V2V links increases according to the number of V2V links, and thus the sum rate of V2I links decreases with the increase in the V2V links. In particular, the performance of the proposed scheme is slightly better than that of the conventional RL-based scheme. Moreover, because the interference from the V2V links to the V2I link increases according to the number of V2V links, the sum rate of the random allocation greatly decreases with the number of V2V links. Figure 6: Average sum rate of V2V links. Figure 7: Average sum rate of V2I links. Figure 8 shows the outage probability as the number of V2V links increases. Here, the outage probability is defined as the probability that a transmitter on the V2V link fails to transmit data within the maximum allowable latency, \(T_{0}\). The outage probability is inversely proportional to the sum rate. Hence, the outage probability increases according to the number of V2V links. Moreover, the outage probability of the random allocation scheme is much higher than that of other schemes. That is, in order to efficiently allocate resources, RL-based resource allocation is required. The major concerns with deep-learning-based approaches are the computational complexity and the memory space, which depend on the number of parameters to be stored and to be computed. In the proposed RL, from (8), the dimension of the state space is \(D_{\text{state}}=(4\times N_{\text{RB}})+2\) (=82 in our simulation) and the dimension of the action space is \(D_{\text{action}}=N_{\text{pwr}}\times N_{\text{RB}}\) (=60 in our simulation), where \(N_{\text{RB}}\) is the number of RBs and \(N_{\text{pwr}}\) is the number of the transmit power levels. Consider a feed-forward network with \(l\) layers, where layer 0 is the input layer and layer \(l-1\) is the output layer. Let the number of neurons of each layer be \(n_{0},n_{1},\ldots,n_{l-1}\).