# Energy-Efficient Resource Allocation Based on Deep Q-Network in V2V Communications
## 5 Simulation Results  
We consider a single-cell system with one base station and 20 V2I links. We follow the simulation setup for the urban case study of Manhattan with 9 blocks of 3 GPP TR 36.885 [11, 40]. The models of vehicle drops, mobility, and channels all follow the evaluation scenario of 3 GPP TR 36.885. Vehicles are dropped on the road according to a spatial Poissonprocess, and the vehicle locations are updated every one time slot in the simulation. A vehicle moves at a constant speed defined in Table 1. The vehicle changes its direction at the intersection to go straight with a probability of 0.5, to turn left with a probability of 0.25, and to turn right with a probability of 0.25. Figure 4 shows the movement of vehicles for 20 s, where there are eight vehicles and one BS. The V2V channel model and V2I channel model are both described in Table 1, according to 3 GPP TR 36.885. Each vehicle communicates with a vehicle nearby. We perform a time-driven simulation, where the simulation clock is advanced in increments of time slot units and the state variables are updated for every time slot. For each slot in the simulation, we calculate the CSI of the V2I and V2V links and the interference power, which results in the state of the DQN. The simulation parameters are summarized in Table 1.  
In the proposed Q-network, the number of neurons in the input layer is set to 82, the number of neurons in the hidden layers is set to [500, 250 \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline Road intersection size & 430 m \(\times\) 250 m \\ Simulation area size & 1300 m \(\times\) 750 m \\ Absolute vehicle speed & 36 km/h \\ Vehicle drop and mobility model & Urban case of A.12 in MGP TP 36.885 [40] \\ V2V path loss model & WINNER + B1 Manhattan [41] \\ V2V shadowing & Log-normal with \(\sigma^{2}=3\) dB \\ V2I path loss model & 128.1 + 37.6 log(\(R\)), where \(R\) in kilometers \\ V2I shadowing & Log-normal with \(\sigma^{2}=8\) dB \\ V2V and V2I fast fading & Rayleigh fading \\ Noise power & \(-\)114 dBm \\ Carrier frequency,\(f_{c}\) & 2 GHz \\ Sub-carrier frequency & 1.5 MHz \\ Number of V2I links, \(L\) & 20 \\ Number of V2V links, \(K\), & [60, 120, 180, 240, 300] \\ Antenna height of subc B type & 25 m \\ Antenna gain of RSU & 8 dBi \\ Noise figure of RSU’s antenna & 5 dB \\ Antenna height of vehicles & 1.5 m \\ Antenna gain of vehicles & 3 dBi \\ Noise figure of vehicle’s antenna & 9 dB \\ Latency constraints for V2V link & 100 ms \\ V2V payload size & 30 Mbits \\ Update time slot duration & 2 ms \\ Simulation time & 400 ms \\ transmit power level of V2V links & [5, 10, 23] dBm \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simulation parameters. Figure 4: Vehicle movements. neurons in the output layer is set to 60. The activation function of the hidden layers uses the ReLU function. The optimizer for training the Q-network uses RMSProp. The detailed parameters of the DQN framework are summarized in Table 2. The DQN is trained for 20,000 episodes, where an episode means 1 simulation time and new vehicles are dropped each time an episode starts. After training the DQN, the simulation is run 1000 times, and the 1000 results are averaged. The proposed resource allocation is compared with the random resource allocation and the conventional RL-based resource allocation of [11] in terms of the average transmit power of the V2V links, the average outage probability of the V2V links, and the average sum rates of the V2V and V2I networks. In the random resource allocation, the transmitter of the V2V link transmits data with randomly selected transmit power through a randomly selected RB. Figure 5 shows the average transmit power of V2V links according to the number of V2V links. As the number of V2V links increases, the average transmit power of vehicles increases in the proposed scheme and the conventional RL-based scheme, but the average transmit power in the random allocation scheme is fixed.