## 4 Simulations  
This paper designs a simulator according to the evaluation method defined for urban cases in Annex A of 3 GPP TR 36.885 [29], which describes in detail the vehicle fading model, density, speed, direction of movement, vehicle passage, V2V data flow, etc. The simulation considers the topology scene of the Internet of vehicles in the two-way and one-way lane area of 375 m wide and 649 m long at the intersection. There is a BS in the center of the scene, and the starting position and driving direction of vehicles are randomly initialized within the region. Other simulation parameters of the system are shown in Table 1. The hardware environment of the simulator is Intel Core I9-10900F processor + 32G memory + Nvidia GeForce RTX3090 graphics card. Tensorflow 1.12.0 and Keras 2.24 are used to build and train the neural network.  
The above algorithm adopts a five-layer fully connected neural network. According to literature [22], the hidden layers of DNN and DQN are set as 3 in this paper. The number of neurons in the three hidden layers of DNN is set to 16, 32 and 16, respectively. The number of neurons in the three hidden layers of DQN was set to 1200, 800 and 600, respectively.  
Here, the RELU activation function \(f(x)=\max(0,x)\) is used for both DNN and DQN, and the linear function is set to the activation function for the output layer in DNN and DQN. In addition, the RMSProp optimizer [30] was used for renewal network parameters, and the study rate was 0.001. The loss function is set as Huber loss [31]. In addition, the exploration rate of the whole neural network was set as linear decay from 1 to 0.01 during training. The number of steps T of each training set is set to 1000, and the update frequency of the target network Q is set to 500 steps. The discount rate \(\gamma\) for training is set to 0.05.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Parameters** & **Values** \\ \hline Number of V2I links & 4 \\ Number of V2V links & 4 \\ Carrier frequency & 2 GHz \\ Normalized channel bandwidth & 1 \\ BS antenna height & 25 m \\ BS antenna gain & 8 dBi \\ BS receiver noise factor & 5 dB \\ Vehicle antenna height & 1.5 m \\ Vehicle antenna gain & 3 dBi \\ Vehicle receiver noise factor & 9 dB \\ Vehicle speed & 10–15 km/h \\ Vehicle fading and movement models & Urban case of A.1.2 in [29] \\ V2I transmit power & 23 dBm \\ V2V transmit power & 10 dBm \\ V2I and V2V link models & Table 2 in [22] \\ \hline \hline \end{tabular}
\end{table}
Table 1: Simulation parameters.  
The size of the empirical memory unit is set to be \(1\times 10^{6}\), and the size of the small sample data is set to 512.  
In order to facilitate performance comparison, the corresponding weights of total V2I link rates and total V2V link rates are set to \(\lambda_{c}=0.1\), \(\lambda_{d}=1\) in accordance with reference [22] in the experiment. This algorithm introduces interference weight of priority link \(\lambda_{f}\) to distinguish link priorities. If \(\lambda_{f}\) is too small, it cannot reflect the superiority of the high-priority link. In contrast, high-priority links overoccupy common link resources, affecting fairness and significantly degrading overall performance. Through the experimental test from Figure 3, the selection \(\lambda_{f}=8\times 10^{5}\) can distinguish the priority well and take into account the overall performance of the system.  
In Figure 4, the reward for each episode increases as the training set increases, and eventually converges. However, the reason why rewards occasionally get smaller on the way is that greedy strategies are adopted in the training, which may lead to poor results in the exploration of unknown environments. However, as the exploration rate decreases in the later training period, the rewards per episode are not particularly bad and tend to converge, indicating the stability of the training process.  
Figure 4: Reward per episode.  
Figure 3: Influence of interference weight on reward.