## 5 Simulation Results Analysis  
### Training Analysis  
This subsection conducts a comparative analysis of three representative deep reinforcement learning algorithms--Deep Q-Network (DQN) [8], double deep Q-network (DDQN) [9], and the proposed dual-priority delayed deep double Q-network (DPD3 QN)--through a multi-dimensional performance evaluation framework. Under identical initialization parameters, the experimental design specifically examines four critical performance dimensions: (1) policy convergence characteristics and training stability, (2) cumulative reward acquisition efficiency, (3) state space exploration capability, and (4) overestimation error suppression effectiveness.  
Figure 6 illustrates the round rewards achieved by the three algorithms. A greater reward signifies that the self-driving vehicle is navigating the highway with increased efficiency. As shown in Figure 6, the learning speed of DPD3 QN is noticeably faster than the other two methods. By approximately the 1200 th iteration, the mean reward of DPD3 QN exceeds that of the other two approaches and sustains this lead until training concludes. By the conclusion of 5000 rounds, the average rewards for DQN, DDQN, and DPD3 QN are 12.48, 12.92, and 13.35, respectively. This improvement can be attributed to the dueling network architecture in DPD3 QN, which accurately assesses the value of the selected action at each step, enabling the autonomous vehicle to identify better decision-making strategies more efficiently.  
Figure 7 illustrates the driving distance for each round throughout the training phase for the three decision-making approaches. The driving distance is mainly affected by collision conditions and vehicle speed. A greater driving range allows autonomous vehicles to maintain high speeds for extended durations without incidents. During the training process, the driving distance of the autonomous vehicle will vary due to the unpredictable initial speeds and positions of nearby vehicles. In the first 800 rounds, the driving distances of the three decision-making methods are similar, and the difference is not obvious. After 800 rounds, the driving distance achieved by the DPD3 QN decision-making approach progressively exceeded that of the other two methods, and this pattern persisted until the conclusion of training. At the end of 5000 rounds, the driving distances of DQN, DDQN, and DPD3 QN were 551 m, 622 m, and 694 m, respectively. Although the performance of the three methods in the exploration phase is similar, DPD3 QN's dual-priority experience replay can identify and utilize key experiences faster, thereby accelerating the learning process and making better choices. This mechanism significantly improves the response  
Figure 6: Average rewards of DPD3 QN, DDQN, and DQN.  
ability of autonomous vehicles to environmental changes and verifies the superiority of this method.  
Figure 8 evaluates the safety performance of the three algorithms through episode duration analysis, where longer operational periods reflect enhanced driving stability, DPD3 QN recorded the highest average step count at 27.34 per episode, surpassing DDQN (24.66) and DQN (22.79). Notably, episodes exceeding 25 steps signify robust long-distance safety, with DPD3 QN achieving 2945 instances beyond 27 steps--substantially higher than DDQN (1709) and DQN (474). This demonstrates DPD3 QN's superior collision avoidance capability, highlighting its safety advantages over baseline methods. Given the critical role of operational reliability in autonomous systems, the DPD3 QN-based decision model exhibits stronger practical viability for real-world deployment.