# PDDQN-HHVBF Routing Protocol Based on Empirical Priority DDQN to Improve HHVBF
## 5 Performance Analysis and Simulation
### Comparative Analysis of Simulation Results  
In this part, we compare the PDDDQN-HHVBF protocol with the state-of-the-arts, which are VBF [12], HHVBF [13], DQELR [3], and ROEVA [29]. Firstly, we study the performance of DQELR and PDDDQN-HHVBF in the convergence speed as shown in Figure 11. In this experiment all nodes move randomly at the speed of 0-3 m/s under the influence of ocean current movement. As the number of iterations increases, the DQELR loss converges to about 16,000 rounds, while the convergence speed of PDDDQN-HHVBF proposed in this paper increases by about 30%, and has converged in 11,000 rounds. This is because PDDDQN-HHVBF uses the DDQN algorithm based on empirical priority, while DQELR is based on the DQN algorithm. The DDQN algorithm based on empirical priority can avoid the strong correlation between the data samples. It can also avoid the phenomenon of overfitting caused by over-concentration of samples sampled from the experience pool. Therefore, compared with the DQELR protocol, PDDDQN-HHVBF has less computational overhead and shorter network delay in network training. The random movement of nodes due to ocean current movement is a major cause of data packets loss in M-UWSNs, so we explore how the free movement speed of node will affect PDR, energy tax, end-to-end delay of the five algorithms as Figures 12-14. We fixed the network size as 1000 nodes (a relatively sparse network). In addition, in this environment, we will also compare the impact of the \"Store-Carry-Forward\" mechanism and the \"Send-Confirm\" mechanism used by AUV on the energy consumption of AUV Figure 15.  
with the historical largest Q value, but this node may becomes a routing hole due to the influence of ocean current movement. Hence, when the nodes moves fast (over 1 m/s), the PDR of the ROEVA is significantly lower than that of PDDQN-HHVBF protocol in this paper. About the DQELR protocol, although considering the movement of nodes, it still use historical Q value of the candidate relay nodes. Although the DQELR better than the ROEVA about PDR under this circumstance, it still worse than the PDDQN-HHVBF protocol. In addition, the other protocols compared with the PDDQN-HHVBF does not consider the situation of the AUV sending data packets. When the AUV cannot find the relay node, the data packet will be lost. The PDDQN-HHVBF protocol performs best in this experimental environment due to the consideration of the number of candidate relay nodes for relay nodes and the \"Store-Carry-Forward\" mechanism. Considering the number of candidate relay nodes, the data packets can be forwarded as far as possible to the node that can successfully forward the data packet to the Sink node and has the smallest probability of routing holes due to the movement of ocean currents. The \"Store-Carry-Forward\" mechanism enables the AUV to find a suitable relay node and avoids data packets loss from the AUV. We continue to explore how the mobility of nodes will affect the energy tax of the five algorithms. In the Figure 13, as the speed of node movement increases, the energy tax of the five algorithms also increases. This is because node mobility reduces the PDR, so more energy is spent to transmit successful data packets. In this experimental environment, the energy tax of VBF protocol and HHVBF protocol is significantly higher than that of DQELR protocol, the ROEVA protocol and the PDDQN-HHVBF protocol. This is because both VBF and HHVBF protocols use broadcast methods to send data packets, which will result in repeated reception of packets and data packets collision. In addition, these two methods do not consider the problem of data path, so data packet is often forwarded to the routing holes, resulting in packets loss and re-transmission consuming a lot of energy. About the ROEVA protocol, it also use broadcast methods to send data packets.