# PDDQN-HHVBF Routing Protocol Based on Empirical Priority DDQN to Improve HHVBF
## 5 Performance Analysis and Simulation
### Comparative Analysis of Simulation Results
The number of network nodes is 500-3000, and the nodes' moving speed is fixed at 1 m/s. In Figure 16, VBF protocol PDR is very low. The reason is that the failure to find a suitable relay node leads to packets loss, and the performance of the VBF protocol is still the worst even when the number of nodes increases. The HHVBF protocol performs much better than the VBF protocol, but still much worse than those protocols using reinforcement learning to find the optimal relay nodes. Comparing the other protocols that use reinforcement learning to find the optimal nodes in sparse M-UWSNs, the PDDQN-HHVBF protocol performs best. The reason is that the proposed protocol can find the optimal nodes according to the in-time number of candidate nodes, but the two protocols use historical Q value to find the relay nodes, which will be moved by ocean current movement. In addition, the proposed protocol also use the \"Store-Carry-Forward\" mechanism, which makes the data packets transmitted by AUV store the data until a suitable relay node is found and forward it if there is no suitable relay node, which reduces the probability of packets loss. When the number of nodes in M-UWSNs is greater than 2500, the PDR of three protocols which use reinforcement learning reach 100%. Therefore, these using reinforcement learning protocols have greater performance in PDR under large-scale M-UWSNs. Figure 17 shows the energy tax trends of the five protocols as the number of nodes increases. When the number of nodes is sparse, the energy tax of the VBF protocol is very large. The reason is that the failure to find a suitable relay node leads to multiple re-transmissions and wastes energy. Even when the number of nodes increases, the performance of the VBF protocol is still the worst. The HHVBF protocol performs much better than the VBF protocol when the number of nodes is small. However, when the number of nodes is greater than 2000, the energy tax performance of the HHVBF protocol begins to increase. This is due to the fact that the number of nodes is too large and the packets is collided and re-transmitted many times. Comparing those use RL protocols, the proposed protocol performs best in sparse network. However, when the number of nodes is greater than 2500, the performance of these three protocols in PDR is very close. The reason is that they all consider energy consumption of nodes. Different from the compared protocols use  
Figure 16: The PDR of VBF, HHVBF, DQELR, ROEVA, and PDDQN-HHVBF protocol in different number of network nodes. the historical Q value, the PDDQN-HHVBF protocol considers the real-time Q value related to the the geographical location information, which will allow packets to be propagated over short distances in M-UWSNs. Thus, in the PDDQN-HHVBF protocol, re-transmits data packets fewer times due low packets collision. Therefore, it consumed less energy to forward data packets. Figure 18 shows the end-to-end delay of the five protocols as the number of nodes increases. The energy tax of the VBF protocol and HHVBF are very large both in the sparse network and the large-scale network. The reason is that the failure to find a suitable relay node leads to multiple re-transmissions and wastes time in sparse network. In addtion, the two protocols will happen more packets collisions compared others. About these use reinforcement learning protocols, the proposed protocol performs worse than compared two protocols in end-to-end delay. This is because that the PDDQN-HHVBF protocol needs to query the Q value of candidate relay nodes, which leads to spending extra time in Q value request and reply, so the delay is higher than the DQELR and the ROEVA protocol. Although the PDR of the proposed protocol is high and the number of re-transmissions is the least, it takes extra time to make the delay higher than the other two protocols in each forwarding. The Q values of the DQELR and the ROEVA protocol consider the link delay, these two protocols perform best in the delay indicators.