# PDDQN-HHVBF Routing Protocol Based on Empirical Priority DDQN to Improve HHVBF
## 6 Conclusions  
In this paper, a routing protocol named PDDQN-HHVBF (Empirical Priority DDQN to Improve Hop-by-Hop Vector-Based Forward) protocol is proposed for M-UWSNs.The proposed protocol finds the optimal relay node to achieve energy saving, reliable data packet transmission and extend network lifetime. In order to reduce packets collision in large-scale M-UWSNs, the protocol uni-cast the packets with short propagation distance to the selected relay nodes calculated by convergent DDQN algorithm based on empirical priority, which converges quickly than DQELR about 30%. In addition, the proposed protocol select the optimal nodes, with max Q value of all candidate relay nodes, to forward. The optimal relay nodes with higher residual energy, shorter distance for transmission and more candidate relay nodes to forward have make higher PDR, lower energy consumed and longer M-UWSNs lifetime. Finally, the \"Store-Carry-Forward\" mechanism used in AUV can effectively reduce the energy consumed of AUV, and it also can progress the PDR of M-UWSNs. The experimental results show that the performance of the PDDQN-HHVBF protocol in PDR, energy tax, and lifetime are better than that of the relevant comparative protocols. However, in order to improve the reliability of data transmission and reduce the energy consumption, the PDDQN-HHVBF exchanges Q values for many times, so the protocol has a relatively long delay compared DQELR and ROEVA. In the future work, we will consider to reduce the delay of the proposed protocol by designing a MAC (Medium Access Control) protocol that is consistent with this routing protocol.  
Methodology, J.B. and Y.L.; Formal analysis, Y.C.; Investigation, Y.L.; Resources, Y.C.; Data curation, Y.L.; Writing--original draft, J.B.; Writing--review & editing, J.B.; Supervision, Y.L.; Funding acquisition, Y.C. All authors have read and agreed to the published version of the manuscript.  
This work was supported in part by the following projects: the National Natural Science Foundation of China through the Grants 61861014, Doctor start-up fund (BS2021025), the Guangxi Nature Science Fund (2015 GXNSFAA139298, 2016 GXNSFAA380226), Guangxi University high level innovation team and outstanding scholar program, Guangxi Science and Technology Project (AC16380094, AA17204086, 1598008-29), Guangxi Nature Science Fund Key Project (2016 GXNSFDA380031), and Guangxi University Science Research Project (ZD 2014146), Natural Science Foundation of Guangxi Province of China (2020 GXNSFAA159090).  
Data Availability Statement: Data available on request from the authors.  
The authors would like to thank the anonymous reviewers for their constructive comments, which greatly helped improve this paper.  
The authors declare no conflict of interest.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Network Performance Metrics** & **0 m/s** & **0.5 m/s** & **1 m/s** & **1.5 m/s** & **2 m/s** & **2.5 m/s** & **3 m/s** \\ \hline PDR (\%) & 100 & 100 & 92.5 & 84.6 & 76.4 & 72.4 & 61.2 \\ energy tax (J) & 355.2 & 395.4 & 427.3 & 496.4 & 558.2 & 608.4 & 687.9 \\ end-to-end delay (s) & 9.48 & 9.57 & 9.69 & 9.75 & 9.93 & 10.03 & 10.23 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Experimental results of different nodes speed with 1000 nodes.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Network Performance Metrics** & **500 Nodes** & **1000 Nodes** & **1500 Nodes** & **2000 Nodes** & **2500 Nodes** & **3000 Nodes** \\ \hline PDR (\%) & 58.2 & 92.5 & 94.8 & 100 & 100 & 100 \\ energy tax (J) & 750.4 & 355.6 & 255.2 & 140.4 & 178.4 & 218.4 \\ end-to-end delay (s) & 9.61 & 9.69 & 9.88 & 10.14 & 10.30 & 10.43 \\ lifetime (\%) & 1 & 1 & 1 & 0.95 & 0.91 & 0.88 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Experimental results of different number of node with 1 m/s.