# Autonomous Obstacle Avoidance with Improved Deep Reinforcement Learning Based on Dynamic Huber Loss
## 5 Conclusions  
This paper proposes an improved DRL framework based on Dynamic Huber loss for UAV obstacle avoidance in dynamic, unknown environments. It is theoretically demonstrated that compared with MSE, Smooth L1 loss, and Huber loss, the Dynamic Huber loss can bring faster convergence speed and higher training accuracy for DRL. Simulation experiments show that during training, Dynamic Huber loss offers significant advantages in stability, convergence, and extreme value control, providing a more stable and reliable optimization direction for model training. At the same time, Soft update allows the UAV to maintain a relatively stable flight due to the gentle update of parameters. In the testing phase, different loss functions are evaluated concerning mission planning success rate and average flight paths, and it is found that the algorithm based on Dynamic Huber loss performs well in both static and dynamic environments. In a static environment, the success rate is as high as 98.85%, and the average path length is 10.73, which can effectively plan the path, avoid obstacles, and reach the target quickly; in a dynamic environment, the success rate is as high as 74.20%, and the average path length is 37.04, which significantly improve the success rate and efficiency of path planning. In dynamic environments that incorporate wind disturbances, the success rate is as high as 70.95%, and the average path length is 40.04, highlighting its anti-interference ability. Meanwhile, compared to Hard updates, Soft updates can track dynamic changes in real time, optimizing decision-making. In summary, the improved DRL framework based on Dynamic Huber loss proposed in this paper has obvious advantages in UAV obstacle avoidance performance. It provides a reliable and efficient solution for UAV path planning.  
Conceptualization, X.L. and X.X.; methodology, X.X.; software, D.Z. and X.L.; validation, C.C., X.L., and X.X.; formal analysis, X.X. and N.C.; resources, N.C.; data curation, N.C. and D.Z.; writing--original draft preparation, X.X.; writing--review and editing, X.X. and X.L.; visualization, X.X. and D.Z.; supervision, C.C.; funding acquisition, X.L. and C.C. All authors have read and agreed to the published version of the manuscript.  
This work was supported by the Shandong Provincial Natural Science Foundation (ZR2021 QF038).  
Not applicable.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Environment** & **Update Method** & **MSE** & **Smooth L1 Loss** & **Huber Loss** & **Dynamic Huber Loss** \\ \hline Static & Soft & 0.10\% & 97.40\% & 98.70\% & 98.85\% \\ \hline \multirow{3}{*}{Dynamic} & Hard & 59.40\% & 60.75\% & 65.90\% & 67.90\% \\  & Soft & 66.90\% & 65.80\% & 70.15\% & 74.20\% \\ \cline{1-1}  & Wind Soft & 61.95\% & 64.35\% & 67.60\% & 70.95\% \\ \hline \hline \end{tabular}
\end{table}
Table 4: Success rate of UAV obstacle avoidance test.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Environment** & **Update Method** & **MSE** & **Smooth L1 Loss** & **Huber Loss** & **Dynamic Huber Loss** \\ \hline Static & Soft & 408.92 & 12.34 & 11.17 & 10.73 \\ \hline \multirow{3}{*}{Dynamic} & Hard & 54.39 & 49.29 & 44.78 & 42.39 \\  & Soft & 43.22 & 44.15 & 40.47 & 37.04 \\ \cline{1-1}  & Wind Soft & 47.96 & 45.37 & 43.25 & 40.04 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Average path lengths of UAV obstacle avoidance tests.  
Data Availability Statement: The data presented in this study are available on request to [EMAIL_ADDRESS]. The data are not publicly available due to enterprise cooperation needs.  
Thanks are extended to the editors and reviewers. Thanks to [PERSON] for contributing to the article.  
The authors declare no conflicts of interest.