# Autonomous Obstacle Avoidance with Improved Deep Reinforcement Learning Based on Dynamic Huber Loss
## 4 Experimental Setup and Simulation Analysis
### Analysis of Training Results  
The training outcomes of DRL, using various loss functions and update methods, are analyzed to evaluate the performance of DRL in solving the UAV obstacle avoidance problem. Figure 4 demonstrates the training process's reward variation of different loss functions (MSE, Smooth L1 loss, Huber loss, Dynamic Huber loss). Figure 4a shows the reward changes for Soft-update-based UAV obstacle avoidance training in a static environment, with significant fluctuations in all curves in the figure. Still, it can be observed that the fluctuation of Dynamic Huber loss is relatively small; especially in the 0-500 round interval, the MSE and Smooth L1 loss reward value fluctuation is obvious, while Dynamic Huber loss is gentler. From 1500 to 2000 rounds, the fluctuation range of Dynamic Huber loss is gradually narrowed and tends to converge. Figure 4b shows the reward variation of UAV obstacle avoidance training based on the Hard update in a dynamic environment. At the early stage of training (0-250 rounds), the fluctuation of MSE, Smooth L1 loss, and Huber loss reward values are more drastic, while Dynamic Huber loss is relatively small and the model training process is smoother. From the entire training process, the fluctuations of the first three are consistently large, while Dynamic Huber loss gradually stabilizes after 1000 rounds, and the range of fluctuations narrows after 1500 rounds, with a clear convergence trend; additionally, its lower limit is high (seldom lower than \(-200\)), and high reward values are reached frequently in the later stages. Figure 4c shows the reward variation of UAV obstacle avoidance training based on the Soft update in a dynamic environment, Dynamic Huber loss is smaller than other functions in fluctuation amplitude, and there is a tendency of convergence after 1000 rounds. Figure 4d shows the reward variation in a dynamic environment with wind interference, which increases the complexity and uncertainty of UAV obstacle avoidance. In this context, Dynamic Huber loss demonstrates an advantage by flexibly adjusting the gradient to handle errors \begin{table}
\begin{tabular}{c c c} \hline \hline
**Serial Number** & **Main Parameters** & **Measure Value** \\ \hline
1 & Discount factor & 0.99 \\
2 & Learning rate & 0.001 \\
3 & Batch size & 128 \\
4 & Maximum number of rounds & 2000 \\
5 & Input neuron & 4 \\
6 & Output neuron & 4 \\
7 & Full connectivity layer & 4 \\
8 & Activation function & Relu, Linear \\ \hline \hline \end{tabular}
\end{table} 
Table 2: DRL parameter settings. Figure 3: Sensitivity analysis of parameters (**(a)** is a sensitivity analysis for learning rates, **(b)** is a sensitivity analysis for discount factors, and **(c)** is a sensitivity analysis for batch sizes). caused by wind interference. Early in training, a larger threshold increases the gradient, allowing the model to quickly reduce larger errors. As training progresses, the threshold decreases, ensuring stable convergence near the optimal solution. In contrast, other loss functions experience greater fluctuations and slower convergence due to their less flexible error handling. Overall, Dynamic Huber loss offers significant benefits in stability and convergence, providing a more stable optimization path that allows the model to learn effective strategies faster and improve obstacle avoidance performance in complex tasks. The UAV obstacle avoidance training reward variation of different updating methods is shown in Figure 5. In UAV obstacle avoidance training experiments using different loss functions, the Soft update method demonstrates several significant advantages. In the MSE experiments, the Soft updating method has less fluctuation in the reward value curve, which ensures the training is stable and accelerates the convergence speed, which is conducive to the UAV making better obstacle avoidance decisions continuously.