# Autonomous Obstacle Avoidance with Improved Deep Reinforcement Learning Based on Dynamic Huber Loss
## 3 A DRL Framework Based on Dynamic Huber Loss and Soft Updates
### Loss Function
#### 3.2.4 Dynamic Huber Loss Function
In contrast, the gradient of the traditional Huber loss function with a fixed threshold remains constant, making it less adaptive to varying error sizes, particularly in the presence of wind interference. As a result, the convergence of gradient descent may be hindered when using the fixed-threshold Huber loss function. According to the convergence theorem for gradient descent, the DynamicHuber loss function is more effective at accelerating convergence. It allows the UAV's control parameters to adjust more quickly, mitigating the impact of wind interference on the UAV's trajectory and enabling the UAV to converge to a state closer to the optimal trajectory more rapidly. A dynamical system is asymptotically stable if its Lyapunov function \(V(\epsilon)\) satisfies (1) \(V(\epsilon)\) is positive definite and (2) the derivative of \(V(\epsilon)\big{(}\dot{V}(\epsilon)\big{)}\) is negatively definite. In a loss function optimization problem, the loss function \(L\) can be considered as a Lyapunov-function-like quantity, and the gradient descent process can be viewed as a dynamical system. In the UAV obstacle avoidance scenario with wind interference, the Dynamic Huber loss function \(L_{DH}(\epsilon,t)\) is analyzed as a Lyapunov function. During the training process, the gradient of the loss function \(\
abla L_{DH}(\epsilon,t)\) changes continuously as the threshold \(\delta_{t}\) is dynamically adjusted. In the early stage of training, a larger threshold value allows the loss function to decrease rapidly when facing a large error caused by wind interference, i.e., \(\dot{V}(\epsilon)<0\), and the system quickly adjusts to the direction of reducing the error. In the late stage of training, a smaller threshold value makes the loss function change more gently in the small error region, which ensures the stability of the system. Due to the fixed threshold, the Huber loss function cannot be adjusted flexibly when dealing with different sizes of wind disturbances like the Dynamic Huber loss function, and it is difficult to meet the requirements of the rapid decline of large errors and stable adjustment of small errors at the same time. Therefore, according to the stability theory, the Dynamic Huber loss function can enable the UAV to optimize the trajectory more stably in the environment with wind interference, suppress the influence of wind interference, and improve the obstacle avoidance performance. The Adam optimizer updates the parameters of the neural network based on the gradient of the loss function through the first-order moment estimation and the second-order moment estimation to accelerate the convergence of the model and improve its stability. While the mean square error is sensitive to outliers and affects training, and the Smooth L1 loss unction has the problem of high fluctuation, the Huber loss function better combines the advantages of MAE and MSE and converges faster and more stably. The Dynamic Huber loss function has an outstanding advantage under Adam optimization, and it performs better in convergence speed and accuracy by dynamically adjusting the threshold value.