## 5 Simulation Analysis
### Scenario 1  
To verify the enhanced effect of combining reinforcement learning DDPG with a linear active disturbance rejection controller in terms of disturbance suppression capability and control accuracy, the position and attitude of the underwater robot are tracked under time-varying external disturbances. The transient performance of the control system under perturbations is evaluated to validate the disturbance rejection and robustness of the DDPG \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Parameter** & **Values** & **Parameter** & **Values** \\ \hline \(m\) & 197 kg & \(Y_{\eta|v|}\) & \(-245.2\) N\({}^{2}\)/m\({}^{2}\) \\ \(I_{z}\) & 25.1 Nms\({}^{2}\) & \(Z_{w}\) & \(-12.6\) N/S \\ \(X_{u}\) & \(-5.24\) N/S & \(Z_{w}\) & \(-367.8\) kg \\ \(X_{u}\) & \(-135.1\) kg & \(Z_{w|w|}\) & \(-547.4\) N\({}^{2}\)/m\({}^{2}\) \\ \(X_{u|u|}\) & \(-109.1\) N\({}^{2}\)/m\({}^{2}\) & \(N_{r}\) & \(-1.52\) N/S \\ \(Y_{\eta}\) & \(-11.1\) N/S & \(N_{r}\) & \(-34.3\) kg \\ \(Y_{\tilde{v}}\) & \(-390.6\) kg & \(N_{r|r|}\) & \(-26.2\) N\({}^{2}\)/m\({}^{2}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: ROV model parameters. Figure 7: Underwater robot (ROV) physical prototype, 3D arrangement of thrusters. LADRC control scheme. Disturbances are introduced during the movement of the ROV as follows:  
\[ \begin{array}{c}f=\left[f_{x},f_{y},f_{z},f_{\psi}\right]^{T}\\ =\left[20 sin(0.4*t)\ 20\ sin(0.4*t)\right.\\ 20\ sin(0.4*t)\ \ 20\ sin(0.4*t)]^{T}\end{array}  \tag{47}\]  
The initial conditions for the underwater robot are set as \([X(0),Y(0),Z(0),\)\(\mathit{Phi}(0)]=0\), with the velocity and angular velocity set as \(u(0)=\
u(0)=w(0)=r(0)=0\). Additionally, for the controller parameters, the PID parameters are set as:  
\[\left\{ \begin{array}{l}K_{p}=\left\{150,150,300,370\right\}\\ K_{i}=\left\{15,15,60,15\right\}\\ K_{d}=\left\{300,300,150,300\right\}\end{array} \right. \tag{48}\]  
The parameters for the Active Disturbance Rejection Control are set as follows: \(b_{0}=10\), \(w_{0}=5\), because \(\beta_{1}=3\omega_{o}\), \(\beta_{2}=3\omega_{o}^{2},\beta_{3}=\omega_{o}^{3}\) which means \(\beta_{1}=15\), \(\beta_{2}=75\), \(\beta_{3}=125\). The relevant DDPG setting parameters are shown in Table 2 above. The underwater robot simulation is designed to run for 100 s, with a simulation step size of 0.01 s. The proposed control algorithm is mainly compared with PID and LADRC under fixed parameters through three-dimensional trajectory tracking, and planar tracking, to verify the degree of improvement in the system's transient performance by the DDPG-LADRC control strategy. The trajectory tracking curve in the inertial coordinate system is:  
\[\left\{ \begin{array}{l}x_{d}=2 sin(0.1\pi t)\ \mathrm{m}\\ y_{d}=2 cos(0.1\pi t)\ \mathrm{m}\\ z_{d}=0.2t\ \mathrm{m}\\ \psi_{d}=0.03\pi t\ \mathrm{rad}\end{array} \right. \tag{49}\]  
First, a feasibility analysis of the parameter optimization for LESO is conducted. Figure 8 compares the observation errors of the LESO optimized by DDPG with those of the fixed-parameter LESO. It can be observed that the fixed-parameter LADRC controller is not precise in tracking total disturbances. In contrast, the DDPG-LADRC can maintain better performance with a shorter time under the constraints of model parameter uncertainty and strong unknown external disturbances in underwater robot trajectory tracking control. The DDPG-LADRC can quickly respond to changes in disturbances and adjust its control strategy promptly to adapt to these changes, thereby enhancing the system's dynamic performance. This indicates that the optimized observer parameters of DDPG-LADRC are effective. Figure 8: Comparison of total disturbance observation errors of two types of observers. The three-dimensional trajectory tracking performance of the ROV under different control schemes, as well as the tracking curves in the XY, XZ, and YZ planes shown in Figure 9, can be observed.