# Velocity Control of a Multi-Motion Mode Spherical Probe Robot Based on Reinforcement Learning
## 4 Simulations
### Simulation Results and Analysis
#### 4.2.2 Rapidly Brake  
Postures  
Three distinct initial states of the robot are set to simulate its various random postures after landing fully. In these states, the robot's directions are consistent with the forward movement direction (north) but with acceleration in different directions: northwest, north, and northeast (as shown in Figure 9). These three acceleration directions represent that the robot may deviate in different directions after landing, providing a more comprehensive scenario for algorithm validation. In the first scenario, related parameters are set as: \(v_{x1}=0.24\) m/s, \(v_{y1}=0\) m/s, \(\alpha_{1}=0.1\) rad, \(\beta_{1}=0.2\) rad, and the remaining parameters are zero. In the second scenario, parameters are set as: \(v_{x2}=0.24\) m/s, \(v_{y2}=0.24\) m/s, \(\alpha_{2}=0.1\) rad, \(\beta_{2}=-0.2\) rad, and the remaining parameters are zero. In the third scenario, parameters are set as: \(v_{x3}=0\) m/s, \(v_{y3}=0.24\) m/s, \(\alpha_{3}=-0.1\) rad, \(\beta_{3}=-0.2\) rad, and the remaining parameters are zero. The  
Figure 9: Three initial scenarios of MMSPR after landing: (**a**) the first scenario; (**b**) the second scenario; (**c**) the third scenario. desired values for the speed of the MMSPR are set as: \(v_{xd}=v_{yd}=0\) m/s. The results of each control method under different states are shown in Figures 10 and 11. To more intuitively compare the effects of various control methods, this section will mainly analyze the simulation results of the first scenario. As seen in Figure 10a, there is a substantial overshoot during the convergence with PID. The convergence time of \(v_{x}\) and \(v_{y}\) is 0.04 s and 0.01 s, respectively, which are slower than the ARLPID and RLPID methods using variable parameters. With RLPID, the speed error converges more rapidly, requiring only 0.01 s for both \(v_{x}\) and \(v_{y}\) to converge. However, the robot's speed fluctuates significantly when disturbances occur. Under SMC, the robot's speed changes relatively smoothly, but it has the longest convergence time, with \(v_{x}\) and \(v_{y}\) taking 1.04 s and 0.535 s,  
Figure 10: Velocity variation of MMSPR under different scenarios: (**a**) the change in velocity in the first scenario (with external interference); (**b**) the change in velocity in the second scenario (with external interference); (**c**) the change in velocity in the third scenario (with external interference). respectively. Additionally, there is a periodic speed fluctuation during the convergence. Compared with the above methods, the overshoot under ARLPID is reduced by about 81% compared to the PID method, eliminating the speed fluctuation observed in RLPID. This method allows \(v_{x}\) and \(v_{y}\) to converge quickly within 0.03 s and 0.01 s, respectively, and shows strong resistance to disturbances. The convergence times for other scenarios are shown in Table 4.  
Figure 11 shows the trajectories of different control methods during braking under different scenarios. In the first scenario, the braking displacement of the robot under PID is longer, with 0.0021 m, compared to that of RLPID and ARLPID, both of which employ variable parameters. The braking displacement under RLPID is 0.0013 m. However, the robot's speed is not completely stable when suffering disturbances from the external environment, resulting in substantial fluctuation in its movement trajectory. The robot's trajectory under ARLPID alters relatively smoothly, with a braking displacement of 0.0013 m, the smallest of \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Scenarios** & **Velocity** & **PID** & **RLPID** & **ARLPID** \\ \hline \multirow{2}{*}{The first scenario} & \(v_{x}\) & 0.04 s & 0.01 s & 0.03 s \\  & \(v_{y}\) & 0.01 s & 0.01 s & 0.01 s \\ \hline \multirow{2}{*}{The second scenario} & \(v_{x}\) & 0.035 s & 0.01 s & 0.025 s \\  & \(v_{y}\) & 0.00 s & 0.00 s & 0.00 s \\ \hline \multirow{2}{*}{The third scenario} & \(v_{x}\) & 0.035 s & 0.01 s & 0.025 s \\  & \(v_{y}\) & 0.01 s & 0.08 s & 0.01 s \\ \hline \hline \end{tabular}
\end{table}