## 4 Experiments And Analysis
### Algorithm Performance Evaluation  
The reward function parameters employed in the experiments are presented in Table 2.  
The experiments involved comparing the traditional DDPG and SAC algorithms with two improved versions of these training algorithms, incorporating a dynamic policy selection mechanism and adaptive rewards. Figure 7 illustrates the growth curve of reward values over 5000 episodes for the traditional algorithms, as well as the corresponding performance of the improved algorithms in Environment 1. The improved algorithms typically converged at around 3000 episodes, whereas the traditional algorithms generally took approximately 4000 episodes to converge. In Environment 2, the introduction of randomly moving obstacles complicated the training environment. To ensure that the trained policy models could handle nearly all situations, the number of training episodes was increased \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Parameter** & **Value** & **Parameter** & **Value** \\ \hline \(\delta\) & 0.1 & \(d_{b}\) & 0.2 \\ \(\rho\) & 2 & \(k_{o}\) & 5 \\ \(\eta\) & 0.05 & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Parameters of the reward function. Figure 5: Network connections in the simulation framework. Figure 6: Diagram of the experimental scene. to 10,000. The resulting reward curves, as shown in the figure, indicate that the improved SAC algorithm began to converge around 6000 episodes, while the traditional algorithms generally converged at around 7000 episodes. Evidently, the improved algorithms exhibited significant advancements over the traditional algorithms. The results presented in the figure demonstrate that the dynamic exploration guidance mechanism and adaptive rewards contribute positively to the convergence speed and stability of the algorithms. To evaluate the model's adaptability to complex environments, we conducted tests within a simulated setting, specifically utilizing the more intricate Environment 2. Figure 8 illustrates the complete process of dynamic obstacle avoidance while reaching the target within a single episode. In environment 2, although the obstacles were designed to move randomly during the training phase to introduce dynamic complexity, they were configured to move along predefined straight-line trajectories during the testing phase (as shown in Figure 9). This setup was intentionally chosen to control variables and ensure consistent environmental conditions across different algorithms, thereby enabling a fair and direct performance  
Figure 8: The entire path-planning and dynamic obstacle-avoidance process of the manipulator. (**a**–**f**) illustrate key stages of the task: (**a**) Initialization of the manipulator; (**b,c**) The manipulator moves toward the target while detecting the surrounding environment; (**d,e**) The manipulator encounters the dynamic obstacle and adjusts its path to avoid it; (**f**) The manipulator successfully reaches the target. Figure 7: Comparison of four algorithms in (**a**) Environment 1 and (**b**) Environment 2.  
comparison. Since both the initial point and the target point were located on the tabletop surface, and the reward function encourages the robot to complete the task in as few steps as possible, the motion of the robot's end-effector showed only minor variation along the z-axis and was primarily confined to the x-y plane. We used the four algorithms to perform multiple simulations in the same dynamic simulation environment. The trajectory diagrams obtained from the experiment are shown in Figure 9. In this figure, a blue sphere is used to denote the starting point, while the red five-pointed star represents the end point. The robot's motion trajectory is depicted in blue, and the motion trajectories of the obstacles are shown in red and pink. It is evident that the robot avoided collision with obstacles throughout its movement.