## 5 Discussion  
In this study, we proposed a reinforcement learning (RL)-based path-planning approach for a robotic manipulator in dynamic obstacle-avoidance tasks, integrating inverse kinematics (IK) with a dynamic exploration guidance mechanism. Path-planning under dynamic conditions involves two fundamental challenges: collision-free navigation around obstacles and successful arrival at designated targets. Due to the large and continuous nature of the state space in dynamic environments, traditional reinforcement learning approaches often require significant computational resources and extensive training iterations to deduce the state-action relationships effectively.  
To overcome these challenges, we introduced a novel hybrid action formulation that combines the deterministic velocity output from an IK module with an adaptive compensatory action generated by a deep reinforcement learning (DRL) policy network. Specifically, this compensatory action dynamically guides the manipulator to effectively avoid obstacles while maintaining goal-directed motion. Furthermore, our method incorporates a dynamic scaling factor to adaptively modulate exploration intensity. This scaling factor rapidly increases at the early training stages, promoting effective exploration and preventing inefficient exploration, and gradually slows down in later stages to refine the manipulator's movements and decision-making precision. Additionally, the designed adaptive reward mechanism derived from an energy function further supports effective policy training by improving the exploration-exploitation balance.  
The effectiveness of our proposed method was clearly demonstrated through rigorous comparisons with traditional DRL algorithms. The experimental results indicate that our improved SAC algorithm achieved an average success rate of 86.65%, significantly higher than the 75.25% success rate of traditional SAC. Likewise, our improved DDPG method reached a success rate of 80.56%, markedly surpassing the traditional DDPG's 71.33%. Moreover, the improved algorithms exhibited considerably lower reward and step variances, implying more stable and robust performance. Specifically, the improved SAC and DDPG recorded reward variances of 4.1266 and 6.6281, respectively, compared to their  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Policy** & **Average Success Rate** & \begin{tabular}{c} **Average** \\ **Reward** \\ \end{tabular} & \begin{tabular}{c} **Reward** \\ **Variance** \\ \end{tabular} & \
\begin{tabular}{c} **Average** \\ **Step** \\ \end{tabular} &
\begin{tabular}{c} **Step** \\ **Variance** \\ \end{tabular} \\ \hline Traditional SAC & 75.25 & 10.0673 & 8.4654 & 28.24 & 20.55 \\ Improved SAC & 86.65 & 11.6967 & 4.1266 & 21.58 & 8.27 \\ Traditional DDPG & 71.33 & 9.1696 & 11.8709 & 30.25 & 27.57 \\ Improved DDPG & 80.56 & 10.8799 & 6.6281 & 24.58 & 11.90 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Test results.  
traditional counterparts (8.4654 and 11.8709). Step variances also decreased significantly from 20.55 to 8.27 (SAC) and from 27.57 to 11.90 (DDPG).  
The superior performance of the proposed approach compared to traditional methods can be attributed primarily to the underlying strategy of transforming an infinite or extensive state space into a manageable finite subset. Traditional reinforcement learning methods often suffer from inefficient convergence due to their exhaustive exploration of irrelevant states, which contributes to higher variance and reduced success rates. In contrast, our dynamic exploration guidance, combined with adaptive reward-shaping and IK-based policy refinement, significantly narrows the exploration scope. This targeted approach not only accelerates convergence but also ensures more informed and precise decision-making, substantially enhancing the robot's real-time path-planning capabilities.