## 3 Method
### Reward Signal  
To address the issue of sparse rewards in DRL-based trajectory-planning for robotic arms within dynamic environments, this study introduces an adaptive reward function that seamlessly integrates existing reward mechanisms with energy function constraints, drawing on the principles of artificial potential field methods and energy functions. This adaptive function facilitates a smooth transition between the two reward mechanisms and carries out dynamic adjustments according to the distance between the end effector and the obstacle to ensure that the path-planning result is both safe and efficient:  
\[R_{ada}=\beta\cdot R_{t}+(1-\beta)\cdot R_{energy} \tag{7}\]  
\[R_{energy}=-\sum \begin{array}{c}6\\ i=1\end{array} \tanh(\parallel q\parallel^{2}) \tag{8}\]  
Specifically, \(\beta\in[0,1]\) is designed such that, when the end effector is sufficiently distant from obstacles, \(R_{t}\) serves as the dominant factor; when the distance between the end effector and an obstacle falls below the critical distance \(d_{b}\), the energy function takes precedence. \(\beta\) can be expressed as follows:  
\[\beta=\frac{\tanh(x)+1}{2} \tag{9}\]  
where  
\[x=\frac{d_{obs}-d_{b}}{k_{o}} \tag{10}\]  
It is evident that the \(\beta\) function yields a smooth transition from 0 to 1: when \(d_{obs}\) is large, \(\beta\) approaches 1; meanwhile, when \(d_{obs}\) is small, it approaches 0. Here, \(k_{o}\) serves as a tuning parameter that adjusts the smoothness of the reward mechanism. \(R_{t}\) is designed to prompt the robotic arm to accomplish the task, which mainly includes the following aspects:  
\[R_{t}=R_{\text{target}}+R_{\text{avoid}}+R_{\text{done}} \tag{11}\]  
The first term on the right-hand side of Equation (11) is designed to guide the end effector toward a specific target position. It is defined based on the difference in the end effector--target distance between the current step and the previous step:\[R_{\text{target}}= \begin{cases}d_{\text{tar}}-d_{\text{tar}}^{\prime}&\text{if $d_{\text{tar}}>\delta$}\\ (d_{\text{tar}}-d_{\text{tar}}^{\prime})\cdot\frac{\delta}{d_{\text{tar}}}&\text{ if $d_{\text{tar}}\leq\delta$}\end{cases}  \tag{12}\]  
In robotic arm path-planning tasks, we do not necessarily expect the end-effector to reach the designated location in a single step every time; instead, a gradual approach strategy is adopted. Specifically, the end effector is controlled to move incrementally at each step, gradually closing in on the target position. In designing the reward feedback, the distance between the end effector's current position and the target point is introduced as a key parameter. Compared with relying solely on a one-step Euclidean distance, this reward function may slow down the movement process, yet it effectively reduces control complexity and meets the path-planning objective under higher-precision requirements. Furthermore, in path-planning, precision and efficiency often impose mutual constraints. To enhance the reward mechanism's effectiveness, when the current state nears boundary conditions (e.g., when the distance to the target point is relatively small), each slight movement in the correct direction should receive a stronger positive reward. This approach guides the robotic arm to gradually refine its path, ultimately achieving high-precision control of the target position. The second term, \(R_{\text{avoid}}\), is the obstacle avoidance reward function designed to guide the robotic arm in evading obstacles:  
\[R_{\text{avoid}}=-\left(\frac{1}{1+d_{obs}}\right)^{\rho} \tag{13}\]  
Here, \(d_{obs}\) represents the smallest gap between the robotic arm and obstacles, and \(\rho\) denotes the decay rate of the reward function. The third term, \(R_{\text{done}}\), is the termination reward function, primarily designed to reward the achievement of the target and collisions with obstacles:  
\[R_{done}=\left\{ \begin{array}{cc}30&d_{\text{tar}}<\eta\\ -15&collision=True\end{array} \right.