## 4 Experimental Results
### Panda Robot Arm Scenarios  
In this subsection, we evaluate the proposed method by more complex robot arm control scenarios: PandaReach-V2 and PandaReachjoints-V2 on a Panda robot arm simulation environment panda-gym ([[https://github.com/qgallouedec/panda-gym](https://github.com/qgallouedec/panda-gym)]([https://github.com/qgallouedec/panda-gym](https://github.com/qgallouedec/panda-gym)), accessed on 15 November 2024). The first task directly controls the three-dimensional position of the end-effector via internal inverse kinematics. The second task is extended from the first one to further control the angle of each joint to further investigate the capability of the proposed method in the Panda robot arm. The reward is designed to reach the randomly generated end-effector position as closely as possible. The observation states of two tasks were designed as a 20-dimensional vector including the position and velocity of the end-effector. The learning curves of all compared approaches are illustrated in Figure 4.  
Table 4 compares the final converged results of all methods. We selected success rate as the evaluation metric following the original paper of panda-gym [37].  
In the PandaReach-V2 task, DDPG struggled to learn and exhibited an unstable learning process, evidenced by a high standard deviation in the success rate. TD3 learned at a nearly 100% success rate, albeit at a bit slower convergence. Among traditional RL methods, only SAC achieved quick and stable convergence. In terms of the approaches using DMP-based policy, the NDP completely failed to learn this task. As a comparison, the proposed method converged to an almost 100% success rate with efficiency second only to SAC. It is worth noting that while our method was slightly less efficient than the neural network policy-based SAC in this task where directly controlling the end-effector position created a relatively simple scenario, it demonstrated superior smoothness in robot trajectories when controlling robot arm joint angles. We explore this advantage in the next case.  
In the PandaReachJoints-V2 task, the agent attempts to move the end-effector to reach the randomly generated target position by controlling the seven joint angles in a Panda robot arm. This task implicitly requires learning the unknown inverse kinematics of a Panda robot, making it significantly more complex than the previous task. Such an increased challenge results in all comparison methods failing to reach the target successfully in the first 50,000 steps. Among all traditional RL approaches based on the neural network policy, the DDPG showed no signs of learning the task, even after about 200,000 interaction steps. TD3 slowly learned to a success rate of around 92%. SAC, despite its advantages in the previous task, underperformed TD3 in this task. It demonstrated good learning efficiency within the first 120,000 steps but turned to unstable afterward, finally converging to a low success rate (85%) with a high standard deviation. For the approaches using the DMP-based policy, the NDP continued to struggle in this task, while the proposed method demonstrated the significant advantages of DMP's smooth trajectories in both task success rate and learning efficiency. DDPG-DMP not only stably converged to a 99% success rate but also enjoyed markedly superior learning efficiency compared to other baselines. These results indicate that the adaptive inverse controller is crucial to employ a DMP-based policy into RL for general applications in robotic systems. Under the proper setting in DDPG-DMP, the DMP-based policy contributes to significant advantages in the learning efficiency.