## 5 Conclusions and Discussion  
In this work, we introduce DDPG-DMP, a novel and effective RL approach that employs a DMP to generate robot control actions. Unlike traditional RL methods that use neural networks to directly output control signals, our method generated proper robot control trajectories with consideration of the velocity and acceleration information, resulting in smooth and effective robot control behaviors. Compared to a related RL approach using DMP, DDPG-DMP leveraged the off-policy actor-critic framework for enhanced learning efficiency and achieved superior feasibility across a wide range of control signals (e.g., torque, joint angle, and end-effector position) without requiring additional human knowledge by proposing an adaptive inverse controller. The experimental results of robot manipulation tasks on OpenAI Gym and Panda simulation demonstrated significant advantages in control performance, learning efficiency, and smoothness of robot actions over related baselines, indicating the potential of our method in the field of robot control.  
However, the current DDPG-DMP had limitations in fully utilizing trajectory information. Beyond accumulating the rewards over a \(K\)-step trajectory, it lacked a mechanism to fully consider the entire trajectory's states and actions (as discussed in Section 3.3). This presents a crucial direction for our future work. We also plan to explore integrating the DDPG-DMP framework with state-of-the-art model-free RL methods such as Continuous Dynamic Policy Programming (CDPP) [18], TD7 [38], and CrossQ [39]. Furthermore, incorporating DMP-based policies into model-based RL methods like Dropout-based Probabilistic Ensembles with Trajectory Sampling (DPETS) [40] and optimizing the initial trajectories of its model predictive controller offers an interesting research direction. From the perspective of engineering implementation, we aim to investigate the potential of DDPG-DMP in various real-world scenarios, including real physical systems such as Panda and UR5 robot arms, quadruped robots, and dexterous hands.  
Conceptualization, S.L. and Y.C. ([PERSON]); methodology, S.L., W.H., C.M. and Y.C. ([PERSON]); software, S.L., W.H. and C.M.; validation, T.S. and Y.C. ([PERSON]); formal analysis, S.L. and Y.C. ([PERSON]); investigation, S.L.; writing--original draft preparation, S.L. and Y.C. ([PERSON]); writing--review and editing, Y.C. ([PERSON]); visualization, S.L. and W.H.; supervision, Y.C. ([PERSON]), T.S. and Y.C. ([PERSON]); project administration, K.X., Y.C. ([PERSON]), T.S. and Y.C. ([PERSON]); funding acquisition, Y.C. ([PERSON]), K.X. and Y.C. ([PERSON]). All authors have read and agreed to the published version of the manuscript.  
This research was supported in part by the National Natural Science Foundation of China under Grants 62103403; in part by Major Program of Science and Technology of Shenzhen Grant No.KJZD20231023100304010 and No. KJZD20231023100509018; and in part by the Key Program of Natural Science Foundation of Shenzhen Grant No. JCYJ20220818101406014.  
**Informed Consent Statement**: Not applicable.  
Data Availability Statement: The raw data supporting the conclusions of this article will be made available by the authors on request.  
Conflicts of Interest: Author [PERSON] was employed by the company Shenzhen ZHAOWEI Machinery & Electronics Co., Ltd. The remaining authors declare that the re-search was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.