## 2 Materials and Methods
### Reinforcement Learning in Continuous Environments
As original scientific contributions of this paper, the following are highlighted:

* Trajectory-level learning from a demonstration method based on spatial coordinate system classification. * A trajectory model for motion planning in Cartesian coordinate systems based on an optimization process with the ability to approximate coordinate systems and avoid obstacles. * A methodology for task-oriented iterative trajectory learning based on initialization through demonstration learning. Future research will focus on finding more efficient strategy search algorithms and exploring the possibility of applying other parameter strategies for motion modeling. Additionally, significant progress in the ease of applying strategy search algorithms can be achieved through research in the areas of automatic learning parameter tuning and automatic extraction of reward/goal functions from a small number of human demonstrations. This would significantly increase and simplify the applicability of such solutions in real-world applications. Another future development for the proposed approaches in this paper can be the transition from the existing designed simulated environment to the real-life robot manipulator in a physically designed industrial environment. Conceptualization, [PERSON] ([PERSON]) and L.G. ; methodology, [PERSON] ([PERSON]) and [PERSON] ([PERSON]); software, [PERSON] ([PERSON]); validation, L.G. ; resources, [PERSON] ([PERSON]); writing--original draft preparation, [PERSON] ([PERSON]); writing--review and editing, [PERSON] ([PERSON]) and L.G. ; supervision, [PERSON] ([PERSON]); project administration, L.G. All authors have read and agreed to the published version of the manuscript. This research received no external funding. Not applicable. The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding author. The authors declare no conflicts of interest. ## Abbreviations

The following abbreviations are used in this manuscript:

DH & Denavit-Hartenberg

DMP & Dynamic Movement Primitives

DOF & Degrees of Freedom

DQN & Deep Q-Network

DRL & Deep Reinforcement Learning

LbD & Learning by Demonstration

NN & Neural Network

ReLU & Rectified Linear Unit

RL & Reinforcement Learning

## References

* [PERSON] et al. (2021) [PERSON]; [PERSON] [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON] Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review. _Robotics_**2021**, _10_, 22. [CrossRef]
* [PERSON] et al. (2020) [PERSON]; [PERSON]; [PERSON] Pick and Place Objects in a Cluttered Scene Using Deep Reinforcement Learning. _Int. J. Mech. Mech. Mechatron. Eng._**2020**, _20_, 50-57. * [PERSON] et al. (2023) [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON]; [PERSON]; [PERSON] [PERSON]; [PERSON] [PERSON] Shared Control of Robot Manipulators with Obstacle Avoidance: A Deep Reinforcement Learning Approach. _IEEE Control Syst. Mag._**2023**, _43_, 44-63. [CrossRef]
* [PERSON] et al. (2021) [PERSON]; [PERSON]; [PERSON]; [PERSON] Crossing the reality gap: A survey on sim-to-real transferability of robot controllers in reinforcement learning. _IEEE Access_**2021**, \(9\), 153171-153187. [CrossRef]
* [PERSON] et al. (2019) [PERSON]; [PERSON]; [PERSON] Comparison of nature-inspired population-based algorithms on continuous optimization problems. _Swarm Evol. Comput._**2019**, _50_, 100490. [CrossRef]
* [PERSON] (2019) [PERSON]; [PERSON] Benchmarking evolutionary algorithms for single objective real-valued constrained optimization-a critical review. _Swarm Evol. Comput._**2019**, _44_, 927-944. [CrossRef]
* [PERSON] et al. (2020) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Optimal Trajectory Planning of Grinding Robot Based on Improved Whale Optimization Algorithm. _Math. Probl. Eng._**2020**, _2020_, 3424313. [CrossRef]
* [PERSON] et al. (2022) [PERSON]; [PERSON]; [PERSON]; [PERSON]; [PERSON] Optimal scheduling for palletizing task using robotic arm and artificial bee colony algorithm. _Eng. Appl. Artif.