# Comparison of Various Reinforcement Learning Environments in the Context of Continuum Robot Control
## 4 Conclusions  
In this study, we have explored the use of RL algorithms in controlling a three-section planar continuum robot. By varying the reward functions, we have compared the performance of different RL environments, providing a concrete example of how the choice of reward function can impact the performance of the RL algorithm in this specific context. While the reward functions we used are specific to this problem, we believe that our findings provide valuable insights for researchers and practitioners working on similar problems in the field of robotics and control systems. The nonlinear aspect of robot kinematics is accounted for by the presented DDPG algorithm, which relies heavily on the reward function. Reward functions do not need to be complex to effectively represent the objectives of the task at hand. Specifically, we found that reward functions based on scaling distance led to good performance indices for the continuum robot, even without the need for complex formulations. This highlights the importance of carefully designing the reward function in RL environments for the effective control of continuum robots. The environment of continuum robots is described in depth, and a framework based on forward and velocity kinematics is used to describe the continuum robot environment. Overall, the paper describes a promising method for controlling continuum robots and emphasizes the potential of RL algorithms in this field. This example demonstrates continuum robot control for a small area, which has the potential to be applied to trajectory tracking, high-level control (such as obstacle avoidance), or moving-target state tasks with additional extensions.  
Figure 9: The mean of rewards during the training for all reward types.  
Looking ahead, there are several promising directions for future research. First, while our study focused on the impact of different reward functions, future research could explore the effects of other aspects of the RL environment, such as the state representation or the choice of RL algorithm. Second, our study used a specific type of continuum robot, the three-section planar robot. Future studies could investigate whether our findings generalize to other types of continuum robots, such as testing our RL algorithm on a 3D continuum robotic system. In addition, we plan to explore the effects of different types of noise on the performance of the RL algorithm. In this study, we used the Ornstein-Uhlenbeck process for generating noise. However, in future work, we aim to compare this with mean-zero Gaussian noise to understand the impact of different noise processes on the learning and performance of the RL algorithm. Next, while we adopted the hyperparameters from a previous study, an interesting direction for future research would be to conduct more extensive hyperparameter tuning to determine if improvements in performance can be achieved. Finally, the paper focused on comparing different RL environments, but we acknowledge the potential value in comparing the RL algorithm with traditional algorithms such as PID and predictive control.  
In conclusion, our study contributes to the growing body of research on the application of RL in the control of continuum robots, and we hope that it will inspire further research and practical applications in this area.  
Conceptualization, T.C.K. and J.K.; methodology, T.C.K. and J.K.; software, T.C.K.; validation, T.C.K.; writing--original draft preparation, T.C.K.; writing--review and editing, J.K.; project administration, J.K. All authors have read and agreed to the published version of the manuscript.  
This research was funded by the Ministry of Education and Science, grant number 0211/SBAD/0122.  
Not applicable.  
The authors declare no conflict of interest.