# RL-Based Sim2 Real Enhancements for Autonomous Beach-Cleaning Agents
## 6 Experiments and Results
### Second Experiment: Navigating with Obstacles in Simulated and Real Environments  
The second experiment addresses the challenge of the positional control of a mobile robot in an environment enriched with the presence of obstacles. This scenario places a target point one meter away, with the robot oriented in the opposite direction, as depicted in Figure 13, representing the simulated environment for this experimental phase. Unlike the first experiment, which was designed to assess the agents' prow  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**Real**} & \multicolumn{2}{c}{**Simulated**} \\ \cline{2-5}
**Index** & **DDPG** & **PPO** & **DDPG** & **PPO** \\ \hline ISE & **9.48** & 11.16 & **9.50** & 11.00 \\ IAE & **12.90** & 14.25 & **12.84** & 14.15 \\ ITSE & **57.64** & 70.84 & **59.03** & 72.03 \\ ITAE & **102.37** & 116.41 & **102.91** & 118.99 \\ \hline \hline \end{tabular} Bold values indicate the best performance (lower index indicates better performance).  
\end{table}
Table 1: Performance index comparison of the first experiment.  
without external complexities, this second scenario tests the agents' abilities to navigate around obstacles while maintaining effective control of their position.  
Figure 14 eloquently reveals the agents' skill in circumventing obstacles en route to the designated target. Here, a significant adaptation by the agents is observed, as they successfully navigate around the obstacles present in the scenario, a stark contrast to the obstacle-free environment of the first experiment.  
A systematic analysis of the performance indices in Table 2 provides key insights into the agents' competencies in steering a mobile robot in an obstacle-laden environment. The DDPG agent consistently outperforms the PPO agent across all measured indices in both simulated and real-world scenarios. Lower values indicate a higher level of precision and effectiveness in guiding the robot toward the target compared to the PPO. This advantage is particularly notable in the real-world environment, where obstacles add complexity to the navigation task.  
The performance indices not only affirm the efficacy of the DDPG algorithm but also highlight the reliability of the simulation environment for training RL agents in obstacle-involving situations. The superior performance of the DDPG agent suggests its potential for precise real-world robotic navigation applications, especially in environments where obstacles present navigational challenges. These findings further underscore the success  
Figure 14: Trajectory graph of the second experiment.  
Figure 13: Simulated environment of the second experiment.  
of the employed Sim2 Real transfer methodologies, reaffirming their utility in tackling complex navigational tasks in practical and cluttered environments.  
The outcomes of this experiment carry significant implications for the deployment of beach-cleaning robots. Unlike controlled laboratory settings, beach environments are dynamic and unpredictable, with variable elements such as wandering beachgoers, animals, and shifting terrain. The ability of the DDPG agent to adeptly handle these challenges in simulation is promising for the future application in beach-cleaning operations, where autonomous robots must operate effectively in the midst of such uncertainties. The experiment thus sets a precedent for the adaptability required in real-world beach-cleaning applications, illustrating the potential of RL-trained robots to contribute meaningfully to environmental preservation efforts.