# RL-Based Sim2 Real Enhancements for Autonomous Beach-Cleaning Agents
## 6 Experiments and Results
### Third Experiment: Path Following in Dynamic Beach Environments  
The third experiment scrutinizes the robot's ability to precisely follow a pre-established route delineated by a sequence of control points, mimicking a beach-cleaning course. This test scenario is pivotal as it mirrors the practical application of the robot in real-life conditions, where navigational efficiency and precision are paramount.  
Figures 15 and 16 visually demonstrate how the robot, under the guidance of reinforcement learning agents, adeptly maneuvers towards the control points, even amidst obstacles. Figure 15 depicts the robot's trajectory in an obstacle-free environment, underscoring the agents' ability to fluidly and accurately adhere to the desired path. In contrast, Figure 16 introduces impediments, simulating the dynamic and ever-changing beach environment where unexpected elements such as beachgoers, animals, or debris must be navigated around.  
In the first two experiments, the models' ability to adapt effectively from simulation to reality was assessed. The robust convergence of performance metrics between simulated and real environments adequately supported the models' adaptability. This experiment focused exclusively on the execution of the task of following a predefined route. Given that the primary emphasis was on task execution rather than a detailed performance analysis, reward index tables were omitted to streamline result presentation and concentrate on the task execution capability.  
This adaptability and precise navigation are crucial for the goals outlined in this article, which seeks to develop an autonomous robotic system for beach-cleaning tasks. The outcomes of the experiment affirm that the agents can adjust their behavior and make real-time decisions to circumvent obstacles and maintain the set route, promising indicators of their real-world applicability.  
Reinforcing the Sim2 Real methodology elaborated throughout the article, the experiment confirms that skills transitioned from simulated to actual environments are not only feasible but also effective. This success holds promising implications for scalability and the potential deployment of beach-cleaning robots across various beach environments.  
\begin{table}
\begin{tabular}{c c c c c} \hline  & \multicolumn{2}{c}{**Real**} & \multicolumn{2}{c}{**Simulated**} \\ \cline{2-5}
**Index** & **DDPG** & **PPO** & **DDPG** & **PPO** \\ \hline ISE & **11.78** & 12.45 & **9.76** & 12.16 \\ IAE & 16.71 & **16.60** & **13.47** & 16.27 \\ ITSE & 97.28 & **95.04** & **63.66** & 92.82 \\ ITAE & 178.29 & **164.31** & **113.57** & 159.50 \\ \hline \end{tabular} Bold values indicate the best performance (lower index indicates better performance).  
\end{table}
Table 2: Performance index comparison of the second experiment.  
The demonstrated effectiveness of transferring learned skills underscores the scalability of the proposed methodology. As the robots showcase adaptability and efficiency in real-world beach landscapes, the potential for deployment in diverse environments becomes evident. The scalability of the approach positions it as a versatile solution with the capability to address the challenges posed by different beach environments, ultimately contributing to more widespread and effective deployment of autonomous beach-cleaning systems.