# Comprehensive Review of Robotics Operating System-Based Reinforcement Learning in Robotics
## 4 Application-Specific Comparison
DDPG struggled with convergence in complex environments, while TD3 showed high reward values and consistentlanding but required a long training time. SAC exhibited a less stable reward curve as it approached peak performance. The SAC proved to be an effective algorithm for managing drone landings among the other three algorithms with respect to the landing task. _Navigation of Legged Mobile Robots_: In [65, 73], the authors offer valuable information on a common challenge in robotics: transferring learned policies from sim-to-real applications. Both works collectively contribute to a broader understanding of the potential of RL for revolutionizing robot control. In both works, the authors discuss the benefits of using different algorithms, such as SAC and PPO in the Gazebo environment before transferring the learned knowledge to real-world applications. In [73], the authors stress the importance of further using DRL to improve the learning of the agent by exposing the agent to a wide range of variations in the simulated environment. This makes the agent more robust and adaptable to real-world conditions. Both works [65, 73] provide a convincing example of how to achieve generalization through simulation-based training. _LiDAR-Based Navigation_: In [81, 86], the authors discuss the use of LiDAR for robot navigation, particularly in the context of autonomous vehicles. The work in [81] focused on developing a DRL-based approach for path exploration using LiDAR data. The authors evaluated their approach on a small-scale RC car in both simulated and real-world environments. One key development in this work involved converting raw LiDAR data into a 2D grid map for input into the neural network. The grid map method outperformed the raw LiDAR data as input, particularly when transferring the trained model to a real-world setting. In [86], the authors focused on improving the AMCL algorithm by using a particle filter-based approach for high-speed localization on a race track to enhance both the accuracy and computational efficiency. Several improvements were implemented in [86] to the AMCL algorithm, such as corridor-optimized LiDAR beam extraction. Instead of sample LiDAR beams at the same angle of movement, they chose the best beams for a race track environment that is more information-rich. This system monitors factors like the estimated pose consistency with the map and the variance of the particle distribution. The system assigns a status level, which indicates the reliability of the current localization, which can inform the subsequent fusion of sensor data. In general, both works contribute to the advancement of LiDAR-based navigation for autonomous vehicles, with each focusing on different aspects of the problem. The work in [81] explores the use of DRL to train a vehicle to navigate unknown environments, showcasing the effectiveness of using a processed LiDAR grid map over raw data. Whereas, ref. [86] focuses on improving a well-established localization approach by introducing modifications that address the specific challenges of high-speed driving on a race track, leading to improved accuracy and computational efficiency. _Improvements to Mapless Navigation_: In [68, 78, 79, 80], the authors discuss how to improve robot navigation in unknown environments by using DRL algorithms in conjunction with traditional methods such as DWA and semantic segmentation. They highlight the limitations of purely DRL-based approaches, such as slow learning, slow robot speeds, and difficulty adapting to real-world scenarios. In [68], the authors address the challenges of real-world applications by integrating semantic segmentation with DRL. The work proposes a system using a DRL model to navigate on the basis of segmented images, reducing the gap between simulation and real-world environments. The authors acknowledge that the system still struggles with camera height variations, suggesting a need for more robustness. In [78, 80], the authors focus on combining DRL with DWA, proposing a hierarchical framework.