# Comprehensive Review of Robotics Operating System-Based Reinforcement Learning in Robotics
## 4 Application-Specific Comparison
It employs the LSTM module to generate local way points for collision avoid ance based on the robot's state, LiDAR data, and goal position. This approach combines the learning capabilities of DRL with the trajectory-smoothing properties of DWA, resulting in a more efficient and reliable navigation system. In [79], this work also leverages sensor fusion to improve navigation. It proposes a system that combines a 2D LiDAR sensor with a camera for obstacle detection. Instead of using a 3D LiDAR, which is more expensive, the system uses a Bird's-Eye View (BEV) projection to transform the 3D point cloud data from the camera into a 2D representation. This approach enhances the robot's environmental perception capabilities without incurring the high cost of a 3D LiDAR. _DWA-Based Navigation_: In [78; 84; 82], the authors describe methods for improving robot navigation by combining DRL with DWA. These works show that DWA can be limited in its ability to adapt to unknown or dynamic environments; however, due its fixed parameters, it may not be optimal in some scenarios. In [78], the work shows a DWA-LSAC framework that uses SAC with the LSTM module to create local way points. The DWA then uses these to smooth the robot navigation path. This approach improves the model's exploration ability in the early stages of training, leading to faster convergence and smoother navigation, particularly in environments with dynamic obstacles. The work in [84] integrates DWA with DRL using the PPO algorithm. This approach introduces new cost functions to DWA, considering the distance from other agents and the goal, resulting in more efficient and collision-free navigation. In [82], the emphasis is on improving DWA by adding new functions to address limitations, such as the tendency to discard safe paths or fall into local minima. Overall, these works highlight the benefits of combining the DWA with RL. The DWA provides a robust method for generating locally feasible and smooth directions that respect the robot's kinematic constraints. RL enables the robot to adapt to dynamic environments and learn optimal parameters for the DWA algorithm, leading to successful navigation. _Physics Engine Advancements_: In [21; 64], the authors developed a toolkit using ROS and Gazebo simulators. This toolkit provides a standardized environment for testing and comparing different RL algorithms and allows researchers to benchmark techniques under controlled virtual conditions. This approach offers significant benefits: it reduces the high cost and time associated with real-world robot training, speeds up the development process, and eliminates potential dangers during the learning phase. In [21], the authors present gym-gazebo using ROS1, and in [64], they present gym-gazebo2, a significant improvement over the initial toolkit gym-gazebo. The gym-gazebo2 offers a more robust and advanced architecture focusing on facilitating the transfer of learned policies to real-world applications. _Moving Toward Benchmarking_: Several works aimed to establish a robust benchmarking system for robotic applications to compare different RL algorithms under varying conditions. This is essential for objectively evaluating the performance of different algorithms and identifying those best suited for specific robotics applications. In [21], the authors mention techniques such as \"_mental rehearsal_\" that involves using real-world data to refine simulated models as a potential approach to address the \"_reality gap_\". In [64], the authors focus on the advantages of using a mobile robot with autonomous capabilities for tasks, such as search-and-rescue operations. In [72], the authors introduce an efficient platform for multiagent robots to solve the complexity of building a robot environment. The platform allows developer to build custom environments by combining different robot modules, regardless of the underlying robot software details.