# Comprehensive Review of Robotics Operating System-Based Reinforcement Learning in Robotics
## 4 Application-Specific Comparison  
Table 4 presents a comprehensive summary of reviewed works--robotic applications, RL algorithms that include model and policy type, application environments, platforms, sensors, OSs, and languages. We further provide application-specific comparative analysis in the below paragraphs. _Enhanced Collision Avoidance_: In [58, 69, 70], the authors describe the evolution of collision avoidance techniques, specifically focusing on the transition from SA-CADRL to GA3C-CADRL. The SA-CADRL builds on previous work [11]. SA-CADRL [58], representing an earlier approach, relies on training a DL network to learn a value function that estimates the time it takes for a robot to reach its goal given its current state in the presence of other agents (pedestrians). This approach, while promising, suffers from limitations due to its dependence on simplified assumptions about pedestrian behavior. In particular, SA-CADRL uses a constant velocity model to project future states, which proves inadequate to capture the dynamic interactions of the crowd pedestrian environments. Recognizing these limitations, in [69], the authors developed the GA3C-CADRL technique that adopts a policy-based learning approach. Instead of relying on value functions and assumptions of the pedestrian behaviors, GA3C-CADRL directly learns a policy that maps the robot observations to actions. This is a significant departure from the value-function-based approach, and it eliminates the need for potentially inaccurate assumptions about pedestrian behaviors. An additional enhancement to GA3C-CADRL is the incorporation of LSTM networks. LSTMs are adept at processing sequences, making agents well suited to handle the variable number of pedestrians encountered in real-world scenarios. The LSTM within GA3C-CADRL receives a stream of agent states as input, encoding this information into a fixed-size representation of the environment, which is then used to determine the robot's next action. The integration of LSTM allows GA3C-CADRL to manage a dynamic number of agents without needing to predefine the network's input size. The development work from SA-CADRL to GA3C-CADRL brings several notable improvements, such as improving scalability in terms of the number of agents. Flexibility is essential in real-world applications, where the density of the agent fluctuates. The more realistic modeling work [70] GA3C-CADRL directly learns a policy, eliminating the need for simplistic and often flawed assumptions about pedestrian behaviors, such as the constant velocity model used in SA-CADRL [58]. This enables GA3C-CADRL to learn more sophisticated collision avoidance strategies by observing actual pedestrian movements. The work in [70] employed a hybrid GPU/CPU approach to accelerate the training process. This allows for more extensive exploration of possible policies, leads to faster convergence, and makes possible the discovery of more effective solutions. Enhanced generalization in [69, 70] was found to improve the learned policy and adaptability. Both these works demonstrate their successful implementation of simulated multiagent collision avoidance among pedestrians, highlighting their potential for real-world deployment. The advancements represented by the GA3C-CADRL algorithm signify a fundamental shift in the approach to collision avoidance for robots navigating in pedestrian-rich environments. The combination of policy-based learning, LSTM networks, and efficient training methodologies results in a more robust, adaptable, and practical solution. _Advancements in UAV Landing Handling_: In [75, 76], the authors share a common goal of showcasing the potential of simulation-based training (using Gazebo) of different algorithms applied to complex UAV tasks. In [75], the authors discuss how the DDPG algorithm is used to enhance UAV landing safety. In [76], the authors compared other algorithms such as DDPG, TD3, and SAC to evaluate the UAV landing approach.