## 2 Related Work
### DRL-Based Approaches  
Adopting DRL in autonomous navigation has yielded promising results in various real-world applications. By learning from experience and a set of policies, DRL enables the agent to make intelligent decisions, creating a neural network model capable of handling unknown scenarios [29]. Learning-based methods have been developed to enable agents to perceive their environment and understand the behavior of dynamic objects. These methods facilitate safe and seamless navigation within 2D scenarios amid other autonomous agents and obstacles. To model the dynamic interactions in the scenarios, DRL-based methods often leverage traditional techniques such as the Social Force model [30], Reciprocal Velocity Obstacles (RVOs) [31], and Optimal Reciprocal Collision Avoidance (ORCA) [32]. Some works have explored data-driven approaches to model and predict human behavior for collision avoidance [33; 34]. From previous approaches, numerous researchers have developed neural network architectures to simulate robots within 2D environments, navigating multiple episodes from the start point to specified goals [35; 36; 37]. Long short-term memory (LSTM) [38; 39] has shown promise in handling dynamic environments with an arbitrary number of other agents, diverging from previous methods with fixed observation sizes. By employing a self-attention mechanism [40], the socially aware RL (SARL) method prioritizes the most relevant dynamic obstacles within its environment. However, research has primarily focused on simple scenarios, often resulting in robots freezing or failing to reach goals in diverse scenarios with static and dynamic obstacles due to a lack of prior experience. Proposals like [41] advocate for enhancing state-of-the-art neural networks by broadening training across multiple scenarios, yielding notably satisfactory results where robots successfully achieve goals. On the other hand, new challenges emerge when robots navigate long paths, particularly when sensor noise disrupts measurements, causing the robot to lose its path and fail to reach the intended goal. Addressing this, studies like [42; 43] propose using probabilistic road maps (PRMs) in conjunction with RL methods for indoor environments. Despite the existence of numerous reward functions for autonomous navigation, some research provides unique insights into reward functions that enhance robot performance [9]. This work highlights the advantages and drawbacks of policies during both training and real-world experiments. Predicting human behavior is a challenging task. In [44], the authors proposed dynamic warning zones that consider the heterogeneous and dynamic nature of humans, including their velocities, dimensions, and step lengths. This approach allows the robot to anticipate human motion in social environments. In [45], the authors use function-oriented bounding capsules (OBCs) to represent humans, capturing their dynamic shapes and velocities. Building upon this existing research, our work concentrates on integrating information obtained from radar for dynamic object detection into the DRL model for autonomous navigation. Additionally, we enhance the reward policy of the neural network to improve algorithm performance during testing and real-world experiments.