# An Enhanced Deep Q Network Algorithm for Localized Obstacle Avoidance in Indoor Robot Path Planning
## 2 Related Work  
In recent years, RL has emerged as a research focus in the field of online path planning due to its immunity to environmental mutations and dynamic obstacles [4]. Reference [12] has shown this by simulating two scenarios: the first scenario where the map changes while the intelligent body is moving, and the second scenario where dynamic obstacles appear on a static map. It is demonstrated that real-time Q Learning can be used for mobile robot path planning and dynamic obstacle avoidance in various environments. Reference [13] introduced priority weights into Q Learning to improve the value assessment of the algorithm when solving practical problems. The improved algorithm significantly improved the average speed and accuracy and was able to find better paths in dynamic environments. Reference [14] proposed an Optimized Q Learning (O_QL) algorithm to address the problems of slow convergence and easy trapping in local optimal solutions of the Q Learning algorithm. O_QL introduced a new Q table initialization method and adopted a new action selection strategy that combines the \(\varepsilon\)-greedy and Boltzmann strategies. In order to avoid sparse rewards, it borrowed the Gaussian idea to use a continuous reward function. Reference [15] adopts the concept of partially guided Q Learning and uses the APF method to improve the classical Q Learning method. The proposed QAPF algorithm overcomes the shortcomings of classical Q Learning such as slow learning speed, long learning time and slow convergence. The above reference has made corresponding improvements to the Q Learning algorithm by iteratively updating the Q table, exploring strategies, reward functions, and learning rates, significantly improving the convergence efficiency and learning speed of the algorithm, enabling it to cope with general unknown dynamic environments, but it cannot solve the \"dimensionality curse\" problem. Q Learning performs well in discrete state and action spaces, but for continuous spaces, discretization is required, which may introduce approximation errors. In 2015, in order to solve the memory overflow problem caused by large amounts of data or upcoming continuous actions in high-dimensional environments, DeepMind proposed the DQN [8] algorithm in Nature, which combines neural networks with Q Learning. The authors of [16] used the PER mechanism in the DQN algorithm. By determining the importance of experience based on TD error, high-priority experience is replayed more frequently, and random priority and importance sampling weight corrections are introduced to ensure sample diversity and unbiased updates. These improvements enable the DQN algorithm to use experience data more effectively, accelerate learning without increasing the number of interactions with the environment, and significantly improve the quality of the strategy, especially in Atari game tests. The authors of [17] introduced maximum entropy into RL, making the strategy more random when choosing actions. The introduction of this entropy regularization increases the diversity of exploration, thereby exploring the environment more effectively in the early stages, and ultimately speeding up the entire learning process. The authors of [18] maximized the expected return and the entropy of the final policy by adding a scaled \"log-policy\" to the instant reward based on maximum entropy RL. This approach makes the M-DQN algorithm the first algorithm to outperform distributional RL without using distributional RL [19]. Reference [20] drew on the network branching structure of Duel DQN to design a dual-branch network that decouples obstacle avoidance and navigation tasks. It also adopted an \"expert experience\" approach to improve random exploration and designed a unique reward function mechanism. Through both simulation and real-world experiments, the algorithm's advantages in terms of convergence and stability were demonstrated. The authors of references [21; 22] both adopted the Double DQN approach to alleviate the overestimation issue in DQN.