# An Enhanced Deep Q Network Algorithm for Localized Obstacle Avoidance in Indoor Robot Path Planning
## 5 Experimental Analysis
### Simulation Comparison Experiment  
We show the cumulative and average rewards obtained by the four algorithms trained based on the model-free approach in the three scenarios (a), (b) and (c) in Figures 8-10, respectively, where the cumulative rewards are represented by circular scatters and the average rewards are depicted in a folded line. And the average rewards obtained from the four algorithms trained in the three scenarios are compared in Figure 11. The simulation experiments were all performed within 500 episodes, and an episode was terminated when the robot collided while searching for the target location or when the number of execution steps exceeded 500. Figure 8: Total and cumulative rewards generated by the four algorithms for each episode in Env1. (**a**) represents the cumulative reward and average reward plots of the DQN; (**b**) represents the cumulative reward and average reward plots of the Duel DQN; (**c**) represents the cumulative reward and average reward plots of the PMR\(-\)DQN; (**d**) represents the cumulative reward and average reward plots of the PER\(-\)D2 MQN. Figure 7: Gazebo experiment environment. (**a**) is the experimental static environment (4-4 m); (**b**) is the experimental dynamic environment (4-4 m); (**c**) is the experimental complex environment (5–5 m). Figure 10: Total and cumulative rewards generated by the four algorithms for each episode in Env3. (**a**) represents the cumulative reward and average reward plots of the DQN; (**b**) represents the cumulative reward and average reward plots of the Duel DQN; (**c**) represents the cumulative reward and average reward plots of the PMR\(-\)DQN; (**d**) represents the cumulative reward and average reward plots of the PER\(-\)D2 MQN. Figure 9: Total and cumulative rewards generated by the four algorithms for each episode in Env2. (**a**) represents the cumulative reward and average reward plots of the DQN; (**b**) represents the cumulative reward and average reward plots of the Duel DQN; (**c**) represents the cumulative reward and average reward plots of the PMR\(-\)DQN; (**d**) represents the cumulative reward and average reward plots of the PER\(-\)D2 MQN. As can be seen in Figure 8, the cumulative reward of the four algorithms in scenarios (a) show some volatility. In terms of average reward, the DQN algorithm has no obvious convergence trend throughout the simulation process; in contrast, the Duel DQN algorithm gradually converges after 300 episodes by changing the network structure, and the average reward tends to be around 4000, but the cumulative reward still fluctuates greatly. The PMR-DQN algorithm, which introduces the PER mechanism, shows better stability, with the average reward gradually converging to around 6700 after 300 episodes, and the fluctuation of the cumulative reward decreases compared with that of Duel DQN. The PER-D2 MQN algorithm shows better convergence after 250 episodes, with the average rewards converging to about 7500, and the fluctuation of the cumulative rewards is minimized, which is a significant improvement in the performance. The comparison of the average rewards in Figure 11a further validates these findings. Figure 9 illustrates the cumulative rewards and average rewards of four algorithms in the scenario (b) with dynamic obstacles. In terms of cumulative rewards, all four algorithms exhibit a certain degree of fluctuation. However, the PMR-DQN and PER-D2 MQN algorithms demonstrate significantly better performance, with cumulative rewards appearing more frequently within the range of (4000, 12,000) after 200 episodes. In contrast, the DQN algorithm shows larger fluctuations in cumulative rewards, while Duel DQN exhibits comparatively reduced volatility. Regarding average rewards, the differences between the algorithms are more pronounced. The DQN algorithm does not exhibit a clear trend of convergence throughout the simulation. In contrast, the Duel DQN algorithm begins to converge after 200 episodes, stabilizing around 3700 after 300 episodes.