## 3 Fusion DQN DWA Algorithm
### Fusion Path Planning Algorithm Framework Based on DQN and DWA  
Based on the traditional path planning algorithm and deep Q network, a fusion path planning algorithm based on deep reinforcement learning and dynamic window method is proposed. The fusion path planning algorithm has the following advantages:  
1. With the \"dynamic window\" feature, the movement of the robot is directly acted on by the speed space, and the movement of the robot is controlled by the linear speed, angular speed, and rotation direction of the robot, so that the robot can maintain a safe speed and avoid collision when it is moving at a speed that is too fast or slow, or is moving at too steep of an angle. 2. Process the map environment with a too large state-action space on the convolutional neural network model, set the \(\varepsilon\)-greedy strategy at the same time, and train the neural network. The simulation results show that the fusion algorithm can effectively avoid the problems of dimensionality disaster and slow convergence, ensuring that the robot has strong robustness in real-time in the complex and changeable environment. Figure 10: Flowchart of DQN algorithm. To enable the robot to complete the task of path planning in the complex and changeable environment and make the robot's movement show the following characteristics: dynamic window, deep reinforcement learning, and dynamic window method. These are integrated to carry out the path planning of the mobile robot. Figure 11 shows a schematic diagram of the fusion path planning framework based on deep reinforcement learning and dynamic window method. First, the robot's moving environment is input in the form of RGB images. The convolutional neural network is used to process the image to reduce computational complexity: Then, in the current state \(s\), the robot uses the \(\varepsilon\) greedy strategy to select an action by interacting with the environment (dynamic window part). After that, the robot will enter the next state \(s^{\prime}\), and a reward r will be generated in the interaction process: Finally, the robot moves by selecting the action corresponding to the maximum Q value. The training samples can be obtained through the mechanism of experience storage, and then the network parameters are updated using stochastic gradient descent and backpropagation algorithms. The main idea of the experience storage mechanism is that during training, the movement information of the robot at each time point is stored in an experience ancestor U, U = (\(s\), \(s^{\prime}\), \(r\), \(a\)). Then, these experience progenitors are saved in the experience storage pool, and the weight of DQN is updated by gradient descent method to train the data. The specific calculation steps of the fusion path planning algorithm based on DQN and DWA are described as follows:  
Step 1: Initialization: the initial state of the robot, including the position information of the starting point and the initial speed. Step 2: During the interaction between the robot and the environment (that is, the dynamic window part), the speed corresponding to the action with the largest Q value (\(v\), \(\omega\)) is selected, and the corresponding reward value r is obtained, thus entering the next state \(s^{\prime}\). Step 3: If the distance between the robot and the nearest obstacle in the next state \(s^{\prime}\) is kept within the safe range, that is, dist (\(v,\omega\)) \(>t\), then perform the next step; otherwise, return to the previous state and select the action again. Step 4: Save the mobile status information of the trained robot in the experience tuple U = (\(s\), \(s^{\prime}\), \(r\), \(a\)), the random gradient descent method is used to update the network parameter 0 of DQN. Figure 11: Schematic diagram of fusion path planning framework based on DQN and DWA. Step 5: Determine whether the final state information of the robot has reached the target point of the path planning experiment.