# LSTM-DQN-APF Path Planning Algorithm Empowered by Twins in Complex Scenarios
## 4 Twin System Modeling and LSTM-DQN-APF Algorithm
### Algorithm Validation Test and Analysis
#### 4.3.4 Experiment 3: Algorithm Validation Test for Urban Warfare Scenario
average flight distance is 71.6 m. For flight time, the average variance of APF is 15.424 s, the average variance of Ge-Apf is 6.87 s, and the average variance of LSTM-DQN-APF is 1.121 s; for flight distance, the average variance of APF is 67.942 m, and the average variance of Ge-Apf is 10 m. The average variance of the flight distance of LSTM-DQN-APF is 9.896 m.  
The starting point coordinates of the UAV are (0,0,0), and the target point coordinates are (10,50,0). The motion parameters of the three spheres are shown in Table 6. As can be seen from Figure 13, there are dynamic obstacles in the scene, including a moving sphere and pedestrian, and the movement direction of the sphere has been marked in the figure. Figure 13a shows the static state; Figure 13b shows the state after 15 s of exercise. As shown in Figure 14a, the task is segmented into three target waypoints: the first waypoint is located at coordinates (4,20,5), the second at (12,25,5), and the third at (10,50,0). As illustrated in Figure 14a, when the UAV approaches the first dynamic obstacle, it begins to adjust its trajectory due to the perceived relative velocity and the repulsive force exerted by the obstacle. Figure 14b depicts the second phase of trajectory planning in a dynamic environment, demonstrating the UAV's successful avoidance of both the dynamic sphere and pedestrians as it continues toward the target waypoint. Upon entering the influence zone of the second dynamic obstacle, the flight path planning system initiates adjustments to the UAV's course based on the obstacle's impact. Figure 14c illustrates the final stage of trajectory planning in a dynamic environment, where the UAV successfully reaches its designated target. As shown in Figure 15, the evolution curves of the number of iterations versus the reward value for DQN and LSTM-DQN-APF are demonstrated. From the figure, it can be seen that DQN shows fast learning ability in the initial phase of the experiment, and its reward value rises rapidly, which implies that it can adapt to the environment relatively quickly and can find a suitable path planning strategy quickly. However, in the middle and late stages, the reward value of DQN shows a more obvious decreasing trend, which may indicate that the algorithm has some problems in the subsequent iterations, such as falling into the local optimal solution or deteriorating its adaptability to the environmental changes. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Sequo** & **Initial Position** & **Radius** & **X-Axis Step Size** & **Y-Axis Step Size** & **Z-Axis Step Size** & **Exercise Cycles** \\ \hline
1 & (\(-\)1,1,5) & 4 & 0.02 & \(-\)0.04 & 0.03 & 100 \\
2 & (3,15,5) & 2 & 0.03 & 0.06 & 0.02 & 100 \\
3 & (5,12,5) & 4 & 0.04 & 0.05 & 0.02 & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Parameters of dynamic obstacles in the 3D environment. Figure 13: Static (**a**) and dynamic states (**b**) in a dynamic environment. In contrast, LSTM-DQN-APF is able to quickly rise to a positive reward value at the beginning of the experiment, which fully demonstrates that the algorithm has a stronger fast learning ability and is able to find an effective strategy for avoiding obstacles faster than the DQN algorithm. LSTM-DQN-APF not only performs well at the beginning of the learning process but also has better stability with relatively small fluctuations in its reward value throughout the iteration process, which further demonstrates the superior performance of the LSTM-DQN-APF algorithm in handling this task. This further indicates that the LSTM-DQN-APF algorithm has better performance in handling this task. Figure 14: Motion trajectory diagram in dynamic environment. Figure 15: Evolution curve of the number of iterations versus reward value. (3) Wind effects  
As shown in Figure 16, a horizontal westward wind field with a wind speed of 5 m/s is added to the simulation environment, and the mission target point is set to (10,0,5) for the virtual UAV to fly towards the target point with a speed of 1 m/s.