## 3 Multiple Destination Global Path Planning Strategy Based on Improved Q-Learning Algorithm
### Multi-Destination Simulation Experiment
In order to prove this point, in the experimental analysis of the algorithms, their running time and the number of fitness function calls are compared. _Path length_ is the length of the planned path. _Path safe_ is determined by using a potential field. The potential field of surroundings due to obstacles can be given a value by linear decreasing potential value. This article considered the specific obstacle placement position shown in Figure 14b. This scheme is one of the complex schemes considered in earlier path planning research. This research compares the use of NSGA_II algorithm to solve the path planning problem. It compares the three representations of integer coding, binary coding and mixed coding. This research selected size of population and number of iterations of dual-target NSGA-II. Size of population is 500, and number of iterations is 800. The mutation rate (pm) was selected as 0.032, and the crossover rate (pc) was selected as 0.9. Using these values for the NSGA-II parameters, the Pareto front obtained in all three scenarios is shown in Figure 14a. Among the three schemes, it was found that the integer-encoded gene representation scheme can produce better results. The integer-encoded gene represents the minimum path distance solution. The minimum path distance solution, the minimum path vulnerability solution and the intermediate compromise solution are shown in Figure 14b. In order to compare the performance of the proposed algorithm and NSGA_II, the previous scenario was repeated using the algorithm (based on MD-Q-learning with minimal obstacles) in this article. In this experiment, a population with a size of 50 was selected and the algorithm was run for 100 iterations. The minimum and maximum values of the local search operation are 60 and 10, respectively. Figure 15a shows the Pareto front obtained by the CSD-based MD-Q-learning algorithm. Figure 15b shows some of the obtained paths. As shown in Figure 15a, the Pareto obtained by the MD-Q-learning algorithm based on the fewest obstacles and the NSGA_II algorithm is close enough to the optimal Pareto front. The MD-Q-learning algorithm based on passing through the fewest obstacles applies more local search operations to these solutions. In this way, appropriate solutions can improve and enhance Pareto solutions. Figure 14: (**a**) Pareto front obtained by NSGA_II using three schemes. Integer encoding means that the scheme performs best. (**b**) Obstacles on a \(32\times 32\) grid use integer coding to represent the solution path obtained by the scheme. Figure 15: (**a**) Pareto front obtained using the MD-Q-learning algorithm based on passing through the fewest obstacles. (**b**) Use of the obstacle arrangement and three paths based on the MD-Q-learning algorithm that passes through the fewest obstacles. In addition, the local search operator will try to improve the smoothness of the path as close as possible to the best Pareto front. Therefore, the path obtained by this article is smoother than the path shown in Figure 14b. Table 1 compares the parameter initialization and performance of NSGA_II and the MD-Q-learning algorithm based on the fewest obstacles. According to the table, the running time of the MD-Q-learning algorithm based on passing the fewest obstacles is 3.04. The running time of the NSGA_II algorithm is 2.99. The MD-Q-learning algorithm based on the fewestobstacles has fewer fitness function calls to achieve the best solution. The Pareto front obtained from the two algorithms is as close as possible to the optimal Pareto front. However, in the MD-Q-learning algorithm based on the fewest obstacles, the final coefficient of variation of Pareto front for different targets is smaller than that of the NSGA_II algorithm. Therefore, Pareto solutions obtained by the MD-Q-learning algorithm have a better distribution in the Pareto front. As mentioned earlier, NSGA_II has MN\({}^{2}\) complexity. MD-Q-learning based on passing through the fewest obstacles has M\({}^{2}\)N\({}^{2}\).