# Reinforcement-Learning-Based Path Planning: A Reward Function Strategy
## 3 Results
### Evaluation of Trajectory-Planning Algorithms  
Figure 8 shows the proposed maps for the evaluation of the algorithms RL and Deep Q-Learning. Four different maps are considered. Figure 8: The structures of the maps used for the test: (**a**) Map 1, with static obstacles and a dynamic obstacle; (**b**) Map 2, with static and dynamic obstacles; (**c**) Map 3, with static obstacles; and (**d**) Map 4, with static obstacles. The first map, proposed in Figure 8a, contemplates the initial position of the agent at the coordinate \((0,\ 0)\), corresponding to the upper-left corner. In this environment, a vertical, obstacle-free trajectory from the initial row to the final row of fixed obstacles is avoided. The only trajectory that has this characteristic is planned by horizontally moving a dynamic obstacle, which changes in each iteration. The objective is located in the lower-right corner, corresponding to position \((9,\ 9)\). The second proposed map, shown in Figure 8b, is a variant of the first map and aims to validate the agent behavior by adding random dynamic obstacles. The third and fourth maps are shown in Figures 8c and 8d, respectively. They contemplate static obstacles, but an initial coordinate different from the coordinate \((0,\ 0)\) is considered. Figure 9a-d shows the graphs of the number of iterations with rewards from the RL algorithm for Maps 1, 2, 3, and 4, respectively. Map 1 shows a total reward of 4539, while Map 2 shows 4381, Map 3 shows 4323, and Map 4 shows 4596. All four graphs indicate a tendency to minimize accumulated rewards. The close values between the accumulated rewards of the maps are due to the fact that each map considers \(10\times 10\) cells, and the action sequences for each map have similar lengths. From iteration 300 onwards, convergence is evident in each map. Figure 10 shows the performance of the proposed Deep Q-Learning algorithm. The rewards versus episode plots for Maps 1, 2, 3, and 4 are seen in Figure 10a, Figure 10b, Figure 10c, and Figure 10d, respectively. The four graphs show a convergence from episode 200 onwards. The results show acceptable performance, since the agent receives minimum penalties along the trajectory. This can be proven with values close to zero from episode 200 onwards. Figure 9: Rewards versus iterations of the RL algorithm. (**a**) Rewards versus iterations, Map 1; (**b**) rewards versus iterations, Map 2; (**c**) rewards versus iterations, Map 3; and (**d**) rewards versus iterations, Map 4. The characteristics to be compared with the four proposed algorithms include the distance traveled, the number of turns during the trajectory, and the training time. Table 3 provides a comparison based on the planted approach. \begin{table}
\begin{tabular}{c c c c c} \hline
**Map** & **Algorithm** & **Distance (m)** & **No. Turns** & **Training Time (s)** \\ \hline \multirow{4}{*}{1} & Reinforcement learning & 13.89 & 3 & 112.40 \\  & Deep Q-Learning & 13.89 & 3 & 526.45 \\  & A* (4 actions) & 22.00 & 10 & 0.011 \\  & A* (8 actions) & 13.89 & 4 & 0.015 \\ \hline \multirow{4}{*}{2} & Reinforcement learning & 13.31 & 4 & 119.46 \\  & Learning & 13.31 & 4 & 556.49 \\  & Deep Q-Learning & 13.31 & 4 & 556.49 \\  & A* (4 actions) & 18.00 & 5 & 0.011 \\  & A* (8 actions) & 13.89 & 7 & 0.014 \\ \hline \multirow{4}{*}{3} & Reinforcement learning & 17.89 & 8 & 127.65 \\  & Deep Q-Learning & 17.89 & 7 & 687.87 \\  & A* (4 actions) & 22.00 & 10 & 0.010 \\  & A* (8 actions) & 18.48 & 9 & 0.014 \\ \hline \multirow{4}{*}{4} & Reinforcement learning & 13.89 & 6 & 98.06 \\  & Deep Q-Learning & 13.89 & 4 & 447.16 \\  & A* (4 actions) & 18.00 & 4 & 0.009 \\ \cline{1-1}  & A* (8 actions) & 13.89 & 8 & 0.013 \\ \hline \end{tabular}
\end{table} 
Table 3: Comparison of the performance of each algorithm on different maps. Figure 10: Reward versus episodes of the RL algorithm. (**a**) Rewards versus episodes, Map 1; (**b**) rewards versus episodes, Map 2; (**c**) rewards versus episodes, Map 3; and (**d**) rewards versus episodes, Map 4.