# Deep-Reinforcement-Learning-Based Planner for City Tours for Cruise Passengers +
## 6 Experimental Results and Discussions
At the end of the training, results show that the planner could provide a path \begin{table}
\begin{tabular}{c c c c} \hline \hline
**\#Episode** & **Training Max Reward** & **Testing Max Reward** &
\begin{tabular}{c} **Mean Residual Time** \\ **(Min)** \\ \end{tabular} \\ \hline
5000 & 1.0 & 0.9 & 131.78 \\
10,000 & 1.1 & 0.8 & 99.87 \\
20,000 & 1.3 & 1.0 & 70 \\
30,000 & 1.1 & 1.0 & 70 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Overview of planner performance with testing data. Figure 5: Trend of reward during training. ranging from two PoIs to four PoIs because of the setting provided to model both the environments and the agent. Table 2 reports the mean, variance, and maximum reward value for each setting. Notably, that these values increase as long as the number of episodes increases. Figure 6 reports a box chart graphically demonstrating the different settings in terms of minimum reward value, first (lower) quartile, median, third (upper) quartile, and maximum reward value. The most uniform distribution appears to be achieved by the 20,000-episode setting, and the reward distribution with a path length of 3 PoIs appears to expose the fairness rewards distribution. Finally, Figure 7 reports one example of applying the dynamic re-planning of the agent's output when an unexpected event happens. We simulated the occurrence of a public transportation strike as a uniform probability distribution. When the probability overcomes a fixed threshold in this study, set to 0.3, the events are triggered. The first PoI is the National Museum of Naples, and the next PoI is the Riviera di Chiaia, but a strike is taking place, so the agents have to re-plan a new itinerary and forward the tourist to Posilipo and then to the port. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**5000**} & \multicolumn{3}{c}{**10,000**} \\  & \multicolumn{3}{c}{**Episodes**} & \multicolumn{3}{c}{**Episodes**} \\ \hline PoI & \#2 & \#3 & \#4 & \#2 & \#3 & \#4 \\ Mean & 0.4 & 0.63 & 0.75 & 0.48 & 0.63 & 0.81 \\ Variance & 0.03 & 0.034 & 0.08 & 0.03 & 0.034 & 0.0097 \\ Max & 1 & 1 & 1 & 1 & 1.1 & 1 \\ \hline  & \multicolumn{3}{c}{**20,000**} & \multicolumn{3}{c}{**30,000**} \\  & \multicolumn{3}{c}{**Episodes**} & \multicolumn{3}{c}{**Episodes**} \\ \hline PoI & \#2 & \#3 & \#4 & \#2 & \#3 & \#4 \\ Mean & 0.21 & 0.71 & 0.77 & 0.51 & 0.66 & 0.79 \\ Variance & 0.017 & 0.029 & 0.008 & 0.01 & 0.02 & 0.02 \\ Max & 0.99 & 1.3 & 0.8 & 1 & 1.3 & 1.1 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Overview of reward performance by number of PoIs. Figure 6: Overview of reward performance with reference to the number of PoIs in the itinerary. In the simulation, each PoI is scheduled individually, and each time the agent provides the best PoIs to maximize the time to visit. However, a strike is taking place, and consequently, the agent has to provide a novel PoI that produces a sub-optional itinerary. With the aim of providing an honest discussion, the following reports a few limitations of the achieved results: The main issue concerns the fixed number of PoIs adopted for the planner's training. We designed only fourteen PoIs for experimental reasons, a number which could be higher in a real use-case scenario. Another issue is training the agent with a high number of episodes; Table 1 reports that the performance does not increase with the number of episodes. Indeed, moving from 20,000 episodes to 30,000 means an increased rate equal to 33%; the testing reward does not improve. This aspect suggests that the training process should be optimized to improve performance as the number of episodes increases, or the learning process should be stopped early to save time.