# A Reinforcement Learning Model of Multiple UAVs for Transporting Emergency Relief Supplies
## 5 Numerical Experiments
### Performance Comparison  
In this study, we evaluated the transport of items for each case against our model by conducting a comparative examination of the transport of five cases with different rates of three objective assignments in QL. The cases are listed in Table 6. In Table 6, Case 1 minimizes the total flight cost based on the study of [PERSON] et al. (2017). Case 2 minimizes the sum of the total flight cost and the penalty cost of items based on the study of [PERSON] et al. (2018). Case 3 minimizes the equity cost based on the study of [PERSON] et al. (2018). Case 4 and Case 5 minimize the sum of three costs and two costs other than the total flight cost, as introduced in our study. QL was performed 10 times for each case in each scenario and analyzed from three perspectives (rapidity, urgency, and equity). For each trial, the best performance (lowest objective value) transportation plan is selected for comparison. Figure 10 shows the total flight time of all UAVs in each case of each scenario. Numerical values denote the average of 10 trials, and error bars denote 95% confidence intervals. As shown in Figure 10, for all scenarios, Case 1 is seen to have the shortest flight time, while for the other cases, the length of flight time is shown to vary from scenario to scenario. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multirow{2}{*}{**Case**} & \multicolumn{4}{c}{**Rate**} \\ \cline{3-5}  & & **Rapidity (\(\lambda_{1}\))** & **Urgency (\(\lambda_{2}\))** & **Equity (\(\lambda_{3}\))** \\ \hline [PERSON] et al. (2017) & Case 1 & 1 & 0 & 0 \\ [PERSON] et al. (2018) & Case 2 & 0.5 & 0.5 & 0 \\ [PERSON] et al. (2018) & Case 3 & 0 & 0 & 1 \\ Our study & Case 4 & 0.33 & 0.33 & 0.33 \\ Our study & Case 5 & 0 & 0.5 & 0.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Rate of reward for each case. Figure 10: Total flight time of the UAV in each scenario. The values are averages of 10 training trials, and the error bars represent 95% confidence intervals. Figure 9: The model of [PERSON] et al. (2018). (**left**) Location coordinates of 1 depot (red) and 10 demand points (blue). (**center**) The optimal route obtained by PSO of [PERSON] et al. (2018). (**right**) The optimal route obtained by QL of our study. Figure 11 shows the penalty cost for each item in each scenario. The penalty cost depends on the ends of the transport time and the urgency of the item. In all scenarios, Case 1 has by far the highest penalty cost for urgent items (especially Item A). Therefore, the transportation strategy of Case 1 is not suitable for the transportation of emergency relief supplies in the event of a disaster that requires rapid transportation of supplies and has a wide range of supply urgency. Case 2, Case 3, Case 4, and Case 5 have similar values for all items in Scenario 1 and Scenario 2. On the other hand, as the size of the instance increases, the penalty cost of Case 3 increases relative to the other cases. Figure 12 shows the total service level of each shelter in each scenario. Case 3 has the lowest cost, and Case 1 has the highest cost in all scenarios. Figure 12, Figure 10, Figure 11, Case 2, Case 4, and Case 5 have similar trends. This could be because the initial demand for all shelters is the same. To minimize the objective function of urgency \(Z_{2}\), a transportation strategy that delivers items with high urgency to all shelters quickly and then delivers items with low urgency to all shelters is effective. Similarly, the objective function equity \(Z_{3}\) can be decreased by transporting a small number of items to each shelter quickly. Figures 13 and 14 show the route and resource allocation to minimize the objective value of 10 trials in Scenario 1 of Case 1 and Case 4, respectively. Figure 13 shows that in each trip, each UAV transports a large number of items to a single shelter. On the other hand, Figure 14 shows that each UAV is transporting supplies to multiple shelters on a single trip.