# A Reinforcement Learning Model of Multiple UAVs for Transporting Emergency Relief Supplies
## 1 Introduction
In conventional UAVRPs for delivery, the objective function is set as follows total traveling/delivery time [17; 18; 19; 20], total travel distance [21; 22], total number of covered tasks [22], additional costs due to outside delivery deadlines [21; 23], location priority [18], and total disutility for the delivery [24]. However, in the post-disaster HL situation, applying these objective functions is not adequate. Equitable relief supplies distribution among recipients is also a critical consideration in post-disaster HL [28]. This is due to the high stakes associated with unsatisfied and/or late-satisfied demand [1]. Although considerations of equity are featured in the land transportation literature [1; 29], the authors are unaware of instances where equity was considered in relation to UAVRPs. Considering equity cost as an objective function in UAVRPs may allow for optimal transportation of goods in disaster situations. In this study, motivated by challenges associated with the transportation of emergency relief supplies to isolated areas by tsunami, we present a model for planning the transportation of supplies by multiple UAVs in unique situations during a disaster. In this case, not only are many different types of supplies needed, but also the disparity in the amounts of supplies to be transported to each shelter must be minimized. In this study, we formulated the problem as an NP-hard UAV routing problem that determines the amount of supplies to be transported by multiple UAVs and destination shelters and then used Q-learning (QL) [30], a typical method of reinforcement learning (RL), to improve the efficiency of planning the transportation of multiple UAVs. The purpose of this study is to propose an efficient logistics planning strategy for disaster response using multiple UAVs. Our proposed method considered the following three perspectives: (i) the rapid transport of supplies, (ii) the urgency of supplies, and (iii) the equity (fairness) in the amount of supplies for each shelter. The main contributions of this study are as follows:* In the UAVRPs, equity metric was implemented to take into account the post-disaster HL situation, demonstrating that the disparity in supplies per evacuation center can be eliminated;
* We proposed a QL algorithm for solving UAVRPs, discussed the parameter settings of QL, and outperformed meta-heuristics methods in the conditions of the previous study;
* We introduced three metrics that are considered important in the transportation of supplies in a post-disaster HL. We tested several transport strategies with different weights of the three metrics and evaluated the response of each strategy to each metric. The remainder of this study is organized as follows: Section 2 reviews previous studies related to disaster response using UAVs and the logistics of UAVs. Section 3 gives an overview of the model constructed in this study. Section 4 describes the proposed method, QL. Section 5 describes the comparison and validation of the methods used in this study, the results, and the discussion. Section 6 gives the conclusion and future work.