# Deep Reinforcement Learning Based Resource Management in UAV-Assisted IoT Networks
## 5 Proposed Method Description
### Round-Robin Scheduling Algorithm for Resource Management  
According to Reference [26], round robin is a scheduling algorithm in which each resource can be assigned with a duration of time and iteration. The algorithm mainly focuses on the time slot and time-sharing-based scheduling, and is applied to ensure that resources are allocated fairly at each time slot. If the RM processes are not assigned or completed at a given time, the allocation queue comes after other resources have arrived, which makes scheduling fair [19]. Therefore, the round robin scheduling-algorithm has various advantages, such as being easy to implement, cyclical, and starvation not occurring. For different resources, it supports first come and first serve to schedule, and gives equal access to all resources to be allocated. When a new resource request comes in, it is added to the end of the queue and is ready to be managed and allocated. Therefore, each resource has a chance of being rescheduled for allocation with a particular time slot. Generally, the following steps are applied for our proposed system to compute the resource management requests:  
1. [label=(0)]
2. Decide the resource that comes first and then start to allocate the resources as a time slot only.
3. Check the other resource requests. If there is a resource request that is available in a one-time slot while another resource request is being filled, the incoming resources are put on a waiting list as a ready queue.
4. After the time slot has gone, check for any more resource requests in the queue. If the existing RM process is not finished, add the existing request to the end of the queue.
5. Take the first request from the waiting ready queue and start to allocate it (same rules),
6. Steps (2)-(4) can be repeated.
7. If the resource request is finished and none are waiting in the queue, then the assigning work is done.  
Figure 4: The flow chart of clustering, training and scheduling phases: DRL; deep reinforcement learning.  
For instance, Table 3 illustrates the resource management process with the round robin scheduling-algorithm. It contains a list of resources (resource 1 (R1), resource 2 (R2), and resource 3 (R3)), duration, the order of the queue, and arrival time slots. Therefore, R1, R2, and R3 are given a waiting time slot of four (4), six (6), and six (6), respectively. Hence, the average waiting time (AWT) of all resource requests is \((4+6+6)/3=5.33\). Lastly, three-time slot considerations are crucial for applying RR as an RM issue.  
Then, we consider the time slot value is one (1):  
\begin{tabular}{c c c c c c c c c} \hline \hline R1 & R2 & R3 & R1 & R2 & R3 & R1 & R2 & R3 & R2 \\ \hline
0 & & & & & & & & & 10 \\ \hline (a) & Completion Time: the time slot at which the resource request allocation assignment ends. (b) & Turnaround Time: the total time that the service request is available in the queue (i.e., turnaround time = completion time \(-\) arrival time). (c) & Waiting Time (WT): the total waiting duration (i.e., waiting time = turnaround time \(-\) burst time). & & & & \\ \hline \end{tabular}