# Distributed Unmanned Aerial Vehicle Cluster Testing Method Based on Deep Reinforcement Learning
## 3 The Formalization of UTDR  
The problem presented in this work can be formalized to allocate the received testing task requests to the UAV cluster network in the time window \(\Delta t\), so as to achieve node load balance and minimize bandwidth resource cost in the system. To solve this problem, this work defines a quintuple \(W=\{ND,\,TK,\,C_{cpu},\,C_{mem},\,C_{link}\}\) to describe this problem scenario. \(ND\) indicates a set of available processing nodes. \(ND(n,\,t)=\{nd_{1},\,nd_{2},\,\ldots,\,nd_{n}\}\). \(t\) indicates the start time of task assignment. \(TK\) represents the set of task requests from users in the \(\Delta t\) time window, \(TK\) (\(m,\,\Delta t,\,t\)) = \(\{tk_{1},\,tk_{2},\,\ldots,\,tk_{m}\}\). \(C_{cpu}\) is the set of remaining CPU resources of \(n\) UAV nodes in \(ND\), \(C_{cpu}(n,\,t)=\{C_{cpu}^{1},C_{cpu}^{2},\,\ldots,\,C_{cpu}^{n}\}\). \(C_{mem}\) is the collection of memory resources currently remaining in \(n\) UAV nodes in \(ND\), \(C_{mem}(n,\,t)=\{C_{mem}^{1},\,C_{mem}^{2},\,\ldots,\,C_{mem}^{n}\}\). If the DAG is known, \(bw(i,\,i^{\prime})\) represents the set of bandwidth resource costs for the communication between node \(i\) and its neighbors \(i^{\prime}\) in the cluster. In view of the task assignment problem of the UAV test proposed in this work, the final solution we obtain should be the optimal network task assignment scheme that can solve both computing resources and bandwidth resources. A description of the main functions is shown in Table 1. The first objective function in this section is based on load balancing. We use the load balancing degree to measure the load balancing effect in a multi-UAV cluster environment. The load balancing degree of the \(i\)th UAV node is calculated as follows:  
\[ \begin{array}{l}WL^{if}=\partial WL^{if}_{cpu}+\beta WL^{if}_{mem},i\in\{1,2,3,\ldots,n\},\\ f\in\{1,2,3,\ldots,z\},\partial+\beta=1.\end{array}  \tag{1}\]  
where \(WL_{cpu}\)\({}^{ip}\) and \(WL_{mem}\)\({}^{ip}\), respectively, represent the CPU resource utilization and memory resource utilization of the \(i\)th UAV processing node after the task is completed according to the solution vector \(f\). Thus, the resource utilization of the \(i\)th node can be expressed as follows:  
\[U^{if}=WL^{if}/R^{i},\,i\in\{1,2,3,\ldots,n\},f\in\{1,2,3,\ldots,z\}. \tag{2}\]\(R^{i}\) can be expressed as follows:  
\[R^{i}=\alpha R^{i}_{cpu}+\beta R^{i}_{mem},\;i\in\{1,2,3,\ldots,n\}. \tag{3}\]  
where \(R_{cpu}\)_if_ and \(R_{mem}\)_if_ represent the total CPU resources and memory resources of the \(i\)th node, respectively. This work uses the standard deviation of resource utilization to measure load balancing \(A\). The load balancing effect can be reflected by the load balancing degree. \(U\)_if_ indicates the current resource utilization of the \(i\)th UAV node, and \(n\) indicates the number of processing nodes. The formula of average resource utilization in a certain period of time is as follows:  
\[Mean(U)=\sum_{i=1}^{n}U^{if}/n. \tag{4}\]  
We can obtain the formula of the load balancing degree, which is as follows:  
\[A=\sqrt{\frac{1}{n}\sum_{i=1}^{n}\left(U^{if}-Mean(U)\right)^{2}}. \tag{5}\]  
By the above formulas, the first objective function presented in this work can be minimized and defined as follows:  
\[A^{f}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}\left(\frac{\alpha WL^{if}_{cpu}+\beta WL ^{if}_{mem}}{\alpha R^{i}_{cpu}+\beta R^{i}_{mem}}-\frac{\sum\limits_{e=1}^{n }\frac{\alpha WL^{ef}_{cpu}+\beta WL^{ef}_{mem}}{\alpha R^{e}_{cpu}+\beta R^ {e}_{mem}}}{n}\right)^{2}}. \tag{6}\]  
Thus, we can achieve the goal of load balancing by minimizing \(A^{f}\). The second objective function of this work is to reduce the cost of bandwidth resources required for inter-node communication.