## 3 CPIO Algorithm with Cooperation-Competition Mechanism
### Design Reward Function  
The MUCS process is modeled as a multiobjective nonlinear programming function with RC, OC, no-fly zone and collision free and the model is given in Equations (7) and (8):  
\[J_{1}=\omega_{1}\sum_{q=1}^{N_{p}}P_{e}^{m,n}(k+q)+\omega_{2}\sum_{q=1}^{N_{p} }\Delta\chi_{e}^{m,n}(k+q)+\omega_{3}\sum_{q=1}^{N_{p}}F_{r}(k+q)+\omega_{4} \sum_{q=1}^{N_{p}}F_{a}(k+q), \tag{7}\]\[s.t. \begin{cases}R_{c}\in R,\\ D_{i}(k+1)=\{(D_{i}(k)-1)\bmod 8,\ D_{i}(k)\bmod 8,(D_{i}(k)+1)\bmod 8\},\\ T_{i,1}(k+1)<T_{i},\\ d_{ij}(k),...,d_{ij}(k+N_{p})>0,\end{cases}  \tag{8}\]  
where \(\omega_{1},\omega_{2},\omega_{3},\omega_{4}\) are weighting factors, which satisfy \(\omega_{1},\omega_{2},\omega_{3},\omega_{4}\in(0,1)\) and \(\omega_{1}+\omega_{2}+\omega_{3}+\omega_{4}=1\). \(R_{c}\) is the searchable region in \(R\). \(T_{i,1}(k)\) and \(T_{i}\) are the searching time and the range of \(UAV_{i}\) respectively. \(d_{ij}(k)=\sqrt{(x_{i}(k)-x_{j}(k))^{2}}+\sqrt{(y_{i}(k)-y_{j}(k))^{2}}\) denotes the Euclidean distance of \(UAV_{i}\) and \(UAV_{j}\). \(N_{p}\geq 1\) is a positive integer. The first and second terms of the \(J_{1}\) are designed to maximize search rewards, which corresponding to the first and second terms of the principles. The reduction of EU of the cell \((m,n)\) can be defined as follows:  
\[\Delta\chi_{e}^{m,n}(k)=\chi_{e}^{m,n}(k+1)-\chi_{e}^{m,n}(k). \tag{9}\]  
The third of the \(J_{1}\) is consistent with the fourth of the principles and is designed to avoid collision between UAVs:  
\[F_{r}(k)= \begin{cases}-100,&d_{\min}(k)\leq\sqrt{2},\\ e^{-\dfrac{1}{d_{\min}(k)}},&d_{\min}(k)>\sqrt{2},\end{cases}  \tag{10}\]  
where \(d_{\min}(k)\) is the minimum element of the \(d_{ij}(k)\) at time step \(k\), and \(d_{ij}(k)\) is given in Appendix A Equation (A1). When the distance between any two UAVs is less than or equal to \(\sqrt{2}\) unit lengths, the reward of this term will be \(-100\). And then the rewards of \(J_{1}\) will also be negative. \(UAV_{i}\) can only choose to search cells away from other UAVs. With regard to the fourth term of the \(J_{1}\), it implies that the further away \(UAV_{i}\) is from the airport, the more rewards it obtains (principle 5):  
\[F_{a}(k)=e^{-\dfrac{1}{d_{\Sigma}(k)}}, \tag{11}\]  
where \(d_{\sum}(k)\) is the sum of all the elements in \(d_{ih}(k)\), which is written in Appendix A Equation (A2). Definition 4: (\(N_{p}\)-step-ahead prediction, \(N_{p}\) SAP). Learning from the idea of rolling optimization in model predictive control theory [31]. In each term of \(J_{1}\) designed in Equation (7), the rewards of the reachable cells to be calculated is not only the next time step, but also the next \(N_{p}\) time steps. Remark 1: Although \(n\) steps are predicted in advance each time, \(UAV_{i}\) flies forward only one-time step. The series of reachable waypoints of \(N_{p}\)SAP of \(UAV_{i}\) maps the predicted cells sequence \(\mathcal{E}_{i}(k)=\{\mathcal{L}_{i}(k+1),...,\mathcal{L}_{i}(k+q),..., \mathcal{L}_{i}(k+N_{p})\}(q=1,...,N_{p})\) from the step \((k+1)\) to the future time step \((k+N_{p})\), which is based on the present position \(L_{i}(k)\) at time \(k\). The reachable cells included in 3 SAP construct an expanding tree, as shown in Figure 1(a). Obviously, the expanding tree generated by \(N_{p}\)SAP contains \(3^{N_{p}}\) alternative paths and the \(l\)-th path can be illustrated as:  
\[\mathcal{P}_{i}^{l}(k)=\{L_{i}^{l}(k+1),...,L_{i}^{l}(k+q),...,L_{i}^{l}(k+N_ {p})\}, \tag{12}\]  
where \(L_{i}^{l}(k+q)\in\mathcal{L}_{i}(k+q)\). The positive aspects of \(N_{p}\)SAP include:  
* Avoid greedy thought corresponding to the sixth term of principles referred in Section 2.3.2
* Circumvent the no-fly zone ahead of time and continue to search the key region
* Prohibit collision between UAVs
* Reduce the impact of communication delays  
As given in Figure 2b, Comparing the reward of \(path_{1}\) and \(path_{2}\), if \(N_{p}=1,J_{1,1}(k+1)>J_{1,2}(k+1)\), the \(path_{1}\) is chosen at this time step.