# User-Centric UAV Group Power Allocation Algorithm

Authors: Yuexia Zhang; Pengfei Zhang
Journal: Wireless Communications and Mobile Computing
 (2021)

---

## Abstract

User-centric unmanned aerial vehicle group (UUAVG) organizes a dynamic user-centric access point group for each user, which can improve user service quality and throughput. However, in UUAVG, unmanned aerial vehicle user equipment (UUE) and device-to-device user equipment (DUE) using the same frequency will cause co-frequency interference and reduce system performance. In response to this problem, a power allocation algorithm of the two-level Stackelbreg game (PAOTLSG) is proposed. The algorithm builds a two-level Stackelberg game model based on UUE and DUE and uses power control coefficients and other parameters to control UAV adjust with the transmit power of DUE to achieve dynamic balance. The optimal transmit power of UAV and DUE is theoretically deduced to achieve the game Nash equilibrium and theoretically prove the existence and uniqueness of the Nash equilibrium solution. The simulation results show that compared with the pricing-based Stackelbreg game (PSG) algorithm, the PAOTLSG algorithm can effectively increase the throughput by 4.65%.


---

## Body

## 1. Introduction

Since the commercialization of 5G networks in 2020, 5G networks have gradually entered people’s lives, but because 5G networks have not yet been widely popularized, they cannot provide people with seamless high-quality communication services [1]. UAVs have the characteristics of high maneuverability, flexible deployment, and strong environmental adaptability. They can provide reliable signal coverage for blind areas of 5G networks and can also provide task offloading for 5G networks in hotspots [2].D2D (device-to-device) communication technology can not only increase the coverage of 5G networks but also greatly increase system throughput [3]. Compared with traditional cellular communication, D2D communication does not need to directly establish link communication through the base station, which can not only effectively reduce the load of the base station but also allows more devices to be connected to the system and increase the system capacity [4].Therefore, combining UAV technology and D2D technology can increase 5G coverage, increase system capacity, and increase system throughput [5, 6]. In the UAVG network architecture with D2D, D2D users (device-to-device user equipment, DUE) reuse the spectrum resources of unmanned aerial vehicle user equipment (UUE) to improve the spectrum utilization of the entire system rate [7]. However, the traditional network architecture cannot meet the different business needs of all users, especially the quality of service (QoS) for users at the edge of the network [8].In order to improve the QoS of edge users, some researchers have proposed a user-centric unmanned aerial vehicle group (UUAVG) network structure [9–12]. In [9], the concept of user-centric is introduced and transformed the traditional base station-centered network structure into user-centered. In [10], the performance analysis of user-centric networks is introduced in detail. In [11], the handover probability in a user-centric network is analyzed. In [12], the expression of coverage probability for suppressing interference was derived through Poisson point process modeling. The user-centered network structure can make users feel that they are in the center of the network, so that the network always follows the users.The user-centric architecture can not only give full play to the advantages of UAV technology and D2D technology but also effectively compensate for the low QoS problems of UAV technology and D2D technology for edge users. Therefore, this article is based on a user-centric architecture, combined with UAV technology and D2D technology, and is committed to improving system throughput and user QoS. However, the existing interference problem in the original D2D network and UAV network architecture has not yet been resolved. How to effectively suppress the interference is still a problem that needs to be resolved.


### 1.1. Related Works

At present, some scholars have researched the complex interference problem in the abovementioned network architecture, and power control is considered to be the most effective interference suppression method.In [13–15] considered the complicated cochannel interference problem in UAVG and improved the throughput of the system by adjusting the transmission power of UAV. In [13], a centralized interference coordination technology was proposed. This scheme obtains the local optimal solution through the continuous approximation method, which can effectively reduce the interference in the network, but the solution solved by this scheme is not the global optimal solution. In [14], a power allocation algorithm based on deep learning was proposed. This algorithm is aimed at the lowest energy consumption. Although the use of deep learning to allocate UAV transmission power can effectively reduce intra-network interference, the algorithm requires a large amount of data is used for training, and the hardware requirements are high. In [15], a power allocation algorithm based on user association was proposed. This algorithm uses a successive approximation to iterate to find the best power allocation. Although this algorithm can effectively reduce cochannel interference, the complexity of the algorithm far exceeds other algorithms.In [16–18] considered the problem of cochannel interference caused by D2D users multiplexing cellular users. In [16], an algorithm to maximize the signal to interference plus noise ratio (SINR) of the user was proposed, although it can effectively reduce intranetwork interference and improve user QoS. However, this algorithm only compares with the situation without power control and lacks comparison with other algorithms. In [17], an algorithm for optimizing transmission rate and power control was proposed. This algorithm can effectively reduce intra-network interference through distributed game theory. However, the proposed D2D user as a relay auxiliary user lacks D2D as a terminal user. In [18], an algorithm to minimize interference and power consumption is proposed. Although this algorithm can effectively reduce interference and improve system throughput, the complexity of the algorithm is much higher than other algorithms.In [19–21] considers the network structure where D2D and UAV coexist. By adjusting the transmission power of UAV and D2D, the interference in the network can be effectively reduced and the system throughput can be improved. In [19], the author aims to maximize the spectrum efficiency. Although the network capacity is increased, the QoS of the access device is not considered, and UAV is a secondary user in this literature and cannot provide communication services for UUE. In [20], the author takes the maximum system throughput as the goal and proposes a hybrid time division/frequency division duplex communication scheme. Although this scheme can effectively improve the system throughput and spectrum efficiency, the proposed frequency utilization rate is far. It is lower than other algorithms that share the whole frequency band. In [21], the author considered the resource allocation problem of the UAV auxiliary network, to maximize throughput. Although the proposed algorithm can effectively reduce the interference in the system, UAV is only used as an auxiliary transmission node and lacks UAV as an air Base station considerations. In [22], the author considered the energy consumption of UAV network and jointly optimized the path and UAV’s velocities along subsequent hops to minimize the total energy consumption while satisfying the requested timeout requirement and energy budget. In [23], the author studied UAV relay-assisted Internet of Things communication networks and jointly optimize the allocated bandwidth, transmission power, and the UAV trajectory to maximize the total system throughput.In the above literatures, the problem of the same frequency interference is considered, the same frequency interference in the network is reduced to a certain extent, and the system throughput is improved. However, they are all not user-centric system models and lack of consideration for edge users.


### 1.2. Contributions

In order to solve the above problems, this paper proposes a two-level game power allocation algorithm (power allocation of two-level Stackelberg game, PAOTLSG). Its main contributions are as follows:(1)
In this paper, a user-centric UAV group network model with D2D communication is proposed. The access point group (APG) is constructed with UUE as the center. APG contains the network transmission model of D2D users. According to the different locations of UAV, UUE, and DUE, two different channel transmission models are considered(2)
This paper is aimed at maximizing the system throughput and constructing a two-level game power allocation algorithm (power allocation of two-level Stackelberg game, PAOTLSG). According to the different transmission power of DUE and UAV, the UAV and DUE are processed in layers. The first layer: first determine the profit function of each UUE and use the Stackelberg game to determine the best attenuation factorβi∗ of the transmit power of each UAV in the APG.The second layer: determine the best transmission power of each UAV according to the best attenuation factorβi∗ of the known UAV transmission power. Then set the benefits function of each DUE and use the Stackelberg game to determine the best transmit power of each DUE pdl∗.In the process of solving, because the UAV transmission power will affect the DUE game result, and the DUE game result will affect the UAV user, the UAV transmission power and the DUE transmission power are mutually restricted and mutually adjusted, in order to obtain dynamic balance. Finally, the best attenuation factor of the UAV transmission powerβi∗and the best transmission power of DUE pdl∗ are obtained. Maximize the benefits of all users(3)
It is theoretically proved that the abovementioned Nash equilibrium solution (βi∗, pdl∗)is the optimal solution, and the existence and uniqueness of the Nash equilibrium solution are proved(4)
The simulation results show that PAOTLSG can effectively reduce interference. Compared with the PSG algorithm, the system throughput can effectively increase the throughput by 4.65%


### 1.3. Paper Structure

The paper continues as follows, Section2 presents the system model under investigation. Then, the algorithm of PAOTLSG is described in Section 3. Next, the solution and proof of Nash equilibrium are in Section 4. Section 5 gives simulation results to illustrate the system performance. Finally, Section 6 concludes this paper.


## 1.1. Related Works

At present, some scholars have researched the complex interference problem in the abovementioned network architecture, and power control is considered to be the most effective interference suppression method.In [13–15] considered the complicated cochannel interference problem in UAVG and improved the throughput of the system by adjusting the transmission power of UAV. In [13], a centralized interference coordination technology was proposed. This scheme obtains the local optimal solution through the continuous approximation method, which can effectively reduce the interference in the network, but the solution solved by this scheme is not the global optimal solution. In [14], a power allocation algorithm based on deep learning was proposed. This algorithm is aimed at the lowest energy consumption. Although the use of deep learning to allocate UAV transmission power can effectively reduce intra-network interference, the algorithm requires a large amount of data is used for training, and the hardware requirements are high. In [15], a power allocation algorithm based on user association was proposed. This algorithm uses a successive approximation to iterate to find the best power allocation. Although this algorithm can effectively reduce cochannel interference, the complexity of the algorithm far exceeds other algorithms.In [16–18] considered the problem of cochannel interference caused by D2D users multiplexing cellular users. In [16], an algorithm to maximize the signal to interference plus noise ratio (SINR) of the user was proposed, although it can effectively reduce intranetwork interference and improve user QoS. However, this algorithm only compares with the situation without power control and lacks comparison with other algorithms. In [17], an algorithm for optimizing transmission rate and power control was proposed. This algorithm can effectively reduce intra-network interference through distributed game theory. However, the proposed D2D user as a relay auxiliary user lacks D2D as a terminal user. In [18], an algorithm to minimize interference and power consumption is proposed. Although this algorithm can effectively reduce interference and improve system throughput, the complexity of the algorithm is much higher than other algorithms.In [19–21] considers the network structure where D2D and UAV coexist. By adjusting the transmission power of UAV and D2D, the interference in the network can be effectively reduced and the system throughput can be improved. In [19], the author aims to maximize the spectrum efficiency. Although the network capacity is increased, the QoS of the access device is not considered, and UAV is a secondary user in this literature and cannot provide communication services for UUE. In [20], the author takes the maximum system throughput as the goal and proposes a hybrid time division/frequency division duplex communication scheme. Although this scheme can effectively improve the system throughput and spectrum efficiency, the proposed frequency utilization rate is far. It is lower than other algorithms that share the whole frequency band. In [21], the author considered the resource allocation problem of the UAV auxiliary network, to maximize throughput. Although the proposed algorithm can effectively reduce the interference in the system, UAV is only used as an auxiliary transmission node and lacks UAV as an air Base station considerations. In [22], the author considered the energy consumption of UAV network and jointly optimized the path and UAV’s velocities along subsequent hops to minimize the total energy consumption while satisfying the requested timeout requirement and energy budget. In [23], the author studied UAV relay-assisted Internet of Things communication networks and jointly optimize the allocated bandwidth, transmission power, and the UAV trajectory to maximize the total system throughput.In the above literatures, the problem of the same frequency interference is considered, the same frequency interference in the network is reduced to a certain extent, and the system throughput is improved. However, they are all not user-centric system models and lack of consideration for edge users.


## 1.2. Contributions

In order to solve the above problems, this paper proposes a two-level game power allocation algorithm (power allocation of two-level Stackelberg game, PAOTLSG). Its main contributions are as follows:(1)
In this paper, a user-centric UAV group network model with D2D communication is proposed. The access point group (APG) is constructed with UUE as the center. APG contains the network transmission model of D2D users. According to the different locations of UAV, UUE, and DUE, two different channel transmission models are considered(2)
This paper is aimed at maximizing the system throughput and constructing a two-level game power allocation algorithm (power allocation of two-level Stackelberg game, PAOTLSG). According to the different transmission power of DUE and UAV, the UAV and DUE are processed in layers. The first layer: first determine the profit function of each UUE and use the Stackelberg game to determine the best attenuation factorβi∗ of the transmit power of each UAV in the APG.The second layer: determine the best transmission power of each UAV according to the best attenuation factorβi∗ of the known UAV transmission power. Then set the benefits function of each DUE and use the Stackelberg game to determine the best transmit power of each DUE pdl∗.In the process of solving, because the UAV transmission power will affect the DUE game result, and the DUE game result will affect the UAV user, the UAV transmission power and the DUE transmission power are mutually restricted and mutually adjusted, in order to obtain dynamic balance. Finally, the best attenuation factor of the UAV transmission powerβi∗and the best transmission power of DUE pdl∗ are obtained. Maximize the benefits of all users(3)
It is theoretically proved that the abovementioned Nash equilibrium solution (βi∗, pdl∗)is the optimal solution, and the existence and uniqueness of the Nash equilibrium solution are proved(4)
The simulation results show that PAOTLSG can effectively reduce interference. Compared with the PSG algorithm, the system throughput can effectively increase the throughput by 4.65%


## 1.3. Paper Structure

The paper continues as follows, Section2 presents the system model under investigation. Then, the algorithm of PAOTLSG is described in Section 3. Next, the solution and proof of Nash equilibrium are in Section 4. Section 5 gives simulation results to illustrate the system performance. Finally, Section 6 concludes this paper.


## 2. System Model

### 2.1. Network Model

The UUAVG D2D downlink network architecture model established in this paper is shown in Figure1. Suppose there are N UAVs in the system to form an APG set. The APG set is represented by ψ, ψ=UAV1,⋯,UAVi,⋯UAVa,⋯UAVN, where UAVi represents the i-th UAV, which the coordinates are xi,yi,zizi>0, UAV−i means all UAVs in UAVG except UAVi. There are M UAV users under each UAV. The set of UAV users is represented by UUE, UUE=UUE11⋯UUEji⋯UUEMN, where UUEji represents the i-th UAV in the UAV group The coordinates of the j-th UAV user under xij,yij,0, UUEj−i represents the j-th UAV user under each UAV in UAVG except the i-th UAV. Assuming that there are K pairs of DUE under each UAV, the DUE set is represented by DUE, DUE=DUE1,⋯,DUEl⋯,DUEK, where DUEl represents the l-th pair of D2D. Each D2D pair contains a transmitting user DT and a receiving user DR. The transmitting user and one receiving user in the first pair of D2D are denoted as DTl and DRl, respectively, and their coordinates are denoted as xdtl,ydtl,0, xdrl,ydrl,0.Figure 1
Downlink network architecture model of UUAVG.It is assumed that all UAVs use OFDM to allocate frequency points for the UUEs they serve. Therefore, there is no interference between different UUEs under the same UAV. However, some UUEs use the same frequency under different UAVs, so UUEs that use the same frequency under different UAVs generate co-frequency interference with each other. The DUE uses the underlay method to compete for the UUE’s spectrum resources, which causes mutual interference between the DUE and the UUE.In order to explain the above problem more clearly, takeUUEjias an example. Assuming that UUEji is located at the origin of coordinates, the distance di,uij from UAVi and UUEji can be expressed as:
(1)di,uij=xi−xij2+yi−yij2+zi2.The distance fromUAVi to the receiving user DR_l2 in the l2-th D2D pair can be expressed by di,drl2 as:
(2)di,drl2xi−xdrl22+yi−ydrl22+zi2.The distanceddtl1,uij from the transmitting user DTl1 to the user UUEji in the l1-th pair of D2D are, respectively, expressed as:
(3)ddtl1,uij=xdtl1−xij2+ydtl1−yij2.The distancesddtl1,drl2 from the transmitting user DTl1 in the l1-th pair of D2D to the receiving user DRl2 in the l2-th pair of D2D are, respectively, expressed as:
(4)ddtl1,drl2=xdtl1−xdrl22+ydtl1−ydrl22.


### 2.2. Propagation Model

Based on the theory of random geometry, it is assumed that all UAV, UUE, and DUE in the Euclidean plane obey the homogeneous Poisson point process (HPPP) distribution according to the respective densitiesλu, λc, and λd. The path space loss considered in this paper is considered jointly by large-scale fading and small-scale fading, and the small-scale fading is a Rayleigh fading channel model with a mean value of 1. The large-scale fading is DUE to the consideration of the nonline of sight transmission (nonline of sight, NLOS) and LOS channel transmission models. The probability of LOS and NLOS is related to environmental parameters [24, 25].Because UAV is in the air, it is flexible and maneuverable, and it is rarely blocked by obstructions. Therefore, LOS (line of sight, LOS) channel transmission model is adopted for line-of-sight transmission, fromUAVi to UUEji and DRl2. Large-scale fading can be expressed as Luij=di,uij−α2 and Ldrl2=di,drl2−α2. Where α2 represents the LOS path loss index, 2<α2.D2D and UUE are on the same ground, and there may be obstructions. Therefore, the NLOS (nonline of sight, NLOS) channel transmission model is adopted. The large-scale fading ofUUEji and DTl1 to DRl2 can be expressed as Luij=duij,drl2−α3 and Ldrl2=ddtl1,drl2−α3. Where α3 represents the NLOS path loss index, and 2<α2<α3.


### 2.3. Interference Analysis

Because this article not only considers the mutual interference between D2D and D2D interference to UAV users but also considers the UAV interference to DUE. In order to describe the interference situation of UUE and DUE more clearly, takeUUEji and DRl as examples to consider the interference situation of two types of users. Among them, the UUEji downlink transmission interference model is shown in Figure 2, with the l-th pair D2D downlink interference model of the receiving user DRl is shown in Figure 3.Figure 2
Interference analysis diagram of UUE.Figure 3
Interference analysis diagram of DUE.In Figure2, taking UUEji as an example, UUEji not only receives other N−1UAV−idownlink interference but also suffers from K pairs of D2D link transmission using the same frequency DTll=1,2,⋯,K Downlink interference from users.Therefore, the signal-to-noise ratio ofUUEji can be expressed as:
(5)γuij=βipt⋅hi,uij∑a=1,a≠1Nβaptha,uij+∑l=1Mpdl⋅gdtl,uij+σui2.Among them,hi,uij represents the channel gain from UAVi to UUEji, and gdtl,uij is the channel gain fromDTl to UUEji. pt is the downlink-rated transmit power of the UAV. βi is the transmission power attenuation coefficient of UAVi0≤βi≤1, and βipt is the transmission power of UAVi. 0<βi<1, which means that the current UAV has adjusted the power of βitimes. pdl is the transmit power of DTl, and σui2 is the background noise power received by UUEi.In Figure3, taking the receiving user DRl in the l-th pairs of D2D pairs as an example, DRl is not only affected by the downlink interference from N UAVs but also by the downlink interference from M−1DTl′ users, where l′=1,2,⋯,Kand l′≠l.The signal-to-noise ratio ofDRl can be expressed as:
(6)γdl=pdl⋅gdtl,drl∑l=1′,l′≠lMpdl′⋅gdtl′,drl+∑i=1Nβipthi,drl+σdrl2,wheregdtl,drl represents the channel gain from DTl to DRl, gdl′,drl represents the channel gain from DTl′ to DRl in other D2D, and hi,drl represents the channel gain from UAVi to DRl. σdrl2 represents the background noise power of DRl.The various abbreviations encountered in the text are listed in Table1.Table 1
List of abbreviations.
AbbreviationProfessional termsUUAVGUser-centric unmanned aerial vehicle groupUUEUnmanned aerial vehicle user equipmentDUEDevice-to-device user equipmentPAOTLSGPower allocation of two-level Stackelberg gameD2DDevice-to-deviceQoSQuality of serviceSINRSignal to interference plus noise ratioAPGAccess point groupHPPPHomogeneous Poisson point processNLOSNonline of sightLOSLine of sight


## 2.1. Network Model

The UUAVG D2D downlink network architecture model established in this paper is shown in Figure1. Suppose there are N UAVs in the system to form an APG set. The APG set is represented by ψ, ψ=UAV1,⋯,UAVi,⋯UAVa,⋯UAVN, where UAVi represents the i-th UAV, which the coordinates are xi,yi,zizi>0, UAV−i means all UAVs in UAVG except UAVi. There are M UAV users under each UAV. The set of UAV users is represented by UUE, UUE=UUE11⋯UUEji⋯UUEMN, where UUEji represents the i-th UAV in the UAV group The coordinates of the j-th UAV user under xij,yij,0, UUEj−i represents the j-th UAV user under each UAV in UAVG except the i-th UAV. Assuming that there are K pairs of DUE under each UAV, the DUE set is represented by DUE, DUE=DUE1,⋯,DUEl⋯,DUEK, where DUEl represents the l-th pair of D2D. Each D2D pair contains a transmitting user DT and a receiving user DR. The transmitting user and one receiving user in the first pair of D2D are denoted as DTl and DRl, respectively, and their coordinates are denoted as xdtl,ydtl,0, xdrl,ydrl,0.Figure 1
Downlink network architecture model of UUAVG.It is assumed that all UAVs use OFDM to allocate frequency points for the UUEs they serve. Therefore, there is no interference between different UUEs under the same UAV. However, some UUEs use the same frequency under different UAVs, so UUEs that use the same frequency under different UAVs generate co-frequency interference with each other. The DUE uses the underlay method to compete for the UUE’s spectrum resources, which causes mutual interference between the DUE and the UUE.In order to explain the above problem more clearly, takeUUEjias an example. Assuming that UUEji is located at the origin of coordinates, the distance di,uij from UAVi and UUEji can be expressed as:
(1)di,uij=xi−xij2+yi−yij2+zi2.The distance fromUAVi to the receiving user DR_l2 in the l2-th D2D pair can be expressed by di,drl2 as:
(2)di,drl2xi−xdrl22+yi−ydrl22+zi2.The distanceddtl1,uij from the transmitting user DTl1 to the user UUEji in the l1-th pair of D2D are, respectively, expressed as:
(3)ddtl1,uij=xdtl1−xij2+ydtl1−yij2.The distancesddtl1,drl2 from the transmitting user DTl1 in the l1-th pair of D2D to the receiving user DRl2 in the l2-th pair of D2D are, respectively, expressed as:
(4)ddtl1,drl2=xdtl1−xdrl22+ydtl1−ydrl22.


## 2.2. Propagation Model

Based on the theory of random geometry, it is assumed that all UAV, UUE, and DUE in the Euclidean plane obey the homogeneous Poisson point process (HPPP) distribution according to the respective densitiesλu, λc, and λd. The path space loss considered in this paper is considered jointly by large-scale fading and small-scale fading, and the small-scale fading is a Rayleigh fading channel model with a mean value of 1. The large-scale fading is DUE to the consideration of the nonline of sight transmission (nonline of sight, NLOS) and LOS channel transmission models. The probability of LOS and NLOS is related to environmental parameters [24, 25].Because UAV is in the air, it is flexible and maneuverable, and it is rarely blocked by obstructions. Therefore, LOS (line of sight, LOS) channel transmission model is adopted for line-of-sight transmission, fromUAVi to UUEji and DRl2. Large-scale fading can be expressed as Luij=di,uij−α2 and Ldrl2=di,drl2−α2. Where α2 represents the LOS path loss index, 2<α2.D2D and UUE are on the same ground, and there may be obstructions. Therefore, the NLOS (nonline of sight, NLOS) channel transmission model is adopted. The large-scale fading ofUUEji and DTl1 to DRl2 can be expressed as Luij=duij,drl2−α3 and Ldrl2=ddtl1,drl2−α3. Where α3 represents the NLOS path loss index, and 2<α2<α3.


## 2.3. Interference Analysis

Because this article not only considers the mutual interference between D2D and D2D interference to UAV users but also considers the UAV interference to DUE. In order to describe the interference situation of UUE and DUE more clearly, takeUUEji and DRl as examples to consider the interference situation of two types of users. Among them, the UUEji downlink transmission interference model is shown in Figure 2, with the l-th pair D2D downlink interference model of the receiving user DRl is shown in Figure 3.Figure 2
Interference analysis diagram of UUE.Figure 3
Interference analysis diagram of DUE.In Figure2, taking UUEji as an example, UUEji not only receives other N−1UAV−idownlink interference but also suffers from K pairs of D2D link transmission using the same frequency DTll=1,2,⋯,K Downlink interference from users.Therefore, the signal-to-noise ratio ofUUEji can be expressed as:
(5)γuij=βipt⋅hi,uij∑a=1,a≠1Nβaptha,uij+∑l=1Mpdl⋅gdtl,uij+σui2.Among them,hi,uij represents the channel gain from UAVi to UUEji, and gdtl,uij is the channel gain fromDTl to UUEji. pt is the downlink-rated transmit power of the UAV. βi is the transmission power attenuation coefficient of UAVi0≤βi≤1, and βipt is the transmission power of UAVi. 0<βi<1, which means that the current UAV has adjusted the power of βitimes. pdl is the transmit power of DTl, and σui2 is the background noise power received by UUEi.In Figure3, taking the receiving user DRl in the l-th pairs of D2D pairs as an example, DRl is not only affected by the downlink interference from N UAVs but also by the downlink interference from M−1DTl′ users, where l′=1,2,⋯,Kand l′≠l.The signal-to-noise ratio ofDRl can be expressed as:
(6)γdl=pdl⋅gdtl,drl∑l=1′,l′≠lMpdl′⋅gdtl′,drl+∑i=1Nβipthi,drl+σdrl2,wheregdtl,drl represents the channel gain from DTl to DRl, gdl′,drl represents the channel gain from DTl′ to DRl in other D2D, and hi,drl represents the channel gain from UAVi to DRl. σdrl2 represents the background noise power of DRl.The various abbreviations encountered in the text are listed in Table1.Table 1
List of abbreviations.
AbbreviationProfessional termsUUAVGUser-centric unmanned aerial vehicle groupUUEUnmanned aerial vehicle user equipmentDUEDevice-to-device user equipmentPAOTLSGPower allocation of two-level Stackelberg gameD2DDevice-to-deviceQoSQuality of serviceSINRSignal to interference plus noise ratioAPGAccess point groupHPPPHomogeneous Poisson point processNLOSNonline of sightLOSLine of sight


## 3. PAOTLSG Algorithm

Since the transmit power of UAV and DUE is both controllable and is a typical distributed power control, this is very similar to a completely static game in a noncooperative game in game theory, so the process of power control can be expressed by the benefits function of a noncooperative game.


### 3.1. Two-Level Game Elements

The PAOTLSG (power allocation of two-level Stackelberg game) algorithm is based on the algorithm proposed by Stackelberg game. Stackelberg game is a noncooperative game. A noncooperative game refers to a process in which any participating user in the network seeks to maximize their own profits selfishly. In the standard game model, the three elements of the game usually include the participants of the game, the decisions of the participants, and the benefits of the participants. In the PAOTLSG model built in this article, the model is expressed asG=ψ,Γ,βi,pdl,Uij,Udl, the basic elements in the model are as follows:
(1)
Game leaders:The member of UAV:ψ=UAV1,⋯,UAVi,⋯,UAVa,⋯,UAVN(2)
Game followers:(7)Γ=DT1,DT2,⋯,DTl1,⋯,DTl2,⋯,DTK.(3)
Leader’s decision set:(8)βi=β1,β2,⋯βN.(4)
Follower’s decision set:(9)pdl=pd1,pd2,⋯,pdl1,⋯,pdl2,⋯pdk.(5)
Leader’s benefits:(10)Uijβi,pdl.(6)
Followers’ benefits:(11)Udlβi,pdl.Since this paper adopts a hierarchical Stackelberg game, it can be divided into two processes: leader decision-making and follower decision-making. The leader game is first performed. The game leader’s goal is to maximize the benefitsUijβi,pdl of all participating leaders and obtain the best attenuation factor of each UAV, which is βi∗. The DUE participating in the second layer of the game will decide its own transmission power according to the best transmission power of the first layer of UAV game, to maximize the follower’s benefits Udlβi,pdl as the goal to obtain the DUE, the best transmit power of pdl∗. In the process of solving, because the UAV transmission power will affect the DUE game result, and the DUE game result will affect the UAV user, the UAV transmission power and the DUE transmission power are mutually restricted and mutually adjusted, in order to obtain a dynamic balance. Finally, the optimal attenuation factor βi∗ of UAV transmission power and the optimal transmission power of DUE pdl∗ are obtained and maximize the benefits of all users.


### 3.2. Benefits of UUE

There is a UUE under each UAV to communicate with it, and each UAV hopes that the UUE that can be served can maximize the rate, so each UAV will always improve its own transmit power to maximize the rate of UUE, that is, to achieve the maximum transmission rateRij. Assuming that the transmission bandwidth of all users is B=1Hz, the UUE transmission rate Rijcan be expressed as:
(12)Rij=Bln1+γuij=log21+γuij.Since the selfishness of UAVs will affect UAV users served by other UAVs, it is necessary to set an interference suppression function for each UAV:(13)Cij=λu∑a=1,a≠iNhi,uaj.βipt.Among them, the UAV’s punishment factorλc>0, the transmission power of each UAVi is βipt. ∑a=1,a≠iNhi,uaj.βipt is UAVi, the total amount of interference to other UAV users. Therefore, the benefits function for the leadership UAV in this article is defined as the user rate obtained by the UAV users served by the UAV minus the interference suppression function of the UAV to other UAV users, and the expression is
(14)Uijβi,pdl=Rij−Cij.Combining formula (5), formula (7), and formula (8), the benefits function of UUE can be expressed as:
(15)Uijβi,pdl=ln1+γuij−λu∑a=1,a≠iNhi,uaj.βipt.The physical meaning of setting the benefits function in this way is to ensure that the average reachable rate of the current user is maximized. Because the first termRij of the benefits function is the user’s reachable rate; the second term of the benefits function is a suppression term to prevent the transmission power of βipt from being too large and affecting other users.


### 3.3. Benefits of DUE

After determining the optimal transmit powerβi∗pt of the leader level game, the follower DUE will decide its own transmitting power according to the determined transmitting power of UAV. Referring to the game model of UAV in the first layer, the transmission rate of DUEs in the second layer is Rdl can be expressed as:
(16)Rdl=ln1+γdl.Similarly, the interference suppression function of the second layer DUE is as follows:(17)Cdl=λd∑i=1Npdl.gdtl,uij.Therefore, the benefits function of DUE can be expressed as:(18)Udlβi,pdl=ln1+γdl−λd∑i=1Npdl.gdtl,uij.The physical meaning of setting the benefits function is to maximize the average achievable rate of the current user, because the first termRdl1 of benefits function is the average achievable rate of user DTl1. The second term Cdl1 of benefits function is a suppression term to prevent the interference of pdtl1 transmitting power to the same frequency UUE and DUE.


### 3.4. Destination of the Game

According to the knowledge of game theory, any decision made by the participants should be satisfied to maximize the leader’s benefitsUij and follower’s benefits Udl. Since the benefits function of leadership is about βi, and that of follower is about pdl, the objective of the optimization is as follows:leader’s benefitsUij:
(19)maxUijβi∗,pdl≥Uijβi,pdls.t.0≤βi≤1λu>00≤pdl≤pdlmaxλdl>0follower’s benefits Udl:
(20)maxUdlβi∗,pdl∗≥Udlβi∗,pdls.t.0≤βi≤1λu>00≤pdl≤pdlmaxλdl>0βi∗ is the best attenuation factor of UAV transmitting power, pdl∗ is the best transmitting power of DTl. According to the optimization objective formulas (14) and (15) and the constraint condition formulas (16) and (17), the hierarchical Stackelberg game is formed. Based on the game, the attenuation factor of UAV’s transmitting power and the transmitting power of DUE are updated iteratively, and the Nash equilibrium point is finally found, so that both UUE revenue and DUE revenue can reach the maximum. Among them, formula (14) guarantees that when the transmission power attenuation factor βi of any UAV is any value, the benefit Uij of UUE will not be greater than that when the transmission power attenuation factor is βi∗. Among them, formula (15) guarantees that when the transmission power pdl of any DUE is any value, the benefit Udl of UUE will not be greater than that when the transmission power is pdl∗. Only when formulas (16) and (17) are satisfied at the same time can we ensure that when UAVG and DUE make any decision, the profit of the whole system will not be greater than that of Nash equilibrium. So βi∗,pdl∗ is the Nash equilibrium solution of this game.


## 3.1. Two-Level Game Elements

The PAOTLSG (power allocation of two-level Stackelberg game) algorithm is based on the algorithm proposed by Stackelberg game. Stackelberg game is a noncooperative game. A noncooperative game refers to a process in which any participating user in the network seeks to maximize their own profits selfishly. In the standard game model, the three elements of the game usually include the participants of the game, the decisions of the participants, and the benefits of the participants. In the PAOTLSG model built in this article, the model is expressed asG=ψ,Γ,βi,pdl,Uij,Udl, the basic elements in the model are as follows:
(1)
Game leaders:The member of UAV:ψ=UAV1,⋯,UAVi,⋯,UAVa,⋯,UAVN(2)
Game followers:(7)Γ=DT1,DT2,⋯,DTl1,⋯,DTl2,⋯,DTK.(3)
Leader’s decision set:(8)βi=β1,β2,⋯βN.(4)
Follower’s decision set:(9)pdl=pd1,pd2,⋯,pdl1,⋯,pdl2,⋯pdk.(5)
Leader’s benefits:(10)Uijβi,pdl.(6)
Followers’ benefits:(11)Udlβi,pdl.Since this paper adopts a hierarchical Stackelberg game, it can be divided into two processes: leader decision-making and follower decision-making. The leader game is first performed. The game leader’s goal is to maximize the benefitsUijβi,pdl of all participating leaders and obtain the best attenuation factor of each UAV, which is βi∗. The DUE participating in the second layer of the game will decide its own transmission power according to the best transmission power of the first layer of UAV game, to maximize the follower’s benefits Udlβi,pdl as the goal to obtain the DUE, the best transmit power of pdl∗. In the process of solving, because the UAV transmission power will affect the DUE game result, and the DUE game result will affect the UAV user, the UAV transmission power and the DUE transmission power are mutually restricted and mutually adjusted, in order to obtain a dynamic balance. Finally, the optimal attenuation factor βi∗ of UAV transmission power and the optimal transmission power of DUE pdl∗ are obtained and maximize the benefits of all users.


## 3.2. Benefits of UUE

There is a UUE under each UAV to communicate with it, and each UAV hopes that the UUE that can be served can maximize the rate, so each UAV will always improve its own transmit power to maximize the rate of UUE, that is, to achieve the maximum transmission rateRij. Assuming that the transmission bandwidth of all users is B=1Hz, the UUE transmission rate Rijcan be expressed as:
(12)Rij=Bln1+γuij=log21+γuij.Since the selfishness of UAVs will affect UAV users served by other UAVs, it is necessary to set an interference suppression function for each UAV:(13)Cij=λu∑a=1,a≠iNhi,uaj.βipt.Among them, the UAV’s punishment factorλc>0, the transmission power of each UAVi is βipt. ∑a=1,a≠iNhi,uaj.βipt is UAVi, the total amount of interference to other UAV users. Therefore, the benefits function for the leadership UAV in this article is defined as the user rate obtained by the UAV users served by the UAV minus the interference suppression function of the UAV to other UAV users, and the expression is
(14)Uijβi,pdl=Rij−Cij.Combining formula (5), formula (7), and formula (8), the benefits function of UUE can be expressed as:
(15)Uijβi,pdl=ln1+γuij−λu∑a=1,a≠iNhi,uaj.βipt.The physical meaning of setting the benefits function in this way is to ensure that the average reachable rate of the current user is maximized. Because the first termRij of the benefits function is the user’s reachable rate; the second term of the benefits function is a suppression term to prevent the transmission power of βipt from being too large and affecting other users.


## 3.3. Benefits of DUE

After determining the optimal transmit powerβi∗pt of the leader level game, the follower DUE will decide its own transmitting power according to the determined transmitting power of UAV. Referring to the game model of UAV in the first layer, the transmission rate of DUEs in the second layer is Rdl can be expressed as:
(16)Rdl=ln1+γdl.Similarly, the interference suppression function of the second layer DUE is as follows:(17)Cdl=λd∑i=1Npdl.gdtl,uij.Therefore, the benefits function of DUE can be expressed as:(18)Udlβi,pdl=ln1+γdl−λd∑i=1Npdl.gdtl,uij.The physical meaning of setting the benefits function is to maximize the average achievable rate of the current user, because the first termRdl1 of benefits function is the average achievable rate of user DTl1. The second term Cdl1 of benefits function is a suppression term to prevent the interference of pdtl1 transmitting power to the same frequency UUE and DUE.


## 3.4. Destination of the Game

According to the knowledge of game theory, any decision made by the participants should be satisfied to maximize the leader’s benefitsUij and follower’s benefits Udl. Since the benefits function of leadership is about βi, and that of follower is about pdl, the objective of the optimization is as follows:leader’s benefitsUij:
(19)maxUijβi∗,pdl≥Uijβi,pdls.t.0≤βi≤1λu>00≤pdl≤pdlmaxλdl>0follower’s benefits Udl:
(20)maxUdlβi∗,pdl∗≥Udlβi∗,pdls.t.0≤βi≤1λu>00≤pdl≤pdlmaxλdl>0βi∗ is the best attenuation factor of UAV transmitting power, pdl∗ is the best transmitting power of DTl. According to the optimization objective formulas (14) and (15) and the constraint condition formulas (16) and (17), the hierarchical Stackelberg game is formed. Based on the game, the attenuation factor of UAV’s transmitting power and the transmitting power of DUE are updated iteratively, and the Nash equilibrium point is finally found, so that both UUE revenue and DUE revenue can reach the maximum. Among them, formula (14) guarantees that when the transmission power attenuation factor βi of any UAV is any value, the benefit Uij of UUE will not be greater than that when the transmission power attenuation factor is βi∗. Among them, formula (15) guarantees that when the transmission power pdl of any DUE is any value, the benefit Udl of UUE will not be greater than that when the transmission power is pdl∗. Only when formulas (16) and (17) are satisfied at the same time can we ensure that when UAVG and DUE make any decision, the profit of the whole system will not be greater than that of Nash equilibrium. So βi∗,pdl∗ is the Nash equilibrium solution of this game.


## 4. Solution and Proof of Nash Equilibrium

In this section, the Nash equilibrium solution is solved theoretically, and the existence and uniqueness of the solution are proved.


### 4.1. Nash Equilibrium Solution of Leader Level (UAV Layer)

The derivation of the benefits function of the leader level is as follows:(21)∂Uij∂βi=11+γij×∂γij∂βi−λu∑a=1,a≠iNhi,uaj.pt.Let∂Uij/∂βi=0, suppose Iij− is the total interference to UUE
(22)Iij−=∑a=1,a≠iNβaptha,uij+∑l=1Mpdl1.gdtl1,uij+σui2.The optimal transmitting power of UAV is as follows:(23)βi∗=hi,uij−λu∑a=1,a≠iNhi,uajIij−λu∑a=1,a≠iNhi,uaj.pt.hi,uij.According to the constraint condition formula (16), it can be determined that the attenuation factor βi∗ of the optimal transmission power of the UAV group is
(24)βi∗=0,βi∗≤0,βi∗,0<βi∗<1,1,βi∗≥1.The Newton iterative method is used to obtain the iterative formula of the iterative transmission power attenuation factor of UAV(25)fβ=βip=1λu∑a=1,a≠iNhi,ui′j.pt×γuijγuij+1.


### 4.2. Nash Equilibrium Solution of Follower Level (DUE Layer)

After the attenuation factorβi of UAV’s optimal transmit power is solved, the optimal transmit power βipt of each UAV can be confirmed. On this basis, DUE will determine its own transmit power according to the optimal transmit power of UAV. Since the transmitting power of UAV has been determined at this time, the benefits of follower Udl does not contain an unknown quantity βi, but uses βi∗ instead. According to the knowledge of game theory, the derivation of the benefits function of the leader level is as follows:
(26)∂Udl∂pdl=11+γdl×∂γdl∂pdl−λd.∑i=1Ngdtl,uij.Let∂Udl/∂pdl=0, suppose Idl− is the total interference to DUE
(27)Idl−=∑l′=1,l′≠lMpdtl′.gdtl′,drl1+∑i=1Nβipthi,drl+σdrl2.The optimal transmitting power ofDTl is as follows:
(28)pdl∗=gdtl,drl−λd.∑i=1Ngdtl,uij.Idl−λd.gdtl,drl.∑i=1Ngdtl,uij.According to the constraint condition formula (17), the transmitting power of DUE can be obtained as follows:
(29)pdl∗=0,pdl∗≤0,pdl∗,0<pdl∗<pdlmax,pdlmax,pdl∗>pdlmax.In order to facilitate the calculation, the Newton iteration method is used in combination with formula (6), formula (25), and formula (26), and the best expression of iteration q of DRl can be obtained as follows:
(30)fpdl=pdlq+1=γdlqγdlq+1×1λd.∑i=1Ngdtl,uij.


### 4.3. Game Process

Therefore,βi∗ in formula (21) and pdl∗ in formula (26) together constitute the Nash equilibrium solution of PAOTLSG algorithm in UUAVG network with D2D network. However, because βi∗ is a function of pdl and pdl∗ is a function of βi∗, it is necessary to repeatedly play games to reach the final Nash equilibrium in solving βi∗ and pdl∗.According to the iterative formula of formula (21) and formula (27), this paper proposes a distributed iterative power allocation algorithm as follows:Algorithm 1
1. Wheni=0, set the initial λu, set attenuation factor β1,β2⋯βi⋯βn=1 of each UAV transmitting power.2. The iterative operation is carried out according to formula (22).3. Until the new transmission powerβip+1=βip, Stop the iteration, otherwise repeat step 2. In order to ensure that the transmission power after the update is unchanged from that before the update and tends to be stable.4. When the iteration is stopped, the best attenuation factorβi∗ of the transmission power of each UAV is obtained.5. Whenl=0, set the initial price λd, according to the best attenuation factor of the UAV transmission power βi∗, set the initial transmission power p0of the DT user, pd1,pd2⋯pdl⋯pn=p0。.6. The iterative operation is carried out according to formula (27).7. Until the new transmit powerpdlq+1−pdlq<ε, stop the iteration, otherwise repeat step 6. Among them is a very small value ε set to ensure that the updated transmit power and the transmit power before the update remains unchanged and tend to be stable.8. When iteration is stopped, the optimal transmission power of each DT user is obtained bypdl∗.9. Since the user’s transmission power will affect the central UAV users, it is necessary to recalculate the interference of DUE received by UAV users. According to formula (20), the optimal transmission power attenuation factor βip+2 of UAV is solved, and if βip+2−βi∗<ε, the iteration ends. At this time, βi∗,pdl∗ is the required value. Otherwise, repeat step 2 to get Nash equilibrium again.


### 4.4. Proof of Nash Equilibrium

Because the proof of Nash equilibrium of leadership and follower is similar, this paper only takes the Nash equilibrium solution of the leadership Game as an example.


#### 4.4.1. Proof of the Existence of Nash Equilibrium Solution for Leaders

Theorem 1.
Price for UAVsλu. There must be Nash equilibrium solution in the noncooperative game between UAV groups.
There are Nash equilibrium solutions in noncooperative games, which need to meet the following three conditions:(1)
The set of all players in the game is limited(2)
The transmit power decision sets of all UAVs are closed and bounded(3)
The benefits function is continuous and quasi concave on the decision set of all UAVsProof of Theorem1.The specific process is as follows:(1)
The leader UAV groupψ=0,1,2,⋯,N participating in the game is a finite set(2)
The decision set of all UAVs isβi=β1,β2,⋯βN. The decision of each UAV is independent and 0≤βi≤1, closed and bounded(3)
The second derivative of benefits function of UAV is obtained as follows:(31)∂2Uij∂βi2=1βipthi,uij+Iij−∗pthi,uij1′=−pthi,uij2βipthi,uij+Iij−2<0.According to formula (28), we can get ∂2Uij/∂βi2<0, so the benefits function is quasiconcave. In conclusion, there is a Nash equilibrium solution in the noncooperative game between UAVs.


#### 4.4.2. Proof of the Uniqueness of Nash Equilibrium Solution for Leaders

Theorem 2.
Price for UAVsλu. The Nash equilibrium solution in the noncooperative game between UAV groups must be unique.
To converge to the unique Nash equilibrium solution of noncooperative game, the following three conditions need to be satisfied:(1)
Nonnegative properties of functions:(32)fβ≥0.(2)
Monotonicity of functions:for anyβa≥βb, there is fβa≥fβb(3)
Expansibility of functions:ifμ>1, then μfβ≥fμβ.Proof of Theorem2.
(1)
According to formula (21), we can get 0≤βi≤1 and βip+1=fβip>0, obviously(2)
By deriving the function in formula (22), we can get that:
(33)fβ′=1λu∑a=1,a≠iNhi,uaj.pt×1γuij+12.Obviously,fβ′>0, so fβ is a monotone increasing function.That is, if there is anyβa≥βb, there will be fβa≥fβb, obviously.(3)
(34)Lβ=μfβ−fμβ.According to formula (27), we can get
(35)μfβ=μλu∑a=1,a≠iNhi,uaj.pt×γuijγuij+1,fμβ=1λu∑a=1,a≠iNhi,uaj.pt×μγuijμγuij+1,so(36)Lβ=μγuijμ−1γuij+1μγuij+1λu∑a=1,a≠iNhi,uaj.pt.∵μ>1, the other items in Lβ are positive
(37)∴Lβ>0,μfβ≥fμβ.In conclusion, there is a unique Nash equilibrium solution of a noncooperative game between UAVs.The solution of the follower level game is similar, so we do not need to do additional verification.


## 4.1. Nash Equilibrium Solution of Leader Level (UAV Layer)

The derivation of the benefits function of the leader level is as follows:(21)∂Uij∂βi=11+γij×∂γij∂βi−λu∑a=1,a≠iNhi,uaj.pt.Let∂Uij/∂βi=0, suppose Iij− is the total interference to UUE
(22)Iij−=∑a=1,a≠iNβaptha,uij+∑l=1Mpdl1.gdtl1,uij+σui2.The optimal transmitting power of UAV is as follows:(23)βi∗=hi,uij−λu∑a=1,a≠iNhi,uajIij−λu∑a=1,a≠iNhi,uaj.pt.hi,uij.According to the constraint condition formula (16), it can be determined that the attenuation factor βi∗ of the optimal transmission power of the UAV group is
(24)βi∗=0,βi∗≤0,βi∗,0<βi∗<1,1,βi∗≥1.The Newton iterative method is used to obtain the iterative formula of the iterative transmission power attenuation factor of UAV(25)fβ=βip=1λu∑a=1,a≠iNhi,ui′j.pt×γuijγuij+1.


## 4.2. Nash Equilibrium Solution of Follower Level (DUE Layer)

After the attenuation factorβi of UAV’s optimal transmit power is solved, the optimal transmit power βipt of each UAV can be confirmed. On this basis, DUE will determine its own transmit power according to the optimal transmit power of UAV. Since the transmitting power of UAV has been determined at this time, the benefits of follower Udl does not contain an unknown quantity βi, but uses βi∗ instead. According to the knowledge of game theory, the derivation of the benefits function of the leader level is as follows:
(26)∂Udl∂pdl=11+γdl×∂γdl∂pdl−λd.∑i=1Ngdtl,uij.Let∂Udl/∂pdl=0, suppose Idl− is the total interference to DUE
(27)Idl−=∑l′=1,l′≠lMpdtl′.gdtl′,drl1+∑i=1Nβipthi,drl+σdrl2.The optimal transmitting power ofDTl is as follows:
(28)pdl∗=gdtl,drl−λd.∑i=1Ngdtl,uij.Idl−λd.gdtl,drl.∑i=1Ngdtl,uij.According to the constraint condition formula (17), the transmitting power of DUE can be obtained as follows:
(29)pdl∗=0,pdl∗≤0,pdl∗,0<pdl∗<pdlmax,pdlmax,pdl∗>pdlmax.In order to facilitate the calculation, the Newton iteration method is used in combination with formula (6), formula (25), and formula (26), and the best expression of iteration q of DRl can be obtained as follows:
(30)fpdl=pdlq+1=γdlqγdlq+1×1λd.∑i=1Ngdtl,uij.


## 4.3. Game Process

Therefore,βi∗ in formula (21) and pdl∗ in formula (26) together constitute the Nash equilibrium solution of PAOTLSG algorithm in UUAVG network with D2D network. However, because βi∗ is a function of pdl and pdl∗ is a function of βi∗, it is necessary to repeatedly play games to reach the final Nash equilibrium in solving βi∗ and pdl∗.According to the iterative formula of formula (21) and formula (27), this paper proposes a distributed iterative power allocation algorithm as follows:Algorithm 1
1. Wheni=0, set the initial λu, set attenuation factor β1,β2⋯βi⋯βn=1 of each UAV transmitting power.2. The iterative operation is carried out according to formula (22).3. Until the new transmission powerβip+1=βip, Stop the iteration, otherwise repeat step 2. In order to ensure that the transmission power after the update is unchanged from that before the update and tends to be stable.4. When the iteration is stopped, the best attenuation factorβi∗ of the transmission power of each UAV is obtained.5. Whenl=0, set the initial price λd, according to the best attenuation factor of the UAV transmission power βi∗, set the initial transmission power p0of the DT user, pd1,pd2⋯pdl⋯pn=p0。.6. The iterative operation is carried out according to formula (27).7. Until the new transmit powerpdlq+1−pdlq<ε, stop the iteration, otherwise repeat step 6. Among them is a very small value ε set to ensure that the updated transmit power and the transmit power before the update remains unchanged and tend to be stable.8. When iteration is stopped, the optimal transmission power of each DT user is obtained bypdl∗.9. Since the user’s transmission power will affect the central UAV users, it is necessary to recalculate the interference of DUE received by UAV users. According to formula (20), the optimal transmission power attenuation factor βip+2 of UAV is solved, and if βip+2−βi∗<ε, the iteration ends. At this time, βi∗,pdl∗ is the required value. Otherwise, repeat step 2 to get Nash equilibrium again.


## 4.4. Proof of Nash Equilibrium

Because the proof of Nash equilibrium of leadership and follower is similar, this paper only takes the Nash equilibrium solution of the leadership Game as an example.


### 4.4.1. Proof of the Existence of Nash Equilibrium Solution for Leaders

Theorem 1.
Price for UAVsλu. There must be Nash equilibrium solution in the noncooperative game between UAV groups.
There are Nash equilibrium solutions in noncooperative games, which need to meet the following three conditions:(1)
The set of all players in the game is limited(2)
The transmit power decision sets of all UAVs are closed and bounded(3)
The benefits function is continuous and quasi concave on the decision set of all UAVsProof of Theorem1.The specific process is as follows:(1)
The leader UAV groupψ=0,1,2,⋯,N participating in the game is a finite set(2)
The decision set of all UAVs isβi=β1,β2,⋯βN. The decision of each UAV is independent and 0≤βi≤1, closed and bounded(3)
The second derivative of benefits function of UAV is obtained as follows:(31)∂2Uij∂βi2=1βipthi,uij+Iij−∗pthi,uij1′=−pthi,uij2βipthi,uij+Iij−2<0.According to formula (28), we can get ∂2Uij/∂βi2<0, so the benefits function is quasiconcave. In conclusion, there is a Nash equilibrium solution in the noncooperative game between UAVs.


### 4.4.2. Proof of the Uniqueness of Nash Equilibrium Solution for Leaders

Theorem 2.
Price for UAVsλu. The Nash equilibrium solution in the noncooperative game between UAV groups must be unique.
To converge to the unique Nash equilibrium solution of noncooperative game, the following three conditions need to be satisfied:(1)
Nonnegative properties of functions:(32)fβ≥0.(2)
Monotonicity of functions:for anyβa≥βb, there is fβa≥fβb(3)
Expansibility of functions:ifμ>1, then μfβ≥fμβ.Proof of Theorem2.
(1)
According to formula (21), we can get 0≤βi≤1 and βip+1=fβip>0, obviously(2)
By deriving the function in formula (22), we can get that:
(33)fβ′=1λu∑a=1,a≠iNhi,uaj.pt×1γuij+12.Obviously,fβ′>0, so fβ is a monotone increasing function.That is, if there is anyβa≥βb, there will be fβa≥fβb, obviously.(3)
(34)Lβ=μfβ−fμβ.According to formula (27), we can get
(35)μfβ=μλu∑a=1,a≠iNhi,uaj.pt×γuijγuij+1,fμβ=1λu∑a=1,a≠iNhi,uaj.pt×μγuijμγuij+1,so(36)Lβ=μγuijμ−1γuij+1μγuij+1λu∑a=1,a≠iNhi,uaj.pt.∵μ>1, the other items in Lβ are positive
(37)∴Lβ>0,μfβ≥fμβ.In conclusion, there is a unique Nash equilibrium solution of a noncooperative game between UAVs.The solution of the follower level game is similar, so we do not need to do additional verification.


## 4.4.1. Proof of the Existence of Nash Equilibrium Solution for Leaders

Theorem 1.
Price for UAVsλu. There must be Nash equilibrium solution in the noncooperative game between UAV groups.
There are Nash equilibrium solutions in noncooperative games, which need to meet the following three conditions:(1)
The set of all players in the game is limited(2)
The transmit power decision sets of all UAVs are closed and bounded(3)
The benefits function is continuous and quasi concave on the decision set of all UAVsProof of Theorem1.The specific process is as follows:(1)
The leader UAV groupψ=0,1,2,⋯,N participating in the game is a finite set(2)
The decision set of all UAVs isβi=β1,β2,⋯βN. The decision of each UAV is independent and 0≤βi≤1, closed and bounded(3)
The second derivative of benefits function of UAV is obtained as follows:(31)∂2Uij∂βi2=1βipthi,uij+Iij−∗pthi,uij1′=−pthi,uij2βipthi,uij+Iij−2<0.According to formula (28), we can get ∂2Uij/∂βi2<0, so the benefits function is quasiconcave. In conclusion, there is a Nash equilibrium solution in the noncooperative game between UAVs.


## 4.4.2. Proof of the Uniqueness of Nash Equilibrium Solution for Leaders

Theorem 2.
Price for UAVsλu. The Nash equilibrium solution in the noncooperative game between UAV groups must be unique.
To converge to the unique Nash equilibrium solution of noncooperative game, the following three conditions need to be satisfied:(1)
Nonnegative properties of functions:(32)fβ≥0.(2)
Monotonicity of functions:for anyβa≥βb, there is fβa≥fβb(3)
Expansibility of functions:ifμ>1, then μfβ≥fμβ.Proof of Theorem2.
(1)
According to formula (21), we can get 0≤βi≤1 and βip+1=fβip>0, obviously(2)
By deriving the function in formula (22), we can get that:
(33)fβ′=1λu∑a=1,a≠iNhi,uaj.pt×1γuij+12.Obviously,fβ′>0, so fβ is a monotone increasing function.That is, if there is anyβa≥βb, there will be fβa≥fβb, obviously.(3)
(34)Lβ=μfβ−fμβ.According to formula (27), we can get
(35)μfβ=μλu∑a=1,a≠iNhi,uaj.pt×γuijγuij+1,fμβ=1λu∑a=1,a≠iNhi,uaj.pt×μγuijμγuij+1,so(36)Lβ=μγuijμ−1γuij+1μγuij+1λu∑a=1,a≠iNhi,uaj.pt.∵μ>1, the other items in Lβ are positive
(37)∴Lβ>0,μfβ≥fμβ.In conclusion, there is a unique Nash equilibrium solution of a noncooperative game between UAVs.The solution of the follower level game is similar, so we do not need to do additional verification.


## 5. Simulation Results

### 5.1. Simulation Parameter

In order to verify the effectiveness of the PAOTLSG algorithm, this paper constructs a user-centered network architecture. The radius of UAV coverage is 500 m, and the users in the coverage are randomly distributed. Within 100 m distance from the UAV to corresponding communication users, the DUE can form D2D pairs within 50 m, and the noise power is -114 dBm. The minimum SINR of UAV users is 6 dB, and that of DUE is 7 dB. Assuming that there are four UAV users and two DUEs, the rated transmitting power of the UAV is 40 dBm, and the maximum transmitting power of DUEs is 20 dBm.λu=10,λdl=10.


### 5.2. Simulation Results and Analysis

As shown in Figure4, the abscissa in the figure is the number of iterations, the ordinate is the user’s transmit power, and the curve is expressed as the transmit power of UAV and DUE, respectively. It can be seen from the figure that with the increase of iteration times, the transmission power of UAV and DUE gradually decreases to a stable state. Among them, the transmitting power of UAV is higher, and its change is obvious, while the transmitting power of DUE is lower and its change is gentle. Among them, in the first four iterations, the initial transmit power of the UAV is higher, and it gradually reduces to a stable state after four games. This is because when setting the initial UAV transmission power, each UAV uses the rated transmission power pt. In the process of the game, each UAV gradually reduces the transmission power to reduce the interference to other users. In the first four iterations, the initial transmit power of DUE is low, the power value does not change, and it is always in a stable state. This is because this paper adopts the hierarchical game, giving priority to the game of leader level. When the game of leader level is not completed, DUE does not participate in the game, so there is no need to adjust the power. During iteration 5-8, the transmit power of the UAV layer does not change, while the transmit power of the D2D layer gradually decreases to a stable state. Therefore, when the D2D layer plays the game, the UAV layer adopts the βi∗pt transmit data without participating in the game. However, DUE needs to reduce its own transmission power to reduce the interference to other DUE. After the eighth iteration, some UAVs reduce the transmit power, and the change is small. This is because after the final pdl∗ is generated after the DUE game, the UAV users are affected by the D2D interference, so the UAV layer game needs to be carried out again. However, because the transmission power of DUE is far less than that of UAV, that is to say, the transmission power of DUE has little influence on the UAV layer game, so the transmission power of UAV has little change. The D2D layer game is based on the UAV layer transmit power βi∗pt, the transmitting power of the UAV layer hardly changes, so the transmitting power of DUE changes little. Similarly, Figure 4 shows the effectiveness and convergence of the proposed algorithm.Figure 4
The curve of transmitting power with iteration times.Figure5 shows the variation curve of system throughput with the number of iterations under different algorithms, where the abscissa is the number of iterations and the ordinate is the system throughput. The two curvilinear in the figure: the square lines and hexagonal star lines represent the PAOTLSG algorithm in this paper and the PSG algorithm in reference [26]. It can be seen from the figure that the two curves are gradually promoted to a stable state with the increase of the number of iterations. In the first four iterations, the system throughput of the PSG algorithm is improved faster than that of the PAOTLSG algorithm. This is because, at the beginning of the iteration, DUE users in the PAOTLSG algorithm do not participate in the game, which causes greater interference to UUE and their own interference, so the system throughput is low. After four iterations, the system throughput of the PSG algorithm does not change, because the iteration is completed at this time, and no power adjustment is needed. In the PAOTLSG algorithm, the game of leader level has just been completed, and the equilibrium solution βi∗ has been solved. At this time, DUE has not participated in the iteration, and all UAVs and DUE have power changes, so the system throughput remains unchanged at this time. After the fourth iteration, the system throughput gradually increases to a stable state. The reason for this change is that DUE to the start of the iteration, the interference of DUE to UUE and the mutual interference between DUE are reduced. After many games, the throughput of the system is gradually improved to a stable state. After stabilization, the throughput of the PAOTLSG algorithm is 4.65% higher than that of the PSG algorithm.Figure 5
The curve of system throughput with the number of iterations.Figure6 describes the system throughput curve with the number of DUEs and compares it with the conventional algorithm in reference [27]. The abscissa is the number of DUE, and the ordinate is the system throughput. With the increase of the number of DUEs, the system throughput shows an upward trend and finally tends to be stable. This is because the increase of DUE will improve the system throughput, but at the same time, it also brings interference to UAV users and other DUE, reducing the signal to interference noise ratio of other users, so the growth rate becomes slower and tends to be stable. Compared with the conventional algorithm, the proposed algorithm can effectively improve the system throughput, because this paper not only considers the interference of UAV group on DUE but also considers the mutual interference between DUE. In addition, when the number of DUE is certain, the increase of UUE will also improve the throughput, but with the increase of UAV users, the system throughput tends to be stable.Figure 6
The curve of system throughput versus the number of D2Ds.


## 5.1. Simulation Parameter

In order to verify the effectiveness of the PAOTLSG algorithm, this paper constructs a user-centered network architecture. The radius of UAV coverage is 500 m, and the users in the coverage are randomly distributed. Within 100 m distance from the UAV to corresponding communication users, the DUE can form D2D pairs within 50 m, and the noise power is -114 dBm. The minimum SINR of UAV users is 6 dB, and that of DUE is 7 dB. Assuming that there are four UAV users and two DUEs, the rated transmitting power of the UAV is 40 dBm, and the maximum transmitting power of DUEs is 20 dBm.λu=10,λdl=10.


## 5.2. Simulation Results and Analysis

As shown in Figure4, the abscissa in the figure is the number of iterations, the ordinate is the user’s transmit power, and the curve is expressed as the transmit power of UAV and DUE, respectively. It can be seen from the figure that with the increase of iteration times, the transmission power of UAV and DUE gradually decreases to a stable state. Among them, the transmitting power of UAV is higher, and its change is obvious, while the transmitting power of DUE is lower and its change is gentle. Among them, in the first four iterations, the initial transmit power of the UAV is higher, and it gradually reduces to a stable state after four games. This is because when setting the initial UAV transmission power, each UAV uses the rated transmission power pt. In the process of the game, each UAV gradually reduces the transmission power to reduce the interference to other users. In the first four iterations, the initial transmit power of DUE is low, the power value does not change, and it is always in a stable state. This is because this paper adopts the hierarchical game, giving priority to the game of leader level. When the game of leader level is not completed, DUE does not participate in the game, so there is no need to adjust the power. During iteration 5-8, the transmit power of the UAV layer does not change, while the transmit power of the D2D layer gradually decreases to a stable state. Therefore, when the D2D layer plays the game, the UAV layer adopts the βi∗pt transmit data without participating in the game. However, DUE needs to reduce its own transmission power to reduce the interference to other DUE. After the eighth iteration, some UAVs reduce the transmit power, and the change is small. This is because after the final pdl∗ is generated after the DUE game, the UAV users are affected by the D2D interference, so the UAV layer game needs to be carried out again. However, because the transmission power of DUE is far less than that of UAV, that is to say, the transmission power of DUE has little influence on the UAV layer game, so the transmission power of UAV has little change. The D2D layer game is based on the UAV layer transmit power βi∗pt, the transmitting power of the UAV layer hardly changes, so the transmitting power of DUE changes little. Similarly, Figure 4 shows the effectiveness and convergence of the proposed algorithm.Figure 4
The curve of transmitting power with iteration times.Figure5 shows the variation curve of system throughput with the number of iterations under different algorithms, where the abscissa is the number of iterations and the ordinate is the system throughput. The two curvilinear in the figure: the square lines and hexagonal star lines represent the PAOTLSG algorithm in this paper and the PSG algorithm in reference [26]. It can be seen from the figure that the two curves are gradually promoted to a stable state with the increase of the number of iterations. In the first four iterations, the system throughput of the PSG algorithm is improved faster than that of the PAOTLSG algorithm. This is because, at the beginning of the iteration, DUE users in the PAOTLSG algorithm do not participate in the game, which causes greater interference to UUE and their own interference, so the system throughput is low. After four iterations, the system throughput of the PSG algorithm does not change, because the iteration is completed at this time, and no power adjustment is needed. In the PAOTLSG algorithm, the game of leader level has just been completed, and the equilibrium solution βi∗ has been solved. At this time, DUE has not participated in the iteration, and all UAVs and DUE have power changes, so the system throughput remains unchanged at this time. After the fourth iteration, the system throughput gradually increases to a stable state. The reason for this change is that DUE to the start of the iteration, the interference of DUE to UUE and the mutual interference between DUE are reduced. After many games, the throughput of the system is gradually improved to a stable state. After stabilization, the throughput of the PAOTLSG algorithm is 4.65% higher than that of the PSG algorithm.Figure 5
The curve of system throughput with the number of iterations.Figure6 describes the system throughput curve with the number of DUEs and compares it with the conventional algorithm in reference [27]. The abscissa is the number of DUE, and the ordinate is the system throughput. With the increase of the number of DUEs, the system throughput shows an upward trend and finally tends to be stable. This is because the increase of DUE will improve the system throughput, but at the same time, it also brings interference to UAV users and other DUE, reducing the signal to interference noise ratio of other users, so the growth rate becomes slower and tends to be stable. Compared with the conventional algorithm, the proposed algorithm can effectively improve the system throughput, because this paper not only considers the interference of UAV group on DUE but also considers the mutual interference between DUE. In addition, when the number of DUE is certain, the increase of UUE will also improve the throughput, but with the increase of UAV users, the system throughput tends to be stable.Figure 6
The curve of system throughput versus the number of D2Ds.


## 6. Conclusion

This paper considers a user-centric UAV swarm with DUE network environment and proposes a PAOTLSG algorithm. The algorithm first adopts the hierarchical Stackelberg game to formulate different profit functions for different game participants, uses the Stackelberg game to find the best attenuation factor of the UAV transmission power, and obtains the best DUE transmission power through the game. And use the knowledge of game theory to prove the existence and uniqueness of the solution. The simulation results show that the algorithm in this paper can effectively improve the system throughput and reduce the interference in the UUAVG. The next step will consider the impact of full-duplex technology on UUAVG.



---

*Source: 1629012-2021-10-31.xml*