Patent Publication Number: US-11641591-B2

Title: Optimization method for UAV-based wireless information and energy transmission

Description:
CROSS REFERENCE TO THE RELATED APPLICATIONS 
     This application is the national phase entry of International Application No. PCT/CN2018/110483, filed on Oct. 16, 2018, which is based upon and claims priority to Chinese Patent Application No. 201810774969.2, filed on Jul. 16, 2018, the entire contents of which are incorporated herein by reference. 
     TECHNICAL FIELD 
     The present disclosure relates to the field of improved energy balancing distribution technologies, and more particularly, to an optimization method for UAV-based wireless information and energy transmission. 
     BACKGROUND 
     Abbreviated as an unmanned aerial vehicle (UAV), a pilotless aircraft is an unmanned aircraft based on wireless remote control and can also be programmed to achieve automatic flight. In recent years, related technologies of the UAV have become more and more mature, playing an important role in the field of wireless communication and wireless charging. The reason why the UAV can play an important role in the field of wireless communication and wireless charging is that the UAV can flexibly move and can be easily arranged wherever needed. Furthermore, due to proximity to a communication objective, a better communication environment can be obtained, and thus the data transmission rate and the energy transmission efficiency may be improved. 
     In China, with the popularization of the Internet of Things, more and more wireless devices are used in industry and daily life, such as smart factories and smart homes. The use of the wireless devices can save wiring costs and beautify space. However, numerous wireless devices such as sensors in the Internet of Things are smaller in size and lower in power. Costs of recycling, charging, and rearranging these wireless devices are high. Charging the wireless devices by using traditional wireless charging technologies often faces the problem that it is difficult to arrange charging base stations nearby. In addition, improving the data transmission rate is also one of the objectives of optimizing the Internet of Things. Therefore, how to use the UAV in information and energy transmission of the wireless devices and improve the data transmission rate and the energy conversion efficiency of the network is a practical research topic. 
     References [1] and [2] both propose the use of microwave wireless charging technologies in cognitive radio networks to improve spectrum utilization and solve the charging problem of the wireless devices. However, the charging base stations used are fixed on the ground and thus cannot move flexibly. Sometimes, it is difficult to arrange the charging base stations nearby the wireless devices. In References [3] and [4], by designing a flight trajectory of the UAV, energy received by the wireless device is maximized. In Reference [5], it is proposed that in the event of natural disasters or malicious attacks on the network, the UAV may be employed to quickly deploy the air communication base stations. The UAV may also participate in the formation of a mobile relay system. In Reference [6], by adjusting the transmission power and path planning, the throughput of the network is maximized. In Reference [7], the UAV may be employed to simultaneously transmit information and energy to the wireless devices. In the above references, either the UAV is not employed to transmit energy, or the UAV is only employed to transmit energy or information. In Reference [7], when the UAV is employed to transmit information and energy, both the energy and the information are incorporated into the same signal. After the wireless device receives the signal, a part of the signal is decoded, and a part of the signal is directly converted into energy. In the system considered, the UAV serves the wireless devices in the form of time division multiplexing, and selects to keep silence, transmit energy or information at each moment. Furthermore, an impact of a hovering height of the UAV on the system data rate may be finally considered. 
     REFERENCES 
     
         
         [1] Pratibha, Kwok Hung Li, and Kah Chan The, “Dynamic Cooperative Sensing—Access Policy for Energy-Harvesting Cognitive Radio Systems,” IEEE Transactions on Vehicular Technology, Volume: 65, Issue: 12, December 2016, pp. 10137-10141. 
         [2] Xiao Lu, Ping Wang, Dusit Niyato, and Ekram Hossain, “Dynamic Spectrum Access in Cognitive Radio Networks with RF Energy Harvesting,” IEEE Wireless Communications, Volume: 21, Issue: 3, June. 2014, pp. 102-110. 
         [3] Jie Xu, Yong Zeng and Rui Zhang, “UAV-enabled multiuser wireless power transfer: Trajectory design and energy optimization,” in Proc. IEEE APCC 
         [4] Jie Xu, Yong Zeng and Rui Zhang, “UAV-Enabled Wireless Power Transfer: Trajectory Design and Energy Region Characterization,” (available on-line at https://arxiv.org/abs/1706.07010). 
         [5] A. Merwaday and I. Guvenc, “UAV assisted heterogeneous networks for public safety communications,” in Proc. IEEE Wireless Commun. Netw. Conf., pp. 329334, 9-12 Mar. 2015. 
         [6] Y Zeng, R. Zhang, and T. J. Lim, “Throughput maximization for UAV-enabled mobile relaying systems,” IEEE Transactions on Communications, accepted (available on-line at arxiv/1604.02517). 
         [7] Xuanke He, Jo Bito and Manos M. Tentzeris, “A drone-based wireless power transfer anc communications platform,” in Proc. WPTC 
       
    
     SUMMARY 
     An objective of the present disclosure is to provide an optimization method for UAV-based wireless information and energy transmission to solve the above technical problems. 
     The present disclosure is implemented as below. There is provided an optimization method for UAV-based wireless information and energy transmission, and the optimization method includes following steps.
         S1: reporting, by a wireless device, an energy state B(t) of the wireless device to a UAV;   S2: detecting, by the UAV, a channel state γ(t) between the UAV and the wireless device; and   S3: selecting, by the UAV, an action space based on estimated revenue maximization according to an electric quantity of the UAV, an electric quantity of the wireless device, and the channel state.   A further technical solution of the present disclosure is as below. The action space includes a silence state, and a state for charging the wireless device or a state for transmitting information to the wireless device.   A further technical solution of the present disclosure is as below. The energy state of the wireless device is classified into a scarcity state, a medium state, and a sufficiency state, respectively corresponding to B(t)&lt;E d , E d ≤B(t)&lt;(1+T−t)E d , and B(t)≤(1+T−t)E d . When B(t)&lt;E d , the wireless device fails to decode, and the UAV does not transmit information to the wireless device. When B(t)≥(1+T−t)E d , the current electric quantity of the wireless device is enough to support decoding of all current and future time slots, and the UAV does not need to determine to charge the wireless device.   A further technical solution of the present disclosure is as below. The energy state of the UAV is classified into the scarcity state, the medium state, and the sufficiency state, respectively corresponding to P f ≤E r (t)&lt;2P f , 2P f ≤E r (t)&lt;(1+T−t)P f , and E r  (t)≥(1+T−t)P f . When P f ≤E r  (t)&lt;2P f , the UAV does not determine to charge the wireless device, otherwise the UAV can do nothing but only keep silence in subsequent time slots. When E r (t)≥(1+T−t)P f , the current electric quantity of the UAV is enough to support information transmission in all current and future time slots, and the UAV does not need to determine to keep silence.   A further technical solution of the present disclosure is as below. The UAV needs to determine the action space in different states. When there is more than one action in the action space, a value needs to be calculated for each action, an action with a maximum value is selected, and the value of the action is defined as Q t (S(t), a(t))≙R t (S(t), a(t))+F t (B(t+1), E r (t+1)). F t (B(t+1), E r (t+1)) represents an estimated future revenue after the time slot t. Q t  represents a total revenue of instantaneous revenue plus the estimated future revenue corresponding to the action a(t) in the state S(t) S(t) represents a system state of the time slot t, a(t) represents the action of the time slot t, and R t  represents the instantaneous revenue of the time slot t.   A further technical solution of the present disclosure is as below. When the electric quantity of the UAV is in different states, there are different calculation methods for the estimated future revenue, and the action of the time slot of the UAV is expressed as       

     
       
         
           
             
               a 
               ⁡ 
               ( 
               t 
               ) 
             
             = 
             
               
                 
                   arg 
                   ⁢ 
                      
                   max 
                 
                 
                   a 
                   t 
                 
               
               ⁢ 
               
                 
                   
                     Q 
                     t 
                   
                   ( 
                   
                     
                       S 
                       ⁡ 
                       ( 
                       t 
                       ) 
                     
                     , 
                     
                       a 
                       t 
                     
                   
                   ) 
                 
                 . 
               
             
           
         
       
         
         
           
             A further technical solution of the present disclosure is as below. When the UAV is in shortage of energy, 
           
         
       
    
                 F   t     (       B   ⁡   (     t   +   1     )     ,       E   r     (     t   +   1     )       )       =   △       {               V   n     ,       a   ⁡   (   t   )     =   0                 0   ,       a   ⁡   (   t   )     =   2             ,             
wherein V n  represents an expected revenue of a next time slot and is expressed as
 
                 V   n     =       log   2     (     1   +         P   f     ⁢   E   ⁢     {     γ   ⁡   (     t   +   1     )     }         P   0         )       ;         
wherein V n  represents the expected revenue of the next time slot, P f  represents a transmission power of the UAV, E represents a mathematical expectation symbol, γ represents a channel state, and P 0  represents a noise power.
         A further technical solution of the present disclosure is as below. When the UAV has a medium energy, it is estimated that the number of times the UAV will charge the wireless device in the future is       

                 n   c     =         arg   ⁢        min     n     ⁢       ❘   &#34;\[LeftBracketingBar]&#34;         ⌊         E   r     (     t   +   1     )       P   f       ⌋     -   n   -     ⌊         B   ⁡   (     t   +   1     )     +       nP   f     ⁢   E   ⁢     {     γ   ⁡   (   t   )     }           E   d       ⌋         ❘   &#34;\[RightBracketingBar]&#34;           ,         
it is estimated that the number of times the UAV will transmit information to the wireless device in the future is
 
                 n   m     =     min   ⁢     {         ⌊         E   r     (     t   +   1     )       P   f       ⌋     -     n   c       ,         ⌊         B   ⁡   (     t   +   1     )     +       n   c     ⁢     P   f     ⁢   E   ⁢     {     γ   ⁡   (   t   )     }           E   d       ⌋       }         ,         
and it is estimated that the future revenue is
 
     
       
         
           
             
               
                 F 
                 t 
               
               ( 
               
                 
                   B 
                   ⁡ 
                   ( 
                   
                     t 
                     + 
                     1 
                   
                   ) 
                 
                 , 
                 
                   
                     E 
                     r 
                   
                   ( 
                   
                     t 
                     + 
                     1 
                   
                   ) 
                 
               
               ) 
             
             
               = 
               △ 
             
             
               
                 n 
                 m 
               
               · 
               
                 
                   
                     log 
                     2 
                   
                   ( 
                   
                     1 
                     + 
                     
                       
                         
                           P 
                           f 
                         
                         ⁢ 
                         E 
                         ⁢ 
                         
                           { 
                           
                             γ 
                             ⁡ 
                             ( 
                             t 
                             ) 
                           
                           } 
                         
                       
                       
                         P 
                         0 
                       
                     
                   
                   ) 
                 
                 . 
               
             
           
         
       
         
         
           
             A further technical solution of the present disclosure is as below. When the UAV has sufficient energy, it is estimated that the number of times the UAV will charge the wireless device in the future is 
           
         
       
    
                 n   c     =         arg   ⁢       min     n     ⁢       ❘   &#34;\[LeftBracketingBar]&#34;         ⌈     T   -   t     ⌉     -   n   -     ⌊         B   ⁡   (     t   +   1     )     +       nP   f     ⁢   E   ⁢     {     γ   ⁡   (   t   )     }           E   d       ⌋         ❘   &#34;\[RightBracketingBar]&#34;           ,         
it is estimated that the number of times the UAV will transmit information to the wireless device is
 
                 n   m     =     min   ⁢     {         ⌈     T   -   t     ⌉     -     n   c       ,     ⌊         B   ⁡   (     t   +   1     )     +       n   c     ⁢     P   f     ⁢   E   ⁢     {     γ   ⁡   (   t   )     }           E   d       ⌋       }         ,         
and it is estimated that the future revenue is
 
     
       
         
           
             
               F 
               
                 t 
                   
               
             
             ( 
             
               
                 B 
                 ⁡ 
                 ( 
                 
                   t 
                   + 
                   1 
                 
                 ) 
               
               , 
               
                 
                   
                     E 
                     r 
                   
                   ( 
                   
                     t 
                     + 
                     1 
                   
                   ) 
                 
                 
                   = 
                   △ 
                 
                 
                   
                     n 
                     m 
                   
                   · 
                   
                     
                       
                         log 
                         2 
                       
                       ( 
                       
                         1 
                         + 
                         
                           
                             
                               P 
                               f 
                             
                             ⁢ 
                             E 
                             ⁢ 
                             
                               { 
                               
                                 γ 
                                 ⁡ 
                                 ( 
                                 t 
                                 ) 
                               
                               } 
                             
                           
                           
                             P 
                             0 
                           
                         
                       
                       ) 
                     
                     . 
                   
                 
               
             
           
         
       
     
     A further technical solution of the present disclosure is as below. A signal transmitted from the UAV to the wireless device is classified into a direct signal and an indirect signal according to different propagation paths. 
     Beneficial effects of the present disclosure are as below. The use of the wireless device can save wiring costs, beautify space, and ensure a smaller size and a lower power. The UAV is used in information and energy transmission for the wireless devices to improve the data transmission rate and the energy conversion efficiency of networks. This solution has a lower time complexity, but its effect is close to the God strategy with a high time complexity. Furthermore, the wireless device can be easily embedded into the UAV system, and higher data transmission rate and energy conversion efficiency can be achieved. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG.  1    is a flowchart of an optimization method for UAV-based wireless information and energy transmission according to an embodiment of the present disclosure; 
         FIG.  2    is a schematic diagram of a system model according to an embodiment of the present disclosure; 
         FIG.  3    is a schematic diagram of a forward algorithm for searching an optimal action according to an embodiment of the present disclosure; 
         FIG.  4    is a schematic diagram showing revenue comparison of three strategies under different T according to an embodiment of the present disclosure; and 
         FIG.  5    is a schematic diagram showing revenue of a two-element control strategy under different heights according to an embodiment of the present disclosure. 
     
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS 
     As shown in  FIG.  1   , it is illustrated a flowchart of an optimization method for UAV-based wireless information and energy transmission according to an embodiment of the present disclosure, which is described in detail as below. 
     Description of System Model 
     A UAV-based downlink wireless information and energy transmission system is considered. In this system, both the UAV and a wireless device are provided with batteries. When the UAV transmits energy to the wireless device, the wireless device stores the energy in its own battery. When the UAV transmits information to the wireless device, the wireless device uses the energy of the battery to receive a signal from UAV and decodes the signal. 
     As shown in  FIG.  2   , this model includes one UAV and a plurality of wireless devices. The UAV has limited energy, and thus in order to save energy, the UAV only adjusts its height but not moves horizontally during the whole working period. The UAV allocates a period of time and a part of energy for each wireless device to serve separately. Therefore, it is only necessary to study the process of the UAV serving a specific wireless device. The UAV serves T time slots for the wireless device. At the beginning of the t th  time slot of the entire time slots, the wireless device reports its energy state B(t) to the UAV, and the UAV may detect a channel state γ(t) between itself and the wireless device. Next, the UAV selects to keep silence, charge the wireless device or transmit information to the wireless device according to an electric quantity of the UAV, an electric quantity of the wireless device, and the channel state. A horizontal distance and a vertical distance from the wireless device to the UAV are represented by L and H, respectively. An action of the UAV is represented by: 
               a   ⁡   (   t   )     ∈     {       0   ⁢     (         keep           silence         )       ,     1   ⁢     (     transmit   ⁢         energy   ⁢         to   ⁢         the   ⁢         wireless   ⁢         device     )       ,   
     2   ⁢     (     transmit   ⁢         information   ⁢         to   ⁢         wireless   ⁢         device     )         }           
remaining service energy is represented by E r  (t), and a transmission power is represented by
 
                     P   ⁡   (   t   )     =     {             0   ,       a   ⁡   (   t   )     =   0                   P   f     ,       a   ⁡   (   t   )     =       1   ⁢         or   ⁢           a   ⁡   (   t   )       =   2               ,               (   1   )               
where P f  represents the operating power of the UAV. The energy required for each time decoding by the wireless device is represented by E d . The system state is expressed as S(t)≙(γ(t) B(t), E r (t)), which is a Markov decision process since the system state of the current time slot is only related to the system state of a previous time slot and the action of the UAV in the previous time slot.
 
     Channel Model
         A signal transmitted from the UAV to the wireless device may be classified into a direct signal and an indirect signal according to different propagation paths. The proportion of the direct signal depends on the height and the density of surrounding buildings, the height of the UAV, and the horizontal angle between the UAV and the wireless device, etc., which is expressed by Formula       

                       p   L     =     1     1   +     a   ⁢     exp   ⁡   (     -     b   ⁡   (     θ   -   a     )       )             ,           (   2   )               
where E r (t) and b represent parameters related to the environment. θ represents the horizontal angle between the UAV and the wireless device, and is calculated as
 
               θ   =         1   ⁢   8   ⁢   0     π     ⁢   arctan   ⁢     H   L         .         
The proportion of the indirect signals is p N =1−p L . In the t th  slot, fading of the direct signal and fading of the indirect signal are respectively as below:
 
γ L ( t )=| h   L ( t )| 2 (√{square root over ( L   2   +H   2 )}) −α     L     (3), and
 
γ N ( t )=| h   N ( t )| 2 (√{square root over ( L   2   +H   2 )}) −α     N     (4),
         where α L  and α N  represent a path fading coefficient of the direct signal and a path fading coefficient of the indirect signal, respectively. h L (t) and h N (t) respectively represent a multipath fading coefficient of the direct signal and a multipath fading coefficient of the indirect signal in the t th  time slot, and both obey Nakagami-m distribution. In this case, a probability density distribution function of the |h L (t)| 2  and a probability density distribution function of the |h N (t)| 2  are as below:       

                         f         ❘   &#34;\[LeftBracketingBar]&#34;       h   L       ❘   &#34;\[RightBracketingBar]&#34;       2       (   x   )     =           m   L     m   L       ⁢     X       m   L     -   1             Ω   L     m   L       ⁢     Γ   ⁡   (     m   L     )         ⁢     exp   ⁡   (     -         m   L     ⁢   X       Ω   L         )         ,   and           (   5   )                                 f         ❘   &#34;\[LeftBracketingBar]&#34;       h   N       ❘   &#34;\[RightBracketingBar]&#34;       2       (   x   )     =           m   L     m   N       ⁢     X       m   N     -   1             Ω   L     m   N       ⁢     Γ   ⁡   (     m   N     )         ⁢     exp   ⁡   (     -         m   N     ⁢   X       Ω   N         )         ,           (   6   )               
where m L  and m N  represent a Nakagami parameter of the direct signal and a parameter of the indirect signal, respectively. Ω t =E {|h L (t)| 2 } and Ω N =E{|h N (t)| 2 } represent a multipath fading power of the direct signal and a multipath fading power of the indirect signal, respectively. Γ(•) represents a Gamma function. The total signal fading is expressed as
 
γ( t )= p   L γ L ( t )+ p   N γ N ( t )  (7).
 
     State, Action and Revenue of an MDP Model
         The optimal design for wireless information and energy transmission of the UAV may be modeled as a restrictive Markov decision process within limited time.   The state space of this MDP is s={(γ(t),B(t),E r (t)):γ(t)∈[0,+∞,B(t)∈[0,B max ],E r (t)∈[0,E p ]}. The action space is:       

             A   =       {       0   ⁢     (         keep           silence         )       ,     1   ⁢     (           transmit   ⁢         energy   ⁢         to               the   ⁢         wireless   ⁢         device           )       ,   
     2   ⁢     (           transmit   ⁢         informaiton   ⁢         to               the   ⁢         wireless   ⁢         device           )         }     .           
The revenue is an information rate, which is expressed as
 
                       R   t     (       S   ⁡   (   t   )     ,     a   ⁡   (   t   )       )     =         log   2     (     1   +         P   ⁡   (   t   )     ⁢     γ   ⁡   (   t   )         P   0         )     ⁢     I   (         B   ⁡   (   t   )     ≥       E   d     ⁢     I   ⁡   (         E   r     (   t   )     ≥     P   f       )     ⁢     I   ⁡   (       a   ⁡   (   t   )     =   2     )         ,                 (   8   )               
where P 0  represents a noise power, and I(•) represents an indicator function.
 
     State Transition
         If the UAV does not have enough energy to transmit a signal, the UAV will keep silence. Therefore, when a strategy is designed, it is only needed to consider the situation that the UAV has enough energy to transmit the signal, i.e., E r (t)≥P f . A state transition function of the UAV and a state transition function of the wireless devices may be respectively expressed as       

                     B   ⁡   (     t   +   1     )     ⁢     {               B   ⁡   (   t   )     ,       a   ⁡   (   t   )     =   0                     B   ⁡   (   t   )     +         P   f     (   t   )     ⁢     γ   ⁡   (   t   )         ,       a   ⁡   (   t   )     =   1                     B   ⁡   (   t   )     -     E   d       ,       a   ⁡   (   t   )     =   2             ⁢         and               (   9   )                               E   r     (     t   +   1     )     =     {                 E   r     (   t   )     ,       a   ⁡   (   t   )     =   0                       E   r     (   t   )     -     P   f       ,       a   ⁡   (   t   )     =       1   ⁢       or   ⁢           a   ⁡   (   t   )       =   2               ,               (   10   )               
and the γ(t) is independently identically distribution in different t.
 
     An Objective Function and a Restriction
         The objective function is expressed as       

                       J   ⁡   (   π   )     =       max   π           ∑     t   =   1       T         R   t     (       S   ⁡   (   t   )     ,     a   ⁡   (   t   )       )           ,           (   11   )               
where π represents an action strategy function, the input is S(t) and the output is a(t). J(π) represents the total revenue under the strategy π. The UAV has limited energy, so the restriction of the model is
 
                           ∑     t   =   1       T       P   ⁡   (   t   )       ≤       E   r     (   1   )       ,           (   12   )               
where E r (1) represents the total energy available for the UAV to serve the wireless device.
 
     Action Selection Strategy
         Three strategies are provided: greedy strategy, two-element control strategy, and God strategy.       

     Greedy Strategy
         The first strategy is the simplest greedy strategy, and the action of the UAV in the t th  time slot is       

     
       
         
           
             
               
                 
                   
                     a 
                     ⁡ 
                     ( 
                     t 
                     ) 
                   
                   = 
                   
                     { 
                     
                       
                         
                           
                             
                               1 
                               , 
                               
                                 
                                   B 
                                   ⁡ 
                                   ( 
                                   t 
                                   ) 
                                 
                                 &lt; 
                                 
                                   E 
                                   d 
                                 
                               
                             
                           
                         
                         
                           
                             
                               2 
                               , 
                               
                                 
                                   B 
                                   ⁡ 
                                   ( 
                                   t 
                                   ) 
                                 
                                 ≥ 
                                 
                                   E 
                                   d 
                                 
                               
                             
                           
                         
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   13 
                   ) 
                 
               
             
           
         
       
     
     Two-Element Control Strategy
         Because the γ(t) and B(t) are continuous, the state S is also continuous. The Markov decision process in this continuous state is particularly difficult to be decoded. Thus, a sub-optimal solution is provided.   The energy state of the wireless device is classified into a scarcity state, a medium state, and a sufficiency state, respectively corresponding to B(t)&lt;E d , E d ≤B(t)&lt;(1+T−t)E d , and B(t)≥(1+T−t)E d . When B(t)&lt;E d , B(t)&lt;E d  the wireless device fails to decode, and the UAV does not transmit information to the wireless device. When B(t)≥(1+T−t)E d , the current electric quantity of the wireless device is enough to support decoding of all current and future time slots, and the UAV does not need to determine to charge the wireless device.   The energy state of the UAV may be likewise classified into the scarcity state, the medium state, and the sufficiency state, respectively corresponding to P f ≤E r (t)&lt;2P f , 2P f ≤E r (t)&lt;(1+T−t)P f , and E r (t)≥(1+T−t)P f . When P f ≤E r (t)&lt;2P f , the UAV does not determine to charge the wireless device, otherwise the UAV can do nothing but keep silence in subsequent time slots. When E r (t)≥(1+T−t)P f , the current electric quantity of the UAV is enough to support signal transmission of all current and future time slots, and the UAV does not need to determine to keep silence.       

     
       
         
           
               
             
               
                 TABLE 1 
               
             
            
               
                   
               
               
                 action space at current time 
               
            
           
           
               
               
            
               
                   
                 E r (t) 
               
            
           
           
               
               
               
               
            
               
                 B(t) 
                 the scarcity state 
                 the medium state 
                 the sufficiency state 
               
               
                   
               
            
           
           
               
               
               
               
               
               
               
            
               
                 the scarcity state 
                 a(t) ∈  
                 {0} 
                 a(t) ∈  
                 {0, 1} 
                 a(t) ∈  
                 {1} 
               
               
                 the medium state 
                 a(t) ∈  
                 {0, 2} 
                 a(t) ∈  
                 {0, 1, 2} 
                 a(t) ∈  
                 {1, 2} 
               
               
                 the sufficiency state 
                 a(t) ∈  
                 {0, 2} 
                 a(t) ∈  
                 {0, 2} 
                 a(t) ∈  
                 {2} 
               
               
                   
               
            
           
         
       
     
     Table 1 lists the action space that the UAV needs to determine in different states. When there is more than one action in the action space, it is needed to calculate a value for each action, and then the action with the greatest value is selected. In the t th  time slot, the value of the action is defined as
 
 Q   t ( S ( t ), a ( t ))≙ R   t ( S ( t ), a ( t ))+ F   t ( B ( t+ 1), E   r ( t+ 1))  (14),
 
where F t (B(t+1),E r (t+1)) represents the estimated future revenue after the time slot t. When the electric quantity of the UAV is in different states, there are different calculation methods provided for F t (B(t+1),E r (t+1)).
 
     When the UAV is in shortage of energy, F t (B(t+1),E r (t+1)) is expressed as 
                       F   t     (       B   ⁡   (     t   +   1     )     ,       E   r     (     t   +   1     )       )       =   △       {               V   n     ,       a   ⁡   (   t   )     =   0                 0   ,       a   ⁡   (   t   )     =   2             ,               (   15   )               
where V n  represents an expected revenue of a next time slot and is expressed as
 
     
       
         
           
             
               
                 
                   
                     V 
                     
                       n 
                         
                     
                   
                   = 
                   
                     
                       
                         log 
                         2 
                       
                       ( 
                       
                         1 
                         + 
                         
                           
                             
                               P 
                               f 
                             
                             ⁢ 
                             E 
                             ⁢ 
                             
                               { 
                               
                                 γ 
                                 ⁡ 
                                 ( 
                                 
                                   t 
                                   + 
                                   1 
                                 
                                 ) 
                               
                               } 
                             
                           
                           
                             P 
                             0 
                           
                         
                       
                       ) 
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   16 
                   ) 
                 
               
             
           
         
       
         
         
           
             When the UAV has a medium energy, it is estimated that the number of times the UAV will charge the wireless device in the future is 
           
         
       
    
     
       
         
           
             
               
                 
                   
                     n 
                     c 
                   
                   = 
                   
                     
                       
                         arg 
                         ⁢ 
                             
                         min 
                       
                       n 
                     
                     ⁢ 
                     
                       
                         
                           ❘ 
                           &#34;\[LeftBracketingBar]&#34; 
                         
                         
                           
                             ⌊ 
                             
                               
                                 
                                   E 
                                   r 
                                 
                                 ( 
                                 
                                   t 
                                   + 
                                   1 
                                 
                                 ) 
                               
                               
                                 P 
                                 f 
                               
                             
                             ⌋ 
                           
                           - 
                           n 
                           - 
                           
                             ⌊ 
                             
                               
                                 
                                   B 
                                   ⁡ 
                                   ( 
                                   
                                     t 
                                     + 
                                     1 
                                   
                                   ) 
                                 
                                 + 
                                 
                                   
                                     nP 
                                     f 
                                   
                                   ⁢ 
                                   E 
                                   ⁢ 
                                   
                                     { 
                                     
                                       γ 
                                       ⁡ 
                                       ( 
                                       t 
                                       ) 
                                     
                                     } 
                                   
                                 
                               
                               
                                 E 
                                 d 
                               
                             
                             ⌋ 
                           
                         
                         
                           ❘ 
                           &#34;\[RightBracketingBar]&#34; 
                         
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   17 
                   ) 
                 
               
             
           
         
       
         
         
           
             It is estimated that the number of times UAV will transmit information to the wireless device in the future is 
           
         
       
    
     
       
         
           
             
               
                 
                   
                     n 
                     m 
                   
                   = 
                   
                     min 
                     ⁢ 
                     
                       
                         { 
                         
                           
                             
                               ⌊ 
                               
                                 
                                   
                                     E 
                                     r 
                                   
                                   ( 
                                   
                                     t 
                                     + 
                                     1 
                                   
                                   ) 
                                 
                                 
                                   P 
                                   f 
                                 
                               
                               ⌋ 
                             
                             - 
                             
                               n 
                               c 
                             
                           
                           , 
                             
                           
                             ⌊ 
                             
                               
                                 
                                   B 
                                   ⁡ 
                                   ( 
                                   
                                     t 
                                     + 
                                     1 
                                   
                                   ) 
                                 
                                 + 
                                 
                                   
                                     n 
                                     c 
                                   
                                   ⁢ 
                                   
                                     P 
                                     f 
                                   
                                   ⁢ 
                                   E 
                                   ⁢ 
                                   
                                     { 
                                     
                                       γ 
                                       ⁡ 
                                       ( 
                                       t 
                                       ) 
                                     
                                     } 
                                   
                                 
                               
                               
                                 E 
                                 d 
                               
                             
                             ⌋ 
                           
                         
                         } 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   18 
                   ) 
                 
               
             
           
         
       
         
         
           
             It is estimated that the future revenue is 
           
         
       
    
     
       
         
           
             
               
                 
                   
                     
                       F 
                       t 
                     
                     ( 
                     
                       
                         B 
                         ⁡ 
                         ( 
                         
                           t 
                           + 
                           1 
                         
                         ) 
                       
                       , 
                         
                       
                         
                           E 
                           r 
                         
                         ( 
                         
                           t 
                           + 
                           1 
                         
                         ) 
                       
                     
                     ) 
                   
                   
                     = 
                     △ 
                   
                   
                     
                       n 
                       m 
                     
                     · 
                     
                       
                         
                           log 
                           2 
                         
                         ( 
                         
                           1 
                           + 
                           
                             
                               
                                 P 
                                 f 
                               
                               ⁢ 
                               E 
                               ⁢ 
                               
                                 { 
                                 
                                   γ 
                                   ⁡ 
                                   ( 
                                   t 
                                   ) 
                                 
                                 } 
                               
                             
                             
                               P 
                               0 
                             
                           
                         
                         ) 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   19 
                   ) 
                 
               
             
           
         
       
         
         
           
             When the UAV has sufficient energy, it is estimated that the number of times the UAV will charge the wireless device in the future is 
           
         
       
    
     
       
         
           
             
               
                 
                   
                     n 
                     c 
                   
                   = 
                   
                     
                       
                         arg 
                         ⁢ 
                             
                         min 
                       
                       n 
                     
                     ⁢ 
                     
                       
                         
                           ❘ 
                           &#34;\[LeftBracketingBar]&#34; 
                         
                         
                           
                             ⌈ 
                             
                               T 
                               - 
                               t 
                             
                             ⌉ 
                           
                           - 
                           n 
                           - 
                           
                             ⌊ 
                             
                               
                                 
                                   B 
                                   ⁡ 
                                   ( 
                                   
                                     t 
                                     + 
                                     1 
                                   
                                   ) 
                                 
                                 + 
                                 
                                   
                                     nP 
                                     f 
                                   
                                   ⁢ 
                                   E 
                                   ⁢ 
                                   
                                     { 
                                     
                                       γ 
                                       ⁡ 
                                       ( 
                                       t 
                                       ) 
                                     
                                     } 
                                   
                                 
                               
                               
                                 E 
                                 d 
                               
                             
                             ⌋ 
                           
                         
                         
                           ❘ 
                           &#34;\[RightBracketingBar]&#34; 
                         
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   20 
                   ) 
                 
               
             
           
         
       
         
         
           
             It is estimated that the number of times UAV will transmit information to wireless devices in the future is 
           
         
       
    
     
       
         
           
             
               
                 
                   
                     n 
                     m 
                   
                   = 
                   
                     min 
                     ⁢ 
                     
                       
                         { 
                         
                           
                             
                               ⌈ 
                               
                                 T 
                                 - 
                                 t 
                               
                               ⌉ 
                             
                             - 
                             
                               n 
                               c 
                             
                           
                           , 
                             
                           
                             ⌊ 
                             
                               
                                 
                                   B 
                                   ⁡ 
                                   ( 
                                   
                                     t 
                                     + 
                                     1 
                                   
                                   ) 
                                 
                                 + 
                                 
                                   
                                     n 
                                     c 
                                   
                                   ⁢ 
                                   
                                     P 
                                     f 
                                   
                                   ⁢ 
                                   E 
                                   ⁢ 
                                   
                                     { 
                                     
                                       γ 
                                       ⁡ 
                                       ( 
                                       t 
                                       ) 
                                     
                                     } 
                                   
                                 
                               
                               
                                 E 
                                 d 
                               
                             
                             ⌋ 
                           
                         
                         } 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   21 
                   ) 
                 
               
             
           
         
       
         
         
           
             It is estimated that the future revenue is 
           
         
       
    
                         F   t     (       B   ⁡   (     t   +   1     )     ,       E   r     (     t   +   1     )       )       =   △         n   m     ·       log   2     (     1   +         P   f     ⁢   E   ⁢     {     γ   ⁡   (   t   )     }         P   0         )         ,           (   22   )               
and finally the action of the t th  time slot may be expressed as
 
     
       
         
           
             
               
                 
                   
                     a 
                     ⁡ 
                     ( 
                     t 
                     ) 
                   
                   = 
                   
                     
                       
                         arg 
                         ⁢ 
                            
                         max 
                       
                       
                         a 
                         t 
                       
                     
                     ⁢ 
                     
                       
                         
                           Q 
                           t 
                         
                         ( 
                         
                           
                             S 
                             ⁡ 
                             ( 
                             t 
                             ) 
                           
                           , 
                           
                             a 
                             t 
                           
                         
                         ) 
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   23 
                   ) 
                 
               
             
           
         
       
     
     God Strategy 
     Because the state space is continuous, this Markov decision process is difficult to get an optimal solution in reverse. However, if all future channel states can be known in advance, the optimal solution can be obtained through forward search. This method requires the God&#39;s assistance and has a high time complexity, thus it is impossible to put this method into practical application. However, this method can be used as a benchmark for other strategies.
         As shown in  FIG.  3   , in the t th  time slot, the total revenue of all action combinations from the t th  time slot to the T th  time slot may be calculated, and then a(t) of a path with the maximum total revenue is selected, which may be expressed as Formula       

                     a   ⁡   (   t   )     =         arg   ⁢        max       a   t             max       a     t   +   1       ,   …         ,     a   T               ∑       t   ′     =   t       T           R     t   ′       (       S   ⁡   (     t   ′     )     ,     a     t   ′         )     .                 (   24   )               
The time complexity of this forward algorithm is 0(3 T ).
         Two simulation experiments are conducted: one is performance comparison of the three strategies, and the other is one-dimensional search for the optimal height of the UAV. In the first experiment, parameters are set as: L=200 m, P f =100 mW, P 0 =−100 dBm, Ω L =Ω N =12 mW, m L =3, m N =2, a=8.5, b=0.33, E d =4 μW·s, Δt=0.1 s, E total =40 mW·s, and B(1)=4 μW·S. The total number T of time slots is increased from 1 to 16, and for each T, 1000 rounds are conducted for each strategy and an average revenue is calculated. As shown in  FIG.  4   , revenues of different strategies are shown. It may be seen that the revenue of the greedy strategy and the revenue of the two-element control strategy have little difference when T is less than or equal to 4. This is because the energy of the UAV is always in the sufficiency state. As T gets closer and closer to 16, the performance of the two-element control strategy is getting better and better than that of the greedy strategy. Finally the performance of the two-element control strategy is increased by 26.05% than that of the greedy strategy, while the performance of the God strategy is only increased by 3.84% than that of the two-element control strategy.       

     In the second simulation experiment, the parameter is set as H=16, which is increased from 10 m to 200 m. As shown in  FIG.  5   , the relation between the revenue of the two-element control strategy and the height of the UAV is shown. As can be seen from  FIG.  5   , the best height is 89 m. 
     The foregoing descriptions are merely preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. Any modification, equivalent replacement and improvement made within the spirit and principle of the present disclosure shall fall into the protection scope of the present disclosure.