## 3 AoI Minimization in UAV-Assisted WSN/IoT Application Architectures
### Trajectory and Energy Management (TE)
#### 4.2.5 TE and Scheduling
The authors did not explore the potential of full duplex communication [95], and the limited onboard energy of UAVs, which affects AoI, was not considered as a constraint during optimization [98]. Similarly, the unreliable channel conditions and the absence of knowledge of CSI were not considered [8]. Sensor nodes were also assumed to have equal importance, which might not reflect the reality of sensor heterogeneity in applications with sensor-specific AoI costs [87]. Similarly battery recharging for sensor nodes [85] and UAVs were not captured within the studied framework and one source-destination pair was investigated [2]. The energy consumption of UAVs was not considered in the age-optimal trajectory design [91]. Finally, the analysis did not consider the optimal scheduling of update packet transmission from different nodes while optimizing the UAV trajectory [81]. Prior to [80], many studies did not consider the energy consumption of the UAV in age-optimal trajectory optimization. This motivates the authors to consider UAV energy constraints by studying age-optimal data collection in UAV-assisted IoT using deep RL. A UAV leaves a depot, flies towards the sensor nodes to collect status update packets, and reaches the destination within a given time frame. While the weighted sum of the AoI of sensors is minimized during UAV flight, the UAV must maintain non-negative residual energy. The problem is formulated into a finite-horizon MDP to obtain optimal flight trajectory and sensor node transmission scheduling. Solving the MDP using dynamic programming is computationally prohibitive due to the high-dimensional state space. Thus, the authors propose a deep-RL-based UAV-assisted data collection algorithm where the UAV decides on which direction to fly and which sensor node it should connect to at each step. Extensive simulations show that the algorithm can reduce the weighted sum AoI compared to other policies. However, the authors did not consider the potential of backscatter communications for improving the energy management in the network [106]. They did not also consider the recharging of the UAVs [90]. In [7], a UAV collects data from a group of clusters by interacting with the cluster heads alone. In this scenario, the authors jointly optimize UAV hovering points and trajectory for minimal AoI data collection. The optimization problem is formulated as a traveling salesman problem with neighborhoods, which is quite challenging due to the continuous (optimization of hovering points) and combinatorial (optimization of visiting order) aspects. To reduce the computational complexity involved in the joint optimization of UAV hovering points and trajectory for minimal AoI data collection, the authors transform the continuous optimization traveling salesman problem with neighborhoods into the Generalized Traveling Salesman Problem (GTSP) using a sampling-based concept. The transformed GTSP is combinatorial in nature and can be solved using exact algorithms, approximate algorithms, or heuristics. However, they cannot balance optimality and complexity effectively. Thus, the authors use ML-based algorithms to solve the GTSP (i.e., UAV trajectory design problem) to achieve this balance. However, this work does not study the use of multiple UAVs for data gathering; it is considered for future work. Although using multiple UAVs adds some complexity to the model, it can potentially reduce the AoI further. Note that a generate-at-will policy was used and thus other policies can be considered in the future. [PERSON] et al. [92] deployed deep RL for trajectory optimization in an energy-harvesting UAV-aided data collection in WSN setup as illustrated in Figure 8b. The main motivation was to extend the operation time of the UAV. The problem is formulated using MDP, and deep-RL-based methods were deployed to optimize the UAV flight trajectory and sensor node transmission opportunities to minimize AoI and energy consumption. EH was adopted to acquire renewable energy from the environment to ensure the UAV enjoys a longer operation time.