###### Abstract  
Recently, using unmanned aerial vehicles (UAVs) to collect information from distributed sensors has become one of the hotspots in the Internet of Things (IoT) research. However, previous studies on the UAV-assisted data acquisition systems focused mainly on shortening the acquisition time, reducing the energy consumption, and increasing the amount of collected data, but it lacked the optimization of data freshness. Moreover, we hope that UAVs can perform long-term data collection tasks in dynamic scenarios within a constantly changing age of information (AoI) and within their own power levels. Therefore, we aim to maximize the quality of service (QoS) based on the freshness of data, while considering the endurance of the UAVs. Since our scenario is not an inertial order decision process with uniform time slots, we first transform the optimization problem into a semi-Markov decision process (SMDP) through modeling, and then we propose a hierarchical deep Q-network (DQN)-based path-planning algorithm to learn the optimal strategy. The simulation results show that the algorithm is better than the benchmark algorithm, and the tradeoff between the system QoS and the safe power state can be achieved by adjusting the parameter \(\beta_{e}\). U 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 2022 20222 2022 2022 2022 20222 2022 20  
attention in recent years. A study by [PERSON] et al. [10] demonstrated how to maximize the acquisition efficiency of the unmanned aircraft system (UAS), modeling the problem as a potential game between UAS and sensor nodes, and proposing a data acquisition method that maximizes energy efficiency under fairness constraints. [PERSON] et al. [11] considered jointly optimizing the wake-up time of the sensor nodes and the trajectory of UAVs. While ensuring the required amount of data is collected, the energy consumption is minimized, and the suboptimal solution is obtained by using the iterative convex optimization technique. [PERSON] et al. [12] considered minimizing the completion time of UAV acquisition tasks, which builds on their previous research on joint optimization. [PERSON] et al. [13] considered the problem of UAV-assisted data acquisition in delay-sensitive application scenarios, and they maximized the available quantity of devices serving the IoT by optimizing the trajectory and resource allocation of UAVs. Most of the existing works of UAV-assisted data acquisition systems focus mainly on shortening the acquisition time, reducing energy consumption, and increasing the amount of collected data, which is aimed at either maximizing system throughput or minimizing delay via designing the trajectory, deployment position, or resource allocation of UAVs [14; 15; 16; 17; 18]. However, for real-time IoT applications, such as air quality monitoring, precision agriculture, and post-disaster early warning, the quality of service (QoS) highly depends on the freshness of information collected by UAVs from the ground nodes. In this paper, we introduce the age of information (AoI) to indicate the freshness of collected data. Different from the delay metric, the AoI is defined as the time elapsed since the latest received status update packet at a destination node [19]. The continuous data collection task is also a research hotspot. Due to its learning ability and generalization ability, deep reinforcement learning (DRL) is very suitable for such dynamic scenarios in which the AoI is constantly changing [20; 21]. In [15; 22], the authors formulated the fresh data collection problem in UAV-assisted IoT networks as a Markov decision process (MDP) and used a DRL-based approach to minimize the AoI. [PERSON] et al. [23] considered the unknown and time-varying traffic generation pattern of ground nodes, and proposed a novel online AoI-based trajectory planning approach using a deep deterministic policy gradient (DDPG). However, the energy of the UAV is limited, and each UAV needs to recharge in continuous data collection tasks.