# Deep Reinforcement Learning for the Agile Earth Observation Satellite Scheduling Problem  
[PERSON]  
1 College of Systems Engineering, National University of Defense Technology, Changsha 410073, China; [EMAIL_ADDRESS] (J.C.); [EMAIL_ADDRESS] (L.H. )1 College of Advanced Interdisciplinary Studies, National University of Defense Technology, Changsha 410073, China; [EMAIL_ADDRESS]  
[PERSON]  
22334  
[PERSON]  
1 College of Systems Engineering, National University of Defense Technology, Changsha 410073, China; [EMAIL_ADDRESS] (J.C.); [EMAIL_ADDRESS] (L.H. )1 College of Advanced Interdisciplinary Studies, National University of Defense Technology, Changsha 410073, China; [EMAIL_ADDRESS]  
[PERSON]  
4 School of Traffic and Transportation Engineering, Central South University, Changsha 410075, China  
constraints of satellites, such as memory and power consumption. Therefore, the AEOSSP is a typical combinatorial optimization problem with complex constraints that has been shown to be an NP-hard problem [1]. With the expansion of AEOS application fields, observation requests become frequent, and observation requirements become diverse. However, with better observation capabilities, AEOSs are still a scarce resource that cannot satisfy the high demand for observations. In addition, some emergencies, such as earthquakes and floods, require satellites to complete observations as soon as possible. Therefore, a fast and efficient scheduling algorithm is essential to improve the utilization rate of satellites. In recent decades, many scholars have studied the AEOSSP. [PERSON] et al. [2] were the first to research the AEOSSP, and described the AEOSSP as a combinatorial optimization problem considering the selection and scheduling of observation tasks. Due to the complexity of the problem, few exact methods have been proposed. [PERSON] et al. [3] proposed a mixed-integer programming model for the AEOSSP and reduced problem complexity by discretizing the continuous observation angle into three angles. They obtained the approximate upper bound of the problem by CPLEX. [PERSON] et al. [4] designed an implicit enumeration algorithm to construct a solution under the framework of depth-first search and designed three pruning strategies. For the AEOSSP, the exact methods all simplify the time-dependent constraint. In addition, ref [5] showed that solutions cannot be obtained in an acceptable time with the CPLEX solver when the number of VTWs exceeds 27. Currently, research on combinatorial optimization problems primarily focuses on heuristics and meta-heuristics. Heuristic and meta-heuristic algorithms can solve large-scale problems and are widely used in practice. A well-designed algorithm can significantly improve efficiency [6; 7; 8]. For AEOSSP, [PERSON] et al. [2] proposed four heuristics: a greedy algorithm, dynamic programming, a constraint planning algorithm, and a local search algorithm. There are also several profit-based construction heuristics [3; 9] and the iterative local search [10]. Many meta-heuristics include the tabu search algorithm [11; 12], the hybrid differential evolution algorithm [13], the improved genetic algorithm [14; 15; 16; 17], and the adaptive large neighbourhood search algorithm [5; 18]. However, the search difficulty and solution time of these algorithms increase dramatically as the problem scale increases. Traditional heuristic and meta-heuristic algorithms cannot meet the requirements of high efficiency and fast response in practical applications. In addition, these rule-based algorithms rely heavily on the designer's experience, and the solution's quality is poor. Therefore, traditional methods are limited by their solution characteristics and cannot produce high-quality results timely. In recent years, deep reinforcement learning (DRL) has been applied to many classical combinatorial optimization problems, such as the travelling salesman problem (TSP) and the vehicle routing problem (VRP). [PERSON] et al.