## 1 Introduction  
Rapid advances have been made in the field of Earth observation technology. Owing to several advantages, such as large-scale observation coverage and being unrestrained by national boundaries, Earth observation satellites (EOSs) have been widely employed in disaster and environmental monitoring [1], agriculture [2], geographic information [3], and other areas [4]. As a new generation of EOSs, agile EOSs (AEOSs) have drawn attention due to their three-axis maneuverability, which enhances the flexibility in performing remote sensing tasks and leads to a larger solution space for AEOS observation task scheduling [5]. Moreover, the demand from users is rising sharply, and the constraints of satellite scheduling are becoming more complex, presenting challenges for AEOS scheduling [6]. Consequently, an effective scheduling algorithm is pivotal to fully utilize the potent observation capabilities of AEOSs. The window-constrained packing problem model was proposed by [PERSON] and [PERSON] in 2000 [7], which marked the beginning of the use of optimization modes to describe satellite scheduling problems. Then, [PERSON] et al. conducted the first comprehensive study on the AEOS scheduling problem (AEOSSP) in 2002 [8]. The AEOSSP has been proven to be NP-hard and has drawn great interest over the past two decades [5]. General solving algorithms applied to AEOSSFs can be divided into exact algorithms, heuristic algorithms, metaheuristic algorithms, and deep reinforcement learning (DRL). For the AEOSSP, exact methods, such as branch and bound [9, 10] and dynamic programming [8], have deterministic solving steps and can only provide satisfactory solutions for small-scale scenarios due to the high time complexity. Due to the explosion of the solution space, researchers have explored heuristic and metaheuristic algorithms with lower computational complexity. Heuristic methods are fast and intuitive and have been applied in practical satellite scheduling, e.g., for Korean Multi-Purpose Satellite 2 (KOMPSAT-2) in Korea [11]. [PERSON] et al. [12] proposed decision rules and an iterated stochastic greedy search for the online decision-making problem. [PERSON] et al. [13] proposed a heuristic approach based on the density of residual tasks for scheduling tasks in a single AEOS. However, the efficiency of a heuristic depends on the specific characteristics of the problem. Without relying on a specific heuristic policy, metaheuristic methods usually provide satisfactory observation plans with more time consumption than heuristic methods. Various metaheuristic algorithms are considered for scheduling AEOSs, including genetic algorithms (GA) [14; 15; 16], tabu search (TS) [17; 18; 19], ant colony optimization (ACO) [20], metaheuristics based on a divide-and-conquer framework [21], differential evolution algorithms [22], and algorithms based on the framework of a large neighborhood search (LNS) [23; 24]. In general, the solution quality of metaheuristics can be improved by designing additional structures. In the LNS framework, \"removal\" and \"insertion\" operators are defined to destroy and repair the solution in each iteration. [PERSON] et al. [25] developed an adaptive large neighborhood search (ALNS), in which the LNS framework was improved by an adaptive selection of operators. Their experimental results showed that ALNS outperformed other algorithms in AEOSSP. However, algorithms with more complex structures tend to require more runtime. In some complex scenarios, metaheuristic methods usually struggle to balance efficiency and solution quality. With the development of artificial intelligence, DRL has demonstrated a strong ability in sequential optimization and shown the potential for a quick response to requirements with the characteristics of \"offline training and online decision-making\". DRL is a combination of deep learning and reinforcement learning. Deep learning has been widely studied in various domains, including remote sensing [26] and visual location [27].