## 1 Introduction  
Earth observation satellites (EOSs) provide large-scale observation coverage and are equipped with optical instruments to take photographs of specific areas at the request of users [1, 2, 3]. EOSs have been extensively employed in scientific research and application, mainly in environmental and disaster surveillance, ocean monitoring, agricultural harvesting, etc. [4]. Recently, agile EOSs (AEOSs) have drawn significant attention as a new generation of these platforms [5]. Unlike conventional EOSs, AEOSs possess three-axis (roll, pitch, and yaw) maneuverability, which amplifies flexibility and expands the solution space of their task scheduling. Cooperatively multiple AEOSs can serve various demands, such as multi-angular observation and wide area observation [6]. Consequently, multi-AEOS (MAEOS) scheduling has become a primary focus of satellite platforms. However, although the number of AEOSs in orbit is increasing, they are still limited because of the vast number of applications they can be used for [7, 8]. Therefore, an effective scheduling algorithm must be developed to improve the efficiency of observation systems. The MAEOS scheduling problem (MAEOSSP) is a large-scale combinatorial optimization problem and has been proven to be NP-hard [9]. Algorithms applied to MAEOSSP are usually classified as exact methods, heuristic methods, metaheuristic methods and machine learning methods [10]. Exact methods, such as mixed-integer linear programming [9], dynamic programming [11], and branch and price [12], provide optimal solutions and are limited to the scale and complexity of corresponding problems. Simple heuristic rules are extensively used in most optimization problems because they are efficient and intuitive [13; 14; 15]. Owing to their low computational complexity, heuristic methods have been widely applied in practical satellite scheduling, such as Pleiades in France [16], FireBIRD in Germany [17], and EO-1 in the USA [18]. Heuristic methods can speed up the process by finding a satisfactory solution, but studies and engineering projects have also indicated that heuristic results can be improved further. Metaheuristic methods typically provide satisfactory observation plans with a longer run time. Various metaheuristic algorithms have been proposed, including genetic algorithms and their variants [19; 20; 21; 22], tabu search [23; 24; 25], simulated annealing [26], ant colonies and their variants [27], differential evolution algorithms [28], and large neighborhood search [29; 30]. However, the search difficulty and solution times of these algorithms increase dramatically as the scale of the problem increases. Traditional exact and metaheuristic algorithms cannot meet the requirements of high efficiency and fast response in practical applications [31]. A major breakthrough in deep reinforcement learning (DRL) was achieved by [PERSON] et al. [32], where their proposed Deep Q-Network (DQN) demonstrated for the first time that DRL could attain human-level performance in complex decision-making tasks. In recent years, DRL has made remarkable achievements in games and has been applied to combinatorial optimization problems [33; 34; 35; 36; 37]. Combinatorial optimization allows one to optimally select variables in a discrete decision space, which is similar to the action selection of DRL. Moreover, with its \"offline training and online decision making\" characteristics, DRL demonstrates the potential for quick responses to requirements and shows a high solving speed compared with metaheuristics [38]. Therefore, DRL has become a promising method for solving satellite scheduling problems. Research efforts on DRL have been dedicated to single-AEOS scheduling [39; 40; 41; 42], but there is little research on DRL for MAEOS scheduling. Research efforts have been dedicated to addressing single-satellite Earth observation scheduling problems through DRL. [PERSON] et al. [39] integrated case-based learning and a genetic algorithm to schedule EOS. [PERSON] et al.