## 1 Introduction
[40] proposed an efficient and fair proximal policy optimization-based integrated scheduling method for multiple tasks using the Satech-01 satellite. [PERSON] et al. [41] formulated a dynamic and stochastic knapsack model to describe the online EOS scheduling problem and proposed a DRL-based insertion process. [PERSON] et al. [42] investigated a general Markov decision process (MDP) model and a Deep Q-Network for AEOS scheduling, and this solution method has also been employed for satellite range scheduling [43]. [PERSON] et al. [44] regarded neural networks as feasible rule-based heuristics in their proposed end-to-end DRL framework. [PERSON] et al. [45] proposed a two-phase neural combinatorial optimization approach to address the EOS scheduling problem. [PERSON] et al. [46] proposed a DRL-based approach that could provide solutions in nearly real time. [PERSON] et al. [10] addressed the AEOS task scheduling problem within one orbit using the deep deterministic policy gradient (DDPG) method. [PERSON] et al. [47] focused on decision-making in the task sequence and proposed a dynamic destroy deep-reinforcement learning (D3 RL) model with two application modes. [PERSON] et al. [48] proposed an attention decision network to optimize the task sequence in the AEOS scheduling problem. [PERSON] et al. [49] presented a DRL algorithm with a local attention mechanism to investigate the single agile optical satellite scheduling problem. [PERSON] et al. [50] introduced a DRL and parameter transfer based approach to solve a multi-objective agile earth observation satellite scheduling problem. [PERSON] et al. [31] investigated a graph attention network-based decision neural network for the agile Earth observation satellite scheduling problem. Compared with single-EOS scheduling, there is little research on DRL for MAEOS scheduling. [PERSON] et al. [51] proposed a DRL-based multi-satellite mission planning algorithm for high and low-orbit AEOSs, which kept the revenue rate difference compared with A-ALNS [30] below 5%. [PERSON] et al. [52] focused on the autonomous mission planning problem for AEOSs, in which the visible time window (VTW) was set as a time interval. [PERSON] et al. [53] tackled the issue of multi-satellite rapid mission re-planning using a deep reinforcement learning method that incorporated mission sequence modeling. [PERSON] et al. [54] addressed a multi-objective learning evolutionary algorithm for solving the multi-objective multi-satellite imaging mission planning problem. [PERSON] et al. [55] developed an improved DRL method to address the MAEOSSP with variable imaging durations. [PERSON] et al. [56] introduced a genetic algorithm (GA) framework incorporating DRL for generating both initial solutions and neighborhood search solutions to address the multitype satellite observation scheduling problem. These studies focus on scenarios in which each task has only one VTW in each satellite. In this study, we propose a DRL-based two-phase hybrid optimization (TPHO) method to generate an observation plan for the MAEOSSP. For each AEOS, tasks may have different numbers of VTWs. We propose an encoder-decoder network to optimize the task sequence and a heuristic method to allocate satellites for tasks. We model the MAEOS scheduling process using with a finite MDP with a continuous state space. The major contributions of this study are summarized as follows:  
1. An MDP for the MAEOSSP is constructed, in which the action space is decomposed into task action and resource action subspaces corresponding to the task sequence and resource allocation subproblems, and a new reward function is proposed. 2. A DRL-based TPHO framework is proposed for the MAEOSSP. An encoder-decoder network in TPHO is designed to determine the task sequence, and a heuristic method is proposed to optimize the resource allocation problem. 3. A comprehensive experiment is conducted to examine the performance of our method. Based on the computational results, the proposed method has proven to be effective and has high time efficiency.