## 1 Introduction
DRL has gradually gained attention in the optimization field [28; 29; 30], and compared with metaheuristics, DRL shows a high solving speed [31]. [PERSON] et al. [32] proposed a long short-term memory-deep deterministic policy gradient (LSTM-DDQN) algorithm to optimize the transmit beamforming and the reconfigurable intelligent surfaces phase-shift matrix. [PERSON] et al. [33] proposed a deep reinforcement learning-based adaptive modulation for orthogonal frequency division multiplexing underwater acoustic communication systems. [PERSON] et al. [34] integrated the DRL networks with a graph to address the traveling salesman problem (TSP). [PERSON] et al. [35] proposed a dynamic attention model embedded in a dynamic encoder-decoder architecture for the vehicle routing problem (VRP). Moreover, researchers have applied DRL to EOS scheduling problems. [PERSON] et al. [36] introduced a case-based learning method to improve the performance of the genetic algorithm. [PERSON] et al. [37] proposed a novel fair integrated scheduling method with proximal policy optimization to solve the task-scheduling problem of the SATech-01 satellite. [PERSON] et al. [38] modeled the EOS scheduling problem as a dynamic and stochastic knapsack problem and employed DRL to determine task insertion. [PERSON] et al. [39] investigated a general solution based on a Deep Q-Network for single EOS scheduling, and the solution framework was also applied to satellite range scheduling [40]. [PERSON] et al. [41] proposed an end-to-end DRL framework in which neural networks are regarded as complex heuristic methods constructed by feasible rules and reward signals. To tackle the EOS scheduling problem with image quality criteria and transition time constraints, [PERSON] et al. [42] proposed a two-phase neural combinatorial optimization approach. [PERSON] et al. [43] proposed a DRL method that can generate decisions in near real-time. [PERSON] et al. [44] employed the deep deterministic policy gradient (DDPG) algorithm to address the time-continuous satellite task-scheduling problem within one orbit. [PERSON] et al. [45] focused on the decision-making of the observation order and simplified the observation time problem. They proposed a dynamic destroy deep-reinforcement learning (D3 RL) model and two applications of the D3 RL model. DRL is composed of three components, including an environment constructed corresponding to a certain optimization problem, deep networks, and a training method. Table 1 presents a comparison of this study and previous studies of the DRL method for EOS scheduling problems. It can be observed from Table 1 that most of the work based on the DRL method focuses on scenarios in which the satellite orbits the Earth once, so that each task has one visible time window (VTW). The start time and end time of VTWs are a part of the inputs in these DRL methods. The sequence length of inputs for each task needs to be the same, and this requirement is naturally met in one-VTW scenarios. However, in a scenario in which the satellite orbits the Earth more than once, tasks may have different numbers of VTWs, which leads to various sequence lengths of inputs. [PERSON] et al. [39] also address a multi-VTW scenario, but its solution quality could be further improved. In this study, we propose a DRL-based attention decision network (ADN) model to generate an observation plan for the AEOSSP. The model is trained using the soft actor-critic (SAC) algorithm. The process of AEOS scheduling is modeled as a finite Markov decision process with continuous state space in which the DRL environment for the AEOSSP with multi-VTW is constructed. To improve the solution quality of the trained ADN, we also propose a start-time-shift-based local search that conducts a quick search on time windows. Experiments show that our method produces better results than other advanced methods. The major contributions of this study are summarized as follows:  
* An ADN model is proposed for the AEOSSP.