# Cooperative Downloading for LEO Satellite Networks: A DRL-Based Approach
## 1 Introduction
Recently, inter-satellite communication-link (ISL)-utilizing cooperative downloading approaches have been also investigated to maximize throughput and solve the resource-under-utilization issue [22; 23; 24]. They leveraged data offloading through ISLs to distribute an appropriate amount of data to each satellite. [PERSON] et al. [22] constructed a graph reflecting the states of satellite-to-ground communication links (SGLs) and ISLs, and then proposed an iterative algorithm adjusting the download time and the offloading data volume of each satellite. [PERSON] and [PERSON] [23] proposed an iterative algorithm considering the energy efficiency of satellites as well as data throughput. [PERSON] et al. [24] constructed a task flow graph and formulated an ILP problem to maximize the amount of offloaded tasks, which was solved by a genetic algorithm. Even though the use of ISLs is promising for data download services, the previous works assumed relatively stationary environments in that all states are known in advance and unchanged while applying iterative algorithms. Meanwhile, in modern networks that are increasingly complex and expanding horizontally/vertically, deep-reinforcement-learning (DRL)-based optimization, which can guarantee high performance with low complexity, has received great attention [25]. In particular, its utility has been proven in dynamic networks such as vehicular networks [26] and aerial networks [27]. Along with these successful applications, DRL has also been considered as a promising approach for satellite networks [28; 29; 30; 31]. [PERSON] et al. [28] proposed a DRL-based handover scheme for highly dynamic LEO satellite networks. [PERSON] et al. [29] proposed a resource management scheme based on DRL to guarantee QoS for large-scale satellite-supported remote Internet-of-things (IoT) networks. [PERSON] et al. [30] proposed a DRL-based power allocation scheme for an RSMA-applied 6G LEO system. [PERSON] et al. [31] investigated a federated learning (FL)-combined DRL system for UAV/LEO satellites to efficiently provide communication resources for ground nodes. However, there has not been a DRL-based cooperative downloading scheme for satellite networks. In this paper, we try to maximize the amount of the downloaded data of satellite networks where multiple LEO satellites and their ISLs are considered. To this end, we formulate the cooperative downloading problem as a Markov decision problem (MDP) with the objective to maximize the amount of downloaded data to GS. To solve the formulated problem, we adopt a soft-actor-critic (SAC)-based DRL algorithm for discretized action space to learn the dynamics of satellite networks and train the optimal policy. For SAC-based training, we design a novel neural network consisting of (1) a graph attention network (GAT) at the input layer to aggregate graph-oriented network states and (2) parallel fully connected (FC) layers at the output layer to control the individual behaviors of satellites. Evaluation results show that the proposed scheme can enhance the average contact time utilization by up to 17.8% compared with independent downloading and randomly offloading schemes, even when initial data distributions are highly biased. The contributions of this paper can be summarized as follows: (1) owing to the high dynamics (e.g., mobility, transmission rate, and trajectory) of LEO satellites, it is quite complex to find the optimal policy for cooperative downloading using conventional approaches. Thus, we introduce the use of DRL to solve the problem, which is a promising approach since high-performance servers are planned to be put into orbits [32].