# Federated Reinforcement Learning Based AANs with LEO Satellites and UAVs
## 2 Preliminaries and Related Works
### Aerial Access Network  
In general, Internet services are provided through core networks on the ground. In order to provide Internet services to a certain area, it is necessary to install a core network infrastructure in the area. Therefore, in rural areas where it is hard to install wired network infrastructures, it is difficult to provide the Internet service to the users in such area. In this case, the wireless backhaul could be applied instead of the wired infrastructure. The Internet service providers, including LigoWave [7], Proxim [8], and FiberLight [9], utilize wireless backhaul to provide Internet services to the customers. Furthermore, to expand the coverage and fill coverage hole, some researchers utilize UAVs as relay of wireless backhaul. [PERSON] et al. proposed a UAV system which can communicate with base stationsto provide the Internet service to the users, where Internet service is unavailable with the wired infrastructure [10; 11]. Moreover, the satellite Internet is also used in areas, such as mountains, seas, and the sky, where Internet services are not provided via ground facilities. The early stage of satellite network has quite high latency and low capacity as the network is mainly based on geostationary Earth orbit (GEO) satellites. However, compared to the past, the construction cost of satellite network was significantly reduced due to the technological progress, including the launch vehicle reuse. Thus, the satellite communication network with LEO satellites, which have quite lower orbit altitude than GEO satellites, has been proposed and deployed [12]. The LEO satellite network can provide low-latency network services due to short distance between the satellites and ground stations. Currently, Starlink [1], OneWeb [2], and Kuiper [3] are providing or preparing Internet services using LEO satellites. In the case of satellites, it costs a lot of money to launch and manage, and there are many restrictions due to the nature of orbiting the Earth. To overcome these limitations, Facebook and Boeing conducted a research on the method to use high-altitude UAVs, also known as high altitude pseudo-satellites [13; 14]. As wireless communication is essential to build and maintain the satellite network, some researches focused on improving utilization of wireless communication. [PERSON] et al. proposed a broadband satellite network based on software-defined networking and network virtualization to achieve efficient cooperation among various resources [15]. [PERSON] et al. proposed a channel selection optimization method based on joint cooperative spectrum sensing with cognitive radio [16]. [PERSON] et al. studied about satellite cognitive communications with spectrum sharing [17]. Furthermore, there were studies on the multi-layer satellite networks which utilize satellites with different orbitals, such as GEO, medium Earth orbit (MEO), and LEO. [PERSON] et al. proposed a multi-layered satellite routing algorithm which effectively utilizes the delay measurements [18]. In addition, many researchers tried to integrate terrestrial networks and satellite networks [19]. A research on satellite Internet of things (IoT) was also conducted to obtain the connectivity of IoT devices using satellite networks. Unlike the above researches, [PERSON] et al. suggested that the bottleneck of satellite networks is due to the limited capacity of communication between satellites and ground stations [6]. Furthermore, [PERSON] emphasized that, due to the nature of satellite, there are bound to be problems related to the congestion in user-dense areas [5]. In summary, the existing researches about the satellite-based AAN mainly focused on wireless communication, such as cognitive radio, spectrum sensing, channel selection, and so on. Furthermore, the previous studies on AAN with UAVs did not actively consider the satellite network as their backbone.