# Federated Reinforcement Learning Based AANs with LEO Satellites and UAVs
## 4 Performance Evaluation
### Evaluate System with Network Simulator
#### 4.2.3 Simulation Result  
In our proposed AAN, the UAVs are deployed to provide network resource to the areas with high network traffic demand. The overall transmitted data size is related to the Internet service capacity of AAN. Therefore, to validate our system with UAVs, we measured the total amount of data transmitted by customer nodes throughout the simulation. We compared the transmitted data size of customer nodes with two types of AANs, satellite-only AAN and UAV-aided AAN. As a result, our proposed UAV-aided AAN provides approximately 3.25 times more communication resources than satellite-only AAN, and Figure 10 represents throughput improvement rate of each area. In the figure, the intensity of color represents the improvement rate of each area, and blue circles represent service ranges of deployed UAVs. As shown in the simulation result, using the proposed system, it is possible to deliver more data from densely populated area. Moreover, the traffic demand of densely populated area can be offloaded to UAVs, so the satellites can provide more resources to rural areas, which increases the throughput of rural areas.  
In addition to the throughput improvement comparison, we compared the network path between the simulations with satellite-only AAN and UAV-aided AAN.  
The network path of satellite-only AAN is represented at Figure 11, and Figure 12 shows the path of UAV-aided AAN. Throughout the simulation, the path of satellite-only AAN changed four times, at 17.0, 85.8, 140.3, and 191.9 s. The initial network path of satellite-only AAN is represented at Figure 10(a), and changed network paths are represented at Figures 10(b),(c) and 10(d),(e). By contrast, the path of UAV-aided AAN changed three times, at 28.9, 114.2, and 140.5 s, which is less than satellite-only AAN. The initial network path of UAV-aided AAN is represented at Figure 11(a), and changed network paths are represented at Figure 11(b)-(d). As we can see the difference between the two cases, the UAVs could  
Figure 10: Simulation environment map with throughput improvement rate.  
Figure 9: Simulation environment map with population distribution and UAVs\({}^{\prime}\) position. (**a**) Raw population distribution. (**b**) Processed population distribution. (**c**) UAVs\({}^{\prime}\) position.  
offload the traffic of densely populated areas, alleviate the bottleneck of the network link between satellites and ground nodes, and increase the total network capacity of the AAN. Furthermore, the UAV-aided AAN has the fewer number of network path changes which could adversely affect communication status, such as temporary communication disconnection, communication re-initialization, etc.  
Based on the network paths of satellite-only and UAV-aided AAN, we compared the round trip time (RTT) of each network path. The comparison result is represented at Figure 13. In the figure, the solid blue line represents the estimated RTT based on  
Figure 11: Network path of the satellite-only AAN. (**a**) t = 0.0 s. (**b**) t = 17.0 s. (**c**) t = 85.8 s. (**d**) t = 140.3 s. (**e**) t = 191.9 s.  
Figure 12: Network path of the UAV-aided AAN. (**a**) t = 0.0 s. (**b**) t = 28.9 s. (**c**) t = 114.2 s. (**d**) t = 140.5 s.  
UAV-aided AAN's network path, and the red dotted line represents the improvement rate of UAV-aided AAN in comparison with satellite-only AAN. As the result, the RTT of UAV-aided AAN is less than up to 9.5 ms and has 5.1% of improvement rate in average compared to the RTT of satellite-only AAN.