## 3 Federated Learning for Space-Air-Ground Integrated Networks
### Device Selection Optimization Strategy Based on Hierarchical Reinforcement Learning
#### 3.4.2 Hierarchical Attention Mechanism  
To optimize device selection in federated learning, we introduce a Hierarchical Attention Mechanism (HAM) that dynamically adjusts participation weights of devices at different levels, thereby improving the convergence efficiency of federated learning while reducing communication overhead. The core objectives of the Hierarchical Attention Mechanism are as follows:  
Device Level: Select optimal devices within each layer (satellites, unmanned aerial vehicles, ground devices). Hierarchical Level: Allocate optimal computational and communication resources among different layers to optimize global model aggregation. We introduce a two-level attention mechanism to select devices:  
Global Layer Attention: Calculates weights for different layers, determining how much computational resource to allocate to each layer. Local Device Attention: Selects specific devices within each layer to participate in federated learning. In calculating global layer attention, we compute the attention weight for each device category (such as satellites, UAVs, ground devices) based on the characteristics of the device category, rather than metrics for individual devices. Specifically, for each device category \(k\) (such as satellites, UAVs, ground devices), its attention weight \(a_{\text{layer},k}\) is calculated as follows:  
\[a_{\text{layer},k}=\frac{e^{\lambda_{1}\cdot\text{avg}(C_{k})+\lambda_{2}\cdot \text{avg}(B_{k})+\lambda_{3}\cdot\text{avg}(n_{k})}}{\sum\limits_{l=1}^{L}e^{ \lambda_{1}\cdot\text{avg}(C_{l})+\lambda_{2}\cdot\text{avg}(B_{l})+\lambda_{3} \cdot\text{avg}(n_{l})}} \tag{16}\]  
where \(a_{\text{layer},k}\) represents the global layer attention weight for device category \(k\) (such as satellites, UAVs, ground devices); \(C_{k}\), \(B_{k}\), and \(n_{k}\) respectively represent the average values or representative indicators of computational capability, bandwidth, and data volume for device category \(k\); and \(\lambda_{1}\), \(\lambda_{2}\), and \(\lambda_{3}\) are weight coefficients that control the influence of computational capability, bandwidth, and data volume when calculating global layer attention. Within each layer, the selection weight \(a_{i}\) of devices is calculated using the computational capability, bandwidth, and data volume of the device, and the global layer attention weight \(a_{\text{layer},k}\) is introduced to adjust the priority of device selection. The specific calculation formula is as follows:  
\[a_{i}=\frac{e^{\tau_{1}C_{i}+\tau_{2}B_{i}+\tau_{3}n_{i}+\tau_{4}a_{\text{layer },k}}}{\sum\limits_{j=1}^{N_{\text{layer},k}}e^{\tau_{1}C_{j}+\tau_{2}B_{j}+ \tau_{3}n_{j}+\tau_{4}a_{\text{layer},k}}} \tag{17}\]  
where \(a_{i}\) represents the local attention weight of device \(i\) in layer \(k\), indicating the contribution of device \(k\) to model aggregation, \(a_{\text{layer},k}\) represents the global attention weight of device category \(k\), used to adjust the priority of different device categories, and \(\tau_{1}\), \(\tau_{2}\), \(\tau_{3}\), and \(\tau_{4}\) are used to adjust the influence of computational capability, bandwidth, data volume, and global layer attention on local device selection. When selecting devices, we need to combine global layer attention weights and local layer attention weights to calculate the probability of a device being selected. This ensures that the priorities of both the device category at each layer and the specific devices within that category are effectively weighted in the selection process. The device selection probability \(P_{i}\) is calculated as follows:  
\[P_{i}=\frac{a_{i}\cdot a_{\text{layer},k}}{\sum\limits_{j=1}^{N_{\text{layer}, k}}a_{j}\cdot a_{\text{layer},k}} \tag{18}\]  
where \(P_{i}\) is the probability of device \(i\) being selected to participate in aggregation, with a higher probability when the device's contribution is higher. \(a_{i}\) represents the local attention weight of device \(i\), indicating the device's contribution to aggregation.