## 3 Federated Learning for Space-Air-Ground Integrated Networks
### Privacy-Preserving Federated Learning
#### 3.5.1 Local Training at Terminal Nodes  
Due to the data distribution heterogeneity among drones, ground user terminals, and other devices, where each device \(i\)'s dataset \(\mathcal{D}_{i}\) may have different statistical properties (such as mean, variance, etc. ), data preprocessing is necessary at the local level to ensure data consistency during the training process and to reduce the impact of data heterogeneity on training results. In space-air-ground integrated networks, sensor data collected by drones, in particular, may be subject to noise interference. Therefore, data denoising and cleaning are required. The denoising and cleaning process is represented as follows:  
\[x_{i,\text{cleaned}}=f_{\text{clean}}(x_{i}) \tag{26}\]  
where \(x_{i}\) represents the original dataset on device \(i\), and \(f_{\text{clean}}(\cdot)\) is the data cleaning function that removes noise and outliers. Since input data \(x_{i}\) from different devices may have different scales and ranges, standardization or normalization is needed to ensure numerical stability during the training process. The standardization of data \(x_{i}\) is represented as follows:  
\[x_{i}^{\text{norm}}=\frac{x_{i}-\mu_{i}}{\sigma_{i}} \tag{27}\]  
where \(\mu_{i}\) is the mean of data \(x_{i}\) on device \(i\), and \(sigma_{i}\) is the standard deviation of data \(x_{i}\) on device \(i\). The normalization of data \(x_{i}\) is represented as follows:  
\[x_{i}^{\text{scaled}}=\frac{x_{i}-x_{\text{min}}}{x_{\text{max}}-x_{\text{min}}} \tag{28}\]  
where \(x_{\text{min}}\) and \(x_{\text{max}}\) are the minimum and maximum values of data \(x_{i}\) on device \(i\), respectively. In the local training phase, each device trains the model using its local dataset \(\mathcal{D}_{i}\) with the goal of minimizing the local loss function. Assuming that device \(i\) uses model \(f_{\theta_{i}}(\cdot)\), the objective is to update the model parameters \(\theta_{i}\) by optimizing the local loss function. Suppose the local dataset for device \(i\) is \(\mathcal{D}_{i}\), the loss function \(L(\theta_{i},\mathcal{D}_{i})\) can be expressed as  
\[L(\theta_{i},\mathcal{D}_{i})=\frac{1}{n_{i}}\sum_{(x_{i},y_{i})\in\mathcal{D} _{i}}\mathcal{L}(f_{\theta_{i}}(x_{j}),y_{j}) \tag{29}\]where \(\mathcal{L}(\cdot)\) is the loss function on device \(i\), \(n_{i}\) is the number of data points on device \(i\), and \(\theta_{i}\) is the parameter of the model on device \(i\). Device \(i\) updates the model parameters \(\theta_{i}\) based on the local dataset using gradient descent as follows:  
\[\theta_{i}^{(t+1)}=\theta_{i}^{(t)}-\eta\
abla_{\theta_{i}}L(\theta_{i}^{(t)}, \mathcal{D}_{i}) \tag{30}\]  
where \(\eta\) is the learning rate used to control the step size of parameter updates. \(\
abla_{\theta_{i}}L(\theta_{i}^{(t)},\mathcal{D}_{i})\) represents the gradient of the loss function \(L0\) with respect to the model parameters \(\theta_{i}\), indicating the direction of steepest descent in the parameter space. To overcome data heterogeneity and device heterogeneity, each device's model can transfer knowledge from other devices through transfer learning. During the local training process, device \(i\) can dynamically select appropriate source tasks for transfer learning based on data and model similarities between devices. The mathematical model for transfer learning can be represented as  
\[\theta_{i}^{\text{adapted}}=\theta_{s}-\eta_{\text{meta}}\
abla_{\theta_{i}}L (\theta_{i},\mathcal{D}_{i}) \tag{31}\]  
where \(\theta_{s}\) represents the model parameters transferred from other devices (source devices), and \(\eta_{\text{meta}}\) is the learning rate for transfer learning. Through transfer learning, device \(i\) can accelerate the training process with limited data by transferring knowledge (i.e., transferred model parameters) from source devices. To protect the privacy of local data on terminal devices, differential privacy is a commonly used privacy protection method.