## 3 Federated Learning for Space-Air-Ground Integrated Networks
### Privacy-Preserving Federated Learning
#### 3.5.1 Local Training at Terminal Nodes
), bandwidth, and network latency to balance privacy protection and training effectiveness is a challenge in federated learning. The goal of this paper is to dynamically adjust the privacy budget \(\epsilon_{i}\) according to device characteristics, finding a balance between privacy protection effect and training effect through optimization strategies. Based on this goal, this paper proposes a dynamic privacy budget allocation strategy based on reinforcement learning (RL) and optimization theory, aiming to optimize the privacy budget of devices through intelligent scheduling to enhance the overall efficiency of federated learning. We view the allocation of privacy budget \(\epsilon_{i}\) as a multi-objective optimization problem, considering both privacy protection and training effect. First, the privacy budget \(\epsilon_{i}\) for device \(i\) can be represented as the following function:  
\[\epsilon_{i}=f(C_{i},B_{i},n_{i},\Delta f_{i}) \tag{34}\]  
where \(C_{i}\) represents the computational capability of device \(i\) (such as CPU/GPU performance), \(B_{i}\) represents the bandwidth of device \(i\), \(n_{i}\) represents the amount of training data on device \(i\), and \(\Delta f_{i}\) represents the sensitivity of the local training process on device \(i\), reflecting the sensitivity of local model updates to data changes. The objective function consists of two main aspects:  
Privacy protection objective: Improve privacy protection effect, reduce the risk of privacy leakage, controlled by the standard deviation of differential privacy noise. The privacy protection objective is inversely proportional to the privacy budget \(\epsilon_{i}\):  
\[Privacy\_Loss_{i}=\frac{1}{\epsilon_{i}} \tag{35}\]  
Training effect objective: Improve the training accuracy of the device model. The training effect objective is related to the size of the noise; the smaller the noise, the better the training effect, usually proportional to \(\epsilon_{i}\):  
\[x_{i}^{\text{norm}}=\frac{x_{i}-\mu_{i}}{\sigma_{i}} \tag{27}\]0Combining these two objectives, the optimization objective for the privacy budget is balanced through a weighted sum approach:  
\[x_{i}^{\text{norm}}=\frac{x_{i}-\mu_{i}}{\sigma_{i}} \tag{27}\]1  
where \(\rho_{1}\) and \(\rho_{2}\) are weight coefficients used to adjust the relative importance of privacy protection and training effect. We adopt a reinforcement learning method to dynamically adjust the privacy budget \(\epsilon_{i}\) for each device, using the Q-learning algorithm to optimize the allocation of privacy budgets. The reinforcement learning strategy is defined as follows:  
State space \(S_{i}^{\prime}\): The state of device \(i\) consists of the device's computational resources \(C_{i}\), bandwidth \(B_{i}\), training data amount \(n_{i}\), and model sensitivity \(\Delta f_{i}\):  
\[x_{i}^{\text{norm}}=\frac{x_{i}-\mu_{i}}{\sigma_{i}} \tag{27}\]2  
Action space \(A_{i}^{\prime}\): The action of device \(i\) is to adjust the privacy budget \(\epsilon_{i}\), i.e., selecting a reasonable privacy budget value. We assume that the privacy budget can be chosen from a discrete set:  
\[x_{i}^{\text{norm}}=\frac{x_{i}-\mu_{i}}{\sigma_{i}} \tag{27}\]3  
where \(M\) is the number of selectable privacy budgets. Reward function \(R_{i}^{\prime}(\epsilon_{i})\): After selecting the privacy budget \(\epsilon_{i}\), device \(i\) receives a reward composed of privacy protection and training effect components:  
\[x_{i}^{\text{norm}}=\frac{x_{i}-\mu_{i}}{\sigma_{i}} \tag{27}\]4  
Optimization process: Device \(i\) learns the optimal allocation strategy for the privacy budget through the reinforcement learning algorithm Q-learning. The device selects the privacy budget based on its current state and updates the strategy according to the reward function \(R_{i}^{\prime}(\epsilon_{i})\) to achieve the optimal balance between privacy protection and training accuracy.