# Reducing Defense Vulnerabilities in Federated Learning:
## 3 Methods
### Selective Neural Pruning in NC-FLD  
The NC-FLD method uses model gradients \(\Delta\theta_{k}^{t}\), where k is the index of users, \(t\) is the FL round in the training process that end users obtain using their own training data and then send to the server. The entire defense mechanism is designed to be implemented on the server side. Weight Pruning (WP) is an algorithm used to eliminate neurons that have little effect on the artificial neural network and to lighten the models [23]. Although the primary purpose of WP is to reduce processing complexity, it is used to select the most impactful neurons in our proposed method.  
Basically, it allows the selection of ineffective neurons by the second-order Taylor expansion of model gradients. However, since calculating the Hessian matrix in all users' models would be a very costly process, ineffective neurons can be selected by using the magnitudes in the model gradients. The selected neurons carry more information about the learning process because they are the most affected neurons. After examining the change in model parameters, \(\psi_{k}^{t}\) neurons exceeding the determined threshold value are marked as they contain high information.  
We consider a neural network model with parameters \(\theta\) distributed across \(n\) participating nodes. For each local model at node \(i\), we define a binary mask \(M_{i}\in 0\), \(1\,|\,\theta\,|\) where \(|\,\theta\,|\) represents the total number of parameters. The pruning operation \(\mathcal{P}(\theta_{i},\,\rho)\) zeros out a fraction \(\rho\) of the smallest magnitude weights, creating a sparse model:  
\[\theta_{i}^{pruned}=\theta_{i}\odot M_{i} \tag{3}\]  
where \(\odot\) represents element-wise multiplication, and \(Mi\) is determined by  
\[M_{i}[j]=1\textit{ if }\theta_{i}[j]>\tau\rho \tag{4}\]  
\[M_{i}[j]=0,\text{ otherwise} \tag{5}\]  
Here, \(\tau\rho\) represents the magnitude threshold corresponding to the desired pruning ratio \(\rho\).  
When considering security implications, if an adversary attempts to inject malicious updates \(S^{\theta adv}\) into the global model, the pruned model update \(\delta\theta_{i}^{pruned}\) from client \(i\) to the server follows:  
\[\delta\theta_{i}^{pruned}=\mathcal{P}\Big{(}\theta_{i}^{new}-\theta_{i}^{old}, \rho\Big{)} \tag{6}\]  
This pruning operation constrains the adversary's degrees of freedom to \((1-\rho)|\theta|\) parameters. For the global aggregation operation \(\mathcal{A}(\cdot)\), the secure aggregation of pruned updates can be expressed as follows:  
\[\theta^{global}=\theta^{current}+\eta\cdot\mathcal{A}\Big{(}\delta\theta_{i} ^{pruned}i=1^{n}\Big{)} \tag{7}\]  
where \(\eta\) represents the learning rate and \(\mathcal{A}(\cdot)\) typically computes a FedAvg aggregation. The pruning threshold \(\tau_{\rho}\) can be dynamically adjusted based on the weight distribution:  
\[\tau_{\rho}=\text{quantile}\Big{(}|\theta_{i}[j]|j=1^{|\theta|},\rho\Big{)} \tag{8}\]  
After pruning operations are completed, the most impactful neurons are selected and stored as \(\theta_{i}^{pruned}\). Please note, this process is only carried out to select neurons. There is no model pruning in the process that will cause a change in the model architecture.