# Federated Learning: A Comparative Study of Defenses Against Poisoning Attacks
## 5 Defense Methods
### PCA + K-Means  
Principal Component Analysis (PCA) is a widely used dimensionality reduction technique in machine learning and statistics [6]. Its primary goal is to transform high-dimensional data into a lower-dimensional representation that captures the most important information while minimizing loss. PCA accomplishes this by identifying the principal components, which are linear combinations of the original features, ordered by the amount of variance they explain. By retaining a subset of these components, PCA allows for a more compact representation of the data, aiding in visualization, analysis, and, in this context, the identification of potential patterns or clusters, such as those indicative of malicious activity in federated learning systems [23].  
The use of dimensionality reduction combined with a clustering algorithm has been proposed as a method to identify and remove malicious clients that perform both targeted and untargeted poisoning attacks [6].  
The central insight underlying this method of defense is that updates from malicious clients will \"look different\" than updates from benign clients. Benign clients are simply trying to improve the overall accuracy of the global model on their training data, while malicious clients have some other goals, such as to degrade the performance of the global model or to covertly install a backdoor subtask.  
We do not need to know how to quantify what characteristics we might expect malicious updates to have. We can develop a method to identify a cluster of updates that are \"generally different\" from the mass of benign updates.  
The algorithm is similar to that used for typical federated learning, except that a dimensionality-2 PCA is performed on a subset of the local updates. The algorithm, as presented below, then plots these two-dimensional points to identify the malicious cluster, but the cluster can also be identified using the K-means clustering algorithm with two centroids, identifying the smaller cluster as malicious [6,7].  
The algorithm for the federated learning system using PCA + K-means as the aggregation model works as follows (Algorithm 1):  
```
Input: numberOfAgents, T (number of iterations) Output: globalWeights // Step 1: Initialization and Configuration  Initialize agents with indices [1,2,..., numberOfAgents]  Initialize globalWeights  Set t = 0 // Step 2: Main Training Loop whilet < Tdo // Step 2: Calculate probabilities for agent selection  probabilities = CalculateSelectionProbabilities(agents) // Step 2: Start parallel training and wait for completion  selectedAgents = SelectAgents(probabilities)  updates = ParallelTraining(selectedAgents) // Step 2: Data Collection and Preprocessing  updateMatrix = CollectUpdates(updates) // Step 2d: Dimensionality Reduction with PCA  reducedData = PCA(updateMatrix) // Step 2: Cluster Analysis using K-Means  clusters = KMeansClustering(reducedData) // Step 2: Cluster Selection to identify anomalies  selectedClusters = IdentifyAnomalousClusters(clusters) // Step 2g: Weighted Aggregation within clusters  aggregatedUpdates = WeightedAggregate(selectedClusters, clusterSizes) // Step 3: Evaluation and Updating  globalWeights = UpdateGlobalWeights(globalWeights, aggregatedUpdates)  EvaluateModel(globalWeights) // Step 4: Close Loop  StoreEvaluationResults(globalWeights)  t = t + 1 endwhile return globalWeights
```  
**Algorithm 1**Identifying malicious model updates using PCA + K-means