# Reducing Defense Vulnerabilities in Federated Learning:
## 3 Methods
### Classification of Dimensionally Reduced Impactful Neurons  
The objective of previous operations is to increase the divergence between malicious users and benign users' model updates. Thus, the classification becomes more accurate. As a result of the classification, the models of \((K-m)\) users determined to be reliable are added to the FedAvg federated collection and a new \(\theta_{global}\) is created. The new global model is sent to \(K\) users, starting a new federated learning tour that will continue for \(FL_{cycle}\) rounds. The One-Class Support Vector Machine (OC-SVM) [25] detects anomalies by creating a boundary around normal data points, labeling those outside this boundary as outliers. In our case, we classify only benign users, and the rest are marked as malicious users. Given a training dataset comprising instances \(\mathsf{x}_{1},\dots,\mathsf{x}_{n}\in\mathsf{X}\) where \(X\) represents the selected neurons after dimensions are reduced, the OC-SVM optimization problem can be formally expressed as follows:  
\[\min_{w,\mathsf{\rho},\mathsf{\rho}}\frac{1}{2}||w||^{2}+\frac{1}{vn}\sum_{i= 1}^{n}\xi_{i}-\mathsf{\rho} \tag{13}\]  
This optimization is subject to the following constraints:  
\[(w\cdot\phi(x_{i}))\geq\mathsf{\rho}-\xi_{i} \tag{14}\]  
\[\xi_{i}\geq 0 \tag{15}\]  
where \(w\) presents the normal vector to the hyperplane, \(\rho\) denotes the bias term, and \(v\in(0,1]\) serves as a regularization parameter that controls the trade-off between training errors and model complexity. The feature mapping function \(\phi\) projects the input data into a higher-dimensional space, while \(\xi_{i}\) represents slack variables that accommodate soft margins. The resulting decision function is defined as \(f(x)=\text{sign}((w\cdot\phi(x))-\rho)\), where positive values indicate benign model updates and negative values denote malicious ones. Agglomerative Clustering (Agg-C) [26] is a hierarchical technique that begins with each data point as an individual cluster, merging the nearest clusters step by step until one cluster or a specified number of clusters is achieved. This method produces a dendrogram, which shows the hierarchical relationships within the model updates. Consider a set \(X=x_{1},\dots,x_{n}\) comprising \(n\) data points. The algorithm initializes with a cluster set \(C=C_{1},\dots,C_{k}\), where \(k=n\), indicating that each data point initially constitutes its own cluster. The clustering process proceeds iteratively by merging the two most similar clusters based on a specified linkage criterion \(d\big{(}C_{i},C_{j}\big{)}\). Three predominant linkage criteria are employed as follows:  
Single linkage, which considers the minimum distance between clusters:  
\[d\big{(}C_{i},C_{j}\big{)}=\text{min}d(x,y):x\in C_{i},y\in C_{j} \tag{16}\]  
Complete linkage, which evaluates the maximum distance:  
\[d\big{(}C_{i},C_{j}\big{)}=\text{max}d(x,y):x\in C_{i},y\in C_{j} \tag{17}\]Average linkage, which computes the mean distance between all point pairs:  
\[d\big{(}C_{i},C_{j}\big{)}=\frac{1}{|C_{i}|\big{|}C_{j}|}\sum_{x\in C_{i}}\sum_{ y\in C_{j}}d(x,y) \tag{18}\]  
where \(d(x,y)\) represents a suitable distance between points \(x\) and \(y\). The iterative merging process continues until a termination criterion is met, such as achieving a pre-determined number of clusters \(k\), reaching a distance threshold \(\tau\) between clusters, or consolidating all points into a single cluster. The k-Nearest Neighbors (k-NN) [27] algorithm classifies data points based on the majority class of their k closest neighbors. For regression, it predicts values by averaging the \(k\) nearest neighbors' outputs. For a query point \(x_{q}\), the classification is determined by the following:  
\[\hat{y}q=\text{arg}max_{y}\sum{(x_{i},y_{i})}\in\mathcal{N}_{k}\big{(}x_{q} \big{)}ly_{i}=y \tag{19}\]  
The selection of the \(k\) value and distance metric presents a significant importance to be effective in the defense. We chose \(k\) as 4 and the Manhattan distance among the model updates.