# Chained Anomaly Detection Models for Federated Learning: An Intrusion Detection Case Study
## 3 Accountable Federated Learning for Anomaly Detection
### Anomaly Detection through Federated Learning
In order for the model averaging to have a similar expected learning capacity as the centralized case, three assumptions have to hold: (a) all the clients share the same initial weights; (b) the weight updates are disseminated to the clients synchronously and irrespective of their  
Figure 3: Centralized versus federated learning (local optimization and sharing of new weights with parameter server). participation in the last global epoch; (c) the learning rate is common amongst all the clients. With these assumptions holding, the only variable between the federated and the centralized mode is how apart the distributions of the local datasets are. Approaching it from the global scope of the server, we would reframe the question as how independent and identically distributed (henceforth IID) are the mini-batches. After we fix the learning rate \(\eta\), each client \(k=1..K\) downloads the weights from the parameter server and then independently calculates \(g_{t}^{k}=\
abla F_{k}(w_{t})\), the average gradient on its local data at the current model \(w_{t}\). The clients can perform this in batch or mini-batch mode and for more than one epoch. Then, the server aggregates these gradients and applies the following update:  
\[w_{t+1}\gets w_{t}-\eta_{t}\
abla f(w_{t})\quad\text{with}\ \ f(w_{t})=\sum_{k=1}^{K}\frac{n_{k}}{n}F^{k}(w_{t})\quad\text{or}\quad \
abla f(w_{t})=\sum_{k=1}^{K}\frac{n_{k}}{n}g_{t}^{k}\]  
where \(f(w_{t})\) is the loss function, \(w_{t}\) the model parameters at time \(t\), and \(n_{k}\) the size of the data partition of client \(k\) so that \(n=\sum_{k}n_{k}\). In equal manner for each client \(k\), the final weight update becomes:  
\[\forall k,w_{t+1}^{k}\gets w_{t}-\eta g_{t}^{k}\quad\quad w_{t+1} \leftarrow\sum_{k=1}^{K}\frac{n_{k}}{n}w_{t+1}^{k}\]  
At the end of each local update, the parameter server computes the weighted average of the local weights \(w_{t+1}^{k}\). At this point, we need to address the underlying distribution of each local dataset. The work by [PERSON] et al. (2017) shows that model averaging, with the aforementioned assumptions holding, is robust towards unbalanced and non-IID data distributions, with a certain overhead in required training epochs to achieve similar model performance. For this intrusion detection use case that learns an autoencoder model based on features of benign network traffic, we can establish a claim for approximately-IID data given that the infrastructure, protocols, and benign user behavior are largely the same for all clients. Moreover, when we consider this unsupervised learning case, the federated gradient descent finds a lower dimensional manifold where the benign traffic data collectively lie, which implies that there is a distribution that describes and generates them, irrespective of what client from which they originate. Ultimately, the efficacy of the federated averaging algorithm compared to the centralized version is validated in non-IID use-cases, which provides a lower bound on expected performance. Figure 4: Flowchart of the federated aggregation of the local weight updates into the master model. Considering the potential upscaling of the federation, for a large value of \(K\) and especially when the deep learning algorithms would be directly run on a heterogeneous set of powerful and resource-constrained clients, the federated learning algorithm can be dynamically adapted to collect weight updates \(w_{t+1}^{k}\) from a smaller random subset of the \(K\) nodes. Otherwise, computing the federated average could significantly delay the computation of an updated deep learning model if the algorithm has to always wait for the slowest contributor. In our experimental setup, we can vary the number of deep learning clients from 1-12 as the CICIDS2017 dataset contains network flow traces of 12 machines being monitored. With just one deep learning client, as shown in Figure 3 on the left, we end up with a centralized learning configuration where each of the 12 monitored systems is connected to the same deep learning client.