# Lightweight Anomaly Detection in Digit Recognition Using Federated Learning
## 7 Experimental Results and Analysis
### Federated Learning Scenario 1: Non-IID--Single Digit per Device
Following the second local training phase (_Training 1_), performance generally improves across all digits, especially with Model 3. The second federated round (_Federated 1_) results in the highest overall F1 scores for all three models, indicating that this is the optimal aggregation point. After receiving the second global model, devices proceed with another local training phase (_Training 2_) and then evaluate performance. During this phase, several devices observe a decline in performance, which activates the global early stopping mechanism. As the majority votes to terminate training, this signals that additional federated updates are unlikely to provide further benefit. A final federated round (_Federated 2_) was conducted to confirm this behavior, and the results showed a clear drop in performance, validating that _Federated 1_ indeed represents the peak in collaborative learning. Among all the tested architectures, Model 3 outperforms the others, achieving an average F1 score of approximately 0.86 after the _Federated 1_ round. This result is very close to the 0.87 F1 score obtained in the centralized setting using the same model (Table 5). This gap is negligible considering the extreme data skew and distributed nature of training, highlighting the robustness of federated learning even in highly non-IID conditions. To further illustrate the AD capability, Table 6 presents the AD rates for each digit on both the MNIST and EMNIST datasets using Model 3 after FL. As shown in the table, the false positive rate for MNIST is approximately 9%, while the true negative rate on EMNIST is about 81.5% across all devices. These results are reasonable considering the simplicity of the model and the similarity between MNIST and EMNIST. The table also includes F1 scores along with their standard deviations across the devices, showing that the variation is larger after local training rounds compared to federated rounds, which aligns with expectations. While not directly comparable, our experimental setup shares similarities with the study by [PERSON] et al. [44], where each device is also trained on a single digit class from the MNIST dataset. Their model achieves a nearly perfect false positive rate of 1% compared to our 9% result. However, their approach differs in several important ways. They use a more complex AE architecture, perform model aggregation after every training epoch, and train on the full MNIST dataset of 60,000 samples (split by class). In contrast, we employ a significantly smaller autoencoder, use only 10,000 MNIST samples (also class-separated), and aggregate less frequently, after every 10 to 20 epochs, depending on the model, which results in a total of approximately 40 to 80 training epochs. Our design offers considerable advantages in terms of resource efficiency: reduced training time, significantly lower memory requirements, and reduced communication overhead by avoiding per-epoch aggregation. In constrained embedded environments, such a trade-off between performance and efficiency can be both practical and justified.