# AutoFL: Towards AutoML in a Federated Learning Context
## 2 Related Work
### Federated Learning: Heterogeneity and Trade-Offs  
System heterogeneity in federated learning is the result of unbalanced computational resources and/or communication bandwidths across the clients, causing stragglers to indeed increase the federated training time. This is why federated learning algorithms typically perform multiple local iterations on a fraction of randomly sampled clients before aggregating the local model updates via the central coordinating server [23; 24]. [PERSON] et al. [5] explored different methods to reduce the costs of uplink communication. These methods include (1) structured updates, where updates are learned from a restricted space that is parameterized with a small number of variables, and (2) sketched updates, where full model updates are learned which are compressed with combinations of various techniques including quantization, random rotations, and subsampling before the update is sent to the server. The authors experiment with CIFAR-10 image data for image classification and the Reddit post data for next-word prediction. Their experiments show that there are trade-offs, amongst others, between the accuracy, the various communication cost reduction techniques, the number of rounds, and clients in the network. However, the experiments start with a predefined neural network, reusing \"Model C\" from [25] for the CIFAR-10 experiment and a custom LSTM model for the Reddit data experiment. Another concern is data distribution heterogeneity, i.e., the data is distributed across the different clients in an unbalanced manner or non-independent and identically distributed (non-i.i.d. ), causing the model not to converge during training. [PERSON] et al. [26] explore how to make communication more efficient for federated learning with non-i.i.d. data. [PERSON] et al. [14] proposed a method to address both system and data distribution heterogeneity based on an adaptive sampling of the clients. Rather than selecting clients uniformly at random or proportional to their amount of training data, the authors propose an optimal client sampling strategy that minimizes the wall clock time for training the model while offering convergence guarantees. Their experiments show a significant reduction of wall clock time even if their method requires more training rounds for reaching the same target loss as that of the baseline methods. Security and privacy are important concerns, and various threats and countermeasures within the frame of federated learning are discussed in topic-specific surveys [1; 4; 15]. These attacks vary from data and model poisoning attacks to membership and property inference attacks, as well as generative adversarial network (GAN) attacks. Typical defenses include differential privacy (DP), secure multi-party computation (MPC), and homomorphic encryption (HE). For example, [PERSON] et al. [13] explore the risk of training with sensitive data in the financial domain and the impact of privacy-enhancing techniques. More specifically, the authors investigate differential privacy to introduce noise to a model's parameters as a way to mitigate the leakage of private data. A drawback of differential privacy is the fact that it typically reduces the accuracy of the trained model. That is why the authors also explore secure multiparty computation such that the coordinating server does not learn private information. They investigate the impact of these techniques on a logistic regression model trained on a real-world credit card fraud dataset. The objective of their research was to offer a framework to an audience with a computer science background but without any prior knowledge of security, privacy, and distributed learning. They implicitly confirm the complexity of federated learning and its many trade-offs. While many of the previous works focused on federating neural networks, classical machine learning methods can also be federated. [PERSON] et al. (2020) recently proposed a novel model called federated forests.