# FedOpt: Towards Communication Efficiency and Privacy Preservation in Federated Learning
## 1 Introduction  
Artificial Intelligence (AI) has been employed in a plethora of application fields in recent years [1]. In this context, as a notable branch of AI, Deep Learning (DL) has been broadly used to empower plenty of data-driven real-world applications, such as facial recognition, autonomous driving and smart grid systems [2; 3; 4]. These DL-based applications usually demand the gathering of large quantities of data from various IoT edge-devices for training high-quality learning models. However, the traditionally centralised DL models require the local edge-devices to upload their private data to a central cloud server, which may cause serious privacy threats [5]. These privacy threats can be mitigated through distributing the local training among multiple edge-devices, which has led to the emergence of Federated Learning (FL) [6]. Federated Learning (FL) resolves this problem by allowing the edge-devices to collaboratively train a DL model on their individually gathereddata, without revealing their private-sensitive data to a centralised server. This privacy-preserving collaborative learning technique is achieved by following three simple steps as illustrated in Figure 1. In the first step, all the users download the global model and the learning parameters from the cloud server. In the second step, the users train the local learning models based on their local data using Distributed Stochastic Gradient Descent (DSGD) [7]. Finally, in the third step, all the users upload the parameters of their locally trained models to the server, where they are aggregated to generate a new global model. These three steps are continuously repeated until the desired convergence level is achieved. However, despite this efficient training scheme, a major issue in FL is the massive communication overhead that generally evolves from the model updates [8]. In specific, following the above described FL protocol, each user has to communicate its full gradient update during each epoch. This update is normally the same size as the fully trained model, where the trained model could be in the size of gigabytes based on the DL architecture and its millions of parameters [9]. This size can easily reach petabytes when the training is conducted on large-scale datasets that require thousands of training epochs. As a result, communication cost with the limited bandwidth makes FL completely infeasible and unproductive. In addition, FL still faces various privacy concerns as shown in recent studies [10]. For example, a malicious user in a network of shared parameters can access the personal images of users from surveillance systems through various attacks. In a different context, several adversaries can similarly violate the emergency responses of autonomous vehicles or change the health records of several patients from their wearable devices [11]. These threats not only result in serious privacy leakages but also bring in unpredictable life losses. As a result, privacy preservation in FL has become an important factor that spurs the further advancements and developments of efficient FL approaches. To the best of our knowledge, none of the existing approaches supports communication efficiency and privacy preservation in FL at the same time [12]. To this end, in this paper, we propose a novel approach, namely, Federated Optimisation (FedOpt), based on Distributed Stochastic Gradient Descent (DSGD) optimisation. The major contributions in this approach are summarised as follows:  
1. FedOpt utilises the novel Sparse Compression Algorithm (SCA) in order to reduce the communication overhead. In particular, SCA extends the existing top-k gradient compression technique and enables downstream compression with a novel mechanism. 2. FedOpt adopts a lightweight homomorphic encryption for efficient and secure aggregation of the gradients. In particular, FedOpt provides a concrete abstract, where additively homomorphic encryption is completely utilised in order to eliminate the key-switching operation and to increase the space for plain-text. 3.