## 3 Research Questions and Communication Efficient Methods
### RQ2-How Can Communication Be More Efficient in an FL Environment?
#### 3.2.5 Compression Schemes
According to the authors in [84], one key problem with regard to federated setting and communication is the communication overhead that is involved in parameter synchronization. The authors inform that this overhead wastes bandwidth and increases the training time while impacting the overall model accuracy. To tackle this, the authors propose a general gradient sparsification (GGS)framework for adaptive optimizers. The framework consists of two important mechanisms: batch normalization updates with local gradients (BN-LG) and gradient correction. The authors determine that updating the batch normalization layer with local gradients could mitigate the impact of delayed gradients and not increase the communication overhead. The authors run their analysis over several models, such as AlexNet, DenseNet-121, CifarNet, etc., and achieve high accuracy results, concluding that gradient sparsification does have a significant impact in reducing the communication overhead. The authors in [85] introduce a compression scheme that is both a communication-efficient and deferentially private federated learning scheme. The authors call it CPFed. The challenge that the authors face when addressing both issues together is one that arises from data compression. Techniques used for compression can sometimes lead to an increased number of training iterations required for achieving some desired training loss due to the compression errors; however, differential privacy could deteriorate with regard to the training iterations. To overcome this paradigm, their proposed CPFed is based on a sparsified privacy masking technique that adds random noise to model updates along with an unbiased random sparsifier before updating the model. The authors can achieve a high communication efficiency through their proposed model. The authors in [86] propose a compression technique that could drastically reduce the communication cost for a distributed training environment. The framework introduced by the authors is a sparse binary compression (SBC) technique; their method integrates techniques that are already present in communication delay and gradient sparsification with a novel binarization method. According to their research, the authors find that the current gradient information for training neural networks with SGD is redundant. Instead, the authors utilize communication delay methods introduced in [6] to introduce temporal sparsity, where gradients are not communicated after every local iteration. From their findings, the authors conclude that, in a distributed SGD setting, both the communication delay and gradient sparsity can be treated as independent types of sparsity techniques. These methods provide higher compression gains, though, in their findings, the authors also find a slight decrease in the accuracy. _Quantization:_ The communication between the devices and the central server includes sharing model updates and parameters that have occurred on the device end. This updating of parameters can be strenuous when it comes to up-linking the model. To aid with this, another compression method referred to as quantization can bring the model parameters to a reasonable size without compromising much on the model accuracy [87]. In this subsection, different techniques and frameworks that have integrated quantization towards achieving communication efficiency in the FL setting are introduced. The authors in [61] utilize their framework, FedPAQ, to reduce the overall communication rounds and the bearing cost. A feature of their framework is to quantize message-passing. The communication bandwidth will mostly be limited in an FL setting, where a limited uplink bandwidth on the participating device end could increase the communication cost, making it more expensive. The authors in [61] employ quantization techniques [88] on the up-links; every local model is quantized before being uploaded, hence reducing the overall communication overhead. Furthering and employing quantization techniques, the authors in [89] use them for their FL analysis.