# Communication-Efficient Vertical Federated Learning
## 2 Background
### Communication-Efficient Federated Learning  
In general distributed learning environments, communication costs have always been a constraint. Since federated learning is a form of distributed machine learning that deals with data privacy issues, this field has attracted a lot of interest in terms of improving communication efficiency. The studies that have been conducted related to communication-efficient FL have focused on the issue of huge communication rounds or bandwidth specifically in the HFL environment. The strategies that have been explored so far that could make communication in FL more efficient include choosing selective clients, reducing the number of model updates, and applying compression schemes to models. Client selection is a strategy for improving communication efficiency, while lowering costs by limiting the number of participants. As a result, just a portion of the parameters are updated over the communication round. [PERSON] et al. [10] proposed a communication-efficient FL framework using a probabilistic device selection scheme such that only those clients are selected for model transmission that have higher probabilities to improve the convergence speed and training loss. [PERSON] et al. [11] proposed a similar approach in which restrictions were imposed on the number of local models sent to the server for aggregation using various protocols (e.g., by random sampling, or by applying thresholds based on the amount of local data or local validation error). As the number of communication rounds between the devices and the central server can be costly, reducing model updates is also a possible solution. [PERSON] et al. [11] also suggested training local models on devices to completion instead of computing increments and then applying ensemble methods to effectively capture information regarding client-specific models, reducing communication rounds to one in the best case. [PERSON] et al. [12] introduced a Partitioned Variational Inference (PVI) for probabilistic models, in which they train a Bayesian Neural Network (BNN) over an FL environment that allows for both synchronous and asynchronous model updates across many machines. Their proposed approach, combined with the integration of other methods, allows for more communication-efficient training of BNN on non-iid federated data. Some other studies that focus on reducing communication overhead in FL settings include a study by [PERSON] et al. [13], in which they designed a one-shot federated learning algorithm FedKT, which uses a knowledge transfer technique that outperforms the other state-of-the-art federated learning algorithms with a single communication round. Furthermore, [PERSON] et al. [14] presented a federated fusion learning approach that allows distribution parameters of local data to be sent to the central server rather than model parameters. Synthetic data are generated at the central  
Figure 1: Iterative Federated Learning Environment (Adapted from [4]). server using those distribution parameters, which are then merged to train a global machine learning model. Hence, the communication between the client and server occurs in a single round. The application of compression schemes to models is a widely used strategy to mitigate the communication cost problem in large-scale machine learning. Gradient quantization is a technique of quantizing the gradients into lower precision values to reduce the number of gradients transmitted. For instance, 1-bit SGD [15] and QSGD [16] can reduce the gradient to 10% of the uncompressed data. SignSGD [17] considers the case, wherein gradients are quantized using only \(+1\) and \(-1\), and shows its convergence by the aid of a majority vote of the clients. Some studies propose that applying quantization to the local or global model can help to improve communication efficiency in FL as well. [PERSON] et al. [12] applied quantization to each local model before sending it to the server, reducing overall communication overhead.