## 2 Background
### Communication
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Methods** & **Description** & **Advantages** & **Considerations** \\ \hline \multirow{4}{*}{Differential updates} & Instead of transmitting the entire model, participants & Significantly reduces the amount of data transmitted, especially when only small parts of the model & Efficient algorithms are needed to calculate and transmit the \\  & & the changes (gradients) in their local model parameters. & parts of the model have changed. & differentials accurately. \\ \hline \multirow{4}{*}{Model compression} & Techniques like quantization, where model parameters are represented with fewer bits, and pruning, where insignificant model weights & Reduces the amount of data that must be transmitted, reducing bandwidth and computational overhead. & Balancing compression levels to maintain model accuracy is crucial. \\  & & are removed, reduce the model size & computational overhead. & is crucial. \\  & before transmission. & & \\ \hline \multirow{4}{*}{Decentralized optimization} & Algorithms like federated averaging allow model & Minimizes communication & \multirow{4}{*}{Requires careful coordination to ensure accurate aggregation.} \\  & updates to be computed locally and averaged among participants, reducing the need to transmit raw data & & & \\  & or gradients. & & \\ \hline \multirow{4}{*}{Edge computing} & Algorithms that intelligently select a subset of clients for participation, reducing the total number of updates transmitted. & Reduces the communication overhead by selecting a representative subset of clients, optimizing the use of bandwidth. & Requires algorithms that balance randomness and representation to avoid biased sampling. \\ \cline{2-2}  & Computation and updates are performed locally on edge devices, reducing the need for frequent communication with a central server. & Minimizes communication by allowing edge devices to handle computations and updates, reducing latency and bandwidth usage. & Ensuring that edge devices have sufficient computational resources and storage capacity is essential. \\ \hline \multirow{4}{*}{Adaptive communication} & Dynamic communication frequency and volume adjustment based on network conditions, participant capabilities, and system requirements. & Optimizes communication overhead in real time, ensuring efficient use of resources. & Requires continuous monitoring and adaptation, potentially introducing computational overhead. \\ \hline \multirow{4}{*}{Cryptography and encryption} & Secure communication protocols use encryption techniques to protect data during transmission. & Ensures data privacy and security, allowing sensitive information to be transmitted securely. & Introduces computational overhead for encryption and decryption processes. \\ \hline \hline \end{tabular}
\end{table} 
Table 6: An overview of some of key communication-efficient methods. nuanced approach to model training, diverging from traditional methods that necessitate the transmission of raw data to a central repository. At the heart of Federated SGD lies the principle of gradient computation at the local device level. Each participating device in the network utilizes its local data to calculate gradients, which represent the partial derivatives of the loss function with respect to the model parameters. This local computation not only preserves the privacy of user data by avoiding raw data transmission but also significantly reduces the volume of data that needs to be communicated across the network. This aspect of Federated SGD is particularly advantageous in scenarios where network bandwidth is limited. * Privacy preservation and data integration. The privacy-preserving nature of Federated SGD is one of its standout features. By enabling local gradient computation, the algorithm ensures that sensitive data remains within the confines of the originating device. These locally computed gradients, encapsulating the necessary information for model updates, are then securely transmitted to a central server [40].