## 2 The Rise of the Federated Learning Approach
### Delving into Decentralized Federated Learning (DFL)
#### 2.3.1 Communication Reduction  
In order to achieve communication reduction, [PERSON] et al., in [PERSON] et al. (2019), proposed a heuristic (CEDFed) in which a one-bit signal is transmitted among selected neighbors in order to reduce the communication requirements of the network. Parameters (which are about to be sent) are firstly encoded to the node \(i\) in which they were trained, and this node \(i\) takes advantage of an encoding matrix \(\Phi\), is accessed by specific nodes belonging to a neighborhood \(M_{i}\). Accordingly, node \(i\), after receiving the one-bit message, decodes it using the _IBCS_ algorithm. Authors also claim that if the \(\Phi\) matrix follows a _Gaussian_ distribution, the dimension \(d_{i}\) of the transmitted dense vector of node \(i\) is upper bounded (\(d_{i}<n\)), and the message is sparsified, then the decoded message has fewer chances to be corrupted. The algorithm was tested, using a linear regression task in a randomly generated decentralized topology, while the competitor was the same algorithm under ideal communication conditions, and the results verified that there are negligible losses in message conversion under the aforementioned assumptions. Furthermore, Decentralized Federated Averaging (DFedAvgM) [PERSON] et al. (2019) is based on the DSGD algorithm. In particular, DFedAvgM proposes a scheme in which participating neighbor nodes communicate with each other after multiple local updates, while DSGD requires communication among the neighbors after each local iteration. In taking into account that the communication cost is the main bottleneck in peer-to-peer communication schemes, DFedAvgM is introduced as more communication-efficient than DSGD, since fewer communication rounds are taking place. Moreover, a quantized version of the proposed algorithm has been introduced, alleviating communication flow and thus time. It is worth highlighting that authors have demonstrated theoretical proofs regarding both convex (for which the worst-case convergence rate coincides with the DSGD one) and non-convex cases, in which the Polyak-Lojasiewicz (PL) condition is used, and under this assumption, quantized DFedAvgM converges faster than the simple DFedAvgM, which mainly depends on both the network topology and the computational resources of the participating nodes. Regarding the experimental evaluation, a CNN deep neural network was trained in both image classification and language modeling tasks, executing the quantized DFedAvgM algorithm, and the results proved the sufficiency of this method, independently of the number of bits transmitted and the data distribution used (either in cases where IID or non-IID data). Finally, the algorithm was also tested for its privacy-preserving capabilities as it is exposed to a membership inference attack (MIA) either in ring-based or d-regular-based network infrastructures. The results verified the efficiency of the algorithm in facing membership attacks, while the lower the AUC, the safer the deployed model (the ideal value of the AUC was proven to be 0.5). Consequently, DFedAvgM demonstrates higher performance than FedAvg in terms of communication costs, paving the way for secure D2D large-scale implementations. In distributed systems, the _gossip_ protocol is an appealing choice for dealing with large-scale networks. In the machine/deep learning field, the _gossip learning_ protocol technique was first introduced in [PERSON] et al. (2019), in which a portion of the participating nodes share the sum of the weights of the global model (GoSGD) in both a fully decentralized and asynchronous manner. In [PERSON] et al. (2019), a gossip-based protocol scheme, namely Combo, was proposed in conjunction with a _split_ learning approach. To elaborate, in every participating node \(i\), the model \(i\) is divided into a specific amount of pieces, in which the trainable parameters are distributed equally and they do not coincide.