## 2 The Rise of the Federated Learning Approach
### Taxonomy of FL Models
#### 2.2.3 Federated Learning Architectures  
Centralized Federated Learning[8] with the FedAvg algorithm was first introduced in order to address the problem of data privacy. The FedAvg algorithm is the reference point in the evolution of centralized federated learning algorithms, since it has been already proven that it converges and achieves a sub-optimal convergence range of O(1/T) when training the model using the _stochastic gradient decent_ algorithm under settings where the convexity of the loss function holds. After FedAvg, a lot of variants were introduced with remarkable convergence rates and high statistical accuracy levels, e.g., DFedSGD, FedProx, SCAFFOLD, VanillaSGD, etc. [21]. However, CFL was also a great solution motivating resource-scarce devices to run DL models on their own. The devices run their own local training for a few epochs and then send their parameters to the server for further calculations, which in turn forwards back the updated weights to the participating entities. However, even if the server mitigates the computational cost of running the collaborative learning, the communication overhead remains at high levels. In taking into consideration that in FL the bandwidth of uploading/downloading data is very small (about 1 MB/s) and we often deal with \"asymmetric\" networks [22], which probably means that not all devices have the same capabilities in the network and specifically the velocity of uploading or downloading differs from device to device, the continual communication among node devices and the server conducted in order for the model to be trained may cause cross-device _overflow_ in communication links. Also, when the energy of these devices is also constrained, this data exchange may deplete all their energy. There are a lot works in the literature that justify the advancements of this technique to small and tiny devices, for instance, the _edge stochastic gradient descent (eSGD)_[11] algorithm for updating weights, based on the importance of parameter values. The eSGD algorithm takes into account a few significant gradients, which are further forwarded to the server in order to update the global model. The eSGD algorithm compares the loss values of two consecutive iterations of the algorithm, and if the present value is smaller than the previous one, it means that these gradients are important for the model training, and thus, these weights are kept. There are a lot works in the literature that justify the advancements of this technique to small and tiny devices, for instance, the _edge stochastic gradient descent (eSGD)_[11] algorithm for updating weights, based on the importance of parameter values. The eSGD algorithm takes into account a few significant gradients, which are further forwarded to the server in order to update the global model. The eSGD algorithm compares the loss values of two consecutive iterations of the algorithm, and if the present value is smaller than the previous one, it means that these gradients are important for the model training, and thus, these weights are kept. Another _selective communication protocol_ is introduced in [23]. [PERSON] et al. present a centralized federated learning paradigm in which not all nodes participate in the learning process but only the nodes that achieve high rewards. The authors propose a Neural _Contextual Combinatorial Bandit (NCCB)_ algorithm for rewarding nodes not only for their contribution, extracted from system _experience_, but also for their data cross-device similarity, while using the K-means clustering algorithm to fit nodes into groups. There is a lot of remarkable work in centralized federated learning [11]; however, the existence of a central server that takes over the orchestration of the training procedure is still considered a disadvantage in recent environments. [PERSON] et al. [24], for instance, proposed a method of encrypted message transmission between the server and the participating nodes in order to enhance data security, with imperceptible client-side overhead for message encryption.