## 2 The Rise of the Federated Learning Approach
### Taxonomy of FL Models
#### 2.2.3 Federated Learning Architectures
However, the existence of a central coordinator remains a \"bottleneck\" in contemporary environments, due to retaining data sovereignty over the devices. In bearing in mind the work in [12], a malicious node may intrude the network and manage to reconstruct the produced-by-devices data that are sent to the server by knowing only their gradients. Also, in cases with cross-silo settings, the organization and storage of all the sensitive parameters, e.g., medical tests, may cause severe problems in the case of an attack to the server. Decentralized Federated Learning (Purely Distributed)  
Decentralized federated learning [18] is a form of federated learning in which there is no server to coordinate the communication among the cooperative nodes and thus the training procedure. Every participating device communicates with its one-hop neighbor node (peer-to-peer or device-to-device networks), and all nodes are usually connected, forming a symmetric topology (see Figure 2), e.g., a ring, d-ring, or grid network topology and more rarely, a fully connected mesh [10] network. It is of a great importance to have symmetric topologies in DFL, meaning that the node resources are balanced, and communication among the nodes is uniform, since these factors impact both the scalability and fault-tolerance of the network directly. However, in [PERSON] et al. (2019), [PERSON] et al. investigated the case when non-symmetric topologies (e.g., star topologies in which the central node has a more crucial role than the rest of the connected nodes) are used and proposed the _push-sum_ protocol in order to address this imbalance. The training procedure continues to be executed until the devices reach a _consensus_[PERSON] et al. (2019), meaning that the global model has converged, e.g., as described in over-the-air approaches [PERSON] et al. (2019); [PERSON] et al. (2020); [PERSON] et al. (2020). This mechanism bridges the gap between data safety and communication overhead, since every device exchanges information and communicates only with its one-hop neighbor. However, due to a lack of memory storage, devices also lack complex algorithms that determine if a node is trusted or provide defense from a contingent attack of a malicious node, e.g., byzantine faults (adversarial learning [PERSON] et al. (2019)). The objectives in DFL are summarized in Figure 3.  
Figure 2: Different topologies in FL: (**a**) star-shaped topology, (**b**) ring-shaped topology, and (**c**) the topology of a mesh network. Hybrid FL  
Although a server-less mechanism is ideal, especially in IoT networks, since device independence is preserved, it is not fully scalable for large-scale networks due to the unreliable connection of D2D communication [31]. Some recent works [32; 33; 34; 35] tried to deploy a hybrid model, or else a _semi-decentralized_ federated learning setting, in order to exploit advantages from both the aforementioned categories. In Semi-DFL, the connection of the entities with the coordinator server is configured with clustering algorithms, e.g., max-min d clustering, k-means, etc., which go beyond the idea of flat networks, and as a result, the central server communicates only with the nodes that represent the respective cluster (the clusterheads). For instance, [PERSON] et al. introduced a tree-shaped hierarchy of layers between devices on the edge and the server, namely, multi-stage hybrid federated learning (_MH-FL_) [36], and later, [PERSON] et al. [31] advanced this idea by introducing two-timescale hybrid federated learning (_TT-HF_), in which the model reaches consensus irregularly. To facilitate training, the network of devices is organized into smaller groups (clusters), with devices sending their model to their sub-clusterhead aperiodically, and the sub-clusterhead also intermittently broadcasting the updated local model to the server. It is worth highlighting that TT-HF converges at a rate of \(O(1/t)\), which is similar to the centralized stochastic gradient decent (SGD) algorithm.