# Rebirth of Distributed AI--A Review of eHealth Research
## 6 FL Architectures in eHealth  
There are two commonly adopted FL architectures in healthcare applications: (1) the conventional distributed and (2) the decentralized architecture [24]. See Figure 9.  
The traditional FL architecture is distributed by a star network that carries a centralization feature embodied in the aggregation actor. In distributed architecture, the participants independently train the model and send the updates to an aggregation server. The aggregation role is conducted by an external, trusted third-party or server. The aggregation server is responsible for collecting the updates from each of the participants, combining them, and producing a new version of the model. On the other hand, in decentralized architecture, also known as peer-to-peer FL approach, there is no single point where the decision is made. The strategy of keeping this role within an external server or delegating it to the participants is mainly dependent on the feasibility of full agreement of all parties (e.g., hospitals) on one trusted, external third-party, or for one of them to play the aggregation actor. Another factor is the type of data being processed--whether it is EHR or IoT-based data. Therefore, we find that most of the proposed solutions in research that deal with EHR from different hospitals adopt the decentralized, not the distributed, approach. For example, in [38] the researchers opted for decentralized rather than distributed for their EHR-based model. They justified this option due to the fact that in a set of different hospitals it would not be feasible to have a fusion center that is trusted by every node (hospital) to collect healthcare data. Instead of using the traditional SGD optimization algorithm with one master and many slaves, they applied fully decentralized, nonconvex stochastic algorithms for federated learning and obtained reasonably good results for health record datasets. In terms of topology, instead of using the typical FL star network, they used a well-connected graph network. Ref. [45] also built its solution using the decentralized approach in working on medical images that resided in different sources. The proposition acknowledges that the variation of medical images, combined with the limited number of medical images, would cause significant variation of parameters updated by clients, leading to bad convergence after aggregation. The core idea of their solution was to alleviate the variations among different clients by transforming the raw medical image data of all clients onto a common image space, via image-to-image translation, without violating the privacy setting in FL. In terms of performance, of the two, decentralized could be the better solution as we find in [26]. In this paper, the authors introduced BrainTorrent, an FL framework without a central server. It trained a complex, fully convolutional neural network in a decentralized fashion using whole-brain image segmentation with 20 classes having severe class imbalance. Through experiments, they proved that BrainTorrent achieved a  
Figure 9: Distributed FL vs Decentralized FLbetter performance than FL with servers under different experimental settings. The typical network topology in decentralized FL architecture is a fully meshed network. Nevertheless, different topologies have been used and some proved out-performance. For example, ref. [63] proved, through empirical results and a theoretical investigation, that the Erdos-Renyi graph, if used as the network topology for a decentralized FL architecture, can outperform a fully meshed network. They concluded that the network topology used in decentralized FL has an impact on the performance of the model convergence as they conducted their comparison with other prevalent networks, namely Scale-Free Networks, Small-World Networks, and Fully Connected Networks. Although decentralized FL is preferred in EHR settings, it is applicable for IoMT-based data settings, as we find in [64].