# A Survey on IoT-Edge-Cloud Continuum Systems: Status, Challenges, Use Cases, and Open Issues
## 2 Supporting Technologies in the IoT-Edge-Cloud Continuum
### Distributed and Decentralized Machine Learning  
As also mentioned in the introductory part, over the last decade, ML algorithms have emerged as a potential solution to relax the computational burden of traditional optimization approaches and provide near-optimum solutions in highly non-convex problems [38]. In centralized ML approaches, data collected directly from different network devices (i.e., mobile terminals, access points, IoT devices, edge servers, etc. ), are sent to a high-performance computing server for proper model training. Afterwards, model inference to all involved devices takes place, if necessary. However, there are several disadvantages with this approach, especially in the modern era of IEC systems: (i) centralized data collection might lead to high computational load, especially for a large number of involved devices and associated datasets, as well as to a single point of failure; (ii) since the vision of the IEC continuum involves multiple connected heterogeneous devices over diverse infrastructures, data preprocessing prior to the actual training of the ML model is necessary, which might increase overall training time and result in system latency deterioration; (iii) frequent transmission of data from IoT devices to centralized servers might drive security and privacy concerns since not all IoT devices have the processing power to execute advanced security protocols; and (iv) computationally demanding ML training might have an impact on the energy footprint of the involved devices. Distributed ML approaches can reduce the centralized computational burden, either by parallelizing the training procedures or by efficiently distributing training data [39; 40]. The first case, which is also known as model parallelism, enables different parts of the model to be trained on different devices (e.g., certain layers of a neural network (NN) or certain neurons per layer are trained per device). In the second case, each ML node takes into account a subset of the training data. Afterwards, model aggregation takes place. Although both aforementioned approaches can improve training times and relax the computational burden, unavoidably, training data offloading still takes place. Consequently, their deployment on privacy-critical applications might be questionable. To overcome the aforementioned issue, the concept of FL has emerged over the last years [41; 42] as a promising approach that ensures distributed ML training on the one hand and privacy protection on the other hand. To this end, training is performed locally on the involved devices, with no need for forwarding training data to external servers. At predefined time intervals, the parameters of the trained model are sent to the central processing node, where the master model is periodically updated. Moreover, since training data remain localized, privacy is enhanced, as was previously mentioned. In addition, with FL, data can be distributed across many devices, which can enable the use of much larger datasets. Moreover, the amount of data transfers and the communication burden are reduced, especially in cases where the data are distributed across devices with limited connectivity or bandwidth. Finally, FL allows the model to be trained on a diverse range of data sources, which can improve its robustness and generalizability, as well as overall training times. For example, focusing on the previously mentioned autonomous driving 5G scenario, using this approach, a predefined set of identical cars can be parallelly trained on different landscapes. Results can then be aggregated and sent back to the autonomous cars in order to cover a wide range of driving reactions. A schematic diagram of FL is shown in Figure 4, in the case of NN training. In this case, each node locally trains the corresponding ML model with the available local data set. The derived parameters (i.e., weights of the NN in the specific case) are sent periodically to the master processing node for proper model aggregation.