# Towards a Decentralized Collaborative Framework for Scalable Edge AI
## 4 Challenges and Motivation  
We discussed traditional learning paradigms in the previous section, and analyzed their primary contrasts. Now, we focus on the significant difficulties presented by these paradigms, which lead to inefficient learning, limited scalability, or privacy concerns [6; 11; 28]. In Table 1, we qualitatively compare the existing approaches over efficiency, scalability, and privacy metrics. Efficiency refers to the ability of a mechanism to achieve good accuracy (or convergence) in a short time. Scalability refers to the ability of a scheme to scale the learning process over a larger number of clients with minimal costs. Privacy refers to the ability of a method to maintain or help enforce the privacy preservation of the client's data. Regarding efficiency, methods with greater control over data distribution (e.g., centralized) would result in model convergence in a short time [19]. On the other hand, methods that can coordinate the operations for many participants (e.g., federated) would result in better abilities for scale [21]. Regarding privacy, methods that keep the user's data private without sharing it (e.g., decentralized or federated) would result in better privacy guarantees [2]. While current centralized methods are the most efficient (due to the lack of non-IID data distribution problems), they are not desirable due to the non-existent or low privacy guarantees. The decentralized and federated methods overcome this issue by providing high privacy guarantees (due to the non-sharing of the data), but they fail to provide satisfactory convergence results quickly. Transfer Learning provides a good comprise, but it typically fails to scale well with large scenarios. Unfortunately, the current paradigms, such as federated, decentralized, and transfer learning, are not efficient or scalable enough to meet the growing demand and adoption of decentralized services and applications [11]. One of the detrimental challenges that the face of the traditional paradigms, which hinders efficiency and scalability, is the heterogeneity of the users and learning environment [6; 7; 11]. When training on decentralized parties, the common types of heterogeneity can be one or a combination of:  
* **Data**: non-uniform data points of the learners in number, type, and distribution. * **Device**: a diverse set of device hardware configurations and communication network settings. * **Behavioral**: dynamic participation availability driven by learners' behavior. We find that heterogeneity, regardless of type, can significantly impact the model quality (as well as fairness), and cause model divergence in the worst case [14]. In Section 6, we support this with experiments that cover many scenarios on five different benchmarks (i.e., models and datasets) and large populations of thousands of users [14]. Since heterogeneity is endemic to decentralized approaches, mitigating its impact on the performance of decentralized methods is desirable. Hence, we identify a pressing need to manage, process and analyze the increasing amount of decentralized data generated by collaborative parties to support the growth of ML and data-driven applications [7; 9; 11; 13]. We seize an opportunity from the proliferation of connected devices and edge computing, which create abundant data at the network's edge. Regardless, there is a lack of frameworks that can do so efficiently, at a low cost, and with fast training time, while observing privacy requirements. Without questions, any centralized approach that collects user data becomes infeasible. Nowadays, it is widely accepted that data are processed locally on edge devices, \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **Efficiency** & **Scalability** & **Privacy** \\ \hline Centralized Learning & High & Medium & Low \\ \hline Federated Learning & Low & Medium & High \\ \hline Decentralized Learning & Low & Low & High \\ \hline Transfer Learning & Medium & Low & Medium \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of existing approaches.