# Towards a Decentralized Collaborative Framework for Scalable Edge AI
## 3 Background  
We present the various methodologies traditionally used for training AI/ML models on big data generated from large-scale IoT, sensors, and mobile devices. Figure 1 depicts the common four approaches for learning largely decentralized data. We discuss them as follows:  
**(a) Centralized Learning (CL)--Figure 1a:** In most cases, the end devices continually feed the generated data into the servers running in the cloud, where they are kept to be processed and analyzed later. These data are evaluated to identify properties that will assist in training AI and ML models using the cloud. High-end servers are used to train these models in the cloud providers' data centers. ML-as-a-service is becoming popular for training models on large amounts of data at scale provided by various cloud providers, such as Google Cloud, Azure, and AWS. Unfortunately, centralized learning (CL) is seeing weaker adoption due to the gathering of user data, which poses serious threats to users' personal information and privacy [28; 29]. Further, the scalability is impeded due to high communication costs associated with data transfer [7; 23]. Even though various works have aimed at reducing the communication overhead in the context of distributed data-parallel learning [30], various challenges remain unsolved [11]. **(b) Federated Learning (FL)--Figure 1b:** In the FL paradigm, the clients are users with end devices (e.g., mobile, IoT, or sensors) using apps generating the data. The client devices maintain the data and never share it with others. So, the clients train local models and engage with the FL server to create a common model. The server assists the clients with the task of aggregating the clients' models and coordinating the FL rounds [6; 7; 8]. The training procedure consists of a series of rounds managed by the server until a specific goal is achieved (e.g., target accuracy). In each cycle, the server selects a small set of clients for participating in the FL round who perform training of the model on their local data and share their new models to create a new global model. The server plays a crucial task in performing this aggregation step. However, this synchronous update step results in issues related to synchronization, dependability, and communication costs [11]. When a training round begins, available clients log in and express their desire to participate with the server. Typically, a client is available if it is fully connected (through IEEE 802.11/WiFinetwork, for instance), has a power source, and is idle [7]. Some recent works have explored battery-powered scenarios [20]. The server then executes a selection algorithm to choose a subset of the large client population. Then, the server sends the selected clients the FL task (i.e., the ML model and hyper-parameter configurations). All the clients run the same number of optimization steps determined by the FL task configuration. Then, the clients upload their new models to the server. The FL server then combines the client models to generate a new global model, which is check-pointed to the server's storage [7]. However, because of the non-IID distribution of the data, FL tends to be less efficient than CL due to the risk of divergence and lower model accuracy and the role of malicious clients aiming to diverge the model [28]. **(c) Decentralized Learning (DL)--Figure 1c:** DL is an alternative way to build generic models based on decentralized data in edge-device environments [9]. In DL, the learners use peer-to-peer communication to coordinate their training of a model tailored to their shared tasks [23]. Thus, device groups can train a shared model while maintaining the data of individual devices. The devices must always be present to repeat the training procedure in lockstep, and stragglers slow down the training because there is no central coordination. As a result, scalability and efficiency are impeded, since devices cannot train independently of slow learners [11].