# Edge Federated Optimization for Heterogeneous Data
## 2 Related Works
### Comparative Summary  
Most of the research integrating federated learning with embedded devices primarily aims at conducting model training in constrained environments or implementing local data processing and model training in edge environments for accelerated data processing and real-time responses. These innovative studies are gradually overcoming challenges in the processing capabilities and communication efficiency of edge devices, invigorating the development in fields such as the IoT and smart devices, thereby demonstrating the potential and possibilities in this domain. Table 1 lists various embedded devices used for different federated learning training tasks.  
It is worth mentioning that finding a proper target density for pruning is nontrivial. Adaptive pruning models have been designed in [39; 41], while [40] combines structured and unstructured pruning. In [39], training was conducted separately on different datasets under both IID and non-IID settings. In the non-IID setting, a single dataset was trained in a non-IID manner by classifying it differently. Note that the above three works use different pruning methods to integrate their proposed algorithms. In contrast, our study employed two entirely distinct datasets for non-IID training with the partial node training strategy proposed in [38], which focuses on retraining weaker nodes in the network. However, when applying this method to our data model, it did not yield the expected results due to the non-IID nature of our dataset, which significantly differs from the independent and IID datasets used in [38]. Despite these challenges, we found the iterative structured pruning strategy mentioned in [40] to be effective during the training process. Therefore, we chose to use a structured pruning method, specifically employing the sparse optimization framework provided by [42], pruning 20% of the nodes, and focusing on training the remaining 80%. Table 2 compares the experimental methods and data formats in the literature.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Works** & **Node** & **Features** & **Data Type** & **Dataset** \\ \hline \multirow{2}{*}{[PERSON] et al. [26]} & \multirow{2}{*}{Arduino Nano 33} & Implementing a KWS application through federated & \multirow{2}{*}{non-IID} & Self-recorded voice \\  & & learning training using the Arduino Nano 33 BLE Sense & & \\  & & development board. & & \\ \hline \multirow{2}{*}{[PERSON] et al. [44]} & \multirow{2}{*}{Raspberry Pi} & In an IoT environment, employing federated learning & \multirow{2}{*}{IID} & \multirow{2}{*}{N-BaloT/LANDER} \\  & & & and an adaptive optimizer for efficient local training. & \\ \hline \multirow{2}{*}{[PERSON] et al. [38]} & \multirow{2}{*}{Nvidia Jetson Nano} & Utilizing sparse learning techniques to freeze & \multirow{2}{*}{IID} & MNIST/ \\  & & significant nodes during local training phases and & & \\  & & retrain weaker nodes. & & \\ \hline \multirow{2}{*}{[PERSON] et al. [45]} & Android smartphones & Investigating the implementation of federated learning & \multirow{2}{*}{IID} & Office-31 \\  & & Nvidia Jetson & on various smartphones and embedded devices using & \\  & devices + Raspberry Pi & the Flower framework. & & \\ \hline \multirow{2}{*}{[PERSON] et al. [39]} & \multirow{2}{*}{Raspberry Pi} & Introduces a method called “PruneFL.”, which reduces & \multirow{2}{*}{IID} & CIFAR-10/ \\  & & model complexity through pruning techniques, & \\  & & enabling its efficient execution on edge devices. & & non-IID & FEMNIST/ CelebA \\ \hline \hline \end{tabular}
\end{table}
Table 1: The comparison of different sensors used in FL.