## 2 Preliminary Studies on Federated Learning and Deep Reinforcement Learning
### Federated Learning for Privacy-Preserving Assurance
To fully make this framework applicable, the implementation requires optimizing the resource \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Domains** & **Summary of Contributions** & **Ref.** & **Year** \\ \hline \multirow{4}{*}{_Background and classification of practical FL scenarios_} & The definition, enabling technologies, taxonomy, application types, challenging issues, and future directions of FL & [32] & 2021 \\ \cline{2-4}  & Collaborative learning and optimization approaches to efficiently train the models in a heterogeneous IoT environment & [33] & 2022 \\ \hline \multirow{4}{*}{_FL for resource-constrained IoT_} & Providing FL approaches for optimizing mobile edge networks, reviewing frameworks for FL execution, and discussing implementation challenges & [34] & 2020 \\ \cline{2-4}  & The enabling technologies, frameworks, protocols, application scenarios, and challenges of FL & [35] & 2020 \\ \hline \multirow{4}{*}{_Comprehensive learning on FL_} & An in-depth study on FL architectures/taxonomies, system designs, on-site deployment scenarios, and future research directions & [36] & 2021 \\ \cline{1-1} \cline{2-4}  & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Contributions of Existing Selected Surveys on Federated Learning. allocation, improving the security, and increasing the robustness. A resource-optimized FL for IoT was proposed in [38] by introducing a dispersed framework that aimed to optimize the placement and allocation with the robustness of distributed learning models. The minimization approach using an integer linear problem was implemented in this system architecture. To specifically tackle the problems in fine-grained detail, a model was split into two sub-problems, namely association and resource block allocation. The association problem considered the adjustment of fixed resource blocks within devices, and the resource block allocation problem tackled the association of fixed devices. Moreover, a further improvement on adaptive FL in resource-constrained systems was presented in [39] by tackling the challenging issues of model parameters from numerous edge nodes with its non-IID data. The convergence of gradient descent was analyzed to cooperate with a control scheme for adaptively managing the global aggregation in mission-critical circumstances. The aim of [39] was to minimize the loss within resource-constrained conditions by optimal tradeoffs between two primary resource types, namely in local model update and global aggregation. In [40], an enhanced asynchronous FL technique was proposed for alleviating the overhead communication latency and advancing the FL performance through adjusted weight aggregation. The modification of DNN layers was labeled as shallow and deep, which learned on general features of various tasks/datasets and ad hoc features, respectively. Well-defined weights lead to fast convergence, less latency per communication round, and accuracy improvement. Furthermore, another use case to be addressed is for non-IID, which possibly degrades the FL convergence expectation. By emphasizing each local model accuracy, user-based service with non-IID data distribution can perform well. To illustrate the applicability in this scenario, [41] proposed a multi-task FL for individual DNNs. The proposed scheme modified the conventional FL by authorizing the participants to customize their DNNs that fit the best for their non-IID data distribution. A new KPI of FL performance on user model accuracy was given for extending the measurement quality. However, privacy-enhanced collaborative architecture and model customization come with a tradeoff against communication costs within FL training phase. To outcome these issues, [42] offered robust and communication-efficient FL solutions in non-IID scenarios, which used a sparse ternary compression framework. The proposed compression scheme modified the existing compression method and handled two major problems: (1) the compression of downstream and (2) caching updates for participant synchronization.