## 5 Application Deployment Strategies
### Industrial Internet of Things  
The exponential growth of IIoT development leads to heterogeneous device taxonomies and complexity in management entities. The control mechanism must take the scalability, privacy, and elasticity into deployment schemes for ensuring a long-term support, reliable architecture, and multi-service efficiencies. IIoT in the new era requires automation and robotic systems for intelligent manufacturing, analysis, and high productivity. To gain automation, AI is a well-fitted paradigm; however, it requires gathering historical experiences and data-driven pattern processing. To secure the data of local equipment or industrial organizations, eFL can be proposed and evaluated to develop a collaborative learning model. In the phases of executing eFL framework, DRL-based approaches make great contributions, such as autonomous data management, efficient resource orchestration, and IIoT client selections.  
In [66], an actor-critic DRL-based device selection for FL-enabled IIoT was proposed. To make the DRL agent applicable, a setup environment must be well-defined with observability of resourceful features and specific characteristics. The authors modeled the structure of a digital twin in IIoT, which consisted of two layers, namely physical and digital twins. Virtual systems bring connectivity between IIoT and digital models. With well-configured interfaces, the agent can obtain the states for figuring out the underlying patterns and for making smart decisions in client selection within each FL communication round. In [69], the authors initialized the environment and agents for all IIoT equipment in the system architecture, and each piece of equipment consisted of task(s) for FL updates. The proposed agent trained the data to evaluate the feedback scores and then updated the significant weights before aggregation. Furthermore, DRL-assisted FL can be used for ensuring the heterogeneity of IIoT data privacy and management [89].  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**DRL/FL-Enabled Domains** & **Contributions** & **Ref.** & **Year** \\ \hline \multirow{8}{*}{_IIoT_} & An optimized asynchronous FL by intelligent DRL-based client selection/clustering approaches to build models in newly integrated digital twin with IIoT environment & [66] & 2022 \\ \cline{2-4}  & Dynamic resource allocation based on DRL for optimizing problems on communication cost, energy consumption, and execution latency of FL-enabled IIoT networks & [69] & 2021 \\ \cline{2-4}  & Improved IIoT client selection in eFL framework by evaluating data quality through deep deterministic policy gradient algorithm & [89] & 2021 \\ \hline \multirow{8}{*}{_Smart Automation_} & Handling computation overhead and privacy-preserving scheme of massive IoT participants in lightweight FL framework & [90] & 2021 \\ \cline{2-4}  & Blockchain-enabled local training and privacy-preserving analytics for FL-enabled real-time smart grid architecture & [91] & 2021 \\ \hline \multirow{8}{*}{_Medical Services_} & A comprehensive review on FL-enabled smart healthcare, which includes DRL-based experience-driven resource management and assessment metrics on data contributions & [92; 93] & 2021 \\ \cline{2-4}  & A thorough study on efficiencies of reinforcement learning for smart healthcare systems in tackling the future edge intelligence (collaborative local-edge AI-based applications) & [94] & 2020 \\ \hline \multirow{8}{*}{_IoV_} & Minimizing system costs of FL frameworks by using a greedy algorithm to optimize the vehicular participant selection based on the quality of data contributions & [74] & 2021 \\ \cline{2-4}  & DRL for experience-driven generation of IoV data training and FL-based framework for secured IoV models & [95] & 2022 \\ \hline \multicolumn{4}{c}{_Environmental Context_} & Analysis of waste and natural disasters for contributing to the \multirow{2}{*}{_Detection_} & [96] & 2020