## 6 Conclusions  
This study presented the UA-FedRL method, an innovative approach designed to tackle the challenges of implementing FL in heterogeneous industrial IoT environments. By dynamically selecting local epochs, UA-FedRL effectively managed the complexities of non-IID datasets and straggler IoT devices, improving accuracy, computation, and communication efficiency. The proposed PWA method further enhanced performance by addressing weight aggregation issues and adjusting the weights of individual models based on their quality. Two commonly used datasets, MNIST and CIFAR-10, were employed to perform extensive experiments of the proposed UA-FedRL method. UA-FedRL obtained an accuracy of 96.45% on the MNIST dataset and 62.75% on the CIFAR-10 dataset, when 90% straggler devices were used to train the model. Furthermore, uncertainty estimation showed the effectiveness of UCB algorithms in achieving good performance in decision-making tasks. These results demonstrate that UA-FedRL outperformed the benchmark in terms of faster convergence and higher training accuracy on both datasets, indicating its potential for enhancing the performance of FL in heterogeneous industrial IoT environments.  
While our proposed UA-FedRL method provided good performance, there are some limitations which include the reliance on stable network conditions. Therefore, the communication between devices and the central server may be affected by unstable connectivity which can impact the overall performance. Additionally, the method's performance may vary depending on the degree of data heterogeneity, as highly variable data distributions might reduce the effectiveness of the quality-based weighting mechanism. To address these  
Figure 16: The uncertainty estimation of the UA-FedRL taking each action in terms of reward.  
limitations, future research could explore adaptive communication strategies to optimize network performance and data normalization techniques to better handle extreme data variability. Furthermore, we can optimize the PWA method by integrating additional factors such as temporal stability for aggregation. Moreover, clustered aggregation can also be investigated to improve robustness in highly heterogeneous environments.  
The following supporting information can be downloaded at: [[https://www.mdpi.com/article/10.3390/app14188299/s1](https://www.mdpi.com/article/10.3390/app14188299/s1)]([https://www.mdpi.com/article/10.3390/app14188299/s1](https://www.mdpi.com/article/10.3390/app14188299/s1)).  
Methodology, A.S.M.S.S. and M.Z.I.; software and coding, M.Z.I. and A.S.M.S.S.; experimentation and formal analysis, M.Z.I. and A.S.M.S.S.; writing--original draft preparation, M.Z.I., and A.S.M.S.S.; writing--review and editing, M.Z.I., A.S.M.S.S., and A.H.; supervision, H.-S.K. and A.H.; funding acquisition, M.Z.I. and H.-S.K. All authors have read and agreed to the published version of the manuscript.  
This work was supported by the National Research Foundation of Korea (NRF) Grant by the Korean Government through MSIT under Grant 2022R1F1A1063662 and the Strengthening R & D Capability Program of Sejong University.  
Not applicable.  
The original contributions presented in the study are included in the article, further inquiries can be directed to the corresponding authors.  
The authors declare no conflicts of interest.