## 1 Introduction
### Weight Aggregation Challenges in Heterogeneous Industrial IoT Scenarios  
The problem of weight aggregation in heterogeneous industrial IoT devices for federated learning is due to inherent complexities and variations in device capabilities, data distribution and available resources. Heterogeneous industrial IoT devices encompass resource-constrained edge servers, limited resources, limited computing power, and battery endurance, which can affect the weight aggregation process during federated learning. In heterogeneous industrial IoT environments, local model updates from individual devices may differ significantly due to the varying data distribution, device resources, and training characteristics. Consequently, the central server must carefully aggregate these diverse local model updates to create a robust and accurate global model. Moreover, non-IID data are common in heterogeneous industrial IoT environments, leading to an imbalanced data distribution across devices. The central server must account for this imbalance during weight aggregation to avoid biases and ensure an accurate representation of the underlying data. IoT devices with limited processing power may not perform as many training iterations on their local data as other IoT devices. Therefore, the aggregation of those straggler IoT devices affects the overall performance of the global model accuracy and convergence. FL with a reinforcement learning approach has evolved in recent years as a potential way to train machine learning models in diverse and decentralized environments. However, the implementation of FL provides considerable hurdles due to the variability in data distribution, computational capacity, and resource availability among IoT devices. There is still potential for advancement in tackling the difficulties of implementing federated learning in dynamic IoT contexts, even though numerous studies on the topics of \"federated learning\" and \"heterogeneous networks\" have been published. This paper proposes an Uncertainty-Aware Federated Reinforcement Learning (UA-FedRL) method, which dynamically selects local epochs to manage heterogeneous industrial IoT devices in federated learning networks more effectively. In combination with the Predictive Weighted Average Aggregation (PWA) method, UA-FedRL provides a comprehensive solution to address the complexities and challenges of implementing federated learning in heterogeneous industrial IoT environments and acquiring high accuracy with communication and computation efficiency. Our proposed methods demonstrate outstanding results in harsh-IoT-environment simulations, outperforming other federated learning approaches in terms of accuracy, communication, and computational efficiency. Moreover, Software-Defined Networking (SDN) technology is implemented in the core network of the IoT environment with a focus on achieving a lower communication latency. The contributions of this study can be summarized as follows:  
* We propose a UA-FedRL method that addresses the challenges associated with the heterogeneity of IoT devices in non-IID data distribution scenarios. The UA-FedRL method dynamically selects local epochs, improving accuracy, computation, and communication efficiency while effectively managing heterogeneous industrial IoT devices in federated learning networks. * We introduce the PWA method to address weight aggregation issues commonly found in heterogeneous industrial IoT scenarios. This method calculates the weight quality of every local IoT device using the predictive log-likelihood of the validation accuracy of the local model. The weights of individual models are adjusted based on their quality to mitigate the impact of heterogeneity and non-IID issues during aggregation. The rest of this paper is structured as follows. Section 2 presents a comprehensive literature overview that highlights key breakthroughs in FL for the IoT and identifies gaps in existing research techniques. Section 3 discusses the problem formulation of the study.