## 3 Problem Formulation  
FL is utilized to accommodate numerous devices that accumulate data and a central server that manages the global learning objective throughout the network of devices. Important notation used in this paper is indicated in Table 1.  
We have \(N\) IoT devices, each with a local dataset \(D_{i}\) and a computational cost represented by \(p_{i}\). The goal is to minimize a federated objective function \(f(w)\) defined as  
\[f(w)=(1/N)\sum_{i=1}^{N}p_{i}F_{i}(w), \tag{1}\]  
where \(F_{i}(w)\) is the local objective function of device \(i\). \begin{table}
\begin{tabular}{c c} \hline \hline
**Notation** & **Definition** \\ \hline \(f(w)\) & Objective function of the FL \\ \hline \(p_{i}\) & Computational cost of training an epoch in a local IoT device \\ \hline \(C(E_{i})\) & Communication cost of training an epoch in a local IoT device \\ \hline \(TC(E_{i})\) & Total cost of training an epoch in a local IoT device \\ \hline \(L\Big{(}f_{w}^{(E_{i})}\cdot x\Big{)}\) & Loss function of a local IoT device in an FL scenario \\ \hline \(TC(E_{i})\) & Total cost of performing an epoch in a local IoT device \\ \hline \(\pi_{\theta\pi}(a_{i,t}\mid s_{i,t})\) & Reinforcement learning agent policy \\ \hline \(Q(s,a,\theta^{*})\) & Action value estimation \\ \hline \(U(s,a)\) & Upper confidence bound for each action \\ \hline \(q_{i}\) & Weight quality score for local IoT device’s trained weights \\ \hline \(\mathcal{S},\mathcal{A},\mathcal{R}\) & State space, action space, and immediate reward function \\ \hline \(s_{t},a_{t}\) & A state and an action taken by the control agent at time step \(t\) \\ \hline \(\gamma\) & Discount factor \\ \hline \(T\) & Number of time steps in an episode \\ \hline \(m\) & Deep learning model size of each local IoT device \\ \hline \(\mathcal{D}\) & Set of local datasets \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Important notation. At each round \(t\) of the FL process, device \(i\) selects a random subset \(S_{i}\) of its local data \(D_{i}\) and performs local training on \(S_{i}\) using the current model parameters \(w_{t}\) to obtain a new set of parameters \(w_{(t+1,i)}\). Then, the device \(i\) sends \(w_{(t+1,i)}\) to a central server. The central server aggregates the received model updates to obtain a new global model parameter \(w_{(t+1)}\):  
\[w_{(t+1)}=(1/N)\sum_{i=1}^{N}w_{(t+1,i)}, \tag{2}\]  
where \(w_{(t+1)}\) is the new global model parameter, \(w_{(t+1,i)}\) is the local model parameter from device \(i\), \(N\) is the number of clients, and \(t\) is the training round. The new global model parameter \(w_{(t+1)}\) is sent back to all devices, and the process repeats until convergence. However, a significant challenge emerges when dealing with heterogeneous industrial IoT devices. These devices are characterized by their diverse computational capacities, available energy resources, and data sizes, which can vary widely from one device to another. This heterogeneity presents a formidable task when it comes to selecting the optimal number of epochs for each IoT device in the context of FL. Currently, FL methods use the same number of epochs for every participating device without considering the heterogeneous issues present in IoT devices. However, if the epoch count is set too low to reduce the usage of energy resources and computation resources, the model might be unfit. Thus, the model fails to learn the underlying patterns in the data and performs poorly on real-world data because it has not adequately captured the complexity of the data. On the other hand, if the epoch count is set too high, the model might overfit the local data on the device. Overfitting occurs when the model learns the training data too well, to the point that it captures not only the underlying patterns but also the noise or random fluctuations in the data. Moreover, it also increases the usage of energy and computational resources, which reduces the usability of IoT devices for a long time.