# A Cluster-Based Optimal Computation Offloading Decision Mechanism Using RL in the IIoT Field
## 2 System Model  
As shown in Figure 1, the system consists of base stations (BSs), MECSs, and IIoT devices. As mentioned in Section 1, we do not consider the mobility of the device because the devices are fixed in the IIoT field. We assume that tasks of the same type are located close to each other. Therefore, IIoT devices are clustered based on distance. The cluster head (CH) is then selected for each cluster. The CH is the device with the highest computation capability within the cluster. The role of the CH is to run the proposed algorithm to determine offloading strategies for devices within the cluster. The reason why the device with the highest computation capability is selected as the CH is that it communicates the information of devices in the cluster and the information of MECSs, in addition to executing the task. Thus, the CH makes an offloading decision regarding the tasks occurring in its own cluster. Each BS is equipped with one MECS, where the task queue of the MECS consists of tasks to be executed on it. The device task queue holds tasks that are to be processed on the device. We use the task execution time model and the energy consumption model to minimize the task execution time and energy consumption and that are the objectives of our proposed paper. \(M\) represents a set of MECSs, with the number of MECSs equal to \(|M|\). \(D\) represents a set of devices, with the number of devices equal to \(|D|\). \(C\) represents a set of tasks, with the number of tasks equal to \(|C|\). A device generates a task with a delay constraint given by a Poisson distribution in each time slot. We assume that the task is not divided and is processed locally or offloaded. The task set created in a time slot \(t\) is \(C(t)\). Each task \(c_{d}(t)\) that is created by the device \(d\) (\(d\in D\)) in a time slot \(t\) consists of a tuple with three properties: \(c_{d}(t)=\big{(}v_{d}(t),w_{d}(t),L_{req}\big{)}\), where \(v_{d}(t)\) (in _Kbits_) is the data size of the task and \(w_{d}(t)\) (in _cycles/bit_) is the number of CPU cycles used per bit to process the task. \(L_{req}\) (in _seconds_) is the delay constraint of the task. Task execution delay is defined differently according to the offloading strategy of the task. To calculate task execution delay, the task transmission rate is essential if the task is transmitted to an MECS or another device. The task transmission  
Figure 1: The proposed system architecture of an IIoT environment. rate depends on the offload location because of using different links. The communication between the devices and MECS uses a cellular link, while the communication between devices is performed through a D2D link. The task transmission rate is computed using the Shannon-Hartley formula, which is a general communication model. The task transmission rate between device \(d\) (\(d\in D\)) and MECS \(m\) (\(m\in M\)) in time slot \(t\) can be described as:  
\[R_{d,m}^{cell}(t)=B^{cell}(t)\log_{2}\left(1+\frac{p_{d,m}^{cell}(t).h_{d,m}^{ cell}(t)}{\sigma^{2}}\right) \tag{1}\]  
where \(B^{cell}(t)\) denotes the channel bandwidth of cellular communication at time slot \(t\) and \(p_{d,m}^{cell}(t)\) is the power required by device \(d\) to transmit data to MECS \(m\) with the least load. \(h_{d,m}^{cell}(t)\) indicates the channel gain between device \(d\) and MECS \(m\) and \(\sigma^{2}\) is the noise power. The task transmission rate between devices \(i\) and \(j\) (\(i,j\in D\)) in time slot \(t\) is described as:  
\[R_{i,j}^{d2d}(t)=B^{d2d}(t)\log_{2}\left(1+\frac{p_{i,j}^{d2d}(t).h_{i,j}^{d2d }(t)}{\sigma^{2}}\right) \tag{2}\]  
where \(B^{d2d}(t)\) denotes the channel bandwidth of the D2D communication at time slot \(t\) and \(p_{i,j}^{d2d}(t)\) is the power required by device \(i\) to transmit data to device \(j\) with the least load. \(h_{i,j}^{d2d}(t)\) indicates the channel gain between devices \(i\) and \(j\). The downlink rate is not considered because the data output size of the task is relatively small.