## 3 Methodology
### Correlated Multi-Objective Energy-Efficient Routing Scheme for SDWSN-IoT
#### 3.2.3 The DOS-RL Routing Scheme  
The realistic conditions of an SDWSN-IoT network are subjected to dynamic changes in resources such as battery capacity, CPU capacity, memory, and bandwidth, as well as link quality changes during network operation. RL-based routing schemes have demonstrated considerable advantages in designing network operation policies that can handle such changes. However, such routing protocols often fail to respond quickly enough to such changes. To address this issue, we propose the DOS-RL routing scheme which allows the RL agent to learn from multiple correlated objectives simultaneously and to adaptively choose the objective it believes is the most promising from the current state. Therefore, we have turned the problem of energy efficiency into an RL process by modeling it into a four-tuple(S, A, P, R), defining the states (S), actions (A), policy (P), and reward functions (R) of the DOS-RL scheme. 1. _The Reward Function (R)_: In reinforcement learning, the agent evaluates the effectiveness of its actions and improves its policy by relying on rewards collected from the environment. The rewards obtained are typically dependent on the actions taken, with varying actions resulting in differing rewards. To implement the proposed DOS-RL scheme, we define three reward functions, each corresponding to one of the correlated objectives: the selection of routing paths with sufficient energy for data forwarding (\(o_{1}\)), load balancing (\(o_{2}\)), and the selection of paths with a good link quality for a reliable data transfer (\(o_{3}\)). Route selection based on any of these objectives is expected to improve the energy efficiency of the IoT network system. Details on each objective of the DOS-RL scheme and its related reward functions are provided below:Energy-consumption: To achieve objective \(o_{1}\), we consider the energy-consumption parameter, which is a critical factor in determining the overall energy efficiency of an SDWSN: for instance, in a scenario where node \(i\) forwards a packet to node \(j\). The reward received by node \(i\) for selecting node \(j\) as its relay node in terms of energy consumption is estimated by the reward function \(R\_E_{i,j}\) for the state-action pair, \((s_{i},s_{j})\) as:  
\[R\_E_{i,j}=Remaining\_Energy(s_{j})-Remaining\_Energy(s_{i}) \tag{4}\]  
where \(Remaining\_Energy(s_{j})\) represents the remaining energy of node \(i\) in state \(s_{i}\) as a percentage; meanwhile, \(Remaining\_Energy(s_{i})\) represents the remaining energy of node \(j\) in state \(s_{j}\) as a percentage. The formula calculates the difference in remaining energy between node \(i\) and \(j\) after the data transmission. A higher value of \(R\_E_{i,j}\) indicates that node \(i\) has consumed less energy in forwarding the packet to node \(j\), which is desirable to achieve the energy efficiency objective \((o_{1})\). 2. Load balance: To optimize energy efficiency and network performance in the SDWSN, the careful selection of relay nodes and balanced workload distribution are crucial. Otherwise, some nodes along overused paths may become overloaded, leading to bottlenecks and delays, which can result in a degraded network performance. For objective \(o_{2}\), we utilize the parameter-available buffer length to estimate the degree of queue congestion in relay nodes. The reward \(R\_Q_{i,j}\) for load balancing as observed by node \(i\) when selecting node \(j\) is computed as follows: \[R\_Q_{i,j}=\frac{Available\_Buffer\_Length(s_{j})}{Max\_Buffer\_Length}-\frac{Load (s_{i})}{Max\_Load}\] (5) where \(Available\_Buffer\_Length(s_{j})\) represents the available buffer length of node \(j\) in state \(s_{j}\), \(Max\_Buffer\_Length\) represents the maximum buffer length of a node, \(Load(s_{i})\) represents the current load of node \(i\) in state \(s_{i}\), and \(Max\_Load\) represents the maximum load capacity of a node. This formula considers both the available buffer length of node \(j\) and the current load of node \(i\).