## 4 Proposed Work
### Coverage Hole Detection and Repair
Hence, the lifetime of the node is computed as,  
\[Lifetime\ of\ node(bn_{m})=\frac{E_{0}}{e_{m}} \tag{21}\]  
The coverage of the node is defined as the area up to which the sensor can sense the data packets and can be calculated as,  
\[Coverage\ (C)=\sum_{i=1}^{n}Area(bn_{m}) \tag{22}\]  
Algorithm 3 explains the process for the optimal mobile node selection is described above using the calculation of necessary factors that affect the mobile node selection. Figure 3 depicts the coverage hole detection and repair process. ```
Input: coverage hole location, mobile nodes around coverage hole Output: optimal node for replacement For nodes near coverage hole do  Calculate the distance using Equation (11);  Select the nearest nodes having less distance;  Calculate the direction for those nodes and save in list L1;  End for  For nodes moving towards hole in L1 do;  Calculate the lifetime of the node using \(E_{bn}(t+1)\);  Select the node with longer lifetime;  Calculate the coverage of those nodes and save in list L2;  End for  Return optimal node
```  
**Algorithm 3** Optimal node selection Let \(\zeta\) denote the state at time stamp t and \(\alpha\) represent the set of actions that each SARSA agent takes in the respective state. During coverage hole repair, the location and sensing of the optimal mobile node is changed which is considered as the change in action. Then the possible action is represented as,  
\[\alpha_{i}=(l_{i},r_{i}) \tag{23}\]  
In which \(l_{i}=(l_{x_{i}},l_{y_{i}})\) is the present position of the mobile node and its future position is represented as \(l^{\prime}_{i}=\big{(}l^{\prime}{}_{x_{i}},l^{\prime}{}_{y_{i}})\). \(r_{i}\) and \(r^{\prime}{}_{i}\) denote the current and future sensing range of the mobile nodes, respectively. \(\mathcal{R}\) is the reward set comprising of rewards \(\gamma\) obtained for the action taken by the agent. Hence, the goal of the agent is to obtain high reward which can be expressed as,  
\[\mathcal{R}_{t}=\sum_{j=0}^{\infty}\delta^{k}\gamma_{t+1+k} \tag{24}\]  
The Q function is represented as,  
\[Q^{\pi}(\zeta,\alpha)=E\{\mathcal{R}_{t}|\zeta_{t}=\zeta,\ \alpha_{t}=\alpha,\pi\} \tag{25}\]  
And the optimal Q function can be formulated as  
\[Q^{*}(\zeta,\alpha)=\underset{\pi}{\max}\
u^{\pi}(\zeta,\alpha) \tag{26}\]  
From the optimal \(Q^{*}\) function, the optimal policy can be determined by,  
\[\pi^{*}=\underset{\pi}{\arg\max}Q^{*}(\zeta,\alpha) \tag{27}\]  
The rule for updating the Q function for the next state \(\zeta^{\prime}\) based on the reward obtained for the action carried out in previous state \(\zeta\) is expressed as,  
\[Q(\zeta,\alpha)\leftarrow\big{(}1-\theta_{\zeta,\alpha}\big{)}Q(\zeta,\alpha )+\theta_{\zeta,\alpha}(\gamma+\underset{\alpha^{\prime}}{\max}Q\big{(}\zeta^{ \prime},\alpha^{\prime})\big{)} \tag{28}\]  
The selected mobile node is positioned at the optimal position to repair the hole. Thus, we repair the coverage hole with the minimum number of mobiles nodes without increasing in complexity. Algorithm 4 explains the process for the MA-SARSA is presented below in which the Q function is initiated arbitrarily for which the state and action is initiated and in each trial step the reward is obtained. This process is repeated until the coverage hole is repaired. Figure 4 depicts the overall flowchart of the proposed MiA-CODER approach. Figure 3: Coverage hole detection and repair.