## 4 Our Work
### Phase I (Pre-Deployment Phase)  
Markov Decision Process Formulation  
The optimal transmission power \(P_{tx}\) in each slot is determined following MDP [15] subject to four prime aspects, i.e., current battery level of the node, event occurrence, data PR and LQ. Consequently, at any time slot t the system state is represented by \[X_{t}=(L_{t},E_{t},PR_{t},LQ_{t}) \tag{6}\]  
where \(L_{t}\varepsilon\{0,1,2,3,4,...,N\}\) denotes the energy level of sensor node at time slot t. \(E_{t}\varepsilon\{1,0\}\) represents an event to be reported in other words whether a node has packet to transmit at time t. For instance, when a data packet is received from upper layer ready for transmission, \(E_{t}\) is 1. It is assumed that at-most one data packet is generated by each node per time slot. \(PR_{t}\varepsilon\{1,0\}\) indicates if data packet transmission rate is low or high; high packet transmission rate intensifies energy depletion rate which in turn increase SAR. \(PR_{t}\) is 1 if \(PR_{t}\ \geq\ PR_{th}\) at present slot and 0 otherwise. Here, \(PR_{th}\) is directly related to SAR threshold of a node specified by an application. Finally, \(LQ_{t}\varepsilon\{1,0\}\) denotes link quality where \(LQ_{t}\) is 1 for stable channel conditions at present slot such that  
\[\frac{\phi_{i}}{(\delta_{0}+\delta_{i})}>\frac{\phi_{i+1}}{(\delta_{0}+\delta_ {i+1})}\text{ s.t. }\ ie\{1,2,...,N\}\]  
and 0 otherwise. Here \(\delta_{0}\) quantifies the energy depleted to run the circuitry and \(\delta_{i}\) for \(0<i\leq N\) gives the amount of required energy (in addition to \(\delta_{0}\)) for data transmission with transmission power \(P_{tx}\) and \(\phi_{i}\) represents corresponding packet success rate. Energy level \(L_{t}\) changes according to the action \(a_{t}\) taken at time slot t, where \(a_{t}\varepsilon\{0,1,2,3,...,K\}\). \[L_{t} = 0\text{ if }E_{rem}(t)=\delta_{0} \tag{7}\] \[= i\text{ if }E_{rem}(t)=\delta_{0}+\delta_{i}\text{ where }1\leq i\leq N\]  
Following MDP, an action \(a_{t}\) is performed at each state \(X_{t}\) that takes the system to the next state \(X_{t+1}\) while resulting in a reward \(R(X_{t},a_{t})\) as shown in Figure 2. Solving the MDP formulation gives us the actions to be performed in each iteration that results in maximum reward. It is to be noted that calculation of both next state and reward only depend on the current state and the action taken at that state. Thus, at any time slot \(t\) reward (\(R\)) is quantified as the probabilities of successful data delivery subject to the input conditions, i.e., possible combination of energy level, event occurrence, PR and channel conditions. \[R(X_{t},a_{t})=\left\{ \begin{array}{ll}(1-p_{off})\times pr_{off}\times(1- lq_{off}),&\text{ if }E_{t-1}=0\text{ \&}PR_{t-1}=0\text{ \&}\\ &LQ_{t-1}=0\text{ \&}L_{t}>\delta_{0}+\delta_{i}\\ (1-p_{off})\times pr_{off}\times lq_{on},&\text{ if }E_{t-1}=0\text{ \&}PR_{t-1}=0\text{ \&}\\ &LQ_{t-1}=1\text{ \&}L_{t}>\delta_{0}+\delta_{i}\\ (1-p_{off})\times(1-pr_{on})\times(1-lq_{off}),&\text{ if }E_{t-1}=0\text{ \&}PR_{t-1}=1\text{ \&}\\ &LQ_{t-1}=0\text{ \&}L_{t}>\delta_{0}+\delta_{i}\\ (1-p_{off})\times(1-pr_{on})\times lq_{on},&\text{ if }E_{t-1}=0\text{ \&}PR_{t-1}=1\text{ \&}\\ &LQ_{t-1}=1\text{ \&}L_{t}>\delta_{0}+\delta_{i}\\ p_{on}\times pr_{off}\times(1-lq_{off}),&\text{ if }E_{t-1}=1\text{ \&}PR_{t-1}=0\text{ \&}\\ &LQ_{t-1}=1\text{ \&}L_{t}>\delta_{0}+\delta_{i}\\ p_{on}\times(1-pr_{on})\times lq_{on},&\text{ if }E_{t-1}=1\text{ \&}PR_{t-1}=1\text{ \&}\\ &LQ_{t-1}=0\text{ \&}L_{t}>\delta_{0}+\delta_{i}\\ p_{on}\times(1-pr_{on})\times lq_{on},&\text{ if }E_{t-1}=1\text{ \&}PR_{t-1}=1\text{ \&}\\ &LQ_{t-1}=1\text{ \&}L_{t}>\delta_{0}+\delta_{i}\\ p_{on}\times(1-pr_{on})\times(1-lq_{off}),&\text{ if }E_{t-1}=0\text{ \&}L_{t}>\delta_{0}+\delta_{i}\\ p_{on}\times(1-pr_{on})\times lq_{on},&\text{ if }E_{t-1}=1\text{ \&}PR_{t-1}=1\text{ \&}\\ &LQ_{t-1}=1\text{ \&}L_{t}>\delta_{0}+\delta_{i}\\ 0,&\text{ Otherwise}\end{array} \right. \tag{8}\]  
(i.e., \(\epsilon\) convergence).