## 4 Our Work
### Phase II (Post-Deployment Phase)  
In Phase II, the obtained policy of Phase I (denoted as \(\pi^{*}\) in Figure 4) is incorporated into each node before deployment to initiate Phase II which focuses on developing network layer protocol. Thus, routing decisions could be made based on this pre-calculated policy to get long-term benefit in terms of energy consumption, successful data delivery and minimal thermal effect as well. This policy can be stored as a data structure where the optimal transmission power corresponding to each system state (\(X_{t}\)) would be listed. The optimal transmission strategy thus obtained can be fed to each node in WBAN as summarized in the routing protocol illustrated in Algorithm 1. The nodes when deployed, may tune themselves to the optimal transmission power according to their working conditions by simply looking into the data structure (\(DS_{j}\)). However, each node tries to establish connection with sink at the suggested optimal transmission power following MDP formulation for present system state. If it succeeds, data is transferred to the sink directly using the specified transmission power level. However, if sink is found to be not reachable at the suggested power level, it looks for relay nodes which are reachable with the same power level. \(p_{on}\) and \(p_{off}\) are mapped in Algorithm 1 using \(flag_{on}\), \(flag_{off}\). \(flag_{on}\) is true to indicate data is sent in the previous time slot and data will be transmitted in the current slot with probability \(p_{on}\); similarly \(flag_{off}\) is true when there is no data transmission in previous slot and data will be transmitted in the next slot with probability \((1-p_{off})\). \begin{table}
\begin{tabular}{l l} \hline \hline
**Terms** & **Description** \\ \hline \(X_{t}\) & A finite set of states \\ \(A_{t}\) & A finite set of actions \((a_{t})\) to be taken \\  & Transition probability matrix, where the state transitions are given by \\  & \(p(x_{(t+1)}\mid x_{t},a_{t})=p(x_{(t+1)}|x_{0}...x_{t},a_{0}...a_{t})\). This matrix plays the key role in finding the next \\ \(P\) & state \(x_{(t+1)}\) which is considered to be a possible consequence of performing an action \((a_{t})\) in a state \((x_{t})\). Hence it is depicted as a set of square matrices one for each action having both \\  & dimensions indexed by states. \\ \(R\) & Reward matrix where each entry gives the immediate reward (or expected immediate reward) \\  & \(r(x_{t},a_{t})\) received for state transition from \(x_{t}\) to \(x_{(t+1)}\) performing action \(a_{t}\). \\ \(\gamma^{\prime}\) & [0,1] Discount factor denoting the importance of future reward in present reward. \\ \(\Pi(x)\) & A policy \(\Pi\) gives an action for each state x, \(\Pi^{*}(x)\) is optimal policy, i.e., the sequence of \\ \(U_{dis}(x)\) & Expected discounted resultant utility value at each state obtained using value iteration process \\ \(P_{tx}\) & Transmission power \\ \(p_{on}\) & [0.5,1]Probability of occurrence of an event in next slot when there is an event in present slot \\ \(p_{off}\) & [0.5,1]Probability of occurrence of no event in next slot when there is no event in present slot \\ \(p_{on}\) & [0.5,1]Probability of exceeding maximum limit of packet transmission rate, \(PR_{th}\) in next slot \\  & when \(PR_{th}\) exceeded in current slot \\ \(p_{off}\) & [0.5,1]Probability of not exceeding \(PR_{th}\) in next slot when \(PR_{th}\) not exceeded in current slot \\ \(l_{qu}\) & [0.5,1]Probability that indicates stable channel condition in next slot when link quality is above \\  & threshold (\(LQ\ >\ LQ_{th}\)) in present slot \\ \(l_{qoff}\) & [0.5,1]Probability that indicates unstable channel condition in next slot when link quality is \\  & below threshold (\(LQ\ <\ LQ_{th}\)) in present slot \\ \(E_{rem}\) & Remaining energy of a node \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Description of frequently used terms.