## 5 Simulation Results
### Experimental Results of Phase I  
Phase, I i.e., pre-deployment phase focuses on analytical formulation of transmission strategy with mathematical calculations before the probable states as defined in Equation (6) are put into effect. The experimental setup of Phase I includes MDP formulation with respect to the input conditions. At the beginning, the state transition matrix P containing all possible transitions from the current state (\(X_{t}\)) to the probable next states (\(X_{t+1}\)) depending on actions \(a_{t}\) performed on \(X_{t}\) have been formulated. Here four actions (\(a_{t}\in\{0,1,2,3\}\)) have been taken into account which are defined in terms of data packet transmission with transmission power \(-20\) dBm, \(-15\) dBm, \(-12\) dBm, \(-10\) dBm respectively (that are standards for WBAN defined by IEEE 802.15.4 [35]) which draw the remaining energy of a node into one of the corresponding five energy levels (\(L_{t}\in\{0,1,2,3,4\}\)). Hence, probability matrix (P) for each action is constructed as \(40\times 40\) matrix. The matrix includes all possible combinations of input variables considering five energy levels \(L_{t}\), two probable values for event \(E_{t}\in\{0,1\}\), packet transmission rate \(PR_{t}\in\{0,1\}\) and link quality \(LQ_{t}\in\{0,1\}\) respectively and thus \(5\times 2\times 2\times 2\) probable states could be defined. Next, reward matrix (R) is developed by estimating rewards following Equation (9) depending on actions \(a_{t}\) performed at each system state \(X_{t}\). In this phase the mathematical calculations are carried out first in Spreadsheet following Equations (1)-(14). The formulation thus formed is solved using value iteration and finite horizon methods, that are executed using R [34] software. R [34] is a simulation tool for statistical computing that has been exploited to carry out experiments of Phase I. At this stage, a set of experiments have been done to study how discount factor (\(\gamma^{\prime}\)) puts an impact on discounted utility value with corresponding number of iterations. The value iteration process iterates until convergence to calculate the utility values for all states that got refined through approximation towards optimal value. Experiments have been performed for three representative input combinations of probability values corresponding to event generation process (\(p_{on},p_{off}\)), PR (\(pr_{on},pr_{off}\)), and link quality (\(lq_{on},lq_{off}\)). These depict the best-case, average-case and worst-casescenario (by regulating probability values) for event occurrence and successful data delivery to sink. Results are plotted in Figure 4(a),b. As found in Figure 4(a), utility value grows gradually for each combination with increasing discount factor \(\gamma^{\prime}<1\) as sooner rewards have higher utility than later ones. In addition, smaller discount factor \(\gamma^{\prime}<1\) leads to smaller horizon and hence the algorithm converges within few iterations which is reflected in Figure 4(b) as well. However, increasing discount factor \(\gamma^{\prime}\) expands the horizon and the number of iterations intensifies accordingly for each combination which gives benefit in longer run. Another experiment has been carried out at this stage before deployment of the network to get insight about the resultant utility obtained from the value iteration process [15] corresponding to each state. The experiment has been performed for two different combinations of probability values (\(p_{on},p_{off},pr_{on},pr_{off},lq_{on},lq_{off}\)) as (0.9,0.55,0.55,0.9,0.9,0.55) and (0.6,0.9,0.55,0.9,0.9,0.55). The combinations are taken such that the values of (\(p_{on},p_{off}\)) pair denote high and low event generation probability respectively (to cover the entire horizon) whereas the other pairs, i.e., (\(pr_{on},pr_{off}\)) and (\(lq_{on},lq_{off}\)) remain unchanged at their best values to provide favorable conditions for data transmission.