# Resource Allocation on Blockchain Enabled Mobile Edge Computing System
## 4 Simulation and Analysis
### Impacts of Weight Coefficients  
Specifically, the impact to the reinforcement learning model by changing the ratio of \(\alpha\), \(\zeta\), \(\beta\), and \(\gamma\) is measured. In the experiment, eight groups of parameters are tested to determine the exact impact of the ratio mentioned above on the convergence, latency, cost and bonus of the model. Other parameters remain the same. We set the training epoch as 2000. There are 100 steps of interaction in each epoch. We set learning rate as \(10^{-4}\), discount factor as 0.99 and the arrival rate  
is closely related to the ratio of \(\alpha\) and \(\beta\). When \(\alpha/\beta>baseline\), the policy tends to lower the latency, when \(\alpha/\beta<baseline\), it tends to lower the cost. Furthermore, from the comparison of group 6 and 7, it can tell that if \(\alpha/\beta\) is becoming smaller and smaller, it can help the algorithm to converge faster. This is because when the agent is observing the environment, it gonna has a certain random magnitude, if the increase in reward brought by choosing a different policy is small compared to the change brought by random search, then the algorithm will not continue to converge. Then some possible impacts that \(\gamma\) may have toward the policy of reinforcement learning model are measured. \(\gamma\) controls \(bonus\), when \(bonus\) increases, the \(t_{mining}\) decreases. That means as long as the policy tends to lower \(t_{mining}\) a bit more than \(cost\), when \(\gamma\times\zeta>0\), the change of \(\gamma\) would not have a great impact on the policy. For example, in group 1: \([\alpha,\zeta,\beta,\gamma]=[100,2,1,20]\), the policy is more inclined to reduce \(t_{mining}\). In this case, no matter how the value of \(\gamma\) changes, there will be no difference to the policy. In order to testify that assumption, a comparative experiment is made: group 1, 4, and 5. In these three groups, the rewards have the same growth pattern. Moreover, relying on the \(cost\) and \(t_{mining}\), we can figure out that the changing of \(\gamma\) has a little impact on the policy of  
Figure 5: Average reward per epoch. Figure 6: The convergence of reinforcement learning; (**a**) Average process delay per epoch; (**b**) average mining delay per epoch; (**c**) average cost per epoch; (**d**) average bonus per epoch. the reinforcement learning model. Their \(cost\) and \(t_{mining}\) converge at the same results. That presents group 1, 4, and 5, which choose nearly the same policy. In group 1, especially, we set \(\gamma\) as 20, the result remains almost unaffected. Here a further analysis is given towards the experiment results above. The emergence of \(baseline\) is within the prediction. It is obvious that \(t_{process}\), \(t_{mining}\) and \(cost\) is correlative, we set the correlation \(f(t_{process},t_{mining},cost)=0\). It can extend to \(f(t_{process}-\Delta t_{1},t_{mining}-\Delta t_{2},cost+\Delta cost)=0\) (Servers with lower computing capabilities are usually cheaper. Servers with higher computing capabilities can lower the latency. That is to say the cost usually increases when the latency is lower). Thus, in a minimum neighborhood, \(\Delta t_{1}\), \(\Delta t_{2}\) and \(\Delta cost\) have a positive correlation as \(\Delta cost\approx k\Delta t_{1}+p\Delta t_{2}\), \(k\), \(p\) is positive. Hence, the change of reward can be described in this neighborhood as: \(\Delta R=-\alpha\cdot(-\Delta t_{1})-\zeta\cdot(-\Delta t_{2})-\beta\cdot( \Delta cost)=(\alpha-\beta k)\Delta t_{1}+(\zeta-\beta p)\Delta t_{2}\). When \(\beta\) is relatively large, \(\Delta R<0\) is minus, which means the policy would lower the \(cost\). When \(\alpha\) is relatively larger, \(\Delta R>0\) is positive, which means the policy would tend to lower the latency. Moreover, from Figure 6a,b, we can tell that the order of magnitude of process delay and mining delay is different.