## 4 Experimental Evaluations and Results
### Attributive Explanations with Beerl  
Our method leverages the transparency of the last layers by implementing the reward decomposition method and separating the reward weight so that the reward components can be compared in a fair manner. This approach benefits in two aspects, which are as follows: (1) each reward component is trained in the same range, which minimizes the explosion or vanishing gradient; (2) the effect of reward prioritization is exposed so it helps humans understand the identification of unwanted behavior. Figure 5 shows the explanation on the Q-function level, i.e., how the Q-function (which is implemented in a neural network model) performs regardless of the priority of the reward component. From Figure 5a, we can see that the RSRP, SINR, and throughput input features have dominant importance to their respective reward components in comparison to other input features. Similarly, the respective input feature becomes the most important for each reward component (other input features do not outweigh their importance in every reward component), as shown in Figure 5b. When NN is employed for the RL agent, it is desirable that the Q-functions work in a normalized range to avoid the exploding or vanishing gradient phenomenon. Explanations at this level show the behavior of NN in approximating Q-values and the fairness of it (i.e., when all components are treated/trained in equal numeric values). Ideally, each feature has a significant contribution to at least one reward component. It is a waste of resources when an input feature is used, but it does not have any impact on the output. Similarly, each reward component should have at least an input feature that significantly affects it. The explanation at the RL agent level shows the correlation between the input and output explanations, where the priority/weights of the reward components are considered. In this manner, we can clearly see which aspect affects the final action of the agent (e.g., whether the neural network as the Q-function, the reward prioritization, or the combination of them). When reward weights are applied and the RSRP reward component is prioritized more than others (\(2.0:1.0:1.0\) for \(k_{RSRP}:k_{SINR}:k_{Throughput}\)), we can see that each input feature still dominates its importance for the respective reward component and vice versa, as shown in Figure 6. Unlike Q-function explanations, an expert may accept RL explanations where a reward component does not contribute significantly to the total reward. The contribution of the reward component can be adjusted using reward  
Figure 4: Explanations generated using the available XAI methods. (**a**) Feature importance generated using [6], which produces similar explanation format as [4,5]. (**b**) Reward components summary generated using [8]. Figure 5: Explanations generated using BEERL at the Q-functions level without applying rewards prioritization. (**a**) Decomposed feature importance at the Q-function level. (**b**) Decomposed feature importance presented per reward component at the Q-function level. prioriitization, and the proportion of reward prioritization can be justified by the domain knowledge expert. The detailed feature importance is shown in Figures 5 and 6 without comparing the importance between the input features and the reward components. Figure 7 shows the importance of them in the same plot so that they can be compared which is also known as a normalized explanation (\(L_{ic}\)). We can see from Figure 7 that the RSRP input feature and the reward component become the top-two most important element in this agent. When reward weights are applied, their importances are pushed further, as shown in Figure 7b. In a local scope (i.e., explanation of a single action), the detailed information produced by our method exposes the contribution that has not been seen in existing methods.