# Iterative Oblique Decision Trees Deliver Explainable RL Models
## 5 Discussion
We investigated another possible variant of an iterative algorithm. In this variant, observations were sampled while the oracle was training (and probably also visited unfavorable regions of the observation space). After the oracle had completed its training, the samples were labeled with the action predictions of the fully trained oracle. These labeled samples were presented to DTs in a similar iterative algorithm as described above. However, this algorithm was not successful. 4. A striking feature of our results is that DTs (predominantly those trained with ITER) can outperform the oracle they were trained from. This seems paradoxical at first glance, but the decision space analysis for MountainCar in Section 4.3 has shown the likely reason. In Figure 5, the decision spaces of the oracles are overly complex (similar to overfitted classification boundaries, although _overfitted_ is not the proper term in the RL context). With their significantly reduced degrees of freedom, the DTs provide simpler, better generalizing models. Since they do not follow the \"mistakes\" of the overly complex DRL models, they exhibit a better performance. It fits to this picture that the MountainCarContinuous results in Figure 2 are best at \(d=1\) (better than oracle) and slowly degrade towards the oracle performance as \(d\) increases; the DT obtains more degrees of freedom and mimics the (slightly) non-optimal oracle better and better. The fact that DTs can be better than the DRL model they were trained on is compatible with the reasoning of [PERSON] [20], who stated that transparent models are not only easier to explain but often also outperform black box models. 5. We applied algorithm ITER to OPCTs. Can CARTs also benefit from ITER? The _solved_-depths shown in column ITER + CART of Table 2a add up to 35\({}^{+}\), which is nearly as bad as EPS + CART (38\({}^{+}\)). Only the _solved_-depth for Pendulum improved somewhat from 10\({}^{+}\) to 7. We conclude that the expressive power of OPCTs is important for being successful in creating shallow DTs with ITER. 6. What are the limitations of our algorithms? In principle, the three algorithms EPS, BB, and ITER are widely applicable since they are model-agnostic concerning the DRL and DT methods used and applicable to any RL environment. However, the limitation of the current work is that, so far, we have only tested environments with one-dimensional action spaces (discrete or continuous). There is no principled reason why the algorithms should not be applicable to multi-dimensional action spaces (OPCTs can be trained for multi-dimensional outputs as well). Still, the empirical demonstration of successful operation in multi-dimensional action spaces is a topic for future work. One obvious limitation of the three algorithms is that they need a working oracle. A less obvious limitation is that algorithm EPS only needs static, pre-collected samples from oracle episodes (in other applications, these might be \"historical\" samples). In contrast, the algorithms BB and ITER really need an _oracle function_ to query for the right action at arbitrary points in the observation space. Which points are needed is not known until the algorithms BB and ITER are run. 7. A last point worth mentioning is the intrinsic trustworthiness of DTs. They partition the observation space in a finite (and often small) set of regions, where the decision is the same for all points. (This feature includes smooth extrapolation to yet-unknown regions.) DRL models, on the other hand, may have arbitrarily complex decision spaces. If such a model predicts action \(a\) for point \(x\), it is not known which points in the neighborhood of \(x\) will have the same action.