EDiT: Interpreting Ensemble Models via Compact Soft Decision Trees Given featurebased data, how can we accurately classify individual input and interpret the result of it? Ensemble models are often the best choice in terms of accuracy when dealing with featurebased datasets. However, interpreting the decision made by the ensemble model for individual input seems intractable. On the other hand, decision trees, although being prone to overfit, are considered as the most interpretable in terms of being able to trace the decision process of individual input. In this work, we propose Ensemble to Distilled Tree (EDiT), a novel distilling method that generates compact soft decision trees from ensemble models. EDiT exploits the interpretability of a treebased structure by removing redundant branches and learning sparse weights, while enhancing accuracy by distilling the knowledge of ensemble models such as random forests (RF). Our experiments on eight datasets show that EDiT reduces the number of parameters of an RF by 6.4 to 498.4 times with a minor loss of classification accuracy.

Heuristic creation of deep rule ensemble through iterative expansion of feature space Abstract   Rule learning approaches, which essentially aim to gerenate a decision tree or a set of xe2x80x9cifthenxe2x80x9d rules, have been popularly used in practice for automatically building rulebased models for prediction tasks, e.g., classification and regression. The key strength of rulebased models is their ability to interpret how an output is obtained given an input, in comparison with models trained by other machine learning approaches, e.g., neural networks. Moreover, ensemble learning approaches have been adopted as a popular way for advancing the performance of rulebased prediction through producing multiple rulebased models with diversity. Traditional approaches of ensemble learning are typically designed to train a single ensemble. In recent years, there have been some studies on creation of multiple ensembles towards increasing the diversity among rulebased models and the depth of ensemble learning. In this paper, we propose a feature expansion driven approach for automatic creation of deep rule ensembles, i.e., the dimensionality of the feature space is increased at each iteration by adding features newly created at the previous iteration. The proposed approach is compared with more recent approaches of rule learning and ensemble creation. The experimental results show that the proposed approach achieves improved performance on various data sets.