Heuristic creation of deep rule ensemble through iterative expansion of feature space Abstract   Rule learning approaches, which essentially aim to gerenate a decision tree or a set of xe2x80x9cifthenxe2x80x9d rules, have been popularly used in practice for automatically building rulebased models for prediction tasks, e.g., classification and regression. The key strength of rulebased models is their ability to interpret how an output is obtained given an input, in comparison with models trained by other machine learning approaches, e.g., neural networks. Moreover, ensemble learning approaches have been adopted as a popular way for advancing the performance of rulebased prediction through producing multiple rulebased models with diversity. Traditional approaches of ensemble learning are typically designed to train a single ensemble. In recent years, there have been some studies on creation of multiple ensembles towards increasing the diversity among rulebased models and the depth of ensemble learning. In this paper, we propose a feature expansion driven approach for automatic creation of deep rule ensembles, i.e., the dimensionality of the feature space is increased at each iteration by adding features newly created at the previous iteration. The proposed approach is compared with more recent approaches of rule learning and ensemble creation. The experimental results show that the proposed approach achieves improved performance on various data sets.

Random rotation ensembles In machine learning, ensemble methods combine the predictions of multiple base learners to construct more accurate aggregate predictions. Established supervised learning algorithms inject randomness into the construction of the individual base learners in an effort to promote diversity within the resulting ensembles. An undesirable side effect of this approach is that it generally also reduces the accuracy of the base learners. In this paper, we introduce a method that is simple to implement yet general and effective in improving ensemble diversity with only modest impact on the accuracy of the individual base learners. By randomly rotating the feature space prior to inducing the base learners, we achieve favorable aggregate predictions on standard data sets compared to state of the art ensemble methods, most notably for treebased ensembles, which are particularly sensitive to rotation.