# OF-AE: Oblique Forest AutoEncoders

Cristian Daniel Alecsa

► To cite this version:

Cristian Daniel Alecsa. OF-AE: Oblique Forest AutoEncoders. 19th IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI), Jun 2023, León, Spain. pp.207-219, 10.1007/978-3-031-34107-6_17. hal-05498062

HAL Id: hal-05498062

https://inria.hal.science/hal-05498062v1

Submitted on 6 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# OF-AE: Oblique Forest AutoEncoders

Cristian Daniel Alecsa<fnref n="1" />,<fnref n="2" />(✉) ID

**Abstract.** We propose an unsupervised ensemble method consisting of oblique trees that can address the task of auto-encoding, which is an extension of the *eForest* encoder introduced in [14]. By employing oblique splits, we will devise an auto-encoder method through the computation of a sparse solution of a set of linear inequalities consisting of feature values constraints. The code for reproducing our results is available at https://github.com/CDAlecsa/Oblique-Forest-AutoEncoders.

**Keywords:** Autoencoder · Oblique Decision Tree · Random Forest · Image reconstruction · Optimization problems

## 1 Introduction

The original CART algorithm [10] partitions the feature space using axis-parallel splits. The training of a classical *Decision Tree* $T$ (denoted as DT) relies on greedy optimization, i.e. the root of the tree is the whole input space $X$ which is split into two disjoint regions, and this process continues in a recursive manner. There are numerous extensions to classical CART methods. As an example, the extremely randomized trees (briefly ERT) [16] is implemented in *SKLearn* as *ExtraTree Regressor* and induces randomization into the optimization of the trees by selecting the threshold fully at random for each candidate feature.

A popular extension of CART is the *Random Forest* algorithm, briefly RF, introduced by Leo Breiman in [9] (see also [8]). The RF method consists of different randomized decision trees $T_1, \dots, T_N$ which are trained independently and which are finally aggregated all together with the average of the underlying individual scores. CART and RF methods can also be perceived as clustering methods. Some examples are the works of Lin and Jeon [21] which shows a connection between RF and nearest neighbor predictors, and the Extremely Randomized Clustering Forest (ERCForest) of Moosmann et. al. [24] (see also [22,25]) which is an ensemble of clustering trees based on spatial partitioning.

<footnote n="1">¹ Romanian Institute of Science and Technology, Cluj-Napoca, Romania
alecsa@rist.ro</footnote>

<footnote n="2">² Technical University of Cluj-Napoca, Cluj-Napoca, Romania</footnote>

<footnote>This work was supported by a grant of the Romanian Ministry of Research and Innovation, CCCDI – UEFISCDI, project number 178PCE/2021, PN-III-P4-ID-PCE-2020-0788, *Object Perception and Reconstruction with deep neural Architectures (OPERA)*, within PNCDI III.</footnote>
A different extension to classical CART methods represent the so-called *Soft Decision Trees* (SDT) considered in [15, 18] that can be considered as the probabilistic variants of CART, which are based upon the idea of splitting the parameters of the tree via gradient descent-type methods. A similar idea was developed for the so-called *Probabilistic Random Forest* (PRF) introduced in [30] where the underlying algorithm takes into account the uncertainty in both features and labels. Some extensions are represented by the *Deep Neural Decision Forest* (DNDF) introduced in [20] where each node is represented by a neural network layer and where the backpropagation algorithm is used in the training process, and the *Adaptive Neural Trees* (ANT) from the work [32]. These types of trees are optimized with respect to a global loss function and the nodes in the DNDF and ANT uses the same input *x* belonging to the dataset. At the same time, SDT and its variants rely on hierarchical decisions, instead of hierarchical features.

## 2 Related Work

An improvement over CART are the *Oblique Decision Trees* (briefly ODT) which partitions the input space $\mathcal{X}$ using multivariate tests. These trees test at each internal node several attributes by utilising linear combinations of features, and it is well known that training ODT is in general associated with a high training run-time cost (see Murthy et. al. [26]).

The pioneering work on oblique trees began with the work of Bremain et. al. [10] which introduced the so-called *Classification and Regression Trees - Linear Combination*, in a nutshell CART-LC. There are multiple extensions of the classical Oblique Tree CART-LC. One of them is the recent CO2 algorithm (*Continuous Optimization of Oblique Splits*) from [28], which focuses on optimizing an objective function through gradient-based methods. A different methodology is considered in [33] where the authors introduced the so-called *HHCART* oblique decision method. The underlying approach is that the original space where the dataset belongs to is transformed into a feature space. Then, one finds an axis-parallel split in the feature space which corresponds to a multivariate oblique split in the original space. The reflected training samples are given by $\hat{\mathcal{D}} = \mathcal{D}\mathcal{H}$, where $\mathcal{D}$ represents the original training data and the Householder matrix is $\mathcal{H} = \mathcal{I} - 2uu^T$, where $u = \frac{e-d}{\|e-d\|_2}$. Here, $d$ represents the dominant eigenvector of the estimated covariance matrix corresponding of a class (suggesting the orientation of that class), and $e$ is the standard basis vector corresponding to a chosen feature. Since the Householder matrix $\mathcal{H}$ is symmetric and orthogonal, a sample point in the transformed space can be mapped with a minimal cost to the original space, due to the fact that $\mathcal{H}\mathcal{H} = \mathcal{I}$. The last Oblique Tree we recall is the *RandCART* algorithm (see [6]) where, as in the case of *HHCART*, an oblique decision split in the original space corresponds to an axis-parallel split in a feature transformed space. The training samples $\mathcal{D}$ are converted into a new feature space $\mathcal{D}'$ of the same dimension as $\mathcal{D}$, for which one generates randomly and independently $p^2$ terms $m_{ij} \sim \mathcal{N}(0, 1)$ which defines a square matrix $\mathcal{M}$,
the latter being decomposed using QR decomposition as $M = QR$, where $Q$ is orthogonal and $R$ is upper triangular. After that, the new feature space is obtained as $\hat{D} = DQ$.

We end the present section by turning our focus to the idea of autoencoders which dates back to [7]. Despite the fact that most autoencoder models are, in general, neural networks (as an example, see [3,19]), only recently autoencoders based upon DT gained significant attention from the Machine Learning community. Irsoy and Alpaydin in [17] introduced an SDT-type method where a sigmoid gating function is used at every internal node, and where the final structure is based on stacking two SDTs with a layer-by-layer training. On the other hand, the interpretable autoencoder introduced in the paper of Aguilar et. al. [2] is based on multiple DT, where the $i^{th}$ tree is trained with all of the features except the $i^{th}$ attribute where this attribute is considered the target class. A different type of model are the Generative Trees (GT) recently introduced in [27] which are generative-like autoencoder models. The last model that we recall is the eForest encoder method from [14] in which the encoding is done by making the input samples go through all of the RF estimators in order to find the path from each tree where every sample goes to. Then, for each sample which will be decoded, it builds up the estimator spaces which represents the restriction on all the features. For decoding, the eForest encoder gathers up the attributes restrictions from all the subspaces and then uses the Maximum Compatibility Rule (MCR) in order to find the lower and upper values of the features.

# 3 Proposed Methodology

In this section we shall present the technique that will aid us extending the eForest encoder to ODT. In what follows, we shall employ different types of ODT coupled simultaneously into a Bagging Regressor which will form an unsupervised encoding-decoding method that will be called **Oblique Forest AutoEncoder**, namely **OF-AE**. For the eForest, the decoder part consists in taking, for a given sample, the feature restrictions from the right branches of the trees paths, while the final application of the MCR is related to taking the maximum bound of the feature values from these features subspaces. On the other hand, for our ODT we can gather, for a given sample which need to be decoded, all of the feature restrictions from the underlying path (by taking into account also the sign of the branches) and form an optimization problem with constraints. Additionally, for the image reconstruction cases we shall add auxilliary constraints such that the feature values belong to the closed interval $[0, 255]$, and we will employ the same technique as in eForest by training a different encoder-decoder pair for every channel. In what follows we shall propose our encoding-decoding method for a given ensemble of ODT $T_1, \dots, T_N$. For every tree of this ensemble that is already trained, the forward encoding process consists in sending an input vector through each individual tree and to retain the weights and the thresholds for every internal node from the path the samples passes through. For some index $i \in \{1, \dots, N\}$, we consider a tree $T_i$ (depicted in Fig. 1). For a sample $x \in \mathcal{X}$, the path traversed by the input vector $x$ is
Now, if we put together all of the sample paths of $x$ from the tree ensemble, and denoting $p$ as the number of features, we obtain the system $Ax \ge b$, with

$$b = \begin{bmatrix} \operatorname{sign}(k_1^1)b_{k_1}^1 \\ \vdots \\ \operatorname{sign}(k_{m_1}^1)b_{k_{m_1}}^1 \\ \vdots \\ \operatorname{sign}(k_1^N)b_{k_1}^N \\ \vdots \\ \operatorname{sign}(k_{m_N}^N)b_{k_{m_N}}^N \end{bmatrix} \quad \text{and} \quad A = \begin{bmatrix} \operatorname{sign}(k_1^1)w_{k_1}^1 \\ \vdots \\ \operatorname{sign}(k_{m_1}^1)w_{k_{m_1}}^1 \\ \vdots \\ \operatorname{sign}(k_1^N)w_{k_1}^N \\ \vdots \\ \operatorname{sign}(k_{m_N}^N)w_{k_{m_N}}^N \end{bmatrix},$$

where $b \in \mathbb{R}^{m_1 \times m_2 \times \dots \times m_N}$ and $A \in \mathbb{R}^{m_1 \times m_2 \times \dots \times m_N \times p}$. Furthermore, for the decoding of a sample $x \in \mathcal{X}$ we consider the optimization problem

$$\begin{cases} \text{minimize} & \|x\|_1 \\ \text{subject to} & Ax \ge b \end{cases}$$

For our codes we made the computing of a sparse solution of the set of the linear inequalities given by the weights and thresholds with the help of the CVXPY package ([1,12]). Also, we have considered only the HHCART and the RandCART as the choices of ODT since they are faster than other types of ODT. Furthermore, we have implemented a unsupervised version of the Bagging Regressor from SKLearn ([29]) by employing another technique from the SKLearn's implementation of RandomTreesEmbedding (which was used in the original version of the eForest encoder) where a one-dimensional target is uniformly generated for the tree ensemble. Our codes are based upon the implementations from [13] and https://github.com/valevalerio/Ensemble_Of_Oblique_Decision_Trees related to [23] (where HHCART was adapted for regression problems using MSE criterion). The original HHCART implementations use the PCA method for transforming the original space to the feature space, but we have utilized the PCA method along with other alternatives: Truncated-SVD, FastICA and Gaussian Random Projection (briefly GRP), with a one-dimensional feature space.

# 4 Experiments

In this section we shall present the results of our approach for various tabular and image datasets.

## 4.1 Datasets and Experimental Setup

Concerning tabular datasets, we consider the following: from SKLearn we will employ the Diabetes dataset (442 samples / 10 features), and from the UCI ML repository we shall use Seeds (210 samples / 7 features) and HTRU2 (17898 samples / 8 features), and also Compas (6172 samples /
13 features) from https://github.com/tunguz/TabularBenchmarks/tree/main/datasets, respectively. On the other hand, for image-related datasets, we will consider as baseline datasets *MNIST* (60000 samples / 28 × 28) and *CIFAR10* (60000 samples / 32 × 32), and the *Oxford Flowers* dataset (8189 samples / various sizes) from https://www.robots.ox.ac.uk/~vgg/data/flowers/102/.

Furthermore, we will also utilize the image dataset used in [31] (which we will succinctly call it *CHD2R - Cultural Heritage Dataset for Digital Reconstruction*) which is comprised of cultural heritage assets and textile artefacts with traditional motifs (732 samples / 256 × 256). For RGB-type images, similar to *eForest* encoder, we will consider for each channel a different **OF-AE** model and then stack the results in order to fully decode an image.

In what follows, the parameters which will be used in our experiments are the following: *max_features* denotes the number of features drawn from the training dataset that are used to train each estimator, *max Samples* is the number of samples which are drawn with replacement from the same training dataset for each base estimator, *max_depth* will denote the maximum depth of each tree, *n_estimators* are the number of estimators (represented as oblique trees) which form the entire ensemble. For the datasets where we utilize the train-test splitting procedure, *test_size* represents the size percentage of the test dataset, while *n_train* and *n_test* will denote the explicit train and test number of samples if these integer values are given.

## 4.2 Tabular Data Reconstruction

In the present subsection we shall consider some basic experiments for the reconstruction of features from various tabular datasets. Our first experiment is related to the *Diabetes* dataset for which we have set *test_size* to 0.25, *n_estimators* to 200, *max_features* to 0.75, *max Samples* 0.5, and *max_depth* is set to 3.

From Fig. (2a) one observes that we obtain almost perfect reconstruction for the chosen numerical features even though we have chosen a moderate number of estimators and a low maximum depth value. Our second experiment relies on the reconstruction of some categorical features from the *Compas* dataset. We have considered the same parameters as in the previous experiment, but the train and test datasets were given independently. The results presented in Fig. (2b) reveals the fact that even categorical features can be reconstructed with the appropriate number of estimators. In the previous experiments, we have used the *HHCART* method. For the *Compas* dataset, we have utilized the *PCA* method, while for the feature transformation for *HHCART* on *Diabetes* dataset we used the *GRP*.

Now we will analyze the impact of various parameters of the oblique trees ensemble. In Fig. 3 we have the variation of the *max_depth*, *max_features*, *max Samples* and *n_train* parameters with respect to the *MSE* values of the decoded test samples reconstruction on the *Diabetes* dataset. Despite the fact that we have not chosen the best values that will give us the lowest *MSE*, Fig. 3 highlights the importance of the parameters underpinning these methods. For the plot concerning the *max_depth* parameter, we have chosen (*n_estimators*, *max Samples*, *max_features*) = (50, 0.5, 0.75), while for the
<figure><img src="image_3.png" /><figcaption>Fig. 2. Feature reconstruction</figcaption></figure>

max_features plot, we have taken (n_estimators, max Samples, max_depth) = (100, 0.5, 3). For max Samples plot we chose (n_estimators, max_features, max_depth) equal to (100, 0.75, 3), while for the n_train plot we used the fixed parameters (n_estimators, max Samples, max_features, max_depth) = (50, 0.5, 0.75, 3).

<figure><img src="image_4.png" /><figcaption>Fig. 3. Ablation study for the OF-AE method on Diabetes dataset</figcaption></figure>

For each plot we have made 10 simulation runs, while the test_size was set to 0.25, and the GRP was used for HHCART.

From Fig. 3 one can easily see that the maximum depth has little influence on the reconstruction error in the situation when the number of estimators is set to a moderate value of 50. Moreover, the maximum number of features seems to slightly increase the MSE values. An explanation for this may be that a large
percentage of the maximum features used reduces the inherent randomization of the ensemble methods, but one can't exclude the possibility that these results may appear due to the fact that the number of simulation runs is not large enough. The most decisive parameters are *max*samples and *n*<sub>train</sub>, namely the *MSE* values are monotonically decreasing when the maximum number of samples or the number of training data points are increased.

For the next experiment, we have used the *Seeds* dataset in order to compare the train and decode time values of the *HHCART* and *RandCART* oblique-type trees, respectively. Here, *n_estimators* are set to 200, *max*samples to 0.5, *max_features* to 0.75 and *max_depth* to 3. For the *Seeds* dataset (where *test_size* was set to 0.25) the high number of trees is utilized in order to emphasize the comparison of the running time of different types of ODT. In Fig. (4a) the line of the categories presented in the barplots represent the variation of 10 simulation runs. We observe that *HHCART* is slower than *RandCART* at training and also at decoding. The major difference lies in the training time where there exists a large variation for the running time of *HHCART*. As noticed before, there are some differences between the fit time of the *HHCART* method in comparison with the training time of the *RandCART* method. Despite this fact, we are interested not only in how fast our ensemble methods are, but how precise they are with respect to reconstruction errors. We investigate the differences in the *MSE* values for the aforementioned oblique-type methods. In Fig. (4b) we have used the *HTRU2* dataset and we considered again 10 runs, 50 sample points for training, 10 test sample points for decoding, along with the following choices: (*max*samples, *max_features*, *max_depth*) = (0.5, 0.75, 3). Despite the fact that the *MSE* values are quite large (since we have chosen the previously mentioned parameters in order to observe the qualitative behaviour of our methods), the boxplots presented in Fig. (4b) show that *HHCART* using *PCA* gives better results than *RandCART*.

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Methods comparison for tabular datasets</figcaption></figure>
## 4.3 Image Reconstruction

In this subsection, we shall study the case of image reconstruction. As for the case of the *eForest* encoder [14], the **OF-AE** method is suitable also for the reconstruction of images belonging to various datasets. In what follows, for the image reconstruction plots, the first row will represent the chosen test samples, while in the second row we will always plot the reconstructed images.

For our first experiment, we consider the *MNIST* dataset, where *n_estimators* is 300, *n_train* is 100, *max Samples* is 1.0, *max_features* is 0.75, and *max_depth* was set to 3. After the training, we have applied the usual encoding-decoding technique of *OF-AE* on 10 test samples.

<figure><img src="image_6.png" /><figcaption>Fig. 5. Image reconstruction of *MNIST* images</figcaption></figure>

From the depiction of Fig. 5 we observe that we obtain a good representation of the test images even though we have used only 100 training samples, similar to *eForest* encoder. We have used lower values for *n_estimators* and *max_depth* than the ones from [14]. For a better decoding quality we can increase the number of trees, but our results suggest that the oblique splits can retain a compact representation of the features. In the previous image reconstruction experiment we have used the *PCA* method, while in our second simulation we will compare the *SSIM* values (also on the *MNIST* dataset) for different methods for transforming the original space. In (Fig. 6a) we observe the *SSIM* values of *PCA* (*eig*), alongside various implementations from *SKLearn: Truncated-SVD* (*svd*), *FastICA* (*fast_ica*), *FactorAnalysis* (*factor*) and *GRP* (*proj*), respectively. In this simulation, *n_estimators* and *n_train* are set to 50, while (*max Samples*, *max_features*, *max_depth*) = (0.75, 0.75, 3).

We observe that the variation in the 10 simulation runs is almost the same for every type of feature transformation method, while the *GRP* seems to have the lowest impact on the reconstruction values.

For our third experiment, we plan to evaluate the *SSIM* values obtained from the decoded samples on the test dataset for the *HHCART* (using *GRP*) and *RandCART*. In Fig. (6b) we have utilized our custom *CHD2R* dataset where all the images are resized to 28 x 28. We took 10 simulation runs and we set (*max Samples*, *max_features*, *max_depth*) = (0.25, 0.5, 30). We observe that we get better results (in the case of *SSIM* values) for the *HHCART* method, and even though we can improve the results by increasing the number of estimators (or modifying other parameters), the results from (Fig. 6b) represent the main aspect of the difference between the aforementioned oblique methods.

We will continue our ablation study with the investigation of the training time of **OF-AE** with respect to the number of estimators, along with the width and the height values of the images, respectively. For the case
when we make the comparison of the image resize values versus the training time, we took (n_estimators, max_sample, max_features, max_depth) = (30, 0.25, 1.0, 10). On the other hand, for the comparison between the number of estimators versus the training time, we chose (resize_value, max_sample, max_features, max_depth) = (28, 0.25, 1.0, 10). For both situations, we used the HHCART oblique trees with the GRP, along with n_train set to 50 and 10 test samples, on the Oxford Flowers dataset. The results depicted in subfigures (Fig. 6c) and (Fig. 6d) suggest a disadvantage our method: the training time grows quickly if we increase the number of trees or the size of the images. The qualitative decoding of the autoencoder deeply depends on the number of ODT (this effectively represents that by increasing the number of estimators, we increase the number of constraints on the features through the weights equations). On the other hand, by increasing the size of the images, we observe that our method suffers from a "curse of dimensionality" problem, which is also present for the eForest encoder (where, for the same reconstruction quality of the decoded images, the number of axis-parallel trees is much larger) and for the tree-type encoder presented in [2] (where for each feature we must construct a DT with the labels as possible pixel values).

For the next experiment we consider some simulations on CIFAR10 using ensemble of RandCART ODT with max_depth = 3. In Fig. 7 we have 6 versions, where, from left to right: on the first row v3 and v1, in the second row v2 and v4, while in the last row v5 and v6. Under each subfigure from Fig. 7 we

<figure><img src="image_7.png" /><figcaption>Fig. 6. Methods comparison for image datasets</figcaption></figure>
have the parameters (`n_train`, `n_estimators`, `max Samples`, `max_features`). From the first row we see that, for a low number of estimators, the results are unsatisfactory. In addition, for the transition between v3 and v1 we get some decoding improvements by increasing the number of trees.

<figure><img src="image_8.png" /><figcaption>**Fig. 7.** Ablation study on CIFAR10</figcaption></figure>

For the second row results we have the transition between v2 and v4 them which is given by decreasing the number of training samples and increasing the number of estimators, which suggest that `n_estimators` have a larger influence over the decoding process than `n_train`. In the last row, the transition between v5 and v6 is given by increasing `n_estimators` and `n_train` which leads to even better results. But, increasing the quality of the reconstructed images, in this transition we have decreased also the `max Samples` parameter.

Our last experiment is the same as the one from [14] regarding model reuse. We have trained **OF-AE** with `n_train` equal to 1000 on CIFAR10, and we have decoded 10 test samples (resized to CIFAR10 dimensions 32 × 32) belonging to *Oxford Flowers* and *CHD2R*. By setting (`n_estimators`, `max Samples`, `max_features`, `max_depth`) = (1000, 0.5, 0.25, 3) and using the *HHCART* with the *GRP*, we get the results from Fig. 8. Quite interestingly, our method trained on a small number of images is able to reproduce almost exactly images from different datasets.

<figure><img src="image_9.png" /><figcaption>**Fig. 8.** Model reuse on the *Oxford Flowers & CHD2R* datasets</figcaption></figure>
# 5 Conclusions and Discussions

In this paper we have extended the *eForest* encoder to unsupervised ensembles of oblique trees. Our method **OF-AE** has the same limitations as the *eForest* encoder and the interpretable autoencoder from [2], i.e. the training time depends heavily on the image dimensions and on the number of estimators, thus our method becomes unfeasible for large datasets. Up to our knowledge, the method introduced in this paper is the only forest-type autoencoder where the encoding-decoding process is represented by a constrained optimization problem, hence this represents a significant departure from the works [4, 5, 11]. It is worth investigating if our methodology can be extended to neural-type trees as in [20, 32], respectively. In addition, a possibility would be to expand our work by drawing upon the methods used in [34] where the discrete tree parameters are learned using a global loss function.

# References

1. Agrawal, A., Verschueren, R., Diamond, S., Boyd, S.: A rewriting system for convex optimization problems. J. Control Decis. 5(1), 42–60 (2018)

2. Aguilar, D.L., Perez, M.A.M., Loyola-Gonzalez, O., Choo, K.K.R., Bucheli-Susarrey, E.: Towards an interpretable autoencoder: a decision tree-based autoencoder and its application in anomaly detection. IEEE Trans. Dependabl. Secur. Comput. (2022)

3. Bengio, Y., Yao, L., Alain, G., Vincent, P.: Generalized denoising auto-encoders as generative models. In: Advances in Neural Information Processing Systems, vol. 26 (2013)

4. Bennett, K.P.: Decision tree construction via linear programming. University of Wisconsin-Madison Department of Computer Sciences, Technical report (1992)

5. Bennett, K.P., Blue, J.A.: Optimal decision trees. Rensselaer Polytechn. Inst. Math Rep. 214, 24 (1996)

6. Blaser, R., Fryzlewicz, P.: Random rotation ensembles. J. Mach. Learn. Res. 17(1), 126–151 (2016)

7. Bourlard, H., Kamp, Y.: Auto-association by multilayer perceptrons and singular value decomposition. Biol. Cybern. 59(4), 291–294 (1988)

8. Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)

9. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

10. Breiman, L., Friedman, J., Stone, C.J., Olshen, R.: Classification and Regression Trees. CRC Press (1984)

11. Brown, D.E., Pittard, C.L., Park, H.: Classification trees with optimal multivariate decision nodes. Pattern Recogn. Lett. 17(7), 699–703 (1996)

12. Diamond, S., Boyd, S.: CVXPY: a python-embedded modeling language for convex optimization. J. Mach. Learn. Res. 17(83), 1–5 (2016)

13. ECNU: Oblique decision tree in python (2021). https://github.com/zhenlingcn/scikit-obliquetree

14. Feng, J., Zhou, Z.H.: Autoencoder by forest. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

15. Frosst, N., Hinton, G.: Distilling a neural network into a soft decision tree. arXiv preprint arXiv:1711.09784 (2017)
16. Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)

17. Irsoy, O., Alpaydin, E.: Autoencoder trees. In: Asian conference on machine learning, pp. 378–390. PMLR (2016)

18. Irsoy, O., Yıldız, O.T., Alpaydin, E.: Soft decision trees. In: Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012), pp. 1819–1822. IEEE (2012)

19. Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)

20. Kontschieder, P., Fiterau, M., Criminisi, A., Bulo, S.R.: Deep neural decision forests. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1467–1475 (2015)

21. Lin, Y., Jeon, Y.: Random forests and adaptive nearest neighbors. J. Am. Stat. Assoc. 101(474), 578–590 (2006)

22. Liu, B., Xia, Y., Yu, P.S.: Clustering through decision tree construction. In: Proceedings of the Ninth International Conference on Information and Knowledge Management, pp. 20–29 (2000)

23. Majumder, T.: Ensembles of Oblique Decision Trees. The University of Texas at Dallas (2020)

24. Moosmann, F., Nowak, E., Jurie, F.: Randomized clustering forests for image classification. IEEE Trans. Pattern Anal. Mach. Intell. 30(9), 1632–1646 (2008)

25. Moosmann, F., Triggs, B., Jurie, F.: Fast discriminative visual codebooks using randomized clustering forests. In: Advances in Neural Information Processing Systems, vol. 19 (2006)

26. Murthy, S.K., Kasif, S., Salzberg, S.: A system for induction of oblique decision trees. J. Artif. Intell. Res. 2, 1–32 (1994)

27. Nock, R., Guillame-Bert, M.: Generative trees: Adversarial and copycat. arXiv preprint arXiv:2201.11205 (2022)

28. Norouzi, M., Collins, M.D., Fleet, D.J., Kohli, P.: Co2 forest: Improved random forest by continuous optimization of oblique splits. arXiv preprint arXiv:1506.06155 (2015)

29. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

30. Reis, I., Baron, D., Shahaf, S.: Probabilistic random forest: a machine learning algorithm for noisy data sets. Astron. J. 157(1), 16 (2018)

31. Stoean, C., et al.: On using perceptual loss within the u-net architecture for the semantic inpainting of textile artefacts with traditional motifs. In: SYNACS Conference Publishing Service (CPS) (2022)

32. Tanno, R., Arulkumaran, K., Alexander, D., Criminisi, A., Nori, A.: Adaptive neural trees. In: International Conference on Machine Learning, pp. 6166–6175. PMLR (2019)

33. Wickramarachchi, D.C., Robertson, B.L., Reale, M., Price, C.J., Brown, J.: HHCART: an oblique decision tree. Comput. Statist. Data Anal. 96, 12–23 (2016)

34. Zantedeschi, V., Kusner, M., Niculae, V.: Learning binary decision trees by argmin differentiation. In: International Conference on Machine Learning, pp. 12298–12309. PMLR (2021)