# Multiple Instance Learning with Trainable Soft Decision Tree Ensembles
## 8 Conclusions  
### Discussion  
An RF-based model to solve the MIL classification problem for small tabular data is proposed. It is based on training decision trees by means of their converting to a neural network of a specific form. Moreover, it uses the attention mechanism to aggregate the bag \begin{table}
\begin{tabular}{l l l l} \hline \hline  & **Elephant** & **Fox** & **Tiger** \\ \hline mi-SVM [15] & 0.822 \(\pm\) N/A & 0.582 \(\pm\) N/A & 0.784 \(\pm\) N/A \\ MI-SVM [15] & 0.843 \(\pm\) N/A & 0.578 \(\pm\) N/A & 0.840 \(\pm\) N/A \\ MI-Kernel [54] & 0.843 \(\pm\) N/A & 0.603 \(\pm\) N/A & 0.842 \(\pm\) N/A \\ EM-DD [55] & 0.771 \(\pm\) 0.097 & 0.609 \(\pm\) 0.101 & 0.730 \(\pm\) 0.096 \\ mi-Graph [56] & 0.869 \(\pm\) 0.078 & 0.620 \(\pm\) 0.098 & 0.860 \(\pm\) 0.083 \\ miVLAD [57] & 0.850 \(\pm\) 0.080 & 0.620 \(\pm\) 0.098 & 0.811 \(\pm\) 0.087 \\ miFV [57] & 0.852 \(\pm\) 0.081 & 0.621 \(\pm\) 0.109 & 0.813 \(\pm\) 0.083 \\ mi-Net [19] & 0.858 \(\pm\) 0.083 & 0.613 \(\pm\) 0.078 & 0.824 \(\pm\) 0.076 \\ MI-Net [19] & 0.862 \(\pm\) 0.077 & 0.622 \(\pm\) 0.084 & 0.830 \(\pm\) 0.072 \\ MI-Net with DS [19] & 0.872 \(\pm\) 0.072 & 0.630 \(\pm\) 0.080 & 0.845 \(\pm\) 0.087 \\ MI-Net with RC [19] & 0.857 \(\pm\) 0.089 & 0.619 \(\pm\) 0.104 & 0.836 \(\pm\) 0.083 \\ Attention [23] & 0.868 \(\pm\) 0.022 & 0.615 \(\pm\) 0.043 & 0.839 \(\pm\) 0.022 \\ Gated-Attention [23] & 0.857 \(\pm\) 0.027 & 0.603 \(\pm\) 0.029 & 0.845 \(\pm\) 0.018 \\
**STE-MIL** & **0.885**\(\pm\) 0.038 & **0.730**\(\pm\) 0.080 & **0.875**\(\pm\) 0.039 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Accuracy measures (the mean and standard deviation) for comparison of the well-known MIL classification models, the RF and the STE-MIL using datasets Elephant, For and Tiger. \begin{table}
\begin{tabular}{l l l} \hline \hline  & **Musk1** & **Musk2** \\ \hline mi-SVM [15] & 0.874 \(\pm\) N/A & 0.836 \(\pm\) N/A \\ MI-SVM [15] & 0.779 \(\pm\) N/A & 0.843 \(\pm\) N/A \\ MI-Kernel [54] & 0.880 \(\pm\) N/A & 0.893 \(\pm\) N/A \\ EM-DD [55] & 0.849 \(\pm\) 0.098 & 0.869 \(\pm\) 0.108 \\ mi-Graph [56] & 0.889 \(\pm\) 0.073 & **0.903**\(\pm\) 0.086 \\ miVLAD [57] & 0.871 \(\pm\) 0.098 & 0.872 \(\pm\) 0.095 \\ miFV [57] & 0.909 \(\pm\) 0.089 & 0.884 \(\pm\) 0.094 \\ mi-Net [19] & 0.889 \(\pm\) 0.088 & 0.858 \(\pm\) 0.110 \\ MI-Net [19] & 0.887 \(\pm\) 0.091 & 0.859 \(\pm\) 0.102 \\ MI-Net with DS [19] & 0.894 \(\pm\) 0.093 & 0.874 \(\pm\) 0.097 \\ MI-Net with RC [19] & 0.898 \(\pm\) 0.097 & 0.873 \(\pm\) 0.098 \\ Attention [23] & 0.892 \(\pm\) 0.040 & 0.858 \(\pm\) 0.048 \\ Gated-Attention [23] & 0.900 \(\pm\) 0.050 & 0.863 \(\pm\) 0.042 \\
**STE-MIL** & **0.918**\(\pm\) 0.077 & 0.854 \(\pm\) 0.061 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Accuracy measures (the mean and standard deviation) for comparison of the well-known MIL classification models, the RF and the STE-MIL by using datasets Musk1 and Musk2. information and to enhance the classification accuracy. The attention mechanism can also be used to explain why a tested bag is assigned a certain label, because the attention shows weights of instances of the tested bag and selects the most influential instances. Numerical experiments with well-known datasets, used by many authors in evaluating MIL models, demonstrated that STE-MIL outperformed many models, including the following: mi-SVM, MI-SVM, MI-Kernel, EM-DD, mi-Graph, miVLAD, miFV, mi-Net, MI-Net, MI-Net with DS, MI-Net with RC, the Attention and Gated-Attention models. An important advantage of STE-MIL is that it successfully combines the positive properties of decision trees to accurately classify small tabular data, with properties of neural networks to learn complex functions in an end-to-end manner. This combination was implemented by introducing soft decision trees and converting the decision trees into neural networks. Another beneficial idea behind STE-MIL was that of using the tree outputs, in the form of the class probability distributions, as embeddings, which allowed us to apply the attention mechanism.