# Multiple Instance Learning with Trainable Soft Decision Tree Ensembles
## 7 Numerical Experiments  
In order to compare the proposed model with other existing MIL classification models, we trained the corresponding models on the well-known datasets Musk1, Musk2 (drug activity) [4], Fox, Tiger, and Elephant (images divided into patches) [15]. Table 1 shows the number of bags \(N\), the number of instances \(n\) in every bag and the number of features \(m\) in instances for the corresponding datasets. The Musk1 dataset contains 92 bags, such that each bag consists of 476 instances having 166 features. The average bag size is 5.17. The Musk2 dataset contains 102 bags such that each bag consists of 6598 instances with 166 features. The average bag size is 64.69. Each dataset (Fox, Tiger and Elephant) contains exactly 200 bags consisting of different numbers of instances with 230 features. Numbers of instances in datasets Fox, Tiger and Elephant are 1302, 1220 and 1391, respectively. The average bag sizes of the datasets are 6.60, 6.96 and 6.10, respectively. The considered datasets were created on the basis of practical tasks. In particular, datasets Musk1 and Musk2 are real drug activity prediction datasets, which allowed us to study how a molecule possesses a musky nature. The task using these databases was to classify molecules as positive (musk) or negative (non-musk) [4]. A molecule exists in multiple conformations (its certain configurations), which can be regarded as unlabeled instances in the MIL terminology, because it is not possible to observe and measure characteristics of a single conformation [9]. At the same time, one can observe the characteristics for the corresponding molecule. Hence, the molecule is labeled positive if one or more of its conformations show muskiness. It can be regarded as a bag. Datasets Fox, Tiger, and Elephant are obtained from images which contain the corresponding animals [15]. Each image is divided into small patches represented by color, texture and shape features. If at least one patch in the image contains an animal, then the image, as a bag, is labeled positive. The accuracy measures for these datasets were also obtained by means of the following well-known MIL classification models; mi-SVM [15], MI-SVM [15], MI-Kernel [54], EM-DD [55], mi-Graph [56], miVLAD [57], miFV [57], mi-Net [19], MI-Net [19], MI-Net with DS [19], MI-Net with RC [19], Attention and Gated-Attention [23]. We investigated Extremely Randomized Trees (ERT) for initialization because they provide better results. At each node, the ERT algorithm chooses a split point randomly for each feature and then selects the best split among these [58]. In the experiments we also used the following: sigmoid function with the trainable temperature parameter \(\omega\), initialized with 10, as an indicator approximation; softmax operation with the trainable temperature parameter \(\tau\), which was also initialized with 10; the number \(T\) of decision trees was 20; the largest depth \(h\) of trees was 5; the dimension \(E\) of each embedding vector was 4; the number of epochs was 2000; the batch size was 20 and the learning rate was 0.01. Accuracy measures (the mean and standard deviations) are computed by using five-fold cross-validation. The best results are in bold in tables. Numerical results for datasets Elephant, Fox and Tiger are shown in Table 2. It can be seen from Table 2 that STE-MIL outperformed on all datasets. Numerical results for datasets Musk1 and Musk2 are shown in Table 3. One can see from Table 3 that the proposed model outperformed all the other models for the dataset Musk1. However, STE-MIL provided the worst result for the dataset \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Data Set** & \(N\) & \(n\) & \(m\) \\ \hline Elephant & 200 & 1391 & 230 \\ Fox & 200 & 1302 & 230 \\ Tiger & 200 & 1220 & 230 \\ Musk1 & 92 & 476 & 166 \\ Musk2 & 102 & 6598 & 166 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: A brief introduction to datasets for classification. Musk2. One of the reasons for this result was that bags in Musk2 consisted of many instances.