# Improving Multi-Instance Multi-Label Learning by Extreme Learning Machine
## 4 Performance Evaluation
### Efficiency  
In this series of experiments, we study the efficiency of MIMELM by testing its scalability. That is, each dataset is replicated different numbers of times, and then, we observe how the training time and the testing time vary with the data size increasing. Again, MIMLSVM+ is utilized as the competitor. Similarly, the MIMLSVM+ algorithm is implemented with a Gaussian kernel, while the penalty factor cost is set from \(10^{-3}\), \(10^{-2}\), \(\ldots\), \(10^{3}\). The MIMELM is implemented with the number of hidden layer nodes set to be 100, 200 and 300, respectively.  
The experimental results are given in Figures 5-8. As we observed, when the data size is small, the efficiency difference between MIMLSVM+ and MIMELM is not very significant. However, as the data size increases, the superiority of MIMELM becomes more and more significant. This case is particularly evident in terms of the testing time. In the Image dataset, the dataset is replicated 0.5-2 times with the step size set to be 0.5. When the number of copies is two, the efficiency improvement could be up to one 92.5% (from about 41.2 s down to about 21.4 s). In the Text dataset, the dataset is replicated 0.5-2 times with the step size set to be 0.5. When the number of copies is two, the efficiency improvement could be even up to 223.3% (from about 23.6 s down to about 7.3 s). In the Geobacter sulfureducens dataset, the dataset is replicated 1-5 times with the step size set to be 1. When the number of copies is five, the efficiency improvement could be up to 82.4% (from about 3.1 s down to about 1.7 s). In the Azotobacter vinelandii dataset, the dataset is replicated 1-5 times with the step size set to be one. When the number of copies is five, the efficiency improvement could be up to 84.2% (from about 3.5 s down to about 1.9 s).  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Geob.**} & \multicolumn{4}{c}{**Evaluation Criterion**} \\ \cline{2-5}  & \(\mathbf{AP\uparrow}\) & \(C\downarrow\) & \(\mathbf{OE\downarrow}\) & \(\mathbf{RL\downarrow}\) \\ \hline ELM & 0.6649 & 1.3720 & 0.5390 & 0.2112 \\ RELM & 0.7818 & 1.1668 & 0.4584 & 0.1796 \\ MCVELM & 0.7892 & 1.1559 & 0.4150 & 0.1626 \\ KELM & 0.8088 & 1.1279 & 0.4049 & 0.1586 \\ DropELM & 0.8107 & 1.1253 & 0.4020 & 0.1582 \\ GEELM & 0.7933 & 1.1499 & 0.4109 & 0.1617 \\ \hline \hline \end{tabular}
\end{table}
Table 8: The effectiveness comparison of ELM and its variants on the Geobacter sulfureducens dataset.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Azoto.**} & \multicolumn{4}{c}{**Evaluation Criterion**} \\ \cline{2-5}  & \(\mathbf{AP\uparrow}\) & \(C\downarrow\) & \(\mathbf{OE\downarrow}\) & \(\mathbf{RL\downarrow}\) \\ \hline ELM & 0.6622 & 1.3610 & 0.6658 & 0.2129 \\ RELM & 0.7928 & 1.1368 & 0.5561 & 0.1778 \\ MCVELM & 0.7968 & 1.1235 & 0.5533 & 0.1757 \\ KELM & 0.8346 & 1.0907 & 0.5283 & 0.1617 \\ DropELM & 0.8524 & 1.0679 & 0.5172 & 0.1583 \\ GEELM & 0.7997 & 1.1194 & 0.5513 & 0.1650 \\ \hline \hline \end{tabular}
\end{table}
Table 9: The effectiveness comparison of ELM and its variants on the Azotobacter vinelandii dataset.  
Figure 5: The efficiency comparison on the Image dataset. (**a**) The comparison of the training time; (**b**) the comparison of the testing time.  
Figure 6: The efficiency comparison on the Text dataset. (**a**) The comparison of the training time; (**b**) the comparison of the testing time.  
Figure 7: The efficiency comparison on the Geobacter sulfurreducens dataset. (**a**) The comparison of the training time; (**b**) the comparison of the testing time.