# Improving Multi-Instance Multi-Label Learning by Extreme Learning Machine
## 4 Performance Evaluation
### Effectiveness  
In this set of experiments, we study the effectiveness of the proposed MIMELM on the four real datasets. The four criteria mentioned in Section 4.2 are utilized for performance evaluation. Particularly, MIMLSVM+ [5], one of the state-of-the-art algorithms for learning with multi-instance multi-label examples, is utilized as the competitor. The MIMLSVM+ (Advanced multi-instance multi-label with support vector machine) algorithm is implemented with a Gaussian kernel, while the penalty factor cost is set from \(10^{-3}\), \(10^{-2}\),..., \(10^{3}\). The MIMELM (multi-instance multi-label with extreme learning machine) is implemented with the number of hidden layer nodes set to be 100, 200 and 300, respectively. Specially, for a fair performance comparison, we modified MIMLSVM+ to include the automatic method for \(k\) and the genetic algorithm-based weights assignment. On each dataset, the data are randomly partitioned into a training set and a test set according to the ratio of about 1:1. The training set is used to build a predictive model, and the test set is used to evaluate its performance. Experiments are repeated for thirty runs by using random training/test partitions, and the average results are reported in Tables 2-5, where the best performance on each criterion is highlighted in boldface, and '\(\downarrow\)' indicates \"the smaller the better\", while '\(\uparrow\)' indicates \"the bigger the better\". As seen from the results in Tables 2-5, MIMLSVM+ achieves better performance in terms of all cases. Applying statistical tests (nonparametric ones) to the rankings obtained for each method in the different datasets according to [45], we find that the differences are significant. However, another important observation is that MIMLSVM+ is more sensitive to the parameter settings than MIMELM. For example, on the Image dataset, the AP values of MIMLSVM+ vary in a wider interval \([0.3735,0.5642]\) while those of MIMELM vary in a narrower range \([0.4381,0.5529]\); the C values of MIMLSVM+ vary in a wider interval \([1.1201,2.0000]\), while those of MIMELM vary in a narrower range \([1.5700,2.0000]\); the \(OE\) values of MIMLSVM+ vary in a wider interval \([0.5783,0.7969]\), while those of MIMLELM vary in a narrower range \([0.6720,0.8400]\); and the \(RL\) values of MIMLSVM+ vary in a wider interval \([0.3511,0.4513]\), while those of MIMLELM vary in a narrower range \([0.4109,0.4750]\). In the other three real datasets, we have a similar observation. Moreover, we observe that in this set of experiments, MIMLELM works better when \(HN\) is set to 200. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Geobacter Sulfurreducens**} & \multicolumn{4}{c}{**Evaluation Criterion**} \\ \cline{3-6}  & \(\mathbf{AP\uparrow}\) & \(C\mathbf{\downarrow}\) & \(\mathbf{OE\downarrow}\) & \(\mathbf{RL\downarrow}\) \\ \hline \multirow{4}{*}{MIMLSVM+} & \(Cost=10^{-3},\gamma=2^{5}\) & 0.6099 & 1.5122 & 0.5583 & 0.2284 \\  & \(Cost=10^{-2},\gamma=2^{4}\) & 0.6529 & 1.2439 & 0.5341 & 0.2488 \\  & \(Cost=10^{-1},\gamma=2^{2}\) & **0.6871** & **1.0488** & **0.4585** & **0.1343** \\  & \(Cost=1,\gamma=2^{5}\) & 0.6755 & 1.0732 & 0.4609 & 0.1873 \\  & \(Cost=10^{1},\gamma=2^{3}\) & 0.6311 & 1.1707 & 0.5097 & 0.1742 \\  & \(Cost=10^{2},\gamma=2^{5}\) & 0.6733 & 1.1219 & 0.4854 & 0.2187 \\  & \(Cost=10^{3},\gamma=2^{1}\) & 0.6268 & 1.2195 & 0.5097 & 0.2122 \\ \hline \multirow{2}{*}{MIMLELM} & \(\mathbf{HN}=100\) & 0.6438 & 1.3902 & 0.5707 & 0.2151 \\  & \(\mathbf{HN}=200\) & 0.6649 & 1.3720 & 0.5390 & 0.2112 \\ \cline{1-1}  & \(\mathbf{HN}=300\) & 0.6495 & 1.4025 & 0.5695 & 0.2142 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: The effectiveness comparison on the Geobacter sulfurreducens (Geob.) dataset.