# Improving Multi-Instance Multi-Label Learning by Extreme Learning Machine
## 4 Performance Evaluation
### Effectiveness
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{} & \multirow{2}{*}{**Image**} & \multicolumn{4}{c}{**Evaluation Criterion**} \\ \cline{3-6}  & & \(\mathbf{AP\uparrow}\) & \(C\mathbf{\downarrow}\) & \(\mathbf{OE\downarrow}\) & \(\mathbf{RL\downarrow}\) \\ \hline \multirow{4}{*}{MIMLSVM+} & \(Cost=10^{-3},\gamma=2^{1}\) & 0.4999 & 1.2100 & 0.6191 & 0.3779 \\  & \(Cost=10^{-2},\gamma=2^{2}\) & **0.5642** & **1.1201** & **0.5783** & 0.3609 \\  & \(Cost=10^{-1},\gamma=2^{3}\) & 0.5142 & 1.1262 & 0.6888 & **0.3511** \\  & \(Cost=1,\gamma=2^{1}\) & 0.4267 & 1.9808 & 0.7391 & 0.3711 \\  & \(Cost=10^{1},\gamma=2^{3}\) & 0.4705 & 1.9999 & 0.7969 & 0.3958 \\  & \(Cost=10^{2},\gamma=2^{5}\) & 0.3735 & 1.9799 & 0.6809 & 0.4513 \\  & \(Cost=10^{3},\gamma=2^{5}\) & 0.4541 & 2.0000 & 0.6950 & 0.3858 \\ \hline \multirow{2}{*}{MIMLELM} & \(\mathbf{HN}=100\) & 0.4381 & 2.0000 & 0.8400 & 0.4750 \\  & \(\mathbf{HN}=200\) & 0.5529 & 1.7410 & 0.6720 & 0.4109 \\ \cline{1-1}  & \(\mathbf{HN}=300\) & 0.4861 & 1.5700 & 0.8400 & 0.4376 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The effectiveness comparison on the Image data set. _AP_: average precision; \(C\): coverage; _OE_: one-error; _RL_: ranking loss; MIMLSVM+: multi-instance multi-label support vector machine; MIMELM: multi-instance multi-label-extreme learning machine. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{} & \multirow{2}{*}{**Text**} & \multicolumn{4}{c}{**Evaluation Criterion**} \\ \cline{3-6}  & & \(\mathbf{AP\uparrow}\) & \(C\mathbf{\downarrow}\) & \(\mathbf{OE\downarrow}\) & \(\mathbf{RL\downarrow}\) \\ \hline \multirow{4}{*}{MIMLSVM+} & \(Cost=10^{-3},\gamma=2^{1}\) & 0.7563 & 1.0295 & 0.3000 & 0.2305 \\  & \(Cost=10^{-2},\gamma=2^{1}\) & 0.7675 & 1.0405 & 0.2650 & 0.1968 \\  & \(Cost=10^{-1},\gamma=2^{1}\) & **0.7946** & 1.0445 & 0.2650 & 0.2025 \\  & \(Cost=1,\gamma=2^{1}\) & 0.7679 & 1.0145 & 0.2600 & 0.1978 \\  & \(Cost=10^{1},\gamma=2^{1}\) & 0.7807 & **1.0041** & 0.2400 & **0.1940** \\  & \(Cost=10^{2},\gamma=2^{1}\) & 0.7763 & 1.0450 & 0.2450 & 0.1953 \\  & \(Cost=10^{3},\gamma=2^{1}\) & 0.7801 & 1.0245 & **0.2350** & 0.1970 \\ \hline \multirow{2}{*}{MIMLELM} & \(\mathbf{HN}=100\) & 0.7476 & 1.0670 & 0.3540 & 0.2075 \\  & \(\mathbf{HN}=200\) & 0.7492 & 1.0928 & 0.3409 & 0.2132 \\ \cline{1-1}  & \(\mathbf{HN}=300\) & 0.7554 & 1.0365 & 0.3443 & 0.2023 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The effectiveness comparison on the Text dataset. Moreover, we conduct another set of experiments to gradually evaluate the effect of each contribution in MIMLELM. That is, we first modify MIMLSVM+ to include the automatic method for \(k\), then use ELM instead of SVM and then include the genetic algorithm-based weights assignment. The effectiveness of each option is gradually tested on four real datasets using our evaluation criteria. The results are shown in Figure 4a-d, where SVM denotes the original MIMLSVM+ [5], SVM+\(k\) denotes the modified MIMLSVM+ including the automatic method for \(k\), ELM+\(k\) denotes the usage of ELM instead of SVM in SVM+\(k\) and ELM+\(k\)+\(w\) denotes ELM+\(k\), further including the genetic algorithm-based weights assignment. As seen from Figure 4a-d, the options of including the automatic method for \(k\) and the genetic algorithm-based weights assignment can make the four evaluation criteria better, while the usage of ELM instead of SVM in SVM+\(k\) slightly reduces the effectiveness. Since ELM can reach a comparable effectiveness as SVM at a much faster learning speed, it is the best option to combine the three contributions in terms of both efficiency and effectiveness. Figure 4: Gradual effectiveness evaluation of each contribution in Multi-instance Multi-label with Extreme Learning Machine (MIMLELM). (**a**) Gradual evaluation on average precision (\(AP\)); (**b**) gradual evaluation on convergence (C); (**c**) gradual evaluation on one-error (\(OE\)); (**d**) gradual evaluation on ranking loss (\(RL\)).