# An Enhanced Multimodal Stacking Scheme for Online Pornographic Content Detection
## 4 Experiment Results
### Analysis Results for Multimodal Ensemble  
In the third experiment, the accuracy and the false negative rates of the different multimodal ensemble schemes for the pornographic content detection are compared, as shown in Table 3. The enhanced stacking scheme proposed in this study, the previous multimodal stacking scheme used in [17], and (as a representative ensemble method) the majority-voting ensemble scheme are compared. The enhanced multimodal stacking scheme suggested in this study is shown to outperform both of the other ensemble schemes in terms of its accuracy and false negative rate.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Visual Classifier with Different Visual Feature Extractions** & **Accuracy** \\ \hline Visual classifier using VGG-16 and Bi-direction RNN & 95.33\% \\ Stacking ensemble with only the three visual classifiers used in [17] & 94.63\% \\ \hline \hline \end{tabular}
\end{table}
Table 1: Detection accuracy comparison based on visual features.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Audio Classifier with Different Auditory Feature Extractions** & **Accuracy** \\ \hline Audio classifier using a multilayered dilated convolution network & 89.16\% \\ Audio classifier in [17] & 61.83\% \\ \hline \hline \end{tabular}
\end{table}
Table 2: Detection accuracy comparison based on auditory features.  
As described in the introduction section, in order to use the scheme in the online service environments, it is necessary to minimize the number of detection omissions that cannot be detected until the final detection stage. The number of harmful contents detected (true positives) and the number of harmful contents that could not be detected (false negatives) by both the enhanced multimodal stacking scheme and the previous scheme used in [17] are analyzed, as shown in Table 4. Using the proposed enhanced multimodal stacking scheme showed that, among the 1500 total harmful contents in the testing data set, 1398 harmful ones (93.20% of the input) were detected by the first fusion classifier. Out of 102 harmful contents that were not detected by the first classifier, 29 (28.43% of the input) were detected by the second video classifier. Among the 73 undetected by both the first and second classifiers, 4 (5.47% of the input) were detected during the third detection stage. In summary, a total of 1431 (95.40%) of the 1500 harmful contents were detected, and 69 (4.60%) were ultimately not detected. Meanwhile, as the results of the experiment with the previous stacking scheme in [17] used the same data, the first stage video classifier detected 1162 (77.47%) harmful contents, the second stage image classifier detected 97 (28.70%) out of the remaining 338 harmful contents, the motion classifier detected 85 (35.3%) of the 241 remaining harmful contents during the third stage, and the audio classifier detected 71 (45.5%) of the 156 harmful contents in the fourth stage. In summary, 1415 (94.3%) out of the 1500 harmful contents were successfully detected, and 85 (5.67%) harmful contents failed to be detected by the previous multimodal stacking method. Since the number of harmful contents that cannot be detected in the online service environments must be minimized, the enhanced multimodal stacking scheme proposed in this paper produces superior performance to the previous multimodal stacking scheme used in [17], with better accuracy, fewer false negatives, and fewer necessary component classifiers.