# An Enhanced Multimodal Stacking Scheme for Online Pornographic Content Detection
## 3 Methods
### Extraction of Implicative Features  
In the second feature extraction step of the proposed scheme, an improved process is used to consider the changes in the input segment over time and extract the implicative features that reflect the correlative relationships with the neighboring signals, as shown in Figure 2, while the first step follows the same process as used in the previous study [17]. In order to extract an implicative visual descriptor for each video segment, the visual feature extractor, which consists of the front part of the pre-trained VGG-16 and the bi-directional RNN in [20], is utilized. In the first stage, the front portion of the pre-trained VGG-16 without the upper-level fully connected (FC) layers is used to extract highly expressive high-level features. In general, the inclusion of the upper-level FC layers during transition learning of the VGG-16 leads to a rapid increase in the number of learning parameters. To reduce the number of learning parameters, the upper-level FC layers of the VGG-16 are removed, and two 1*1 convolution layers are applied. In order for the first 1*1 convolution layer to create an output similar in size to the FC layer (which produces 4096 outputs) of the VGG-16, the output feature map of size (7, 7, 84) with (widths, heights, channels) is configured by receiving the feature map of size (7, 7, 512) from the VGG-16. The second 1*1 convolution layer also configures an output feature map of size (7, 7, 21) to create an output similar to the output by the last FC layer of the VGG-16 with a size of 1024. The output vector becomes 1029 in size after the flattening operation. A frame image of (224, 224, 3) is  
Figure 1: Overall procedure of the proposed multimodal stacking scheme. then entered as the input for the VGG-16 in the feature extractor. Since a vector of 1029 is extracted as a static feature, a sequence of 10 static features is extracted for a single video segment composed of 10 sampled frames. Next, in the second part of the visual feature extractor, instead of using the simple average vector of the static features as in [17], a bi-directional RNN using LSTM [26; 27] is used to singularly express the implicative visual feature vector that expressively implicates the bi-directional correlations between the static features that change over time. The Bi-directional RNN extracts the mutual correlative relationships of the forward and reverse directions from a sequential input by using the LSTM cells of each direction. These extractions are combined into a vector and input into the FC layers using the concatenation operation, thereby creating an implicative visual descriptor of size 1024 that infers the visual features of the content segment from the sequence of the static feature vectors. Moreover, unlike [17], which extracts and uses the static features from the auditory elements of the input content, the auditory feature extractor for extracting an implicative auditory descriptor for each audio segment is designed to use a multilayered dilated convolutional block as in [23]. The auditory feature extractor is composed of two parts. In the first part, a transformation technique that uses a mel-scaled spectrogram [28] is used to extract the static features from an audio segment. When an audio segment is entered as the input, a spectrogram of (431, 1, 128) is extracted as a static feature. The spectrogram used in [28] segments the 0 Hz-8192 Hz band frequency values using 128 filter banks for the unit length determined by the sampling rate of the audio to express the representative energy values. In case the sampling rate of the audio segment is 44 KHz, 431 samples of 128-dimensional static features are extracted per segment.