Porn Streamer Recognition in Live Video Based on Multimodal Knowledge Distillation

Chinese Journal of ElectronicsVolume 30, Issue 6 p. 1096-1102 Original Research PaperFree Access Porn Streamer Recognition in Live Video Based on Multimodal Knowledge Distillation WANG Liyuan, WANG Liyuan wangliyuan@emails.bjut.edu.cn Faculty of Information, Beijing University of Technology, Beijing, 100124 China Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124 ChinaSearch for more papers by this authorZHANG Jing, Corresponding Author ZHANG Jing zhj@bjut.edu.cn Faculty of Information, Beijing University of Technology, Beijing, 100124 China Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124 ChinaSearch for more papers by this authorYAO Jiacheng, YAO Jiacheng Faculty of Information, Beijing University of Technology, Beijing, 100124 China Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124 ChinaSearch for more papers by this authorZHUO Li, ZHUO Li Faculty of Information, Beijing University of Technology, Beijing, 100124 China Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124 ChinaSearch for more papers by this author WANG Liyuan, WANG Liyuan wangliyuan@emails.bjut.edu.cn Faculty of Information, Beijing University of Technology, Beijing, 100124 China Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124 ChinaSearch for more papers by this authorZHANG Jing, Corresponding Author ZHANG Jing zhj@bjut.edu.cn Faculty of Information, Beijing University of Technology, Beijing, 100124 China Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124 ChinaSearch for more papers by this authorYAO Jiacheng, YAO Jiacheng Faculty of Information, Beijing University of Technology, Beijing, 100124 China Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124 ChinaSearch for more papers by this authorZHUO Li, ZHUO Li Faculty of Information, Beijing University of Technology, Beijing, 100124 China Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing, 100124 ChinaSearch for more papers by this author First published: 01 November 2021 https://doi.org/10.1049/cje.2021.07.027 This work is supported by the National Natural Science Foundation of China (No.61971016, No.61531006) and the Beijing Education Committee Cooperation Beijing Natural Science Foundation (No.KZ201910005007). AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract Although deep learning has reached a higher accuracy for video content analysis, it is not satisfied with practical application demands of porn streamer recognition in live video because of multiple parameters, complex structures of deep network model. In order to improve the recognition efficiency of porn streamer in live video, a deep network model compression method based on multimodal knowledge distillation is proposed. First, the teacher model is trained with visual-speech deep network to obtain the corresponding porn video prediction score. Second, a lightweight student model constructed with MobileNetV2 and Xception transfers the knowledge from the teacher model by using multimodal knowledge distillation strategy. Finally, porn streamer in live video is recognized by combining the lightweight student model of visualspeech network with the bullet screen text recognition network. Experimental results demonstrate that the proposed method can effectively drop the computation cost and improve the recognition speed under the proper accuracy. I. Introduction With the popularity of Internet plus, live streaming video (live video for short)continues to be overwhelming and has penetrated into our daily life. Different from recorded video, live video has strong ability of attracting fans and user stickiness. Some streamers show porn content in live video for seeking profit and popularity, who bring a great harm to the network environment. Statistics show that there are nearly 4 million online users and more than 3000 live rooms in the daily peak hours of large-scale live streaming platforms. Therefore, it puts forward high real-time online supervision requirements for porn streamer recognition in live video. Lately, deep learning has made great breakthroughs in the fields of visual, audio and text processing and analysis[1-3]. Wehrmann et al.[4] proposed Adult content recognition with deep neural networks (ACORDE) that comprises both Convolution neural network (CNN) and Long short term memory (LSTM) recurrent networks for adult content detection in videos, reaching to 99% accuracy. Furthermore, multimodal deep learning also provides a new approach for porn steamer recognition. For example, Takahashi et al.[5] proposed an Audio event net (AENet) to extract audio features, then combined visual and audio features to improve the performance on action recognition. Despite reaching a higher accuracy for video content analysis by using multimodal deep learning, it is not satisfied with practical application demands of porn streamer recognition in live video because of numerous parameters, complex structures and low speed of deep network model. In order to improve the recognition efficiency of porn streamer, it is thus extremely necessary to compress and optimize the deep network model to remove redundant structures and parameters, so as to speed up the model processing[6]. To address the above issues, several deep network model compression techniques have been proposed[7]. As one of the representative methods of deep network model compression, knowledge distillation[8] makes the deep model more compact as well as significantly drop computing cost by following the “Student-Teacher” (S-T) learning framework. Specifically, knowledge distillation constructs an S-T learning framework to “distill” knowledge learned by teacher model and then transfer it to student model, thereby reduce the model complexity and generate a deep network compression model. Considering that both Xception and MobileNetV2 are deep separable convolutions with less parameters to balance between performance and complexity, we will construct the student model based on Xception and MobileNetV2. In this paper, a porn streamer recognition method in live video based on multimodal knowledge distillation is proposed by compressing deep network model to construct a lightweight porn streamer recognition network with few parameters redundancy and faster processing speed so as to satisfy the practical application demands. First, the teacher model is trained with visual-speech deep network to obtain the corresponding porn video prediction score. Second, a lightweight student model constructed with MobileNetV2 and Xception is transferred the knowledge from the teacher model by using multimodal knowledge distillation strategy. Finally, porn streamer in live video is recognized by combining the lightweight student model of visual-speech network with the bullet screen text recognition network for improving recognition speed under the proper accuracy. II. Method In our previous work[9,10], a multimodal deep network was designed to recognize porn streamer by using visual, speech, and bullet screen text. By our conclusion, the bullet screen text recognition network's speed is faster, and visual-speech deep network is the main factor affecting the speed of multimodal deep network. This is because that discovering high-level semantics information from the visual-speech modal data uses a cascade of multiple layers of nonlinear processing units for feature extraction, while bullet screen text is an explicit semantic clue. Therefore, our purpose of knowledge distillation is mainly to compress visual-speech deep network model. Through the above analysis, we design a general architecture for porn streamer recognition based on multimodal knowledge distillation, as shown in Fig.1. Firstly, the visual-speech deep network as a teacher model is constructed by multimodal attention-gated CNN + Bidirectional Gated recurrent unit (Bi-GRU), then a FC (Fully connected) layer is applied with the softmax function. Considering the difficulty of training and the performance of the teacher's model, we set the temperature of softmax function to 5 to obtain better training effect. After the above steps, the model outputs the predicted score which is saved as the soft targets, the ground truth is saved as the hard targets. Then, the lightweight porn streamer recognition network as a student model is constructed with Xception /MobileNetV2, global pooling layer and FC layer. Next, the distillation loss function is calculated to transfer knowledge from teacher model to student model by two cross-entropy functions. Finally, the student model is combined with the bullet screen text recognition network, which is constructed by Bidirectional encoder representation from transformers (BERT) model[11], mapping layer and FC layer, and a lightweight multimodal deep network is constructed to recognize the porn streamers in live video. Fig. 1Open in figure viewerPowerPoint The overall architecture of porn streamer recognition based on multimodal knowledge distillation 1. Knowledge distillation strategy To reduce the computing cost of complex models, Hinton et al.[8] introduced the knowledge distillation technology. In conventional supervised learning tasks, prior knowledge often exists in the form of ground truth. Such ground truth ordinarily adopts 0-1 encoding called a hard target. The outputs of a deep network model are prediction probabilities, in which the differences among these probabilities can reflect the differences between the category information and the prediction target. This probability distribution contains the “knowledge” that can help the model to learn about category differences. For a live video recognition task, the live video will be recognized and divided into porn streamers and normal streamers. Therefore, the probability output of the model is an array that contains both positive and negative probabilities. The hard target comes from the ground truth, which is accurate but has little information; the soft target comes from the model output, which has more information but has some errors. Therefore, the porn streamer recognition model for live video should be trained based on the combination of soft targets and hard targets, so as to achieve more efficient knowledge transfer performance. Based on the above analysis, our multimodal knowledge distillation strategy is to transfer knowledge from the teacher to the student model. First, visualspeech deep network is used as the teacher model, in which the probability distribution is calculated with the temperature controlled softmax function F as follows: F = exp x i / T ∑ k = 1 n exp x k / T (1) where x i is the input, T is the hyperparameter representing the temperature that lets the model realize the “distillation” process. When T takes a larger value, the output of the softmax function can be made smoother, as shown in Fig.2. Fig. 2Open in figure viewerPowerPoint Softmax function regulated by T Consequently, adopting larger T values in the training of the teacher model can produce more uniform soft targets. Then both hard targets and soft targets are entered into a lightweight student model to learn knowledge from the teacher model based on soft targets, which is corrected to a certain extent by using hard targets to improve the recognition ability. Finally, the performance of student model is improved by training based on the above “temperature rise” (T value rise) process and the normal “temperature” (change T value to 1) is recovered in actual recognition practice to “distill” knowledge from the teacher model to the student model. As a result, the entire knowledge distilling process is completed. 2. Design of multimodal knowledge distillation network for porn streamer recognition in live video Many excellent deep network models have presented an outstanding comprehensive performance in accuracy and speed. The representative deep network models are shown in Table 1[12]. Table 1. Outstanding deep networks with excellent performance[12] Model Size Top-1 accuracy Top-5 accuracy Parameter Xception 88 MB 0.790 0.945 22,910,480 ResNet50 98 MB 0.749 0.921 25,636,712 ResNeXt50 96 MB 0.777 0.938 25,097,128 InceptionResNetV2 215 MB 0.803 0.953 55,873,736 MobileNetV2 14 MB 0.713 0.901 3,538,984 DenseNet121 33 MB 0.750 0.923 8,062,504 NASNetMobile 23 MB 0.744 0.919 5,326,716 It is observed that Xception[13] and MobileNetV2[14] are of the best comprehensive performance among all the network models, in which the top-1 accuracy in Xception network can reach 79%, and the MobileNetV2 model has the least params and the top-1 accuracy can reach 71. 3%. The running time of the porn streamer recognition model in live video can be optimized in two aspects as follows: 1) The higher time consumption of multimodal data processing mainly includes the extraction time of video frame, optical flow image and spectrogram; 2) The model-based reasoning time refers to the time computation that the model extracts features and outputs categories after it accepts inputs. Therefore, we will propose two types of multimodal knowledge distillation networks from the above two aspects with the Xception model and MobileNetV2 model. 1) Multimodal knowledge distillation network based on Xception In order to extract visual-speech information from live video effectively, we use the video frame, optical flow image and spectrogram in live video to respectively to extract spatial, motion and speech features to recognize the porn streamer. Although such a trick can promote the model recognition performance, the process of extracting optical flow image and spectrogram in the data preprocessing indeed takes time. Besides, the model has to process video frame, optical flow image and spectrogram simultaneously, which greatly raises the computation. The advantage of the knowledge distillation technology is to utilize the knowledge storage with soft targets and transfer it to the homogeneous/heterogeneous student model. Accordingly, a student model designed with the video frame as the input may effectively reduce the time consumption caused by data preprocessing and multimodal data size. In addition, since the model only accepts the inputs of video frame, the student model must be constructed based on the model with strong recognition capability, which will ensure the accuracy of recognition while improving the recognition speed. Based on the above analysis, the Xception network of knowledge distillation (called KD-Xception)is proposed as shown in Fig.3. First, the visual-speech deep network is construct by multimodal attention-gated CNN + Bi-GRU as teacher model, in which the multimodal video features are extracted from visual and speech information, such as video frames, optical flow images and spectrograms. The soft targets containing visualspeech information are computed by softmax function. Then, the KD-Xception only takes video frames as input and uses soft targets and hard targets for training in order to reduce the time consumption of data preprocessing effectively and improve the discriminative ability of the student model. Finally, the prediction score of KD-Xception and the bullet screen text network are fused at the decision-level to recognize porn streamer to improve the recognition speed while maintaining the accuracy. Fig. 3Open in figure viewerPowerPoint Architecture of multimodal knowledge distillation network based on Xception 2) Multimodal knowledge distillation network based on MobileNetV2 In the porn streamer recognition task, the timeconsuming come from the data processing and the model-based reasoning. Therefore, a lightweight visualspeech network can be designed as the student model to effectively improve the speed of model-based reasoning. MobileNetV2 now is one of the most outstanding lightweight models that is characterized by a faster reasoning speed and an impressive accuracy. Thus, the MobileNet network of knowledge distillation (called KD-MobileNet)is established based on MobileNetV2, which is designed to process inputs of video frame, optical flow image and spectrogram respectively with three parallel MobileNetV2 models. Then, the concatenate function is utilized to the “channel” dimension of three feature maps that are the outputs of three different models. This hybrid features are then applied to the subsequent processing as multimodal features to improve the recognition accuracy effectively. Finally, after combining the KD-MobileNet and bullet screen text recognition network to construct a multimodal deep network, the porn streamers by softmax function are recognized as shown in Fig.4. Fig. 4Open in figure viewerPowerPoint Architecture of multimodal knowledge distillation network based on MobileNetV2 3) Multimodal deep network compression for porn streamer recogition Next, the knowledge distillation is utilized to compress the multimodal deep model. First, the visual-speech deep network and bullet screen text recognition network are trained. The classification activation function of visual-speech deep network as the teacher model is changed to softmax function with T = 5 , and soft targets with category knowledge are extracted from video frames, optical flow images and spectrograms by using teacher model. Then, KDXception and KD-MobileNet are applied as student models. Finally, the distillation loss function is used to guide the student model to fit the probability distribution in soft targets. During the training process, knowledge information is transferred from teacher model to student model. The distillation loss function L is defined as follows: L = α L soft + β L hard (2) where α , β are adjustment parameters, L soft is the loss function for learning soft targets. By minimizing L soft , the output distribution of student model is continuously approaching the output distribution of teacher model as follows: L soft = - ∑ i N p i T log q i T (3) where N is the output dimension of teacher model and student model, p i T as Eq.(4) is the output of the i-th dimension of teacher model when temperature is T, as known as soft targets, and q i T as Eq.(5) is the output of the i-th dimension of student model when temperature is T. p i T = exp x i t / T ∑ k N exp x k t / T (4) q i T = exp x i s / T ∑ k N exp x k s / T (5) where x k t is the output of teacher model before softmax of the k-th dimension and x k s is the output of student model before softmax of the k-th dimension. Student model continuously learns the probability distribution of teacher model by L soft loss function, but the output of the teacher model is not absolutely correct. To compensate for this deviation, L hard as Eq.(6) is used to optimize the distillation loss. L hard = - ∑ i N g i log q i 1 where q i 1 = exp x i s ∑ k N exp x k s (6) where g i is the ground truth of the i-th category as known as hard targets, q i 1 is the output probability of the student model of the i-th category when temperature is 1. KD-Xception and KD-MobileNet can learn knowledge from soft targets and hard targets by using distillation loss function. Then, the prediction results of the student model and bullet screen text network are fused at the decision level as follows: P last = a P V + b P T (7) where a , b are the adjustment parameters, P V is the prediction results of the student model and P T is the prediction results of the bullet screen text network. At last, the recognition performance is improved by the above knowledge distillation method effectively. III. Experimental Results and Analysis In order to verify the effectiveness and feasibility of the proposed method, three experiments are conducted. The experimental platform is a PC with 3.40GHZ CPU, 16GB RAM, Ubantu16.04 LTS operating system and NVIDIA TITAN XP GPU. The framework is implemented by Python 3.6 and Keras2.2.4, accelerated by CUDA and Cudnn. In Experiment I, the accuracy of porn streamer recognition based on multimodal knowledge distillation is compared with the state-of-the-art methods. In Experiment II, we compare the parameters and model size of proposed porn streamer recognition model with the state-of-the-art methods. In Experiment III, the recognition speed of porn streamer recognition in live video based on multimodal knowledge distillation is evaluated by a series of experiments. In all experiments, parameter α , β are all set to 0.5 as empirical value. To illustrate our work, we collected a real-world data from Internet to build Beijing University of Technology Streamer Dataset Version 2 (BJUT-SD V2) dataset, which contains 1154 normal, 2528 porn streamer live videos and 120000 bullet screen text as shown in Fig.5. Fig. 5Open in figure viewerPowerPoint Examples of the BJUTSD V2 dataset 1. Experiment I: Accuracy comparison of porn streamer recognition based on multimodal knowledge distillation In this experiment, the accuracy of porn streamer recognition based on multimodal knowledge distillation is compared with the state-of-the-art methods. We present the objective results of porn streamer recognition with different modality on BJUT-SD V2 as shown in Table 2, in which V s , V m , S, and T represent spatial, motion, speech and text modality respectively. Here, according to the design of each network structure, C3D[15], I3D[16], NLResNet50[17] and MMMU-BA[18] adopt combinations of V s , V s + V m , V s and V s + S , and KD-Xception only accepts the video frames (Vs modality) as input. From Table 2, the accuracy of the proposed KD-MobileNet+BERT is 66.26%, higher than other methods except the teacher model. When compared with the model before distilling, the accuracy only drops by 2.98%. The proposed KD-Xception+BERT also the second most accurate. The accuracy of KD-Mobile Net V s + BERT drop by 5.39% and 5.48% respectively compared with KD-Xception+BERT and KD-MobileNet+BERT, which demonstrate the necessity of using complex backbone networks in KD-Xception and the necessity of utilizing multimodal information in KD-MobileNet. The experimental results show that the proposed method has better performance and its loss is little compared to the model before distillation. Table 2. Accuracy comparison of porn streamer recognition in live video based on multimodal knowledge distillation Method Modality Accuracy (%) C3D V s 60.23 I3D V s + V m 62.47 NL-ResNet50 V s 54.30 MMMU-BA V s + S 48.85 Before distillation V s + V m + S + T 69.24 KD-MobileNet + BERT (w/out V m &S) V s + T 60.78 KD-Xception + BERT V s + T 66.17 KD-MobileNet + BERT V s + V m + S + T 66 . 26 2. Experiment II: Parameter number and model size comparison of multimodal knowledge distillation based visual-speech network In Expriment II, we verify the effect of knowledge distillation on visual-speech network model, and the comparison results of parameter number and model size are shown in Table 3. Although the parameter numbers of KD-Xception+BERT and KD-MobileNet+BERT are larger, most of the parameters are in BERT model that used to process bullet screen text. Without BERT model, the parameter number of KD-Xception is 22910480 and its model size is about 88MB. The KD-MobileNet is more lightweight than KD-Xception, the parameter number is 8292964 and its model size is about 49. 3MB. The KD-Xception and the KD-MobileNet have fewer parameters than the C3D, I3D, and NL-ResNet50 network, and the model size of them are also smaller. MMMU-BA model has the least parameters and the smallest model. This is mainly because the MMMUBA is based on GRU and has a simple structure. Compared with the network before knowledge distillation, KD-Xception and KD-MobileNet reduce the parameters and model size significantly. After knowledge distillation, the proposed method transfers the knowledge from complex teacher model to the lightweight student model, thereby reducing the calculation and storage cost while maintaining performance. Table 3. Parameter number and model size comparison of multimodal knowledge distillation based visual-speech network Method Modality Parameter Size C3D V s 79,991,015 299 MB I3D V s + V m 25,000,000 102 MB NL-ResNet50 V s 30,764,050 242 MB MMMU-BA V s + S 2,298,902 27 . 7 MB Before distillation V s + V m + S 35,586,382 192 . 7 MB KD-Xception V s 22,910,480 88 MB KD-MobileNet V s + V m + S 8,292,964 49 . 3 MB KD-Xception+BERT V s + T 131,719,184 503 MB KD-MobileNet+BERT V s + V m + S + T 117,101,668 607 MB 3. Experiment III: Running time comparison of porn streamer recognition in live video In this experiment, we compare the running time of our method with the state-of-the-art methods on BJUT-SD V2 dataset, as shown in Table 4. The running time of this method was obtained by inputting a series of video frames and calculating the average of their processing time. The running time of the proposed porn streamer recognition network based on KD-Xception and bullet screen text network is faster than the I3D, NL-ResNet50. When compared with the model before distillation, the running time reduces 627.1s. The running time of the proposed KD-MobileNet+BERT is 728.6s reduced by 75.7s after knowledge distillation. This is because porn streamer recognition model before distillation and KD-MobileNet needs to process video frames, optical flow images, spectrograms and bullet screen text simultaneously, while KD-Xception only needs to process video frames and bullet screen text, and the amount of input data is reduced about 2/3. Since the network structure is simplified, the running time is greatly improved. Based on the above results, we can see that the proposed porn streamer recognition based on multimodal knowledge distilling can improve the running time under the proper accuracy. Table 4. Running time comparison of porn streamer recognition in live video based on multimodal knowledge distillation Method Modality Running time(s) C 3 D V s 25.06 I3D V s + V m 923.54 NL-ResNet50 V s 176.32 MMMU-BA V s + S 116.03 Before distillation V s + V m + S + T 801.3 KD-MobileNet + BERT (w/out V m &S) V s + T 155.7 KD-Xception+BERT V s + T 174.2 KD-MobileNet+BERT V s + V m + S + T 728 . 6 IV. Conclusions In this paper, a porn streamer recognition in live video based on multimodal knowledge distillation is proposed. we utilize visual-speech deep network as a teacher model to extract soft targets. Then a lightweight visual-speech network is trained using soft and hard targets and knowledge is transferred from the teacher model to the student model. Finally, by following the multimodal deep feature fusion strategy, we fuse the outputs of the student model and output of the bullet screen text recognition network at the decision-level. Three experiments are conducted to evaluate the performance of our porn steamer recognition network based on multimodal knowledge distillation compared with the state-of-the-art methods in the light of accuracy, running time, parameter number, and model size. The experimental results show that the proposed porn steamer recognition in live video based on multimodal knowledge distillation can obtain an encouraging performance, which effectively drop the computation cost and improve the recognition speed under the proper accuracy. In the future work, we will optimize our model compression strategy and compress the bullet screen text recognition network to build a simpler model. We will also consider about combining knowledge distillation with model quantification strategies to optimize the model structure, reduce the parameter and further improve the recognition speed. Biographies WANG Liyuan is a postgraduate at the Faculty of Information Technology, Beijing University of Technology (BUT). His research interest is video processing and analysis. ZHANG Jing (corresponding author) is a professor at the Beijing University of Technology. She received her Ph.D. degree in circuit and system from the Beijing University of Technology in 2008. She is the author of more than 60 journal papers and has written four book chapters. Her current research interests include image/video processing, recognition, and retrieval. YAO Jiacheng is a postgraduate at the Faculty of Information Technology, BUT. His research interest is video processing and analysis. ZHUO Li is a professor at BUT. She received the Ph.D. degree in pattern recognition and intelligent system from BUT in 2004. Her current research interests include image/video processing, image recognition, and medical image processing. References 1X. Sun, L. Ma and G. Li, “Multi-vision attention networks for on-line red Jujube grading”, Chinese Journal of Electronics, Vol. 28, No. 6, pp. 1108– 1117, 2019. Wiley Online LibraryWeb of Science®Google Scholar 2D. Meng and L. Sun, “Some new trends of deep learning research”, Chinese Journal of Electronics, Vol. 28, No. 6, pp. 1087– 1091, 2019. Wiley Online LibraryWeb of Science®Google Scholar 3L. Wu, Y. Rao, H. Yu, et al., “A multi-semantics classification method based on deep learning for incredible messages on social media”, Chinese Journal of Electronics, Vol. 28, No. 4, pp. 754– 763, 2019. Wiley Online LibraryWeb of Science®Google Scholar 4J. Wehrmann, G.S. Simões, R.C. Barros, et al., “Adult content detection in videos with convolutional and recurrent neural networks”, Neurocomputing, Vol. 272, No. 1, pp. 432– 438, 2018. CrossrefGoogle Scholar 5N. Takahashi, M. Gygli and L.V. Gool, “AENet: Learning deep audio features for video analysis”, IEEE Transactions on Multimedia, Vol. 20, No. 3, pp. 513– 524, 2018. CrossrefWeb of Science®Google Scholar 6Y. Cheng, D. Wang, P. Zhou, et al., “Model compression and acceleration for deep neural networks: The principles, progress, and challenges”, IEEE Signal Processing Magazine, Vol. 35, No. 1, pp. 126– 136, 2018. CrossrefWeb of Science®Google Scholar 7P. Luo, Z. Zhu and Z. Liu, “ Face model compression by distilling knowledge from neurons”, AAAI Conference on Artificial Intelligence, Phoenix, USA, pp. 3560– 3566, 2016. Google Scholar 8G. Hinton, O. Vinyals and J. Dean, “ Distilling the knowledge in a neural network”, arXiv preprint, arXiv:1503.02531, 2015-3-9 Google Scholar 9L. Wang, J. Zhang, Q. Tian, et al., “Porn streamer recognition in live video streaming via attention-gated multimodal deep features”, IEEE Transactions on Circuits System Video Technology, https://doi.org/10.1109/TCSVT.2019.2958871, 2019. Google Scholar 10L. Wang, J. Zhang, M. Wang, et al., “Multilevel fusion of multimodal deep features for porn streamer recognition in live video”, Pattern Recognition Letters, https://doi.org/10.1016/j.patrec.2020.09.027, 2020. Google Scholar 11J. Devlin, M.W. Chang, K. Lee, et al., “ BERT: Pre-training of deep bidirectional transformers for language understanding”, Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, pp. 4171– 4186, 2019. Google Scholar 12S. Bhutani, “ Keras documentation”, https://keras.io/api/applications/mobilenet/#mobilenetv2-function, 2020-9-29. Google Scholar 13F. Chollet, “ Xception: Deep learning with depthwise separable convolutions”, IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 1800– 1807, 2017. CrossrefGoogle Scholar 14M. Sandler, A. Howard, M. Zhu, et al., “ Mobilenetv2: Inverted residuals and linear bottlenecks”, IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp. 4510– 4520, 2018. CrossrefGoogle Scholar 15D. Tran, L. Bourdev, R. Fergus, et al., “ Learning spatiotemporal features with 3D convolutional networks”, IEEE International Conference on Computer Vision, Santiago, Chile, pp. 4489– 4497, 2015. CrossrefGoogle Scholar 16J. Carreira and A. Zisserman, “ Quo vadis, action recognition? A new model and the kinetics dataset”, IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 4724– 4733, 2017. CrossrefGoogle Scholar 17X. Wang, R. Girshick, A. Gupta, et al., “ Non-local neural networks”, IEEE Conf. on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp. 7794– 7803, 2018. CrossrefGoogle Scholar 18D. Ghosal, M.S. Akhtar, D. Chauhan, et al., “ Contextual intermodal attention for multi-modal sentiment analysis”, Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 3454– 3466, 2018. Google Scholar Volume30, Issue6November 2021Pages 1096-1102 FiguresReferencesRelatedInformation