A Stacking-based Efficient Method for Toxic Language Detection on Live Streaming Chat (2022)

Abstract: In a live streaming chat on a video streaming service, it is crucial to filter out toxic comments with online processing to prevent users from reading comments in real-time. However, recent toxic language detection methods rely on deep learning methods, which can not be scalable considering inference speed. Also, these methods do not consider constraints of computational resources expected depending on a deployed system (e.g., no GPU resource).This paper presents an efficient method for toxic language detection that is aware of real-world scenarios. Our proposed architecture is based on partial stacking that feeds initial results with low confidence to meta-classifier. Experimental results show that our method achieves a much faster inference speed than BERT-based models with comparable performance.

## introduction with the rapid growth of online social platforms, posting text comments to a content have become a familiar part of people's lives for growing human connection and advertising purposes. however, these comments can include toxic language, which can be harmful or offensive to others. toxic comments lead to damages of the user experience, human well-being, and even product promotion. particularly, toxic language is a very common problem in live feeds on video streaming services (e.g., youtube) @xcite . toxic comments can appear more frequently in live broadcasting, since users tend to impulsively post comments in real-time with less introspection during live streaming @xcite . it is not possible to manually rule out toxic comments from a large number of comments continuously posted across multiple live feeds. we aim for an automatic detection system to address toxic comments, such as figure 1 . to capture powerful latent features for detecting toxic language, recent existing methods rely on deep learning techniques such as bert. although the techniques have performed well on the task, the following issues can be raised when assuming application to comments on live streaming. 1. inference speed: during live streaming, many posted comments can stream to all viewers in realtime. to promptly prevent both viewers and video contributors from reading toxic comments, online processing should be crucial. however, deep learning-based methods might not be scalable for online processing due to slow inference speed. 2. computation resource: a computational resource-friendly method is essential from an industry point of view. even if the inference speed is satisfied with one or more gpus at the poc phase, there should be computational resource constraints, depending on the requirements of a deployed system when assuming real-world operations, such as api on a mobile application. to make up for the above two issues, we propose an efficient method for toxic comment detection on live streaming chat. our target includes offensive, insulting, and obscene expressions, similar to the expressions used by @xcite . we handle japanese comments since our method is meant to be deployed on a japanese live streaming service. our proposed architecture is a stacking-based two-layer classification model in which detection results with lower confidence scores in fasttext classification are re-classified by lightgbm with five features. thus, our method does not require a gpu environment. that enables a developer to facilitate deploying a system with low computation resources and enough reproducibility. experimental results show that our method achieves up to 17 times faster inference speed than several bertbased methods with a comparable f1 score under an online processing setting. ## related work many methods for toxic comment detection have been proposed, mainly for twitter @xcite @xcite @xcite and comments to online news articles @xcite . traditional methods use lexical patterns based on offensive words @xcite and opinion words @xcite . since malicious viewers create various toxic words in diverse writing styles, this approach is not robust to variants of words in the lexicon. in addition, it is costly to regularly update the lexicon to keep a deployed toxic detection system accurate @xcite . in recent years, many methods utilize deep learning-based methods such as bert and lstm, using sentiment information (brassard-gourdeau and khoury, 2019; @xcite @xcite , topic contents @xcite , and context information such as text replies @xcite and attention-based context vectors @xcite . @xcite explored how bertbased models affect the relationship between performance and fairness for toxic comment detection. here, fairness means equalized performance across various sensitive groups such as religion and race. however, none of the studies in this section explore performance that consider inference speed and computational resources for toxic comment detection for real-world applications. comment characteristics in video live streaming chats are fundamentally different from twitter posts and news articles. there is no information on replies (i.e., parent-child relationship) in live streaming chats. moreover, the comments are often short, which lacks context and topic information. according to @xcite , 63% of messages in live streaming chats on youtube contained fewer words than the average word count of 8 with standard deviations of 7. in video live streaming chat on twitch foot_0 , @xcite applied a fine-tuned roberta model to toxic comment detection. we show the efficiency of our method by making a comparison with various bert-based models. edge computing can be an applicable solution to address latency and scalability challenges for nlp services with deep learning @xcite . there is a wide range of deployed candidates for edge computing architectures and deep learning models. the deployment should be carefully considered to accomplish system requirements. thus, we leave the applicability of deep learning with edge-computing in our task for future research. ## data collection we create annotated data for toxic comment detection on the video live streaming domain. we use "niconico-doga comment data" ( @xcite provided by the national institute of informatics foot_1 . niconico-doga is one of the largestscale japanese video streaming services. in the whole dataset, we used the file lists from 0000.zip to 0005.zip, and labeled them as toxic/non-toxic comments via human annotation. in total, 168,071 comments were annotated, which comprise 21,156 toxic comments and 146,915 non-toxic comments. we randomly divided the annotated dataset into a training set, development set, and test set at a ratio of 80%, 10%, and 10%, respectively. the statistics are shown in figure 3 shows the architecture of our proposed method, which comprises two layers. we first use fasttext classification model @xcite as described in §4.2 in order to prioritize fast inference speed. our preliminary experiments with our dataset showed that the lower the prediction probability for the fasttext classification, the lower the f1 score. thus, there is much room for improvement in results of lower prediction probability as described in the triangle area in figure 4 . the results motivate us to utilize another classification model for the case where the fasttext model has less confidence. after getting classification results by the fast-text, we use the stacking technique @xcite . this technique is a simple but effective way that predictions of different classifiers are fed to a meta-level classifier to generate final results. for efficiency, the second classifier is applied only to the first results with lower confidence. we figure 4: macro f1-score by prediction probability for fasttext classification model will describe the meta classifier and used features in §4.3. ## conclusion although many methods have been proposed to detect toxic comments on online social platforms, these methods have paid no attention to inference speed and constraints of computational resources for real-world applications. we presented a fast and computational resource-friendly method. our method does not require gpu resources, which faclitate being adjustable with a requirement of a deployed system. we proposed a two-layer clas-sification model that efficiently utilizes stacking techniques with five features. experimental results showed that all of the proposed features were effective independently. under the online processing setting, our method achieved a much faster inference speed than fine-tuned bert-based methods, with the comparable f1 score. our method is going to be deployed on our service soon. we leave the issues raised in error analysis for future research.