Detect Profane Language in Streaming Services to Protect Young Audiences (2021)

Abstract: With the rapid growth of online video streaming, recent years have seen increasing concerns about profane language in their content. Detecting profane language in streaming services is challenging due to the long sentences appeared in a video. While recent research on handling long sentences has focused on developing deep learning modeling techniques, little work has focused on techniques on improving data pipelines. In this work, we develop a data collection pipeline to address long sequence of texts and integrate this pipeline with a multi-head self-attention model. With this pipeline, our experiments show the self-attention model offers 12.5% relative accuracy improvement over state-of-the-art distilBERT model on profane language detection while requiring only 3% of parameters. This research designs a better system for informing users of profane language in video streaming services.

## introduction streaming services such as netflix and prime video have dramatically changed the media habits of young people, with six-in-ten primarily watching television today with streaming services @xcite . the increased exposure of online content has raised concerns about profane language appeared in these contents @xcite @xcite . exposure to profane language can increase aggressive thoughts, angry feelings, physiological arousal, and aggressive behavior @xcite . profane language is a type of language that includes dirty words, swearing, and obscenity contents. previous research has focused on developing automated techniques to detect profane language in user generated contents on social media. for example, there have been growing interests in detecting hate speech and racism on twitter @xcite @xcite . some recent works have also studied offensive contents in youtube @xcite . however, few studies have focused on profane language detection in streaming services that host movies and tv shows. recent works have shown the importance of data techniques such as pre-processing and augmentation in improving machine learning models. for example, there has been research on applying transfer learning or semi-supervised learning for learning word embedding and addressing insufficient data issues in tasks with limited sample sizes @xcite . in addition, using text pre-processing methods such as text normalization, lowercasing, lemmatizing, tokenizing and multiword grouping can help increase sentiment, topic and polarity classification accuracy @xcite . however, few studies have focused on improving data techniques to better handle long sequence of text appeared in streaming videos. research on addressing data issue have primarily focused on improving data quantity, rather than quality. also, our problem has its novelty in that the data sets of most previous studies are on written text, which can have a different distribution from the spoken-form video captions. in this work, we study the problem of profane language with a specific online video streaming service, amazon prime video (pv), as an example. specifically, we develop a data pipeline that can be integrated sentence level model to automatically predict the level of profanity in video titles according to their caption contents. we collect data from both the targeted service and public data set, and augment training data by merging multiple data sources. our experiments show that this data collection pipeline that can be used to address long sequence of text and help non-hierarchical models to achieve state-of-the-art performance. using this pipeline, we train a multi-head self-attention model on embedding pre-trained on pv caption dataset, and show this simple self-attention model (with 2 million parameters) can outperform the pre-trained distilbert model (with 66 million parameters) that is fine-tuned on the same dataset by 9% accuracy. ## related work profane language is a type of language that includes dirty words, swearing, and obscenity contents. previous research in this area has primarily focused on detecting profane language in social media. for example, a recent work studied the diffusion of profanity in sina weibo, one of the largest chinese social media platforms @xcite . research on abusive and hate speech detection (a close related research area to profane language detection) has focused on developing automatic techniques to identify racists and sexist on twitter @xcite , reddit @xcite , and youtube @xcite . however, few studies have focused on detecting profane language in video stream services such as netflix, hulu, and prime video. research on this area has also shifted from using traditional machine learning methods to using deep learning methods. for example, while early work uses traditional machine learning classifiers, such as logistic regression, support vector machine, and tree-based methods @xcite , there has been a growing interest in applying lstm, cnn, and bert (bidirectional encoder representations from transformers) for detecting racism, sexism, hate, or offensive content @xcite @xcite . however, one of the limits of these deep learning method is its capacity of working memory. this is because processing long texts (even for bert) will quadratically increasing memory and time consumption and slicing the text by a sliding window or simplifying transformers, suffer from insufficient longrange attentions or need customized cuda kernels @xcite . due to this issue, these prior methods are not directly applicable for detecting profane language in video stream services since the video captions are often very long, with an average length of hundreds of sentences per video. ## data our data collection pipeline aims to address two challenges. first, there is no labeled data for profane language detection task in streaming service. to address this challenge, we collect data from a popular streaming service, design an annotation guideline, and hire human annotators to label the levels of profanity in video titles. then, we augment training data by collecting additional labeled contents from a publicly available database. last, we perform pre-processing on the collected data to improve its quality. second, the title caption is so long that it is very challenging for a model to learn from context. to address this challenge, we develop a pipeline for creating sentence-level profanity labels using domain knowledge and title-level labeling information. we provide details about our data collection pipeline in the sections below. ## experiments we integrated several models to our data pipeline and conducted experiments at both title level and sentence level. for the title level methods, we include xgboost and logistic regression, and deep learning methods such as an augmented version of the hierarchical attention network. we will introduce more details of these methods below. for the sentence level methods, we apply distilbert, rule based method that is used to create labels, and a sentence level multi-head attention model with and without the knowledge distillation soft target step. the purpose is to check whether the model learns the context information well, and whether the soft target helps. for each method, we have also fitted the model with only the 2.6k titles from prime video to study the effects of augmented data from mpaa. ## results in this section, we present our experiment results for detecting the presence of profane language and the level of profane language for video titles. ## conclusion in this paper, we presented a data collection pipeline to generate the high quality sentence-level label for profane language detection in a streaming service. this pipeline included specific knowledge distillation and active learning ideas to refine such labels. we applied data augmentation, collected training data from both the targeted streaming service and public open source, and applied a workflow to fill the gap caused by the rating policy inconsistency. we built a multi-head self-attention model for sentence level detection and aggregated the detections to title level for rating prediction. the experiment showed the proposed model outperformed all the baselines including the hierarchical attention network and distilbert, and also beat the rules that created the labels. in addition, the output attention weights showed success in locating the right keywords. future research directions include the exploration on how the proposed pipeline will help detect more general profanity defined in multimodality, such as visual and audio.