Adult Content Detection on A rabic T witter: Analysis and Experiments (2021)

Abstract: With Twitter being one of the most popular social media platforms in the Arab region, it is not surprising to find accounts that post adult content in Arabic tweets; despite the fact that these platforms dissuade users from such content. In this paper, we present a dataset of Twitter accounts that post adult content. We perform an in-depth analysis of the nature of this data and contrast it with normal tweet content. Additionally, we present extensive experiments with traditional machine learning models, deep neural networks and contextual embeddings to identify such accounts. We show that from user information alone, we can identify such accounts with F1 score of 94.7% (macro average). With the addition of only one tweet as input, the F1 score rises to 96.8%.

## introduction disclaimer: due to the nature of this research, we provide examples that contain adult language. we follow academic norms to present them in an appropriate form, however the discretion of the reader is cautioned. in recent years, twitter has become one of the most popular social media platforms in the arab region @xcite . on average, arab users post more than 27 million tweets per day @xcite . such popularity has also spawned a number of spammers who exploit the popularity to post malicious content. such malicious content may contain pornographic references or advertisement. we refer to such content as adult content. adult content may have deliberating effects on many, particularly among those of younger age groups. users who fall for the pornographic advertisements are at risk of losing money and sensitive information to the spammers. due to the massive amount of user-generated content on twitter, it is impossible to detect such accounts manually and this calls for automatic detection -the focus of this paper. twitter's policy prohibits users from posting adult content foot_0 . however, the methods deployed for detecting spams such as adult content are mostly expanded from english and are not very effective for detecting accounts who post adult content in other languages as such as the case of arabic @xcite . traditional methods such as filtering by list of words are not effective since spammers use smart ways such as intentional spelling mistakes to evade such filtering @xcite . despite the dire need of eliminating adult content from arabic social media, there has been a very few notable works @xcite @xcite in the field. in contrast to the existing work @xcite @xcite that rely on extracting collection of tweets from each account to classify whether they post adult content, we present a dataset and several models aimed at classifying accounts based on minimal information. by minimal information, we mean user information such as username, user description or just one random tweet from each account. in our study, we focus on using textual features to detect adult content and we leave multimedia (e.g. images) and social network features for future work. our dataset consists of 6k manually annotated twitter accounts who post adult content and 44k ordinary twitter accounts in addition to a tweet from each account (a total of 50k accounts and tweets). we perform extensive analysis of the data to identify characteristics of these accounts. lastly, we experiment extensively with traditional machine learning models such as support vector machines (svm) and multinomial naive bayes (mnb), deep learning models such as fasttext and contextual embedding models (bert). we analyze contribution of each information available (username, user description, or single tweet) to the performance of the models. since accounts that post adult-content want to attract others, their usernames and user descriptions are often catchy and contain references that are indicators of them posting adult content. we demonstrate that with just username and user description, we can detect these accounts with macro-averaged f1 score of 94.7%. with addition of single tweet as available information, we achieve macro-averaged f1 score of 96.8%. detecting accounts who post adult content with minimal information (e.g. from username or description) will allow such accounts to be detected early and possible warning messages can be sent to users to protect them from potential harm or inappropriateness. the contribution of this work can be summed as: 1) providing the largest dataset of twitter accounts that is manually annotated for adult content detection in arabic, and we make it available for researchers. 2) exploring the dataset to learn silent features used in the domain as well as features related to users and their profiles. we show that user information can be used for early detection of adult accounts even before tweeting, and when they are combined with tweet text, results are improved. the paper is structured as follows: in section 2, we discuss related work in the field. in section 3, we describe the data collection method and present analysis of the data. in section 4, we present our experimental setups and results. in section 5, we examine features learned by our best model and perform error analysis that provides insight on how to improve the data and models in the future. lastly, in section 6, we present conclusions of our work. ## related work despite the fact that many social media platforms enforce rules and conditions about the content shared on their platforms, malicious users attempt to circumvent these rules and guidelines. researchers have attempted different approaches for exposing malignant content. spam detection in particular has gained a lot interest among researchers (e.g., @xcite @xcite @xcite ). spam detection is a generalized approach for detecting unsolicited messages. our focus in this paper is on the more concentrated field of detecting adult-content, which categorically includes pornographic references. for english language, there is a number of works devoted to adult-content detection in terms of analyzing the social networks or the content itself. mitchell et al. @xcite study exposure to adult content and its relation to age/gender. singh et al. @xcite propose random forest classifier to detect pornographic spammers on twitter. cheng et al. @xcite propose an iterative graph classification technique for detecting twitter accounts who post adult content. harish et al. @xcite study deep learning based methods for detecting inappropriate content in text. in arabic, however, the field of adult-content detection is still relatively unexplored. a related field that has been explored recently in arabic is abusive/hate-speech detection. there has been a few recent works @xcite @xcite in the areas of offensive and hate-speech detection. however, offensive language and hate-speech have few fundamental differences with adult-content. while offensive language and hate-speech typically consist of profanity and attack on individuals or groups, adult-content may contain profanity but primarily consist of pornographic references. more concentrated work on adult-content detection have been conducted by @xcite @xcite . in @xcite , a list of hashtags was used to automatically construct dataset of tweets that contain adult content. in @xcite , 500 twitter accounts were manually annotated for adult-content posts. both @xcite use traditional machine learning models such as support vector machine (svm) or multinomial naive bayes (mnb) for classification. using statistical features of tweet text for classification was proposed in @xcite . although @xcite perform some analysis of screennames, they do not use them or any other user information for classification. while @xcite explore number of tweets, followers and following by the accounts, they do not utilize username or user description either. these works rely on collection of tweets from each user for classification. ## data description we describe the method used to collect the dataset, some statistics and observations about it including most frequent words, emojis and hashtags. we show also the geographical distribution of adult accounts and some differences between our dataset and previous datasets. ## experiments to train classifiers for automatic detection of adult tweets, we split the data into training set (70%), development set (10%), and a test set (20%). we experiment extensively with 1) different classifiers, and 2) dataset variants with different degree of information available about accounts. although we conducted experiments on different preprocessing techniques such as removing diacritics or normalizing arabic text, we did not notice any significant improvement (between 0.1%-0.2%) in performance. we omit these experiments to make room for more significant results. ## error analysis the confusion matrix of predictions by our best system, arabert trained on user information+tweet, is shown in non-adult accounts that are detected as adult: this occurred 65 times. we found that in 70% of these cases, they were annotated incorrectly in the reference, for example when an account has (cash and serious) in either user information or tweet text, this account should be marked as adult as such term is commonly used by adult accounts. this suggests that automatic classification can be used iteratively to detect possible annotation errors. the rest of the errors were due to the existence of frequently-used words in adult accounts such as (massage) but these words can be used also by non-adult accounts. adult accounts that are detected as non-adult: this occurred 74 times. only 7 of these cases were due to errors in the reference annotation while majority of errors were due to: i) using unseen words in the training data (ex: creative spelling of some dialectal adult words); ii) complex cases where combining features in user profiles can intuitively reveal adult accounts to human annotators, e.g. when a screen name is "k3eut8i8t3pfmy..." and the user described himself as (extraordinary romantic) and the tweet is an invitation to come in private. for classifiers, it maybe difficult to capture such complex intuition. ## conclusion we presented a dataset for detecting twitter accounts who post adult content in arabic tweets. we performed extensive analysis of the data to identify characteristics of such accounts. in our experiments, we have shown that support vector machines and contextual embedding models arabert and multilingual bert can detect these accounts with impressive reliability while having access to minimal information about the accounts. in the future, we aim to explore if similar methods can be adopted to identify accounts who post other variants of undesirable content such as unsolicited advertisement. also, we plan to experiment tools that detect adult content in multimedia (e.g. in images) and compare performance with our model that depends only on textual information.