Hater- O -Genius Aggression Classification using Capsule Networks (2020)

Abstract: Contending hate speech in social media is one of the most challenging social problems of our time. There are various types of anti-social behavior in social media. Foremost of them is aggressive behavior, which is causing many social issues such as affecting the social lives and mental health of social media users. In this paper, we propose an end-to-end ensemble-based architecture to automatically identify and classify aggressive tweets. Tweets are classified into three categories - Covertly Aggressive, Overtly Aggressive, and Non-Aggressive. The proposed architecture is an ensemble of smaller subnetworks that are able to characterize the feature embeddings effectively. We demonstrate qualitatively that each of the smaller subnetworks is able to learn unique features. Our best model is an ensemble of Capsule Networks and results in a 65.2% F1 score on the Facebook test set, which results in a performance gain of 0.95% over the TRAC-2018 winners. The code and the model weights are publicly available at https://github.com/parthpatwa/Hater-O-Genius-Aggression-Classification-using-Capsule-Networks .

## introduction even though social media offers several benefits to people, it has caused some negative effects due to the misuse of freedom of speech by a few people. aggression is a behavior that is intended to harm other individuals who do not wish to be harmed @xcite . aggressive words are commonly used to inflict mental pain on the victim by showing covert aggression, overt aggression or by using offensive language @xcite . the process of manually weeding out aggressive tweets from social media is expensive and in-definitely slow. so, there is a growing need to build and analyze automatic aggression classifiers. in this paper, we propose an architecture that is an ensemble of multiple subnetworks to identify aggressive tweets, where each subnetwork learns unique features. we explore different word embeddings for dense representation @xcite , deep learning (cnn, lstm), and capsule networks @xcite . our best model (figure 1 ) uses capsule network, and gives a 65.20% f1 score, which is a 0.95% improvement over the model proposed by @xcite . we also release the code and the model weights. ## related work the challenge of tackling antisocial behavior like abuse, hate speech, and aggression on social media has recently received much attention. researchers like @xcite tried detecting abusive language by using machine learning and linguistic features. other researchers like @xcite used cnns and lstms, along with gradient boosting, to detect hate speech. the trac-2018 shared task @xcite , aimed to detect aggression, was won by aroyehun and gelbukh (2018), who used deep learning, data augmentation, and pseudo labeling to get a 64.25% f1 score. another team @xcite , used deep learning along with data augmentation and hand-picked features to detect aggression. however, in order to develop an endto-end automated system, one cannot use handpicked features as they may vary from dataset to dataset. @xcite experimented with capsulenets for detecting aggression and achieved a 63.43% f1 score. our work differs from theirs as we experiment with architectures (fig. 1 ) that are an ensemble of multiple subnetworks. recently, @xcite used pooled bil-stm and nlp features to achieve 67.7% f1 score on the trac-2018 facebook data. the trac-2020 shared task @xcite released a data set @xcite of aggression and misogyny in hindi, english and bengali posts. @xcite tried an ensemble of bert to achieve the best performance on most tasks. safi samghabadi et al. (2020) used bert in a multi-task manner to solve the task, whereas @xcite used lstm and cnns. ## dataset to identify the type of aggression, we use the english train dataset, and the facebook (fb) test dataset provided by the 2018 trac shared task @xcite . the data collection and annotation method is described in @xcite . the training data is combined with the augmented data provided by @xcite . the final distribution is given in we observe that the dataset contains some tweets which have improbable annotations. for example, the tweet "mr. sun you are wrong, pakistan produces one thing that is ' terrorists' and through cpec pak will increase the supply of this product throughout world. wait you will feel the touch of their product in your muslim dominated ## preprocessing and embeddings the tweets are first converted to lower case. next, we remove digits, special characters, emojis, urls, and stop words. we restrict the continuous repetition of the same character in a word to 2 (e.g. 'suuuuuuper' is converted to 'suuper'). each tweet is tokenized and converted into a sequence of integers. the maximum sequence length is restricted to 150. to have dense representation of tokens, the following word embedding features are used: ## proposed architecture we propose an architecture that combines features that are learned from an ensemble of subnetworks and leverages the feature representation to classify aggression. all models optimize the categorical crossentropy loss function using adam optimizer. all the dense layers, except the final layer, have relu activation. all the cnn layers are followed by @xmath0.5. every model is an ensemble of smaller subnetworks. each subnetwork (sn) has the following configuration: each sn of the model uses a different configuration for the cnn layer or embedding. therefore each sn learns different information and generates different features. the output of each sn is flattened and merged and is passed as input to dense layers. the last dense layer has three neurons and a softmax activation function, which gives a probability score to each of the three classes, and the one with the highest score is the predicted class. ## results and discussion from figures 3 , 4 and 5 are t-sne (van der maaten and hinton, 2008) graphs, which depict the output of sn1-3 of cn1, respectively. we visualize the feature embeddings in all the sns, and we observe that each sn is able to characterize the features distinctly due to the variability in the network configurations. when all the sns are combined in an ensemble network, the feature representation is further improved. the inter-class variability is predominant, as can be validated in fig. 6 . this can be attributed to the fact that all 3 sns have comfigure 3: flatten vector of subnetwork1 figure 4: flatten vector of subnetwork2 plimentary feature representations. as observed from the confusion matrix of cn1 model ( fig. 2 ), nag is the easiest to detect. it is because most of the tweets in the data are nag. the performance is better on oag than on cag, despite there being more training examples of cag as oag is more explicit and hence easier to identify, as opposed to the more indirect cag @xcite . cag, because of its covert nature is the most difficult to classify. the confusion of cag can also be observed in figure 6 , where cag is overlapping with nag and oag. the confusion can also be seen by analyzing some cag tweets predicted as nag: "hundreds of people were killed by your friends in bombay, where were you at that time." "what's next? soon we will be told to have a bullock cart and give up cars? or live in a shed using candles?" "chit fund operators n loan sharks r more honest" ## conclusion and future work we perform experiments to identify aggressive tweets by applying dl and capsule networks on preprocessed data. we show that capsulenets are efficient for aggression detection. we use an ensemble-based model and qualitatively show that each subnetwork learns unique features which help in classification. our best model uses capsulenets and results in a 65.20% f1 score, which is an improvement over most of the existing solutions. in the future, we would like to explore other capsulenet architectures using different routing algorithms. a more in-depth analysis of cag tweets could improve the performance on them.