Abstract: Wide usage of social media platforms has increased the risk of aggression, which results in mental stress and affects the lives of people negatively like psychological agony, fighting behavior, and disrespect to others. Majority of such conversations contains code-mixed languages[28]. Additionally, the way used to express thought or communication style also changes from one social media plat-form to another platform (e.g., communication styles are different in twitter and Facebook). These all have increased the complexity of the problem. To solve these problems, we have introduced a unified and robust multi-modal deep learning architecture which works for English code-mixed dataset and uni-lingual English dataset both.The devised system, uses psycho-linguistic features and very ba-sic linguistic features. Our multi-modal deep learning architecture contains, Deep Pyramid CNN, Pooled BiLSTM, and Disconnected RNN(with Glove and FastText embedding, both). Finally, the system takes the decision based on model averaging. We evaluated our system on English Code-Mixed TRAC 2018 dataset and uni-lingual English dataset obtained from Kaggle. Experimental results show that our proposed system outperforms all the previous approaches on English code-mixed dataset and uni-lingual English dataset.

Introduction: The exponential increase of interactions on the various social media platforms has generated the huge amount of data on social media platforms like Facebook and Twitter, etc. These interactions resulted not only positive effect but also negative effect over billions of people owing to the fact that there are lots of aggressive comments (like hate, anger, and bullying). These cause not only mental and psychological stress but also account deactivation and even suicideBIBREF1. In this paper we concentrate on problems related to aggressiveness. The fine-grained definition of the aggressiveness/aggression identification is provided by the organizers of TRAC-2018 BIBREF0, BIBREF2. They have classified the aggressiveness into three labels (Overtly aggressive(OAG), Covertly aggressive(CAG), Non-aggressive(NAG)). The detailed description for each of the three labels is described as follows: Overtly Aggressive(OAG) - This type of aggression shows direct verbal attack pointing to the particular individual or group. For example, "Well said sonu..you have courage to stand against dadagiri of Muslims". Covertly Aggressive(CAG) - This type of aggression the attack is not direct but hidden, subtle and more indirect while being stated politely most of the times. For example, "Dear India, stop playing with the emotions of your people for votes." Non-Aggressive(NAG) - Generally these type of text lack any kind of aggression it is basically used to state facts, wishing on occasions and polite and supportive. The additional discussion on aggressiveness task can be found in Kaggle task , which just divided the task into two classes - i.e., presence or absence of aggression in tweets. The informal setting/environment of social media often encourage multilingual speakers to switch back and forth between languages when speaking or writing. These all resulted in code-mixing and code-switching. Code-mixing refers to the use of linguistic units from different languages in a single utterance or sentence, whereas code-switching refers to the co-occurrence of speech extracts belonging to two different grammatical systemsBIBREF3. This language interchange makes the grammar more complex and thus it becomes tough to handle it by traditional algorithms. Thus the presence of high percentage of code-mixed content in social media text has increased the complexity of the aggression detection task. For example, the dataset provided by the organizers of TRAC-2018 BIBREF0, BIBREF2 is actually a code-mixed dataset. The massive increase of the social media data rendered the manual methods of content moderation difficult and costly. Machine Learning and Deep Learning methods to identify such phenomena have attracted more attention to the research community in recent yearsBIBREF4. Based on the current context, we can divide the problem into three sub-problems: (a) detection of aggression levels, (b) handling code-mixed data and (c) handling styles (due to differences in social media platforms and text entry rules/restrictions). A lot of the previous approachesBIBREF5 have used an ensemble model for the task. For example, some of them uses ensemble of statistical modelsBIBREF6, BIBREF7, BIBREF8, BIBREF9 some used ensemble of statistical and deep learning modelsBIBREF10, BIBREF11, BIBREF12 some used ensemble of deep learning models BIBREF13. There are approaches which proposed unified architecture based on deep learningBIBREF14, BIBREF15, BIBREF16, BIBREF17, BIBREF18, BIBREF19 while some proposed unified statistical modelBIBREF7. Additionally, there are some approaches uses data augmentation either through translation or labeling external data to make the model generalize across domainsBIBREF14, BIBREF10, BIBREF7. Most of the above-discussed systems either shows high performance on (a) Twitter dataset or (b) Facebook dataset (given in the TRAC-2018), but not on both English code-mixed datasets. This may be due to the text style or level of complexities of both datasets. So, we concentrated to develop a robust system for English code-mixed texts, and uni-lingual texts, which can also handle different writing styles. Our approach is based on three main ideas: Deep-Text Learning. The goal is to learn long range associations, dependencies between regions of text, N-grams, key-patterns, topical information, and sequential dependencies. Exploiting psycho-linguistic features with basic linguistic features as meta-data. The main aim is to minimize the direct dependencies on in-depth grammatical structure of the language (i.e., to support code-mixed data). We have also included emoticons, and punctuation features with it. We use the term "NLP Features" to represent it in the entire paper. Dual embedding based on FastText and Glove. This dual embedding helps in high vocabulary coverage and to capture the rare and partially incorrect words in the text (specially by FastText BIBREF20). Our "Deep-text architecture" uses model averaging strategy with three different deep learning architectures. Model averaging belongs to the family of ensemble learning techniques that uses multiple models for the same problem and combines their predictions to produce a more reliable and consistent prediction accuracy BIBREF21. This is the simplest form of weighted average ensemble based predictionBIBREF22 where, each ensemble member contribute equally to predictions. Specifically in our case, three different models have been used. The following contains the intuition behind the selection of these three models: Deep Pyramid CNN BIBREF23 being deeper helps to learn long range associations between temporal regions of text using two-view embeddings. Disconnected RNN BIBREF24 is very helpful in encoding the sequential information with temporal key patterns in the text. Pooled BiLSTM In this architecture the last hidden state of BiLSTM is concatenated with mean and max-pooled representation of the hidden states obtained over all the time steps of Bi-LSTM. The idea of using mean and max pooling layers together is taken from BIBREF25 to avoid the loss of information in longer sequences of texts and max-pooling is taken to capture the topical informationBIBREF26. NLP Features In each of the individual models, the NLP features are concatenated with last hidden state before the softmax classification layer as meta-data. The main aim is to provide additional information to the deep learning network. The intuition behind the NLP features are the following: Emotion Sensor Dataset We have introduced to use of emotion sensor features, as a meta-data information. We have obtained the word sensor dataset from Kaggle. In this dataset each word is statistically classified into 7 distinct classes (Disgust, Surprise, Neutral, Anger, Sad, Happy and Fear) using Naive Bayes, based on sentences collected from twitter and blogs. Controlled Topical Signals from Empath. Empath can analyse the text across 200 gold standard topics and emotions. Additionally, it uses neural embedding to draw connotation among words across more than 1.8 billion words. We have used only selected categories like violence, hate, anger, aggression, social media and dispute from 200 Empath categories useful for us unlikeBIBREF12 which takes 194 categories. Emoticons frequently used on social media indicates the sense of sentenceBIBREF17, BIBREF19, BIBREF9. Normalized frequency of POS tags According to BIBREF12, BIBREF11, BIBREF7, BIBREF15 POS Tags provide the degree of target aggressiveness. LikeBIBREF12, we have used only four tags (a) adjective (JJ, JJR, JJS), (b) adverb (RB, RBR, RBS), (c) verb (VB, VBD, VBG, VBN, VBP, VBZ) and (d) noun (NN, NNS, NNP, NNPS) (See Penn-Treebank POS Tags for abbreviations and the full list). The main reason behind the selection of these four tags is to just identify words related to persons, activities, quality, etc, in the text. Sentiment polarity obtained from VADER Sentiment Analysis BIBREF27 (positive, negative and neutral) like used in BIBREF15, BIBREF10, BIBREF11, BIBREF7. It helps to demarcate aggressiveness with non-aggressiveness in the text. The block diagram of the proposed system is shown in Figure FIGREF22. The proposed system does not use any data augmentation techniques like BIBREF14, which is the top performer in TRAC (in English code-mixed Facebook data). This means the performance achieved by our system totally depends on the training dataset provided by TRAC. This also proves the effectiveness of our approach. Our system outperforms all the previous state of the art approaches used for aggression identification on English code-mixed TRAC data, while being trained only from Facebook comments the system outperforms other approaches on the additional Twitter test set. The remaining part of this paper is organized as follows: Section SECREF2 is an overview of related work. Section SECREF3 presents the methodology and algorithmic details. Section SECREF4 discusses the experimental evaluation of the system, and Section SECREF5 concludes this paper.

Question: Which psycholinguistic and basic linguistic features are used?
Answer: Emotion Sensor Feature, Part of Speech, Punctuation, Sentiment Analysis, Empath, TF-IDF Emoticon features