Amsqr at S em E val-2020 Task 12: Offensive Language Detection Using Neural Networks and Anti-adversarial Features (2020)

Abstract: This paper describes a method and system to solve the problem of detecting offensive language in social media using anti-adversarial features. Our submission to the SemEval-2020 task 12 challenge was generated by an stacked ensemble of neural networks fine-tuned on the OLID dataset and additional external sources. For Task-A (English), text normalisation filters were applied at both graphical and lexical level. The normalisation step effectively mitigates not only the natural presence of lexical variants but also intentional attempts to bypass moderation by introducing out of vocabulary words. Our approach provides strong F1 scores for both 2020 (0.9134) and 2019 (0.8258) challenges.

## introduction keeping social media platforms free from unwanted publications such as spam, scam, phishing, hate speech, targeted attacks and fake news is still an active research topic nowadays. this is due not only to the relative low cost of creating fake accounts, bots @xcite and forging online identities but also to the large amount of personal information made available on the internet which makes targeting certain groups and individuals easier than ever. while some of these threats were seen before affecting traditional messaging platforms such as email and sms, the reach and adoption of social media applications have amplified their impact, requiring additional cost and effort to mitigate. the use of offensive language as a vehicle to attack individuals and communities poses challenges not only for humans, which are prone to subjective and biased judgement @xcite but also for automatic moderation systems. the inherent ambiguity when dealing with messages which are often short, can be written in mixed languages, contain informal words and are usually subject to adversarial modifications render naïve filtering approaches such as word lists or re-purposed spam detection models ineffective. for this reason, in order to solve this problem more sophisticated approaches such as state of the art natural language processing (nlp) is needed. in order to keep track and measure the progress in the area on offensive language detection several english datasets with annotations for hate @xcite , @xcite , targeted @xcite and personal attacks @xcite were released over the last years. likewise, public evaluations such as hateval @xcite and offenseeval @xcite were recently introduced highlighting the need for stronger baselines to assess the performance of more complex systems. this paper evaluates the method and system submitted to the shared task 12 of semeval-2020: multilingual offensive language identification in social media @xcite for the subtask a (english) based on an stacked ensemble of neural networks. the rest of the document is organised as follows: in section 2, we review related work relevant for detecting abusive language. in section 3 we describe our layered model approach including our anti-adversarial strategy based on text normalisation and stacking-based ensembling. in section 4 we show the results obtained in the test and evaluation datasets. finally, in section 5 we draw our conclusions and outline future work. while adding more complexity, combining several models can effectively reduce classification bias and variance. we have seen good results using voting ensembles @xcite and stacked generalisation @xcite when applied to this particular problem. ## methodology the goal of subtask-a is determining if a tweet is either offensive or not offensive, which conceptually translates to a binary classifier using f1-macro as scoring function. however, during the exploratory data analysis of the training set we've identified group of instances where users intentionally crafted offensive messages to bypass profanity and moderation filters. for this reason, our design choices have an anti-adversarial strategy in mind. best performing models in previous benchmarks @xcite , @xcite were based on popular pre-trained embeddings and architectures, either using transfer learning or leveraging these directly. while this is quite convenient in terms of computing cost it also introduces potential weaknesses which can be exploited in a black-box scenario. by guessing the base architecture the model was built upon, since there is a reduced set of high-quality pretrained models, an attacker could launch more successful black-box attacks @xcite . this is usually performed via input perturbation such as introducing synonyms @xcite , flipping characters @xcite or including targeted keywords and typos @xcite . being even possible to steal the whole model altogether @xcite in more sophisticated attacks. ## results our offensive text classification system obtained strong results across different datasets which are summarized in interestingly, bert models fine tuned on kaggle toxic dataset had a high correlation with the test set for this year challenge, even improving slightly final ensemble results when compared against the identity hate and toxic classes. such correlation is not present in the previous year test set, where models trained on olid outperformed the rest by a considerable margin. there is another apparent trend reversal observed in normalised models, on the 2019 test set individual models with normalisation outperformed their non-normalised equivalent while in the current test set results were comparable for both normalised and not normalised. labelling shifts of certain keywords that caused the system to fp may be worth of further analysis: 79% of the tweets containing the pattern "sick|disgusting|sucks" were labelled as offensive in olid, in comparison with a 55% when considering test set gold labels. some examples of this disagreement can be found at ## conclusion and future work in this paper we describe our system and method for detecting offensive tweets built for semeval-2020 task 12 -subtask a. our design choices had an adversarial environment in mind and therefore we've made use of anti-adversarial features such as text normalisation and ensemble learning, obtaining strong results in 2 evaluation datasets. in a future work we would like to explore different attack and defence scenarios for this particular problem.