pwc-archive/papers-with-abstracts huggingface dataset:
IRIT at TRAC 2020
Authors: Rami, Faneva risoa, Josiane Mothe
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Aggression Identification, Language Modeling, Language Modelling, Misogynistic Aggression Identification

This paper describes the participation of the IRIT team in the TRAC (Trolling, Aggression and Cyberbullying) 2020 shared task (Bhattacharya et al., 2020) on Aggression Identification and more precisely to the shared task in English language. The shared task was further divided into two sub-tasks: (a) aggression identification and (b) misogynistic aggression identification. We proposed to use the transformer based language model BERT (Bidirectional Encoder Representation from Transformer) for the two sub-tasks. Our team was qualified as twelfth out of sixteen participants on sub-task (a) and eleventh out of fifteen participants on sub-task (b).
Bagging BERT Models for Robust Aggression Identification
Authors: Julian Risch, Ralf Krestel
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Aggression Identification, text-classification, Text Classification

Modern transformer-based models with hundreds of millions of parameters, such as BERT, achieve impressive results at text classification tasks. This also holds for aggression identification and offensive language detection, where deep learning approaches consistently outperform less complex models, such as decision trees. While the complex models fit training data well (low bias), they also come with an unwanted high variance. Especially when fine-tuning them on small datasets, the classification performance varies significantly for slightly different training data. To overcome the high variance and provide more robust predictions, we propose an ensemble of multiple fine-tuned BERT models based on bootstrap aggregating (bagging). In this paper, we describe such an ensemble system and present our submission to the shared tasks on aggression identification 2020 (team name: Julian). Our submission is the best-performing system for five out of six subtasks. For example, we achieve a weighted F1-score of 80.3{\%} for task A on the test dataset of English social media posts. In our experiments, we compare different model configurations and vary the number of models used in the ensemble. We find that the F1-score drastically increases when ensembling up to 15 models, but the returns diminish for more models.
Scmhl5 at TRAC-2 Shared Task on Aggression Identification: Bert Based Ensemble Learning Approach
Authors: Han Liu, Pete Burnap, Wafa Alorainy, Matthew Williams
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Aggression Identification, Binary Classification, Ensemble Learning, General Classification

This paper presents a system developed during our participation (team name: scmhl5) in the TRAC-2 Shared Task on aggression identification. In particular, we participated in English Sub-task A on three-class classification ({`}Overtly Aggressive{'}, {`}Covertly Aggressive{'} and {`}Non-aggressive{'}) and English Sub-task B on binary classification for Misogynistic Aggression ({`}gendered{'} or {`}non-gendered{'}). For both sub-tasks, our method involves using the pre-trained Bert model for extracting the text of each instance into a 768-dimensional vector of embeddings, and then training an ensemble of classifiers on the embedding features. Our method obtained accuracy of 0.703 and weighted F-measure of 0.664 for Sub-task A, whereas for Sub-task B the accuracy was 0.869 and weighted F-measure was 0.851. In terms of the rankings, the weighted F-measure obtained using our method for Sub-task A is ranked in the 10th out of 16 teams, whereas for Sub-task B the weighted F-measure is ranked in the 8th out of 15 teams.
The Role of Computational Stylometry in Identifying (Misogynistic) Aggression in English Social Media Texts
Authors: Antonio Pascucci, Raffaele Manna, Vincenzo Masucci, Johanna Monti
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Aggression Identification, Misogynistic Aggression Identification

In this paper, we describe UniOr{\_}ExpSys team participation in TRAC-2 (Trolling, Aggression and Cyberbullying) shared task, a workshop organized as part of LREC 2020. TRAC-2 shared task is organized in two sub-tasks: Aggression Identification (a 3-way classification between {``}Overtly Aggressive{''}, {``}Covertly Aggressive{''} and {``}Non-aggressive{''} text data) and Misogynistic Aggression Identification (a binary classifier for classifying the texts as {``}gendered{''} or {``}non-gendered{''}). Our approach is based on linguistic rules, stylistic features extraction through stylometric analysis and Sequential Minimal Optimization algorithm in building the two classifiers.
Aggression Identification in English, Hindi and Bangla Text using BERT, RoBERTa and SVM
Authors: Arup Baruah, Kaushik Das, Ferdous Barbhuiya, Kuntal Dey
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Aggression Identification, Misogynistic Aggression Identification

This paper presents the results of the classifiers we developed for the shared tasks in aggression identification and misogynistic aggression identification. These two shared tasks were held as part of the second workshop on Trolling, Aggression and Cyberbullying (TRAC). Both the subtasks were held for English, Hindi and Bangla language. In our study, we used English BERT (En-BERT), RoBERTa, DistilRoBERTa, and SVM based classifiers for English language. For Hindi and Bangla language, multilingual BERT (M-BERT), XLM-RoBERTa and SVM classifiers were used. Our best performing models are EN-BERT for English Subtask A (Weighted F1 score of 0.73, Rank 5/16), SVM for English Subtask B (Weighted F1 score of 0.87, Rank 2/15), SVM for Hindi Subtask A (Weighted F1 score of 0.79, Rank 2/10), XLMRoBERTa for Hindi Subtask B (Weighted F1 score of 0.87, Rank 2/10), SVM for Bangla Subtask A (Weighted F1 score of 0.81, Rank 2/10), and SVM for Bangla Subtask B (Weighted F1 score of 0.93, Rank 4/8). It is seen that the superior performance of the SVM classifier was achieved mainly because of its better prediction of the majority class. BERT based classifiers were found to predict the minority classes better.
LaSTUS/TALN at TRAC - 2020 Trolling, Aggression and Cyberbullying
Authors: L{\"u}tfiye Seda Mut Alt{\i}n, Alex Bravo, Horacio Saggion
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

This paper presents the participation of the LaSTUS/TALN team at TRAC-2020 Trolling, Aggression and Cyberbullying shared task. The aim of the task is to determine whether a given text is aggressive and contains misogynistic content. Our approach is based on a bidirectional Long Short Term Memory network (bi-LSTM). Our system performed well at sub-task A, aggression detection; however underachieved at sub-task B, misogyny detection.
Spyder: Aggression Detection on Multilingual Tweets
Authors: Anisha Datta, Shukrity Si, Urbi Chakraborty, Sudip Kumar Naskar
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

In the last few years, hate speech and aggressive comments have covered almost all the social media platforms like facebook, twitter etc. As a result hatred is increasing. This paper describes our (\textbf{Team name:} \textbf{Spyder}) participation in the Shared Task on Aggression Detection organised by TRAC-2, Second Workshop on Trolling, Aggression and Cyberbullying. The Organizers provided datasets in three languages {--} English, Hindi and Bengali. The task was to classify each instance of the test sets into three categories {--} {``}Overtly Aggressive{''} (OAG), {``}Covertly Aggressive{''} (CAG) and {``}Non-Aggressive{''} (NAG). In this paper, we propose three different models using Tf-Idf, sentiment polarity and machine learning based classifiers. We obtained f1 score of 43.10{\%}, 59.45{\%} and 44.84{\%} respectively for English, Hindi and Bengali.
BERT of all trades, master of some
Authors: Denis Gordeev, Olga Lykova
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: All, text-classification, Text Classification

This paper describes our results for TRAC 2020 competition held together with the conference LREC 2020. Our team name was Ms8qQxMbnjJMgYcw. The competition consisted of 2 subtasks in 3 languages (Bengali, English and Hindi) where the participants{'} task was to classify aggression in short texts from social media and decide whether it is gendered or not. We used a single BERT-based system with two outputs for all tasks simultaneously. Our model placed first in English and second in Bengali gendered text classification competition tasks with 0.87 and 0.93 in F1-score respectively.
SAJA at TRAC 2020 Shared Task: Transfer Learning for Aggressive Identification with XGBoost
Authors: Saja Tawalbeh, Mahmoud Hammad, Mohammad AL-Smadi
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Sentence, Transfer Learning

we have developed a system based on transfer learning technique depending on universal sentence encoder (USE) embedding that will be trained in our developed model using xgboost classifier to identify the aggressive text data from English content. A reference dataset has been provided from TRAC 2020 to evaluate the developed approach. The developed approach achieved in sub-task EN-A 60.75{\%} F1 (weighted) which ranked fourteenth out of sixteen teams and achieved 85.66{\%} F1 (weighted) in sub-task EN-B which ranked six out of fifteen teams.
FlorUniTo@TRAC-2: Retrofitting Word Embeddings on an Abusive Lexicon for Aggressive Language Detection
Authors: Anna Koufakou, Valerio Basile, Viviana Patti
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Aggression Identification, Word Embeddings

This paper describes our participation to the TRAC-2 Shared Tasks on Aggression Identification. Our team, FlorUniTo, investigated the applicability of using an abusive lexicon to enhance word embeddings towards improving detection of aggressive language. The embeddings used in our paper are word-aligned pre-trained vectors for English, Hindi, and Bengali, to reflect the languages in the shared task data sets. The embeddings are retrofitted to a multilingual abusive lexicon, HurtLex. We experimented with an LSTM model using the original as well as the transformed embeddings and different language and setting variations. Overall, our systems placed toward the middle of the official rankings based on weighted F1 score. However, the results on the development and test sets show promising improvements across languages, especially on the misogynistic aggression sub-task.
AI\_ML\_NIT\_Patna @ TRAC - 2: Deep Learning Approach for Multi-lingual Aggression Identification
Authors: Kirti Kumari, Jyoti Prakash Singh
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Aggression Identification

This paper describes the details of developed models and results of team AI{\_}ML{\_}NIT{\_}Patna for the shared task of TRAC - 2. The main objective of the said task is to identify the level of aggression and whether the comment is gendered based or not. The aggression level of each comment can be marked as either Overtly aggressive or Covertly aggressive or Non-aggressive. We have proposed two deep learning systems: Convolutional Neural Network and Long Short Term Memory with two different input text representations, FastText and One-hot embeddings. We have found that the LSTM model with FastText embedding is performing better than other models for Hindi and Bangla datasets but for the English dataset, the CNN model with FastText embedding has performed better. We have also found that the performances of One-hot embedding and pre-trained FastText embedding are comparable. Our system got 11th and 10th positions for English Sub-task A and Sub-task B, respectively, 8th and 7th positions, respectively for Hindi Sub-task A and Sub-task B and 7th and 6th positions for Bangla Sub-task A and Sub-task B, respectively among the total submitted systems.
Multilingual Joint Fine-tuning of Transformer models for identifying Trolling, Aggression and Cyberbullying at TRAC 2020
Authors: Sudhanshu Mishra, Shivangi Prasad, Shubhanshu Mishra
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Computational Efficiency

We present our team {`}3Idiots{'} (referred as {`}sdhanshu{'} in the official rankings) approach for the Trolling, Aggression and Cyberbullying (TRAC) 2020 shared tasks. Our approach relies on fine-tuning various Transformer models on the different datasets. We also investigated the utility of task label marginalization, joint label classification, and joint training on multilingual datasets as possible improvements to our models. Our team came second in English sub-task A, a close fourth in the English sub-task B and third in the remaining 4 sub-tasks. We find the multilingual joint training approach to be the best trade-off between computational efficiency of model deployment and model{'}s evaluation performance. We open source our approach at https://github.com/socialmediaie/TRAC2020.
Aggression and Misogyny Detection using BERT: A Multi-Task Approach
Authors: Niloofar Safi Samghabadi, Parth Patwa, Srinivas Pykl, Prerana Mukherjee, Amitava Das, Thamar Solorio
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Abusive Language, Aggression Identification, Hate Speech Detection, Misogynistic Aggression Identification, Multi-Task Learning

In recent times, the focus of the NLP community has increased towards offensive language, aggression, and hate-speech detection.This paper presents our system for TRAC-2 shared task on {``}Aggression Identification{''} (sub-task A) and {``}Misogynistic Aggression Identification{''} (sub-task B). The data for this shared task is provided in three different languages - English, Hindi, and Bengali. Each data instance is annotated into one of the three aggression classes - Not Aggressive, Covertly Aggressive, Overtly Aggressive, as well as one of the two misogyny classes - Gendered and Non-Gendered. We propose an end-to-end neural model using attention on top of BERT that incorporates a multi-task learning paradigm to address both the sub-tasks simultaneously. Our team, {``}na14{''}, scored 0.8579 weighted F1-measure on the English sub-task B and secured 3rd rank out of 15 teams for the task. The code and the model weights are publicly available at https://github.com/NiloofarSafi/TRAC-2. Keywords: Aggression, Misogyny, Abusive Language, Hate-Speech Detection, BERT, NLP, Neural Networks, Social Media
Automatic Detection of Offensive Language in Social Media: Defining Linguistic Criteria to build a Mexican Spanish Dataset
Authors: Mar{\'\i}a Jos{\'e} D{\'\i}az-Torres, Paulina Alej Mor{\'a}n-M{\'e}ndez, ra, Luis Villasenor-Pineda, Manuel Montes-y-G{\'o}mez, Juan Aguilera, Luis Meneses-Ler{\'\i}n
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Abusive Language

Phenomena such as bullying, homophobia, sexism and racism have transcended to social networks, motivating the development of tools for their automatic detection. The challenge becomes greater for languages rich in popular sayings, colloquial expressions and idioms which may contain vulgar, profane or rude words, but not always have the intention of offending, as is the case of Mexican Spanish. Under these circumstances, the identification of the offense goes beyond the lexical and syntactic elements of the message. This first work aims to define the main linguistic features of aggressive, offensive and vulgar language in social networks in order to establish linguistic-based criteria to facilitate the identification of abusive language. For this purpose, a Mexican Spanish Twitter corpus was compiled and analyzed. The dataset included words that, despite being rude, need to be considered in context to determine they are part of an offense. Based on the analysis of this corpus, linguistic criteria were defined to determine whether a message is offensive. To simplify the application of these criteria, an easy-to-follow diagram was designed. The paper presents an example of the use of the diagram, as well as the basic statistics of the corpus.
Offensive Language Detection Explained
Authors: Julian Risch, Robin Ruff, Ralf Krestel
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: BIG-bench Machine Learning, General Classification, Interpretable Machine Learning

Many online discussion platforms use a content moderation process, where human moderators check user comments for offensive language and other rule violations. It is the moderator{'}s decision which comments to remove from the platform because of violations and which ones to keep. Research so far focused on automating this decision process in the form of supervised machine learning for a classification task. However, even with machine-learned models achieving better classification accuracy than human experts, there is still a reason why human moderators are preferred. In contrast to black-box models, such as neural networks, humans can give explanations for their decision to remove a comment. For example, they can point out which phrase in the comment is offensive or what subtype of offensiveness applies. In this paper, we analyze and compare four explanation methods for different offensive language classifiers: an interpretable machine learning model (naive Bayes), a model-agnostic explanation method (LIME), a model-based explanation method (LRP), and a self-explanatory model (LSTM with an attention mechanism). We evaluate these approaches with regard to their explanatory power and their ability to point out which words are most relevant for a classifier{'}s decision. We find that the more complex models achieve better classification accuracy while also providing better explanations than the simpler models.
Detecting Early Signs of Cyberbullying in Social Media
Authors: Niloofar Safi Samghabadi, Adri{\'a}n Pastor L{\'o}pez Monroy, Thamar Solorio
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Abusive Language

Nowadays, the amount of users{'} activities on online social media is growing dramatically. These online environments provide excellent opportunities for communication and knowledge sharing. However, some people misuse them to harass and bully others online, a phenomenon called cyberbullying. Due to its harmful effects on people, especially youth, it is imperative to detect cyberbullying as early as possible before it causes irreparable damages to victims. Most of the relevant available resources are not explicitly designed to detect cyberbullying, but related content, such as hate speech and abusive language. In this paper, we propose a new approach to create a corpus suited for cyberbullying detection. We also investigate the possibility of designing a framework to monitor the streams of users{'} online messages and detects the signs of cyberbullying as early as possible.
Lexicon-Enhancement of Embedding-based Approaches Towards the Detection of Abusive Language
Authors: Anna Koufakou, Jason Scott
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Abusive Language, Word Embeddings

Detecting abusive language is a significant research topic, which has received a lot of attention recently. Our work focuses on detecting personal attacks in online conversations. As previous research on this task has largely used deep learning based on embeddings, we explore the use of lexicons to enhance embedding-based methods in an effort to see how these methods apply in the particular task of detecting personal attacks. The methods implemented and experimented with in this paper are quite different from each other, not only in the type of lexicons they use (sentiment or semantic), but also in the way they use the knowledge from the lexicons, in order to construct or to change embeddings that are ultimately fed into the learning model. The sentiment lexicon approaches focus on integrating sentiment information (in the form of sentiment embeddings) into the learning model. The semantic lexicon approaches focus on transforming the original word embeddings so that they better represent relationships extracted from a semantic lexicon. Based on our experimental results, semantic lexicon methods are superior to the rest of the methods in this paper, with at least 4{\%} macro-averaged F1 improvement over the baseline.
Current Challenges in Web Corpus Building
Authors: Milo{\v{s}} Jakub{\'\i}{\v{c}}ek, Vojt{\v{e}}ch Kov{\'a}{\v{r}}, Pavel Rychl{\'y}, Vit Suchomel
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

In this paper we discuss some of the current challenges in web corpus building that we faced in the recent years when expanding the corpora in Sketch Engine. The purpose of the paper is to provide an overview and raise discussion on possible solutions, rather than bringing ready solutions to the readers. For every issue we try to assess its severity and briefly discuss possible mitigation options.
Out-of-the-Box and into the Ditch? Multilingual Evaluation of Generic Text Extraction Tools
Authors: Adrien Barbaresi, Ga{\"e}l Lejeune
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5
Tasks: Diversity

This article examines extraction methods designed to retain the main text content of web pages and discusses how the extraction could be oriented and evaluated: can and should it be as generic as possible to ensure opportunistic corpus construction? The evaluation grounds on a comparative benchmark of open-source tools used on pages in five different languages (Chinese, English, Greek, Polish and Russian), it features several metrics to obtain more fine-grained differentiations. Our experiments highlight the diversity of web page layouts across languages or publishing countries. These discrepancies are reflected by diverging performances so that the right tool has to be chosen accordingly.
From Web Crawl to Clean Register-Annotated Corpora
Authors: Veronika Laippala, Samuel R{\"o}nnqvist, Saara Hellstr{\"o}m, Juhani Luotolahti, Liina Repo, Anna Salmela, Valtteri Skantsi, Sampo Pyysalo
Date: 2020-05-01 00:00:00
Proceeding: LREC 2020 5

The web presents unprecedented opportunities for large-scale collection of text in many languages. However, two critical steps in the development of web corpora remain challenging: the identification of clean text from source HTML and the assignment of genre or register information to the documents. In this paper, we evaluate a multilingual approach to this end. Our starting points are the Swedish and French Common Crawl datasets gathered for the 2017 CoNLL shared task, particularly the URLs. We 1) fetch HTML pages based on the URLs and run boilerplate removal, 2) train a classifier to further clean out undesired text fragments, and 3) annotate text registers. We compare boilerplate removal against the CoNLL texts, and find an improvement. For the further cleaning of undesired material, the best results are achieved using Multilingual BERT with monolingual fine-tuning. However, our results are promising also in a cross-lingual setting, without fine-tuning on the target language. Finally, the register annotations show that most of the documents belong to a relatively small set of registers, which are relatively similar in the two languages. A number of additional flags in the annotation are, however, necessary to reflect the wide range of linguistic variation associated with the documents.