pwc-archive/papers-with-abstracts huggingface dataset:
TuEval at SemEval-2019 Task 5: LSTM Approach to Hate Speech Detection in English and Spanish
Authors: Mihai Manolescu, Denise L{\"o}fflad, Adham Nasser Mohamed Saber, Masoumeh Moradipour Tari
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Hate Speech Detection

The detection of hate speech, especially in online platforms and forums, is quickly becoming a hot topic as anti-hate speech legislation begins to be applied to public discourse online. The HatEval shared task was created with this in mind; participants were expected to develop a model capable of determining whether or not input (in this case, Twitter datasets in English and Spanish) could be considered hate speech (designated as Task A), if they were aggressive, and whether the tweet was targeting an individual, or speaking generally (Task B). We approached this task by creating an LSTM model with an embedding layer. We found that our model performed considerably better on English language input when compared to Spanish language input. In English, we achieved an F1-Score of 0.466 for Task A and 0.462 for Task B; In Spanish, we achieved scores of 0.617 and 0.612 on Task A and Task B, respectively.
Tw-StAR at SemEval-2019 Task 5: N-gram embeddings for Hate Speech Detection in Multilingual Tweets
Authors: Hala Mulki, Chedi Bechikh Ali, Hatem Haddad, Ismail Babao{\u{g}}lu
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Hate Speech Detection

In this paper, we describe our contribution in SemEval-2019: subtask A of task 5 {``}Multilingual detection of hate speech against immigrants and women in Twitter (HatEval){''}. We developed two hate speech detection model variants through Tw-StAR framework. While the first model adopted one-hot encoding ngrams to train an NB classifier, the second generated and learned n-gram embeddings within a feedforward neural network. For both models, specific terms, selected via MWT patterns, were tagged in the input data. With two feature types employed, we could investigate the ability of n-gram embeddings to rival one-hot n-grams. Our results showed that in English, n-gram embeddings outperformed one-hot ngrams. However, representing Spanish tweets by one-hot n-grams yielded a slightly better performance compared to that of n-gram embeddings. The official ranking indicated that Tw-StAR ranked 9th for English and 20th for Spanish.
UA at SemEval-2019 Task 5: Setting A Strong Linear Baseline for Hate Speech Detection
Authors: Carlos Perell{\'o}, David Tom{\'a}s, Alberto Garcia-Garcia, Jose Garcia-Rodriguez, Jose Camacho-Collados
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: General Classification, Hate Speech Detection

This paper describes the system developed at the University of Alicante (UA) for the SemEval 2019 Task 5: Shared Task on Multilingual Detection of Hate. The purpose of this work is to build a strong baseline for hate speech detection, using a traditional machine learning approach with standard textual features, which could serve in a near future as a reference to compare with deep learning systems. We participated in both task A (Hate Speech Detection against Immigrants and Women) and task B (Aggressive behavior and Target Classification). Despite its simplicity, our system obtained a remarkable F1-score of 72.5 (sixth highest) and an accuracy of 73.6 (second highest) in Spanish (task A), outperforming more complex neural models from a total of 40 participant systems.
UNBNLP at SemEval-2019 Task 5 and 6: Using Language Models to Detect Hate Speech and Offensive Language
Authors: Ali Hakimi Parizi, Milton King, Paul Cook
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: General Classification, Language Modeling, Language Modelling, text-classification, Text Classification

In this paper we apply a range of approaches to language modeling {--} including word-level n-gram and neural language models, and character-level neural language models {--} to the problem of detecting hate speech and offensive language. Our findings indicate that language models are able to capture knowledge of whether text is hateful or offensive. However, our findings also indicate that more conventional approaches to text classification often perform similarly or better.
Vista.ue at SemEval-2019 Task 5: Single Multilingual Hate Speech Detection Model
Authors: Kashyap Raiyani, Teresa Gon{\c{c}}alves, Paulo Quaresma, Vitor Nogueira
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Hate Speech Detection

This paper shares insight from participating in SemEval-2019 Task 5. The main propose of this system-description paper is to facilitate the reader with replicability and to provide insightful analysis of the developed system. Here in Vista.ue, we proposed a single multilingual hate speech detection model. This model was ranked 46/70 for English Task A and 31/43 for English Task B. Vista.ue was able to rank 38/41 for Spanish Task A and 22/25 for Spanish Task B.
YNU NLP at SemEval-2019 Task 5: Attention and Capsule Ensemble for Identifying Hate Speech
Authors: Bin Wang, Haiyan Ding
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Hate Speech Detection

This paper describes the system submitted to SemEval 2019 Task 5: Multilingual detection of hate speech against immigrants and women in Twitter (hatEval). Its main purpose is to conduct hate speech detection on Twitter, which mainly includes two specific different targets, immigrants and women. We participate in both subtask A and subtask B for English. In order to address this task, we develope an ensemble of an attention-LSTM model based on HAN and an BiGRU-capsule model. Both models use fastText pre-trained embeddings, and we use this model in both subtasks. In comparison to other participating teams, our system is ranked 16th in the Sub-task A for English, and 12th in the Sub-task B for English.
YNU\_DYX at SemEval-2019 Task 5: A Stacked BiGRU Model Based on Capsule Network in Detection of Hate
Authors: Yunxia Ding, Xiaobing Zhou, Xue-jie Zhang
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Sentence

This paper describes our system designed for SemEval 2019 Task 5 {``}Shared Task on Multilingual Detection of Hate{''}.We only participate in subtask-A in English. To address this task, we present a stacked BiGRU model based on a capsule network system. In or- der to convert the tweets into corresponding vector representations and input them into the neural network, we use the fastText tools to get word representations. Then, the sentence representation is enriched by stacked Bidirectional Gated Recurrent Units (BiGRUs) and used as the input of capsule network. Our system achieves an average F1-score of 0.546 and ranks 3rd in the subtask-A in English.
Amrita School of Engineering - CSE at SemEval-2019 Task 6: Manipulating Attention with Temporal Convolutional Neural Network for Offense Identification and Classification
Authors: Murali Sridharan, Swapna TR
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: General Classification

With the proliferation and ubiquity of smart gadgets and smart devices, across the world, data generated by them has been growing at exponential rates; in particular social media platforms like Facebook, Twitter and Instagram have been generating voluminous data on a daily basis. According to Twitter{'}s usage statistics, about 500 million tweets are generated each day. While the tweets reflect the users{'} opinions on several events across the world, there are tweets which are offensive in nature that need to be tagged under the hateful conduct policy of Twitter. Offensive tweets have to be identified, captured and processed further, for a variety of reasons, which include i) identifying offensive tweets in order to prevent violent/abusive behavior in Twitter (or any social media for that matter), ii) creating and maintaining a history of offensive tweets for individual users (would be helpful in creating meta-data for user profile), iii) inferring the sentiment of the users on particular event/issue/topic . We have employed neural network models which manipulate attention with Temporal Convolutional Neural Network for the three shared sub-tasks i) ATT-TCN (ATTention based Temporal Convolutional Neural Network) employed for shared sub-task A that yielded a best macro-F1 score of 0.46, ii) SAE-ATT-TCN(Self Attentive Embedding-ATTention based Temporal Convolutional Neural Network) employed for shared sub-task B and sub-task C that yielded best macro-F1 score of 0.61 and 0.51 respectively. Among the two variants ATT-TCN and SAE-ATT-TCN, the latter performed better.
bhanodaig at SemEval-2019 Task 6: Categorizing Offensive Language in social media
Authors: Ritesh Kumar, Guggilla Bhanodai, Rajendra Pamula, Maheswara Reddy Chennuru
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: General Classification

This paper describes the work that our team bhanodaig did at Indian Institute of Technology (ISM) towards OffensEval i.e. identifying and categorizing offensive language in social media. Out of three sub-tasks, we have participated in sub-task B: automatic categorization of offensive types. We perform the task of categorizing offensive language, whether the tweet is targeted insult or untargeted. We use Linear Support Vector Machine for classification. The official ranking metric is macro-averaged F1. Our system gets the score 0.5282 with accuracy 0.8792. However, as new entrant to the field, our scores are encouraging enough to work for better results in future.
BNU-HKBU UIC NLP Team 2 at SemEval-2019 Task 6: Detecting Offensive Language Using BERT model
Authors: Zhenghao Wu, Hao Zheng, Jianming Wang, Weifeng Su, Jefferson Fong
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Language Identification, Sentence

In this study we deal with the problem of identifying and categorizing offensive language in social media. Our group, BNU-HKBU UIC NLP Team2, use supervised classification along with multiple version of data generated by different ways of pre-processing the data. We then use the state-of-the-art model Bidirectional Encoder Representations from Transformers, or BERT (Devlin et al, 2018), to capture linguistic, syntactic and semantic features. Long range dependencies between each part of a sentence can be captured by BERT{'}s bidirectional encoder representations. Our results show 85.12{\%} accuracy and 80.57{\%} F1 scores in Subtask A (offensive language identification), 87.92{\%} accuracy and 50{\%} F1 scores in Subtask B (categorization of offense types), and 69.95{\%} accuracy and 50.47{\%} F1 score in Subtask C (offense target identification). Analysis of the results shows that distinguishing between targeted and untargeted offensive language is not a simple task. More work needs to be done on the unbalance data problem in Subtasks B and C. Some future work is also discussed.
CAMsterdam at SemEval-2019 Task 6: Neural and graph-based feature extraction for the identification of offensive tweets
Authors: Guy Aglionby, Chris Davis, Pushkar Mishra, Andrew Caines, Helen Yannakoudakis, Marek Rei, Ekaterina Shutova, Paula Buttery
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: General Classification, Language Identification, text-classification, Text Classification

We describe the CAMsterdam team entry to the SemEval-2019 Shared Task 6 on offensive language identification in Twitter data. Our proposed model learns to extract textual features using a multi-layer recurrent network, and then performs text classification using gradient-boosted decision trees (GBDT). A self-attention architecture enables the model to focus on the most relevant areas in the text. In order to enrich input representations, we use node2vec to learn globally optimised embeddings for hashtags, which are then given as additional features to the GBDT classifier. Our best model obtains 78.79{\%} macro F1-score on detecting offensive language (subtask A), 66.32{\%} on categorising offence types (targeted/untargeted; subtask B), and 55.36{\%} on identifying the target of offence (subtask C).
CN-HIT-MI.T at SemEval-2019 Task 6: Offensive Language Identification Based on BiLSTM with Double Attention
Authors: Yaojie Zhang, Bing Xu, Tiejun Zhao
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Language Identification

Offensive language has become pervasive in social media. In Offensive Language Identification tasks, it may be difficult to predict accurately only according to the surface words. So we try to dig deeper semantic information of text. This paper presents use an attention-based two layers bidirectional longshort memory neural network (BiLSTM) for semantic feature extraction. Additionally, a residual connection mechanism is used to synthesize two different deep features, and an emoji attention mechanism is used to extract semantic information of emojis in text. We participated in three sub-tasks of SemEval 2019 Task 6 as CN-HIT-MI.T team. Our macro-averaged F1-score in sub-task A is 0.768, ranking 28/103. We got 0.638 in sub-task B, ranking 30/75. In sub-task C, we got 0.549, ranking 22/65. We also tried some other methods of not submitting results.
ConvAI at SemEval-2019 Task 6: Offensive Language Identification and Categorization with Perspective and BERT
Authors: John Pavlopoulos, Nithum Thain, Lucas Dixon, Ion Androutsopoulos
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Language Identification

This paper presents the application of two strong baseline systems for toxicity detection and evaluates their performance in identifying and categorizing offensive language in social media. PERSPECTIVE is an API, that serves multiple machine learning models for the improvement of conversations online, as well as a toxicity detection system, trained on a wide variety of comments from platforms across the Internet. BERT is a recently popular language representation model, fine tuned per task and achieving state of the art performance in multiple NLP tasks. PERSPECTIVE performed better than BERT in detecting toxicity, but BERT was much better in categorizing the offensive type. Both baselines were ranked surprisingly high in the SEMEVAL-2019 OFFENSEVAL competition, PERSPECTIVE in detecting an offensive post (12th) and BERT in categorizing it (11th). The main contribution of this paper is the assessment of two strong baselines for the identification (PERSPECTIVE) and the categorization (BERT) of offensive language with little or no additional training data.
DA-LD-Hildesheim at SemEval-2019 Task 6: Tracking Offensive Content with Deep Learning using Shallow Representation
Authors: S Modha, ip, Prasenjit Majumder, Daksh Patel
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: General Classification, Information Retrieval, Retrieval

This paper presents the participation of team DA-LD-Hildesheim of Information Retrieval and Language Processing lab at DA-IICT, India in Semeval-19 OffenEval track. The aim of this shared task is to identify offensive content at fined-grained level granularity. The task is divided into three sub-tasks. The system is required to check whether social media posts contain any offensive or profane content or not, targeted or untargeted towards any entity and classifying targeted posts into the individual, group or other categories. Social media posts suffer from data sparsity problem, Therefore, the distributed word representation technique is chosen over the Bag-of-Words for the text representation. Since limited labeled data was available for the training, pre-trained word vectors are used and fine-tuned on this classification task. Various deep learning models based on LSTM, Bidirectional LSTM, CNN, and Stacked CNN are used for the classification. It has been observed that labeled data was highly affected with class imbalance and our technique to handle the class-balance was not effective, in fact performance was degraded in some of the runs. Macro F1 score is used as a primary evaluation metric for the performance. Our System achieves Macro F1 score = 0.7833 in sub-task A, 0.6456 in the sub-task B and 0.5533 in the sub-task C.
DeepAnalyzer at SemEval-2019 Task 6: A deep learning-based ensemble method for identifying offensive tweets
Authors: Gretel Liz De la Pe{\~n}a, Paolo Rosso
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Language Identification, Part-Of-Speech Tagging

This paper describes the system we developed for SemEval 2019 on Identifying and Categorizing Offensive Language in Social Media (OffensEval - Task 6). The task focuses on offensive language in tweets. It is organized into three sub-tasks for offensive language identification; automatic categorization of offense types and offense target identification. The approach for the first subtask is a deep learning-based ensemble method which uses a Bidirectional LSTM Recurrent Neural Network and a Convolutional Neural Network. Additionally we use the information from part-of-speech tagging of tweets for target identification and combine previous results for categorization of offense types.
NLP at SemEval-2019 Task 6: Detecting Offensive language using Neural Networks
Authors: Prashant Kapil, Asif Ekbal, Dipankar Das
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Deep Learning, POS

In this paper we built several deep learning architectures to participate in shared task OffensEval: Identifying and categorizing Offensive language in Social media by semEval-2019. The dataset was annotated with three level annotation schemes and task was to detect between offensive and not offensive, categorization and target identification in offensive contents. Deep learning models with POS information as feature were also leveraged for classification. The three best models that performed best on individual sub tasks are stacking of CNN-Bi-LSTM with Attention, BiLSTM with POS information added with word features and Bi-LSTM for third task. Our models achieved a Macro F1 score of 0.7594, 0.5378 and 0.4588 in Task(A,B,C) respectively with rank of 33rd, 54th and 52nd out of 103, 75 and 65 submissions.The three best models that performed best on individual sub task are using Neural Networks.
Emad at SemEval-2019 Task 6: Offensive Language Identification using Traditional Machine Learning and Deep Learning approaches
Authors: Emad Kebriaei, Samaneh Karimi, Nazanin Sabri, Azadeh Shakery
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Data Augmentation, Language Identification

In this paper, the used methods and the results obtained by our team, entitled Emad, on the OffensEval 2019 shared task organized at SemEval 2019 are presented. The OffensEval shared task includes three sub-tasks namely Offensive language identification, Automatic categorization of offense types and Offense target identification. We participated in sub-task A and tried various methods including traditional machine learning methods, deep learning methods and also a combination of the first two sets of methods. We also proposed a data augmentation method using word embedding to improve the performance of our methods. The results show that the augmentation approach outperforms other methods in terms of macro-f1.
Embeddia at SemEval-2019 Task 6: Detecting Hate with Neural Network and Transfer Learning Approaches
Authors: Andra{\v{z}} Pelicon, Matej Martinc, Petra Kralj Novak
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Language Identification, Transfer Learning

SemEval 2019 Task 6 was OffensEval: Identifying and Categorizing Offensive Language in Social Media. The task was further divided into three sub-tasks: offensive language identification, automatic categorization of offense types, and offense target identification. In this paper, we present the approaches used by the Embeddia team, who qualified as fourth, eighteenth and fifth on the tree sub-tasks. A different model was trained for each sub-task. For the first sub-task, we used a BERT model fine-tuned on the OLID dataset, while for the second and third tasks we developed a custom neural network architecture which combines bag-of-words features and automatically generated sequence-based features. Our results show that combining automatically and manually crafted features fed into a neural architecture outperform transfer learning approach on more unbalanced datasets.
Fermi at SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media using Sentence Embeddings
Authors: Vijayasaradhi Indurthi, Bakhtiyar Syed, Manish Shrivastava, Manish Gupta, Vasudeva Varma
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Position, Sentence, Sentence Embeddings

This paper describes our system (Fermi) for Task 6: OffensEval: Identifying and Categorizing Offensive Language in Social Media of SemEval-2019. We participated in all the three sub-tasks within Task 6. We evaluate multiple sentence embeddings in conjunction with various supervised machine learning algorithms and evaluate the performance of simple yet effective embedding-ML combination algorithms. Our team Fermi{'}s model achieved an F1-score of 64.40{\%}, 62.00{\%} and 62.60{\%} for sub-task A, B and C respectively on the official leaderboard. Our model for sub-task C which uses pre-trained ELMo embeddings for transforming the input and uses SVM (RBF kernel) for training, scored third position on the official leaderboard. Through the paper we provide a detailed description of the approach, as well as the results obtained for the task.
HAD-T\"ubingen at SemEval-2019 Task 6: Deep Learning Analysis of Offensive Language on Twitter: Identification and Categorization
Authors: Himanshu Bansal, Daniel Nagel, Anita Soloveva
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Language Identification

This paper describes the submissions of our team, HAD-T{\"u}bingen, for the SemEval 2019 - Task 6: {``}OffensEval: Identifying and Categorizing Offensive Language in Social Media{''}. We participated in all the three sub-tasks: Sub-task A - {``}Offensive language identification{''}, sub-task B - {``}Automatic categorization of offense types{''} and sub-task C - {``}Offense target identification{''}. As a baseline model we used a Long short-term memory recurrent neural network (LSTM) to identify and categorize offensive tweets. For all the tasks we experimented with external databases in a postprocessing step to enhance the results made by our model. The best macro-average F1 scores obtained for the sub-tasks A, B and C are 0.73, 0.52, and 0.37, respectively.
HHU at SemEval-2019 Task 6: Context Does Matter - Tackling Offensive Language Identification and Categorization with ELMo
Authors: Alex Oberstrass, er, Julia Romberg, Anke Stoll, Stefan Conrad
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Language Identification

We present our results for OffensEval: Identifying and Categorizing Offensive Language in Social Media (SemEval 2019 - Task 6). Our results show that context embeddings are important features for the three different sub-tasks in connection with classical machine and with deep learning. Our best model reached place 3 of 75 in sub-task B with a macro $F_1$ of 0.719. Our approaches for sub-task A and C perform less well but could also deliver promising results.
Hope at SemEval-2019 Task 6: Mining social media language to discover offensive language
Authors: Gabriel Florentin Patras, Diana Florina Lungu, Daniela Gifu, Tr, Diana abat
Date: 2019-06-01 00:00:00
Proceeding: SEMEVAL 2019 6

User{'}s content share through social media has reached huge proportions nowadays. However, along with the free expression of thoughts on social media, people risk getting exposed to various aggressive statements. In this paper, we present a system able to identify and classify offensive user-generated content.