pwc-archive/papers-with-abstracts huggingface dataset:
T\"eXtmarkers at SemEval-2020 Task 10: Emphasis Selection with Agreement Dependent Crowd Layers
Authors: Kevin Glocker, Stefanos Andreas Markianos Wright
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020

In visual communication, the ability of a short piece of text to catch someone{'}s eye in a single glance or from a distance is of paramount importance. In our approach to the SemEval-2020 task {``}Emphasis Selection For Written Text in Visual Media{''}, we use contextualized word representations from a pretrained model of the state-of-the-art BERT architecture together with a stacked bidirectional GRU network to predict token-level emphasis probabilities. For tackling low inter-annotator agreement in the dataset, we attempt to model multiple annotators jointly by introducing initialization with agreement dependent noise to a crowd layer architecture. We found our approach to both perform substantially better than initialization with identities for this purpose and to outperform a baseline trained with token level majority voting. Our submission system reaches substantially higher Match m on the development set than the task baseline (0.779), but only slightly outperforms the test set baseline (0.754) using a three model ensemble.
UIC-NLP at SemEval-2020 Task 10: Exploring an Alternate Perspective on Evaluation
Authors: Philip Hossu, Natalie Parde
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020

In this work we describe and analyze a supervised learning system for word emphasis selection in phrases drawn from visual media as a part of the Semeval 2020 Shared Task 10. More specifically, we begin by briefly introducing the shared task problem and provide an analysis of interesting and relevant features present in the training dataset. We then introduce our LSTM-based model and describe its structure, input features, and limitations. Our model ultimately failed to beat the benchmark score, achieving an average match() score of 0.704 on the validation data (0.659 on the test data) but predicted 84.8{\%} of words correctly considering a 0.5 threshold. We conclude with a thorough analysis and discussion of erroneous predictions with many examples and visualizations.
YNU-HPCC at SemEval-2020 Task 10: Using a Multi-granularity Ordinal Classification of the BiLSTM Model for Emphasis Selection
Authors: Dawei Liao, Jin Wang, Xuejie Zhang
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Language Modeling, Language Modelling, Ordinal Classification

In this study, we propose a multi-granularity ordinal classification method to address the problem of emphasis selection. In detail, the word embedding is learned from Embeddings from Language Model (ELMO) to extract feature vector representation. Then, the ordinal classifica-tions are implemented on four different multi-granularities to approximate the continuous em-phasize values. Comparative experiments were conducted to compare the model with baseline in which the problem is transformed to label distribution problem.
3218IR at SemEval-2020 Task 11: Conv1D and Word Embedding in Propaganda Span Identification at News Articles
Authors: Dimas Sony Dewantara, Indra Budi, Muhammad Okky Ibrohim
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles, Information Retrieval, Propaganda span identification, Retrieval

In this paper, we present the result of our experiment with a variant of 1 Dimensional Convolutional Neural Network (Conv1D) hyper-parameters value. We describe the system entered by the team of Information Retrieval Lab. Universitas Indonesia (3218IR) in the SemEval 2020 Task 11 Sub Task 1 about propaganda span identification in news articles. The best model obtained an F1 score of 0.24 in the development set and 0.23 in the test set. We show that there is a potential for performance improvement through the use of models with appropriate hyper-parameters. Our system uses a combination of Conv1D and GloVe as Word Embedding to detect propaganda in the fragment text level.
Hitachi at SemEval-2020 Task 11: An Empirical Study of Pre-Trained Transformer Family for Propaganda Detection
Authors: Gaku Morio, Terufumi Morishita, Hiroaki Ozaki, Toshinori Miyoshi
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Propaganda detection, Propaganda span identification

In this paper, we show our system for SemEval-2020 task 11, where we tackle propaganda span identification (SI) and technique classification (TC). We investigate heterogeneous pre-trained language models (PLMs) such as BERT, GPT-2, XLNet, XLM, RoBERTa, and XLM-RoBERTa for SI and TC fine-tuning, respectively. In large-scale experiments, we found that each of the language models has a characteristic property, and using an ensemble model with them is promising. Finally, the ensemble model was ranked 1st amongst 35 teams for SI and 3rd amongst 31 teams for TC.
JUST at SemEval-2020 Task 11: Detecting Propaganda Techniques Using BERT Pre-trained Model
Authors: Ola AlTiti, Malak Abdullah, Rasha Obiedat
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles, Language Modeling, Language Modelling

This paper presents the submission to semeval-2020 task 11, Detection of Propaganda Techniques in News Articles. Knowing that there are two subtasks in this competition, we have participated in the Technique Classification subtask (TC), which aims to identify the propaganda techniques used in a specific propaganda span. We have used and implemented various models to detect propaganda. Our proposed model is based on BERT uncased pre-trained language model as it has achieved state-of-the-art performance on multiple NLP benchmarks. The performance results of our proposed model have scored 0.55307 F1-Score, which outperforms the baseline model provided by the organizers with 0.2519 F1-Score, and our model is 0.07 away from the best performing team. Compared to other participating systems, our submission is ranked 15th out of 31 participants.
NLFIIT at SemEval-2020 Task 11: Neural Network Architectures for Detection of Propaganda Techniques in News Articles
Authors: Matej Martinkovic, Samuel Pecar, Marian Simko
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles, Propaganda span identification

Since propaganda became more common technique in news, it is very important to look for possibilities of its automatic detection. In this paper, we present neural model architecture submitted to the SemEval-2020 Task 11 competition: {``}Detection of Propaganda Techniques in News Articles{''}. We participated in both subtasks, propaganda span identification and propaganda technique classification. Our model utilizes recurrent Bi-LSTM layers with pre-trained word representations and also takes advantage of self-attention mechanism. Our model managed to achieve score 0.405 F1 for subtask 1 and 0.553 F1 for subtask 2 on test set resulting in 17th and 16th place in subtask 1 and subtask 2, respectively.
PsuedoProp at SemEval-2020 Task 11: Propaganda Span Detection Using BERT-CRF and Ensemble Sentence Level Classifier
Authors: Aniruddha Chauhan, Harshita Diddee
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles, Propaganda detection, Sentence

This paper explains our teams{'} submission to the Shared Task of Fine-Grained Propaganda Detection in which we propose a sequential BERT-CRF based Span Identification model where the fine-grained detection is carried out only on the articles that are flagged as containing propaganda by an ensemble SLC model. We propose this setup bearing in mind the practicality of this approach in identifying propaganda spans in the exponentially increasing content base where the fine-tuned analysis of the entire data repository may not be the optimal choice due to its massive computational resource requirements. We present our analysis on different voting ensembles for the SLC model. Our system ranks 14th on the test set and 22nd on the development set and with an F1 score of 0.41 and 0.39 respectively.
SkoltechNLP at SemEval-2020 Task 11: Exploring Unsupervised Text Augmentation for Propaganda Detection
Authors: Daryna Dementieva, Igor Markov, Alexander Panchenko
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles, Data Augmentation, Propaganda detection, Text Augmentation

This paper presents a solution for the Span Identification (SI) task in the {``}Detection of Propaganda Techniques in News Articles{''} competition at SemEval-2020. The goal of the SI task is to identify specific fragments of each article which contain the use of at least one propaganda technique. This is a binary sequence tagging task. We tested several approaches finally selecting a fine-tuned BERT model as our baseline model. Our main contribution is an investigation of several unsupervised data augmentation techniques based on distributional semantics expanding the original small training dataset as applied to this BERT-based sequence tagger. We explore various expansion strategies and show that they can substantially shift the balance between precision and recall, while maintaining comparable levels of the F1 score.
Team DiSaster at SemEval-2020 Task 11: Combining BERT and Hand-crafted Features for Identifying Propaganda Techniques in News
Authors: Anders Kaas, Viktor Torp Thomsen, Barbara Plank
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles

The identification of communication techniques in news articles such as propaganda is important, as such techniques can influence the opinions of large numbers of people. Most work so far focused on the identification at the news article level. Recently, a new dataset and shared task has been proposed for the identification of propaganda techniques at the finer-grained span level. This paper describes our system submission to the subtask of technique classification (TC) for the SemEval 2020 shared task on detection of propaganda techniques in news articles. We propose a method of combining neural BERT representations with hand-crafted features via stacked generalization. Our model has the added advantage that it combines the power of contextual representations from BERT with simple span-based and article-based global features. We present an ablation study which shows that even though BERT representations are very powerful also for this task, BERT still benefits from being combined with carefully designed task-specific features.
Transformers at SemEval-2020 Task 11: Propaganda Fragment Detection Using Diversified BERT Architectures Based Ensemble Learning
Authors: Ekansh Verma, Vinodh Motupalli, Souradip Chakraborty
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles, Ensemble Learning

In this paper, we present our approach for the {'}Detection of Propaganda Techniques in News Articles{'} task as a part of the 2020 edition of International Workshop on Semantic Evaluation. The specific objective of this task is to identify and extract the text segments in which propaganda techniques are used. We propose a multi-system deep learning framework that can be used to identify the presence of propaganda fragments in a news article and also deep dive into the diverse enhancements of BERT architecture which are part of the final solution. Our proposed final model gave an F1-score of 0.48 on the test dataset.
TTUI at SemEval-2020 Task 11: Propaganda Detection with Transfer Learning and Ensembles
Authors: Moonsung Kim, Steven Bethard
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Propaganda detection, Transfer Learning

In this paper, we describe our approaches and systems for the SemEval-2020 Task 11 on propaganda technique detection. We fine-tuned BERT and RoBERTa pre-trained models then merged them with an average ensemble. We conducted several experiments for input representations dealing with long texts and preserving context as well as for the imbalanced class problem. Our system ranked 20th out of 36 teams with 0.398 F1 in the SI task and 14th out of 31 teams with 0.556 F1 in the TC task.
UAIC1860 at SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles
Authors: Vlad Ermurachi, Daniela Gifu
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles, Sentence

The {``}Detection of Propaganda Techniques in News Articles{''} task at the SemEval 2020 competition focuses on detecting and classifying propaganda, pervasive in news article. In this paper, we present a system able to evaluate on sentence level, three traditional text representation techniques for these study goals, using: tf*idf, word and character n-grams. Firstly, we built a binary classifier able to provide corresponding propaganda labels, propaganda or non-propaganda. Secondly, we build a multilabel multiclass model to identify applied propaganda.
UMSIForeseer at SemEval-2020 Task 11: Propaganda Detection by Fine-Tuning BERT with Resampling and Ensemble Learning
Authors: Yunzhe Jiang, Cristina Garbacea, Qiaozhu Mei
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles, Ensemble Learning, Propaganda detection

We describe our participation at the SemEval 2020 {``}Detection of Propaganda Techniques in News Articles{''} - Techniques Classification (TC) task, designed to categorize textual fragments into one of the 14 given propaganda techniques. Our solution leverages pre-trained BERT models. We present our model implementations, evaluation results and analysis of these results. We also investigate the potential of combining language models with resampling and ensemble learning methods to deal with data imbalance and improve performance.
UNTLing at SemEval-2020 Task 11: Detection of Propaganda Techniques in English News Articles
Authors: Maia Petee, Alexis Palmer
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Articles

Our system for the PropEval task explores the ability of semantic features to detect and label propagandistic rhetorical techniques in English news articles. For Subtask 2, labeling identified propagandistic fragments with one of fourteen technique labels, our system attains a micro-averaged F1 of 0.40; in this paper, we take a detailed look at the fourteen labels and how well our semantically-focused model detects each of them. We also propose strategies to fill the gaps.
WMD at SemEval-2020 Tasks 7 and 11: Assessing Humor and Propaganda Using Unsupervised Data Augmentation
Authors: Guillaume Daval-Frerot, Yannick Weis
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Data Augmentation

In this work, we combine the state-of-the-art BERT architecture with the semi-supervised learning technique UDA in order to exploit unlabeled raw data to assess humor and detect propaganda in the tasks 7 and 11 of the SemEval-2020 competition. The use of UDA shows promising results with a systematic improvement of the performances over the four different subtasks, and even outperforms supervised learning with the additional labels of the Funlines dataset.
YNUtaoxin at SemEval-2020 Task 11: Identification Fragments of Propaganda Technique by Neural Sequence Labeling Models with Different Tagging Schemes and Pre-trained Language Model
Authors: Xin Tao, Xiaobing Zhou
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Language Modeling, Language Modelling

We only participated in the first subtask, and a neural sequence model was used to perform the sequence tagging task. We investigated the effects of different markup strategies on model performance. Bert that performed very well in NLP was used as a feature extractor.
AlexU-BackTranslation-TL at SemEval-2020 Task 12: Improving Offensive Language Detection Using Data Augmentation and Transfer Learning
Authors: Mai Ibrahim, Marwan Torki, Nagwa El-Makky
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Data Augmentation, Language Identification, Transfer Learning, Translation

Social media platforms, online news commenting spaces, and many other public forums have become widely known for issues of abusive behavior such as cyber-bullying and personal attacks. In this paper, we use the annotated tweets of the Offensive Language Identification Dataset (OLID) to train three levels of deep learning classifiers to solve the three sub-tasks associated with the dataset. Sub-task A is to determine if the tweet is toxic or not. Then, for offensive tweets, sub-task B requires determining whether the toxicity is targeted. Finally, for sub-task C, we predict the target of the offense; i.e. a group, individual, or other entity. In our solution, we tackle the problem of class imbalance in the dataset by using back translation for data augmentation and utilizing the fine-tuned BERT model in an ensemble of deep learning classifiers. We used this solution to participate in the three English sub-tasks of SemEval-2020 task 12. The proposed solution achieved 0.91393, 0.6300, and 0.57607 macro F1-average in sub-tasks A, B, and C respectively. We achieved the 9th, 14th, and 22nd places for sub-tasks A, B and C respectively.
ALT at SemEval-2020 Task 12: Arabic and English Offensive Language Identification in Social Media
Authors: Sabit Hassan, Younes Samih, Hamdy Mubarak, Ahmed Abdelali
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Language Identification

This paper describes the systems submitted by the Arabic Language Technology group (ALT) at SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media. We focus on sub-task A (Offensive Language Identification) for two languages: Arabic and English. Our efforts for both languages achieved more than 90{\%} macro-averaged F1-score on the official test set. For Arabic, the best results were obtained by a system combination of Support Vector Machine, Deep Neural Network, and fine-tuned Bidirectional Encoder Representations from Transformers (BERT). For English, the best results were obtained by fine-tuning BERT.
Amsqr at SemEval-2020 Task 12: Offensive Language Detection Using Neural Networks and Anti-adversarial Features
Authors: Alejandro Mosquera
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020

This paper describes a method and system to solve the problem of detecting offensive language in social media using anti-adversarial features. Our submission to the SemEval-2020 task 12 challenge was generated by an stacked ensemble of neural networks fine-tuned on the OLID dataset and additional external sources. For Task-A (English), text normalisation filters were applied at both graphical and lexical level. The normalisation step effectively mitigates not only the natural presence of lexical variants but also intentional attempts to bypass moderation by introducing out of vocabulary words. Our approach provides strong F1 scores for both 2020 (0.9134) and 2019 (0.8258) challenges.
BRUMS at SemEval-2020 Task 12: Transformer Based Multilingual Offensive Language Identification in Social Media
Authors: Tharindu Ranasinghe, Hansi Hettiarachchi
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Language Identification

In this paper, we describe the team \textit{BRUMS} entry to OffensEval 2: Multilingual Offensive Language Identification in Social Media in SemEval-2020. The OffensEval organizers provided participants with annotated datasets containing posts from social media in Arabic, Danish, English, Greek and Turkish. We present a multilingual deep learning model to identify offensive language in social media. Overall, the approach achieves acceptable evaluation scores, while maintaining flexibility between languages.
CoLi at UdS at SemEval-2020 Task 12: Offensive Tweet Detection with Ensembling
Authors: Kathryn Chapman, Johannes Bernhard, Dietrich Klakow
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: BIG-bench Machine Learning, Language Identification, regression, XLM-R

We present our submission and results for SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020) where we participated in offensive tweet classification tasks in English, Arabic, Greek, Turkish and Danish. Our approach included classical machine learning architectures such as support vector machines and logistic regression combined in an ensemble with a multilingual transformer-based model (XLM-R). The transformer model is trained on all languages combined in order to create a fully multilingual model which can leverage knowledge between languages. The machine learning model hyperparameters are fine-tuned and the statistically best performing ones included in the final ensemble.
CyberTronics at SemEval-2020 Task 12: Multilingual Offensive Language Identification over Social Media
Authors: Sayanta Paul, Sriparna Saha, Mohammed Hasanuzzaman
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Feature Engineering, Language Identification

The SemEval-2020 Task 12 (OffensEval) challenge focuses on detection of signs of offensiveness using posts or comments over social media. This task has been organized for several languages, e.g., Arabic, Danish, English, Greek and Turkish. It has featured three related sub-tasks for English language: sub-task A was to discriminate between offensive and non-offensive posts, the focus of sub-task B was on the type of offensive content in the post and finally, in sub-task C, proposed systems had to identify the target of the offensive posts. The corpus for each of the languages is developed using the posts and comments over Twitter, a popular social media platform. We have participated in this challenge and submitted results for different languages. The current work presents different machine learning and deep learning techniques and analyzes their performance for offensiveness prediction which involves various classifiers and feature engineering schemes. The experimental analysis on the training set shows that SVM using language specific pre-trained word embedding (Fasttext) outperforms the other methods. Our system achieves a macro-averaged F1 score of 0.45 for Arabic language, 0.43 for Greek language and 0.54 for Turkish language.
DoTheMath at SemEval-2020 Task 12 : Deep Neural Networks with Self Attention for Arabic Offensive Language Detection
Authors: Zoher Orabe, Bushr Haddad, Nada Ghneim, Anas Al-Abood
Date: 2020-12-01 00:00:00
Proceeding: SEMEVAL 2020
Tasks: Word Embeddings

This paper describes our team work and submission for the SemEval 2020 (Sub-Task A) {``}Offensive Eval: Identifying and Categorizing Offensive Arabic Language in Arabic Social Media{''}. Our two baseline models were based on different levels of representation: character vs. word level. In word level based representation we implemented a convolutional neural network model and a bi-directional GRU model. In character level based representation we implemented a hyper CNN and LSTM model. All of these models have been further augmented with attention layers for a better performance on our task. We also experimented with three types of static word embeddings: word2vec, FastText, and Glove, in addition to emoji embeddings, and compared the performance of the different deep learning models on the dataset provided by this task. The bi-directional GRU model with attention has achieved the highest score (0.85{\%} F1 score) among all other models.