pwc-archive/papers-with-abstracts huggingface dataset:
SarcasmDet at SemEval-2021 Task 7: Detect Humor and Offensive based on Demographic Factors using RoBERTa Pre-trained Model
Authors: Dalya Faraj, Malak Abdullah
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Task 2

This paper presents one of the top winning solution systems for task 7 at SemEval2021, HaHackathon: Detecting and Rating Humor and Offense. This competition is divided into two tasks, task1 with three sub-tasks 1a,1b, and 1c, and task2. The goal for task1 is to predict if the text would be considered humorous or not, and if it is yes, then predict how humorous it is and whether the humor rating would be perceived as controversial. The goal of the task2 is to predict how the text is considered offensive for users in general. Our solution has been developed using RoBERTa pre-trained model with ensemble techniques. The paper describes the submitted solution system{'}s architecture with the experiments and the hyperparameter tuning that led to this robust system. Our model ranked third and fourth places out of 50 teams in tasks 1c and 1a with F1-Score of 0.6270 and 0.9675, respectively. At the same time, the model ranked one of the top 10 models in task 1b and task 2 with an RMSE scores of 0.5446 and 0.4469, respectively.
IITK@LCP at SemEval-2021 Task 1: Classification for Lexical Complexity Regression Task
Authors: Neil Shirude, Sagnik Mukherjee, Tushar Shandhilya, Ananta Mukherjee, Ashutosh Modi
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, regression, Task 2

This paper describes our contribution to SemEval 2021 Task 1 (Shardlow et al., 2021): Lexical Complexity Prediction. In our approach, we leverage the ELECTRA model and attempt to mirror the data annotation scheme. Although the task is a regression task, we show that we can treat it as an aggregation of several classification and regression models. This somewhat counter-intuitive approach achieved an MAE score of 0.0654 for Sub-Task 1 and MAE of 0.0811 on Sub-Task 2. Additionally, we used the concept of weak supervision signals from Gloss-BERT in our work, and it significantly improved the MAE score in Sub-Task 1.
Alejandro Mosquera at SemEval-2021 Task 1: Exploring Sentence and Word Features for Lexical Complexity Prediction
Authors: Alejandro Mosquera
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Feature Engineering, Lexical Complexity Prediction, regression, Sentence

This paper revisits feature engineering approaches for predicting the complexity level of English words in a particular context using regression techniques. Our best submission to the Lexical Complexity Prediction (LCP) shared task was ranked 3rd out of 48 systems for sub-task 1 and achieved Pearson correlation coefficients of 0.779 and 0.809 for single words and multi-word expressions respectively. The conclusion is that a combination of lexical, contextual and semantic features can still produce strong baselines when compared against human judgement.
CompNA at SemEval-2021 Task 1: Prediction of lexical complexity analyzing heterogeneous features
Authors: Giuseppe Vettigli, Antonio Sorgente
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction

This paper describes the CompNa model that has been submitted to the Lexical Complexity Prediction (LCP) shared task hosted at SemEval 2021 (Task 1). The solution is based on combining features of different nature through an ensambling method based on Decision Trees and trained using Gradient Boosting. We discuss the results of the model and highlight the features with more predictive capabilities.
PolyU CBS-Comp at SemEval-2021 Task 1: Lexical Complexity Prediction (LCP)
Authors: Rong Xiang, Jinghang Gu, Emmanuele Chersoni, Wenjie Li, Qin Lu, Chu-Ren Huang
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, Sentence, Word Embeddings

In this contribution, we describe the system presented by the PolyU CBS-Comp Team at the Task 1 of SemEval 2021, where the goal was the estimation of the complexity of words in a given sentence context. Our top system, based on a combination of lexical, syntactic, word embeddings and Transformers-derived features and on a Gradient Boosting Regressor, achieves a top correlation score of 0.754 on the subtask 1 for single words and 0.659 on the subtask 2 for multiword expressions.
DeepBlueAI at SemEval-2021 Task 1: Lexical Complexity Prediction with A Deep Ensemble Approach
Authors: Chunguang Pan, Bingyan Song, Shengguang Wang, Zhipeng Luo
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Data Augmentation, Lexical Complexity Prediction, Lexical Simplification, Reading Comprehension

Lexical complexity plays an important role in reading comprehension. lexical complexity prediction (LCP) can not only be used as a part of Lexical Simplification systems, but also as a stand-alone application to help people better reading. This paper presents the winning system we submitted to the LCP Shared Task of SemEval 2021 that capable of dealing with both two subtasks. We first perform fine-tuning on numbers of pre-trained language models (PLMs) with various hyperparameters and different training strategies such as pseudo-labelling and data augmentation. Then an effective stacking mechanism is applied on top of the fine-tuned PLMs to obtain the final prediction. Experimental results on the Complex dataset show the validity of our method and we rank first and second for subtask 2 and 1.
CS-UM6P at SemEval-2021 Task 1: A Deep Learning Model-based Pre-trained Transformer Encoder for Lexical Complexity
Authors: Nabil El Mamoun, Abdelkader El Mahdaouy, Abdellah El Mekki, Kabil Essefar, Ismail Berrada
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Deep Learning, Lexical Complexity Prediction

Lexical Complexity Prediction (LCP) involves assigning a difficulty score to a particular word or expression, in a text intended for a target audience. In this paper, we introduce a new deep learning-based system for this challenging task. The proposed system consists of a deep learning model, based on pre-trained transformer encoder, for word and Multi-Word Expression (MWE) complexity prediction. First, on top of the encoder{'}s contextualized word embedding, our model employs an attention layer on the input context and the complex word or MWE. Then, the attention output is concatenated with the pooled output of the encoder and passed to a regression module. We investigate both single-task and joint training on both Sub-Tasks data using multiple pre-trained transformer-based encoders. The obtained results are very promising and show the effectiveness of fine-tuning pre-trained transformers for LCP task.
Cambridge at SemEval-2021 Task 1: An Ensemble of Feature-Based and Neural Models for Lexical Complexity Prediction
Authors: Zheng Yuan, Gladys Tyen, David Strohmaier
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, Multi-Task Learning, regression

This paper describes our submission to the SemEval-2021 shared task on Lexical Complexity Prediction. We approached it as a regression problem and present an ensemble combining four systems, one feature-based and three neural with fine-tuning, frequency pre-training and multi-task learning, achieving Pearson scores of 0.8264 and 0.7556 on the trial and test sets respectively (sub-task 1). We further present our analysis of the results and discuss our findings.
hub at SemEval-2021 Task 1: Fusion of Sentence and Word Frequency to Predict Lexical Complexity
Authors: Bo Huang, Yang Bai, Xiaobing Zhou
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, Sentence

In this paper, we propose a method of fusing sentence information and word frequency information for the SemEval 2021 Task 1-Lexical Complexity Prediction (LCP) shared task. In our system, the sentence information comes from the RoBERTa model, and the word frequency information comes from the Tf-Idf algorithm. Use Inception block as a shared layer to learn sentence and word frequency information We described the implementation of our best system and discussed our methods and experiments in the task. The shared task is divided into two sub-tasks. The goal of the two sub-tasks is to predict the complexity of a predetermined word. The shared task is divided into two subtasks. The goal of the two subtasks is to predict the complexity of a predetermined word. The evaluation index of the task is the Pearson correlation coefficient. Our best performance system has Pearson correlation coefficients of 0.7434 and 0.8000 in the single-token subtask test set and the multi-token subtask test set, respectively.
Manchester Metropolitan at SemEval-2021 Task 1: Convolutional Networks for Complex Word Identification
Authors: Robert Flynn, Matthew Shardlow
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Complex Word Identification, regression

We present two convolutional neural networks for predicting the complexity of words and phrases in context on a continuous scale. Both models utilize word and character embeddings alongside lexical features as inputs. Our system displays reasonable results with a Pearson correlation of 0.7754 on the task as a whole. We highlight the limitations of this method in properly assessing the context of the target text, and explore the effectiveness of both systems across a range of genres. Both models were submitted as part of LCP 2021, which focuses on the identification of complex words and phrases as a context dependent, regression based task.
UTFPR at SemEval-2021 Task 1: Complexity Prediction by Combining BERT Vectors and Classic Features
Authors: Gustavo Henrique Paetzold
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, Prediction

We describe the UTFPR systems submitted to the Lexical Complexity Prediction shared task of SemEval 2021. They perform complexity prediction by combining classic features, such as word frequency, n-gram frequency, word length, and number of senses, with BERT vectors. We test numerous feature combinations and machine learning models in our experiments and find that BERT vectors, even if not optimized for the task at hand, are a great complement to classic features. We also find that employing the principle of compositionality can potentially help in phrase complexity prediction. Our systems place 45th out of 55 for single words and 29th out of 38 for phrases.
RG PA at SemEval-2021 Task 1: A Contextual Attention-based Model with RoBERTa for Lexical Complexity Prediction
Authors: Gang Rao, Maochang Li, Xiaolong Hou, Lianxin Jiang, Yang Mo, Jianping Shen
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, Task 2

In this paper we propose a contextual attention based model with two-stage fine-tune training using RoBERTa. First, we perform the first-stage fine-tune on corpus with RoBERTa, so that the model can learn some prior domain knowledge. Then we get the contextual embedding of context words based on the token-level embedding with the fine-tuned model. And we use Kfold cross-validation to get K models and ensemble them to get the final result. Finally, we attain the 2nd place in the final evaluation phase of sub-task 2 with pearson correlation of 0.8575.
CSECU-DSG at SemEval-2021 Task 1: Fusion of Transformer Models for Lexical Complexity Prediction
Authors: Abdul Aziz, MD. Akram Hossain, Abu Nowshed Chy
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, Lexical Simplification, Sentence, Text Generation

Lexical complexity prediction (LCP) conveys the anticipation of the complexity level of a token or a set of tokens in a sentence. It plays a vital role in the improvement of various NLP tasks including lexical simplification, translations, and text generation. However, multiple meaning of a word in multiple circumstances, grammatical complex structure, and the mutual dependency of words in a sentence make it difficult to estimate the lexical complexity. To address these challenges, SemEval-2021 Task 1 introduced a shared task focusing on LCP and this paper presents our participation in this task. We proposed a transformer-based approach with sentence pair regression. We employed two fine-tuned transformer models. Including BERT and RoBERTa to train our model and fuse their predicted score to the complexity estimation. Experimental results demonstrate that our proposed method achieved competitive performance compared to the participants{'} systems.
CLULEX at SemEval-2021 Task 1: A Simple System Goes a Long Way
Authors: Greta Smolenska, Peter Kolb, Sinan Tang, Mironas Bitinis, H{\'e}ctor Hern{\'a}ndez, Elin Askl{\"o}v
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Feature Engineering, Lexical Complexity Prediction, Prediction, Word Embeddings

This paper presents the system we submitted to the first Lexical Complexity Prediction (LCP) Shared Task 2021. The Shared Task provides participants with a new English dataset that includes context of the target word. We participate in the single-word complexity prediction sub-task and focus on feature engineering. Our best system is trained on linguistic features and word embeddings (Pearson{'}s score of 0.7942). We demonstrate, however, that a simpler feature set achieves comparable results and submit a model trained on 36 linguistic features (Pearson{'}s score of 0.7925).
RS\_GV at SemEval-2021 Task 1: Sense Relative Lexical Complexity Prediction
Authors: Regina Stodden, Gayatri Venugopal
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, Prediction, Word Embeddings

We present the technical report of the system called RS{\_}GV at SemEval-2021 Task 1 on lexical complexity prediction of English words. RS{\_}GV is a neural network using hand-crafted linguistic features in combination with character and word embeddings to predict target words{'} complexity. For the generation of the hand-crafted features, we set the target words in relation to their senses. RS{\_}GV predicts the complexity well of biomedical terms but it has problems with the complexity prediction of very complex and very simple target words.
UNBNLP at SemEval-2021 Task 1: Predicting lexical complexity with masked language models and character-level encoders
Authors: Milton King, Ali Hakimi Parizi, Samin Fakharian, Paul Cook
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, Prediction

In this paper, we present three supervised systems for English lexical complexity prediction of single and multiword expressions for SemEval-2021 Task 1. We explore the use of statistical baseline features, masked language models, and character-level encoders to predict the complexity of a target token in context. Our best system combines information from these three sources. The results indicate that information from masked language models and character-level encoders can be combined to improve lexical complexity prediction.
ANDI at SemEval-2021 Task 1: Predicting complexity in context using distributional models, behavioural norms, and lexical resources
Authors: Armand Rotaru
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction

In this paper we describe our participation in the Lexical Complexity Prediction (LCP) shared task of SemEval 2021, which involved predicting subjective ratings of complexity for English single words and multi-word expressions, presented in context. Our approach relies on a combination of distributional models, both context-dependent and context-independent, together with behavioural norms and lexical resources.
JUST-BLUE at SemEval-2021 Task 1: Predicting Lexical Complexity using BERT and RoBERTa Pre-trained Language Models
Authors: Tuqa Bani Yaseen, Qusai Ismail, Sarah Al-Omari, Eslam Al-Sobh, Malak Abdullah
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Binary Classification, Text Simplification, Transfer Learning

Predicting the complexity level of a word or a phrase is considered a challenging task. It is even recognized as a crucial step in numerous NLP applications, such as text rearrangements and text simplification. Early research treated the task as a binary classification task, where the systems anticipated the existence of a word{'}s complexity (complex versus uncomplicated). Other studies had been designed to assess the level of word complexity using regression models or multi-labeling classification models. Deep learning models show a significant improvement over machine learning models with the rise of transfer learning and pre-trained language models. This paper presents our approach that won the first rank in the SemEval-task1 (sub stask1). We have calculated the degree of word complexity from 0-1 within a text. We have been ranked first place in the competition using the pre-trained language models Bert and RoBERTa, with a Pearson correlation score of 0.788.
cs60075\_team2 at SemEval-2021 Task 1 : Lexical Complexity Prediction using Transformer-based Language Models pre-trained on various text corpora
Authors: Abhilash Nandy, Sayantan Adak, Tanurima Halder, Sai Mahesh Pokala
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, Task 2

The main contribution of this paper is to fine-tune transformer-based language models pre-trained on several text corpora, some being general (E.g., Wikipedia, BooksCorpus), some being the corpora from which the CompLex Dataset was extracted, and others being from other specific domains such as Finance, Law, etc. We perform ablation studies on selecting the transformer models and how their individual complexity scores are aggregated to get the resulting complexity scores. Our method achieves a best Pearson Correlation of 0.784 in sub-task 1 (single word) and 0.836 in sub-task 2 (multiple word expressions).
C3SL at SemEval-2021 Task 1: Predicting Lexical Complexity of Words in Specific Contexts with Sentence Embeddings
Authors: Raul Almeida, Hegler Tissot, Marcos Didonet Del Fabro
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Sentence, Sentence Embeddings

We present our approach to predicting lexical complexity of words in specific contexts, as entered LCP Shared Task 1 at SemEval 2021. The approach consists of separating sentences into smaller chunks, embedding them with Sent2Vec, and reducing the embeddings into a simpler vector used as input to a neural network, the latter for predicting the complexity of words and expressions. Results show that the pre-trained sentence embeddings are not able to capture lexical complexity from the language when applied in cross-domain applications.
Stanford MLab at SemEval-2021 Task 1: Tree-Based Modelling of Lexical Complexity using Word Embeddings
Authors: Erik Rozi, Niveditha Iyer, Gordon Chi, Enok Choe, Kathy J. Lee, Kevin Liu, Patrick Liu, Zander Lack, Jillian Tang, Ethan A. Chi
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, Prediction, Reading Comprehension, Word Embeddings

This paper presents our system for the single- and multi-word lexical complexity prediction tasks of SemEval Task 1: Lexical Complexity Prediction. Text comprehension depends on the reader{'}s ability to understand the words present in it; evaluating the lexical complexity of such texts can enable readers to find an appropriate text and systems to tailor a text to an audience{'}s needs. We present our model pipeline, which applies a combination of embedding-based and manual features to predict lexical complexity on the CompLex English dataset using various tree-based and linear models. Our method is ranked 27 / 54 on single-word prediction and 14 / 37 on multi-word prediction.
archer at SemEval-2021 Task 1: Contextualising Lexical Complexity
Authors: Irene Russo
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: LEMMA, Lexical Complexity Prediction

Evaluating the complexity of a target word in a sentential context is the aim of the Lexical Complexity Prediction task at SemEval-2021. This paper presents the system created to assess single words lexical complexity, combining linguistic and psycholinguistic variables in a set of experiments involving random forest and XGboost regressors. Beyond encoding out-of-context information about the lemma, we implemented features based on pre-trained language models to model the target word{'}s in-context complexity.
katildakat at SemEval-2021 Task 1: Lexical Complexity Prediction of Single Words and Multi-Word Expressions in English
Authors: Katja Voskoboinik
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Lexical Complexity Prediction, regression

This paper describes systems submitted to Se- mEval 2021 Task 1: Lexical Complexity Prediction (LCP). We compare a linear and a non-linear regression models trained to work for both tracks of the task. We show that both systems are able to generalize better when supplied with information about complexities of single word and multi-word expression (MWE) targets simultaneously. This approach proved to be the most beneficial for multi-word expression targets. We also demonstrate that some hand-crafted features differ in their importance for the target types.
GX at SemEval-2021 Task 2: BERT with Lemma Information for MCL-WiC Task
Authors: Wanying Xie
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: LEMMA, Machine Translation, Task 2, Translation, Word Embeddings

This paper presents the GX system for the Multilingual and Cross-lingual Word-in-Context Disambiguation (MCL-WiC) task. The purpose of the MCL-WiC task is to tackle the challenge of capturing the polysemous nature of words without relying on a fixed sense inventory in a multilingual and cross-lingual setting. To solve the problems, we use context-specific word embeddings from BERT to eliminate the ambiguity between words in different contexts. For languages without an available training corpus, such as Chinese, we use neuron machine translation model to translate the English data released by the organizers to obtain available pseudo-data. In this paper, we apply our system to the English and Chinese multilingual setting and the experimental results show that our method has certain advantages.
hub at SemEval-2021 Task 2: Word Meaning Similarity Prediction Model Based on RoBERTa and Word Frequency
Authors: Bo Huang, Yang Bai, Xiaobing Zhou
Date: 2021-08-01 00:00:00
Proceeding: SEMEVAL 2021
Tasks: Binary Classification, Sentence, Task 2

This paper introduces the system description of the hub team, which explains the related work and experimental results of our team{'}s participation in SemEval 2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation (MCL-WiC). The data of this shared task is mainly some cross-language or multi-language sentence pair corpus. The languages covered in the corpus include English, Chinese, French, Russian, and Arabic. The task goal is to judge whether the same words in these sentence pairs have the same meaning in the sentence. This can be seen as a task of binary classification of sentence pairs. What we need to do is to use our method to determine as accurately as possible the meaning of the words in a sentence pair are the same or different. The model used by our team is mainly composed of RoBERTa and Tf-Idf algorithms. The result evaluation index of task submission is the F1 score. We only participated in the English language task. The final score of the test set prediction results submitted by our team was 84.60.