CSECU - DSG at S em E val-2021 Task 7: Detecting and Rating Humor and Offense Employing Transformers (2021)

Abstract: With the emerging trends of using online platforms, peoples are increasingly interested in express their opinion through humorous texts. Identifying and rating humorous texts poses unique challenges to NLP due to subjective phenomena i.e. humor may vary to gender, profession, age, and classes of people. Besides, words with multiple senses, cultural domain, and pragmatic competence also need to be considered. A humorous text may be offensive to others. To address these challenges SemEval-2021 introduced a HaHackathon task focusing on detecting and rating humorous and offensive texts. This paper describes our participation in this task. We employed a stacked embedding and fine-tuned transformer models based classification and regression approach from the features from GPT2 medium, BERT, and RoBERTa transformer models. Besides, we utilized the fine-tuned BERT and RoBERTa models to examine the performances. Our method achieved competitive performances in this task.

## introduction the exponential evolution of technologies and social platforms increases the growth of usergenerated content. therefore, detecting specific information from a pile of web data is ubiquitous. humor, like most figurative language, poses interesting linguistic challenges to nlp, due to its emphasis on multiple word senses, cultural knowledge, and pragmatic competence. the automated humor detection and rating is one of the challenging and promising tasks for their significance in the field of opinion mining, sentiment analysis, and emotion intelligence domain. the first two authors have equal contributions. numerous works have been done on humorous text identification task. @xcite proposed pre-trained bert architecture to identify humorous jokes from reddit dataset, puns, and short jokes dataset. @xcite ) used bert and glove embeddings with linear support vector classifier (svc), naive bayes, and random forest for predicting the final label to detect sarcasm in tweets. @xcite extracted features pertaining to sarcasm, humor, hate speech, as well as sentiment and ensemble them for sentiment classification. in @xcite , the pre-trained bert is adapted to the distantly supervised ner (named entity recognition) task with early stopping. some evaluation campaigns related to automatic humor detection also has been performed. @xcite includes a logistic regression baseline, a bilstm + attention-based learner, and a transfer learning approach with bert to tackle the internet humor at semeval-2020 task 8. however, most of the existing work addressed the humorous text identification problem as a binary classification task. but the humorous rating of a text can be varied at different scales based on its contents. moreover, a humorous text may portray offensive context to other users due to a variety of users' perceptions. to bridge this gap @xcite introduced a shared task at semeval-2021 focusing on detecting and rating humorous and offensive texts. the task comprises of four subtasks: task 1a is to predict whether a text is humorous or not, task 1b is to rate a humorous text with a score between 0 and 5, task 1c represents the subjectivity of humor appreciation by foretelling a humorous text is controversial or not, and task 2 intend to rate a text with a value between 0 and 5 based on its offensiveness. to tackle the challenges of this task, we employed a stacked embedding of various transformer models including gpt-2 medium, bert, and text text gpt-2 medium gpt-2 medium bert bert roberta roberta predicted label predicted label stacked embedding simple feed-forward linear architecture roberta with a simple linear architecture. besides, we conduct experiments and examine the individual performance of these three models. we used transformer-based models as they can learn the context of the sentence effectively and pushed the state-of-the-art for a wide range of downstream nlp tasks. the rest of the paper is structured as follows: section 2 describes our proposed framework. section 3 illustrates our experiment and evaluation. finally, we come to end with some conclusion and future research directions in section 4. ## proposed framework in this section, we present the details of our proposed approach for humorous and offensive text identification and rating. the overview of our proposed framework is depicted in figure 1 . given a text, we have extracted embedding features from three state-of-the-art transformerbased models including gpt-2 medium, bert, and roberta. the extracted embeddings are then unified through the stacked embedding scheme and the unified feature vector is then passed to the simple feed-forward linear architecture to obtain the prediction score. ## conclusion and future plan in this paper, we have tackled the problem of identifying and grading humorous texts as defined in semeval-2021 task 7. achieving high performance in humor and offensive text identification or ranking is hard due to its diverse contextual form. we have presented transformer-based language models including gpt-2 medium, bert, and roberta in a unified architecture using a stacked embedding scheme. experimental results demonstrated its effectiveness for the classification tasks. however, we have seen that finetuned transformer models performed better in the regression task. in the future, we have a plan to focus on direct inducing topic information into the transformerbased models. we also intend to explore how to tune the parameters for regression tasks in more efficient ways, which could yield better performances.