H umor AAC at S em E val-2020 Task 7: Assessing the Funniness of Edited News Headlines through Regression and Trump Mentions (2020)

Abstract: In this paper we describe our contribution to the Semeval-2020 Humor Assessment task. We essentially use three different features that are passed into a ridge regression to determine a funniness score for an edited news headline: statistical, count-based features, semantic features and contextual information. For deciding which one of two given edited headlines is funnier, we additionally use scoring information and logistic regression. Our work was mostly concentrated on investigating features, rather than improving prediction based on pre-trained language models. The resulting system is task-specific, lightweight and performs above the majority baseline. Our experiments indicate that features related to socio-cultural context, in our case mentions of Donald Trump, generally perform better than context-independent features like headline length.

## introduction humor is an essential part of natural everyday human communication. a huge range of research was devoted to the nature of humor in various fields such as psychology, linguistics and philosophy resulting in various theories such as incongruity-resolution theory, non-cooperative theory, superiority theory, release theories, semantic script theory and general theory of verbal humor @xcite . none of these theories could give a complete and comprehensive explanation of humor phenomena so far. for the field of computational linguistics humor is being quite a difficulty as well. because of its complexity and inherent subjectivity, the development of automatic humor recognition and assessment poses a great challenge in computational linguistics, and therefore is a popular subject in various shared task competitions @xcite and independent research @xcite @xcite . this challenge is related to the meaning extraction task in the domain of sentiment analysis since a great deal of naturally produced text contains humor in one form or another. the system described in this paper was created for the humor assessment task within the international workshop on semantic evaluation semeval-2020 @xcite . this task is aiming to develop a system that automatically evaluates the humorousness of the edited headlines, made funnier with short one-word edits, and evaluated by human annotators. we believe, that this task can be an important step on a way to the development of better speech recognition systems and to the understanding of humor as a phenomenon in general. the presented system is based on ridge regression and neural networks. in essence we pass a set of count-based, semantic, and context-oriented features into the ridge regression algorithm. the main discovery of our research is the importance of task-and data-specific features on the performance of a humor-detection system. due to the social, context-dependent nature of humor, the addition of two features pertaining to donald trump, originally added only for experimental purposes, significantly improved our score. therefore we can conclude that social context features like this are even more relevant to the perceived funniness than some other statistical features, for example those proposed by @xcite . this finding can help to improve humor detection systems by taking social context into account. an automatic way of detecting a social context is a perspective topic for future research. ## experimental setup the competition's evaluation metric is the root-mean-square error (rmse) for task 1 and label accuracy for task 2. to evaluate our system, we both used a 80/20 train/test split on the train set as well as using the development set (both sets provided by task organizers). the final evaluation to determine our ranking was based on an additional test set. for all evaluation not through codalab, building and tuning linear models, and feature extraction with tf-idf we used the python scikit-learn foot_1 library. for tokenizing, part-of-speech tagging and semantic similarity information (wordnet) as well as calculating the edit distance, nltk foot_2 was used. as input for our neural network, headlines were preprocessed using keras foot_3 and transformed with glove. we chose the pre-trained 100d vector variant sourced from wikipedia and gigaword 5 containing 6 billion tokens foot_4 . the neural networks themselves are also implemented with keras. we did not tune our neural networks when it comes to the number of layers or parameters but we used early stopping as a callback to prevent overfitting. ## conclusions to conclude our research we would like to reflect a bit more on a humor nature and humor recognition challenges. humor is not only a linguistic phenomenon, but it can be represented by text and therefore provides a significant challenge for the nlp field. it is clear that humor cannot be accessed only by a language model by itself, no matter how good it is. therefore the humor detection systems should be provided with some context not only in the sense of linguistic context but a broader, socio-cultural context. in our case such a context was provided by manual data exploration and implementation of corresponding features, but there are ways of doing it automatically. just checking if a sentence contains a reference to trump is not enough information to assess funniness of course, especially since his specific relevancy in culture might fade over the years. one of the possible ways to detect social-context related features would be named entity recognition in the dataset. names can be crucially important to the task of obtaining social context, since they carry a lot of social-related information, being at the same time a references to certain common ground of readers. thus in our example trump denotes not just a person, but a whole social phenomenon related to him, including associations, attitudes and ideas. therefore recognising, categorising and evaluating named entities can be helpful to detect socio-cultural context, being significant for the overall performance of a humor detection system.