Hasyarasa at S em E val-2020 Task 7: Quantifying Humor as Departure from Expectedness (2020)

Abstract: This paper describes our system submission Hasyarasa for the SemEval-2020 Task-7: Assessing Humor in Edited News Headlines. This task has two subtasks. The goal of Subtask 1 is to predict the mean funniness of the edited headline given the original and the edited headline. In Subtask 2, given two edits on the original headline, the goal is to predict the funnier of the two. We observed that the departure from expected state/ actions of situations/ individuals is the cause of humor in the edited headlines. We propose two novel features: Contextual Semantic Distance and Contextual Neighborhood Distance to estimate this departure and thus capture the contextual absurdity and hence the humor in the edited headlines. We have used these features together with a Bi-LSTM Attention based model and have achieved 0.53310 RMSE for Subtask 1 and 60.19% accuracy for Subtask 2.

## introduction humor is a highly intellectual and communicative activity that promotes laughter and provides amusement. a great sense of humor refers to the ability to perceive things as funny. with the advances in natural language processing (nlp) techniques, there has been considerable research effort in developing artificial sense of humor in machines. much of the existing work on humor recognition is done in evaluating whether a given text is humorous or non-humorous using datasets such as twitter, yelp reviews, reddit jokes and other sources @xcite . however, the problem dealt in semeval-2020 task 7: assessing the funniness of edited news headlines is not about classifying a given text as humorous or non-humorous. it is about how a micro-edit on a news headline can change it from non-humorous to humorous @xcite . there are two subtasks in this task. subtask 1: given the original and edited headline, the task is to predict the mean funniness of the edited headline. subtask 2: given the original headline and the two edited versions of the original headline, the task is to predict which edited version is the funnier of the two edited versions. the evaluation criteria, as set by the organizers for subtask 1 is root mean squared error (rmse) and for subtask 2 is accuracy. in this work, our contribution to the challenge is to model two novel features namely contextual semantic distance (csd), and contextual neighborhood distance (cnd) which are based on our observation that the departure from expected state/ actions of the situations/ individuals is the cause of humor in the edited news headlines. the remainder of the paper is organized as follows: in section 2, we present related work. in section 3, we give a brief description of hasyrasa -our system for quantifying humor, discuss the proposed features and compare our results with a model that incorporates similarity features. results and analysis are presented in section 4 and section 5 respectively. section 6 concludes the paper. ## related work the task of assessing humor in micro-edited text such as news headlines is a less explored research area. most of the previous work is focused on humor recognition which is formulated as a binary classification problem i.e., determining whether a given text is humorous or non-humorous. various datasets like pun of the day @xcite , reddit jokes @xcite , 16000 one-liners @xcite , short jokes foot_0 have been used or created for recognizing humor. researchers have adopted traditional machine learning algorithms like svm and naive bayes @xcite , deep learning architectures like convolution neural network (cnn) @xcite , long short-term memory (lstm) @xcite , bidirectional long-short term memory (bi-lstm) @xcite , transformer architecture @xcite for humor detection. @xcite discussed about theories of humor and @xcite argued humor is directly correlated with incongruity. however, the challenge in the present work is to detect humor created when a headline is edited. for this purpose we propose the use of two novel features -contextual semantic distance (csd) and contextual neighborhood distance (cnd), and compare our results with a model that incorporates similarity features (sf). we employ cnn @xcite , lstm @xcite and attention mechanism @xcite to ( 1 ) predict intensity of humor in edited news headline and, (2) predict which edited news headline is more humorous of the two edited versions. from among the existing research, work done by @xcite is semantically closest to our position on unexpectedness as a source of humor. ## system description in this section we describe our system: 'hasyarasa'. the word hasyarasa in sanskrit means (creating) a mood of laughter. we employ hasyarasa to quantify the contextual unexpectedness, which is the cause of humor in the edited news headlines. in the following subsections, we start with the discussion of novel features in our approach. next, we give a detailed description of our model architecture and baselines. the data preprocessing and training details are described in the appendix. ## results our results for subtask 1 and subtask 2 are reported in in order to understand the effectiveness of the features, we experimented with different permutations of features with attention mechanism. the results are shown in ## analysis the features csd, cnd and sf are engineered keeping in mind the characteristics of the dataset. we theorized that quantifying the incongruities in the edited news headlines focusing on the departure from expected state which is along the lines of @xcite will correlate with humor scores and our experiments validate this theory. while the csd feature captures the relevance of the edited word keeping in mind just the context of the original headline, the cnd feature builds on existing knowledge of the world to capture the contextual relevance. the datasets provided by the organizers of the competition are mostly biased towards american politics. therefore, we selected kaggle's all the news dataset for building the knowledge graph in order to evaluate the cnd feature. the knowledge graph implementation is heavily dependent on the corpus it is built on. therefore, bigger and more elaborate news headlines datasets will help build a stronger knowledge graph. such a graph will be more efficient in capturing the contextual neighborhood. humor can also be estimated by comparing the representations of original and edited headlines. sf which is primarily an architecture specific feature computes the similarity between original and edited headlines representations from attention layer for estimating humor. the computation of both csd and cnd features depend upon word embeddings (glove in our case) for calculating the cosine similarity between the words. hence the efficiency of these features for this problem is limited by the limitations of word embeddings itself. after conducting elaborate experimentation, we have observed that even though the usage of csd and cnd improves the performance in terms of rmse (for subtask 1) and accuracy (for subtask 2), the improvement is not significant over that of the standard bilstm attention model. one of the primary drawbacks of word embeddings (word vector space models in general) is that words with multiple meanings are collated into a single vector representation in the semantic space. in linguistic terms, figures of speech such as polysemy and homonymy are not handled properly using word embedding models. for example, in the following news headline taken from the dataset-'rex tillerson : us has direct channels to pyongyang', it is impossible to understand if the word channel(s) is related to its word sense implying a strait (geography) or the word sense of a frequency band (telecommunications) or any other sense associated with the word channel(s). such usages of words with multiple meanings are often found in english texts. neologism is defined as a relatively new or isolated term, word, or phrase which is in the process of entering colloquial usage, but has not yet been fully accepted into mainstream language. an example of neologism is the word webinar which has its roots in the words web and seminar and essentially means an online seminar. neologisms are often driven by changes in culture and technology and can be found very often in daily news headlines. word embedding models like glove and word2vec have no way to learn the representation of out-of-vocabulary (oov) words, which often causes difficulties for the model to understand such neologistic word usages. in order to tackle this problem, we have also experimented with custom embeddings created over news headlines dataset like kaggle's all the news dataset. however, we did not obtain any better result than what we did with glove embeddings, which can be primarily attributed to the amount of vastness and elaborateness of such news datasets. so, we believe a bigger and more elaborate news headlines dataset would surely boost the scores to a higher degree. ## conclusion in this paper, we presented our system on the task of assessing humor in edited news headlines in semeval-2020. our experiments with csd, cnd and sf features combined with attention mechanism proved to be reasonably effective in assessing humor in edited news headlines. future work involves collecting more extensive news headlines datasets and building a stronger knowledge graph. using the datasets provided in the competition, it would be an interesting exercise to approach the problem for generating a humorous headline given a news headline and predicting the target word of the headline that should be micro-edited.