U ni T uebingen CL at S em E val-2020 Task 7: Humor Detection in News Headlines (2020)

Abstract: This paper describes the work done by the team UniTuebingenCL for the SemEval 2020 Task 7: “Assessing the Funniness of Edited News Headlines”. We participated in both sub-tasks: sub-task A, given the original and the edited headline, predicting the mean funniness of the edited headline; and sub-task B, given the original headline and two edited versions, predicting which edited version is the funnier of the two. A Ridge Regression model using Elmo and Glove embeddings as well as Truncated Singular Value Decomposition was used as the final model. A long short term memory model recurrent network (LSTM) served as another approach for assessing the funniness of a headline. For the first sub-task, we experimented with the extraction of multiple features to achieve lower Root Mean Squared Error. The lowest Root Mean Squared Error achieved was 0.575 for sub-task A, and the highest Accuracy was 0.618 for sub-task B.

## introduction with the rise of social media and social platforms such as twitter and facebook, the automatic interpretation of shorter amounts of text becomes increasingly more significant. the task of "reading between the lines" is so far often constricted to detecting offensive language or slurs. however, detecting funniness in written content also motivates research due to several reasons. discovering humorous content can help understand text on a deeper level and get its intention. it not only helps comprehending the author, but also the readers of the text and why they favor it. humor can be viewed as a natural language phenomenon and therefore linguistically interesting, but it also proves a lot of potential for modern psychology. using humor can be a way to gain intimacy, reduce stress, and can improve feelings and overall wellbeing @xcite . it is often linked to soft skills like character strength and can therefore also be an asset for human resources, resulting in a change of hiring processes. humor is a communication skill that requires high levels of world knowledge and discourse insight. it is often provoked by an unexpected change of events, and has a lot to do with language usage and ambiguity. when used appropriately, humor can enrich human communication and serve as a tool to establish relationships. humor has a sociological function and can therefore also be abused. humor recognition can help detect such abusive humor, but also it may improve human-computer interaction. the task of automatic humor recognition is quite challenging because for a human to create or detect humor, world knowledge plays an important role. this is especially difficult in tasks where evaluation takes place without context dialogue to consider. humor can also come in many variations, such as humor using anecdotes, self-deprecation, vulgarity and many more. with machine learning becoming more popular, it is a very interesting and contemporary unsolved task to assess how machines "learn" humor, especially in short text. the task of classifying to which extent a text is considered funny can be useful for fine-graining solutions for humor recognition problems. given a data set of english news headlines and a corresponding dataset of the same, micro-edited headlines, the task of the competition was to estimate the funniness of the edited versions. the corresponding paper can be found under @xcite . the headlines were single sentences taken from news. they were micro-edited, which means that one word was exchanged with the intention to make the sentence humorous. sub-task a aimed at classifying the edited news headline on a scale from 0 to 3, given the original headline as a reference. in the provided data sets, the edits were classified by five human judges, and individual and mean scores were given. the goal was to predict the mean funniness of the headline. for example, given the original headline "recent scandals highlight trump 's chaotic management style.", the system had to rank the funniness of the edited version: "recent scandals highlight trump 's chaotic fashion style.". given two edited versions of the same original headline, sub-task b asked to predict which one of the edited versions is funnier. in the example above, a second edited version would be provided for comparison. for both sentences, the grades and mean grades were provided in the data sets. we participated in sub-task a and sub-task b. the goal of this paper is to describe our work done in the semeval 2020 task 7. for task information, please consult the paper released by the authors of this task @xcite . ## background computational research in the field of humor detection has had quite some history and is rooted in language theories. to dismantle humor, societal norms and structures have to be taken into account. the variety of the task allows for different analyses depending on the type of humor and the expression form. however, the detection of humor in text has been widely restricted to bigger amounts of text and shorter content has only recently been put into focus. taylor and marlack focused on one specific type of joke, namely wordplays. they aimed to extract structural patterns of jokes @xcite . two years later, purandare and litman identified humorous speech by analyzing spoken comedic conversations, taking into account gender and speaker effect on humor-prosody @xcite . since humor plays such a big role in societal structures, different theories were developed to explain the structure that lies underneath a good joke. the incongruity theory states that for a statement to be humorous, two different interpretations have to be present. raskin 1984 used it as a basis for his semantic-script theory of verbal humor that states that a text has to be compatible with two different scripts, such that the two scripts overlap partly or fully and are opposite @xcite . the superiority theory @xcite suggests that humor stems from a feeling of superiority to others or previous events. minsky supports the relief theory, that states humor is a way to express taboos and societal unacceptable thoughts @xcite . newspapers have been an important resource for decades and centuries. headlines have been a big selling factor, and research has been done to unveil how headlines influence buyers @xcite . with the rise of social media, news have spread all over the internet. with anonymity and accessibility, there come some risks such as abuse of the web for propaganda, racism and other negative topics. on the other hand, the internet is spreading humor, and with social platforms such as twitter rising, short, humorous text paragraphs become popular. when detecting humor in news, the headline often gives the seriousness away. it is therefore an important part of news analysis to also focus on the headline. ## system description the core method used for our system was a l2 regularized linear model making use of the scikit-learn library provided in python @xcite . input sentences were modified to lower-case, numericonly text and saved for easy location of the edited word. both original and edited version of the input were encoded using tf-idf weighting found in the scikit-learn library. using two kinds of embeddings for the input sentence and adding the edit distance we aimed to facilitate humor recognition. an alternative approach was created by using an lstm model using the keras library provided in python (chollet and others, 2015). we constructed a twin lstm architecture to be able to process the two versions of the input sentence simultaneously and create a merged output. our code can be viewed following this link: https://github.com/leahannah/unituebingencl . 5000 original headlines with two edited versions were provided in the data set. they were split randomly into a training set (64 %), a development set (16 %) and a test set containing the remaining twenty percent of the data. to use the provided data in the most efficient way, we modified it to fit our needs. we removed non-alphanumeric characters with a regular expression @xcite . the input sentences were saved as lower-case strings with either the original or the edited word, which could be varied by changing a parameter. the numerical rankings by the judges provided in the data were saved as floats. we encoded sentences as n-grams by weighting them with tf-idf provided in the scikit-learn library. we chose to use elmo embeddings as provided by allennlp @xcite . in order to use elmo, a sentence array with clearly marked edited word was saved. for the lstm approach the sentences were padded to have uniform length, determined by the maximum sentence length in the training set. tuples containing the original and the edited word are saved for the embeddings. in sub-task b, we used a similar preprocessing structure as for sub-task a. we stored one id for each sentence pair and for each of the sentences all other data was saved in the same manner as in task 1. ## results figure 2 shows the results of our model tuning process for sub-task a. trials i to x show the improvement of the ridge regression model over time. the last data point shows the twin lstm structure in comparison. from the yielded scores we derived that the glove 100d embeddings were the best dimension of embeddings for our purpose. trial x lead to the best results, and was therefore used for the competitions test set. it outperformed the twin lstm structure and became our main approach. in the evaluation phase, we submitted predictions by both systems, the twin lstm structure and the l2 linear model. with the linear model, we achieved a rmse of 0.539 and with the lstm model a rmse of 0.575. therefore, the linear model predicted more accurately than the lstm network. this left us on ranking place 18 of 49 participants in the competition. for sub-task b, we adjusted the linear model from sub-task a to fit the new task structure. the parameters