UTFPR at S em E val-2020 Task 7: Using Co-occurrence Frequencies to Capture Unexpectedness (2020)

Abstract: We describe the UTFPR system for SemEval-2020’s Task 7: Assessing Humor in Edited News Headlines. Ours is a minimalist unsupervised system that uses word co-occurrence frequencies from large corpora to capture unexpectedness as a mean to capture funniness. Our system placed 22nd on the shared task’s Task 2. We found that our approach requires more text than we used to perform reliably, and that unexpectedness alone is not sufficient to gauge funniness for humorous content that targets a diverse target audience.

## introduction it is no secret that the natural language processing community has been developing a growing interest in increasingly challenging text classification and regression tasks in recent years. nowadays the community is coming together to find effective solutions to tasks that delve ever deeper into human subjectivity, such as identifying offensive language and racist remarks @xcite , categorizing complex and nuanced emotions @xcite , identifying sarcasm @xcite , quantifying suicidal tendencies @xcite , just to name a few. it is easy to see how a reliable approach to such tasks could help both businesses and institutions in more reliably and cost-effectively gauging public opinion, monitoring user behavior, driving the creation of new policies, products, services, and so on. similarly, gauging how humorous a certain piece of content is could also help the entertainment industry, for example, to both evaluate and maybe even automatically create quality content for its users. automatic humor recognition has been addressed in many insightful contributions already. @xcite , for instance, explore the use of many different kinds of word embeddings and neural models in judging whether or not a certain piece of text is humorous. similarly, @xcite combine an extensive set of features with tree-based machine learning algorithms to identify humor and irony in text. what these contributions have in common is the fact that, because identifying humor is something inherently subjective and complex, they combine numerous, sometimes hard to obtain resources to perform well. in this contribution, we attempt to take a different approach and try to address this task as minimalistically as possible to establish a foundation for more elaborate strategies. as it has been discussed in many academic contributions @xcite @xcite , one of the most important characteristics of quality humorous content is its unexpectedness, regardless of the means through which it is delivered (text, audio, or video). because unexpectedness is something that could potentially be captured through natural language processing techniques, we decided to create a system that tries to gauge funniness by using unexpectedness alone. because of our interest in potentially adapting our approach to under-resourced languages, we also set out to make ours an unsupervised system that uses as few resources as possible. in this paper, we describe the utfpr systems submitted to semeval-2020's task 7: assessing humor in edited news headlines. section 2 describes the task and associated datasets, section 3 showcases our approach and preliminary results obtained on the dev set, section 4 reveals our performance on the official shared task, and section 5 summarizes our contributions and conclusions. ## task summary the utfpr systems were submitted to the semeval-2020's task 7: assessing humor in edited news headlines @xcite . in this task, participants were challenged to conceive systems that quantify and/or compare the funniness of news headlines that had been modified for humorous effect. the task was divided into two sub-tasks: we chose to participate in task 2, exclusively. the training, dev, and test set for task 2 were composed of 9381, 2355, and 2960 instances, respectively. each instance was composed of a news headline with a highlighted word, two replacements for the highlighted word, and a score indicating which one of the replacements made the headline funnier. as discussed by @xcite , the annotation process of the dataset was quite extensive, involving both academics and amazon turkers. both the replacements and the funniness scores were produced by humans subject to monetary compensation. several measures were taken to prevent annotator's fatigue and ensure annotation quality.