Comparing Apples to Oranges: A Dataset & Analysis of LLM Humour Understanding from Traditional Puns to Topical Jokes (2025)

Abstract: Humour, as a complex language form, is derived from myriad aspects of life. Whilst existing work on computational humour has focussed almost exclusively on short pun-based jokes, we investigate whether the ability of Large Language Models (LLMs) to explain humour depends on the particular form. We compare models’ joke explanation abilities from simple puns to complex topical humour that requires esoteric knowledge of real-world entities and events. To this end, we curate a dataset of 600 jokes across 4 joke types and manually write high-quality explanations. These jokes include heterographic and homographic puns, contemporary internet humour, and topical jokes. Using this dataset, we compare the zero-shot abilities of a range of LLMs to accurately and comprehensively explain jokes of different types, identifying key research gaps in the task of humour explanation. We find that none of the tested models (including reasoning models) are capable of reliably generating adequate explanations of all joke types, further highlighting the narrow focus of most existing works on overly simple joke forms.

## introduction the perception and understanding of humour is essential to human interaction and entertainment. consequently, the ability to correctly process jokes is paramount to achieving a human-like understanding of ambiguity and social context in language. however, most existing work on computational humour has narrowly focussed on the simpler task of humour detection @xcite @xcite , which allows models to learn from confounding signals to detect certain joke types @xcite . on the other hand, humour explanation tasks a computational model with the more challenging objective of explicitly outlining how and why a presented text is humorous @xcite . 1 content warning: some of the presented jokes may be considered offensive or distasteful to different individuals. in addition to the general focus on joke detection, existing works on all humour processing tasks (i.e., detection, explanation, and generation) have mostly focussed on short-form puns such as "yesterday i accidentally swallowed some food colouring. the doctor says i'm ok, but i feel like i've dyed a little inside" @xcite , where humour arises from the interpretation of the punning word (i.e., "dyed") with a phonetically similar word with different meaning (i.e., "died") in the case of heterographic puns, or the polysemy of a word for homographic puns @xcite . whilst puns are commonplace, the humour is primarily self-contained, requiring only a common-sense understanding of the world (e.g., that food colouring "dyes" things) and an understanding of the semantics of dictionary words (e.g., dye/die). on the other hand, much of the humour encountered online, on tv panel shows, and in stand-up comedy routines, is based on contemporary topical knowledge of evolving pop-culture phenomena and news events, where a full appreciation of the humour relies on potentially esoteric knowledge, rather than commonsense @xcite , requiring highly complex reasoning and knowledge retrieval. we present examples of each joke type that we investigate in figure 1 . the homographic pun exploits the dual meaning of "croaked" as both a style of speech and a euphemism for dying; the heterographic pun relies on the phonetic similarity between "lint" and "leant"; the non-topical joke plays on the trope of an absentee father (not) returning like a boomerang; and the topical joke refers to the animal welfare organisation peta's high euthanasia rates and a reference to the movie forrest gump. each joke type plays on polysemy, phonetics, social constructs, and esoteric knowledge, respectively. to address this research gap, we assess the ability of large language models (llms) to explain this range of joke forms in a zero-shot setting. as a result, we present the first work on assessing whether stateof-the-art llms are equally able to explain different joke formats, from those that require an understanding of basic semantics and phonetics, to those that require esoteric knowledge of pop culture and news events, therefore assessing the models' abilities to perform complex reasoning and knowledge retrieval. we summarise our main contributions below. in presenting these contributions, we address the primary research questions of (rq1) whether different joke formats have an impact on the ability of llms to explain humour and by extension, (rq2) whether findings from existing work on computational humour are generalisable to the different jokes found in everyday life. humour detection similarly, humour detection has evolved from using lexical and syntactic featurebased methods @xcite to training language-model-based classifiers @xcite @xcite . @xcite additionally present work using humour to aid in the detection of political parody, whilst @xcite and @xcite investigated the impact of demographic variables on humour perception. furthermore, some works have approached humour detection from a multimodal standpoint, using elements such as audience laughter to aid in detection @xcite . humour explanation lastly, humour explanation provides an account for why a given text is funny. consequently, this task requires models to exhibit extensive reasoning capabilities. whilst explaining standalone puns usually only requires a model to identify the pun word and its alternative interpretations in the given context, more complex jokes require an understanding of common sense, social constructs, and esoteric knowledge to identify key incongruities. unlike the task of detection, explanation requires models to not only be able to identify humour, but also "understand" it to provide a textual explanation. @xcite presented a shared task concerning humour, in which subtask 3 pertains to humour interpretation by tasking models with assigning the correct wordnet sense keys to given instances of pun words. lima inácio et al. (2023) tackle the humour explanation task indirectly by investigating what specific elements humour classifiers learn. regarding the production of natural language explanations, @xcite perform the task of humour explanation on multimodal data from caption contests in the new yorker, generating explanations as to why a particular best-rated caption is funny. finally, @xcite investigate the ability of llms to explain a wide range of linguistic phenomena that require non-linear reasoning, such as tautologies and sarcasm. our work empirically investigates the performance gap for joke explanation on different types of humour, assessing the challenges presented by contemporary topical jokes in particular. ## methodology in the following section, we present our hypotheses relating to the research questions posed in §1, in addition to the models that we evaluate and our scoring criteria for the explanations. ## human evaluation average explanation lengths alongside accuracy and completeness scores can be seen in figure 3 . firstly, the lengths of explanations remain consistent within models regardless of joke type, suggesting that the length of a joke does not necessarily impact the required explanation length. notably, the geminibased models produce the shortest explanations across all joke types, whilst gpt-4o-based models consistently generate the longest explanations, opting for more verbose outputs. however, the shorter explanations produced by the gemini models do not seem to negatively impact their performance in terms of accuracy or completeness. despite being more concise, the gemini models appear to be efficient, focusing on the essential elements of an explanation without adding superfluous detail. in terms of overall performance, completeness scores are generally lower than accuracy scores across all models. this difference indicates that the models are more likely to miss key details in their explanations than to hallucinate incorrect information or misinterpret jokes entirely. this trend suggests that whilst the llms may recognise the core components of a joke, they often fail to fully elaborate on all relevant aspects that contribute to humour. when comparing model performance, gpt-4o consistently outperforms all others, demonstrating the highest accuracy and completeness scores across joke types. interestingly, gpt-4o mini outperforms gemini pro in some specific cases, such as in accuracy on homographic jokes, as well as in both accuracy and completeness for non-topical and topical jokes. additionally, open-source llama 70b, outperforms most other models except gpt-4o, whilst the r1-based reasoning models underperform compared to most other non-reasoning models. ## automatic evaluation in addition to the previously presented human evaluation, we provide the results of automatic metrics in ## case study example llm explanations of a randomly selected joke from the topical subset are shown in figure 5 . we omit the r1 reasoning models for space. the joke of interest refers to a 2018 internet phenomenon called the "tide pod challenge", where teenagers would dare each other to consume toxic laundry pods (tide being a specific brand). the central element to the joke's explanation is the suggestion that this new "teenage snack food market" increased sales, ergo generating enough revenue for the company to afford notoriously expensive advertising at the super bowl american football event. interestingly, all of the full-size models inferred the reference to this challenge in their explanations. however, the smaller counterparts consistently omitted this information or presented misinterpretations in their explanations. gpt-4o mini, for example, explains the joke by suggesting that "teenagers are known for being messy eaters", which caused increased sales. on the other hand, llama 8b and gemini flash focus on the idea that the humour stems from "absurdity", without making any explicit mention of the real-world phenomenon, thereby failing to inform the user of important context. further discussion of explanation characteristics can be found in appendix a.5. ## discussion firstly, regarding h1, that traditional puns will be easier for models to explain than highly-rated jokes from reddit, we found this to be true, with the non-topical and topical subsets significantly impacting explanation success rates. similarly, h2 concerned the hypothesis that homographic puns (which rely on polysemy) are easier to explain than heterographic puns, which was found to be the case in our tests. we observed that heterographic puns more frequently lead to hallucination (i.e., a lower accuracy score), owing to their reliance on phonetic characteristics, with llms having only implicit knowledge of how similar different words may "sound". h3 posited that the topical subset would present the most difficulty for llms. our testing revealed a statistically significant decrease in explanation success rates for topical humour compared to all other joke formats, with many explanations falling short of correctly identifying contemporary real-world references. finally, h4 stated that the larger models would outperform smaller variants. our results showed that this was the case, with r1 70b outperforming r1 8b, gpt-4o outperforming mini, gemini pro outperforming gemini flash, and llama 70b outperforming llama 8b. as predicted, the performance discrepancy was most noticeable within the topical subset. whilst these jokes were significantly shorter than the nontopical subset (see figure 2 ), their reliance on subtle references to esoteric entities and events required the models to identify information that was not explicitly stated if they were to correctly identify the humour. this creates an environment ripe for hallucinations, as the models attempt to generate an explanation for a joke where they often cannot identify the location of the punchline without additional information. we further hypothesise that the larger models struggled less with topical jokes, relative to smaller models, due to the additional parameters allowing the storage of more specific knowledge that does not appear in the training data as frequently as common sense concepts (e.g., that hot things cause burns, or that sexual relations with a blood relative are taboo). ## conclusion we presented the first empirical assessment of how well llms are able to explain the humour found within jokes of varying types, including simple puns and jokes that rely on contemporary topical knowledge. our findings demonstrate that the primary focus on puns in prior work is not representative of the abilities of llms to accurately and comprehensively explain a more broad array of joke types. furthermore, we also observe that no existing state-of-the-art models can consistently explain jokes of any type correctly.