Bridging the Creativity Understanding Gap: Small-Scale Human Alignment Enables Expert-Level Humor Ranking in LLM s (2025)

Abstract: Large Language Models (LLMs) have shown significant limitations in understanding creative content, as demonstrated by Hessel et al. (2023)’s influential work on the New Yorker Cartoon Caption Contest (NYCCC). Their study exposed a substantial gap between LLMs and humans in humor comprehension, establishing that understanding and evaluating creative content is key challenge in AI development. We revisit this challenge by decomposing humor understanding into three components and systematically improve each: enhancing visual understanding through improved annotation, utilizing LLM-generated humor reasoning and explanations, and implementing targeted alignment with human preference data. Our refined approach achieves 82.4% accuracy in caption ranking, significantly improving upon the previous 67% benchmark and matching the performance of world-renowned human experts in this domain. Notably, while attempts to mimic subgroup preferences through various persona prompts showed minimal impact, model finetuning with crowd preferences proved remarkably effective. These findings reveal that LLM limitations in creative judgment can be effectively addressed through focused alignment to specific subgroups and individuals. Lastly, we propose the position that achieving artificial general intelligence necessitates systematic collection of human preference data across creative domains. We advocate that just as human creativity is deeply influenced by individual and cultural preferences, training LLMs with diverse human preference data may be essential for developing true creative understanding.

## introduction warning: this paper contains potentially offensive content due to the nature of humor. the emergence of large language models has revolutionized many domains of artificial intelligence, yet their ability to understand and evaluate creative content remains notably limited. this limitation is particularly evident in humor ranking, as demonstrated by several studies on the new yorker cartoon caption contest (hessel et al., 2023; zhang * equal contribution, random draw. @xcite . these studies exposed a substantial gap between llms and human performance in ranking humorous captions. as shown by our test of the latest llms on the caption ranking task in figure 1 , various scaling efforts have not yielded significant improvement. in this paper, we successfully close the gap between llms and human experts. our work revisits this challenge by decomposing the ranking task into three components: visual understanding, cartoon-caption comprehension, and alignment with audience preferences (see figure 2 ). human experts demonstrate extraordinary ability in all three components. previous research by @xcite suggests that llms struggle with the second component-they are unable to comprehend the humor in cartoon-caption pairs. however, our study reveals that current llms, even with sophisticated prompting, struggle most with the third component-understanding the preferred jokes of specific audiences. while audience preferences are subjective in nature, it is a necessary and crucial ability that enables human experts to choose the best jokes tailored to their audiences. our initial attempts to bridge the gap between llms and human experts through various personabased prompting techniques showed minimal impact, suggesting a fundamental limitation in how llms understand human preferences. the breakthrough came through explicit finetuning on human preference data from the caption contest crowd. combined with improvements in the other components, we dramatically increased ranking performance from 67% to 84.7% accuracy, matching or exceeding human expert performance. this success extends to an even more challenging variant of the task where caption pairs are substantially closer in crowd preferences. lastly, our results highlight a broader challenge in the ability of llm to understand the subgroup and individual preferences for subjective and creative tasks. in section 5, we argue that the focus of the research community on problems with verifiable rewards, in domains such as mathematics and coding, may be insufficient to achieve a sophisticated understanding and abilities for creative works. humor and llms. research on computational humor has evolved significantly-from early rulebased, template-driven systems that generated puns via fixed linguistic rules @xcite to modern large language models that capture the nuances of human wit. finetuned transformer encoders like bert have yielded near-human performance in humor detection and rating @xcite , while very large generative llms such as gpt-3 and palm demonstrate emergent reasoning capabilities, explaining semantic incongruity in anti-jokes via chain-of-thought prompting @xcite . to enhance creative generation, paradigms like creative leap-of-thought (clot) prompting encourage unexpected conceptual associations @xcite , and multimodal approaches incorporating auditory cues have improved the modeling of phonetic ambiguities critical for pun understanding @xcite . curated humor datasets have exposed llm limitations and driven progress in generation tasks @xcite , while real-world evaluations by stand-up comedians reveal that, despite fluent outputs, llm-generated humor can appear generic compared to human creativity @xcite . notably, under controlled evaluations, ai-generated jokes can rival human-produced humor (new york post, 2024), and llms have been used as humor judges in the crowd score framework, producing funniness scores correlating with human judgments @xcite . challenges remain in producing contextually rich, culturally sensitive, and genuinely surprising humor, motivating ongoing research into more sophisticated modeling and training paradigms @xcite . humor detection still remains a hard problem, especially in the new yorker caption contest ranking task as described below. new yorker cartoon caption contest. recent advances in computational humor have been bolstered by the availability of large, well-curated datasets derived from the new yorker cartoon caption contest. previous works used this dataset to analyze the complex interplay between visual cues and linguistic humor, shedding light on the mechanisms that make captions amusing @xcite . the seminal work of bob mankoff, whose editorial work shaped the contest's creative process, provides essential context and insight into what constitutes successful humor in this setting @xcite . however, recent studies have demonstrated that state-of-the-art ai models struggle to fully capture the nuanced judgment required to select and explain winning captions @xcite . together, these works underscore the utility of the new yorker dataset as a powerful benchmark for advancing our understanding of humor in both human and machine-generated contexts. llm post-training/alignment post-training alignment of large language models (llms) can be achieved through a blend of supervised fine-tuning (sft) on instruction datasets and/or lightweight persona-or role-based prompting at inference time. early instruction-tuning work showed that finetuning on a modest set of natural-language instructions yields large zero-shot gains on unseen tasks @xcite . subsequent scaling along task diversity and model size continued this trend @xcite @xcite . complementing sft, persona prompting steers generation by prefacing the user query with a concise role description; @xcite formalize this "prompt programming" paradigm among others @xcite @xcite . moral-reasoning systems such as delphi demonstrate that persona-style conditioning can also be baked into the training data to yield consistent ethical judgments @xcite . in this paper, we find persona-based prompting to be insufficient for current llms to understand subgroup human preferences. as preference learning is a binary output task, we adopt supervised finetuning for state-of-art llms to obtain expert level accuracy. notably, by training a better preference model, our method can also serve as the critical first step in training llms to generate funnier captions through reinforcement learning @xcite . ## cartoon caption ranking task the new yorker cartoon caption contest is a longstanding weekly feature hosted by the new yorker magazine, in which a captionless cartoon is published and readers are invited to submit humorous captions. each week, over 6,000 captions are submitted. from contest #530 to contest #895, a bandit-based crowdsource rating system @xcite has been employed, allowing users to score captions as "funny", "somewhat funny", or "unfunny" (see figure 4 ). at the end of each contest, a complete crowdsourced ranking of captions is obtained based on their perceived humor. over the past eight years, the dataset of cartoons, captions and their rankings @xcite has proven invaluable for computational humor research. notably, prior work by @xcite and @xcite has leveraged the caption contest dataset to benchmark both humor understanding and generation, two essential domains of humor reasoning. to evaluate caption understanding, we employ the pairwise ranking task, a method widely used to study humor @xcite @xcite @xcite . we adopt the variant described by @xcite . in this task, given a cartoon description foot_0 , evaluators (models or humans) compare two captions at a time, each randomly sampled from distinct ranking tiers. specifically, one caption is drawn from a high-ranked group (ranks #1-10) and the other from a lower-ranked group (ranks #1000-1009) (see figure 3 ). this sampling strategy allows us to directly measure an evaluator's ability to discern differences in humor quality while controlling task difficulty through the selection of ranking tiers. additionally, we conduct a more challenging variant by asking models to distinguish between captions sampled from midranked tiers (ranks #30-39 versus ranks #300-309). the results highlighted in figure 1 underscore the persistent gap between state-of-the-art models and human expertise in humor evaluation, motivating our investigation into novel approaches to enhance model performance on this task. ## experiments we break the ranking challenge into three components -visual understanding, humor reasoning, and targeted alignment to human crowd preferences. this parallels the human experts' abilities: visually perceiving cartoons, understanding and creating humorous content, and deliberately tailoring jokes to specific audiences. generating cartoon description is a crucial first step in understanding the humor correctly. however, we found 23.5% of the gpt-4o generated cartoon descriptions in @xcite have erroneous descriptions. therefore, in section 4.1, we employ an ai-assisted annotation with humanin-the-loop assistance to fix cartoon descriptions. in section 4.2, we find that the o1-preview model can explain captions correctly and demonstrates extensive humor reasoning more than 85% of the time. we therefore generate such explanations, which serve as intermediate reasoning steps that inform the final pairwise comparison of captions. to better align our system with human crowd preferences, we implement two different strategies in section 4.3. first, we conducted extensive personabased system prompting, which does not exhibit significant improvements. our second, more sophisticated approach directly finetunes the model based on a set of ground truth rankings collected in the crowdsource ranking. this second approach significantly improves the ranking accuracy, and closes the performance gap between llms and human experts. throughout this section, we use a random train/test split (see figure 3 ) with 279 cartoons for training and 100 for testing. the training set is also used for sampling 5-shot in-context learning, with five meaningfully sampled caption pairs per cartoon. all reported results are evaluated on the test set. ## position: llms need more human preference data for creative domains our empirical findings on humor ranking highlight a significant challenge in developing more capable language models. while recent advances in llms have demonstrated remarkable capabilities in analytical reasoning and structured problem-solving, our results suggest that creative domains present a unique hurdle that requires special attention. we argue that this challenge stems from two fundamensecond, creative excellence requires understanding and internalizing group-specific preferences and cultural contexts. this is particularly evidenced by creative experts, who excel at tailoring their content towards specific audiences. in our experiments, persona-based prompting failed to improve caption ranking, while direct preference learning proved effective. this highlights a crucial gap in current llm capabilities in understanding the individualand subgroup-level preferences. while the new yorker caption contest provides us with extensive ranking data from a specific audience, collecting similarly comprehensive preference data for every creative domain, cultural group, and individual taste remains prohibitively difficult. for instance, how might we gather equivalent preference data for domains like musical composition, architectural design, or scientific research, where expert judgment is highly specialized? how do we gather the same type of preference data across different groups of people with distinctive tastes? these observations lead us to propose that improving llms on creative domains fundamentally requires solving the challenge of preference understanding. current reinforcement learning based techniques can improve performance once we have reliable judgment models, but the path forward requires models that can develop generalizable insights about preferences across different contexts, domains and audiences. ## conclusion our work demonstrates that by decomposing humor evaluation into visual comprehension, reasoning, and preference alignment components, llms can achieve expert-level performance in humor evaluation. while persona-based prompting showed limited success, direct finetuning on crowd preferences yielded dramatic improvements, suggesting that systematic collection of human preference data across creative domains may be essential for achieving true creative understanding in ai systems. looking ahead, our high-performing model can serve as a reliable verifier for humor generation, enabling inference-time scaling techniques @xcite to improve creative output. this creates a promising pathway for advancing both humor understanding and generation capabilities in ai systems.