On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generation
Authors: Wenbo Shang
Yuxi Sun
Jing Ma
Xin Huang
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal humor generation
multi-role framework
LLM
humor retrieval
Primary area: generative models

Abstract: Humor is a commonly used and intricate human language in daily life. Humor generation, especially in multi-modal scenarios, is a challenging task for large language models (LLMs), which is typically as funny caption generation for images, requiring visual understanding, humor reasoning, creative imagination, and so on. Existing LLM-based approaches rely on reasoning chains or self-improvement, which suffer from limited creativity and interpretability. To address these bottlenecks, we develop a novel LLM-based humor generation mechanism based on a fundamental humor theory, GTVH. To produce funny and script-opposite captions, we introduce a humor-theory-driven multi-role LLM collaboration framework augmented with humor retrieval (HOMER). The framework consists of three LLM-based roles: (1) conflicting-script extractor that grounds humor in key script oppositions, forming the basis of caption generation; (2) retrieval-augmented hierarchical imaginator that identifies key humor targets and expands the creative space of them through diverse associations structured as imagination trees; and (3) caption generator that produces funny and diverse captions conditioned on the obtained knowledge. Extensive experiments on two New Yorker Cartoon benchmarking datasets show that HOMER outperforms state-of-the-art baselines and powerful LLM reasoning strategies on multi-modal humor captioning.

Published as a conference paper at ICLR 2026
ON THE WINGS OF IMAGINATION: CONFLICTING SCRIPT-BASED MULTI-ROLE FRAMEWORK FOR HU- MOR CAPTION GENERATION
Wenbo Shang', Yuxi Sun', Jing Ma', Xin Huang!* 'Hong Kong Baptist University {cswbshang, csyxsun, majing, xino hkbu. eduu hk
ABSTRACT
Humor is a commonly used and intricate human language in daily life. Humor generation, especially in multi-modas scenarios, is challenging task for large lan- guage models (LLMs), which is typically as funny caption generation for images, requiring visual understanding, humor reasoning, creative imagination, and Existing LLM-based approaches rely on reasoning chains or self-improvement, which suffer from limited creativity and interpretability. To address these bottle- necks, we develop novel LLM-based humor generation mechanism based on fundamental humor theory, GTVH. To produce funny and script-opposite cap- tions, we introduce a jumoo-theoryy-drive work augmented with humor retrieval (HOMER). The framework consists three LLM-based roles: (1) conflicting-sctt scripptggs-ppp--e- extractor that grounds humor in key script oppositions, forming the basis caption generation; (2) strrivaalaugmenned hi- erarchical imaginator that identifies key humor targets and expands the creative space of them through diverse associations structured as imagination trees; and (3) caption generator that produces funny and diverse captions conditioned the obtained knowledge. Extensive experiments on two New Yorker Cartoon bench- marking datasets show that HOMER outperforms state-of-the-art baselines and powerful I.LM reasoning strategies on multi-modal humor captioning. Our codes are available at httpss//giiuub..co/hhaan-n-iibie
multi-role LLM collaboration frame-
Officaa---iiii-l-iinn infffiiial-mmllmmmnnttttiin
INTRODUCTION
Multi-modal humor generation has emerged to be important for exploring whether large language models (LLMs) can handle human-level linguistic and cognitive complexity (Wang al. 2025
Attardo 2024 Oring 2016 Horvitz et al., 2024; Cocchieri et al.] [025] Baluja 2025). Funny caption generation is typical task of multi-modal humor generation, which aims to generate funny caption for given image. This involves combining visual understanding of cartoons with humor understanding. creative imagination, and stylistic expression (Zhang et al. 2024 Wang et al, 2025),
which is technically challenging and even difficult for human beings.
However, current LLMs have been validated to have a weak inherent humor generation mecha-
nism (Mirowski et al. 2024 Horvitz et al. 2024 Pawar al. 2025 Gorenz & Schwarz, [024 Coc- chieri et al.] 2025 Jentzsch & Kersting 2023). To improve the humor generation ability of LLMs, few existing methods typically rely generic prompting (Zhang al. 2024 Chen et al. 2024 multi-hop reasoning for self-improvemmnt (Zhong et al. (024), or task-specific tuning (Wang et al. 2025) to better steer model outputs towards funnier captions. Unfortunately, these methods, solely guided by the LLM-inherent humor mechanism, capture surface humor language rather than deep humor logical reasoning and creative humor imagining, leading to limited creativity and originality. For example, consider the cartoon in Figure [[a), current LLMs (e.g.. GPT-4o) and the state-of- the-art CLoT (Zhong al., 2024) demonstrate good ability to generate semantic caption for describing the meeting, table, and caffeine, but lack enough humor for funny and deep imagination.
"Corresponding Author.

---

Published as a conference paper at ICLR 2026
(a) Funny Caption Generation
GPT-40
Visual Understanding
CLoT
Visual Understanding
Generated Caption:
Humor Understunclyy
Generated Caption:
Humnr Understanding
When the boss said bring ideas the table, nobody wanted to he caught under caffeenaadd
When the team meeting runs caffeine and ambition... mostly caffeine.
[Humor Imaginatiort
Humor Imagination
Stylistic Expression
Stylistic Expression
(b) Funny Caption Generation with GTVH Mechanism
Language
Step 1: Extract Situation
meeting room, cups
Step 3:
Step 4: Imagination
Step 5: Generate a funny caption. HR says we can expense a cow now.
Situation
Sefect Targer coffee cups
Step 2: Identify Confiict Normal coffee cups vs. oversized cups
coffee-milk
Target
-cOW
Target
Narrative strategy
Script opposition
Figure 1: A comparison of our HOMER with GPT-4o and CLoT models in funny caption generation.
To address the above limitations, we propose novel LLM-based humor generation framework, leveraging the well-established General Theory of Verbal Humor (GTVH) to generate script- opposite humor captions (Attardo & Raskin 1991; Ruch et al. 1993; Attardo 2016 Oring 2016]] Shang et al., 2022). The GTVH models humor creation through several interconnected knowledge resources, offering natural fit for image-based humor caption generation. Therefore, based method centers on script opposition, enabling it to capture diverse humor mechanisms and generalize across a broad range of various images. Continue the above example of a cartoon in Figure the situation is an office meeting, with the script opposition between ordinary coffee cups and oversized ones (see Steps 1-2), which establish the core logic foundation for humor genera- tion. The target of humor is the oversized cups (see Step 3), which disrupt the expected norm. The imagination of the target operates through an associative chain (coffee + milk -+cow,, amplifying the absurdity (see Step 4). The narrative strategy frames this exaggeration as a visual twist, while the language condenses into funny caption (see Step 5) as shown in Figure [b). The humor point of Figure bT) lies here. In terms of references (c.g., person, tables, and chairs), the size of coffee cups is super large. Thus, the key conflicting script, i.e., gigantic coffee cups vs. normal ones. The ground-truth humorous caption is "Could you please pass me a cow?", highlighting that large coffee cups need a large amount of milk, which is even needed to produce by the whole cow. This is ridiculously abnormal and funny. Our generated caption, "HR says we can expense a cow now", which has a similar humor effect as the ground-trutt -ruth and even playfully exaggerating workplace coffee consumption by involving the expense department of Human Resources (HR). As result, our caption has a better humorous effect than that of GTP-40 and CLoT in Figure Ta).
GTVH-
Technically, we propose a humor-theory-drrven multi-role LLM collaboration framework augmented with humor retrieval (HOMER) for funny image caption generation. HOMER iden- tifies a clear and interpretable humor mechanism reliant on the collaboration of three roles of L.LMs: Confticting-script Extractor extracts a detailed situation description from the image and analyzes contrast and incongruity elements based on the definition of script opposition, capturing the core humor logic and essential humor creativity. The result of the extractor is the basis of the generation process. Hierarchical Imaginator aims identify and enhance the critical humor target in the image. To expand its creative space, the imaginator conducts the humorous imagination of targets through our designed imagination trees, which are built by multi-view associations with LLM and humor-relevance retrieval from our collected joke database. Caption Generator combines the de- tailed situation description, confficting scripts, and diverse imaginative trees of targets to generate funny captions in a configuration of the five knowledge resources. Extensive experiments on two public New Yorker Cartoon benchmarks demonstrate the superiority of HOMER against of the-art competitors by achieving 77% improvement average.
2
HOMER
In this section, we present the problem of funny caption generation and our framework, HOMER.
2

---

Published as a conference paper at ICLR 2026
(a) Conflicting Script Extractor
Caption Generator Timo
Situation
Situation description:
Pt
Knowledge resources
aypiccal corporate meeting room..... oversazd steaming
Selection of imagination path
Script opposition
Conflicting scripts:
Searching over imagination tree
Generated Caption:
Normal coffee cups Oversized
HIR says expense
I
cups......
Selection of Seripts and Targets
Bash goveration
Image I
HOMER
Global LLM association
Humor-relevance retrieval imagination
Select
Sugar
Stirer
Caffee
Coffee
Image
entittiss
labll
Coffee
Frew
Jake DB
Retrieval
Pruning
O
Chair Wood
Local vlew of association
H
Ooo
Situation
Steaming mugs
Drew
Spoon
description
entities
Seleet
Formal
Caffee
Spoon
Backhone of imagination trees
with all retrieved entities
Imagination trees
Imagination trees
Tim
(b) Hierarchieal Imaginator
Key entities
LLM-ussccaatd entities
Humor retrieval entities
Figure 2: Framework of HOMER with three LLM-based roles: (a) Conflicingg script extractor, de- riving a detailed situation description and conflicting scripts as the basis of humor generation. (b) Hierarchical imaginator, identifying and enhancing the humor target with multi-view LLM asso- ciations and humor-relevance retrieval imagination. diverse captions conditioned the obtained knowledge.
(c) Caption generator, generating funny and
Fundamental Humor Theory. The GTVH humor theory is the theoretical foundation for our HOMER, modeling humor as the interaction of several knowledge resources: script opposition, situation, target, narrative strategy, and language (Attardo & Raskin 1991). Central to humor is script opposition, which captures conflicts between semantic frames (scripts). It underlies humor by establishing expectations and then violating them, thereby enabling exaggeration absurdity. In Figure [] (b), the conflict between a professional office setting with unexpected gigantic cups juxtaposes scripts of routine and hyperbole, yielding humorous reading. This script opposition leverages surprise, incongruity, and cognitive resolution, which are central to effective and engaging humor, More examples can be found in Section 3.4 and Appendix B.10
Problem Formulation. Given an input image I, the funny caption generation task aims to generate relevant and funny caption Cap(I) for image The ground-truth of this task is composed of human-written funny captions. The goal of tackling this task is to assess whether the generated Cap(T) derived by the multi-role models wins against human-written captions.
Overview of HOMER. The key idea of generating humorous caption by our HOMER framework to extract conflicting scripts from the given image and imagine script-opposition funny based on LLM association and joke database. As shown in Figure 2 HOMER contains three LLM-bbssed roles, which are the conflict script extractor Extract(-). the hicrarchical imaginator Imagine(-). ), and the caption generator Gen(-). Extract(I) + (C, D) yields script oppositions C and situation + Tin identifies key humor targets and derives target imagi- nation tree. Ω E NS LA sets the narrative strategy and selects linguistic style. With prompt (C,, D, Tim Q), the generator generates Cap(T) = Gen(@(C, D, Tim,?)).
description D. Mangine(TTC.
2.1 CONFLICTING SCRIPT EXTRACTOR
To ensure the extraction of precise and comprehensive conflicting scripts, our L.MM-base conflict- ing script extractor Extract(-) first analyzes the scene in the given image I and derives script- opposition-central situation description as the contextual background of the funny caption (c.g., meeting room, professional figures, steaming cups, natural serious expressions in Figure 2) in- cluding location, characters, facial expressions, and actions, while emphasizing inherent conflicting elements (e.g., oversized steaming cups) in J. Next, Extract(-) is designed systematically identify and analyze conflicting or incongruous elements in the image I. As GTVH posits, the definition of script opposition is the relation between conflicting contrasting semantic frames (scripts) in

---

Published as a conference paper at ICLR 2026
joke. Building this definition, we design prompt 4 (-)ript()) to guide Extract(-) to analyze and extract all relevant conflicting scripts that exist in the situation description D and the image I. For- mally, the set of confficting scripts C' is derived as D Extract(I), C Extract @ ssrript (I, D)). D and serve as the foundation of the whole generation process.
2.2 HIERARCHICAL IMAGINATOR
Based on the constructed humor foundation, our hierarchical imaginator Imagine(-) first identifies a set of key entities { described in C. and D as candidate humor targets of the funny caption. Then, to enrich knowledge about the identified targets and expand its creativity, Imagine(-) enhances each target 4 by conducting diverse imaginative associations. To capture diverse and high-quality imag- inative associations, Imagine(.) designed as a hierarchical architecture to combine multi-view LLM frce-associations with humor-relevance retrieval imagination to construct set of imagination trees Tim- Particularly, multi-view LLM associations serve as deep-pattern imagination, forming backbone chains of Trm. while humor-relevanee retrieval serves as broad broad-pattem imagination to expand Tim by discovering relevant humor associations in our collected joke database. When con- structing Tim. humor-relevance score H(e (8) .) is introduced to quantitatively measure the degree of humorous relevance between backbone entities ef and retrieved entities s, pruning Tim and removing retrieved entities with weak humor relevance.
Identify candidate targets from local and global views. Define the set of views V = (loc.glob); The local observation Ote is from the detailed situation description D, capturing fine-grained enti- ties or unexpected features within the image (e.g.. oversized cups, professional figures). The global observation Oglob leverages the image I to emphasize the obvious entities in the scene (e.g., cups, table). Coarse-grained and fine-grained entities can evoke different LLM associations (e.g., coffee cups, tea, figured people, etc). For each view E V, the imaginator extracts entities from O2, as candidate targets that are most relevant to conflicting scripts C. Formally,
O, C + Ent(Ov, C)2 Ent(Ov, C) = (m. Troot [Ent(O,, E V).
m is dependent on LLM analysis of O, and C. Identified candidate targets in T Tooo serve ancestor nodes of a forest of imagination trees, thereby guiding subsequent imaginative exploration.
Deep imagination forms the backbone chains of Tim- Deep-pattons imaginative chains from each to E Troot are modeled as first-order association process through an LLM-driven association function (-hain(-) with possible relations (e.g., ingredient, container, source, etc.) For an ordered
free-association chain TV = (e6% (1) (i)) the construction process is
c+1 Schain (c(")),
0, O...,
where ep t, and each successor e,+1 is imagined solely from its direct predecessor er The length is adaptively determined by LLMs with empirical average length E R3 4. The recursive procedure fchain() (-) enables progressively deeper levels of imaginative reasoning, ensuring each en- tity is conditionally dependent on its predecessor. After constructing two views of backbone chains (T E Ent(O,, C), E V}. the imaginator merges local and global-view chains by aligning identical entities and removing duplicates. For example, entities "coffee" and "coffee cups" can be merged into "coffee cups". As result, each candidate target f; is associated with a unique and multi-view imagination tree Tú, forming the backbone chains of imaginative trees T.
Broad imagination expands the imaginative chains of Tim To expand the backbone of the imag- ination tree with relevant humor associations in daily life, we design a humor-relevance retrieval from our collected joke database J, which is reorganized from 12 open-source joke datasets. First, the imaginator conducts top-K relevant joke retrieval. For each LLs-associated entity cl E T,, the imaginator constructs a query embedding Zg femb (D,C. ef") For each joke j e J with embed- ding Zj. we calculate the cosine similarity sim 2,). The jokes are retrieved by ranking all j according to sim( z.y, 2j), i.e., Jropk [j E J J[simm [zj, 2ng) 2 sim(zz,, 2,), Vjj E J \ &Jeppk) ), ensuring selected jokes are relevant to both the query entity (()) and the foundation of humor D and C. femb() (-) can be the statistical embedding method for efficiency, or other LM-based methods. Then, for each retrieved joke j E Jropk, , the imaginator tokenizes and lemmatizes j into a set of
tokens& E, as leaf nodes for the query node (ff) Finally, the imaginator conducts HOMER-pruning
4

---

Published as a conference paper at ICLR 2026
with designed humor-relevance score H(e deriving high-quality imagination trees Tim-
(3) ,) filter out leaf nodes with weak humor relevance,
HOMER-pruning To filter out leaf nodes with weak humor relevance Er design humor- relevance score H(e) (f) ,e), where & E Eg. H(e))) ,e) builds on three key scores, which are relevance- (8) , humor-frequency Hfreq and POS-diversity scores Haiv (E) follows. H(e e) (e,))e (ee)) ,0) (freq()) + (aiv(E).
opposition (e
We then retain tokens for which rank(H(e (1) E)) < 8, thereby pruning the imagination tree Ti where 8 is the desired rank threshold. rank( ee)) denote the rank of according to its humor- relevance score for E Ej.
Term-1: Delevance-Opposiiee
score. Inspired by GTVH, we design the relevance-opposition score (e,), :) between entities e and E as a joint function of semantic similarity and concep- tual opposition, thereby capturing semantic relevance and surprise incongruity essential to humor.
To accurately measure (e (e),)) ,e). we utilize WordNet (Miller 1995), which affords structured semantic relations for reliable sense discrimination and similarity assessment. Specifically, target semantic similarity (TSS) is quantified using the Wu-Palmer similarity Sim (ts Let Se, and S.
represent the sets of synsets associated with e and For Se, Se # 0,
TSS($e, 8e) max Simwup (Se,: sz). ESe, E:
(2)
Otherwise, Simmup ( O. Conceptual opposition (CO) measured as the Jaccard dis- similarity between concept sets of of and E. For given synset s, its concept set R(s) is de- neighboring concepts, including its synonyms, hypernyms, hyponyms,
fined as the union meronyms, and holonyms, denoted by Hyper(s), Hypo(s), Mero(s), and Holo(s), respectively. Thus, R(s) = s HyHerr(s)) Hypo(s) U Mero(s) U Ho)o(s). The Jaccard overlap between senses Se, Se, and s; S
[R(se, R(s))l
Jaccolse,,88 [((a,) U R(se)l
if (((,,) R(se)] > 0.
Otherwise, Jacco(Ser S 8²) = 0. Therefore, we formulate the conceptual opposition as (0(se,,8)) =
max Ney ESS,, E8,
Jacco(se, 8E), CO(se,, 8x) E [0, 1].
(3)
CO(sE, 82) treats opposition as low overlap between the lexical neighborhoods induced by core semantic relations. Thus, model two opposing tendencies: semantic similarity TSS(ss, se), and conceptual opposition CO(se, :8)) (e .) should designed based on three criteria: (i) dominated by semantic similarity; (ii) incorporating similarity- gated, bounded bonus for concep- tual opposition; and (iii) established through a principled balance between two opposing tendencies. To achieve it, we introduce shaping function [0, 1] - [0, that is increasing and bounded, with (0) = 0 and a moderate peak as similarity grows. Thus, based on Eq and Eq. 3 the calculation of the relevance-opposition score can be formulated
H (e42 ,) TSS( Se,, 8e) + (TTSS 8e)) se), with f(x) x exp(-x)). (4)
Detailed proof of convergence and monotonicity can found in Appendix
Term-2: Humor-Frequeee score. (f)e quantifies the importance of based on its empiri- cal occurrence frequency, defined as the geometric mean of token frequency and normalized joke frequeney over Jrapk:
E Ejehoax count(E, E,) EjeJspx I/2 E Eg] |eappl
Htreq (8) =
(5)
Ejeroox Eg
count(E, E,) is the multiplicity of in The indicator function I(-) equals I if the argument is true and otherwise. Higher (e) indicates that appears frequently and across many jokes in Jropks evidencing statistically meaningful association with (6) and C in context D.
5

---

Published as a conference paper at ICLR 2026
Term-3: POS-Diversity score. (e) assesses the lexical richness of e based parts of speech (POS), Np denotes the POS inventory in WordNet. M(e) is the occurrence number of e tagged with p P. Higher (E) indicates that token has more lexical ambiguity, thereby creating more opportunities for puns and wordplay in funny captions. The normalized POS-diversity score is
Epep (N((c) >
(a)
PP
Therefore, based on Eq. Eq. Eq. and Eq.
H(ef" ,e) is calculated to retain leaf nodes for which rank(H (1) E)) S 6, thereby deriving diverse and high-quality imagination trees Tim-
(H((e))
2.3 CAPTION GENERATOR
Our caption generator Gen(-) aims to generate funny and diverse captions Cap(T) for the given image / based on the obtained knowledge. Specifically, Gen(-) begins by randomly selecting key conflicting scripts E C and relevant humor target t2 E Toot from all candidate targets. Note that not all candidate targets are used in the final caption. For each candidate target t,, the associated imagination tree T; E Tim is traversed to enumerate all possible paths from the ancestor node to the leaf nodes through depth-first search (DFS), denoted by P.. A single imagination path P; E P; is then sampled, representing a creative chain of humorous associations. The generation prompt is constructed by integrating the situation description D, the selected conflicting scripts C, the creative imagination path P. of selected humor target fú, and the generation options Ω E NS x LA, where Ω specifies the narrative strategy and linguistic style. Formally, the prompt can be represented as (C, D, P,, S2), which is then fed into the LLM-based caption generator producing the final funny caption, i.e., Cap(T)= = Gen(#(C. D, S2).
EXPERIMENTS
Datasets. We evaluate the performance on two real-world New Yorker datasets, Human in Al (Zhang et al. 2024) and Electronic sheep (Hessel et al. 2023), including cartoon images, standard cartoon descriptions, humor- ous captions, and ranking of captions based their humorous degree, as detailedly shown in T Following the settings in Human in AI, we evaluate all models by comparing the generated captions against three groups of human-written captions at different ranking levels, which include #top10, #200-#209, #1000- #1009. As Electronic sheep has three ranking pairs of captions per cartoon, we split it into two groups. Higher ranking captions in all pairs form the High-Humor group. Otherwise, the Low-Humor group. In particular, we collect and reorganize 11 one-liner joke datasets through multi-stage data processing as our humor retrieval dataset. We provide more details of our humor retrieval dataset in Appendix B.I
Table I: Statistics of datasets.
Datasets
Human in Al Electronic sheep
#Cartoons
365
679
Avg #captions
6,044
6
#Groups Ranking
2
Global GPT-4o
Pairwise Human
Description
Competitors and Metrics. We evaluate HOMER against four state-of-the-art models for humor generation: Humorous AI (Zhang et al. 2024). LoL (Wang et al. 2025), Phunny (Chen et al. 2024), and CLoT (Zhong al.) 2024). Additionally,
also compare with three widely-adapted and ad- vanced reasoning strategies: chain of thought (CoT) (Wei et al.) 2022). few-shot reasoning (Alayrac) al. 2022), and self-consistency (Wang et al.] (023). To assess the reliable measure for creative caption generation (Zhang et al.) 2024) Hessel et al. 2023), we use the unbiased Pass K metric to
measure the probability that HOMER -e-----geeeeteed humorous captions win the human- written caption
over multiple i trials (Liu et al.
2024
Mohammadi et al. 2025 Zhang et al. 2025 Yu et al. 2024).
1 N
N
m
pass @ k
Σ
where N denotes the total number of images, n, is the number of captions generated for the i-th image, and c; is the number of captions evaluated funnier than the human caption. For compre- hensive evaluation, we report the results at K = 31,3,5 5}. Each pass@ K calculation is the average
value of five trials. More details of competitors and metrics in Appendix B.2 and B.3

---

Published as a conference paper at ICLR 2026
Table 2: Performance (%) of funny caption generation (mean pass @k over 5 runs) on two datasets with four base LLMs (GPT-4o, Claude-4, Qwen-VL. and LLaVA-1.55. Higher scores are better.
Electric sheep
#Top10
4200-209
#1000 1039
High Hummor
Low Humor
Method
61
613
68.5
61
683
65.
68
633
68.5
@
6
@R3
68-5
GPT-40
CoT
45.79
70.59
79.61
57.28
82.85
85.56
61.58
86.90
85.63
57.52
76.13
81.19
63.64
77.76
84.01
Few-shot
58.07
78.91
82.44
65.12
81.14
84.27
65.59
88.39
90.83
50.67
69.33
80.67
55.67
72.66
83.66
Self-consistency
62.03
77.96
82.93
68.09
84.45
87.72
69.42
85.51
88.93
48.57
64.95
74.23
62.02
70.47
78.78
HumorousAl
62.11
81.24
85.15
69.38
85.32
87.86
73.46
67.39
80.57
69.41
85.33
LoL.
$6.30
75.21
81.01
64.50
80.85
85.21
67.29
83,83
88.73
61.26
79.22
84.55
64.73
80.60
84.73
Phunny
16.09
27.47
32,94
20.38
41.25
23.80
38,74
45,99
26.22
36.
45.92
29.31
48.05
CLoT
61.17
75,29
80,00
59.52
72.47
76.47
68, 70,70
78.00
81.17
63.33
71.83
77.33
67.49
81.16
87.83
Ours
66.41
83.70
89.18
73.40
88.38
92.57
76.32
90.50
94.19
75.53
89.21
92.10
79.45
91.48
93.81
(mprov.(8)
+6.92
+3.03
+4.77
+5.79
+3.59
+5.36
+3.89
+2.39
+3.70
+12.1
+0..7
+8.93
+14.4
+12.7
Claude-4
CaT
37.31
47.62
51.01
40.03
48.70
51.01
42.27
41.87
51.67
57.52
69.00
72.51
63.50
74.39
78.01
Fow-shot
61.67
78.70
82.67
70.00
85.13
88.67
69.19
83,
87.00
32.67
54,00
66.67
48.33
63.67
68.33
Self-comsistencyy
60.73
76.90
85.33
68.73
57.72
74.39
79.13
87.66
HumorousAl
62.86
78.86
82.67
70.39
83.46
86.33
68.66
82.06
85.98
59.40
77.66
83.33
65.69
83.13
89.83
58.06
75.19
80.00
68.40
84,40
88.67
67.06
83.30
87.89
60.60
79.33
81.00
66.23
83.04
87.66
Phunny
14.24
32.87
46,40
16:99
34,51
45.75
18.03
40.06
53,59
20.16
38.58
48.33
30.33
50,20
60.41
CL.T
43.15
53.25
56.00
50.91
59.05
62.00
53.65
61.44
63.00
41.67
62.00
68.33
51.83
72.83
79.
64.67
71.27
71.06
85.47
62.27
81.37
71.75
95.19
Improv:
+2.88
+4.83
+5.24
+ 11225
1.44
+1.87
+2,70
+2.56
+ 1.26
+2.75
+2.57
+4.33
+6.44
+8.09
+5.96
Qwen-V VL (7B)
Col'
16.76
27.33
33.01
25.46
38.69
44.44
22.85
35.11
40.63
19.06
30.90
36.66
26.83
37.39
41.83
Few-shet
19.60
29.59
19.38
30.96
28.69
45.19
Self-consistency
15.86
24.99
28.67
26.13
37.23
41.67
25.06
36.53
41.33
15.26
21.61
24.74
19.93
28.28
33.16
HumorousAl
18.26
27.53
30.67
27.67
38.90
43,33
25.40
37.00
41.33
11.80
16.83
18.99
16.56
25.56
29.16
LoL.
15.12
23.84
28.17
19.86
32.61
38.14
21.37
34.29
40.54
17.58
28.53
31.50
24.94
38.24
44.13
Phunny
5.92
9.25
11.11
6.18
10.37
14.81
2.96
8.89
14.81
4.44
13.33
22.22
10.55
20.55
30.56
CLoT
21.67
36.67
43.33
27.33
48.33
23.00
29.33
8.00
21.10
26.67
18.66
33.33
41.67
Ours
24.06
41.75
49.59
33.65
53.57
62.19
32.92
50,52
58,53
22.74
36.18
41.58
29.62
42.37
47.42
Improv.
+11.0
+13.8
+14.4
+23.4
+22.2
+28.7
+24.4
+30.2
+25.4
+17.3
+16.8
+13.4
+3.24
+4.77
+4.93
LLaVA-1.5 (7B)
1.11
2.89
3.33
2.22
1.08
3.66
5.56
7.22
11.66
13.89
Row-shot
8.44
10.89
12.22
20.44
23.33
24.44
16.44
18.67
18.89
14.00
16.00
16.67
20.67
25.26
27.50
Self-consistency
3.78
6.22
6.67
7.11
8.44
8.89
8.00
9.44
10.00
1.37
3.99
6.67
7.99
10.67
13.33
HumorousAl
4.00
10.00
13.33
9.33
14.67
20.00
18.67
20.00
21.73
11.11
17.78
22.22
22.78
28.33
30.55
Lol.
1.33
4.09
6.67
14.67
17.33
22.25
12.00
17.33
20.00
1.90
5.71
9.52
15.24
24.76
28.57
Phunny
3.89
10.00
13.33
15.67
24.67
13.33
22.13
4.17
13.33
22.36
10.55
20.57
27.01
CL.T
6.40
8.00
8.00
1.8
2.40
4.00
16.00
19.60
20.00
13.33
16.11
16.67
24.44
27.78
28.98
Ours
10.22
15.22
19.56
22,89
27.67
31.11
20.67
24.44
27.67
19.33
23.17
25.00
30.16
34.33
35.83
Improw
+21.0
+39.7
+46.7
+11.9
18.6
+26.1
+0.77
+22.2
+22.5
+38.0
+30.3
+.1.8
+23.4
+21.1
+17.2
Implementation Details. In the hierarchical imaginator, we impose the top-k relevant jokes = 5
for balancing efficiency and effectiveness, and the threshold of humor-relevant entities 5 = 5. For caption generation, all base LLMs use the temperature of to ensure the creative generation of funny captions, leaving all other parameters at their default values. For the humor evaluator, GPT-5 and other humor evaluators use the temperature of to guarantee the stability and reproducibility of evaluation results. Additionally, we fine-tune the Humor-tuned LLaMa3 on ranked caption pairs split 8:1:1 into training, validation, and test sets. Some hyperparameters are analyzed in Section 3.3 and Appendix B.4 More ablation studies and detailed prompt design are provided Appendix and G respectively. All experiments are conducted on 2 NVIDIA RTX 4090 (16 GB) GPUs.
3.1 RELIABILITY OF HUMOR EVALUATOR
To measure the reliability of different evalua- tors, as reported in Table 3 we compare their ranking accuracy in judging human-written caption pairs, which are randomly selected across 200 different contests. Our assess- ment involves five evaluators: two open-source LLMs with sum token logits and rewards (i.e., ILLMa 3-8B and Humor-tuned LLaMa 3-8B),
Table Ranking accuracy (%) of evaluators.
Evaluator
Humor in AI Electronic sheep
LLaMa 3
53.5
$2.0
Humor LLaMa3
60.0
58.0
Qwen-Turbo
55.5
54.0
GPT-4.1
68.5
67.0
GPT-5
73.5
70.0
7

---

Published as a conference paper at ICLR 2026
Group
Group
Group
100
80
60
40
pass
pass
pass-2
pass-
pass- F223
pass
pass
pass
pass
022
acopco
303 Wio relcreeeencce
w/o, Fro.occccccy
i/c vo_ddversyy
Figure 3: Ablation study of humor-relevance score.
Table 4: Ablation studies of HOMER modules with GPT-4o. Inclusion (V) or exclusion (x).
Module
Image-Only
IeD
I+C
1+
14CTTm
I+D+Tim
1+D+C
1+D+C+Tim
Image (7)
Situation Description (D)
x
Conflict Scripts (C)
Imagination Tree (Tim)
x
x
Generator
V
Homor in Al
Electric shoop
#Top10
#200-209
#1000-1009
Better Group
Worse Group
Method
@1
@3
@5
@1
@
@:
@
@3
@5
@
@3
@
@3
@
Imuge-Only
20.20
38.30
51.00
21.00
36,00
42.99
27.99
43.30
53,00
17.67
31.35
36.67
25.67
43.50
51.67
I+D
50.60
69.49
78.00
47.20
66.50
74.00
53.60
68
73.00
41.66
65.16
74.99
51.50
74.00
83.33
IeC
41.80
$9.70
67.00
37.20
50.70
57.00
44.99
60.00
66.00
15:25
27.83
33.33
19.67
32.83
40.83
I+
20.00
35,90
43.00
20.60
34,00
40.00
28.60
44.09
51.00
15.00
26.67
33.33
29.33
44.91
52.49
I+C+Tim
35.40
53.89
60.00
33.00
$2.60
61.00
41.00
58.70
65.99
24.00
43.00
55.00
39.17
59.50
67.50
I+D+Tim
34.40
56.50
67.00
36.20
51.20
56.00
42.60
59.90
67.00
36.67
57.83
68.33
51.00
72.50
80.00
I+D+C
57.40
75.50
80.00
56.80
74.70
79.99
63.00
78. 10
83.00
60.33
74.83
78.33
67
82.00
86.19
I+D+C+Tm
66.41
83,70
89.18
73.40
88.38
92.57
76.32
90.50
94.19
75.53
89.21
92.10
79.45
91.48
93.81
as well as three advanced closed-source LLMs (i.e., Qwen- Turbo, GPT-4.1, and GPT-5). The results indicate that GPT-5 demonstrates superior performance as a humor evaluator. Although Humor- tuned LLaMa 3 shows noticeable improvements, its effectiveness still remains limited. Therefore, we adopt GPT-5 as our primary humor evaluation model. The humor fine-tuning is in Appendix B.7
3.2 FUNNY CAPTION GENERATION
Table 2 demonstrates that our HOMER significantly outperforms seven state-of-the-art baselines on two real-world New York Cartoon Contest datasets, achieving average improvements of 8.62% pass@1 1, 6.48% on pass @3, and 5.91% @ with GPT-4o. Unlike leading methods of multi-modal humor generation. such as HumorousAl and CLoT, which focus on reasoning chains, HOMER's core distinction is the explicit modeling of the funny caption generation step by step through a humor-theory-driven multi-role framework. These improvements underscore three key insights: (1) Incorporating humor theory into the generation process provides explicit guidance to LLMs, resulting in captions that are not only more humorous but also more interpretable. In contrast to methods that rely on heuristic reasoning strategies, humor theory enables systematic, step-by-step generation process, offering greater generation control and interpretability, enhancing the humor quality of the generated captions. (2) Imagination plays a critical role in humor genera- tion. Since humor creation is inherently creative, solely relying the LLMs intrinsic imagination may lead to repetitive and limited outputs. By introducing multiple perspectives and diverse imagi- nation patterns, LLMs can generate funnier and more original captions.
(3) A multi-role framework facilitates the complex and challenging task of multimodal humor generation by breaking down into several more precise and refined steps, thereby enhancing the quality of humorous captions.
3.3 ABLATION STUDIES
Ablation on Three Key Modules. We first exhaust all ablation choices in three humor-theory- driven modules to generate humorous captions in Table Results in Thow that (1) removing any single module consistently degrades performance, verifying the necessity of D,C, Tim in the multi-modal caption generation. The largest performance drop, seen in I+D+Tm. highlights the
8

---

Published as a conference paper at ICLR 2026
2.0
Group 2-9
Group 200-209
Group 1000-1009
Group 1-9
Group 2002099
Group 1000-1009
2.5
pamp
Figure 4: k hyperparameter.
Figure 5: hyperparameter.
significance of conflict script C. as a basis caption generation. (2) Both conflict scripts and sit- uation descriptions are critical for deriving imagination trees. Compared I+Tim. both I+C+Tim and I+D+7im contribute to significant improvements. (3) Inadequate guidance of imagination leads to performance drops, which may lead irrelevant and nonsensical caption generation. Compared with the image-only variant, I+7m leads to performance drops.
Ablation on humor-relevance score H(-). We ablate the calculation of relevance-opposition fre- quency, and diversity scores in H.) in Figure 3 The w/o relevance variant (removing relevance score calculation) consistently results in a significant performance drop, validating the necessity and effectiveness of modeling semantic relevance and conceptual opposition. The w/o frequency and w/o diversity variants also show great drop, indicating that they are useful for exploring imagination.
Robustness of hyperparameters. We conduct an ablation study on the only two hyperparameters in our method: the number of retrieved jokes k and the number of humor-relevant entities 6, both of which are varied across [3. 5, 7, 9]. The results, shown in Figures and indicate that our method remains stable across the entire range of values tested, showing strong robustness. We provide the detailed analysis in Appendix B.4
Generalization across visual domains. To assess the gen- eralization ability of our HOMER, we evaluate HOMER
Table 5: Results on Meme data (%).
on a public ImgFlip meme (Hwang & Shwartz, 2023).
Method
pass
pass @
which contains diverse range of images, including realis-
CoT
74.33
86.
Few-Shot
66.67
81.67
tic, comic, cartoon, and synthetic images. We evaluate the generated meme captions against the ground truth meme captions. T shows that HOMER consistently outper- forms strong competitors by approximately 5.4% on aver- age, validating the effectiveness and powerful generaliza- tion ability of our HOMER across different visual domains. More results are in Appendix Table 9
Self-consistency
70.00
90.83
HumorousAl
75.00
80.00
LoL
71.67
81.12
Phunny
21.67
31.67
CL.T
76.67
88.33
HOMER
83.33
96.67
3.4
CASE STUDY
We show two cases in Figure 6 showing the explicit intermedi- ate results of caption generation. For Case 1, the extractor records the core opposition, normal cof- fee cups vs. gigantic cups. The hierarchical imaginator then ex- pands traceable imagination path from the chosen target cof- fee cups mcik--crremm-xoow, supported by retrieval. Fi- nally, the GTVH-guided gener- ator generates HR says we can expense a cow now. This cap- tion suggests that employees, drinking large quantities of cof- fee. humorously claim HR al- lows them to expense a whole cow for milk. The exaggeration aligns with both the image and the office culture. For Case 2, the script opposition is communication from typical col-
Case I.
HOMER
Caliisia strits
Whet the meeting nass caffiine
Normal coffee cups vs. Giganic cups
mostl coffeie.
Koger
Coffre cups
wark Troashatle saboddy expected tn be los parts with plant maag!!
When the
Ckoore snggeatte pash-
[Caffee caps, nilk,, cream, cow)
HamorousAl
Gesorand Cayaan:
When the
Dd says
robodly resccad too
Ciree 2
HOMER
ClaT
Closen Corlictin zergu.
Wamn
your Bestll..
Typical colleague Mini-dummy
Taget
Smaller man
Lei zazui nest quater tixy say peb inen?
Closen sxepestio pash
[Sunller man. puppet, working!
Humerumnl
Geverind Captios. Is Bring Your Inner Critic Work alreads?
Lioze, bad rews
dellveriin
HOMER
CleT
Dangerons high-altiende Casual pizza delivery
When you catre high delivery we tike itaally.
Chuzs lirstt
Pizza delivery
Beiwary snywher:? Don'
Chouzn Wugh
HamorouxAl
[Pizza, crust] [Great height, opper]
Was oan ligh delivry, oot eata high ralk
Geoortte Caplin.
Now that's what call 'upper delivery!
Figure 6: Casc Study.

---

Published as a conference paper at ICLR 2026
league through a mini-dummy the fundamental joke logic. The hierarchical imaginator de- rives smaller man -uup-ppppet-wworkkng imagination chain of humor target Smaller man. Finally, the GTVH-guided generator generates the funny caption Is it Bring Your Inner Critic to Work Day already? that fuses idiom subversion with personification to resolve the visual incongruity in a fresh way. For Case 3, which is normal image but with incongruity, the core script opposition is the dangerous high-altitude work vs. the casual pizza delivery, forming the basis of the joke. The hierarchical imaginator derives two imagination chains: great huperr and pizza-+crust Then, the generator derives the funny caption Now that's what we call 'upper crust' delivery!. This caption cleverly plays on the idiom "upper crust," connecting the visual context of pizza being delivered at a great height to the phrase's 's meaning of high quality or elite status.
3.5 HUMAN EVALUATION
We conduct a human evaluation in which 20 raters scored seven captions corresponding to seven methods on five-point funniness rating rule (1: not funny, 2: slightly funny, 3: mod- erately funny, 4: funny, 5: very funny), follow- ing standard caption evaluation practice (Kasai jet al.] 2022, Levinboim et al.) (202T). There are total of 5600 rating scores for evaluation. Ta- ble reporrss the mean ratings (# std) for seven representative methods. Our method achieves
Table 6: Mean human ratings with std (+)).
Method
Humor in AI
Electronic sheep
CoT
2.47.1 0.67
2.20 0.78
Few-shot
2.96.1 0.70
2.56 1.00
Self-consistency
2,66 0.59
2.25 0,56
CLoT
2,95 0.77
2.53# 0.73
HumorousAl
1011
2.24 0.81
LoL
3.16 0.84
2.40 0.82
Ours
3.54 1 0.59
3.311 0.85
the highest mean score (> 3.0 on the five-point funniness scale), indicating that human raters gener- ally judged its captions be over moderately funny. Inter-rater agreement is relatively substantial, with Cohen's kappa k = 0.49, following agreement measurements in human studies (Hallgren 2012]. Humor is inherently subjective, individuals differ in their interpretations of humor well as in their understanding of images. Therefore, к = 0.49 reflects an acceptable level of agree- ment among annotators, given the expected subjectivity in humor evaluation tasks. Detailed human evaluation can be found in Appendix B.8
4 RELATED WORK
Humor creativity in LLMs. With the emergence of LLMs, exploring the linguistic capability of
LLMs on human-challenging linguistic phenomena (Shang & Huang 2025 Sun al. 2025 Zhou
ct al. 2025a), such as multi-modal humor generation, has attracted rapidly growing interest from researchers (Horvitz et al. 2024 Cocchieri et al.) 2025 Baluja 2025 Wang al. 2025 Attardo 2024). Benchmark evaluations show that prominent models (c.g.. GPT-4 variants) can detect sur-
face humor cues yet struggle with originality and comedic quality (Zhang et al. 2024 Wu et al, 2025). To improve the humor generation ability of LLMs, prior methods typically rely on generic prompting (Zhang et al. 2024; Chen et al., (2024). multi-hop reasoning for relf-improvement (Zhong et al. 2024), or task-specific tuning (Wang et al.) 2025) to better steer model outputs towards fun- nier captions. However, they still sufter limitations of interpretability and creativity. Therefore, we propose humor generation mechanism driven by humor theory and augmented by hierarchical creative imagination process. Classical related works can be found in Appendix F
5 CONCLUSIONS
In this paper, we propose a HOMER humor generation framework to address the limitations of in- terpretability and creativity in prior approaches. Anchored in the famous theory GTVH, HOMER employs three coordinated roles: a conffict-script extractor that identifies detailed situation descrip- tions and script oppositions, hierarchical imaginator that stimulates imaginative associations with retrieval, and a caption generator that generates funny captions using the obtained knowledge re- sources. This modular design shows explicit control over humor logic and materials, enabling tar- geted editing and more original creativity. Extensive experiments demonstrate consistent improve- ments over seven state-of-the-ar baselines for multimodal humor captioning.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work is supported by grants from the Research Grants Council of the Hong Kong Special Administrative Region, China (No. 12201923).
REFERENCES
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language model for few-shot learning. Advances in neural information processing systems, 35:23716- 23736, 2022.
Miriam Amin and Manuel Burghardt. A survey approaches to computational humor generation. In Proceedings of the 4th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pp. 29-41, 2020.
Issa Annamoradnejad and Gohar Zoghi. Colbert: Using bert sentence embedding for humor detec- tion. arXiv preprint arXiv:2004. /2765, 1(3), 2020.
Salvatore Attardo. Translation and humour: approach based the general theory of verbal humour (gtvh). In Translating Humour, pp. 173-194. Routledge, 2016.
Salvatore Attardo. Linguistic theories of humor, volume 1. Walter de Gruyter GmbH & Co KG, 2024.
Salvatore Attardo and Victor Raskin. Script theory revis (it) ed: Joke similarity and joke represen- tation model. Humor: International Journal of Humor Research, 1991.
Ashwin Baluja. Text is not all you need: Multimodal prompting helps ilms understand humor. In Proceedings of the 1st Workshop on Computational Humor (CHum), pp. 9-17, 2025.
Kim Binsted, Anton Nijholt, Oliviero Stock, Carlo Strapparava, Graeme Ritchie, Ruli Manurung, Helen Pain, Annalu Waller, and Dave O'Mara. Computational humor. IEEE intelligent systems, 21(2):59-69, 2006.
Andrew Cattle and Xiaojuan Ma. Recognizing humour using word associations and humour anchor extraction. In Proceedings of the 27th International Conference Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018, pp. 1849-1858, 2018. URL
Dushyant Singh Chauhan, Gopendra Vikram Singh, Navonil Majumder, Amir Zadeh, Asif Ekbal, Pushpak Bhattacharyya, philipp Morency, and Soujanya Poria. M2h2: A multimodal multiparty hindi dataset for humor recognition in conversations. In Proceedings of the 2021 international conference on multimodal interuction, pp. 773-777, 2021.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint (tt:::/00..0377 2021.
Yang Chen, Chong Yang, Tu Hu, Xinhao Chen, Man Lan, Li Cai, Xinlin Zhuang, Xuan Lin, Xin Lu, and Aimin Zhou. Are u a joke master? pun generation via multi-stage curriculum learning towards humor 11m. In Findings of the Association for Computational Linguistics ACL 2024, pp. 878-890, 2024.
Lukas Christ, Shahin Amiriparian, Alice Baird, Panagiotis Tzirakis, Alexander Kathan, Niklas Müller, Lukas Stappen, Eva-Maria Meßner, Andreas König, Alan Cowen, et al. The muse 2022 multimodal sentiment analysis challenge: humor, emotional reactions, and stress. In Proceedings of the 3rd International on Multimodal Sentiment Analysis Workshop and Challenge, pp. 5-14, 2022.
Alessio Cocchieri, Luca Ragazzi, Paolo Italiani, Giuseppe Tagliavini, and Gianluca Moro. "what do you call dog that is incontrovertibly true? dogma": Testing Ilm generalization through humor. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics Vollme 1: Long Papers), pp. 22922-22937, 2025.

---

Published as a conference paper at ICLR 2026
Tomas Engelthaler and Thomas T Hills. Humor norms for 4,997 english words. Behavior research 2018.
methods,
Drew Gorenz and Norbert Schwarz. How funny is chatgpt? comparison of human-and ai-produced jokes. Plus one, 19(7)=0035334, 2024.
Kevin A Hallgren. Computing inter-rater reliability for observational data: an overview and tutorial. Tutorials in quantitative methods for psychology, 8(1):23, 2012.
Laura Hanu and Unitary team. Detoxify. Github. http://fithub.commmiiarraiadddd 2020.
Md Kamrul Hasan, Wasifur Rahman, AmirAli Bagher Zadeh, Jianyuan Zhong, Md Iftekhar Tan- veer, Louis-Prn Louis-Philiippe Morency, and Mohammed Ehsan Hoque. Ur-funny: A multimodal language dataset for understanding humor. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2046 2056, 2019.
Jack Hessel, Ana Marasović, Jena D Hwang, Lillian Lee, Jeff Da. Rowan Zellers, Robert Mankoff, and Yejin Choi. Do androids laugh at electric sheep? humor "understanding" benchmarks from the new yorker caption contest. In Proceedings of the 6t Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pp. 688-714, 2023.
Zachary Horvitz, Jingru Chen, Rahul Aditya, Harshvardhan Srivastava, Robert West, Zhou Yu, and Kathleen Mckeown. Getting serious about humor: Crafting humor datasets with unfunny large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computa- tional Linguistics (Volume 2: Short Papers), pp. 855-869, 2024.
Eunjeong Hwang and Vered Shwartz. Memecap: A dataset for captioning and interpreting memes. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 1433-1445, 2023.
Marcio Lima Inácio, Gabriela Wick-Pedro, and Hugo Gonçalo Oliveira. What do humor classifiers learn? an attempt to explain humor recognition models. In Proceedings of the 7th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pp. 88-98, 2023.
Sophie Jentzsch and Kristian Kersting. Chatgpt is fun, but it is not funny! humor is still challenging large language models. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pp. 325-340, 2023.
Jungo Kasai, Keisuke Sakaguchi, Lavinia Dunagan, Jacob Morrison, Ronan Le Bras, Yejin Choi, and Noah A Smith. Transparent human evaluation for image captioning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies, pp. 3464-3478, 2022.
Yoon Kim. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746-1751. 2014.
Tomer Levinboim, Ashish V Thapliyal, Piyush Sharma, and Radu Soricut. Quality estimation for image captions based on large-scale human evaluations. In Proceedings the 2021 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 3157-3166, 2021.
the
Junnan Liu, Hongwei Liu, Linchen Xiao, Ziyi Wang, Kuikun Liu, Songyang Gao, Wenwei Zhang, Songyang Zhang, and Kai Chen. Are your llms capable of stable reasoning? arXiv preprint arXiv:2412.13117, 2024.
Lizhen Liu, Donghai Zhang, and Wei Song. Exploiting syntactic structures for humor recognition. In Proceedings of the 27th International Conference on Computational Linguisties, COLING 2018, Santa Fe, New Mexico, USA, August 20-26, 2018, pp. 1883,1883, 2018a. URL https:
12

---

Published as a conference paper at ICLR 2026
Lizhen Liu, Donghai Zhang, and Wei Song. Modeling sentiment association in discourse for humor recognition. In Proceedings of the 56th Annual Meeting of the Association for Computational URL https::/aclaattaaaa
Linguistics (Volume 2: Short Papers), pp. 586-591, 2018b.
Junie-Anne Meaney, Steven Wilson, Luis Chiruzzo, Adam Lopez, and Walid Magdy. Semeval 2021 task 7: Hahackathon, detecting and rating humor and offense. In Proceedings of the 15th Inter- national Workshop on Semantic Evaluation (SemEval-200) -0211), pp- 105-119, 2021.
George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11);
39-41, 1995.
Piotr Mirowski, Juliette Love, Kory Mathewson, and Shakir Mohamed. A robot walks into bar: Can language models serve as creativity supporttools for comedy? an evaluation of Ilms' humour alignment with comedians. In Proceedings of the 2024 ACM Conference Fairness, Account- ability, and Transparency, pp. 1622-1636, 2024.
Mahmoud Mohammadi, Yipeng Li, Jane Lo, and Wendy Yip. Evaluation and benchmarking of Ilm agents: A survey. In Proceedings of the st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pp. 6129-6139, 2025.
Elliott Oring. Joking asides: the theory, analysis, and aesthetics of humor. University Press of Colorado, 2016.
Badri N Patro, Mayank Lunayach, Deepankar Srivastava, Hunar Singh, Vinay P Namboodiri, et al. Multimodal humor dataset: Predicting laughter tracks for sitcoms. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 576-585, 2021.
Siddhesh Pawar, Junyeong Park, Jiho Jin, Arnav Arora, Junho Myung, Srishti Yadav, Faiz Ghifari Haznitrama, Inhwa Song, Alice Oh, and Isabelle Augenstein. Survey of cultural awareness in language models: Text and beyond. Computational Linguistics, pp. 1-96, 2025.
Willibald Ruch, Salvatore Attardo, and Victor Raskin. Toward an empirical verification of the general theory of verbal humor. Humor: International Journal of Humor Research, 1993.
Wenbo Shang and Xin Huang. A survey of large language models generative graph analyties: Query, learning, and applications. IEEE Transactions on Knowledge and Data Engineering, 37 (12):6799-68) 2025. doi: 10.1 2..1099TKKDD.22255369999
Wenbo Shang. Jing Wei, Runhui Song, Yan Xu, and Binyang Li. Conversational humor identifica- tion based on adversarial learning on chinese sitcoms. In International Conference on Cognitive Computing, pp. 24-34. Springer, 2021.
Wenbo Shang, Jiangjiang Zhao, Zezhong Wang, Binyang Li, Fangchun Yang. and Kam-fai Wong. "i know who you are": Character- based features for conversational humor recognition in chinese. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 2927-2932, 2022.
Yuxi Sun, Wei Gao, Hongzhan Lin, Jing Ma, and Wenxuan Zhang. Explainable ethical assess- ment on human behaviors by generating conflicting social norms. In Kentaro Inui, Sakriani Sakti, Haofen Wang, Derek F. Wong, Pushpak Bhattacharyya, Biplab Banerjee, Asif Ekbal, Tan- moy Chakraborty, and Dhirendra Pratap Singh (eds.), Proceedings of the 14th International Joins Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chap- ter of the Association for Computational Linguistics, pp. 166-184, Mumbai, India, December 2025. The Asian Federation of Natural Language Processing and The Association for Computa- tional Linguistics. ISBN 979-8-89176-298-5 URL /tthtttsss//cllll/ http::///aaaahahhhaccaococccc hww.y.hooooooooooooooooooooooooo 2 ijenlp- long. 11/
Dmitry Vikhorev, Daria Galimzianova, Svetlana Gorovaia, Elizaveta Zhemchuzhina, and Ivan P Yamshchikov. Cleancomedy: Creating friendly humor through generative techniques. arXiv preprint ar XrXiv:2412.0920) 2024.
Han Wang, Yilin Zhao, Dian Li, Xiaohan Wang, Xuguang Lan, Hui Wang, et al. Innovative thinking, infinite humor: Humor research of large language models through structured thought leaps. In The Thirleenth International Conference Learning Representations 2025.
13

---

Published as a conference paper at ICLR 2026
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe V Le. Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24837, 2022.
Orion Weller and Kevin Seppi. Humor detection: A transformer gets the last laugh. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Inter- national Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3621-3625, 2019.
Orion Weller and Kevin Seppi. The rjokes dataset: a large scale humor collection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 6136-614 63666141,14 2020.
Orion Weller, Nancy Fulda, and Kevin Seppi. Can humor prediction datasets be used for humor gen- eration? humorous headline generation via style transfer. In Pruceedings of the Second Workshop on Figurative Language Processing, pp. 186-191, 2020.
Zhikun Wu, Thomas Weber, and Florian Müller. One does not simply meme alone: Evaluating co-creativity between Ilms and humans in the generation of humor. In Proceedings of the 30th International Conference on Intelligent User Interfaces, pp. 0082-1092, 2025.
Yubo Xie, Junze Li, and Pearl Pu. Uncertainty and surprisal jointly deliver the punchline: Ex- ploiting incongruity-based features for humor recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1th International Joint Con- ference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 2: Short Papers), Vir- tual Event, August 1-6, 2021, pp. 33-39, 2021. URL httt::://ooi.org/1011666533
2021.acl-shore.
Hiroaki Yamane, Yusuke Mori, and Tatsuya Harada. Humor meets morality: Joke generation based on moral judgement. Information Processing & Management, 58(3):102520, 2021.
Diyi Yang, Alon Lavie, Chris Dyer, and Eduard Hovy. Humor recognition and humor anchor extrac- tion. In Proceedings the 2015 conference on empirical methods in natural language processing, pp. 2367-2376, 2015.
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhen- guo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models. In ICLR, 2024.
Nima Zargham, Vino Avanesi, Leon Reicherts, Ava Elizabeth Scott, Yvonne Rogers, and Rainer Malaka. "funny how?" serious look humor in conversational agents. In Proceedings of the 5th International Conference on Conversational User Interfaces, pp. 2023.
Chaoyun Zhang, Shilin He, Jiaxu Qian, Bowen Li, Liqun Li, Si Qin, Yu Kang, Minghua Ma, Guyue Liu, Qingwei Lin, et al. Large language model-brained gui agents: A survey. TMLR, 2025.
Jifan Zhang, Lalit Jain, Yang Guo, Jiayi Chen, Kuan Zhou, Siddharth Suresh, Andrew Wagenmaker, Scott Sievert, Timothy T Rogers, Kevin G Jamieson, et al. Humor ai: Massive scale crowd- sourced preferences and benchmarks for cartoon captioning. Advances in Neural Information Processing Systems, 37:125264 125286, 2024.
Shanshan Zhong, Zhongzhan Huang, Shanghua Gao, Wushao Wen, Liang Lin, Marinka Zitnik, and Pan Zhou. Let's think outside the box: Exploring leap-of-thoughe in large language models with creative humor generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 13246-13257 2024.
Yichao Zhou, Jyun-Yu Yu Jiang, Jieyu Zhao, Kai-Wei Chang, and Wei Wang. "the boating store had its best sail ever": Pronunciation-attentivv contextualized pun recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, acl-main.
1ttp://11.1111////// 2020, pp. 813-822, 2020. URL pattps://doi..r http:::///oa..oogg......55////// 75
14

---

Published as a conference paper at ICLR 2026
Zhanke Zhou, Rong Tao, Jianing Zhu, Yiwen Luo, Zengmao Wang, and Bo Han. Can language mod- els perform robust reasoning in chain-of-thoon thhuughtt prompting with noisy rationales? In NeurlPS, 2024.
Zhanke Zhou, Xiao Feng, Zhaocheng Zhu, Jiangchao Yao, Sanmi Koyejo, and Bo Han. From passive to active reasoning: Can large language models ask the right questions under incomplete information? In ICML, 2025a.
Zhanke Zhou, Zhaocheng Zhu, Xuan Li, Mikhail Galkin, Xiao Feng. Sanmi Koyejo, Jian Tang, and Bo Han. Landscape of thoughts: Visualizing the reasoning process of large language models. arXiv preprint arXiv:2503 2266,,,,,6,, 2025b.
Yanyan Zou and Wei Lu. Joint detection and location english puns. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Pupers), pp. 2117-2123, 2019. URL fttps://doi.crg/11
18653/v1/n19-1217
15

---

Published as a conference paper at ICLR 2026
Algorithm 1 HOMER Framework.
Require: A cartoon image I, the number of top-k relevant jokes k, the threshold of retrieved entities rank Generated funny cartoon captions Cap(!).
Ensure:
1: Phase I: Script Extraction: (C, Extract(/):
C: Script Oppositions, D: Situation Description
f- Extract(I):
Define script opposition 4 (ssrrppe(-) and design prompt as $ Pecripes D);
4: C E- Extract scripe (T, D));
5: Phase II: Imagination: Tim 4- Imagine(T, C, D);
6: Choose candidate targets from local and global perspectives:
{loc.glob}, Troot t:
8: Ore <- D. Ogiah +- I; D Local: fine-grained situation description. Global: obvious scene entities
9:
for Vv E V do
10: Ent C) ft,,... tm Troot + Toot U Ent(Ou,C))
Deep-pattern imagination forms backbone chains:
12:
for vt, E Troot
13:
T (ta), C t;i
Initialize chain by to D. is determined by LLMs
14:
for YT E (0,,... n2 do
15:
6 = fchain (et)) = 14 E Troot }:
Ti + (e
Ti
16:
17:
T (Til4, Tiooe) s Smerge (T)):
Align local and global entities across T''
18:
Broad-pattern imagination through humor- retrieval:
19:
for V4: Troot, Yes") T. do
20:
2y LembbD,C,,
21:
for Vjj do
22:
simm(Zg: 2j) )
f- T Z,
||
23:
Jesp = argtopk( (rmopkissmmii (z), 2,)): jEH
24:
for j Jopk do (61,,2,-4 from j:
25:
Tokenize and lemmatize the joke
26:
for Ve Ej do (e) .e)
27:
TSS (s) ((TSS(s.,, (s),))
> Relevance score
hopk hepp com)((s,8)) Ese 1sEEfy
28:
Have
> Frequency score
Isc E8
Meopkl
Hdiv(E)) (5)) Eper 1[N())>0))
29:
Diversity score
[P]
30:
Hle,)) ,5) = (el" (e)"),))++ (e) + Hano(e);
31:
Prune: Exeat + [e|rank(H((ef" ,,) < 6):
32:
Ti + T E)le E Eleat); }. Tim + (Tilts E Troot::
33:
Phase III: Generation: Cap(/) &- Gen(C, D, Ti, R);
34:
+ Sample(C), < SeleofTargets(Troos roots C); Randomly select conflict scripts and relevant targets
35:
P. < PT))(T;) by DFS, T; E Timi
Enumerate paths from ancestor leaf;
36:
P. + SamplePath(P.):
b Randomly sample one imagination path D Construct the GTVH--gudee prompt.
37:
(CC, Pi,
38:
Cap(T) +- Gen(# P.
39:
return Cap(!);
A HOMER ALGORITHM.
We propose HOMER, three-phase framework for humorous image captioning, summarized in Al- gorithm [ Phase (lines 1-4) extracts core conflicting scripts via an extractor. Phase II (lines 5-32) expands humorous imagination with a hierarchical imaginator by (i) initializing candidate humor targets from local and global views guided by the confficting scripts (lines 5-10), (ii) performing deep-pattent imagination via LLM--drvee associations to form free- as-osciciiiiinn backbone chains (lines 11 11-17), and (iii) conducting humor-relevance retrieval grow the chains into imagination trees (lines 18-32). Phase III (lines 33-39) employs a -uVdded generator to generate funny captions conditioned five constructed knowledge resources.
16

---

Published as a conference paper at ICLR 2026
B EXPERIMENTS
B. DATASETS
Humor Retrieval Database Construction. particular, we collect and reorganize 11 humor benchmarking datasets our humor retrieval dataset, which are from Pun of a Day (Yang et al. 2013). Short Jokes (Annamoradnejad & Zoghi 2020). Reddit Jokes (Weller
Seppil 2019).
rJoke (Weller & Seppil 2020), SemEval 2021 Task 7 (Meaney et al. (2021). TED Laughter (Kim
2014). HumorNorm (Engelthaler Hills 2018). CleanComedy (Vikhorev et al. 2024). ShortJoke
CrowdTruth and and Dad Joke
Multi-stage data curation. We construct our humor retrieval database through a multi-stage data curation process. First, we collect several publicly available humor-related datasets. Next, as an initial fitterigg step, we employ humor rating information provided within the datasets to eliminate entries rated as not funny; specifically, all jokes with humor rating lower than 3 are discarded. Subsequently, we perform data cleaning to remove noise and ensure quality. To further refine the corpus, we eliminate duplicate jokes as well jokes that exhibit excessive textual similarity. In particular, for any pair of jokes sharing more than 80% of their English words, we retain only the longer version. After completing these operations, our finalized humor retrieval database comprises a total of 335,570 jokes.
Comparison of joke database and our test dataset. The joke database consists primarily of one-liner text jokes, explicitly excluding cartoons or image captions, whereas the test set comprises original, publicly submitted captions from the New Yorker Caption Contest. The database used exclusively for text-only humor tasks (humor detection, rating, and joke generation), while the test set is reserved for multimodal humor tasks, specifically funny caption generation, ensuring clear separation of modalities and application contexts. A summary of formats, sources, and task usage is provided in Table
License and curation policy: All datasets used are publicly available and we follow strict curation protocol to prevent cross-modal leakage: (i) we exclude any datasets that are multimodal or have been used for multimodal applications; (ii) the corpus is restricted text-only humor, focusing short, one-liner jokes with simple structure; and (iii) we will remove items that are near-duplicates or overly similar content in our multimodal captioning evaluation, minimizing any risk of overlap.
Table 7: Overview of humor-related corpora
Corpus
Data Format
Source(s)
Intended Use
Short Jokes
One-liner jokes
Various joke websites
Text-only humor
Reddit Jokes
One-liner jokes
Reddit (r/Jokes)
Text-only humor
Pun of the Day
One-liner jokes
Pun of the Day website
Text-only humor
rJoke
One-liner jokes
Reddit (r/Jokes)
Text-only humor
SemEval 2021 Task
One-liner jokes
SemEval 2021 Task 7
Text-only humor
TED Laughter
Speech
TED Talks
Text-only humor
HumorNorm
Words
English lexical resources
Text-only humor
CleanComedy
One-liner jokes
Reddit, Twitter, other platforms
Text-only humor
CrowdTruth
One-liner jokes
Various joke websites
Text-only humor
Dad Jokes
One-liner jokes
Grin, Dad Joke
Text-only humor
Our Test Dataset
Cartoons with captions
New Yorker Caption Contest
Multimodal humor
Data distribution tested datasets. Our method generalizes beyond images with overt anoma- lies, effectively handling a diverse range of humor sources, including unexpected logic, contextual incongruity, personification, and role reversals. Its foundation script opposition, a central concept in the General Theory of Verbal Humor (GTVH) and related accounts, where humor emerges from
httpss//iihibbcmmmmmmmn///oaooos hhttp:://githubbco Tow.e//oootttttttt///otottotttoo For-Huoor-DDteeeiiiie http:://wwww..aal..ccocc///eeeee
17

---

Published as a conference paper at ICLR 2026
surprising conflicts between competing scripts. We statistic different types of script opposition in the dataset across the following dimensions:
Abnormalities: visual elements that deviate from everyday norms.
Unexpected logic: outcomes that defy conventional expectations (e.g., role reversals).
Contextual incongruity: entities actions that inconsistent with their context.
Exaggeration: phenomena behaviors presented in extreme form.
Ambiguity: multiple plausible interpretations that invite playful confusion.
Personification: nonhuman entities endowed with human traits, intentions, or roles.
Table Data distribution in the Humor in Al dataset
Humor Basis
Occurrences
Percentage
Abnormalities
I22
34%
Unexpected Logic
70
19%
Contextual Incongruity
67
18%
Exaggeration
14%
Ambiguity
55
15%
Total
365
100
B.2 COMPETITORS
Recent advances in multimodal and language-based humor generation have led the develop- ment of several benchmark methods. HumorousA et al., (024) and CLon et al., 2024) represent state-of-the-ar approaches for multimodal humor generation, typically leveraging advanced reasoning strategies to create contextually appropriate and funny captions. In contrast, LoL(Wang al., 2025) addresses dialogue- based humor generation through a specialized fine- tuning framework tailored conversational contexts. Additionally, Phunny et al.) 2024) focuses on the generation puns, specifically targeting linguistic wordplay and double meanings.
B.3 METRICS
Unbiased @ Calculation. To evaluate the performance of our method, we employ the un- biased pass@1 k metric (Chen et al. (02I), which estimates the probability that at least one of randomly selected captions funnier than the human-written caption. For given image, we sam- ple candidate captions from the model, and let e denote the number of captions that are funnier than the ground truth human caption among them. The unbiased pass@; for this task is computed as (*-9)
where (k) denotes the binomial coefficient. This formula corrects for the bias that
may arise when multiple winner captions exist among the samples. Averaging over all N images in
the dataset, the overall unbiased pass ls metric calculated
N
pass @k k
Σ N
where n; the total number of generated captions for the i-th image. We set n, = our ex- periments, and c; is the number of captions in i sampled captions judged to be funnier than the corresponding human caption by the evaluator. This unbiased estimation provides a reliable mea- sure of the model's win rate given multiple sampling trials.
B.4 HYPERPARAMETER ANALYSIS
Analysis of & and in the joke database retrieval. Optimal performance is observed when i: is set to 3, 5, or and when 8 set to or 7, suggesting that retrieving too few humor instances results in an insufficient imagination space, whereas retrieving too many introduces noise, such as unrelated
18

---

Published as a conference paper at ICLR 2026
Group 1-9
Group 200-209
Group 1000-1009
1.00
0.75
0.50
pass@ 1
pass@ 1
pass @
0.25
pass@
pass@
pass@
pass@ 5
pass@ 5
pass@ 5
0.00
T=0
T=0.5 Temperature T
T=1
T=0
T=0.5
T=1
T=0
T=0.5
T=1
Temperature T
Temperature T
Figure 7: Robustness of the LLM temperature.
Table Performance comparison on Meme dataset with pass @k metrics
Method
pass
pass@2
pass
CoT
74.33
86.
95,83
Few-Shot
66.67
81.67
91.67
Self-consistency
70.00
90.83
91.67
HumorousAI
75.00
80.00
83,33
LoL
71.67
81.12
97.54
Phunny
21.67
31.67
33.33
CLoT
76.67
88.33
91.67
HOMER
83.33
96.67
98.86
entities, which can degrade performance. Therefore, we select = and in our method to strike a balance between maintaining high performance and minimizing the noisy inducing.
evaluate stability our method under different sampling temperatures of the LLM by varying the temperature parameter across the values 0, 0.5, and 1, as shown in Figure 7 Experimental results demonstrate that our approach achieves consis- tently strong performance across all tested temperature settings, indicating robustness changes in the sampling temperature. The results suggest that the effectiveness of our method not sensitive to the choice of temperature within this range, underscoring its practical reliability and generalizability in diverse decoding scenarios.
Robustness to Temperature Variation.
B.5 COMPARISON OF HUMOR FREQUENCY SCORE AND TF-IDF
The distinction between TF-IDF and our humor-rclevance metric lies in their weighting schemes: TF-IDF down-weights globally common tokens, whereas approach up-weights tokens and hu- mor concepts that recur in jokes relevant to given situation and its conflicting scripts. We aim to identify entities that are both salient and frequently used reliable components for humor genera- tion. The objectives of the two methods are opposite. In experiments, our humor-frequency score outperformed TF-IDF-based baseline, shown in Table 10
Table 10: Comparison of scoring methods on pass @k
Scoring Method
pass @
pass @
p
TF-IDF Humor-Freee Frequeryyy (Ours)
63.00
79.83
86.33
66.41
83.70
89.18
B.6 EVALUATION ON FOUR DIMENSIONS
We conduct comprehensive qualitative and quantitative analyses across four key dimensions of multi-modal humor generation, leveraging both expert LLMs (GPT-5) and automated metrics: (1) visual understanding, (2) humor understanding, (3) humor imagination, and (4) stylistic ex- pression. The results shown in Table an and Table 12
19

---

Published as a conference paper at ICLR 2026
For the dimensions requiring nuanced linguistic and cognitive sudgment-vvisst understanding. hu- mor understanding, and stylistic expression we use GPT-5 as an expert evaluator. Specifically, for each image, GPT-5 ranks the humorous captions generated by eight different methods according to well-defined criteria for each dimension, assigning ranks from best (#1) to worst (#8). The average rank for each method (lower is better) is reported ensure fair and consistent assessment. To enhance reliability, each ranking is conducted over five repeated trials. For the dimension of humor imagination, we utilize two established automated metrics: (i) n-gram diversity, which quanti- fies lexical variety, and (ii) NLI diversity, measuring the percentage of non-entailing caption pairs Roberta-large natural language inference model. Higher scores on
as judged by a state-of of-the-atr these metrics indicate a greater capacity for imagination, as producing wider variety of humorous captions reflects the model's ability to generate diverse and creative comedic ideas.
Table 11: Humor AI Dataset: Comparative results across four dimensions
Model
Visual Under-
Humor Under-
Stylistic pression
Ex-
Humor Imagination
standing
standing
Avg. Rank (L)
Avg. Rank (4)
Avg. Rank (L)
3-gram (1)
NL. Diversity (%) (f)
CoT
5.5
4.8
5.6
0.87
85.3
Few-Shot
4.4
3.7
3.8
0.84
81.6
Self-Consistency
5.9
5.4
5.7
0.88
85.9
HumorousAl
4.9
0.59
70.0
Lol.
4.9
4.6
4.4
0.92
89.3
Phunny
5.9
5,6
5,4
0.69
81.9
CLoT
4.5
3.8
0.46
HOMER (Ours)
2.5
3.2
2.3
0.98
91.5
Table 12: Electronic Sheep Dataset: Comparative results across four dimensions
Model
Visual Under-
Humor Under-
Stylistic pression
Ex-
Humor Imagination
standing
standing
Avg. Rank (L)
Avg. Rank (L)
Avg. Rank (4)
3-gram
NLI Diversity (%)
CoT
3.8
4.3
4.2
0.88
84.2
Few-Shot
4.5
4.5
4.1
0.89
83.1
Self-Consistency
4.2
4.5
0.84
86.40
HumorousAl
6.9
6.7
6.1
0.92
88.5
LoL
4.5
4.6
5.7
0.94
91.9
Phunny
6.8
6,8
6.9
0.89
85.2
CLoT
3.5
3.2
2.7
0.83
78.9
HOMER (Ours)
1.8
0.98
92.2
B.7 TWO-STAGE HUMOR TUNING STRATEGY OF HUMOR EVALUATOR
For the Humor-tuned LLaMa 3, utilize two-stage training strategy: supervised fine-tuning (SFT) followed by Direct Preference Optimization (DPO). This process encourages the model to align with human preferences for humor by assigning higher rewards funnier and lower rewards to less funny captions in the benchmarking ranking. During inference, Humor-tuned LLaMa assigns reward score each caption. We then assess whether the model can correctly predict the ground-truth ranking by verifying that the higher reward corresponds to the higher rank.
B.8 HUMAN EVALUATION.
Procedure of human evaluation. Human evaluation caption funniness is conducted using a stan- dardized procedure. As shown in Figure for each cartoon image, human annotators are presented with seven candidate captions and asked to rate the funniness of each caption on scale from to 5. The assessment proceeds in two steps: first, evaluators must decide whether the caption is relevant to the given cartoon. the caption deemed irrelevant, it automatically receives a score of 1.0. the caption is relevant, annotators then assess its comedic quality according to the following scale: Not Funny (1.0), Slightly Funny (2.0), Moderately Funny (3.0), Funny (4.0), and Very Funny (5.0). This protocol ensures that both relevance and humor systematically appraised and provides
20

---

Published as a conference paper at ICLR 2026
Table 13: Human evaluation of seven methods: Mean, Standard Deviation (SD), and Median of Humor Ratings by 20 Raters (1-5 Scale).
Dataset Method
Humor in Al
Electronic sheep
Mean
SD
Median
Mean
SD
Median
CoT
2.47
0.6697
2.50
2.20
0.7756
2.45
Few-shot
2.96
0.6992
2.80
2.56
1.0065
3.10
CLoT
2.95
0.7732
2.75
2.53
0.7314
2.60
Self-consistency
2.66
0.5949
2.60
2.25
0.5520
2.40
HumorousAl
3.01
0.7301
2.75
2.24
0.8093
1.85
LoL.
3.16
0.8338
3.40
2.40
0.8236
2.15
Ours
3.54
0.5862
3.65
3.31
0.8491
3.20
fine-grained numeric measure of each caption's effectiveness in eliciting amusement. The human evaluation was conducted on a total of 2,800 data points, calculated as 20 cartoon images were eval- uated by 20 human raters across 7 different caption generation methods (20 images 20 raters x methods).
Example of H Human Evaluation
Instructions
Based on the cartoon image, rate the funniness the six cartoon captions (1-5). First, determine whether it is relevant. not, score 1.0. If relevant, score the funniness.
Not Fumny (1.0), Slightly Funny (2.0). Moderately Funny (3.0). Funny (4.0), Very Funny (5.0).
Figure 8: Example of human evaluation.
Detailed human evaluation. The results of our human evaluation, summarized in Table [3] demon- strate that our method consistently outperforms six baseline methods across humor datasets, "Humor in AI" and "Electronic Sheep." 20 human raters scored each method using 1-5 scale. Our approach achieves the highest mean humor ratings on both datasets. These results collectively indi- cate the robustness and effectiveness of our approach in generating humorous content as perceived by human evaluators.
B.9 DIVERSITY EVALUATION.
We assess caption diversity using two established metrics. The results are shown in Tables 14 and 15
N-gram (Distinct-N) diversity: a lexical-variability measure computed as the ratio of unique n-grams the total number of generated n-grams (typically for (1),,33).
NLI Diversity: the percentage of caption pairs classified non-entailing by widely adopted RoBERTa-large natural language inference model, capturing semantic variety be- yond surface form.
Higher scores on either metric indicate more diverse of captions. Across both metrics, our results provide strong empirical evidence that HOMER generates more diverse funny captions, con- sistently outperforming all baselines.
B.10
CASE STUDIES
21

---

Published as a conference paper at ICLR 2026
Table 14: Humor in AI Dataset: n-gram coverage and NLI diversity
Model
I-gram (1)
2-gram (1)
3-gram (T)
NLI Diversity (1)
HumorousAl
0.45
0.55
0.59
70.0%
LoL
0.64
0.83
0.87
89.3%
Phunny
0.50
0.64
0.69
81.9%
CLoT
0.35
0.43
0.46
51.5%
Our Method
0.76
0.94
0.98
91.5%
Table 15: Electronic Sheep Dataset: n-gram coverage and NLI diversity
Model
1-gram (1)
Z-gram (1)
3-gram (1)
NLI Diversity (1)
HumorousAl
0.65
0.87
0.92
88.5%
LoL
0.63
0.88
0.94
91.9%
Phunny
0.64
0.84
0.89
85.2%
CLoT
0.58
0.78
0.83
78.9%
Our Method
0.72
0.94
0.98
92.2%
In Figure 9 the conflict between a professional office setting with unexpected horseplay juxta- poses scripts of routine and hyperbole, yield- ing humorous reading. This script opposi- tion leverages surprise, incongruity, and cog- nitive resolution, which are central to effective and engaging humor.
Situation:
Shansharn
B.11 HARMFUL DETECTION.
In the office with professional collcagues
Script opposition:
Professional behavior vs. unexpected horseplay
We evaluate harmful content in HOMER's 's gen- erated captions using Detoxify (Hanu & Unitary team 2020), widely used toxicity detector, across seven dimensions: toxicity, severe toxi- city, obscene, identity attack, insult, threat, and sexual explicit, as shown in Figure IO On Hu- mor in AI, average scores the dataset in seven dimensions very low, summing 0.023 (< 0.03). On Electronic Sheep, the sum of toxic- ity is 0.015 (< 0.02). These consistently low scores indicate negligible harmful content, sug- gesting that our HOMER generates captions that largely avoid abusive, threatening, or sexually explicit language.
Target: Narrative strategy: Language:
Cowgirl Short narrative
Wordplay
Funny caption generated by HOMER: Well, Janet, I admire your drive, but this isn't exactly what meant by taking the reins work.
Figure 9: GTVH-guided Example.
We also evaluate more harmful content in captions generated by HOMER using Detoxify (Hanu & Unitary team, 2020), as shown in Figure [I Specifically, harmfulness toxicity, severe toxicity, obscene lan- guage, identity attack, insult, threat, and sexual explicitness. This evaluation is performed on two datasets, Humor in AI and Electronic Sheep, and covers captions produced by three base LLMs: Claude-4, Qwen-VL, and LLaVA. The results con- sistently demonstrate that harmful content scores are very low across all toxicity dimensions, datasets, and base models. These findings indicate that HOMER reliably generates captions that are safe, minimizing the risk of toxic, offensive, otherwise inappropriate outputs in diverse settings.
texicity Severe texicity
toxicity (96)
40
assessed
obscane
across seven dimensions:
30
identity attack
insult
20
threat
10.
sexual explicit 1.52
Humor Al
Electronic sheep
Figure 10: Harmful detection.
22

---

Published as a conference paper at ICLR 2026
60
lostrity
loality
lostrity
Sum ol toxieity (96)
:00
Sum of toxieity g
40
identity_ pitackk
obscene
Sum boxicity
40
iderilly_sthe vitfac
obscene
40
obscene
identity_nii wttaak
30
Insut
30
Insut
30
Insut
20
sootaal explicct
20
sesual explitt
20
seouaal explect
4.13
10
3.43
3.13
8.40
Electronic sheep
sheep
Electronic sheep
(a) Claude-4
(b) Qwen-VL VL
(c) LLaVA
Figure 11: Harmful detection of three base models.
B.12 COMPUTATIONAL COSTS
Efficiency of HOMER. Table T preseent the number of API calls required at each stage of HOMER for both the full model and the naive model. The extractor and generator stages make the same number of API calls in both models. However, in the naive model, the imaginator stage does not make any API calls because the imagination results of LLM association can be pre-processed and saved in advance. Humor-retrieval does not need API calls. In contrast, the full model involves three API calls at the imaginator stage, reflecting real-time imagination processing. This comparison highlights the efficiency gained by pre-processing imagination results in the naive model.
Table 16: Number of API calls for each stage of HOMER
Model
Full Model
Naive Model
Extractor
2
2
Imaginator
0
Generator
2
2
Fair comparison under the same LLM calls. For fairness, each baseline generates multiple inde- pendent humorous captions and selects the best one for each output. The number of attempts was set to match exceed HOMER's 's seven LLM calls. We show the pass@) @ I results evaluated by GPT-5 as Table 17 Despite with equal or greater call budgets, baselines show modest gains and do not close the performance gap with HOMER, indicating that HOMER's superiority is from its collaborative, structured framework rather than a higher computational budget.
Table 17: LLM calls per output and pass @ performance across methods.
Method
LLM Calls per Output
pass @ (%)
HumorousAl
(=3 calls 3 repeats)
62.7
LoL
8 (=4 calls x 2 repeats)
58.0
Phunny
(=3 calls 3 repeats)
19,1
CLoT
7 (=7 calls X repeats)
61.2
HOMER (Ours)
66.4
B.13 SIGNIFICANCE TEST AND AGREEMENT EVALUATION
We assess statistical significance with a two-sided Wilcoxon signed-rank test on Pass@k scores and quantify agreement between GPT-5 and human judgments via correlation analysis. The results show that our method significantly outperforms all baselines (p <0.05), as shown Table 18
We measure the agreement between GPT-5 scores and human ratings per caption via the Pearson correlation coefficient. The results in Table 19 show that GPT-5 and humans are positively strongly correlated.
23

---

Published as a conference paper at ICLR 2026
Table 18: Pairwise significance test results for Humor in Al (left) and Electronic Sheep (right)
datasets (a = 0.05).
Comparison Pair
p-value
Significant?
Comparison Pair
p-value
Significant?
Ours vs. CoT
0.0000
Yes
Ours vs. CoT
0.0000
Yes
Ours vs. Few-Shot
0.0027
Yes
Ours Few-Shot
0.0120
Yes
Ours vs. Confisooni
0.0031
Yes
Ours vs. Self- Consistency
0.0001
Yes
Ours vs. CLoT
0.0153
Yes
Ours vs. CLoT
0.0011
Yes
Ours HumorAI
0.0036
Yes
Ours vs. HumorAl
0,0001
Yes
Ours LoI.
0.0001
Yes
Ours vs. Lol.
0.0004
Yes
Ours vs. Phunny
0.0000
Ours Phunny
0.0000
Yes
Humor in AT dataset
Electronic Sheep dataset
Table Pearson correlation coefficients by evaluator pair
Evaluator Pair
Pearson coefficient
Humor in AI
0.8608
Electronic Sheep
0,8533
B.14 PERFORMANCE WITH GPT4.1 AS THE EVALUATOR
We conducted additional experiments using GPT-4.1 (the second-strong evaluator in Table 2) to @ and statistical significance. Results corroborate GPT-5 findings: HOMER con- sistently outperforms all baselines in most cases, with significant differences (p <0.05), as shown in Table 20 and 21
Agreement Evaluation. We measure the agreement between GPT-4.1 scores and human ratings per caption. The results show that GPT-4.1 exhibits moderate positive alignment with humans with 0.5639 Pearson coefficient.
B.15 ANALYSIS OF DATA LEAKAGE
Different data sources and formats. The joke database primarily comprises one-liner jokes sourced from Pun of the Day. TED Talks, Reddit (r/jokes), and short-joke websites, and explicitly excludes cartoon or image captions. contrast, the test set consists of original, publicly submitted captions from the New Yorker Caption Contest.
Different task usage. The joke database is used exclusively for text-only humor tasks (e.g., humor detection, humor rating, and joke generation). The test set is reserved for a multimodal humor task, such as funny caption generation, ensuring clear separation of application contexts.
Empirical negligible overlap. We evaluate potential leakage by quantifying instance-level overlap between the test set and the joke database using the exact-maich and normalized comparisons of caption answers. All metrics yielded zero overlap, indicating no evidence of leakage between the joke database and the testing data.
16 ABLATION ON JOKE DATABASE
We have conducted experiments of ablation the scale of our joke dataset, varying the proportion from 0% to 100%. At 0%, the model relies solely on the GTVH-guided structure. The results indicate that our model's performance increases steadily, further demonstrating the necessity of our joke database. Combined with the ablation studies in Table 4, the performance of the joke database alone, without our designed humor mechanism, shows significant drop. These results validate both our novel GTVH-based framework and joke database.
C THEORETICAL ANALYSIS OF Hrel
We define the relevance-opposittoon score as
Hhale))
TSS(s 8c) S(TSS(SE,, 8c)) CO(se,,
24

---

Published as a conference paper at ICLR 2026
Table 20: Pass @k results for the Humor in AI dataset across different subsets.
Methods
pass @ 1
pass
pass
Top 10
CoT
61.0
76.0
81.9
Few-Shot
67.8
76.5
82.4
Self-Consistency
69.6
90.1
92.7
CLoT
60.8
74.8
80.0
HumorAI
68.2
85.5
89.6
LoL
70.6
89.3
93.3
Phunny
16.8
32.5
42.0
Our HOMER
74.6
90.6
95.0
#200-1109
Co'I
64.2
81.1
85.0
Few-Shot
71.6
87.9
90.0
Self-Consistency
69.0
86.0
88.9
CLoT
63.6
74.1
78.0
HumorAI
72.8
86.5
90.0
LoL
73.2
90.9
93.9
Phunny
17.8
31.1
38.0
Our HOMER
75.2
91.5
95.0
#1000-1009
CoT
70.2
84.5
89.7
Few-Shot
73.8
89.2
90.8
Self-Consistency
73.8
88.2
91.0
CLoT
73.4
82.4
84.0
HumorAI
74.4
87.8
91.0
LoL.
74.6
90.9
94.5
Phunny
25.4
38.4
45.0
Our HOMER
77.2
89.6
95.1
Table 21: Pairwise significance test results.
Comparison Pair
p-value
Significant? (a 0.05)
Ours vs. CoT
0.0021
Yes
Ours vs. Few-Shot
0.0089
Yes
Ours vs. Self-Consistency
0.0091
Yes
Ours vs. CLoT
0.0030
Yes
Ours vs. HumorAI
0.0108
Yes
Ours vs. LoI.
0.0142
Yes
Ours vs. Phunny
0.0000
Yes
where TSS(-.) measures semantic similarity and CO(--) quamtifies conceptual opposition, with f(x) = x exp( -x) serving as a similarity-gater modulation function. This formulation fulfills the following criteria:
(i) Dominance by Semantic Similarity: The term TSS(se, se) is the primary additive component, ensuring that the overall score increases monotonically with greater semantic similarity, regardless of the value of CO.
(ii) Similarity- Gated, Bounded Bonus for Conceptual Opposition: The CO term multiplied by f(TSS), which serves I = 1 and decays to 0 as x + 0 or æ + oo, the contribution conceptual opposition substantial only for intermediate semantic similarity and is suppressed for both very low and very high TSS. Moreover, since [(x)] is maximized at e!_ and CO is assumed to be bounded (e.g., |CO| s 1), the bonus (or penalty) term is inherently bounded in magnitude.
adaptive gate. Since f(x) exp(=z) achieves its maximum
(iii) Principled Balance Benween Competing Tendencies: The function f provides a smooth and principled balance between rewarding similarities and oppositions: it modulates the influence of
25

---

Published as a conference paper at ICLR 2026
Table 22: Performance by database scale across different groups.
Group
Scale of Joke DB
pass (%)
pass @3 (%)
pass @5
Top 10
0%
58.2
73.2
78.4
25%
63.4
82.0
87.1
50%
65.7
78.5
81.7
75%
66.1
80.3
87.6
100
66.4
83.7
89.2
#200-209
0%
60.3
75.1
80.8
25%
66.2
84.9
88.7
50%
69.0
86.3
91.4
75%
71.5
87.2
92.1
100 %
73.4
88.4
92.6
#1000-1009
0%
63.2
78.7
83.7
25%
69.4
81.8
85.7
50%
72.9
84.3
90.0
75%
74.6
86.2
93.8
100 %
76.3
90.5
94.2
opposition such that opposition is only beneficial when the two sentences are neither too similar nor completely unrelated, reflecting a nuanced interplay between similarity and opposition.
In summary, the score Hree is monotonic in TSS when CO 0, bounded for all inputs, and ex- presses principled, interpretable balance between semantic similarity and conceptual opposition.
D FAILURE ANALYSIS
HOMER struggles with purely formal or inherently non-humorous images, especially when script opposition is difficult detect. These cases challenging even for humans, and the lack of narrative content and clear humor cues results captions with limited humor.
E STYLE CONTROL
Our modular architecture enables controlled stylistic conditioning through curated imagination trees and the design of instructions prompts. For example, the imagination tree can retrieve relevant semantic ambiguities among jokes in the joke database. Then, the Generator reinforce selected imaginative entities through explicit style directives guided by the designed instruction.
F RELATED WORKS
Classical Computational Humor. Computational humor, as challenging branch of computational
linguistics, employs computational methods study humor (Binsted et al. (2066) Wang al. 2025),
mainly including humor recognition (Cattle & Ma 2018 Liu et al.) [201 Saj Xic et al. 202 Zhou
al. 2020 Zou Lu, 2019. 2019,1am [iu al. [01 Shang et al. 2021) humor explanation (Hessel al.
2023 Patro et al. 2021 Amin & Burghardi 2020). and humor generation (Amin & Burghardt 2020
Weller et al. 2020 Yamane et al. [202]] | Zargham al. (023)) tasks. Classical computational humor
research focused rule-based, statistical approaches, and multimodal techniques for detecting and
modeling humor across text, audio, and vision [Inácio et al. 2023: Amin & Burghardi, 2020 Yang
al. 2015 Chauhan al. 2021 Christ et al. 2022 Hasan et al. (2019). Recently, incorporating
LLM reasoming techniques (Zhou et al. 2024
2025b into computational humor has become an
interesting and meaningful direction.
26

---

Published as a conference paper at ICLR 2026
G
PROMPT DESIGN
Prompt design in conflicting script extractor. We design the prompt to instruct the conflicting script extractor analyze scripl-opposition-cenaall situation description the contextual back- ground of the funny caption and then systematically identify and analyze conflicting or incongruous elements in the image The prompt includes the definition of script opposition, GTVH theory and the description of the image. It to analyze and extract all relevant conflicting scripts that exist in the situation description D and the image I. The detailed prompt can be found in Appendix Figure [2]
Prompt design in hierarchical imaginator. There are two views of the imagination. The lo- cal observation Ohe is from the detailed situation description D, capturing fine-grained entities or unexpected features within the image (e.g., oversized cups, professional figures). The global obser- vation Oloob leverages the image to emphasize the obvious entities in the scene (e.g., cups, table). For each view, the detailed prompt can be found in Appendix Figure 13
Prompt design in caption generator. The generation prompt is constructed by integrating the sit- uation description D, the selected conflicting scripts C, the creative imagination path P. of selected humor target t, and the generation options NS x LA, where specifies the narrative strategy and linguistic style. Formally, the prompt can be represented &(C, D, P., 92), which is then fed into the LLM--bsed caption generator producing the final funny caption. The detailed prompt can be found in Appendix Figure 14
H LLM USAGE CLAIM
In this paper, LLMs are utilized for polishing writing and are strictly confined to improving coher- ence, grammar, and style within textual content. No additional functionalities are incorporated.
27

---

Published as a conference paper at ICLR 2026
Figure 12: Example Prompt for Conflict Script Extractor
SYSTEM
Based on the Script Opposition theory from the General Theory of Verbal Humor (GTVH), analyze the given cartoon description and identify two or more conflict scripts exists in the description. A script refers to a bundle of knowledge expectations about a particular situ- ation. Script opposition occurs when two conflicting scripts (i.e., scenarios, expectations, or frames) are brought into contrast within the description, creating basis for potential humor. In your answer, list pairs of conflicting scripts (each as phrases or a short sentence) that are opposed contrasted within the description. Just present the conflicting script pairs directly.
USER
Description of the image: Inside a grand stone-walled throne room, a king sits stiffly on an ornate throne, his face marked by a mix of weariness and unease...
ASSISTANT
1. Imminent threat vs. calm indifference. 2. Royal authority and power vulnerability and danger...
Figure 13: Example Prompt for Hierarchical Imaginator
(Global View)
SYSTEM
Given conflict scripts and a cartoon, your task is to identify the main entities mentioned. For each identified entity, generate a logical chain of three relevant entities, each based directly on the previous one. Associations may include ingredients, containers, sources, related objects, or common companions. Output JSON: key is the entity, value is a list of three such imaginations.
USER
Image: Encoded cartoon image.
Conflicting scripts: 1. Imminent threat calm indifference. Royal authority and power vs. vulnerability and danger...
ASSISTANT
[king, crown, head)...
(Local View)
SYSTEM
Given conflict scripts and cartoon description, your task is to identify the main entities mentioned. For each identified entity, generate a logical chain of three relevant entities, each based directly on the previous one. Associations may include ingredients, containers, sources, related objects, or common companions. Output JSON: key is the entity, value is list of three such imaginations.
USER
Description: Inside grand stone-walled throne room, king sits stiffly on ornate throne, his face marked by a mix of weariness and unease... Conflicting scripts: 1. Imminent threat calm indifference. 2. Royal authority and power vs. vulnerability and danger...
ASSISTANT
[throne, chair, table], (stone-walled walled throne room, chandelier, hanging)...
28

---

Published as a conference paper at ICLR 2026
Figure 14: Example Prompt for Caption Generator
SYSTEM
Using the provided free-association chains, conflict scripts, and the cartoon description, generate a witty, funny and smart caption that spotlights the central incongruity and naturally combines key keywords in chains. Consider techniques such as narrative setups, linguistic styles and puns, but keep it short, concise and suitable as a cartoon tagline.
USER
Description: Inside grand stone-walled throne room, king sits stiffly on an ornate throne, his face marked by mix of weariness and unease...
Conflicting Scripts: Imminent threat vs. calm indifference. Royal authority and power vs. vulnerability and danger...
Free-associating chains of cartoon: [thread, chandelier, hanging]. [...]
ASSISTANT
Caption: When you're re hanging by thread, but the intern's still their trial period.
29


Reviews:
[{"review_id": "mmBwFKlNj2", "reviewer": "Reviewer_VAjU", "summary": "HOMER is a humor-theory–guided, three-role LLM for New Yorker-style cartoon captions that (1) extracts script oppositions, (2) builds retrieval-augmented “imagination” trees, and (3) generates captions; on two cartoon benchmarks it beats strong prompting baselines, with ablations showing each role matters.", "questions": "1. How sensitive are results to the choice of evaluator? Please report pass@K with a second strong evaluator and give 95% CIs; also correlate evaluator scores with human ratings per-caption", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Clear multi-role architecture grounded in humor theory; maps situation, script opposition, target, narrative strategy, and language to concrete modules \n2. Consistent SOTA gains across datasets/groups and base models; effect sizes reported\n3. Thorough ablations: module inclusion matrix (Tbl. 4) and score-term ablations (Fig. 3). \n4. Toxicity audit across seven dimensions shows low scores", "weaknesses": "1. My biggest concern for methodology is for creative generation, the desireable generation can be diverse. Is there an way to enforce the diversity of the generated response and ensure all generated caption can beat the baseline. For a single type of humor generation, hacking the benchmark with certain prompt structure could still occur. In addition, the author should also measure the diversity for the generation to ensure that actual creative generation is produced. \n\n2. Primary automatic metric uses GPT-5; while Table 2 benchmarks evaluators, no significance tests or cross-evaluator robustness are reported (e.g., bootstrap CIs on pass@K). Add statistical tests and report agreement between GPT-5 and humans per-caption.\n3. Retrieval compliance & leakage: Retrieval set lists several joke corpora but lacks licensing and dedup/leakage controls vs. test captions. Report licenses, filtering, and overlap stats between retrieved text and ground-truth or generated captions.\n4. Reproducibility gaps: Prompts, seeds, and full code release status are unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "IdPOM0EoKU", "reviewer": "Reviewer_hHh4", "summary": "The paper proposes HOMER, a novel framework for generating humorous captions for cartoon images. The approach is grounded in the General Theory of Verbal Humor (GTVH) and employs three coordinated LLM-based roles: (1) a conflicting-script extractor that identifies incongruities in images, (2) a hierarchical imaginator that builds imagination trees through LLM associations and humor-relevance retrieval from a joke database, and (3) a caption generator that produces humor based on the extracted knowledge. The method is evaluated on New Yorker Cartoon datasets, showing improvements over baselines.", "questions": "1. **Generalization**: How does HOMER perform on other humor generation tasks beyond cartoon captioning? Have you tested on joke completion, pun generation, or conversational humor?\n\n2. **Example quality**: Could you provide more compelling examples that better demonstrate the humor quality? The current examples seem to lack the wit typically associated with New Yorker captions.\n\n3. **Ablation on retrieval database**: What happens to performance as the joke database size varies? How sensitive is the method to the quality/style of jokes in the database?\n\n4. **Failure analysis**: What types of images or situations does HOMER struggle with? Can you provide a systematic analysis of failure modes?\n\n5. **Comparison fairness**: How many inference calls do the baseline methods use? Is the comparison fair given HOMER's computational overhead?\n\n6. **Style control**: Can HOMER generate different styles of humor (e.g., dark humor, slapstick, wordplay) by modifying the retrieval database or prompts?\n\n7. **Zero-shot performance**: How does HOMER perform without the joke retrieval component, relying only on the GTVH-guided structure?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. **Theoretically grounded approach**: Unlike prior work that relies on generic prompting or self-improvement, HOMER explicitly incorporates established humor theory (GTVH), providing better interpretability and control over the humor generation process.\n\n2. **Novel hierarchical imagination mechanism**: The combination of deep-pattern LLM associations with broad-pattern humor-relevance retrieval is creative and well-motivated, expanding the creative space while maintaining relevance.\n\n3. **Strong empirical results**: The paper demonstrates consistent improvements (average ~7% on pass@1) across multiple baselines and LLM backbones, with comprehensive evaluation including human studies showing statistical significance.\n\n4. **Thorough experimental analysis**: The paper includes extensive ablations, hyperparameter analysis, harmful content detection, and human evaluation with appropriate inter-rater agreement metrics.\n\n5. **Clear modular design**: The three-role framework provides good interpretability and allows for targeted improvements to individual components.", "weaknesses": "1. **Questionable humor quality in examples**: The showcased example \"HR says we can expense a cow now\" doesn't seem particularly funny, raising concerns about whether the quantitative improvements translate to genuinely humorous outputs. More compelling examples would strengthen the paper's claims.\n\n2. **Limited scope**: The method is only evaluated on cartoon caption generation. It's unclear whether the approach generalizes to other humor formats (jokes, puns, situational comedy, etc.).\n\n3. **Computational overhead**: The method requires multiple LLM calls (7 total per caption generation according to Table 7) \n\n5. **Limited human evaluation**: Only 12 raters participated\n\n6. **Complexity concerns**: The framework involves many components (WordNet for semantic relations, multiple scoring functions, GTVH knowledge resources) that may be difficult to reproduce or adapt.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0rJT4YVPdx", "reviewer": "Reviewer_SKPc", "summary": "This paper proposes a novel framework for humour generation based on the General Theory of Verbal Humour. This framework is composed of three modules: 1. conflicting scripts extraction to find the out-of-expectation target with both global and local views, 2) humour-relevance retrieval to construct the hierarchical imagination tree from their collected joke database, and 3) final script generation. The experimental results on two humor generation datasets validate the effectiveness of this framework.", "questions": "1. Line 38, are there other types of images for humour generation, besides cartoons?\n2. Figure 2, Where is the direct link between Coffee and Sugar (demonstrated in the Global view)?\nHow to decide the Coffee node rather than the Cup node? In this case, it should be Cups, as it could be coffee, tea, or other drinks. And the conflicting elements are the sizes of the cups. How to ensure that these terms make sense? Could you please explain the humour in case 1? As for me, case 2 is humorous. \n3. Term-2, It seems this equation would find those common words with a high humour-frequency score. What about trying the TF-IDF idea?\n4. Line 262, Are all the candidate targets utilised or integrated in the final caption? How are they connected with the final caption?\n5. Table 1, 6044, why are there so many captions?\n6. Table 2: How do you convert the global ranking into the pairwise ranking for the Human-in-AI dataset?\n7. line 290-291, How to calculate the win rate? Which competes against which?\n8. Human evaluation: How many data points are evaluated? They are not stated in the appendix, either.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Grounding on the GTVH humour theory, it generates humour from conflicting elements in a novel way. \n2. And considering both the similarity and the conceptual opposition. Utilising an external joke database for \"humour\" element retrieval.", "weaknesses": "1. Gaps between the figure illustration and the paragraphs:\n1.1. Figure 1 lists 1. visual understanding, 2. humour understanding, 3. humour imagination, and 4. stylistic expression, and they are marked with checks and crosses. This leads the reader to expect an assessment along these dimensions, while the study provides no qualitative or quantitative evaluation of them.\n\n2. Confusing Delivery\n2.1 abstract, line 13-17.\n2.2 The last line on page 1.\n2.3 line 100, \"The goal of tackling this task is to assess ...\"\n2.4 \"extract conflicting script\", e.g., line 103. Where are the scripts to extract? It appears that the authors are referring to the \"strange\" elements in the image, which contradict the norm.\n2.5 line 290-291, \"To assess the reliable win rate ...  use Pass rate (pass@K) to measure the win rate \"\n\n3. The paper's theoretical grounding, fundamental humour theory, GTVH, should be introduced more. Only line 139 mentions it.\n\n\ntypos\nline 82, \";\" should be '.'\nline 190, redundancy \")\" and \"}\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pAFnbdSyIP", "reviewer": "Reviewer_gAin", "summary": "This paper proposes a new framework for generating humorous image captions, called HOMER. It consists of three LLM-based components. First, the Extractor derives a general description from the image and identifies objects that contradict normal expectations as the basis for humor generation. Next, the Imaginator expands the humorous space both deeply and broadly by constructing imagination trees using humor-relevant retrieval, then pruning them based on humor-relevance scores. These results are finally fed into the Caption Generator to produce the desired humorous caption.", "questions": "- The quantitative experimental results are quite thorough, but the presentation of the generated humorous captions seems rather sparse. From the example shown in Figure 2, I feel that some of the generated captions don't quite match the original images—the humor may be too far-fetched. Could you provide more examples of the generated captions?\n- I have some doubts about the generalizability of your pipeline. If there are no particularly abnormal elements in the image, is this method still applicable? And how is the humor basis determined in such cases?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Compared to previous work, this paper breaks away from the inherent humor mechanisms of LLMs. Instead of enhancing the model's ability to generate humorous image captions through prompt engineering or task-specific fine-tuning, it innovatively designs three distinct modules to complete the humor caption generation progressively. I find the overall pipeline design to be reasonable and well-structured.\n- The design of the Imaginator is quite novel, as it takes into account both the depth and breadth of imagination. Moreover, the pruning strategy based on humor relevance scores is also fairly reasonable.", "weaknesses": "- The construction of the imagination tree relies too heavily on the joke database. The construction process of the joke database is not mentioned in the paper.\n- Maybe the range of data that this pipeline can handle is quite limited. (Details see Questions)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)