Culture In a Frame: C$^3$B as a Comic-Based Benchmark for Multimodal Culturally Awareness
Authors: Yuchen Song
Andong Chen
Wenxin Zhu
Kehai Chen
Xuefeng Bai
Muyun Yang
Tiejun Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Large Language Models
Cultural Awareness
Cross-Cultural Benchmark
Comics
Multilingual Evaluation
Multitask Evaluation
Primary area: datasets and benchmarks

Abstract: Cultural awareness capabilities have emerged as a critical capability for Multimodal Large Language Models (MLLMs). However, current benchmarks lack progressed difficulty in their task design and are deficient in cross-lingual tasks. Moreover, current benchmarks often use real-world images. Each real-world image typically contains one culture, making these benchmarks relatively easy for MLLMs. Based on this, we propose C$^3$B (\textbf{C}omics \textbf{C}ross-\textbf{C}ultural \textbf{B}enchmark), a novel multicultural, multitask and multilingual cultural awareness capabilities benchmark. C$^3$B comprises over 2000 images and over 18000 QA pairs, constructed on three tasks with progressed difficulties, from basic visual recognition to higher-level cultural conflict understanding, and finally to cultural content generation. We conducted evaluations on 11 open-source MLLMs, revealing a significant performance gap between MLLMs and human performance. The gap  demonstrates that C$^3$B poses substantial challenges for current MLLMs, encouraging future research to advance the cultural awareness capabilities of MLLMs.

Published as a conference paper at ICLR 2026
CULTURE IN A FRAME: B AS A COMIC-BASED BENCHMARK FOR MULTIMODAL CULTURALLY AWARENESS
Yuchen Song!. Andong Chen', Wenxin Zhu', Kehai Chen², Xuefeng Bai Muyun Yang'. Tiejun Zhao?.1
Harbin Institute of Technology, Harbin, China
Harbin Institute of Technology, Shenzhen, China
songyuchn@1 com ands6911 98gma 1 com, wenx i nzhu@stu wettetittttt...e...e.eeeeeeeeeet .ww.. [chenkehai , ba xuefeng, yangmuyun, tjzhao (achao)@hiie @hi edu.cd
ABSTRACT
Cultural awareness capabilities have emerged as a critical capability for Multi- modal Large Language Models (MLLMs). However, current benchmarks lack progressed difficulty in their task design and are deficient in cross-lingual tasks. Moreover, current benchmarks often real-world images. Each real-world im- age typically contains one culture, making these benchmarks relatively easy for MLLMs. Based on this, we propose C (Comics Cross- Cultural Benchmark), a novel multicultural, multitask and multilingual cultural awareness capabilities benchmark. C comprises structed on three tasks with progressed difficulties, from basic visual recognition higher-level cultural conflict understanding, and finally cultural content gen- eration. We conducted evaluations on 11 open-source MLLMs, revealing a sig- nificant performance gap between MILMs and human performance. The gap demonstrates that C poses substantial challenges for current MLLMs, encour- aging future research to advance the cultural awareness capabilities of MLLMs.
Cross-Cultural
2000 images and over 18000 QA pairs, con-
INTRODUCTION
( CVDA
Co giliendi
Choiow:
Degain seda
Multoulture
Maltiinual
Stngle form
When
Guestion:
#1 Da Aumt/DA
What cul cose bedlgroudd
Ouration
Tame ansner ooing caratio basac
Cholcas:
Mouse for viluto daking pegpetiieeeenn
Caseddoon
L.Ager Capan Norway
thees contact image pliasa covation
Answer St
That dbactcan cofore
(cc, (ereeerrrrr for taa parmos (Troipal
Nuticelars
monolngua
Single taak
Viauall Racogration
Culburrl Conntliit Understooniigg
Caltural Contant Generation
Figure Comparison between C and previous culture awareness capability benchmarks. In comparison with existing benchmarks for cultural awareness capabilities, CB is compatible with multicultural, multilingual, and multitask contexts, thereby facilitating more thorough evaluation.
Multimodal Large Language Models (MLLMs) have become more and more important in many as- pects of daily living, such machine translation (Chen et al., 2025), image captioning (Anantha Ra-
"These authors contributed equally.
Corresponning author

---

Published as a conference paper at ICLR 2026
makrishnan et al., 2025) and visual question answering (Huynh al., 2025). Users who interact with these models often discover such a situation: current models perform well in a reetrr- culture context but perform badly in non- Western culture contexts (Singh et al., 2025; Nayak et al., 2024; AlKhamissi et al., 2024; Burda-Lassen et al., 2025; Naous et al., 2024). This imhalance shows that MI.LMs need to improve cultural awareness capabilities which refers to capabilities of MILMS to understand and process cultural contexts (Pawar ct al., 2024).
Benchmarks are essential build MLLMS with strong cultural awareness capabilities (Cohen and Howe, 1988; Reuel et al., 2024). While existing benchmarks have laid foundational framework for evaluation, there remains room for improvement. From a multicultral perspective, existing bench- marks for evaluating cultural awareness capabilities of MLLMs mainly focus on real-world im- ages (Arora et al., 2025; Romero et al., 2024; Xu et al., 2025). single real-world image typically contains one culture, making these benchmarks relatively easy for MILMs. From multitask per- spective, most of these benchmarks include one question per data sample, which makes difficult to evaluate MLLMs across multiple dimensions. From a multilingual perspective, languages are carriers of cultural meaning (Kramsch, 2014). concept may have no direct equivalent expression across languages. Therefore, adding multi- lintii-ligual tasks to benchmarks will introduce appropriate complexity, enabling us to evaluate MLLMs more comprehensively.
To address these issues, we propose B (Comics Cross- Cooss-Cultural Benchmark), novel multicultural, multitask and multilingual cultural awareness capabilities benchmark, containing 2220 images and 18789 QA pairs. In contrast real-world images, select comies as the primary medium in our benchmark. Comics differ from real-world images: they often depict fictional scene. Real-world images often tied the specific, singular cultural contexts of real-life scenarios, but fictional scenes in comics are free from such constraints. This enables comics to condense numerous cultures into single frame, creating a more complex context, raising the bar for evaluation. CBB consists of tasks that form a logical chain. Based on difficulty levels, these tasks progress from basic visual recognition to higher-level cultural conflict understanding. and finally to cultural content generation. This multitask arrangement enables more comprehensive evaluation of ML. In cultural gener- ation task, CB incorporates languages (Japanese, Russian, Thai, English, and Spanish), reducing the limitations of previous monolingual benchmarks. The differences between C and previous cultural awareness capability benchmarks are presented in Figure 1.
In this study, we conduct a comprehensive evaluation of 11 open-source MLLMs B. The re- sults confirm the value of CBB, as they reveal a significant performance gap between MLLMs and human performance. Furthermore, the results provide critical insights into the current state of cul- tural awareness capabilities of MILMs. Specifically, MILMS should enhance their understanding of lesser-known cultures and their abilities process cultural conflicts. Our contributions can be summarized as follows:
1. We propose CB novel comic-centric, multicultural, multitask and multilingual cultural awareness capabilities benchmark.
2. CBB incorporates 3 tasks with escalating difficulty, evaluating cultural awareness capabili- ties of MLLMs comprehensively through progressively challenging tasks.
3. We benchmark 11 MILMs with C$B, which presents an initial set of evaluations on this benchmark, establishing a baseline for future research on MLLMS with strong cultural awareness capabilities.
2 RELATED WORK
2.1 MULTIMODAL LARGE LANGUAGE MODELS
Recent years have witnessed rapid advancements in Multimodal Large Language Models (MLLMs).
The multimodal capabilities of MLLMs enable large language models to address a broader range of tasks (Caffagni et al., 2024). Models like BLIP-2 (Li et al., 2023), LLaVA (Liu et al., 2023b; 2024: 2023a) and Qwen- VL (Bai et al., 2023) perform well in tasks such as visual question an- swering (Dong et al., 2024) and multimodal machine translation (Chen et al., 2025). Moreover, SormLM-XC2.. (Zhang et al., 2024) has good long contexoll input and output capabilities. which enables many advanced features, such high resolution image understanding. 1lama3.2 series
2

---

Published as a conference paper at ICLR 2026
models (Grattafiori al., 2024) are optimized for many multimodal tasks, becoming baseline for many methods.
2.2 MULTIMODAL BENCHMARK ON CULTURAL AWARENESS CAPABILITIES
Recent multimodal benchmarks on cultural awareness capabilities mainly focus on real-world im- ages to evaluate MLLMs. Among various recent benchmarks (Romero et al., 2024; Burda-Lassen al., 2025; Arora et al., 2025; Schneider et al., 2025; Yang et al., 2025), CVQA (Romero al., 2024) adopts multipl--choiee format to evaluate the cultural awareness capabilities of MLLMs using real-world images. Notably, CVQA covers 30 languages, enabling evaluation from a multi- lingual perspective. CVQA relies on single form of task, restricting its ability to comprehensively evaluate the cultural awareness capabilities of MLLMs across diverse interaction scenarios. Cultur- alVQA (Nayak et al., 2024) also evaluates MLLMs with real world images, covering 11 countries in 2378 images. More recently. GIMMICK (Schneider et al., 2025) incorporates six tasks for eval- uating the cultural awareness capabilities of MILMs. The images included still exhibit relatively low cultural density. Recently, GLOBALRG (Bhatia et al., 2024) has utilized images sourced from 50 countries and integrated visual grounding tasks benchmark the cultural awareness capabilities of MLLMs. However, this framework does not account for multilingual settings. Moreover, ALM- bench (Vayani et al., 2025) encompasses over 100 languages and 19 different cultural domains, yet the images contained ALM-bench are still real-world images. AlKhamissi et al. (2025) explores the development of more sophisticated cultural benchmarks, providing complementary lens for multimodal cultural reasoning. In parallel, Karamolegkou et al. (2024) introduces a culture-centric evaluation subset wherein a subset of images is annotated with multiple cultural associations. Over- all, B integrates the advantages of these benchmarks, forming multitask, multicultural, and multilingual benchmark.
2.3 MULTIMODAL BENCHMARKS ON COMICS
Multimodal benchmarks centered on comics primarily focus basic visual tasks. For Western-style comics, the eBDtheque dataset (Guérin et al., 2013) was the first publicly released comic dataset, featuring spatial and semantic annotations for 100 pages of Western comics. The COMICS dataset (lyyer et al., 2017) includes over 1.2 million comic panels. offering resources for future research. For Japanese-style comics, Manga109 (Matsui et al., 2016) comprises 21,142 comics pages, with primary focus multimedia applications. More recently, CoMix (Vivoli et al., 2024) integrates comics into new dataset but remains focused on visual tasks such as speaker identification and character naming.
Given the current landscape of multimodal benchmarks for cultural awareness capabilities and comics, propose CB The differences between C and previous works are presented in Table 1. From the table, we observe that benchmarks for evaluating cultural awareness capabilities have not simultaneously integrated multicultural images, multitask settings, and multilingual tasks. Ad- ditionally, none of them incorporate progressive difficulty tiers. Regarding comic-centered datasets, most are designed for cultural evaluation purposes.
Table 1: The difference between C and previous works. We analyze the works along three dimensions: whether the dataset is Multicultural for every image, Multitask for every data sample, Multilingual, whether its primary task is Cultural Awareness Capabilities and whether the tasks within have Progressed Difficulty.
Benchmarks
Multicultural Multitsk
Multilingual
Cultural Awareness Capabilities
Progressed Difficulty
CVQA (Romero al. 2024)
x
MOSAIC-I.S, (Bunda-Lassen 2025)
CoMix (Vivali et al,, 2024)
2024)
x
BDthequu (Guétin et 2013)
X
x
Manga (Malsui 2016)
X
CultualVQA
GIMMICK (Schneider et 2025)
CBB
3

---

Published as a conference paper at ICLR 2026
3 B: COMICS CROSS-CULTURS UNDERSTANDING BENCHMARK
B is a novel comic-centric benchmark designed to comprehensively evaluate MLLMs' cultural awareness capabilities. C features multicultural diversity, multitask setting and multilin- gual coverage. To comprehensively evaluate MLLMs, C incorporates three tasks with increasing difficulty (detailed in Section 3.1). For data construction (detailed in Section 3.2), we design multi- agent method to create culturally rich comics and their annotations, ensuring both efficiency and quality of images. In Section 3.3, some necessary statisties of CB is presented. An overview of CB is presented in Figure 2. Some data samples of C is presented in Appendix B.
3.1 TASKS
design tasks with escalating difficulty to evaluate different dimensions of cultural awareness capabilities of MLLMs. The first task, Culture-aware Object Extraction (Extraction @ Culture), eval- uates the visual recognition and basic cultural understanding capabilities of MLLMs. The second task, Cultural-conflict Object Detection (Conflict Culture), focuses evaluating their abilities to understand cultural conflicts. The third task, Culturally-aligned Content Generation (Genera- tion @ Culture), measures their multilingual generation capabilities when provided multimodal cul- tural contexts. The question template for each task is presented in Appendix A.
C3B
Open-ender ended
Images: 2220 QA Pairs: 18780
Hard
Plaase translate fallowing senterce from Japanien to
Mulllask Mulliouitre Mutiiigual
It's sill not enough
Outhuralallnedd Goraration
Q5: Machine Translation
02: Culture aware Objects Detestion
Description conflicts
What eojest can reflect cortain ou lare
Mork Americar the
Se chooolation should be
Level2: Culuralccnnictt Detection
Q3: Identification cultural conflicts
Q1: Background Culture idontiiiction
Whic
lulu doss be X7?
Arr budid in 3ha magg??
Easy
Dose-erded
ChRi
Levelt:
Figure 2: Overview of C. C evaluates MILMs across three dimensions: Object Identification (foundational vision capability based on culture), Conflict Identification (cultural conflict under- standing), and Culturally-aligneed Content Generation (comprehensive cultural generation). Culture-aware Objects Extraction (Extraction @ Culture): This task requires the MILMM not only to identify objects within the image but also to understand whether or not the object is related to specific cultural context. In this task, set two questions:
Question 1 (Q1): Background Culture Identification We task MLLMs with identifying the background culture of comic pages. Multiple valid answers may exist for single question.
2. Question (Q2): Culture-aware Objects Detection this question, MLLMs are re- quired to identify all culturally representative objects in the image. We set up multiple options, with each option including several culturally representative objects. The MLLM needs to choose option that contains exactly all the objects present in the image.
Cultural-conffict Objects Detection (Conflict@ Culture): This task is designed to evaluate MLLMs' capabilities in identifying culturally conflicts within comics. In CBB, if two distinct cul- tures are depicted within a single image, the instance is classified as a cultural conflict. In this task, also questions:
4

---

Published as a conference paper at ICLR 2026
1. Question 3 (Q3): Identification of cultural confficts In this question, MLLMs must de- termine whether there is cultural conflict the presented image.
2. Question 4 (Q4): Deseription culture conflicts When cultural conflict is detected, MLLM is subsequently required to specify which objects of the answer to Q2 contradict the culture to the answer of QI.
Culturally-aligned Content Generation (Generaion @ Culture): This task is designed evalu- ate the cultural generation capabilities of MLLMs. As for the specific task form, we select ma- chine translation. We have set up translation tasks for the language pairs of Sapanese-English (JA- EN), -aaaaaaaaeeuusseee (JA-RU), sapanese-Germal (JA-DE), Japanese- -haan (JA-TH), and Japanese- Spanish (JA-ES). These language pairs roughly cover languages from various continents, enabling comprehensive evaluation of the overall translation capabilities of MLLMs.
3.2 CONSTRUCTION OF B
The construction process of C comprises two parts: image collection and annotation. An overview of the entire process is presented Figure 3.
Image Collection: For task Extraction @ Culture and @onflict@ ContuictCCuttre,, we design a comic gener- ation pipeline with the help doubao APIs'. The pipeline consists of key stages: (1) prompt generation to specify cultural conflict scenarios, followed by (2) image creation based on gener- ated prompts. This process illustrated in Figure 3a. Subsequent prompt generation, manual verification is performed on the generated prompts to assess the presence of harmful content.
For the Generation @ Culture task, we source images from Manga109 (Matsui et al., 2016). We manually selected 197 comics images that are closely related to culture and contain more culturally representative objects as well as cultural conflicts.
(a)
Plesse culu contlictss lis fime
that
All Cutture China, Japan, Thallnd, Cultural Objects Katana, Toahousee
Output
fourad in Hiecstann grea siert patss inoo Mizoi
Stap1: Prompt goner adon
Stop2: Generata comic pages
Stepd: Get all outures and objects
(c) Arnotation for Extratoon@Cutture and ConticralCutture
Stept: Get cultures and objecte the
Build COE OA Pair
Cutture List: Native America, Chira
Cuttural Objects List: Natho Ameriaa Hai...
Ctena
Ineland
Cholces:
D2 Choices: (Nathe Amorica Hat...! -]
All Cutture List: Chins,
All Cuhural Objects List: Katana, Teshouse,
Input
dlomett
Refined Answers Nativo America Hat should not China.
contlida with the cutture cullame het...
conticti detection
Output
kualvv Amnrics Hat shouldd non be Amerrcca,
Step7: Answer Refinement
(c) Ancotation for Generatian&Cuuhae
Stepli: Multieget Ground-Iruhh tranolation generation
Translation:
13te bicod..
Step9: Transiation Refinement
Final Transiation +ill you, wont... (1 just lile your is all..
Bource Sentence:
Advice: natural and common ray.
And Translation
latt S......-.L.B Matt 137 .....
Translation
Transtator
Revinwer
Figure 3: The construction process of CB The process contains steps: Comics Generation, Annotation for Extraction@: Culture and Conflict@ Culture and Annotation for Generation @ Culture.
Annotation: The annotation for the tasks of Extraction @ Culture and @onflict@Culll refers the process of QA pairs creation (illustrated in Figure 3b). For the task Extraction @ Culture, first
https:/wwwwwoolcnngnncccoom

---

Published as a conference paper at ICLR 2026
manually compiled two lists: containing all distinct cultures present in the comic pages, and the other listing all culturally representative objects. Next, we manually created QA pairs for each comic page. For QI, we identified the cultures presented in given image, then randomly selected additional cultures from the precompiled culture list to form a total of 5 options. For Q2, we first manually created a list of all culturally representative objects in the image, then generated 5 options by randomly modifying this list by either adding object, deleting object, or deleting 2 objects.
For the task Conflict@ Culture, we implemented an annotation pipeline consisting of two stages with the help of Deepseek-V3 V3 (DeepSeek-A) AI, 2024) (illustrated in Figure 3c). The prompt setting is presented in Appendix C.I. The details of the pipeline is:
1. Automated Cultural Conflict Detection: Given the background cultures and culturally representative objects in the comics page, Deepseek will analyze each object identify if the object conflicts with one of the background cultures. After this, it generates either formatted conflict description or "No".
2. Manual Verification and Correction: We manually inspect all generated results, focusing primarily on verifying whether the model misjudged the existence of conflicts. Subsequent to this, we check if the generated formatted conflict descriptions contained formatting in- consistencies or culture-related inaccuracies. The examples of these two kinds of errors are presented in Appendix D.
For the task Generation Culture, annotation refers the process of creating ground-truth transla- tions. We design a multi-agent process to generate ground-truth translations, which involves two specialized agents: (1) a Translator responsible for generating translations, and (2) Reviewer re- sponsible for checking the translation based on the input sentence and giving suggestions. The Translator first provides a rough translation based on the extracted dialog from Manga109, the Reviewer regards the translation as good translation, the translation. Otherwise, the Reviewer will examine three specific types of potential errors: con- textual inconsistencies, basic translation errors, and related inaccuracies. The suggestions generated by the Reviewer are subsequently integrated into the prompt. The Translator then uses produce new translation. In the annotation process, we employed DeepSeek-V3 V3 the base model, with the prompt used presented C.2. More details on annotation are presented in E.
will conduct manual verification to finalize
3.3
DATA STATISTICS
To ensure comprehensive evaluation, BB includes total of 2,220 images and more than 18000 QA pairs. An overall statistics of CB is presented in Table 2.
Table 2: Statistics of C
Statistic
Number
Images
2220
Exiraction Culturek ConfticreCuutuure
1024
Generation Calture
1197
Japsneso/Westem Style Images
1697/523
QA Pairs
18789
Extraction Culture
1023
Conflict& Culture
1023
Generaaion
16743
Figure 4: The cultures CB covers are pre- sented in a world map. Regions shaded in blue indicate that the culture is included in BB
No. All Calture Covered
77
Min//aa Possible Answers of Extrostion&Cullure
1/5
In task Extraction @Cultrre, there are 1023 images. Each image corresponds to one QA pair. In Q1, the number of correct numbers ranges from to 5 among 5 candidate options. This variability increases the difficulty for MLLMs to process the task.
In task Conflict @ Culture, the images used are the same task Extraction @ @Culture, covering both Japanese comics and American comic styles. This design choice is intentional because the Ex- traction @ Culture and @onflict@ Culture tasks evaluate the visual signal processing capabilities of

---

Published as a conference paper at ICLR 2026
MLLMs. In these two tasks, introducing comics of different styles would make the process more complicated and challenging.
In task Generation @ Culture, cach QA pair consists of a source sentence and its ground-truth transla- tion 5 languages. All 16743 QA pairs cover 77 distinct cultures (visualized in Figure 4), enabling comprehensively benchmark the understanding different cultures of MLLMs.
4 BB EVALUATION ON EXISTING LLMS
Models: To evaluate performance of MLLMs on our C benchmark, we choose 11 open-source MLLMs. Specifically, we select SPHINX (Lin al., 2023), Monkey (Li et al., 2024), MiniGPT. v2 (Chen et al., 2023), mPLUG-OW13 (Ye et al., 2024), LLaVA family models (Liu et al., 2023b; 2024; 2023a), IntemLLM-XC2.5 (Zhang et al., 2024), Llama3.2 (Grattafiori et al., 2024), Qwen2.5- VL (Bai et al., 2025) and Intern VL2 (Chen et al., 2024).
Metries:
For task Extraction Culture and ConflictCCulturac we use accuracy (ACC) as the eval- uation metric. For Q4, because it is constructed based on models' answers Q1 and Q2, we have designed composite ACC metric CACC:
(1) where a, b, e represent preset weighting parameters, ACC(Q,) denotes the accuracy score for Ques- tion i. These weighting parameters quantify the respective contributions the final evaluation. In our experiments, the values a, b, c are 0.3, 0.3 and 0.4. QI and Q2 contribute approximately equally to Q4, while Q4 should dominate. To simultaneously consider the contributions of the three tasks, we set the hyperparameters in this way.
CACC(Q,) = x ACC(Q1) + ACC(Q2) + c ACC(Q,)
For Generation @ Culture, adopt the conventional metric BLEU (Papineni et al., 2002), supple- mented by COMET (Rei et al., 2022) and BLEURT (Sellam et al., 2020) to align with current standards in L..M-based translation research.
Evaluation Settings All evaluations were conducted on a Ubuntu server equipped with an H20- NVLink GPU featuring 96GB of memory. We adhered strictly the official inference example codes provided by the respective model developers.
5 EXPERIMENT RESULTS
MAIN RESULTS FOR TASK EXTRACTION CULTURE AND CONFLICT CULTURE
Table 3: Main Results of task Extraction @ Culture and @onflict@ Culture in C For task Con- flict@ Culture, two types accuracy metrics are used to evaluate model performance. Specifically, Q4 is constructed based on the answer of MLLM to QI and Q2. The calculation process of CACC of Q4 considers the answers to QI and Q2. In contrast, ACC refers to the accuracy Q4 calculated without referencing answers to Q1 and Q2. The highest performance is marked in bold.
Methods
Extraction Culture
Conflict@ Culture
Conflict@ (Q4)
Q1 (ACC)
Q2 (ACC)
Q3 (ACC)
ACC
CACC
SPHINX
29,9
5.28
69.2
0.16
10.6
Monkey
28.4
5.08
33.0
0.19
10.1
MiniGPT-22
18.0
6.84
66.3
0.51
7.7
mPLUG-OW13
24.9
15.2
30.8
2.03
12.8
L.LVAI.5-7B
32.5
2.93
56.3
0.00
10.6
L.AVA-NeXT
16.5
39.8
0.88
0.00
16.9
LLaVA-OV
39.0
32.4
53.1
3.87
23.0
InternLM-XC225
46.0
50.9
68.5
1.94
29.8
Llama3.2
46.0
59.0
44.9
2.76
32.6
Qwen2.5-VI.
53.7
55.9
63.1
3.20
34,2
InternVL.
46.0
50.9
68.5
0.01
29.1
Table presents the results for task Extraction @eneettionnCulln Culture and Conffict@Culture. Qwen2.5-Vi demonstrates optimal performance, with Q1 outperforming the second-ranked models (InternL
7

---

Published as a conference paper at ICLR 2026
XC2.5, Llama3.2 2. and InternVL2) by 7.7 points and Q4 achieving 1.6-point lead over the second- ranked model (Llama3.2) 2).. We also find that all models achieve extremely low ACC in Q4 due to the influence from Q1 and Q2.
As for task Extraction @ Culture, in QI, LLaVA-NeXT performs the worst, because tends to de- scribe the image rather than answer the question directly. We name this phenomenon "Turn-a- deaf-ear" (shown in Appendix F.1). ). In Q2, Llama3.2 performs the best, exceeding average perfor- mance by 72.7%. In this task, LLaVA1.5-7B performs the worst, because it keeps answering "A". We name this failure pattern as "Taak---shot-in--th--aar and an example is shown in Appendix F.2.
As for task Conflict@ Culture, in Q3, SPHINX performs the best, exceeding the second-ranked model (InterLL M-X2.5 and InternVL2) by 0.7 points. In Q4, we find that models from the LLaVA series yield considerably lower results. Specifically, both LLa and LLaVA-NeXT achieves 0.00 ACC. Through case study (detailed in Appendix F.3), we find that, LLaVA-NeXX persistently outputs "Nothing". indicating a lack of cultural conflict comprehension capability, while LLaVA1.5. 7B cannot follow instructions properly, and we name this phenomenon "stubbornness".
5.2 MAIN RESULTS FOR TASK GENERATION @ CULTURE
Table 4 shows the results for task Generation @ Culture. The results show that Qwen- 2.5-VL demon- strates the most robust multilingual capabilities, as outperforms other models across all cultural generation tasks. In contrast, MiniGPT-v2 exhibits notably poor performance. Specifically, it achieves a BLEU score of 0 in most tasks. This underperformance can be attributed to the weak cotructioo-followinng ability, as presented in Appendix F.4. This error pattern might be caused by the poor understanding capability of comic pages. Additionally, LLaVA-NeXT tends to repeat the source sentence when handling the tasks except JA-EN. Among all tasks, the performance of all models in JA-TH is the poorest while they all perform the best in JA-EN tasks. These results indi- cate that the multilingual capabilities when given cultural contexts needs to be enhanced, and greater support for low-resource languages should also be strengthened.
Table 4: Main Result of task Generation @ Culture. The result is presented in the format BLEU/CCMET/BBEURY The highest performance marked in bold.
Methods
JA-EN
JA-DE
JA-RU
JA-ES
JA-TH
SPHINX
2/24..
1.12/54.3/33.1
0.27/45.0/19.
66255.
0.02/38.7/16.
Monkey
3.88/62.0/433
1.62/55.5/33.5
0.73/48.2/17.7
2.81/56.3/29.3
0.63/50.2/18.0
MiniGPT-v2
5/3309
1/15//
0,00/30.0/13.9
0.00/36.5/21.6
0.00/32.2/14.5
mPLUG-OWL3
6.21/58.4/39.
4.97/56.0/36.8
3.97/57.6/33..5
4.75/55.7/34.6
2.02/51.9/25.2
LLaVAI.5-7B
5.94/62.0/41.0
2.74/54.2/29.7
1.39/53.8/23.
4.23/62.2/38.4
1.1//449/10.2
LLaVA-NeXT
2.84/
0.00/46.4/10.5
0.14/418/111
0.00/48.7/6.87
0.28/42. 1/12.5
LLAVA-OV
000/58.3/35.2
3.37/49.6/24.7
2.78/49 2.78/0
3.24/53.4/26.5
3.77/42.8/13.8
InternLM-XC25
4.99/66.6/49)
108/59.2/42.0
1.90/63 6/40.6
2.33/65.3/44.7
0.53/53.7/211
Llama3.2
5.05/54.6/35.2
4.27/51.3/11.8
170/47.0/20.1
5.73/55.0/31.1
0.99/46.9/17.4
Qwen2.5-VI.
13.2/70.9/53.8
12.0/67.7/52.1
8/74.5
14.5/72.0/54.3
9.72/67.8/42.2
InternVL2
7,20/66.6/47.6
3.52/58.5/40.4
3,33/3.8/440
5.32/66.8/46.9
0.74/45.3/24.5
5.3 IMPACT FOR Q4 FROM QI AND Q2
In C3B, answering Q4 requires models correctly answer Q1 and Q2. To evaluate the influence of QI and Q2 on Q4, we first calculate the correlation coefficients (Formula 2) between Q1 and Q4 and between Q2 and Q4, which yielded values of 0.56 and 0.51, respectively. The result demonstrates that QI and Q2 exhibit moderate correlations with Q4.
R(Q., Q+) Cov(ACC(()),ACC(O..)
(2)
/Var(ACC(Q)))ar((cc(Q..)
where R represents the correlation coefficient between two questions, Cov refers to the covariance, Var refers to the variance, and ACC(Q,) is the accuracy value of Question
8

---

Published as a conference paper at ICLR 2026
Table 5: Performance of Q4 when answers Q1 and/or Q2 are omitted. Q1/Q2 Answer means that only Q1/Q2 answer is provided. Q1&Q2 Answer means both Q1 and Q2 answers are pro- vided.
Moreover, conducted an ablation study Q1 and Q2. We added the answers to Q1/Q2 into the prompt template of Q4 to observe the impact model performance. CACC represents the average performance of all 11 MILMs, and the results are presented in Table 5. The results indicate that incorporating the QI answer into the Q4 prompt will enhance performance (+0.003), but adding the Q2 answer will change the perfor- mance. The greatest performance boost is observed when both QI and Q2 answers are included (+0.533). Although Q1 and Q2 show moderate correlations with Q4 (0.56 and 0.51), the ab- lation results reveal only marginal improvement when adding Q1/Q2 answers. This suggests that the positive correlations mainly reflect an intrinsic consistency across related questions, while the direct prompt incorporation of Q1/Q2 answers is not always effectively leveraged by MILMs.
Method
CACC
Base Prompt
19.231
QQ Answer
19.234
+ Q2 Answer
19.231
+ QI&Q2 Answer
19.764
5.4 GENERATED COMIC PAGES ANALYSIS
To evaluate B's cultural diversity, we compute three measuresCulture Density Per Image (CDP), Density (CAD)and compare them against
Cultural Breadth Intensity (CBI), and Coverage-Adjuse cultural QA datasets. CDPI is defined as the quantification of the average number of distinet cul- tures depicted within an image. CBI is designed to measure both cultural density and the total num- ber of cultures depicted in an image. CAD incorporates log-scaling to alleviate excessive rewards arising from large cultural lists. The CDPI, CBI and CAD of a dataset are presented in Equation 3. 4 and 5. In the equations D represents a complete image dataset, which { Ig,.... In). CultureInImage(44 43) denotes the number of cultures in the image I,, Neutturss refers to the number of cultures occurring in the dataset and [D] as the cardinality of D.
a
CDPI(D) D
CultureIn (mmaue)))
(3)
CBI(D) CDPI(D) x Nenltures CAD(D) DD) x logz (Noutturs + 1)
(4)
(5)
The results presented in Table 6, indicating that C shows significantly higher cultural diversity compared to similar datasets. This highlights its improved ability to eval- uate cultural awareness capabilities of MLLMs.
Table 6: culture-related datasets.
Statistics of different
Dataset
CDPI
CBI
CAD
CVQA
1.00
4.91
5.5 SCORES OF DIFFERENT CULTURE
CulturalVQA
1.00
3.58
MOSAIC-L.5
1.00
43
5.46
To examine how well MLLMs can recognize diverse cul- tures, we conducted culture-specific evaluation of models'
GIMMICK
1.00
144
7.18
C
2.28
175.56
14.29
correctness to mono-culture QA task (QI). The results, visualized in Figure 5, indicate that the per- formance of MLLMs differs substantially across cultural groups. In detail, representative cultures (e.g., Cambodia and Japan) are reliably identified, while lesser-known cultures (e.g., Finnish and Somalia) exhibit notably higher error rates. The results show that the cultural awareness capabilities of MLLMs for lesser-known cultures should be enhanced.
03-
02-
01
80
Tuno
ge
Figure 5: Model Accuracy on Q1 by Culture.

---

Published as a conference paper at ICLR 2026
5.6 HUMAN-MODEL ANALYSIS
Table 7: Result of Human-Model Analysis. Q4 is con- structed based on the answer of MLLM to Q1 and Q2. The calculation process of ACC of Q4 considers the answers to Q1 and Q2. In contrast, ACC refers to the accuracy of Q4 calculated without referencing models' answers to Q1 and Q2. IRA stands for Inter-rater Agreement Score. PTA stands for Per-tier Time-to-answer.
To validate the effectiveness of CBB, conducted human evaluation (more detail in Appendix G). First, We categorized task difficulties based on the number of distinct cultures in each image: (easy), 2/3 (medium), and 4/5 (hard). This tiered classi- fication allows evaluation across dif- ferent complexity levels. Next, we randomly selected 100 questions per tier. For each tier, calculated two metrics: the average accuracy performance of human and MILMs. The results are presented in Table 7. Three graduate student volun- teers participated in the experiment, with each completing all the ques- tions. We then calculated their average accuracy to quantify their performance.
Method
Q1
Q2
Q3
Q4
IRA PTAIScconds)
ACC
ACC
CACC
Eas Questions
Human Evalusation
92.0
77.0
100.0
60.0
74.7
95%
27.01
Model Performance
48.0
28.8
50.5
1.0
23.8
Ouestions
Human Evaluation Model Per Verformance
78.0
60.0
100.0
45.0
59.1
91%
$2.92
32.6
27.1
54.0
1.69
18.6
Hard Quevtions
Human Evaluaatio
50.0
93%
Model Performance
25.7
25.5
41.7
1.27
15.9
We find that the human performance significantly better than that of MLLMs, particularly in Q3, where the human performance achieves all 100% accuracy. Constrained by the performance in Q1 and Q2, the human ACC result relatively low, yet it still exceeds the MILMs' performance. The results show that C is challenging for MLLMs and the cultural awareness capabilities of MLLMs need to be improved.
CONCLUSION
We propose C a novel comic-based, multicultural, multitask, and multilingual cultural awareness capabilities benchmark. Applying comics as the core medium, we enable comprehensive evaluation of cultural awareness capabilities of MLLMs, with the dataset encompassing large number of im- ages, QA pairs, and cultural contexts. C contains three tasks with progressed difficulties, from basic visual recognition to higher-level cultural conflict understanding, and finally to cultural content generation. We benchmarked 11 open-source MLLMs on C revealing significant performance gap between MLLMs and human performance. Specifically, current MLLMs lack proficiency in understanding less well-known cultures and processing cultural conflicts, highlighting areas for im- provement. We anticipate that C²B will serve as a critical tool to support and advance research on MLLMs' cultural awareness capabilities.
7 ACKNOWLEDGEMENTS
We want to thank all the anonymous reviewers for their valuable comments. The work was supported by the National Natural Science Foundation of China (62376075, 62276077, 62406091, 62350710797 and U23B2055), Guangdong Basic and Applied Basic Research Foun- dation (2024A151501 1205, 2026A 151501 1718888) and Shenzhen Science and Technology Program (KQTID20240729102154466 and 2DSSSS202306660012222000
ETHICS STATEMENT
Our BB benchmark incorporates images created by Doubao. Through a rigorous human annota- tion and verification process (Appendix E), we have made every effort to mitigate the majority of cultural biases and stereotypes. Prior to the creation of images, and removed harmful content, thereby ensuring the high quality of the created images.
manually checked all prompts
Regarding the definition of cultural conflict, we have explicitly clarified that the cultural conflict in our task refers to co-occurrence conflict rather than aggressive conflict. Our work is not intended to cause any harm individuals or groups
10

---

Published as a conference paper at ICLR 2026
REFERENCES
B. AlKhamissi, M. EINokrashy, M. Alkhamissi, and M. Diab. Investigating cultural alignment of large language models. In L.-W. Ku, A. Martins, and V. Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Pupers), pages 12404-12422, Bangkok, Thailand, Aug. 2024. Association for Computational Linguistics, Linguuttiss doi: 10. 10018863//1/20222.cllooog https: http://aaaannnt hwt.s/hoolaaatooooooooooo orgg 2024 1a.4..ac--20 -long..o..... 671/.
URL
M. AlKhamissi, Y. Xiao, B. AlKhamissi, and M. Diah, Hire your anthropologist rethinking culture benchmarks through an anthropological lens, 2025. URL https: / aarxiv.i org/ abs
A. Anantha Ramakrishnan, A. A. Ramakrishnan, and D. Lee. RONA: Pragmatically diverse image caption- ing with coherence relations. In V. Padmakumar, K. Gero, T. Wambsganss, S. Sterman, T.-H. Huang, D. Zhou, and Chung, editors, Proceedings of the Fourth Workshop on Intelligent and Interactive Writ- ing Assistants (In2Writing 2025). pages 74-86, 86, Albuquerque. New Mexico, US. May 2025. Association for Computational Linguisties. ISBN 979-8-88 8-89176-929----1
doi: 18.18653/v 1/tp:///2225...22 in2writingn 1.8. URL
https: // ho org/ 2025 in2writ ing-1.8/.
S. Arora, M. Karpinska, H.-T. Chen, I. Bhattacharjee. M. lyyer, and E. Choi. CaLMQA: Exploring cul- turally specific long-form question answering across 23 languages. In W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), pages 11772-111 118177 Vienna, Austria, July 2025. Associa- tion for Computational Linguistics. ISBN 979-8-89176-251-0 doi: 10. 12116665///202555-oooooo https: : / anthol logy- .rgg 2025 act 1 ongg 5578/.
URL.
J. Bai, S. Bai, S. Yang, S. Wang, S. Tan, P. Wang, J. Lin, C. Zhou, and J. Zhou. Qwen-vl: A versatile vision- language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308. /2966, 2023.
S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang. S. Wang, J. Tang, H. Zhong, Y. Zhu, M. Yang, Z. Li, Wan, Wang, W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and Lin. Qwen2.5-vl technical report. arXiv preprint 2rXhv:2502.1392 2025.
M. Bhatia, S. Ravi, A. Chinchure, F. Hwang, and V. Shwartz. From local concepts universals: Evaluating the multicultural understanding of vision-language models. In Y. Al-Onaizan, M. Bansal, and Y.-N. Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6763-6782, Miami, Florida, USA. Nov. 2024. Association for Computational Linguistics. doi: 10.18653/ v1/2024.emnlp m85. URL https: //tc///aaaaaa awwaantnnnttoooooooooooo org/ 2024. momm-.mmmnnne 385/.
O. Burda-Lassen, A. Chadha, S. Goswami, and V. Jain. How culturally aware are vision-language models?, 2025. URL https: / arxiv www.iivww...vvoooooooooooooeeeee abs / 20002 11.05..77777 17755.
D. Caffagni, F. Cocchi, L. Barsellott. N. Moratelli. S. Sarto, L. Baraldi, L. Baraldi, M. Cornia, and R. Cucchiara. The revolution of multimodal large language models: A survey. In L.-W. Ku, A. Martins, and V. Srikumar, ed- itors, Findings of the Association for Computational Linguistics: ACL 2024, pages 35500-1318, Bangkok, Thailand, Aug. 2024. Association for Computational Linguistics. doi: 10.18633/11/2224 findii............gggss... 807. URL https: / aaw.annnchhoooooooo
org/ 2024. findings-act 807/.
A. Chen, Y. Song, K. Chen, X. Bai, M. Yang, L. Nie, J. Liu, T. Zhao, and M. Zhang. Make imagination clearer! stable diffusion-basse based visual imagination for multimodal machine translation. In W. Che. J. Nabende. E. Shutova, and M. T. Pilchvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 26567-26583, Vienna, Austria, July 2025. As- sociation for Computational Linguistics. ISBN 979-8-89176-251- doi: 10. 12255////2025..clllls 1289. URL https: ac www.annnooooooooooooooccoc wwwpoog..o...ooooooooo.oo.ooo..e 201//02555.0002 acl long 1289/.
J. Chen, D. Zhu, X. Shen, X. Li, 7. Liu, P. Zhang, R. Krishnamoorthi, V. Chandra, Y. Xiong, and M. Elhoseiny. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning, 2023. URI. https: / / /arxiv.iiivv arggabs//// 2310 09478.
Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic vinual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185-24198, 2024.
P. R. Cohen and A. E. Howe. How evaluation guides ai research: The message still counts more than the medium. Al Magazine, 94))35,, Dec. 1988. doi: 10.160%/iima..v944.95 URL https://ojee /ttps:///ossaaaaeeee aaai. org/ atrs/aimaazzinee index. pttdee.ppp//pppe
DeepSeck-AT. Deepseek-v3 technical report, 2024. URL
194377.
11

---

Published as a conference paper at ICLR 2026
DeepSeck-AL. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025. URL https://arxiv: www.abs.///oooooooo.o.oooo..aeas 2501. 12944. 12948.
R. Dong, C. Han, Y. Peng, Z. Qi, Z. Ge, J. Yang, L. Zhao, J. Sun, H. Zhou, H. Wei, X. Kong, X. Zhang, K. Ma, and L. Yi. DreamLLM: Synergistic multimoaal comprehension and creation. In The Twelfth Inter- national Conference on Learning Representations, 2024. URL https: oppnrreeeiiiieeeeeee net/ / forum? id-y0 KGvd9Bw.
A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, A. Yang, A. Fan, A. Goyal, A. Hartshorn, A. Yang, A. Mitra, A. Sravankumar, A. Korenev, A. Hinsvark, A. Rao, A. Zhang. A. Rodriguez, A. Gregerson, A. Spataru. B. Roziere, B. Biron, B. Tang. B. Chern, C. Caucheteux, C. Nayak. C. Bi, C. Marra, C. McConnell, C. Keller, C. Touret, C. Wu, C. Wong. C. C. Ferrer. C. Nikolaidis. D. Allonsius, D. Song. D. Pintz, D. Livshits, D. Wyatt, D. Esiobu, D. Choudhary. D. Mahajan, D. Garcia-Olano, D. Perino, D. Hupkes, E. Lakomkin, E. AlBadawy. E. Lobanova, E. Di- Zhang, G. Synnaeve, G. Lce, G. L. Anderson, G. Thattai, G. Nail, G. Mialon, G. Pang, G. Cucurell, H. Nguyen, H. Korevaar, H. Xu, H. Touvron, I. Zarov, I. A. Ibarra, I. Kloumann, 1. Misra, I. Evtimov, J. Zhang. J. Copet, J. Lee, J. Geffert, J. Vranes, J. Park, J. Ma- hadeokar, J. Shah, J. van der Linde, J. Billock, J. Hong, J. Lee, J. Fu, J. Chi, J. Huang. J. Liu, J. Wang. Yu, J. Bitton, J. Spisak, J. Park, J. Rocca, Johnstun, J. Saxe, J. Jia, K. V. Alwala, K. Prasad, K. Upasani, K. Plawiak, K. Li, K. Heafield, K. Stone, K. El-Arini, K. Iyer, K. Malik, K. Chiu, K. Bhalla, K. Lakho- tia, L. Rantala Rantala-Yeary, L. van der Maaten, L. Chen, L. Tan, L. Jenkins, L. Martin, L. Madaan, L. Malo, L. Blecher, L. Landzaat, L. de Oliveira, M. Muzzi, M. Pasupuleti. M. Singh, M. Paluri, M. Kardas, M. Tsim- poukelli, M. Oldham, M. Rita, M. Pavlova. M. Kambadur. M. Lewis, M. SL. M. K. Singh, M. Hassan, N. Goyyal, N. Torabi, N. Bashlykov, N. Bogoychev, N. Chatterji, N. Zhang. O. Duchenne, O. Çelebi, P. AI- rassy, P. Zhang, Ii, P. Vasie, P. Weng, P. Bhargava,
nan, E. M. Smith, F. Radenovic, F. Guzmán,
Dubal, Krishnan, P. S. Koura, P. Xu, Q. He, Q. Dong, R. Srinivasan, R. Ganapathy, R. Calderer, R. S. Cabral, R. Stojnic, R. Raileanu, R. Maheswari, R. Girdhar, R. Patel, R. Sauvestre, R. Polidoro, R Sumbaly. R. Taylor, R. Silva, R. Hou, R. Wang. S. Hos- seini, S. Chennabasappa. S. Singh, S. Bell, S. S. Kim, S. Edunov, S. Nie, S. Narang, S. Raparthy, S. Shen, S. Wan. S. Bhosale, S. Zhang. S. Vandenhende, Batra, Whitman, S. Sootla, S. Collot, S. Gururangan. S. Borodinsky, T. Herman, T. Fowler, T. Sheasha, T. Georgiou, T. Scialom, T. Speckbacher, T. Mihaylov, T. Xiao, U. Karn, V. Goswami, V. Gupta, V. Ramanathan, V. Kerkez, V. Gonguet, V. Do, V. Vogeti, V. Albiero, V. Petrovic, W. Chu, W. Xiong. W. Fu, W. Meers, X. Martinet, X. Wang, X. Wang. X. E. Tan. X. Xia, X. Xie, X. Jia, X. Wang. Y. Goldschlag, Y. Gaur, Y. Babaei, Y. Wen, Y. Song, Y. Zhang. Y. Li. Y. Mao, Z. D. Coudert, Z. Yan, Z. Chen, Z. Papakipos, A. Singh, A. Srivastava, A. Jain, A. Kelsey, A. Shajnfeld, A. Gangidi, A. Vic- toria, A. Goldstand, A. Menon, A. Sharma, A. Boesenberg, A. Baevski, A. Feinstein, A. Kallet, A. Sangani, A. Teo, A. Yunus, A. Lupu, A. Alvarado, A. Caples, A. Gu, A. Ho, A. Poulton, A. Ryan, A. Ramchandani, A. Dong. A. Franco, A. Goyal, A. Saraf, A. Chowdhury, A. Gabriel, A. Bharambe, A. Eisenman, A. Yazdan, B. James, B. Maurer, B. Leonhardi. B. Huang. B. Loyd, B. D. Paola, B. Paranjape, B. Liu, B. Wu, B. Ni. B. Hancock, B. Wasti, B. Spence, B. Stojkovic, B. Gamido, B. Montalvo, C. Parker, C. Burton, C. Mejia, C. Liu, C. Wang, C. Kim, C. Zhou, C. Hu, C.-H. Chu, C. Cai, C. Tindal, C. Feichtenhofer, C. Gao, D. Civin, D. Beaty, D. Kreymer, D. Li, D. Adkins, D. Xu, D. Testuggine, D. David, D. Parikh, D. Liskovich, D. Foss, D. Wang, D. Le, D. Holland, E. Dowling, E. Jamil, E. Montgomery. E. Presani, E. Hahn, E. Wood, E.-T. Le, E. Brinkman, E. Arcaute, E. Dunbar, E. Smothers, F. Sun, Kreuk, F. Tian, F. Kokkinos. F. Ozgenel, Cag- gioni, F. Kanayet, F. Seide, G. M. Florez, G. Schwarz, G. Badeer, G. Swee, G. Halpern, G. Herman, G. Sizov, Guangyi. Zhang, G. Lakshminarayanan, H. Inan, H. Shojanazeri, H. Zou, H. Wang. H. Zha, H. Habeeb, H. Rudolph, H. Suk, H. Aspegren, H. Goldman. H. Zhan, I. Damlaj. Molybog, I. Tutanov, I. Lcontiadis, L-- E. Veliche, Gat, Weissman, J. Geboski, Kohli, J. Lam. J. Asher, J.- Gaya, J. Marcus, Tang, J. Chan, Zhen, J. Reizenstein, J. Teboul, Zhong, J. Jin, J. Yang, J. Cummings, J. Carvill, Shepard, J. McPhie, J. Torres, J. Ginshurg, J. Wang, K. Wu, K. H. U, K. Saxena, K. Khandelwal, K. 7and, K. Matosich, K. Veer- araghavan, K. Michelena, K. Li, K. Jagadeesh, K. Huang, K. Chawla, K. Huang, L., Chen, I. Garg, I., A, L. Silva, L. Bell, L. Zhang. L. Guo, L. Yu, L. Moshkovich. L. Wehrstedt, M. Khabsa. M. Avalani, M. Bhatt, M. Mankus, M. Hasson, M. Lennie. M. Reso. M. Groshev, M. Naumov, M. Lathi, M. Keneally. M. Liu, M. L. Seltzer, M. Valko, M. Restrepo, M. Patel, M. Vyatskov, M. Samvelyan, M. Clark, M. Macey, M. Wang. M. Hermoso, M. Metanat, M. Rastegari, M. Bansal, N. Santhanam, N. Parks, N. White, N. Bawa, N. Singhal, N. Egebo, N. Usunier, N. Mchta, N. P. Laptev, N. Dong, N. Cheng, O. Chernoguz, O. Hart, O. Salpekar, O. Kalinli, P. Kent, Parekh, P. Saab, Balaji, P. Rittner, P. Bontrager, P. Roux, P. Dollar, P. Zvyagina, P. Ratanchandani, P. Yuvraj, Q. Liang, R. Alao, R. Rodriguez, R. Ayub, R. Murthy, R. Nayani, R. Mitra, R. Parthasarathy, R. Li, R. Hogan, R. Battey, R. Wang. R. Howes, R. Rinott, S. Mehta, S. Siby, J. Bondu, S. Datta, S. Chugh, S. Hunt, S. Dhillon, S. Sidorov, S. Pan, S. Mahajan, S. Verma, S. Yamamoto, S. Ra- maswamy, S. Lindsay, S. Lindsay. S. Feng, S. Lin, S. C. Zha, S. Patil, S. Shankar, S. Zhang. S. Zhang, S. Wang. S. Agarwal, S. Sajuyigbe, S. Chintala, S. Max. S. Chen, Kehoe, Satterfield Govindaprasad, S. Gupta, S. Deng. S. Cho, S. Virk, S. Subramanian, S. Choudhury, S. Goldman, T. Remez, T. Glaser, T. Best, T. Koehler, T. Robinson. Li, Zhang, T. Matthews, T. Chou, T. Shaked, V. Vontimitta. V. Ajayi, V. Montanez, V. Mohan, V. Kumar, V. Mangla, V. Ionescu, V. Poenaru, V. T. Mihailescu, V. Ivanow, W. Li,
W. Wang, W. Jiang, W. Bouaziz, W. Constable, X. Tang, X. Wu, X. Wang, X. Wu, X. Gao, Y. Kleinman,
12

---

Published as a conference paper at ICLR 2026
Y. Chen, Y. Hu. Y. Jia, Y. L., Y. Zhang, Y. Zhang. Adi, Y. Nam. Yu, Wang, Y. Zhao, Y. Hao, Y. Qian. Y. Li, Y. He, Z. Rait, Z. Vito,, Z. Rosnbrick, Z. Wen, Z. Yang, Z. Zhao, and Z. Ma. The llama herd of models, 2024, URL https: / //tpaaaavvv///////////...///aava www.iiiworg......e....ooo..oooo. abs/ abs /2002 21783.
C. Guérin, C. Rigaud, A. Mercier, F. Ammar-Boudjelal. K. Bertet, A. Bouju,J.CC. Burie, G. Louis, J.M. Ogier, and A. Revel. ebdtheque: representative database of comics. In Proceedings of the 12th International Conference on Document Analysis and Recognition (ICDAR), pages 1 1145-1149, 2013.
N. Huyah, M. R. Bouadjenek, S. Aryal, I. Razzak, and H. Hacid. Visual question answering: from early developments to recent advances - a survey, 2025. URL https:/ /ttp://axxv//g///........... abs aaaaaab///// 22001.511.000 03939.
M. Iyyer, V. Manjunatha, A. Guha, Y. Vyas, J. Boyd-Graber, H. Daumé, and L. Davis. The amazing mysteries of the gutter: Drawing inferences between panels in comic book narratives. In 2017 JEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6478 6487, 2017. doi: 10.1109/CVPR.200 2017.2017.666 1.66.66.....
A. Karamolegkou, P. Rust, R. Cui, Y. Cao, A. Søgaard, and D. Hershcovich. Vsiounlanguage models under cultural and inclusive considerations. In N. Soni, I. Flek, A. Sharma, D. Yang. S. Hooker, and H. A. Schwartz, editors, Proceedings of the 1st Human-Centered Large Language Modeling Workshop, pages 53- 66, TBD, Aug. 2024. ACL. doi: 10. N0.18663/v1/2024 h0033//1l002. 1.5. URL https: : 2024. mucclnm-ii 1.5/.
.org/
C. Kramsch. Language and culture. AILA Review, 27(1)300-55 2014. ISSN 1461-0213. doi: https///oi.orr//10 27.00ka. URL https: /www.
Form. com/ content/ http///ooottttttaaaa/aaaaaaaaaee jttr/oournaaaaaaa 1075/
Li, D. Li, S. Savarese, and S. Hoi. B1ip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models, 2023. URL httttps:/arxii..rrrr
/2001 12597.
Z. Li, B. Yang, Q. Liu, Z. Ma, S. Zhang, J. Yang, Y. Sun, Y. Liu, and X. Bai. Monkey; Image resolution and text label are important things for large multi-modal models, 2024. URL https: org/ abs/ 2311.0660 06607.
Lin, C. Liu, R. Zhang, Gao, L. Qiu, H. Xiao, H. Qiu, C. Lin, W. Shao, K. Chen, J. Han, S. Huang. Y. Zhang, X. He, H. Li, and Y. Qiao. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-mode modal large language models, 2023. URL https: : // /arxiv. org/ abs 07575.
H. Liu, C. Li, Y. Li, and Y. J. Lee. Improved baselines with visual instruction tuning, 2023a.
H. Liu, C. Li, Q. Wu, and Y. J. Lee. Visual instruction tuning, 2023b.
H. Liu, C. Li, Y. Li, B. Li. Y. Zhang. S. Shen, and Y. J. Lee. Llava-next: Improved reason- ing, oct, and world knowledge, January 2024. URL 2024-00 01 - 30- 1 1lava -next/.
1c/blo//
Y. Matsui, K. Ito, Y. Aramaki, A. Fujimoto, Ogawa, T. Yamasaki, and K. Aizawa. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications, 76(20);1181218888 Nov. 2016. ISSN 1573-7721. doi: 10. 10.0077/111442-01--400000 URL http:// http://dx.doi..ooo doi http://.xxdo...//////.// 1000/007//
s11042 60006-40222222
T. Naous, J. Ryan, A. Ritter, and W. Xu. Having beer after prayer? measuring cultural bias in large language models. In L.-W. Ku, Martins, and V. Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 16366-16393, Bangkok, Thailand, Aug. 2024. Association for Computational Linguistics. doi: 10. a2l www.elooo....... 862. URL https: : / / accl anthol ooy org/ 2222222222220 - 1 long. 862/
Nayak, K. Jain, Awal, S. Reddy, S. V. Steenkiste, L. A. Hendricks, K. Stanezak, and Agrawal. Bench- marking vision language models for cultural understanding. In Y. Al-Onaizan, M. Bansal, and Y.NN. Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5769-5790, Miani, Florida, USA, Nov. 2024. Association for Computational Linguistics. doi: 18.18653/ 1120024emmnlpmmale 322. URL https: : / acl wcw.nnooloogoo...ooooooooo wwwps/gy..r...oooeeee.eee... wwt. 2024. mam.mlpmmain..99.
K. Papineni, S. Roukos, Ward, and J. Zhu. Bleu: a method for automatic evaluation of machine translation. In P. Isabelle, E. Charniak, and D. Lin, editors, Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311-318. Philadelphia, Pennsylvania. USA, July 2002. Association for Computational Linguistics. doi: 10.31 171071310730 1073113.. URL hotps: / / hot ogy .org/ P02-1040/.
13

---

Published as a conference paper at ICLR 2026
S. Pawar, J. Park, J. Jin, A. Arora, J. Myung, S. Yadav, F. G. Haznitrama, I. Song, A. Oh, and L. Augenstein. Survey of cultural awareness in language models: Text and beyond, 2024, URL abs/2411, 00860.
R. Rei, J. G. C. de Souza, D. Alves, C. Zerva, A. C. Farinha, T. Glushkova, A. Lavie, L. Coheur, and A. F. T. Martins. COMET-22 22: Unbabel-IST 2022 submission for the metrics shared task. In P. Koehn, L. Barrault, O. Bojar, F. Bougares, R. Chatterjee, M. R. Costa-jussà, C. Federmann, M. Fishel, A. Fraser, M. Freitag, Y. Graham, R. Grundkiewicz, Guzman, B. Haddow, M. Huck, A. Jimeno Yepes, T. Knemi, A. Martins, M. Morishita, C. Monz, M. Nagata, Nakazawa, M. Negri, A. Névéol, M. Neves, M. Popel, M. Turchi, and
M. Zampieri. editors, Proceedingy of the Seventh Conference on Machine Trunslation ( (WMT), pages 578- 585, Abu Dhabi, United Arab Emirates (Iybrid), Dec. 2022. Association for Computational Linguistics. URL https:///////m // ////acaaantt loothoooogo , org/ 2022.202222222 wwtt 1./..
A. Reuel, A. Hardy, C. Smith, M. Lamparth, M. Hardy, and M. J. Kochenderfer. Betterhench: As- sessing ai benchmarks. uncovering issues, and establishing best practices.
A. Globerson, L. Mackey, D. Belgrave. A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 21763-21813. Curran Associates, Inc.. 2024. URL
neurips cappaper fi set_s__aan____eaaaeeeeee Track.
(a88e839977ff887757777777777 pdf.
Paper-
D. Romero, C. Lyu, H. A. Wibowo, T. Lynn, I. Hamed, A. N. Kishore, A. Mandal, A. Dragonctti, A. Abzaliev, A. L. Tonja, B. F. Balcha, C. Whitehouse, C. Salamea, D. J. Velasco, D. I. Adelani, D. L. Meur, E. Villa- Cueva, F. Koto, F. Farooqui, F. Belcavello, G. Batnasan, G. Vallejo, G. Caulfield, G. Ivetta, H. Song, H. B. Ademtow, H. Maina, H. Lovenia, I. A. Azime, J. C. B. Cruz, Gala, J. Geng, J.-G. Ortiz-Barajas, Back, Dunstan, A. Alemany, K. Y. Nagasinghe, L. Benotti, L. F. Haro. M. Viridiano, M. Estecha- M. Jouitteau, M. Mihaylov, M. F. M. Imam, M. F. Adilazuarda, M. Gochoo, M.-E. Otgonbold, N. Etori, O. Niyomugisha, P. M. Silva, P. Chitale, R. Dabre, R. Chevi, R. Zhang, R. Diandaru, S. Cahyawijaya, S. Góngora. S. Jeong, S. Purkayastha, T. Kuribayashi, T. Clifford, T. Jayakumar, T. T. Torrent, T. Ehsan, V. Araujo, Y. Kementchedjhieva, Z. Burzo, Z. W. Lim, Z. X. Yong, O. Ignat, J. Nwatu, R. Mihalcea, T. Solorio, and A. F. Aji. Cvqa: Culturally- diverse multilingual visual question answering benchmark, 2024, URL https: : / /tt///aaaaa war.aaivvvv..o..... wtw.or/////////o///a/e.o./eeaaee
Garitagoitia, M. C. B. Cabrera, M. Rodríguez-Cantell
abs / 2406. 05967.
F, Schneider, C. Holtermann, C. Biemann, and A. Lauscher. GIMMICK: Globally inclusive multimodal mul- titask cultural knowledge henchmarking. In W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar, editors, Findings of the Association for Computational Linguistics: ACI. 2025, pages 9605-9668, Vienna, Austria, July 2025. Association for Computational 1. Linguistics. ISBN 979-8-89176-256-6
doi: 10.18653/v1/203 findings- ac1.500. URL ht tpst /t////aaaaaa httpaaaaanttoooooooooooooooooooa wwg.:/oooooooooooooooooooooooeee 2025. findings acw.acc..00.. 500/.
T. Sellam, D. Das, and A. Parikh. BLEURT: Learning robust metrics for text generation. In D. Jurafsky, J. Chai, N. Schluter, and J. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Compu- tational Linguistics, pages 7881 17892, Online, July 2020. Association for Computational Linguistics. doi: 10. 10..8865/1/2020..c wal.0aaccc-maaaa main.704. URL https: ac/ httpa/aaaannooooooooooooooooooot org/ 2020 wel-mmainn 704/.
Singh, A. Romanou. C. Fourrier, D. I. Adelani, J. Ngui, D. Vila- Suero, P. Limkonchotiwat, K. Marchi- sio. W. Q. Leong. Y. Susanto. R. Ng. S. Longpre. S. Ruder, W.-Y. Ko, A. Bosselut, A. Oh, A. Mar- tins, L. Choshen, D. Ippolito. E. Ferrante, M. Fadaee, B. Ermis, and S. Hooker. Global MMLU: Under- standing and addressing cultural and linguistic biases in multilingual evaluation. In W. Che, J. Nabende, E. Shutova, and M. T. Pilchvar, editors, Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Papers), pages 18761-18799, Vienna, Austria, July 2025. As- sociation for Computational Linguistics. ISBN 979-8-89176-2514 doi: 10.18653/v1/202. acl-long.919 URL /eaetss:/aclle
org/ 20252025522222 acl long. 919/.
A. Vayani, D. Dissanayake, H. Watawana, N. Ahsan, N. Sasikumar, O. Thawakar, H. B. Ademtew, Y. Hmaiti, A. Kumar, K. Kuckreja, M. Maslych, W. A. Ghallabi, M. Mihaylov, C. Qin, A. M. Shaker, M. Zhang, M. K. Ihsani, A. Esplana. M. Gokani, S. Mirkin, H. Singh. A. Srívastava, E. Hamerlik, F. Izzati, F. A. Maani, S. Cavada, J. Chim. R. Gupta, S. Manjunath, K. Zhumakhanova, F. H. Rabevohitra, A. Amirudin, M. Ridzuan, D. Kareem, K. More. K. Li. P. Shakya, M. Saad, A. Ghasemaghaei. A. Djanibekov, D. Azi- zov, B. Jankovic, N. Bhatia, A. Cabrera, J. Obando-Ceron, O. Otieno, F. Farestam, M. Rabbani, S. Baliah, S. Samjeev, A. Shtanchacv, M. Fatima, Nguyen, A. Kareem, T. Aremu, N. Xavier, A. Bhatkal, H. Toyin, A. Chadha, H. Cholakkal, R. M. Anwer, M. Felsberg, J. Laaksonen, T. Solorio, M. Choudhury, Laptev, M. Shah, S. Khan, and F. Khan. All languages matter: Evaluating Imms on culturally diverse 100 languages, 2025. URL https: /arxiv..vvv.. org/ abs/abs//22 /2112111111 16508.
E. Vivoli, M. Bertini, and D. Karatzas. Comix: A comprehensive benchmark for multi-task comic understand- ing, 2024. URL htt.t::p://// httppp/::////awioo/////aa////// 00000000055.0
14

---

Published as a conference paper at ICLR 2026
P. Xu, Y. Wang. S. Zhang, X. Zhou, X. Li, Y. Yuan, F. Li, S. Zhou, X. Wang. Y. Zhang, and HI Zhao. Tec-bench: Benchmarking the traditional chinese culture understanding capabilities of milms, 2025. URL https: //arxiv. org/ abs/ 2505. 11275.
Yang. Zhang. Ren. Z. Xu. X. Zhang. Song, Lin, F. Xia. Cultural bias matters: cross-cultural benchmark datasct sentiment-enrichet model for understanding multimodal metaphors. In W. Che, J. Nahende, E. Shutova, and M. T. Pilchvar, editors, Proceedings of the 63nd Annual Meet- ing of the Association for Computational Linguistics (Volume I: Long Papers), pages 26301-26317, Vi- enna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-251-00 doi: 10.18653/v 1/2.33/w/2222.cc/m wal.25..cc--onogg. 1275. URL https: / /ac.aahhlaaalyyyyoocoooco cxg/ 202..025..00 acl long. 1275/.
J. Ye, H. Xu, H. Liu, A. Hu, M. Yan, Qian, Zhang, F. Huang, and Zhou. mplug-ow13: Towards long image-sequence understanding in multi-modal large language models, 2024. URL https:/ //ariiv. org/ abs arg/abs// /2408. 04840.
Zhang, X. Dong, Y. Zang, Y. Cao, R. Qian, L. Chen, Q. Guo, Duan, B. Wang. L. Ouyang, Zhang, W. Zhang, Y. Li, Y. Gao, P. Sun, X. Zhang, W. L. Li, W. Wang, H. Yan, C. He, X. Zhang. K. Chen, J. Dai, Y. Qiao, D. Lin, and J. Wang. Internlm-xcool xcomposer-2.5; A versatile large vision language model supporting long-cone contextual input and output, 2024. URL ht https:/ aarxiv....... org/ abs///b///// / /4007 03320.
15

---

Published as a conference paper at ICLR 2026
Input Image:
Q1: Which culture does the background in the comics page belong to? Choose all the possible answers.
Choices for Q1:
A. Cambodia
B. Greece
C. Brazil
D. Australia
E. China
Answer to Q1: C
Q2: What object can reect a certain culture in the picture? You should choose the most appropriate answer.
Choices for Q2:
A. A'An Australian surfer', kilt', Arabian sheikhs in Brazilian', rainforest hut', 'indian sari']
B. ['An Australian surfer', kilt', Arabian sheikhs in a Brazilian', rainforest hut', 'batik
robe has clash"]
C. ['An Australian surfer', kilt', Arabian sheikhs in a Brazilian', rainforest hut', 'a group of inuit elders']
D. ['An Australian surfer', kilt', Arabian sheikhs in a Brazilian', rriinforsst hut']
E. [' ['An Australian surfer', kilt'," Arabian sheikhs in Brazilian")
Answer to Q2: D)
Q3: lease answer the following question based on the image. there a culture contradicts
in this image? Answer to Q3: Yes.
Q4: If there is contradict in this image, please answer the following question. For each object ['An An Australian surfer', kilt', Arabian sheikhs Brazilian', rainforest hut' ]. if that object and certain culture in ["Brazil", "Australia") conflict with each other, please output it. The format "Something should not in Some Culture". For example, "Katana should not be in America", "Chinese calligrapher should be in Africa". If there isn' 'n't contradict, output nothing.
Answer to Q4:
1. An Australian surfer should not be in Brazil, kilt should be in Australia.
3. kilt should not be in Brazil.
Arabian sheikhs Brazilian should not in Australia.
5. rainforest hut should not be in Australia.
Figure 6: A data sample of Extraction @ Culture and Conflict @ Culture.
16

---

Published as a conference paper at ICLR 2026
BRC U Later
SP0 8- Tam cadt
SRC Hile BEStE BIRIEEEGEEEEI EI +HEBS
ATOT:
with folls like that.
Jit:: 2713 A.tri....... IGI adore...
SIR to / d Aufte 6
why you keep your open....
BRC: 2:0
S....... Piuu Whe curben Bdlm
TST Too nanyy doopless rights... Pre got koop meds
Figure 7: A data sample of Generation Culture. During each inference, a single source sentence and the comic page are input the MLLM.
A QUESTION SETTING FOR EACH TASK
A.1 EXTRACTION @ CULTURE
Question 1: Which culture does the background in the comics page belong to? Choose all the possible answers.
Question 2: What object can reflect a certain culture the picture? You should choose the most appropriate answer.
A.2 CONFLICT@ CULTURE
Question 3: Please answer the following question based on the image. Is there culture contradicts in this image?
Question 4: If there is contradict in this image, please answer the following question. For each object in the Ag. that object and a certain culture in A₁ conflict with each other, please output it. The format is "Something should not be in Some Culture". For example, "Katana should not be in America", "Chinese calligrapher should not be Africa". If there isn't contradict, output nothing.
where A refers to the list of cultures of the MLLM's answer to Q1, and A2 refers to the list of cultural representative objects of the MLLM's answer to Q2.
A.3 GENERATION @ CULTURE
Translate following sentences from Japanese into English. Output the result line by line. The sentences are:
B
SOME DATA SAMPLE FROM C²
B. EXTRACTION @ CULTURE AND CONFLICT @ CULTURE
An example of Extraction @ Culture and Conflict Culture is presented in Figure 6.
17

---

Published as a conference paper at ICLR 2026
B.2 GENERATION @ CULTURE
An example of Generation @ Culture is presented in Figure 7.
C PROMPT SETTING FOR DATA COLLECTION AND ANNOTATION
C.1 PROMPT SETTING FOR QA PAIR CREATION CONFLICT CULTURE TASK
Only output the answer. You need to check if there is anything in the element list that conflicts with the culture in the culture list. If there is, you should output the result as "Something should not be in Somewhere." ". Otherwise, just output "No". For element, as aconflict. For example, the culture list is [Japan, China] and the element list is [katana, Chinese tea]. Then you should output "katana should not be China" and "Chinese should not bein Japan".
conflicts with any one of the culture in the list, you should count
As for acceptable results. For example, some acceptable results are "Katana should not be inAmerica", "Chinese calligrapher should not be in Africa" and "No". Now output the result for: The element list le. The culture list is 4p.
Figure 8: The prompt setting for QA pair creation in @onflict@ Culture Task.
In the prompt setting of @onflict@ Culture (presented in Figure 8), and Ip denote the list of cultur- ally representative objects and culture of image annotated in task Extraction Culture respectively.
C.2 PROMPT SETTING FOR ANNOTATION IN GENERATION CULTURE
The model we apply in annotation in Generation @ Culture for creating B is DeepSeek- RI (DeepSeek-AI, 2025). The prompt we use for Translator is presented in Figure 9. The prompt we use for Reviewer is presented Figure 10. Upon receiving the review feedback, the prompt provided to the Translator is presented in Figure 11.
Output the answer line by line. Please only output the answer. Translate following sentence from Japanese to English: s.
Figure 9: The prompt used for the Translator, where denotes the source sentence that is to be translated.
Output the answer line by line. Only output the translation, not suggestions. Please only output the modified translation. You need to pay attention to whether there are any incon- sistencies in context, whether there are culturally related errors and whether there are any translation errors. The source is:s and the translation is:l.
Figure 10: Translator.
The prompt used for the Reviewer, where denotes the translation provided by the
18

---

Published as a conference paper at ICLR 2026
Output the answer line by line. Please only output the answer. Translate following sentence from Japanese to English: s. Here is some advice: a
The prompt used for the Translator once the review is provided, where denotes the review generated by the Reviewer.
Figure 11:
Input:
Only output the answer. You need to check if there is anything in the element list that conflicts with the culture in the culture list. If there is, you should output the result as "Something should not be in Somewhere." Otherwise, just output "No". For an element, conflict. For example, the culture list is [Japan, China] and the element list is [katana, Chinese tea]. Then you should output "katana should not be China" and "Chinese tea should not bein Japan".
conflicts with any one of the culture in the list, you should count
As for acceptable results. For example, some acceptable results are "Katana should not be in America", "Chinese calligrapher should not be in Africa" and "No". Now output the result for: The element list is [China, Japan, Brazil]. The culture list is [Chinese calligrapher, Japanese Katana, Brazilian Dancer]. Output with Error: In China, Brazilian dancers are rare... Corrected Output: Brazilian dancers should not be China.
Figure 12:
An example of formatting inconsistency. The format provided is shaded in blue, and the output with error doens't follow it.
D EXAMPLES OF ERRORS IN MANUAL VERIFICATION IN CONFLICT @ CULTURE
D.I FORMATTING INCONSISTENCY
This type of error occurs when the format of the conflict descriptions generated by the models fails to conform to the format provided. An example provided in Figure 12.
D.2 CULTURE-RELATUD ERRORS
This type of етrо occurs when the conflict descriptions generated by the models contains cultural errors. An example is provided Figure 13.
E ANNOTATION DETAILS
All annotations were performed by three graduate students and two undergraduate students, with distinct division of labor across annotation tasks. All of the annotators come from Asian countries. Among the five annotators, three are graduate students engaged culture-related research. The quality of annotation is ensured by credible online resources (e.g. Wikipedia).
For the annotation of cultural lists and cultural object lists in Image Collection phase, the two un- assigned 512 images (accounting
dergraduate students were tasked with this work. where each for 50% the total dataset) for annotation. Subsequent to this initial annotation phase, the three graduate students conducted comprehensive review of the annotated content and rectified any notation errors identified based on the prompts which are used to generate images. The criteria for this phase are defined follows:
Fidelity: An object counted evokes perceived association with a specific cultural context.
19

---

Published as a conference paper at ICLR 2026
Input:
Only output the answer. You need to check if there is anything in the element list that conflicts with the culture in the culture list. If there is, you should output the result as "Something should not be in Somewhere.", Otherwise, just output "No".
For element, if it conflicts with any one of the culture in the list, you should count it as a conflict. For example, the culture list is [Japan, China] and the element list [katana, Chinese tea]. Then you should output "katana should not be in China" and "Chinese tea should not bein Japan".
As for acceptable results. For example, in America", "Chinese calligrapher should not be in Africa" and "No". Now output the result for: The element list is [China, Japan, Brazil]. The culture list is [Chinese calligrapher, Japanese Katana, Brazilian Dancer].
acceptable results "Katana should not be
Output with Error: Japanese Katana should not be in Japan. Corrected Output: Japanese Katana should not be in China. Japanese Katana should not be Brazil.
Figure 13: An example of culture-related errors. In our example, Japanese katana should not be the country except Japan.
2. Granularity: For an object associated with a culture shared across multiple countries, regional-level description shall be adopted.
3. Security: An image shall be excluded if it contains aggressive depictions, which refer to racist content, sexual explicitness, and other analogous content that may cause harm to individuals.
Following the annotation rectification phase, we calculated that 237 out of the total 1,023 images underwent revisions. The inter-annotator agreement score was determined to 76.8%.
For the annotation of question-answar (QA) pairs corresponding to the three tasks, all five annotators participated in the verification phase. Initially, each annotator independently revised the annotation conducted
results; subsequent to this individual revision step, joint review of each question collectively by the entire annotation team. Finally, the questions corresponding to 755 images were manually revised, resulting in correction rate of 73.8%. In this phase, the criteria are defined as follows:
Culture Fidelity: All annotators were instructed retrieve relevant information from cred- ible online sources (e.g. Wikipedia) to verify the accuracy of the specific forms of cultural objects depicted in the image. In cases where inaccuracies were identified, the annotations were revised based on the outcomes of group discussions.
The guidelines for the annotation of cultural lists and cultural object lists are presented as follows:
1. Fidelity & Granularity: If an element in the image shows a country's cultural traits, please label it with the country name. If reflects a regional cultural feature, please use the region or continent namelike "Middle East," "Asia," "South Pacific," and so on.
2. Security: there's any stereotype or offensive content in the image or prompt, mark it for removal.
3. Example:
When labeling elements in the image, try keep the labels close possible to the prompt used to generate the image. Here's 's an example: "A Brazilian footballer in Scottish kilt has shouting match with Chinese ping-pong players in Beijing sports complex." such case, if the elements mentioned in the prompt appear in the image and can be matched to the entities in the prompt (e.g., the footballer in the image corresponds to a "Brazilian footballer." " etc.), label them exactly as "Brazilian Footballer." " "Scottish Kilt," and so on.
4. Credibility: For the cultural elements in the image, look them up Wikipedia if you not familiar with the culture. You can label them only they match the relevant information there; otherwise, skip the labeling.
20

---

Published as a conference paper at ICLR 2026
F
CASE STUDY
During evaluation, we observed several unexpected behavioral patterns that led to suboptimal question- -answeinnn performance in MLLMs.
F.1 ERROR CASES IN Q1
"Turn-a-deaf-ear". This behavioral pattern is particularly occurred in LLaVA-NeXT, where the model frequently defaulted to describing the image rather than direct question-answering when pre- sented with clear instructions. We name this behavoir as "Turn-a-deaf-ear" An example is provided in Figure 14.
We assume that this behavioral pattern likely stems from the fine-tuning dataset's compositional bias, which predominantly trains MLLMs for image description tasks rather than image-grounded question answering.
We need to mention that, in Q3, LeXVA-NeXT fails to predict the result properly due to the same reason as Q1.
Input Image:
Input: Which culture does the background in the comics page belong to? Choose all the possible answers. Please only generate answer.
A. Australia
B. France
C. Middle-east
D. Indonesia
E. South america
Ground-truth Answer: A,B,C
Output: The background in the comics page appears to be inspired by a setting that could be from variety.
An example of "Turn-a-deaf-ear" is presented. The MLLM is asked to output the correct choices but it outputs the detail of the comics page instead.
Figure 14:
F.2 ERROR CASE IN Q2
"Take-a-shoo--n-thh--aaa This behavioral pattern particularly observed in LLaVA1.5-78
when the model attempted to answer Q2. For this question, the model exhibited a tendency to output "A" the response.
We calculated the frequency that LLaVA1.5-7B outputs the choice "A" and the result is 78.4, which says that the model cannot understand the cultures properly and tends to output answer.
21

---

Published as a conference paper at ICLR 2026
F.3 ERROR CASE Q4
Keep answering "Nothing": This behavioral pattern is particularly occurred in LLaVA-NeXT, where the model always answering "Nothing", which shows lack of cultural conflict comprehen- sion capability. An example is presented in Figure 15.
Input Image:
Input: If there is contradict in this image, please answer the following question. For each object in ('Brazilian samba dancer', Swiss dirndl engages', dispute with Turkish Janis- saries', Chinese pagoda'], if that object and certain culture the [Turkey, Switzeland, Brazil, China] conflict with each other, please output it. The format is "Something should be Some Culture". For example, "Katana should not be in America", "Chinese callig- rapher should not be in Africa". If there isn' an't contradict, output nothing. Ground-truth Answer: "Swiss dirndl engages should be Brazil, Output: Nothing.
Figure 15: An example of keeping answering nothing. MLLM keeps answering "nothing" instead of the description of cultural conflicts.
"Stubbornness": This behavioral pattern is particularly occurred in LLaVA-1.5-7B. where the model frequently defaulted to follow the instruction without thinking. We name this behavior as "Stubbornness". An example is provided in Figure 16.
F.4 ERROR CASE IN Q5
Poor instruction following capabilities MiniGPT: For Question 5, our instruction MLLMs was directly output the translated sentence without including any redundant explanatory content. Nevertheless, MiniGPT tended to elaborate on how the translation was performed and also repeated the identical sentence second time. An example is presented in Figure 17.
G DETAILS ON HUMAN-MODER ANALYSIS
Regarding the human-model analysis, three undergraduate students volunteered to participate. Each question answered separately, with one-hour time limit per question, and was presented in the same format used for the MI.LMs.
H EFFECT OF OCR INTEGRATION ON MT TASK PERFORMANCE
We conducted additional experiments to evaluate the impact of OCR on the Generation @ Culture task. Our experimental settings are follows: one group directly adopted the pre-extracted text from Manga109, while the other group utilized OCR technology for text extraction. We use BLEU as the evaluation metric The results are presented in Table 8.
22

---

Published as a conference paper at ICLR 2026
Input Image:
Fight!
If there is contradict in this image, please answer the following question. For each ob- ject in ['French chef in a Native', American headdress is wrestling'. African tribal chiefs', Russian Orthodox church'), if that object and certain culture in the [France, America, Russia] conflict with each other, please output it. The format is "Something should not be in Some Culture". For example, "Katana should not in America", "Chinese calligrapher should not be in Africa" If there isn't contradict, output nothing.
Input:
Ground-truth Answer: "French chef Native should be United States.", Output:Something should not be in Some Culture.
Figure 16: An example of "Stubbornness" is presented. The MLLM fails generate the required substitutions, leaving both the placeholder terms 'Something" and 'Some Culture' unmodified in its output.
Sure, here are the translations of the sentences you provided:
1. kuru na!: This is an interjection used to express excitement or surprise, similar to "Wow!" or "Whoa!" in English.
2. kuru na!: This is another way of expressing excitement surprise, similar to "Wow!" or "Whoa!"
An example of poor instruction following capability of MiniGPT. Our instruction clearly required MiniGPT to directly output the target translations without additional content. Nev- ertheless, the model responded with redundant descriptive text (e.g., explaining the sentence type and emotional connotation) and duplicated translations for the identical Japanese input. Both issues indicate its fallure adhere the constraint of instruction.
Figure 17:
From the results, it is evident that OCR significantly degrades model performance. Since we did not utilize OCR in our experiments, the observed lower model performance is not attributable to OCR-related factors.
Table Main Result of effect of OCR on MT tasks. The highest performance is marked bold.
Methods
JA-DE
JA-RU
JA-ES
JA-TH
OCR
No OCR
OCR
No OCR
OCR
No OCR
OCR
No OCR
Qwen2.5-VL.
0.92
12.0
0.97
8.74
1.32
14.5
1.22
9.72
LLaVA1.5-7B
0.64
2.74
0.24
1.39
0.62
4.23
0.32
1.13
Llama3.2
0.33
4.27
0.22
1.70
0.45
5.73
2.73
0.99
23


Reviews:
[{"review_id": "91boTuzPo1", "reviewer": "Reviewer_h7gx", "summary": "This paper proposes benchmarking MLLM in cultural and multilingual understanding via tasks involving comic panel as the image input. This benchmark were generated with a pipeline of both automated generated data and human in-the-loop for verification, and have 3 sub-tasks, covering various languages depening on the task. Lastly, author benchmarked various models, noting the challenge of the benchmark.", "questions": "-", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "The use of comic-panel as a benchmark is interesting and unique. This benchmark is decently sized, multilingual.", "weaknesses": "My biggest concern is the decision to use AI-generated images for this particular benchmark. If we hypothesize that current MLLMs are not yet proficient at handling multicultural contexts, then generating cultural images in this way could be harmful and potentially misleading. Anecdotally, the examples in this paper show exaggerated depictions of cultures (see relevant reading: https://arxiv.org/pdf/2407.14779v1).\n\nThis issue is further compounded by the lack of human validation for the generated images. Without proper grounding (e.g., visual references), there is no guarantee that the generated cultural images are accurate or appropriate.\n\nThe author also does not seem to consider what truly constitutes a culture, beyond surface-level representations such as “Native American” or related objects. While these can serve as proxies for culture, cultural understanding goes much deeper than that. The paper could benefit from a more robust definition of what the author considers a “cultural benchmark,” which would also clarify how the list of cultures, objects, or related proxies was derived.\n\nFigure 5 further illustrates the lack of care in defining culture. The plot is confusing, it includes “Asia” alongside “Indonesia,” “Korea,” and “Japan.” Are these not all Asian cultures? What, then, constitutes “Asian” culture? This connects back to the earlier issue of validation: just as the generated images require validation, the questions themselves should also be validated, ideally by native speakers and individuals familiar with the respective cultures.\n\nMinorly, the author claims that existing cultural VQA datasets are weaker because they use real-world images, implying those are easier. I am not convinced by this reasoning. Real-world images reflect the actual use cases for these models, and there is no guarantee that such tasks are easier.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "30yA3FbcVy", "reviewer": "Reviewer_oEmH", "summary": "The paper introduces C3B (Comics Cross-Cultural Benchmark), a comic-centric evaluation suite for cultural awareness in MLLMs. C3B contains 2,220 images and 18,789 QA pairs, spanning three escalating tasks: (i) culture and object detection, (ii) cultural-conflict object detection and description, and (iii) machine translation. The dataset mixes generated comics via Doubao APIs with automatic and manual annotations of culturally representative objects and conflicts (DeepSeek-V3-assisted) and Manga109 pages for translation with a translator–reviewer multi-agent pipeline for references. The authors benchmark 11 open-source MLLMs, report low performance, especially on conflict description (Q4), and show a consistent gap vs. human accuracy across difficulty tiers.", "questions": "- How were stereotypes and sensitive portrayals handled during generation and annotation? Any expert review (region-specific annotators) or exclusion lists? A brief ethics appendix would help, given synthetic cultural content concerns.\n- For Q4, do you have independent gold conflict triples (culture, object, contradiction) for a subset, rather than relying on model-conditioned answers? If not, could you release human-vetted conflict graphs to decouple recognition errors from reasoning?\n- The human evaluation study needs further elaboration. You bucket difficulty by the number of cultures per image, then sample 100 items per tier. Could you add inter-rater agreement and per-tier time-to-answer? Also, show model vs. human delta broken down by specific culture sets (e.g., lesser-known vs. globally familiar).\n- Background Culture Identification and Culture-aware Object Detection: How are gold labels established? Please describe annotator guidelines, the source of culture/object lists, and any expert review. Include inter-annotator agreement (e.g., Cohen’s κ) and examples of borderline cases. Explicitly discuss bias mitigation (e.g., stereotype checks, exclusion lists, region-specific reviewers).\n- How do you define cultural conflict? Provide an explanation and decision rules with concrete examples from the annotation/internal validation process and guidelines. Clarify whether conflicts are culture–culture, culture–object, or object–object, and how multi-culture scenes are handled.\n- Clarify whether the Generation@Culture task uses OCR, classic machine translation, or general-purpose generative LLMs for translation. Is the input the manga image or the transcription of the text presented in the image?\n- Low BLEU: disentangle error sources. Are low scores primarily due to translation quality or OCR extraction errors? If OCR is used on Manga109, I would recommend first reporting OCR accuracy or manual correction rates. State whether you have ground-truth transcriptions for source text from Manga109; if yes, use them to isolate translation quality from text extraction noise.\n\n**Grammar mistakes/Wording fixes**\n- Line 13: “Cultural awareness capabilities has emerged” → “Cultural awareness capabilities have emerged.”\n- “current benchmarks lack progressed difficulty?” → It is unclear what is meant by progressed difficulty in the abstract, maybe progressive? I would rephrase: “Existing benchmarks do not offer a clearly staged progression of difficulty.” or something that shows that you mean progressively challenging tasks.\n- Lines 125: “integrates comics and comicss” →  Do you mean and manga? It is not clear.\nLines 249-251: This is not good English and is confusing to the reader. \"Annotation for task...\" -> \"The annotation for the tasks of\" AND \"For task Extraction@Culture\" -> \"For the task of Extraction@Culture\"\n- Lines 260–269: I would recommend avoiding the future tense in the methods section. For example, replace “We will manually inspect…” with “We manually inspect (or inspected)…” (past or present).\n- Lines 420–424: Rewrite for grammar and specificity. I assume you mean: “To evaluate C3B’s cultural diversity, we compute three measures—Culture Density Per Image (CDPI), Cultural Breadth Intensity (CBI), and Coverage-Adjusted Density (CAD)—and compare them against cultural QA datasets.” \n- Figure 1: “Is there a culture contradicts in this image?” → “Does this image contain a cultural contradiction?”, “If there is contradict in this image…” → “If a contradiction is present, answer the following questions.”, “For each object in the answer to your question 2 -/ contradiction!!” → Please rephrase the instruction clearly; e.g., “For each object identified in Q2, explain the cultural contradiction it creates.”, Labeling error: Use “C3B” instead of “C3UB.”\n- Figure 2: Same labeling error—“C3B,” not “C3UB.”\n- Table 3: Again, “C3B,” not “C3UB.”\n- Figure 5 caption: “Scores of QA Pairs in different culture” is ambiguous. Specify the metric: “Error rates of QA pairs by culture”. Also define the denominator (per pair? per culture? macro/micro?).\n\n**Comments**\n- Given that culture-related benchmarks are still relatively scarce, I recommend expanding the Related Work 2.2 section to include recent datasets that explicitly address cultural diversity in multimodal evaluation. For instance, the VizWiz cultural extension ([Karamolegkou et al., 2024](https://aclanthology.org/2024.hucllm-1.5.pdf)) demonstrates that existing real-world benchmarks, such as VizWiz, encode cultural concepts that were not taken into account in the original crowdsourced annotations. Thus, it introduces a culture-centric evaluation subset in which some images are annotated with more than one culture (e.g., a US brand package of matcha tea, a French wine packaged by Tesco in the UK, containers of Asian food from an Australian meal delivery company, etc.). This work is relevant to your focus on cross-cultural visual understanding and includes images taken in real-world settings by people who are blind that reference more than one culture. Another relevant work that seems to be missing is GlobalRG ([Bhatia et al., 2024](https://aclanthology.org/2024.emnlp-main.385.pdf)), which introduces a multicultural benchmark and two novel tasks—Retrieval across Universals and Cultural Visual Grounding—designed to assess vision-language models' performance across culturally diverse and culture-specific concepts. Including this paper would strengthen the discussion on cultural inclusivity and dataset diversity in VLM evaluation. In addition, ALM-bench ([Vayani et al., 2025](https://arxiv.org/pdf/2411.16508)) represents the largest-scale effort to date to evaluate multimodal models across 100 languages and 13 cultural dimensions, explicitly measuring cultural and linguistic inclusivity. Its scope and methodological framing make it a strong point of comparison for your benchmark. You may also consider referencing [Zhong et al., 2025](https://arxiv.org/html/2510.05931v2), which introduces a complementary perspective on multimodal cultural reasoning and could help position C3B more clearly within the rapidly growing ecosystem of cross-cultural LMM evaluation.\n- Lines 270–280: I would recommend adding a quantitative summary of that inspection (e.g., number reviewed, acceptance rate, inter-rater agreement).\n- For instruction-following issues (e.g., “stubbornness” in LLaVA-1.5-7B), this is a prompt compliance problem rather than cultural understanding. Consider simple schema validation (e.g., JSON schema with pydantic) to enforce output format and reduce evaluation noise; note it as an implementation detail, not a scientific claim.\n- For cultural density measures (CDPI/CBI/CAD), add brief intuition + formulae in main text and validate that higher density correlates with lower model accuracy on held-out items.\n- If you claim “progressive difficulty,” define how difficulty is constructed and validated (e.g., number of cultures per image, ambiguity, required cross-culture reasoning) and show that model performance monotonically decreases with the proposed difficulty index (or explain deviations). You should also clarify that by progressive difficulty, you also include the dimension of how many cultural objects are contained in one image (as it is only briefly mentioned in the human-model analysis).\n- To compensate for the limitations of synthetic data and artificial tasks, you could draw inspiration from [Evaluating Multimodal Language Models as Visual Assistants for Visually Impaired Users](https://aclanthology.org/2025.acl-long.1260/) (Karamolegkou et al., ACL 2025). That work grounds multimodal evaluation in real-world needs, such as question answering and image captioning for accessibility. You should acknowledge it in related works, since MLMMs are now being used in the real world. Incorporating similarly practical, human-centered tasks would make C3B more ecologically valid and demonstrate the benchmark’s relevance beyond synthetic cultural scenarios.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "- Framing culture evaluation around comics is novel: unlike many real-world images that encode a single culture, comics can “condense numerous cultures into a single frame”, which plausibly raises task difficulty, but also introduces the problem of artificially generated data that do not depict real-world needs.\n- The dataset combines multitask + multicultural + multilingual integration and progressive difficulty. Something that is not clarified very well in the paper is that in C3B, there are two different dimensions of progressive difficulty. One is in the three-level structure (recognition → conflict reasoning → multilingual generation), which provides a progressive, multitask view missing in several prior cultural benchmarks that are monolithic or single-task. The second one is in the number of cultural objects/conflicts identified in one image.", "weaknesses": "- Problems with tasks, prompt set-up, and phrasing. The prompts, as depicted in the Figures, contain grammatical mistakes and appear to be too complicated. See questions/comments for further details. I would recommend actually phrasing the first task as recognition (what cultures/objects appear as multiple choice (q1) and open-ended (q2) questions). Then, I would rename cultural conflict understanding to cultural conflict reasoning, emphasizing that the goal is not merely to detect a contradiction (q3), but to reason about its cause and cultural context (q4). For the Generation@Culture task, I would reconsider using machine translation as the generative component. While translation can reflect cultural nuances, it does not fully test a model’s ability to generate culturally informed or contextually appropriate content. Instead, I suggest reframing this task toward culture-aware image captioning or cultural storytelling, where models must produce natural-language descriptions or short narratives that reflect the depicted cultures’ symbols, values, or interactions. This adjustment would make the task hierarchy more coherent (recognition → reasoning → generation) and align better with standard multimodal evaluation practices. \n- Problems with dataset bias and validation. A significant share of C3B relies on generated comics and LLM-mediated conflict labels. This can introduce artifact-driven shortcuts or stereotype leakage from generators/LLMs—risks widely discussed for culture benchmarks and synthetic imagery. The paper partially mitigates with manual verification, but a deeper bias and authenticity audit (e.g., culture-expert review, stereotype checklists) would strengthen claims about real-world cultural awareness.\n- Problems with annotation quality and manual validation. The paper does not clearly describe how annotators (models, agents, or authors) were instructed to identify cultures, objects, or cultural conflicts, nor how manual validation was performed or measured (e.g., number of validators, agreement scores, or correction rate). Please provide explicit annotation details and quality-control metrics to ensure transparency and reproducibility.\n- Problems with ground truth for conflict reasoning. Q4 depends on Q1/Q2 labels plus model answers; the paper reports a composite CACC with fixed weights. While motivated, this mixes upstream recognition errors with conflict reasoning, and the ablation shows only marginal gains when injecting Q1/Q2 answers directly into the Q4 prompt. Stronger gold annotations for conflicts (independent of model responses) and sensitivity checks on CACC weights would clarify what Q4 truly measures.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7R2dbAhUc8", "reviewer": "Reviewer_MQS5", "summary": "The paper introduces a comics cross-cultural benchmark ($\\mathrm{C}^3$B) designed to measure MLLMs' cultural awareness beyond Western-centric settings. Motivated by limits in prior datasets, including single culture and real image focus, one-question tasks, and weak multilingual coverage, $\\mathrm{C}^3$B uses fictional comic scenes to pack multiple cultures into a single frame and raise task difficulty. The benchmark contains 2,220 images and 18,789 QA pairs spanning 77 cultures, aiming to evaluate models in multicultural, multitask, and multilingual contexts.\n\n$\\mathrm{C}^3$B comprises three progressively harder tasks forming a logical chain: culture-aware object extraction, cultural-conflict detection, and culturally-aligned content generation. Evaluating 11 open-source MLLMs on $\\mathrm{C}^3$B, the authors find a clear gap between models and humans, especially on cultural-conflict reasoning.", "questions": "- How well do the skills measured on $\\mathrm{C}^3$B transfer to real photos? Have you conducted any cross-benchmark evaluations?\n- Given that Doubao generated the comic pages and DeepSeek-V3 handled conflict labels and translations, what measures did you take to prevent bias or information leakage from those models?\n- What was the justification for selecting the 0.3/0.3/0.4 weights for Q1/Q2/Q4?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- Solid benchmark scale and diversity (2,220 images, 18,789 QAs, and 77 cultures).\n- Annotation pipeline combines automated detection with manual verification.\n- Multilingual coverage across five languages (Japanese, Russian, Thai, English, and Spanish) reduces monolingual bias.", "weaknesses": "- Frame-level evaluation may underutilize narrative or contextual reasoning across pages.\n- CACC uses fixed weights (0.3 / 0.3 / 0.4) without dynamic justification.\n- Comics differs from real-world photos, making transfer to natural imagery uncertain.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QbJr9gXTVo", "reviewer": "Reviewer_6ZTX", "summary": "This paper introduces C³B (Comics Cross-Cultural Benchmark), a novel benchmark aimed at evaluating Multimodal Large Language Models (MLLMs) for their cultural awareness capabilities across multiple languages and cultural contexts. Unlike prior datasets that rely on real-world images—each typically representing a single cultural context—C³B uses comics as the core medium. The authors argue that comics can depict multiple, fictional, and mixed cultural elements within a single frame, thus presenting richer and more complex cross-cultural scenarios. \n\nC³B evaluates:\n1. Culture-aware Object Extraction (Extraction@Culture) – basic recognition of cultural cues and artifacts;\n2. Cultural-conflict Object Detection (Conflict@Culture) – detecting contradictions between cultural elements;\n3. Culturally-aligned Content Generation (Generation@Culture) – multilingual cultural translation tasks (Japanese → EN/RU/DE/TH/ES).\n\nThe benchmark was used to evaluate 11 open-source MLLMs (e.g., Qwen2.5-VL, LLaVA-NeXT, InternLM-XC2.5, Llama3.2). Results indicate that Qwen2.5-VL performs best overall, particularly in multilingual translation, but all models fall far behind human performance—especially in understanding cultural conflicts and less-known cultures. Human evaluation shows a consistent 40–60% gap over MLLMs, confirming C³B’s difficulty and diagnostic potential.", "questions": "Since DeepSeek-V3 was used in annotation, did you compare its conflict-detection output to human-only annotations to quantify model bias?\nHave you performed any human evaluation of cultural fidelity (i.e., are the depicted elements realistic to their culture)?\n\noverall, it's a good paper.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "### Innovative use of comics as a medium\nThe idea of using comics for cross-cultural benchmarking is both creative and effective. Comics enable a richer mixture of visual and textual cues that naturally embed multiple cultures, allowing for deeper testing of model reasoning and perception.\n\n### Progressive multitask structure\nThe three-level design (extraction → conflict → generation) provides a structured evaluation pipeline that mimics human-like cultural reasoning progression—from perception to comprehension to production.\n\n### Multilingual and multicultural scope\nC³B covers 77 cultures and 5 languages, far broader than prior cultural benchmarks like CVQA or GIMMICK. The inclusion of low-resource languages such as Thai enhances diversity and fairness.\n\n### Detailed methodological design\nThe construction pipeline—using multi-agent annotation with both human verification and LLM assistance (DeepSeek-V3)—is well-documented and reproducible. Figures 2–3 (pp. 4–5) clearly illustrate the dataset creation process.\n\n### Comprehensive evaluation and analysis\nThe experiments include cross-task correlation analysis (Table 5), cultural diversity metrics (Table 6), and per-culture accuracy (Figure 5). These analyses go beyond standard benchmarking, offering valuable insights into model cultural biases and limitations.", "weaknesses": "### Limited evaluation diversity in the generation task\nThe Generation@Culture task focuses solely on machine translation. This limits the benchmark’s exploration of more nuanced cultural generation (e.g., culturally appropriate narration, idiomatic expression, or story understanding).\n\n### Use of AI-assisted annotation may introduce bias\nAlthough manually verified, the use of DeepSeek-V3 for conflict detection and translation generation could embed model-specific biases, especially in culturally sensitive interpretations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)