A Structured Framework for Evaluating and Enhancing Interpretive Capabilities of Multimodal LLM s in Culturally Situated Tasks (2025)

Abstract: This study aims to test and evaluate the capabilities and characteristics of current mainstream Visual Language Models (VLMs) in generating critiques for traditional Chinese painting. To achieve this, we first developed a quantitative framework for Chinese painting critique. This framework was constructed by extracting multi-dimensional evaluative features covering evaluative stance, feature focus, and commentary quality from human expert critiques using a zero-shot classification model. Based on these features, several representative critic personas were defined and quantified. This framework was then employed to evaluate selected VLMs such as Llama, Qwen, or Gemini. The experimental design involved persona-guided prompting to assess the VLM’s ability to generate critiques from diverse perspectives. Our findings reveal the current performance levels, strengths, and areas for improvement of VLMs in the domain of art critique, offering insights into their potential and limitations in complex semantic understanding and content generation tasks.

## introduction large language models (llms) have demonstrated remarkable performance on general nlp benchmarks, yet their applicability in culturally embedded, humanistic domains remains limited. in highcontext interpretive tasks such as art criticism, clinical narrative analysis, or historical commentary, model performance depends not only on linguistic fluency or factual accuracy, but also on deeper forms of cognitive alignment-epistemic sensitivity, rhetorical coherence, and cultural adaptability. a representative and particularly demanding testbed for such capabilities is chinese art commentary. this genre, especially when analyzing works like traditional landscape or court paintings, involves symbolic interpretation, aesthetic judgment, and deeply situated cultural discourse. existing multimodal llms are rarely evaluated in this space. standard benchmarks such as mme @xcite and mmbench @xcite focus on object recognition or task-oriented visionlanguage reasoning, while frameworks like art-gpt @xcite emphasize captioning and factual grounding. these methods largely overlook interpretive nuance and disciplinary diversity. meanwhile, humanistic commentary often exhibits non-linear logic, specialized lexicons, and varied stylistic conventions, particularly in chinese art contexts where rhetorical strategies such as yijing (意境, artistic conception) or qiyun shengdong (气韵生动, spiritual resonance) are essential but difficult to quantify @xcite . without appropriate grounding, llms risk producing synthetic outputs that mimic surface patterns but fail to demonstrate epistemic alignment @xcite . this growing mismatch calls for new paradigms in evaluation and adaptation. to address these challenges, we introduce vulca-the vision-understanding and language-based cultural adaptability framework. vulca is a structured evaluation and enhancement framework designed to assess how well vlms align with domain-specific interpretive practices in culturally situated tasks. our work centers on chinese art commentary, but the methodology generalizes to other multimodal and epistemically rich domains such as religion, medicine, or history. vulca combines three core components: (1) a multi-dimensional human expert benchmark (mheb) constructed from 163 art commentaries annotated across five cultural capability dimensions; (2) a persona-guided recontextualization mechanism using eight interpretive personas and a domain-specific knowledge base; and (3) a joint evaluation pipeline integrating vector-space semantic alignment with rubric-based capability scoring. commentaries are generated from annotated traditional chinese paintings, and their alignment with expert patterns is evaluated with and without interventions. as a result, we produce five contributions: (i) the definition of vulca, a new structured framework for assessing and enhancing vlms in culturally grounded, multimodal reasoning tasks; (ii) we construct mheb, a high-quality human benchmark of chinese art commentary annotated across five capability dimensions; (iii) we develop and evaluate persona-guided recontextualization interventions using eight expert personas and a domain-specific knowledge base; (iv) we demonstrate over 20% improvement in symbolic reasoning and over 30% improvement in argumentative coherence on gemini 2.5 pro using our proposed method; and (v) we establish the generalizability of our evaluation methodology to other epistemically rich domains such as religion, history, and education. together, our work highlights the need for new evaluation paradigms that go beyond benchmark metrics and toward measuring how well llms can adapt to the interpretive demands of real-world, interdisciplinary contexts. ## related work missing evaluation dimensions for cultural reasoning. despite significant advances in multimodal evaluation, current benchmarks primarily target factual understanding rather than cultural interpretation. existing benchmarks for large or multimodal language models, such as @xcite , emphasize factual accuracy or instruction following, seldom addressing symbolic interpretation or epistemic alignment. recent cultural evaluation efforts like m3exam @xcite and seed-bench @xcite , although primarily focused on multimodal understanding with limited cultural coverage, begin to incorporate cultural knowledge but focus on factual recall rather than interpretive reasoning. @xcite , for instance, evaluates stylistic generation but lacks formal metrics for interpretive depth. while prior work explores aesthetic reasoning @xcite , these studies rarely offer structured, multi-capability evaluation. our work addresses this gap by introducing cultural adaptability, operationalized through a multi-dimensional human expert benchmark with capability rubrics, enabling quantitative comparison in high-context domains like chinese art. limitations of persona conditioning without grounding. building on evaluation gaps, current persona-based approaches show promise but remain limited in cultural domains. persona use in llm evaluation shows promise for style control @xcite , yet most methods lack structured knowledge grounding, especially in epistemically rich domains. while recent work on role-playing @xcite and character conditioning demonstrates behavioral adaptation, these approaches often rely on surface-level stylistic changes rather than deep domain expertise. our method addresses this limitation by combining persona simulation with curated domain-specific knowledge to guide generation towards symbolic reasoning and cultural interpretation, not just stylistic alignment, offering a controlled intervention mechanism. ## methodology this research aims to comprehensively evaluate visual language models (vlms) capabilities in generating critiques for traditional chinese painting, assessing their understanding of image content, commentary quality, and adaptability to guided perspectives. the workflow involves: framework construction, developing a quantitative analytical framework from human expert commentaries, including defining evaluative dimensions and critic personas; vlm evaluation experiment design, creating structured protocols for vlm critique generation under conditions like persona-based and baseline prompting; and experimentation and result analysis, implementing experiments, collecting vlm critiques, and analyzing them with the developed framework to assess capabilities and intervention impacts. figure 1 provides an overview of this framework and its components. a cornerstone is the quantitative framework benchmark for vlm critiques, built upon human expert commentaries on chinese art. to ensure objective, reproducible, and fine-grained evaluation, an automated capability assessment framework was developed. this involves feature extraction, multi-dimensional capability scoring, profile assignment, and visualization, using a zero-shot classification model for fine-grained evaluative labels. the scoring covers painting element recognition, chinese painting understanding, and language usage, each with a dedicated rubric. this structured, rule-based approach enhances objectivity and facilitates large-scale benchmarking @xcite . ## results we present our results from semantic alignment, capability profiling, and the effects of persona-guided interventions on vlms. all evaluations are made with respect to the mheb, using both vector-space analysis and rubric-based scoring. ## conclusion this research introduced vulca, a quantitative framework for evaluating vlm-generated critiques of traditional chinese painting. our experiments demonstrate that persona and knowledge-based interventions significantly enhance vlm performance, achieving closer alignment with human expert standards. the study underscores the importance of culturally grounded approaches for developing vlms capable of nuanced engagement with specialized domains, paving the way for more sophisticated ai-assisted cultural analysis across diverse contexts. dedicating 15-20 minutes per commentary to ensure high-quality annotations. we acknowledge the palace museum beijing, national palace museum taipei, and the metropolitan museum of art for providing access to their museum catalogs and documentation of giuseppe castiglione's "twelve months" series, which formed the foundation of our expert commentary corpus. we also thank the art history scholars whose peer-reviewed publications and monographs on qing court painting provided essential domain expertise for this work. we acknowledge the use of ai-powered tools in this research. claude code assisted with code development and debugging throughout the experimental implementation. claude also provided english language refinement and editorial suggestions during the manuscript preparation. all scientific insights, experimental design, and final editorial decisions remained under full human control and responsibility.