MIMIC-Bench: Exploring the User-Like Thinking and Mimicking Capabilities of Multimodal Large Language Models
Authors: Jiajie Teng
Huiyu Duan
Sijing Wu
Jiarui Wang
Xilei Zhu
Jianing Jin
Wei Shen
Xiongkuo Min
Guangtao Zhai
Venue: ICLR 2026 Poster (2026)
Keywords: MLLMs
Benchmark Evaluation
User-Like Thinking
Comment Simulation and Discrimination
Instruction-Tuned Multimodal Models
Primary area: datasets and benchmarks

Abstract: The rapid advancement of multimodal large language models (MLLMs) has greatly prompted the video interpretation task, and numerous works have been proposed to explore and benchmark the cognition and basic visual reasoning capabilities of MLLMs.  However, practical applications on social media platforms demand MLLMs that can emulate user-like thinking and behavior when interpreting user-generated videos, which has been rarely studied in current research. To bridge the gap and get closer to general practical artificial intelligence (AI), we first construct **MIMIC-Data**, a large-scale dataset containing 150K+ user-shared videos with corresponding information including captions, tags, comments, *etc.*  Then, we present **MIMIC-Bench**, a large-scale benchmark building upon curated 4,000 user-shared videos from MIMIC-Data, which is designed to evaluate user-like thinking and mimicking capabilities of MLLMs in real-world video contexts.  MIMIC-Bench not only supports user-like thinking challenges including creator intent, user feedback interpretation, *etc.*, but also introduces a novel comment imitation task to assess whether MLLMs can generate human-like responses to video content.  Based on MIMIC-Data and MIMIC-Bench, we develop **MIMIC-Chat**, which integrates spatial and temporal features into a large language model, and finetunes the model to perform user-like thinking and mimicking tasks. Extensive experiments conducted based on 24 existing MLLMs and our MIMIC-Chat model show that current MLLMs exhibit limited capabilities to perform human-like thinking and responses, and MIMIC-Chat performs better to some extent.  We hope MIMIC-Bench can contribute to the advancement of human-aligned video understanding in the multi-modal era. The MIMIC-Data, MIMIC-Bench, and MIMIC-Chat will be released upon the publication.

Published as a conference paper at ICLR 2026
MIMIC-BENCH: EXPLORING THE USER-LIKE THINK- ING AND MIMICKING CAPABILITIES OF MULTIMODAL LARGE LANGUAGE MODELS
Jiajie Teng', Huiyu ; Wu', Jiarui Wang', Xilei Zhu', Jianing Jin', Wei Shen', Xiongkuo Min', Guangtao Zhai" Shanghai Jiao Tong University
ABSTRACT
The rapid advancement of multimodal large language models (MLLMs) has greatly prompted the video interpretation task, and numerous works have been proposed to explore and benehmark the cognition and basic visual reasoning ca- pabilities of MLLMs. However, practical applications demand MILMs that can emulate user-like thinking and behavior when interpret- ing seer-generated videos, which has been rarely studied in current research. To bridge the gap and get closer to general practical artificial intelligence (AI), we first construct MIMIC-Data. large-scale dataset containing 150K+ user-shared videos with corresponding information including captions, tags, comments, etc. Then, we present MIMIC-Bench, large-scale benchmark building upon curated 4,000 user-shared videos from MIMIC-Data, which is designed to evaluate user- like thinking and mimicking capabilities of MLLMs in real-world video contexts. MIMIC-BB Beich not only supports user-like thinking challenges including creator intent, user feedback interpretation, etc., but also introduces novel comment imi- tation task to assess whether MILMs can generate human-like responses to video content. Based on MIMIC-Data and MIMIC-Bench, we develop MIMIC-Chat, which integrates spatial and temporal features into large language model, and finetunes the model to perform user-like thinking and mimicking tasks. Extensive experiments conducted based on 24 existing MLLMs and our MIMIC-Chat model that current MILMS exhibit limited capabilities to perform human-like think- ing and responses, and MIMIC-Chat performs better to some extent. We hope MIMIC-Bench can contribute to the advancement of human-aligned video under- standing in the multi-modal era. The MIMIC-Data, MIMIC-Bench, and MIMIC- Chat will be released upon the publication.
social media platforms
INTRODUCTION
In recent years, video platforms have rapidly emerged as mainstream media for social interaction and knowledge sharing. Beyond passive viewing, users actively participate in shaping the video ecosystem through content creation, commenting, and engagement. A single user-generated video is often accompanied by rich metadata and viewer interactions, such as titles, descriptions, tags, and comments, etc., which reflect not only the video content but also how humans perceive, interpret, and respond to it. These contents embody high-level human cognition and social behavior, offering a rich yet underexplored resource for evaluating machine intelligence. Studying how multi-modal large language models (MLLMs) interpret such human-centric signals is crucial for advancing toward general-purpose AI systems that can truly understand and engage into social and cultural contexts.
Driven by the effectivvness of integrating multimodal information into large language models (LLMs) to acquire the scene perception ability (Radford et al., 2021; Li et al., 2021; Chen et al., 2020: Wang et al., 2022: Cho et al., 2021; Wang al., 2021), recent progress in MLLMs (Zhang et al., 2025; Bai et al., 2025; Gao et al., 2024; Chen et al., 2024a) have led to strong performance on tasks involving visual understanding. temporal reasoning. and open-ended language generation. Powered by scalable visual encoders and multi-frame alignment mechanisms, these models achieve impressive results on several benchmarks (Li al., 2024b; Fu al., 2024; Mangalam et al., 2023), etc., which assess the capabilities of MLLMs in video captioning, action recognition, and event prediction. However, these benchmarks typically rely on curated visual-only datasets and designer-
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Favorite Count
Shore Count
Comment
Like Count
MLLLA. MinlC-Chat (Curs)
Tags
Category
world
Description
#
Topic
Video
Imege
(+) All the informebion contained User Generated Video (U6C)
(b) User-like Thinking Task
(6) User Mimicking Task
Figure I: Overview MIMIC-Bench. (a) Left: Multi-source metadata from user-shared videos. (b) Center: User-like thinking task: across three axes CIU, CAM, and UIU. (c) Right: User-like mimicking task: human-like comment simulation pipeline.
crafted questions, focusing on low- or mid-level perception tasks that emphasize what happens in the video while neglecting how humans think, feel, and react. The lack of real-world user context and organically generated content limits their capacity evaluate whether MILMS can simulate human cognition and language behavior in authentic video scenarios. Although some studies have begun to explore emotions in multimodal data such as EmoLLM (Yang et al., 2024), etc., the practical applications are more complex than simple emotional tasks.
To bridge this gap, we first introduce MIMIC-Data, which comprises 150K+ user-shared videos and corresponding metadata information, as shown in Figure 1(a). Then, construct MIMIC- Bench, large-scale benchmark designed to evaluate user-like thinking and behavioral capabilities of MILMs in real-world, user-centric video applications. Unlike prior benchmarks that focus pri- marily on visual recognition and rely on synthetic or resigner-curated questions, MIMIC-Bench is grounded in real user-generated content and targets the human-like video understanding. i.e., how creators and viewers interpret, react to, and communicate about video content. As shown in Fig- ((b)(c), the benchmark consists of two components: (1) User-like Thinking Task that spans three structured reasoning axes, including Creator Intent Understanding (CIU), Content Attribute Matching (CAM), and User Interaction Understanding (UIU), and (2) User-like Mimicking Task that evaluates whether MLLMs can generate and identify human-lik comments on video content. Notably, MIMIC- shifts the focus from factual QA to human- cognitive tasks, espe- cially comment imitation, which directly evaluate whether MLLMs can approximate human-like reasoning and expression.
We also introduce MIMIC-Chat, multi-modal model designed to simulate human thinking and expression. Built upon the spatial and temporal visual encoders and a large language model back- bone, it is trained over 150k video samples from the training set of MIMIC-Dad enabling joint learning of video semantics and human-style responses. In summary, our key contributions are:
We construct MIMIC-Data, a large-scale dataset containing all metadata information for over 150K user-shared videos.
We propose MIMIC-Bench, the first large-scale benchmark designed to evaluate human- aligned reasoning and communicative behavior of MLLMs in practical video understand- ing. Especially, introduce novel comment imitation task that offers new lens to assess human- like thinking and mimicking capabilities of MLLMs.
We develop MIMIC-Chat, multi-modal model trained on 150k+ real-world samples, enabling the joint modeling of video semantics and social cognition.
We conduct extensive experiments on MIMIC-Bench, and compare the performance of 24 sate-offthe-art art MLLMs and our MIMIC-Chat. Results show that current MLLMs exhibit limited capabilities on MIMIC-Bench, and MIMIC-Chat performs better to some extent.
2

---

Published as a conference paper at ICLR 2026
Parent Childb-Child
Chinese Resthebic Traditanna Chicaie fation
Spontancous Captures
Beauty & Style
Sharing Fun Moments
9%
26 %
Daily
1. %
MIMIC Data
FO
Homanities Ares Denicel Paintery Caligraphy
Sports
13%
%
Pets
The pur paneery
Film & Editine
Nature Travel
days
Trerel Loys. Scanary Shaning
Mimic-Date Overview. We select 150,000+ videos, and each video is guaranteed to be of high quality and line with public aesthetics. The videos be categorized into categories (Daily Life can be divided into: Spontaneous Captures, Parent-Child, Sharing Fun Moments).
Figure 2:
RELATED WORK
2.1
MULTI-MODAL LARGE LANGUAGE MODELS FOR VIDEO UNDERSTANDING
Recent advances in large language models (LLMs) (Eysenbach et al., 2023; Chiang et al., 2023) have shown strong performance open-ended reasoning, contextual understanding, and instruction following (Radford et al., 2019; Brown et al., 2020; Ouyang al., 2022; Touvron et al., 2023). Building on this progress, multi-modal large language models (MLLMs) (Dai et al., 2023; OpenAl, 2023; Team et al., 2023; Zhu et al., 2023) have emerged jointly process visual and textual inputs. Early works (Alayrac et al., 2022; Chen et al., 2022) introduced vision- Lasion-language pretraining, while recent models (Liu al., 2023; Sun al., 2024) improved image-language alignment. As video becomes more prominent, MLLMs have been extended to handle temporal dynamics and multi- frame reasoning. Representative models like Video-LLaMA (Zhang al., 2023; Cheng al., 2024; Zhang et al., 2025), Intern VL (Chen et al., 2023; 2024b; Gao et al., 2024; Chen et al., 2024a), and the Qwen-V VL series (Bai al., 2023; Wang al., 2024a; Bai al., 2025) combine visual encoders with language backbones to perform well on video captioning. action localization, and Q&A, leveraging spatiotemporal pooling, contrastive learning. and dense alignment. Despite excelling factual un- derstanding. most MLLMs fall short in modeling human cognition. These gaps highlight the need reflecting real-world user intent and communication and motivates our work.
2.2 VIDEO BENCHMARKS AND EVALUATION PROTOCOLS
Numerous benchmarks (Fu al., 2023; Li al., 2023; Yu et al., 2023; Liu al., 2025; Ning al., 2023; Wang et al., 2024b) and datasets (Marino et al., 2019; Goyal et al., 2017; Song et al., 2024; Duan al., 2025; Xu al., 2025; Wang al., 2025a; Yang et al., 2025) have been developed to evaluate MI.MMs on video understanding. Early works (Jang et al., 2017; Chowdhury et al., 2018) focus on short-form video QA, testing recognition of visual entities and actions. Recent bench- marks (Li et al., 2024b; Fu et al., 2024; Mangalam et al., 2023; Wu et al., 2024) extend evaluation to multimodal alignment, instruction following, long-range modeling, and multi-turn dialogue. These resources have improved content-level evaluation, primarily assessing "what happens in the video" via structured queries, while overlooking how humans interpret, react to, or linguistically engage with videos. Most evaluations stress visual recognition but neglect emotional and linguistic en- gagement with videos, while EmoLLM (Yang et al., 2024) touches affective reasoning yet leaves human-style commenting, social interaction, and expressive realism underexplored. User-generated signals like comments, descriptions. and feedback are rarely included, despite their relevance on real-world platforms. This limits assessment of models' ability simulate human cognition and communication in natural contexts. In contrast, our benchmark incorporates real user content and authentic responses evaluate MILLMs from numan--liignn perspective.
BENCHMARK CONSTRUCTION
HIMIC-BENCH
In this section, we introduce MIMIC-Bench. benchmark for evaluating human -linnnd reasoning and communicative behavior in real-world, user-centrie video scenarios. It follows three prin- ciples: Realism-derriingg tasks from authentic human interactions rather than synthetic prompts; Task Orientation Orientation--desinnnin evaluations around multiple-choice and generation tasks with prac-
3

---

Published as a conference paper at ICLR 2026
tical grounding; and Human Alignment- -leasuint-measurringg models' ability simulate human cognition and expression in the video domain.
3.1 DATASET OVERVIEW
Data Source and Initial Collection. To construct MIMIC-Bench, we first collect over 150,000 user-generated videos from mainstream short video platforms (TikTok and YouTube), which form the foundation of MIMIC-Data. This dataset spans diverse topics such as lifestyle, education, travel, beauty, and humor (Figure 2). Each video is paired with metadata, including titles, tags, descriptions, comments, and interaction statistics, all sourced from publicly accessible content in compliance with platform policies. To ensure quality and usability, we applied initial filtering to remove videos with low resolution, excessive noise, or corruption, as well as those lacking mean- ingful metadata (e.g., empty titles, no comments, very short duration). We prioritized samples with rich viewer
meaaunee by comment volume and like count- mount-enuuing that selected videos reflect natural and diverse human responses. This high-quality pool forms the basis for both benchmark construction and model training.
We further examine the video duration distribution of MIMIC-Data, which serves the source pool for benchmark construction. Specifically, 32.76% of the videos are between 1-10 seconds, 49.08 59.08% fall within 10-45 seconds, 11.05% span 45-180 seconds, and 7.11% exceed 180 seconds in length. While MIMIC-Data predominantly consists of short-form videos, it also contains a non- trivial proportion of medium- and long-duration content. This diversity in video length ensures that the subsequent benchmark subset is grounded in a realistic distribution of user-generated video scenarios, rather than being restricted to narrowly defined short clips.
Multi-Source Metadata Composition. Each video is represented as a multi-modal, multi-field unit combining content with surrounding human interactions. Beyond the visual and audio signals, each sample includes user-level metadata: title (creator intent), tags and topic labels (categorical annotations), description (context or emotional framing), category, user comments (subjective im- pressions), and popularity indicators (like counts). These components capture not only what the video presents but also how creators frame it and how viewers respond cognitively and emotionally. Unlike datasets focused solely on video-text pairs, our structure reflects the full communication loop of online video ecosystems.
Benehmark Subset Construction. To build tasks, we further filtered the 150,000 videos using a ranking score based on engagement metrics (likes, favorites, shares), publisher influence, and thematic relevance. We selected the top 2% from TikTok and top 5% from YouTube, yielding 4,000 highly engaging and representative samples.
These videos combine high content quality with rich user interaction and serve as the core resource for constructing challenging, cognitively demanding evaluation tasks.
To avoid data leakage and memorization, MIMIC-Bench was constructed prior to model training through the above multi-stage filtering process. All benchmark videos are strictly excluded from the training set, and the remaining non-overlapping videos constitute the final MIMIC-Data used for training. As result, MIMIC-Bench and MIMIC-Data are disjoint at the video level and serve dis- tinct purposes, with the former used exclusively for evaluation and the latter for large-scale training.
To ensure reliability, we implemented a three-step quality control process: selecting highly interac- tive videos with rich viewer feedback, carefully designing distractors to avoid ambiguity, and con- ducting large-scale manual review of over 20,000 QA entries from 4,000 videos. This procedure guarantees accuracy, coherence, and unambiguous semantics across the benchmark. The distribu- tion of tasks reflects real-world user behavior rather than being artificially balanced, ensuring that the benchmark maintains ecological validity and mirrors authentic application scenarios.
Fairness and Validity To enhance fairness and temporal validity, we further filtered out samples that relied on ephemeral trends, niche memes, or culture-specific references. By retaining only general and widely comprehensible semantics-such as emotional tone, social context, and stylistic cues-MINE BIL--MMMIC-BL Benchh remains accessible across cultural backgrounds and stable over time.
All data collection strictly follows the copyright policies and terms of service of the respective plat- forms. We do not redistribute video files and only release benchmark annotations and metadata necessary for academic research. Additional details on copyright compliance, de-identification pro- cedures, exclusion criteria, and risk mitigation protocols are provided in Appendix F.
4

---

Published as a conference paper at ICLR 2026
3.2 TASK SUITE DESIGN
3.2.1 USER-LIKE THINKING TASK
To assess whether MLLMs can reason about video content in human-aligned way, we design seven single-choice tasks grouped into three axes: Creator Intent Understanding (CIU), Content Attribute Matching (CAM), and User Interaction Understanding (UIU). These tasks leverage rich metadata from MIMIC-Data to evaluate both perceptual and cognitive understanding. These tasks grounded in rich video metadata and evaluate both perceptual and cognitive understanding. De- tailed information about the MIMIC-BeNn benchmark, are provided in Appendix.
Creator Intent Understanding (CTU). This axis includes two sub-tasks. In Title Selection, the model selects the title most likely assigned by the creator, based on thematic and stylistic cues. In Description Selection, it identifies the description best aligned with the video's content and intent. Ground- come from original metadata; distractors are sampled from unrelated videos to ensure semantic contrast and minimize stylistic leakage.
Content Attribute Matching (CAM). This axis comprises three tasks. In Tag Matching, the model selects the tag that best captures content-level semantics. Topic Matching requires identifying the most relevant thematic label, while Category Matching asks for the correct predefined content category. All answers are derived from original annotations, and distractors are drawn from distinct videos with non-overlapping labels.
User Interaction Understanding (UTU). This axis focuses on modeling viewer behavior. In Com- ment Matching, the model selects the comment most likely reflecting genuine viewer feedback; the ground-trute the top-liked comment, and distractors come from unrelated categories. In Commens Popularity, the model chooses the most-liked comment among four from the same video (top-1, 1, top- 10, top-50, top-100), based on subtle linguistic and contextual signals. All tasks follow a four-way single-choice format with one correct answer. The design emphasizes semantic confusion, distractor diversity, and option balance, ensuring that task success reflects genuine understanding rather than superficial cues or memorization. While some metadata-based tasks might appear straightforward, the overall suite is cognitively demanding. In particular, comment-related tasks (e.g.. Comment Matching and Comment Popularity) require nuanced reasoning about social interaction, linguistic appeal, and collective preferences, making them non-trivial and discriminative.
The structured user-like thinking tasks in MIMIC-Bench are designed with varying levels of cogni- tive complexity. For clarity, we provide a more detailed discussion of task difficulty characteristics, along with representative challenging examples of the seven user-like thinking tasks, in Appendix D.
3.2.2 USKR-LIKE MIMICKING TASK
Beyond structured reasoning, we introduce a novel task- Comment Imitation to evaluate whether MILMS simulat human-like comments in social video contexts. This task emphasizes linguistic creativity, emotional expression, and social reasoning rather than factual correctness.
Task Motivation. As the second core component of MIMIC-Bench this task captures the imagi- native, affective, and context-aware nature of real viewer comments. It evaluates capability: gener- ating authentic, human-style comments. Although the task inevitably involves subjective judgment, we explicitly design the protocol and aggregation strategy to ensure reproducibility and comparabil- ity, transforming subjective perception into systematic evaluation dimension.
Task Setup. For each of the 4,000 benchmark videos, we collect the top-5 most-liked real com- ments. Each of the 24 MLLMss then generates one comment per video, producing a pool of 5 real and 24 model-generated comments, anonymized and randomly shuffled.
Human Evaluation. We conduct al human study using the same setup to provide a perceptual reference. These annotations serve as both performance upper bound and an indicator of which Al-generated comments "deceive" human judges.
Evaluation Metrics. We use two metrics: Imitation Quality, based on the percentage of com- ments judged as human and their average realism scores (0-5). Beyond serving as benchmark component, this protocol and its associated metrics also provide methodological contribution, offering a reusable framework for future studies on human-likeness evaluation multimodal gener- ation.
This task creates a unique generannn + judgment + scoring- a lens to assess models' social and linguistic alignment with humans in multimodal settings.
5

---

Published as a conference paper at ICLR 2026
leontin
A of
2et astill silee keny Solee da
Sporial Encesler
Temporal
LoRA_e B
LLM Talenizer
the fhow Evtre shety conieet for
Spotial Projector (ML))
Tempural Projector (MLF)
de painy currt pretasdiyy
V
and reep tertory de for foure 3 plas the mas surts
C. or
oud Nouer
LoRA_A
L1M (IntemLMM2CCht-BB))
LoRA_ 8
Try kocwr cach phus He wite pulltiong pirtbrn
dble alo
Trainable
(LM Decoder
Figure 3: Overview of the proposed MIMIC- Chat architecture. Given sampled video frames, spatial and tem- poral features are extracted using dual encoders and projected into language space via lightweight MLPs. The projected visual tokens and task specific instructions are fused and processed by LoRA- InternLM2- Chat-8B model. The unified interface supports both structured reasoning tasks (e.g., classification) and open- ended comment generation, enabling human-aligned video understanding.
4 MIMIC-CHAY
In this section, present MIMIC-Chat, unified multimodal framework tailored for human- aligned video understanding. The model supports both User-like Thinking Tasks and User-like Minicking Tasks in MIMIC-Bench, enabling assessment of perception, cognition, and linguistic behavior within a cohesive pipeline.
4.1 ARCHITECTREE OVERVIEW
MIMIC- MIMIC-Chat composed of three key components: a spatiotemporal video encoder, a task task-guided instruction formatter, and causal language model. This architecture supports both single-choice classification and free-form comment generation under unified input format.
The model takes video V and task- instruction T as input, and produces the output Y via:
Y = LM((VD)), (SPP)))
(1)
Here, (VV denotes visual tokens extracted and projected from the video, while [VTD] and [SEP] are special tokens marking the multimodal boundaries. This design enables multimodal fusion and instruction-folloiin through shared interface. It balances open-ended expressiveness with dis- criminative precision, making it suitable for all tasks MIMIC-Bench detailed architecture diagram is shown in Figure 3.
4.2
VIDEO ENCODING AND VISUAL TOKEN EXTRACTION
To convert raw video into structured multimodal input, we adopt a spatiotemporal encoder with a lightweight projection layer, ensuring compatibility with the language model backbone.
Frame Sampling and Preprocessing. We adopt dual-branch design: the spatial branch uni- formly samples 8 frames to capture scene-level cues, while the temporal branch consumes the full frame sequence to preserve fine-grained dynamics and causal context. All frames are resized and center-cropped to fixed resolution before encoding.
Spatiotemporal Encoding via TimeSformer. The processed frames are input to spatiotemporal encoding pipeline composed of a spatial encoder (operating on sampled frames) and a temporal encoder (operating on the full sequence), which together capture complementary spatial layouts and temporal dependencies. It outputs fixed-length token sequence:
((V) {VI, U2,..., Un,,. 0. E Rd
(2)

---

Published as a conference paper at ICLR 2026
MLP-Based Projection. We align visual tokens with the language space via a lightweight MLP:
MLP((;)
(3)
For simplicity, Eq, (3) shows a generic projection: in practice, we use distinct spatial and temporal projectors before fusion. The transformed sequence ((V) ensures effective vision-language inte- gration. We distinguish spatial projector (mapping tokens from the sampled-frame branch) and a temporal projector (mapping tokens from the full-sequence branch). Their outputs are subsequently fused within the language model via gated integration, ensuring complementary spatial-temporal reasoning.
Multimodal Input Construction. We concatenate the visual tokens <(V) with the natural lan- guage instruction T', using special tokens [VID] and [SEP] to delineate modalities. Each token is assigned modality-awan positional embeddings.
Language Model Integration. We employ InternLM-8B as the causal language model. It re- ceives the multimodal sequence and generates either a structured classification output or free-form textual response depending on the task type.
This video encoder module forms the foundation for MIMIC-Chat's high-level reasoning and gen- eration capabilities, providing a consistent visual representation for diverse understanding tasks. For a more detailed explanation of the model architecture and training con- figurations, refer to Appendix, where we provide additional insights on the system architecture and benchmark construction.
video
4.3 TRAINING DATA CONSTRUCTION - MIMIC-DATA
We build an instruction-tuning dataset named Mimic- Data from 150,000+ high-quality videos, each paired with multiple QA examples aligned with MIMIC-Bench tasks. This ensures consistency between training and evaluation in both structure and intent.
For the seven single-choice tasks (CIU, CAM, UIU), each sub-task is framed using a standardized prompt (e.g., "Based the video, choose the most appropriate tag/title/commmntt) followed by four candidate options. The correct answer guides the model to align visual-linguistic cues with se- mantic labels. For the comment generation task, we natural and imaginative comment for this video.") ") and provide the top-5 most-liked user com- ments references. This helps the model learn express emotional nuance, associative thinking, and stylistic variation.
unified prompt per video (e.g., "Generate
4.4 TRAINING OBJECTIVE AND OPTIMIZATION
We use supervised instruction tuning to train MIMIC-Chat on tasks, optimizing for accurate, human-like responses to video and instructions.
Training Loss. All training samples are cast as qucstion-answer pairs, allowing us to apply a standard language modeling loss:
Y Σ
LIM
log Plyt X, Vee)
(4)
This formulation unifies structured classification (e.g., choice "A") and open-ended generation (e.g, full comments) under single decoding objective.
Multi-task Learning. We employ a unified architecture without task-specific heads. Batches from different tasks are randomly sampled, enabling the model to generalize across diverse task types by interpreting the instruction and visual context.
Optimization Strategy. We tne-tune the full model except for the frozen visual backbone. Train- ing leverages standard techniques including mixed-precision computation, dropout, gradient clip- ping, and label smoothing for improved stability and generalization. This unified objective aligns spatial-temporal semantics with human-centria reasoning goals, supporting both discriminative and generative tasks in MIMIC-Bench
7

---

Published as a conference paper at ICLR 2026
Table I: Accuracy (%) on the seven reasoning tasks in MIMIC Bench, covering three cognitive axes: Creator Intent Understanding (TiS: Title Selection, DeS: Description Selection), Content Attribute Matching (TaM: Tag Matching, ToM: Topic Matching, CaM: Category Matching), and User Interaction Understanding (CoM: Com- Matching, CoP: Comment Popularity). Overall denotes the average accuracy across all tasks. The highest, second highest, and third-highest scores are highlighted in purple green and pink respectively.
Type
Overall
Tasks
ToM f
CaM
CoM
CoP f
Vide (Mazz al. 2024)
27.6
33.1
24.8
23.3
16.7
23.8
22.1
24
(1in Vad
2023)
27.0
41.2
68.3
32.4
17.0
24.6
25.8
3166
VideoCha12 (Li 202)0b)
37.7
33.2
30.1
66.1
34.2
32.9
26.7
33.6
MoXA-NoXT
23.5
VideoLLaMA2
36.1
27.1
VideoLLaMA3 (Zhang et 2025)
78.7
$8.0
78.6
82.0
19.5
51.3
28.3
58.7
MiniCPM-V (Yio ata.. 2024)
77.6
65.4
77.0
78.6
49.1
$5.0
28.9
592
MimLPM-a (Yo et 202.4)
71.1
00.7
84.8
833.5
50.2
54.2
32.0
59.5
CogVLM2 (llog
2024)
51.1
37.5
K3.6
70.3
55,0
41.9
29.9
502
Qwea2-V1 (2B)
48.6
47.3
44.3
Qwee2 (Wan) 2024a)
27.3
Qwen2.5 (3B) (Bai al.. 2025)
78.7
51.2
24.7
85.7
43,9
56.1
29.5
59.0
Qwen2 (7B))Wang (Wang et ct., 20246)
80.8
54.1
72.6
88.0
45.6
58.1
29.0
59.9
Qwen2.5. (722) (Wang cl. 20244)
85.5
79..3
79.8
93.3
50.5
67.3
33.1
66.7
IntemV1.2 (28) (Chenn al, 20124))
46.1
27.3
67.3
62.7
45.1
31.5
23.8
41.9
IntenVVL2
2024b)
52.8
Inten
72.9
51.1
306
Intemm112.5:44B) (Chen
20246)
79.2
33.3
85.2
53.7
57.4
28.7
60.
1 (8R) (Chen et al., 20246)
83.5
47.6
86.6
89.8
50.0
64.0
31.6
64.5
IntemVVseu2.5 (Wang al., 2025e)
83.2
71.7
K7.5
90.1
$3.5
64.4
32.7
66.3
3L (78B) (Zho al, 2025)
87.4
75.1
80.1
90.5
51.5
70.2
33.3
67.5
ChaGPT-4o (Achiam
2023)
87.9
80.3
51.3
70.9
33.5
682
Gemini2.5-pen (Comanici 2025)
a3 KEl-Kiskky al,, 2025)
93.2
$6.1
85.7
92.1
55.2
7744
455
74.6
Human
85.1
77.2
78.7
90.6
60.0
85.9
51.1
73.1
MIMIC-Chat(Oum
90.4
$7.1
86.7
92.5
55.7
78.3
43.6
74.1
Parameter-Efficien Tuning. Following (Hu et 2022), we integrate LoRA modules into select attention layers of InternLM-8B and jointly train them with the multimodal projector. This approach reduces memory usage while preserving performance. We further examine the contribution of the temporal encoder, spatial/temporal projectors, and LoRA through ablation studies Appendix.
EXPERIMENTAL EVALUATION
5.1 EXPERIMENTAL SETUP
We evaluate MIMIC-Chat and 24 baseline--including 21 open-source MLLMs and 3 powerful proprietary models well human participants on two key tracks in MIMIC-Bench: (1) User- like Thinking Tasks, which cover structured reasoning problems based on metadata; and (2) User- like Mimicking Tasks, which assess comment generation.
Implementation Details. For baseline models, we follow each model's official preprocessing strategy (including frame sampling and resolution). For MIMIC-Chat, we adopt the dual-branch design described Section the spatial branch uniformly samples frames, while the temporal branch processes the full frame sequence. Prompts follow the format defined in Section 4. Subjec- tive evaluation is conducted via web interface. Three independent annotators judged each comment on two dimensions: (i) whether it was human- Al-written, and (ii) realism score between 0 and 5. We summarize key ablation findings here and provide detailed results in the Appendix. Remov- ing the temporal encoder either projector leads substantial performance drops, while excluding LoRA causes consistent but smaller degradation.
Regarding inference efficiency, MIMIC-Chat requires approximately 24 GB GPU memory during evaluation and takes about 5-10 seconds to process a 20-second video clip on a single GPÚ.
5.2 USKR-LIKE THINKING TASK PERFORMANCE
We evaluate all participants on the seven structured reasoning tasks in MIMIC-Bench, spanning three axes: creator intent understanding (CIU), content attribute matching (CAM), and user interaction understanding (UIU). Accuracy is reported per task and averaged across all subtasks. Results are summarized in Table 1.
Human participants achieve an overall accuracy of 73.1%, maintaining strong consistency across all reasoning axes and setting the upper bound on UIU tasks, particularly in Comment Matching (85.9%) and Comment Popularity (51.1%). MIMIC-Chat, our fine-tuned model, attains an overall accuracy of 74.1 %,, ranking third among all participants-stighe behind two frontier proprietary
8

---

Published as a conference paper at ICLR 2026
models (Gemini-2.5-PPr at 75.1% and o3 at 74.6%) but surpassing all spen- baselines, includ- ing very large ones such as Qwen2.5-VL-725 (66.7%) and InterVL3-788B (67.5 (67.5%). MIMIC-Chat achieves top-three performance across nearly all subtasks, with notable strength in creator intent understanding (TiS 90.4%, DeS 87.1 %) and content attribute matching (CaM 55.7%).
Despite these gains, UIU remains the most challenging axis: while MIMIC- Chat improves substan- tially over prior open models, its performance on Comment Popularity (43.6%) still lags far behind human annotators. This highlights the intrinsic difficulty of modeling user-centric semantics and interaction cues.
A closer comparison across task types reveals that this gap is not primarily caused by perceptual limitations. Most models perform relatively better on perceptual or surface-aligned tasks (e.g.. Tis and ToM), but struggle on tasks requiring higher-level social-cognitive reasoning (e.g.. CaM and CoP), such as inferring human intent, emotion, and sociocultural cues beyond literal content. This pattern indicates that current failures stem more from insufficient human-centered commonsense reasoning than from visual understanding itself.
Overall, the results confirm that human-aligned fine-tuning enables MIMIC-Chat to close the gap with, and in some cases surpass, much larger models, underscoring the importance of task-specific alignment rather than sheer model scale alone.
5.3 USER-LIKE MIMICKING TASK PERFORMANCE
We evaluate each model's ability to generate human-like comments through comment simulation task. Table 2 summarizes the results. Here, the first column (Models) denotes the comment sources: each row reports evaluation results for comments generated by the corresponding model, while the Human row represents real user comments originally collected from video-sharing platforms. Ad- ditional details of the annotation protocol and bias mitigation strategies for the comment mimicking task are provided in Appendix D.5.
Comment Simulation. For each video, all models generate one comment, which is then evaluated by human annotators. The simulation quality is measured in three ways: (1) the percentage of model's comments judged as "human": (2) the distribution of realism scores (0-5); and (3) the mean realism score per model. Each comment is anonymously presented to multiple human evaluators, who are asked to determine whether it was written by a human or an AI, and to rate its human- likeness on 0-5 scale. Inter-annotator agreement reached 91.95%, calculated as the proportion of samples on which at least two annotators gave consistent judgments. This high level of agreement indicates that the evaluation is stable and less susceptible to individual annotator bias. The "Judged as Human" metric reflects the proportion of model's comments that were classified as human- written. The "Score@k" distribution indicates the percentage of all comments receiving score k and "Mean Score" reports the overall average realism score across the model's 4,000 generated comments.
As shown in the table, MIMIC-Chat significantly outperforms all baseline models in the key met- ric of being judged as human (64.24%), more than triple most existing models, and second only to real human comments (87.57%). Its average realism score (2.88) is also notably higher than that of other models. These results suggest that MIMIC-Chat produces more natural, human-aligned com- ments in both linguistic style and semantic coherence. These findings highlight that MIMIC-CH -Chtt achieves the most human-aligned comment generation performance among all evaluated models.
We attribute this advantage fundamental behavioral difference: most baseline models tend to generate literal or content-descriptive comments-suchh as rephrasing video scenes or restating vi- sual facts-which are easily recognized as Al-written by human annotators. In contrast, MIMIC- Chat demonstrates stronger capacity for simulating human-like like thinking patterns, often exhibit- ing signs of divergent, associative, or emotionally reflective reasoning- ttriinnggttttss more characteristic of genuine human responses in open-domain video discussions. Beyond open-source baselines, the inclusion of stronger proprictary models (e.g., ChatGPT-40, Gemini2.5-pre and 03) reveals a narrowing gap in human-likeness. These models achieve substantially higher realism scores and human- judgment rates than most open-source MLLMs. Nevertheless, MIMIC-Chal remains the most competitive among open-source systems and still exhibits notable margin over proprietary counterparts in the "Judged as Human" metric. Taken together, these results indicate that while re- cent closed-source models are improving rapidly in simulating human-like comments, there remains
9

---

Published as a conference paper at ICLR 2026
Table 2: Evaluation of comment simulation across participants, based on human judgments. The results include human- -inean- classification rates, realism score distributions (0- and mean scores. The highest, second highest, and hirdht scores are highlighted in purple green and pink respectively.
Commuctt Simuslation
Models Coodebs/iiee 'Tisk
Judged as
Judged
Score Distribution( %)
Moss
Human (%6
A1 1%6)
Score
Somee0
Semr@1
Scar
Score
Scoe884
Sccre
VideoChaGGP 2024)
Video-LLaVA
6.30
95.70
63.16
25. 48
2,07
2.44
1.97
VideoChar2 (1l ot 2024b)
21,02
78.95
53.66
21.54
3.79
9,20
7.70
3.92
1.08
LaVA-NeeT (1. ot, 2054g)
7.02
92.98
63.20
27 53
2.12
2.93
1.90
2.27
0.60
VideolLaMAA (Cheng
20144)
2,70
97.30
73.57
20.37
1.23
0.31
1.56
0.94
Video (Zhang
1.28
MiniCPM-V V (Yao et 2024))
8.17
91.83
61.78
26.98
2.99
2,64
2.37
3.25
0.67
MiniCPM-o Yao et al, 2024)
6.85
93.15
06.43
24.00
2.11
1.25
2,81
2.81
0.58
CogVLM2 (Hong et.l., 2024)
7,50
92.50
68.20
21 40
2,50
4,80
1,70
1.10
0.54
Qwes2-V1. (2.a2))1.(2R)) (Wing et at. 2034a)
2.30
2,31
2,16
Qwen2-VL (Wang
2024s)
6.99
2/40
Qwen25-VL (Bal et 2025)
14.18
85.82
59.61
23.76
2.38
3,93
4.83
5.49
0.87
Qwes2.5 (VB) al., 2025)
10.25
89.75
67.76
20.73
1.19
2,50
3.27
4,46
0.66
InteraVL2 (28) (Chen et al. 20246)
1.03
98.97
$0.59
17.66
0.56
0.29
0.29
0.51
0.24
Iatera (48) (Chon 20240)
77.14
17.70
IstereV1.2
26. 36
[nternVL.2.5 (4B) (Chen 2024g)
16.26
83.74
54.24
26. 58
2.93
4.30
5,70
6.16
0.99
[n2eraVL2255 (8) (Chen el al. 2024a)
10.90
89.10
59.47
26.54
2,86
2.56
4,21
4,36
0.79
Videc2 (Wang et al., 2025b)
12.67
87.33
55.58
27.60
3.77
4,45
4,45
4.15
0.87
Qwea25-VL (728) at. 2025)
26.93
42.59
IateraVL3
al.,
41.23
18.87
5.48
13.85
ChariPT-4o (Achiam et 2023)
40.52
59.68
28.65
21.32
9.71
10.91
17.64
11.77
2,03
Gemini2,5-pe (Comanici al, 2025)
41.87
58.13
29.51
16.84
11.78
9.97
18:31
13.59
2.12
a3 (EI- Kishky al., 2025)
43.36
56.64
28.68
19.71
8:25
11.33
19.56
12:47
2.11
MIMIC-Chal (ours)
64.24
7.62
15.23
26.82
17.23
2,88
a sizable gap compared to genuine human responses, underscoring the challenge of fully capturing the subtleties of human reasoning and expression.
To ensure fairness, we also fine-tuned several strong open baselines on MIMIC-Data under identical settings. While their performance improved, they still lagged behind MIMIC- suggesting that our advantage stems not merely from access to task-aligned data but from the proposed architecture and training strategy. Full results are reported in Appendix.
In addition, to examine potential biases introduced by engagement- based video selection and as- sess robustness beyond the benchmark distribution, we conduct additional out-of-domain and cross- benchmark evaluations. Detailed analyses and results are reported in Appendix E.
CONCLUSION
We presented MIMIC-Data and MIMIC-Bench, large-scale dataset and benchmark for evaluat- ing human-aligned video understanding, and introduced MIMIC-Chat, a model trained to bridge the gap between perception and human-like reasoning. Across 24 competitive MLLMs, results show that while recent systems advance perception, they still fall short in mimicking human thought and expression. MIMIC-Cha narrows this gap, setting a new state of the among open-source mod- els. We believe this work opens new direction toward socially and cognitively aligned multimodal intelligence.
ACKNOWLEDGEMENT
This work supported in part by the National Natural Science Foundation of China under Grants 62401365, 62225112, 62271312, 62132006. U24A20220, and in part by the China Postdoctoral Science Foundation under Grants BX20250411 1, 2025M773473.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our work adheres to the ICLR Code of Ethics. This work introduces MIMIC-Bench, benchmark designed to evaluate multimodal large language models (MLLMs) on human-aligned video under- standing tasks, and MIMIC-Chat, a model fine-tuned for this purpose.
All videos and metadata in MIMIC-Data were collected exclusively from publicly accessible sources. We only include content that is publicly available at the time of collection and do not access private, restricted, or paywalled material. We respect the copyright policies and terms of service of the hosting platforms. The benchmark does not redistribute raw video files; instead, we release annotations, benchmark splits, and associated metadata necessary for research evaluation. Users are required to retrieve original content directly from the respective hosting platforms un- der their applicable terms of service. We also provide a contact channel to accommodate content removal requests if needed.
To protect privacy, we exclude personally identifiable information (PII) during data collection and curation. We do not release uploader identities, user handles, account IDs, contact information, or other identifying metadata. Content involving private individuals, minors, sensitive personal circumstances, identifiable private locations is excluded based predefined filtering criteria. Both automated filtering rules and manual review procedures are applied to minimize the inclusion of sensitive or personal content.
To mitigate risks of reputational harm, we exclude videos or annotations that contain defamatory statements, targeted harassment, unverified allegations, or highly sensitive personal topics (e.g., medical conditions, criminal accusations, political attacks directed at specific individuals). The dataset is intended strictly for academic research and benchmarking purposes. It designed for profiling, surveillance, or other applications that could adversely impact individuals or communities.
Further details on dataset collection procedures, copyright compliance measures, de-identification rules, exclusion criteria, and responsible release protocols are provided in the Appendix.
REPRODUCIBILITY STATEMENT
We have made significant efforts to ensure the reproducibility of work. The main paper details the construction process of MIMIC-Bench, the task definitions, and the evaluation protocols. We describe the design of MIMIC-Chat, including its dual-branch spatial-temporal encoder, training strategy, and optimization settings. Appendix C and D provide further details on model architec- ture, training configurations, dataset construction, and visualization examples. We plan to release the benchmark data splits, evaluation code, and MIMIC-Cha training scripts upon publication to facilitate replication and follow-up up research.
LLM USAGE STATEMENT
Large Language Models (LLMs) were used only as part of the evaluation in our benchmark, where 24 baseline models (both open-source and proprietary) were compared against our proposed MIMIC-Chat. L.MMs were not used to generate any dataset samples, annotations, or experimental results. All dataset construction, annotation, and analysis were conducted independently by the au- thors. We used LLMs in limited capacity for language polishing of the manuscript, but no section of the research design, data collection, or core analysis was generated by LLMs. The authors take full responsibility for the content of this paper.
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.087.. 2023.
Jean- Ctan-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: visual language
11

---

Published as a conference paper at ICLR 2026
model for few-shot learning. Advances in Neural Information Processing Systems, 35:23716- 23736, 2022.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, local- ization, text reading, and beyond. arXiv preprint arXiv:2308. 12966, 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint ar 2r0.i:2502. 139233 2025.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Xi Chen, Xiao Wang. Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Gryener, Basil Mustafa, Lucas Beyer, al. Pali: A jointly-scaled multilingual language-image model. arXiv preprint arXiv:2209.066 06794, 2022.
Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. In ECCV, 2020.
Zhe Chen. Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing. Muyan Zhong, Qing- long Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. In- ternvl: Scaling up vision foundation models and aligning for generic visual- lingu--in tasks. arXiv preprint arXiv:2312. 14238, 2023.
Zhe Chen, Weiyun Wang. Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Yc, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024a.
Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, al. How far are we gpt-4v? closing the gap com- mercial multimodal models with open-source suites. arXiv preprint arXiv:2404. /682/, 2024b.
Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and Lidong Bing. Videollama 2: Advancing apatial-tempord modeling and audio understanding in lids arXiv preprint 1.Xi::2406007466 2024. URL https: arxii.iii.....vv. orgg alss/ 2406. 07476.
Wei-Lin Chiang, Zhuohan Li, 7i Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna. Imsys. org (accessed 14 April 2023), 2(3):6, 2023.
Jaemin Cho, Jie Lei, Hao Tan, and Mohit Bansal. Unifying vision-and-language tasks via text generation. In ICML, 2021.
Muhammad Iqbal Hasan Chowdhury, Kien Nguyen, Sridha Sridharan, and Clinton Fookes. Hi- erarchical relational attention for video question answering. In 20/8 25th IEEE International Conference on Image Processing (ICIP), pp. 593-603. IEEE, 2018.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.06261 2025.
Wenliang Dai. Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li. Pascale Fung, and Steven Hoi. Instructblip: Towards general- purpose vision-language models with instruction tuning. arXiv preprint arXiv:2305.06550 2023.
12

---

Published as a conference paper at ICLR 2026
Huiyu Duan, Qiang Hu, Jiarui Wang, Liu Yang, Zitong Xu, Lu Liu, Xiongkuo Min, Chunlei Cai, Tianxiao Ye, Xiaoyun Zhang, et al. Finevq: Fine- greeggraied user generated content video quality assessment. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp- 3206-3217, 2025.
Ahmed El- Ei-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, al. Competitive programming with large reasoning models. arXiv preprini arXiv:2502.0680) 2025.
Gunther Eysenbach et al. The role of chatgpt, generative language models, and artificial intelligence in medical education: a conversation with chatgpt and a call for papers. JMIR Medical Education, 9(1): (1))e46888, 2023.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang. Xiawu Zheng, Ke Li, Xing Sun, and Rongrong Mme: A comprehensive evaluation benchmark for multimodal large language models. ArXiv, abs/2306. (aN/2306.13394, 2023.
Chaoyou Fu, Yuhan Dai, Yondong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, et al. Video-mme: The first-ever comprehensive evalua- tion benchmark of multi-modal Iims in video analysis. arXiv preprint arXiv:2405.21077 2024.
Zhangwei Gao, Zhe Chen, Erfei Cui, Yiming Ren, Weiyun Wang, Jinguo Zhu, Hao Tian, Shenglong Ye, Junjun He, Xizhou Zhu. et al. Mini-internvl: A dexible-transfer pocket multimodal model with 5% parameters and 90% performance. arXiv preprint arXiv:2410. 2024.
Yash Goyal, Tejas Khot, Douglas Summers-Stay. Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 6904 6913, 2017.
Wenyi Hong, Weihan Wang, Ming Ding, Wenmeng Yu, Qingsong Lv, Yan Wang, Yean Cheng, Shiyu Huang, Junhui Ji, Zhao Xue, et al. Cogvlm2: Visual language models for image and video understanding. arXiv preprint arXiv:2408. 16500, 2024.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL https: http//e/eeeeeeeeeeeeeeeeeeeeeeee https:://penreviiww id=nzevKeeFYEB.
net forum?
Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. Tgif-qa: Toward spatio- temporal reasoning in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2758-2766, 2017.
Li, Hao Zhang, Kaichen Zhang, Dong Guo, Yuanhan Zhang, Renrui Zhang, Feng Li, Ziwei Liu, and Chunyuan Li. Llava-next: What else influences visual instruc- tion tuning beyond data?, May 2024a. URL https:/ https:://llaaaalll.. github io/blog/ 2024-05- 25- 1lava next ablations/.
Bohao Li, Rui Wang, Guangzhi Wang. Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench. Bench- marking multimodal Iims with generative comprehension. ArXiv, abs/2307.16122, 2023.
Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven
Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. NeurlPS, 2021.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang. Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mybench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 22195-22206, 2024b.
Bin Lin, Bin Zhu, Yang Ye, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. arXiv preprint 2.X1122311.10122, 2023.
Haotian Liu, Chumyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv
preprint ar arXiv:2304.084. 08485, 2023.
13

---

Published as a conference paper at ICLR 2026
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European Conference Computer Vision, pp. 216-233. Springer, 2025.
Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), 2024.
Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. Egoschema: A diagnostic bench- mark for very long-form video language understanding. Advances in Neural Information Process- ing Systems, 36:46212--405 46244, 2023.
Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. Proceedings of the IEEEJeyf conference on computer vision and pattern recognition, pp. 3195-3204, 2019.
Munan Ning, Bin Zhu, Yujia Xie, Bin Lin, Jiaxi Cui, Lu Yuan, Dongdong Chen, and Li Yuan. Video-bench: A comprehensive benchmark and toolkit for evaluating video-based large language models. arXiv preprint
2023.
OpenAI. Gpt-4 technical report. ArXiv, abs/2303.08774, 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal. Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730-27744, 2022.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog. 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In ICML, 2021.
Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token sparse memory for long video understanding. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition. pp. 18221-18232, 2024.
Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu, Yueze Wang, Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multimodal models are in-context learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 14398-14409, 2024.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312. /800, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307. 099888 2023.
Jiarui Wang, Huiyu Duan, Juntong Wang, Ziheng Jia, Woo Yi Yang, Xiaorong Zhu, Yu Zhao, Jiay- ing Qian, Yuke Xing, Guangtao Zhai, et al. Dfbench: Benchmarking deepfake image detection capability of large multimodal models. In Proceedings of the 33rd ACM International Conference on Multimedia, pp. 12666-12673, 2025a.
Peng Wang. An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. Ofa: Unifying architectures, tasks, and modalities through a simple peuuence-to-sequnee learning framework. In ICMI., 2022.
14

---

Published as a conference paper at ICLR 2026
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-langue model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024a.
Weihan Wang, Zehai He, Wenyi Hong. Yean Cheng, Xiaohan Zhang, Ji Qi, Shiyu Huang, Bin Xu, Yuxiao Dong, Ming Ding, et al. Lvbench: An extreme long video understanding benchmark. arXiv preprint arXiv:2446.08000 08035, 2024b.
Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xiangyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, Min Dou, Kai Chen, Wenhai Wang, Yu Qiao, Yali Wang. and Limin Wang. Internvideo2.5: Empowering video milms with long and rich context modeling. arXiv preprint arwiv:2501. 2025b.
Zirui Wang, Jiahui Yu, Adams Wei Yu, Zihang Dai, Yulia Tsvetkow, and Yuan Cao. Simvlm: Simple visual language model pretraining with weak supervision. arXiv:2108. 10904, 2021.
Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B Tenenbaum, and Chuang Gan. Star: A benchmark for situated reasoning in real-world videos. arXiv preprint 2405::240,.0771,, 2024.
Zitong Xu, Huiyu Duan, Bingnan Liu, Guangji Ma, Jiarui Wang, Liu Yang. Shiqi Gao, Xiaoyu Wang. Jia Wang, Xiongkuo Min, et al. Lmm4edit: Benchmarking and evaluating multimodal im- age editing with Imms. In Proceedings of the 33rd ACM International Conference Multimedia, pp. 6908- 6918-6917.220 2025.
Liu Yang, Huiyu Duan, Ran Tao, Juntao Cheng, Sijing Wu, Yunhao Li, Jing Liu, Xiongkuo Min, and Guangtao Zhai Odi-bench: Can milms understand immersive omnidirectional environments? arXiv preprint arXiv:2510. 1549, 2025.
Qu Yang, Mang Ye, and Bo Du, Emollm: Multimodal emotional understanding meets large language models. arXiv preprint arXiv:2406.1644 2024.
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Miniepm-v: A gpt-4v level milm your phone. arXiv preprini 240i::2008.01800 2024.
Weihao Yu, Zhengyuan Yang, Linjie Li. Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. ArXiv, 0ls/2308.004900, 2023.
Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang. Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Lidong Bing, and Deli Zhao. Vidcollama 3: Frontier multimodal foundation models for image and video under- standing. arXiv preprint 2xXi:2501113100 2025. URL //ar///aavv/a////aa//eaa org abrgg/bbs/255 2501. 13106.
Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual lan- guage model for video understanding. arXiv preprint arXiv:2306.02858 2023, URL https:
//arxiv. wwt.abs..//...e....o..o........a /2066 02858.
Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, HongFa Wang, Yatian Pang, Wenhao Jiang, Junwu Zhang, Zongwei Li, et al. Languagebind: Extending video-language pretraining to n- modality by sanguage-based semantic alignment. arXiv preprint arXiv:2310.01852 2023.
Jinguo Zhu, Weiyun Wang. Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprini arXiv:2504. 10479, 2025.
15

---

Published as a conference paper at ICLR 2026
A ABLATION STUDIES
A.I ABLATION SETUP
To isolate the contribution of each key component in MIMIC-Chat, we perform controlled abla- tions under unified training and evaluation pipeline. Unless otherwise specified, all settings strictly follow the main paper ($4-$5); same training data (MIMIC-Data), instruction format, loss, opti- mizer, batch size, number of epochs, inference hardware (A6000 GPUs), and evaluation metrics. For thinking tasks we report accuracy on the seven multiple-choice subtasks; for mimicking we report human-judged comment simulation metries (Judged as Human, Score @ k. Mean Score). We do not include cource-identification results in the appendix.
Backbone and Inputs (constant across variants). We use the full dual-branch video pathway described in $4 as the reference ("Full"): spatial branch that uniformly samples 8 frames for scene- level cues, and a temporal brunch that processes the full frame sequence for fine-grained dynamics. Visual tokens from both branches projected by branch- branch-specific MLP projectors and fused into the language model with gated integration. The causal LM is InternLM-SB with LoRA modules enabled in the Full configuration.
Ablated Variants. We construct four ablation variants by toggling one component at time while keeping all other factors identical:
w/o LoRA (No-LoRA): Disable all LoRA adapters in InternLM-SB and freeze the LM parameters; only the multimodal projector(s) remain trainable. This tests the role of parameter-efficien language adaptation for instruction alignment.
w/o Temporal Encoder (No-TempEnc): Remove the temporal branch (full-sequence pro- cessing); the model uses only the spatial branch with 8 uniformly sampled frames. This probes the importance of explicit temporal modeling.
w/o Temporal Projector (No-TempProj): Keep both branches but remove the temporal- specific projector; temporal tokens are routed through the spatial projector (shared MLP) before fusion. This examines whether a dedicated temporal projection space is necessary.
w/o Spatial Projector (No-SpatProj): Keep both branches but remove the projector; spatial tokens routed through the temporal projector (shared MLP). This complements the previous variant and tests sensitivity to projector specialization.
Fairness Controls. All variants use the same prompts and decoding settings as the Full model. Frame sampling, resolution, and preprocessing follow $5. When branch is removed (e.g., No- TempEnc), the remaining branch and its projector are unchanged; when a projector is removed (c.g., No- Vo-TempProj/NNo-- Spattrro),,,, tokens passed through the remaining projector to keep token dimen- sionality and downstream interfaces intact. This design ensures that any performance difference can be attributed the ablated component rather than confounds in optimization or data.
A.2 QUANTITATIVE COMPARISON ON THINKING TASKS
We quantify the contribution of each core component the seven multi-choice tasks (CIU: Tis/DeS; CAM: TaM/ToM/CaM: UIU: CoM/CoP). All settings (data, preprocessing, prompts) are held fixed in the main experiments; only the indicated module is removed or altered.
Key observations. The ablation results reveal that both temporal modeling and LoRA-based adap- tation are indispensable. Removing the temporal encoder causes the largest overall drop (74.1 -> 65.8; -8.3 pp). with especially severe declines in DeS (-33.8 pp) and UIU tasks (CoM: - 12.9 pp. CoP: -11.0 pp), underscoring the need for end-to-end temporal reasoning. LoRA adaptation equally critical: ablating it reduces Overall to 66.9 (-7.2 pp), driven by sharp declines semantics- heavy tasks such as DeS (-22.7 pp). TaM (-16.3 pp), and both UIÚ subtasks. The temporal pro- jector further contributes complementary gains, as its removal lowers Overall to 67.5 (-6.6 pp) and disproportionately weakens CAM and UIU performance, while the spatial projector supports static semanties, with its absence (Overall 68.7; -5.4 pp) most affecting DeS (-18.9 pp). Together,
16

---

Published as a conference paper at ICLR 2026
Table Accuracy (%) on seven structured reasoning tasks after ablating components MIMIC-Chat. CIU includes Title Selection (TiS) and Deseription Selection (DeS); CAM includes Tag//opic/Cateegry Matching (TaM/ToM/CaM)); UIU includes Comment Matching (CoM) and Comment Popularity (CoP). Overall is the average over all seven tasks. The full model's scores are highlighted in purple
Task Type
CIU
CAM
UIU
Overall
Models/ Tasks
TIS
DeS
TaM
ToM
CaM 1
CoM
CoP1
MIMIC-Chat (full)
90.4
87.1
86.7
92.5
55.7
78.3
43.6
74.1
w/o LoRA
88.3
64.4
70.4
89.8
47.3
68.2
34.1
66.9
wlo Temporal Encoder
$3.3
87.5
89.5
51.4
65.4
32.6
65.8
w/o Temporal Projector
75.4
72.8
48.6
70.3
34,5
who Spatial Projector
89.3
68.2
84.2
91.7
50.2
72.6
35.1
68.7
these findings highlight that temporal components drive interaction- and context-sensitive reasoning. while LoRA secures treator-intent and textual alignment, and only their integration allows the full model to achieve a balanced advantage across CIU, CAM, and UIU.
A.3 DISCUSSION
The ablation results highlight several broader implications. First, temporal modeling and LoRA- based language adaptation are complementary: the former underpins interaction- and context- sensitive reasoning (UIU, CAM), while the latter ensures fine-grained textual alignment (CIU, DeS/TaM). Second, different task categories stress distinet modalities-CII benefits most from semantic adaptation, whereas UIU requires strong temporal grounding-indiiating that balanced multimodal integration is essential for generalizable video understanding. Finally, the consistent superiority of the full model suggests that parameter-efficient language tuning and temporally- aware encoding are not just additive improvements, but jointly critical for bridging the gap between perception -riive reasoning and socially aligned interpretation, reinforcing the design principles behind MIMIC- Chat.
B FINE-TUNING OTHER MODELS
B.1 SETUP
To assess whether the improvements of MIMIC-Chat stem solely from access to MIMIC-Data, we fine-tuned several strong video-language models under identical conditions. Specifically, we se- lected three representative backbones covering different architectures and scales: Qwen2.5-VL-7B, VL.255B.. and Intern Video2.5-8B. These models were chosen because of their strong base- line performance and wide adoption in the community.
For faimess, all models were fine-tuned on the same training split of MIMIC-Data that was used to train MIMIC-Chat, and evaluated on the official test set of MIMIC-Bench. The fine-tuning procedure followed a unified setup:
Data. The full MIMIC-Das training split was used without task-specific resampling. All structured and generative tasks share the same preprocessing pipeline.
Optimization. We employed AdamW optimizer with learning rate of 2 x 10-5, cosine decay, and batch size of 128. Training was run for epochs with early stopping on the validation set.
LoRA. For parameter-efficient adaptation, LoRA modules were applied to the language backbone of each model, while vision encoders were kept frozen. This ensured efficiency and comparability across different backbones.
Evaluation. All models
evaluated the seven structured reasoning tasks (CIU, CAM, UIU) and the mimicking tasks, using accuracy for multi chotiechoiie and metrics for generative tasks.
17

---

Published as a conference paper at ICLR 2026
Table 4: Accuracy (%) on the seven structured reasoning tasks after linettuning existing models on MIMIC- Data. CIU includes Title Selection (TiS) and Description Selection (DeS); CAM includes Tag/topic/Category Matching (TaM/ToM/CaM); UIU includes Comment Matching (CoM) and Comment Popularity (CoP). Over- all is the average over all seven tasks. The full model's scores are highlighted in purple
Tisk Type
CIU
CAM
UIU
Overall
Models/ / Tasks
TiS
DeS
TaM f
ToM
CaM
CoM 1
CoP
DWe-2.5-VLL-7B VL-7B
80.8
54.1
72.6
88.0
43.6
58.1
29.0
59.9
IntemVL2.5-8B
83.5
47.6
86.6
89.8
50.0
64.0
31.6
64.5
IntemVideo2.5-88
83.2
71.7
87.5
90.1
53.5
64.4
32.7
66.3
fine-huned on MIMIC-Dota
Qwen2.5-VL VB- (1)
85.1
60.2
80.1
89.7
46.8
64.7
32.3
66.4
IntemVLL2.5888 (f))
85.6
87.5
89.5
51.4
65.4
32.6
Intern Vntemiideo2.5550 (ft)
87.5
76.2
90.3
91.3
51.3
66.9
33.1
68.1
MIMIC-Chat (Ours)
90.4
87.1
86.7
92.5
55.7
74.3
43.6
74.1
This setup ensures that any observed performance differences are attributable to model architecture and adaptation capacity, rather than data imbalance training procedure.
B.2 RESULTS
We report the performance of fine-tuned models compared with MIMIC-Chat across both structured reasoning and mimicking tasks.
Key findings. Fine-tuning MIMIC-Data consistently improves all baseline models, with Qwen2.5-VL.cn VL-7B gaining from 59.9 to 66.4 overall accuracy and Intern Video..8dd2 rising from 66.3 to 68.1. The strongest gains appear in semantics-heavy tasks such as DeS (e.g., +6.1 for Qwen2.5- and TaM (+7.5), highlighting the value of domain-specific alignment. Never- theless, MIMIC-Chat remains clearly ahead across nearly all tasks, especially in DeS (87.1 vs. 76.2 for the best fine-tuned baseline) and CoP (43.6 vs. 33.6), confirming that its superior architecture and training design contribute substantially beyond data fine-tuning alone.
B.3 DISCUSSION
The fine-tuning experiments demonstrate that existing MILMs, when adapted to MIMIC-Data, can indeed improve their performance on user-centric reasoning tasks. Models such as Qwen2.5-VL- 7B and InternVidco2.55. show consistent gains in both creator-intent understanding (c.g., DeS) and content-attribur matching (c.g., TaM), validating the importance training data that reflects human communicative patterns. However, the improvements tuned baselines remain substantially behind MIMIC-Chat in both overall accuracy and in the most challenging sub-tasks. This suggests that while cannot substitute for architectural innovations and multi-stage training pipelines explicitly designed for human-like reasoning. In other words, access the same data is not MMMiientt-MMMIC-...MM Cha''s advantage lies in how it integrates LoRA-based based language adaptation, temporal-spatee modeling, and spse-ip objectives to achieve balanced and robust performance across all axes of MIMIC- Bench.
incremental: even the best fine-
fine-tuning enhances alignment, it
C MODEL ARCHITECTURE AND IMPLEMENTATION DETAILS
In Section 4 of the main paper, we provided brief overview of the MIMIC-Chal architecture, which integrates a video encoder, an instruction formatter, and language model into unified framework. This section supplements that overview by elaborating implementation details and training configurations, including hardware setup, tinettunngg strategies, visual input processing, and optimization techniques.
18

---

Published as a conference paper at ICLR 2026
C.I TRAINING ENVIRONMENT AND HARDWARE CONFIGURATION
All experiments were conducted on high-performance server equipped with six NVIDIA RTX A6000 GPUs (each with 48 GB memory), using CUDA 12.2 and driver version 535.179. Model training was implemented with PyTorch, and distributed optimization was realized through torchrun and DeepSpeed Stage 1, enabling parameter offloading and mixed-precision (bf16) training for enhanced memory and efficiency.
C.2 FINE-TUNING STRATEGY AND MODULE CONFIGURATION
MIMIC-Chat adopts a parameter-efficiet instruction tuning strategy, updating only key compo-
nents:
Language Model (LLM)) LoRA modules injected into the attention sublayers of
IntemLMM-CChat-8, leveraging low-rank adaptation to reduce trainable parameters.
Freezing Strategy: Both the vision backbone and LLM backbone are frozen during train- ing, while projection layers and LoRA modules remain trainable.
Vision Projection: Spatial and temporal features are independently projected into the lan- guage space via two MLPs to preserve visual-linguistic alignment.
Additionally, 6 mixediprecision training and gradient checkpointing are enabled reduce mem- ory usage without compromising performance.
C.3 VIDEO INPUT PROCESSING AND VISUAL TOKEN CONSTRUCTION
Each video sample undergoes standardized frame sampling and preprocessing:
The spatial encoder uses 8 frames uniformly sampled from each video, while the temporal encoder processes the full sequence of frames to capture fine-grained temporal dynamics.
Frame Sampling:
Image Processing: All frames are center-cropped and resized to 448x448 resolution.
The spatial and temporal encoders extract static and dynamic informa- tion. The spatial encoder processes the uniformly sampled frames, while the temporal encoder consumes the full frame sequence to capture temporal continuity.
Feature Encoding:
Projection and Tokenization: Features from both spatial and temporal encoders are pro- jected into the language model token space form visual tokens.
Input Construction: Visual tokens and natural language instructions concatenated, with [VID] and [SEP] tokens denoting modality boundaries.
A dynamic patch control mechanism (up to 6 patches) and thumbnail token injection are introduced accommodate longer videos and enhance contextual representation.
C.4 TRAINING CONFIGURATION AND OPTIMIZATION
To ensure performance and stability, we adopt the following training settings:
Epochs: 50
Per-device batch size: 2: Global batch size: (via gradient accumulation)
Learning rate: 4e-5 with 3% warm-up
Input resolution: 448x448
Max dynamic patches: 6
Optimizer: AdamW with weight decay 0.01
Scheduler: Cosine decay
Gradient clipping: enabled
19

---

Published as a conference paper at ICLR 2026
Max sequence length:
4096 tokens
Grouped training: samples are grouped by token length to accelerate convergence
training logs recorded via TensorBoard; best-performing checkpoints and LoRA weights are saved periodically
Monitoring:
C.5 ENGINEERING OPTIMIZATIONS FOR SYSTEM ROBUSTNESS
To support long-context, large-scale multimodal training, we introduce several engineering enhance- ments:
Lazy-loading dataset class for robust video streaming with corrupted frame handling;
Custom trainer with LoRA-only weight saving to facilitate model deployment and abla- tion analysis;
Dynamic image preprocessing that adapts patch numbers and resolutions on-the-fly to control memory usage:
Multi-task training support. enabling unified classification and generation under instruction-basse bassd prompts.
D BENCHMARK CONSTRUCTION AND VISUALIZATION EXAMPLES
In Section of the main paper, we outlined the construction MIMIC-Bench and the motivation for its design. This section provides additional implementation details regarding how the 4,000 benchmark videos were selected and scored, including the criteria used to ensure their human-centric relevance and linguistic richness. It also supplements the dataset composition and preparation steps that underpin our evaluation tasks.
D. SELECTION AND SCORING CRITERIA
To ensure that the benchmark accurately reflects human-style interpretation and communicative be- havior, we curated 4,000 videos from the larger MIMIC-Data pool of 150,000+ user-shared videos. The selection process involved multi-stage filtering pipeline:
(1) Engagement Scoring, Each video was assigned composite engagement score to measure real- world interaction. The score combines the log-normalized values of like count, favorite count, share count, and comment count, computed as:
Engagement Score = a loog((ikk)) (aLLLike++>) + 1-3-1o (og(Fovorite) + Po--oo((Shar + log( Comment) (5)
We set = 1.0, 3 = 0.8, = 0.5, and 5 = 1.2 to place greater emphasis on comments, which better reflect human intent and understanding.
(2) Metadata Integrity. After sorting by engagement score, we retained only those videos with complete metadata fields, including title, description, tags, and topic. We further ensured that each video contains at least five unique, high-quality user comments and is free from decoding errors or anomalously short durations.
(3) Semantic Coverage and Diversity. To ensure diverse coverage across topics and expression styles, we adopted the following constraints:
Top 2% videos from TikTok and top 5% from YouTube were selected.
The selected pool spans 8 major categories (e.g., lifestyle, travel, beauty) and 20+ subcate- gories.
The distribution of comment types was controlled include exclamatory, inquisitive, as- sociative, and ironic styles.
This multi-dimmensinall curation strategy ensures that MIMIC-Bench captures both high user gagement and rich numan-centert semantics, laying a robust foundation for downstream evaluation of multimodal models user-aligned reasoning and mimicking capabilities.
20

---

Published as a conference paper at ICLR 2026
Aglino age woef
those Kapp foodlie in Fecul Laterden EPT7 da
Iereis and the samel E. Thraei busier meni Viir Do
Musil Holant tese eptee
Cod docaaaatii du fitheee only conpl and sonr Caber cemve
dey sear of
spien liel le decein iie dai coneeeen qhic in
libee
dnciing
Dupoiten oan't Time thee (Tand tren on bode dener
Norad Mee das aelin amodt and
Leneeco 2018
8:00 Lonkh
Figure 4: Illustration of the responses from the MiniCPM-V and MIMIC-Chat (Ours) models on user-like reasoning tasks. Ground truth is marked in red in the question, and model responses and correctness follow the corresponding question.
D.2 QUALITATIVE EXAMPLES OF MODEL RESPONSES
To better illustrate the performance of different multimodal large language models (MILMs)) on MIMIC-Bench tasks, we present series of representative model response examples in this sup- plementary material. These examples cover a variety of tasks such as title selection, tag matching. comment imitation, demonstrating each model's ability to interpret real-world user videos and gen- crate human-aligned responses.
Each example includes the following components:
Task input: the multimodal metadata associated with the video, along with the task prompt:
Model-genee gener--ge responses: the outputs from set of baseline MLLMs, as well as our proposed MIMIC-Chat model;
Ground-truth or reference answers: provided for comparison to evaluate model correct- ness or human human-likeness.
As shown in Figures 4, 5, and 6, the visualized outputs display the input prompts, the model pre- dictions, and whether the generated results match the expected answers. These examples qualita- tively complement the quantitative results in the main paper, highlighting each model's strengths and weaknesses across tasks involving higher-level reasoning, creative intent recognition, and user interaction interpretation.
We hope these examples will deepen understanding of the challenges posed by MIMIC-Bende and inspire the development of more human-aligned multimodal systems.
21

---

Published as a conference paper at ICLR 2026
deleed mando
Thias oieue onmer ane fame bate fartier ded
aod vides minder riaciia
X
X
hale And tat oeme arap untiee sovea star 1i twin and
X
X
demaerdee uden aps
partted by
X
Neaoiiile firm for Io Haard feedii mrdken dee den fn Grt (nadly
cyterm memttr lf
bateert of Coferrie aew a0t for odler. Fupele C
Foan
lodea tieruttlll with
canee bery sumtoall Mel other
fand
hightght
Loa alouad Bope cott www.ttaateee.... wwt.
X
cnl bume der tie in Seer
Sa GUB Inmora duy indooiiti 1o Aeciccal Our
deart coffl
X
The the rido
The Solel Ku ver Spe
Eoinale de air lia an cing of prand and 21
A BOSAET Via sollen and 2ww.py
X
Figure 5: Illustration of the responses from the Video-ChatGPT, Intern Video-LLaVA VA, and Lide models on user-like reasoning tasks. Ground truth is marked in red in the question, and model responses and correctness follow the corresponding question.
22

---

Published as a conference paper at ICLR 2026
Choos 1ile
Seser 22 PDM
ont orrgent and repiri do mou thd misntte alio thr -ppn fir Ganiig
Vilemm Valuou U/ Thene dey, you diy mo
Slandt, indlutie (hesdi LACEE TEE E. vald.
doset mocr dill: Eo book desrcian
Ber offir GG0000
Bliicing For
X
CogVLM
X
TEU
(ACRLLAAA soade Cut riihtea
Tee CogVLM
X
T CogVLM
(G4 HATT
goolde it ool 2l04 viden
tee Aedilii of Iabor
CogVVM
X
X
mie to our colicc an bere apoliirt boitt
CogVLM
X
Beik
4'dc aouuyy seceeclieee reme BN aar Beal ower 1 lusal del tuoll
CogVLM
MAXT
Figure 6: Illustration of the responses from the Qwen2-VL, VideoChat2, VLaVeTT and CogVLME models on user-like reasoning tasks. Ground truth is marked in red in the question, and model responses and correctness follow the corresponding question.
23

---

Published as a conference paper at ICLR 2026
paleyppye andl ules 5.3 adlig
Owent di H472
Quest di (P-772
X
thons, srrees
Viderge le de thut
with cild
fhr soactt sade diaiii hepi oe cilei des dld Fociall
de
Qwend di (772
X
Sibe mee the fpym Phaqpeayy
Sorea Wies lillanng perolts odual clder
Millonng Bulg commun
bub mey dee comer depplle
Cet Prggl whoue rikes Me uad fromat.
X
Noclut menie Hill The focs qpicd
sudddadd radingg
Qua2 5 (V72
X
Gemini
Openall
Hanopeote
fochh deth
wortel Of
Geriini Feally lir
Wener ond be ofve The cntt comer ththe codee and Thag
Geriini
Gemini
Gemini
Gemini
A- ciail
X
X
Figure 7: Illustration of the responses from the ChatGPT-40, Qwen2.5-VL(72B), Gemini2.5-pro, and Openai-o3 models on user-like reasoning tasks. Ground truth is marked in red in the question, and model responses and correctness follow the corresponding question.
24

---

Published as a conference paper at ICLR 2026
D.3 DIFFICULTY CHARACTERISTICS OF USER-LIKE THINKING TASKS
The user-like thinking tasks in MIMIC-Bench are deliberately designed to span multiple levels of cognitive complexity. As reported in the main paper. the majority of tasks remain challenging for current multimodal large language models (MILMss, with substantial gaps to human performance on many user-oriented reasoning problems. Although small subset of tasks may appear relatively simple from a human perspective, they play an important role in comprehensively probing different aspects of user-like thinking and providing coverage over realistic interaction scenarios.
At a coarse level, some user-like thinking tasks, such as Title Selection (TiS), involve relatively direct alignment between the video content, its associated metadata, and the candidate options. Even in these seemingly straightforward cases, models must still capture how human users abstract and summarize video content into concise, platform -aayfo titles. In contrast, higher-level tasks, including Category Matching (CaM), Comment Matching (CoM), and Comment Popularity (CoP), demand reasoning that goes well beyond surface-level cues. These tasks require models to infer implicit attributes, thematic foci, emotional or stylistic undertones, and preference patterns that govern which comments are considered relevant, engaging, or popular by real users.
Importantly, the difficulty of user-like thinking tasks does not primarily stem from visual complexity, but from the need to bridge perceptual and textual information with human-centric cognition. For instance, solving CoM and CoP tasks often hinges understanding why a particular comment resonates with viewers, how reflects shared background knowledge or community norms, and how subtle shifts in tone or stance can alter perceived appropriateness or popularity. Such challenges expose fundamental gaps between current MILMs and the way humans interpret, evaluate, and respond short-form video content in everyday platforms.
To concretely illustrate these characteristics, Figure 7 presents set of representative and challeng- ing examples drawn from the user-like thinking tasks, including Tis, CaM, CoM, and CoP. In these cases, successful task completion depends on implicit reasoning about content attributes, expressive intent, or user perspective, rather than merely matching superficial keywords or literal descriptions. Together, these examples highlight that, even when the task interface may look simple, the underly- ing cognitive demands in MIMIC-Bend are substantial for current MILMs, and that covering both simpler and more complex user-like thinking scenarios is necessary for holistic evaluation.
D.4 EXTENDED DESCRIPTION OF MIMIC-DATA
MIMIC-Data is the foundational dataset for constructing all tasks in MIMIC-Bench. It contains over 150,000 user-generated short videos collected from multiple public video-sharing platforms. While the main paper already outlines the high-level data pipeline and task mappings, this section provides additional implementation details regarding its structure and usage.
Each data sample is stored in structured JSONL format, with fields including video_par ppto,, title, description, tags, topic, and list of user comments. Every video is associ- ated with least five real user comments. All text fields are pre-cleaned by removing duplicates, empty or meaningless entries, and normalizing punctuation and encoding formats to ensure natural linguistic quality.
During task construction, each video may yield multiple question answer pairs depending on the completeness of its metadata and the number of available comments. All training prompts are for- mulated in unified instruction-folloinn format, where the task type and target field are explicitly encoded (c.g., "Please select the most likely title," " or "Which comment is most popular?").
MIMIC-Date is also structurally well-suited for supporting the full range of tasks in MIMIC-Bende All evaluation samples are derived directly from original metadata fields without requiring additional human annotations. In particular, for the comment imitation tasks, select the top five most-liked user comments per video and control the stylistic diversity of samples across expressive, associative, declarative, and rhetorical styles to better support modeling of human-like language behavior.
In future releases, we plan to extend MIMIC-Dai with multilingual versions and enhanced semantic annotations support broader research in multimodal reasoning and generation.
25

---

Published as a conference paper at ICLR 2026
Mayor
VoR, breathtaking viaw nountatin and the vaterfall. I can inagine the feeling of being there and experiancli anding the rush the water. It' besuliful sight to betold.
You think this comment isi AI
Humen
Reelice Scaues (O 52 OC 02 :33 O4 O5
18/18
Previoux Conmennt
Next Comment
Extt
# Usss use MESIKIN TADE
This video exoiures breathtaking view of nountaiacus
waterfall
casceding dosa Lha Tave, The midly stavegliere and the lush
add the nountaina read winding through driv iig along it create senee of adventure and exploration. The video
besuty of the
perfect repreuentation of the an uetaca beauty of the ai en and thilll traveling through such picturesque location.
You think this comment lsc Al
O Hunse 03
Real iem Scores (0 53
C1 02
04 06
Video 2/90
Comment 14/18
Previ ous Comment
Nex: Comment
Prit
Plyee
USEE- KIMIINNG
Vow, such a transformetion! Love the elegeat vedding crees,
You think this comment is: OlAI
Hunan
Roulion Scores (O G3 O.
O
03
4 O
Video 3/90
Comment 6/18
Previous Comment
Exit
Rlyer
10 utsssus uracicn TASI
x
That' quite the fashion show in propreza!
You think this O
Human
Beal Scores (0- G> 00
#
G6
Video 7/90
Conment 15/17
Previous Comment
Nexc Corment
Exit
Figure 8: Illustration of the annotation interface used in our User-like Mimicking Task. For each video, human annotators are presented with: (1) a displayed video clip, (2) one candidate comment generated either by an MILM or human, (3) a forced-choice judgment (AI vs. Human) on the com- ment's origin, and (4) realism score from 0-5 assessing how human-like the comment appears. We ensures consistent evaluation across all samples. This setup mirrors the model evaluation pipeline in our benchmark and provides a controlled environment for assessing human-centered cognitive and mimicking abilities.
26

---

Published as a conference paper at ICLR 2026
D.5 COMMENT MIMICKING EVALUATION PROTOCOL
The goal of the user-like comment mimicking task is to assess whether models can generate com- ments that exhibit human-like expressive characteristies commonly observed on social media plat- forms. Such expressions often involve emotional nuance, metaphor, sarcasm, cultural context, and implicit cross-context associations. At present, there no stable or widely accepted automatic metric that can reliably evaluate these open-ended, human-centered properties. Therefore, human evaluation is not a limitation of our design, but a necessary component, consistent with common practice in dialogue evaluation, story generation, and multimodal language generation benchmarks.
Evaluation Dimensions. The evaluation focuses on the human-likeness of generated comments and adopts two complementary dimensions: (1) Comment Origin Discrimination, where annotators judge whether a comment is more likely written by human or an Al model; and (2) Scoring. where annotators assign score from 0 to 5 reflecting how closely the comment resembles authentic human expression. These two dimensions directly align with the objective of measuring user-like expressive quality and have shown strong discriminative power across different models in our experiments.
Blind and Independent Annotation Protocol. To ensure fairness, reduce bias, and improve con- sistency, we adopt blind, multi-annotator evaluation protocol. For each video, real user comments and model-generated comments are mixed into unified pool and presented in randomized order. Annotators are only shown the video content and a single comment at a time; the source of the comment (human model) and the identity of the generating model are fully hidden. Importantly, model-generater comments are not directly compared against specific real comments, as such com- parison could implicitly reveal comment origin and conflict with the design of the origin prediction task.
Each comment is independently evaluated by multiple annotators using a unified web-based inter- face (see Figure 8 for an example of the annotation interface). Annotators receive brief instructions and illustrative examples prior to the formal evaluation to ensure consistent understanding of the criteria. A fixed scoring rubric with clear definitions is used to reduce ambiguity in interpreting human-likeness.
Bias Mitigation and Consistency. Several strategies are employed to mitigate potential annota- tion bias and variability: (i) multiple independent annotators are used reduce the influence of individual subjectivity; (ii) randomized presentation is applied to avoid ordering effects; and (iii) disagreement cases are resolved using simple aggregation rules, such as majority voting, with rare highly disputed items reviewed by more experienced annotator. These practices follow well- established methodologies in open-ended generation benchmarks and improve the stability and re- producibility of the evaluation.
Scalability Considerations. We acknowledge that human evaluation introduces scalability chal- lenges, which are shared by all benchmarks involving open-ended language generation. Due to the diversity and subjectivity of social media comments, it is difficult to define single ground truth for automatic comparison. Nevertheless, MIMIC-Bench and MIMIC-Data contain multiple high- quality human comments for each video, forming a rich distribution of authentic user responses. This structure provides foundation for exploring more scalable evaluation strategies in the future, such as hybrid L.M -assisted filtering or iitributionnawaar metrics, which we identify as an impor- tant direction for future work.
E ANALYSIS ON ENGAGEMENT BIAS AND OUT-OF-DOMAIN GENERALIZATION
E. RATIONALE OF USING HIGHER-EENAAGEMENT VIDEOS
The inclusion of subset of siggeeeengagement videos (top 2-5%) in HIMIC-Benchh motivated by methodological considerations rather than by any specific social or cultural preference. Our
27

---

Published as a conference paper at ICLR 2026
Table 5: Out-of-domain (OOD) evaluation results on videos with different engagement profiles.
Performance patterns remain consistent with those observed on HIMIC-Benche
Model Tasks
CIU
UIU
CAM
Overall
DeS
TaM
ToM
CaM
CoM
CoP
Qwen2.5-VL-7B
81.1
53.6
79.3
89.6
43.8
58.5
29.3
62.2
Intern
51.2
86.8
90.6
49.5
64.5
31.3
65.5
MIMIC-Chal (Ours)
92.5
87.5
87.4
93.2
56.2
78.4
43.8
77.0
primary objective is to ensure content quality, semantic clarity, and the availability of reliable real- user interaction signals, which are essential for evaluating user-like cognition and communicative behavior.
For tasks centered on user comments and human-centric understanding, eegher-ngaagment videos typically exhibit a more stable semantic focus and more representative comment distributions. Such properties are crucial for constructing dependable mimicking and interpretation tasks, as they reduce noise caused by ambiguous content, sparse feedback, or incidental viewer reactions. In contrast, low-engagement videos often lack sufficient or diverse user responses, making it difficult to reliably assess human-aligne reasoning or language behavior.
To mitigate potential biases arising from popularity-based selection, MIMIC-Bench further cu- rated through careful multi-stage manual filtering process. This process explicitly avoids stylistic homogeneity and cultural narrowness. The final benchmark spans wide range themes, nar- rative forms, and social contexts, including daily-life moments, instructional content, emotional self-expression, pets, food, travel, outdoor activities, and creative works. As a result, no single style, genre, cultural pattern dominates the benchmark, even though some videos exhibit higher engagement.
Overall, the of higher-engagement videos serves quality-control mechanism support reliable human-centry evaluation. rather than as proxy for social preference or cultural bias. In the following sections, we further examine this choice through out-of-domain and cross-benchmark analyses.
E.2 OUT-OF-DOMANL EVALUATION ON DIFFERENT ENGAGEMENT PROFILES
To further assess whether the engagement bageg video selection introduces unintended bias, we con- duct out-of-domain (OOD) evaluation videos with engagement profiles different from those used in HLMIC-Benchl Specifically, we construct a small OOD subset by randomly sampling videos with varied and generally lower engagement levels, while preserving the same task formulation and evaluation protocol as the original benchmark.
We evaluate several representative MLLMs, including Qwen2.5-VL-7B., Intern and our MIMIC-Chat, on this OOD subset. The results are summarized in Table 5. Although absolute performance varies across models, all three exhibit performance trends that are highly consistent with those observed on HIMIC-Bench.
Notably, MIMIC-Chat continues to achieve consistent improvements over both Qwen2.5-VL-7B and Intern VL2.5-8B across all task groups, including Creator Intent Understanding (CIU), User Interaction Understanding (UIU), and Content Atrribtee Matching (CAM). The preservation of clear performance margins under distribution shift suggests that the benchmark tasks capture intrinsic model capabilities rather than exploiting specific engagement patterns video popularity cues.
Overall, the strong agreement between in-domain and OOD evaluations indicates that MIMIC- Bench is not overly sensitive to engagement- selection, and that MIMIC-Chat's gains gen- eralize reliably to videos with different interaction statistics.
E.3 CROSS- BENCHMAR GENERALIZATION ANALYSIS
To further evaluate whether the improvements achieved by MIMIC-Chat generalize beyond MIMIC- Bench, conduct additional experiments several widely used video reasoning benchmarks,
28

---

Published as a conference paper at ICLR 2026
Table 6: Cross-benchmark results on selected MVBench subtasks.
Model
Lide
VideoChatGPT LLaVA VideoChar2 IntemVVL.2.553 MIMIC-Chat
Action Antonym
51.0
62.0
63.0
83.5
87,9
90.1
Action Count
34.0
30.5
34.0
39.0
54,4
58.5
Egocentric Navigation
30.0
29.5
27.0
35.0
30.1
35.5
Table Cross-benchmark results on selected TemporalBench subtasks.
Task
IL2.5n
MIMIC-Chat
Motion Direction / Orientation
58.6
60.4
Action Effector
51.4
52.3
Motion Magnitude
59.5
61.4
including MVBench and TemporalBench. Although these benchmarks were not explicitly designed to assess human-centric cognition comment mimicking, some of their subtasks overlap with the cognitive dimensions emphasized in MIMIC-Bench, such as action semantics, temporal reasoning, and egocentric understanding. Therefore, they provide meaningful complementary evidence for cross-benchmark generalization.
Table 6 reports results on representative MVBench subtasks, including Action Antonym, Ac- tion Count, and Egocentric Navigation. These tasks involve higher-level human action reason- ing, quantity-sensitie
inference, and first-person spatial understanding. Across all three sub- tasks, MIMIC-Chal consistently outperforms multiple baseline models, including its base model VnterVVL2.5888 The observed gains suggest that the improvements induced by MIMIC-Data are not limited to the benchmark setting, but extend to more diverse action- and perspective-orientee reasoning scenarios.
Table 7 presents results selected TemporalBench subtasks, including Motion Direc- fion/Oricntation, Action Effector, and Motion Magnitude. These tasks require fine-grained mo- tion interpretation, recognition of acting body parts, and estimation of physical magnitude. Such skills closely correspond to the content-atrribtte understanding, human-interaction reasoning, and dynamic semantic extraction emphasized in MIMIC Bench. MIMIC-Char again achieves consis- tent improvements over Intern VL2.5-8B across all three subtasks, indicating enhanced temporal and motion reasoning capability.
Overall, despite substantial differences in task formulation, supervision signals, and data distribu- tions between these external benchmarks and MIMIC-Bench, MIMIC-Chat demonstrates stable per- formance gains across heterogeneous tasks. This cross- cross-benchmark consistency supports the claim that the observed improvements arise from enhanced general reasoning and human-cene -entric abstrac- tion abilities, rather than from overfitting to the structure or data characteristics of MIMIC-Ben -BBchhhhh
F DATASET RELEASE, COPYRIGHT COMPLIANCE, AND RISK MITIGATION
This section provides a detailed description of the procedures and safeguards adopted in the con- struction and release of MIMIC-Data and MIMIC-Bench. In response to the ethical requirements specified in the meta-review, document practices for respecting copyright and platform terms of service, preventing the release of identifiable personal information, and mitigating potential risks of reputational harm.
The following subsections describe (1) data source compliance and platform policies, (2) copyright respect and redistribution constraints, (3) privacy protection and de-identification procedures, and (4) safeguards against reputational misuse. These measures are designed to ensure that the benchmark support academic research while minimizing legal, ethical, and social risks.
29

---

Published as a conference paper at ICLR 2026
F. DATA SOURCE AND PLATFORM COMPLIANCE
All videos included in MIMIC-Data collected exclusively from publicly accessible content on major online video-sharing platforms at the time of collection. We only consider content that is openly viewable without authentication beyond standard platform access and do not access private accounts, restricted materials, paywalled content, or content requiring elevated permissions.
Data collection procedures were designed to comply with the publicly stated terms of service of the respective hosting platforms. We do not employ methods that bypass access controls, circum- vent technical safeguards, or scrape non-public data. The dataset construction process focuses on retrieving publicly available video identifiers and associated metadata necessary for benchmark con- struction and evaluation.
Importantly, MIMIC-Bench does not redistribute raw video files. Instead, we release benchmark annotations, task splits, evaluation prompts, and metadata required for academic research. Users of the benchmark are expected to access original video content directly through the respective hosting platforms under their applicable terms of service.
We also maintain contact mechanism for content-related concerns. If a content owner or platform requests removal of specific entries from the benchmark, we will review and, where appropriate, remove the corresponding metadata and annotations from future releases.
F.2 COPYRIGHT RESPECT AND REDISTRIBUTION POLICY
MIMIC-BenCe and MIMIC-Data do not claim ownership of the original video content hosted on third-party platforms. All copyrights remain with the respective content creators and hosting ser- vices. The benchmark is constructed solely for academic research purposes and does not alter or redistribute copyrighted media.
As part of our release policy, we do not provide raw video files, downloadable media content, or tools that facilitate bypassing platform restrictions. Instead, the released materials consist of annotations, benchmark task definitions, evaluation prompts, and structured metadata necessary for reproducible research. Researchers accessing the benchmark are responsible for complying with the terms of service and copyright policies of the original hosting platforms when retrieving video content.
The dataset released under research-only usage policy. It is not intended for commercial re- distribution, large-scale content replication, or deployment in systems that violate platform policies or infringe upon creator rights. Any downstream use must respect applicable intellectual property regulations and platform-specific rules.
If any copyright holder or platform operator identifies content within the benchmark that raises concerns, we commit to reviewing the request and removing associated metadata or annotations from subsequent releases where appropriate.
F.3 PRIVACY PROTECTION AND DE-IDENTIFICATION PROCEDURES
We adopt multiple layers of safeguards to prevent the inclusion or release of personally identifiable information (PII) MIMIC-Data and MIIMI--Bench...
First, we do not release uploader identities, account names, profile links, user IDs, contact infor- mation, or other metadata that could directly identify individuals. Only benchmark-relevant annota- tions, task prompts, and non-identifying metadata are retained for research evaluation purposes.
Second, we implement filtering criteria to exclude content that centers on private individuals, minors, sensitive personal cireumstances. Vidcos that prominently expose private residential addresses, personal contact details, or vulnerable individuals are excluded during dataset construction. We also avoid including content that primarily documents personal disputes, medical conditions, other highly sensitive life events involving identifiable individuals.
Finally, the released benchmark does not provide structured indexing retrieval mechanisms de- signed to profile or track specific individuals. The dataset is organized around task-level evaluation rather than person-level aggregation, thereby reducing the risk of individual targeting.
30

---

Published as a conference paper at ICLR 2026
These measures collectively aim to ensure that the released data does not contain identifiable per- sonal information and does not facilitate individual identification.
F.4 REPUTATIONAL HARM RISK MITIGATION AND RESPONSIBLE USE
In addition copyright and privacy safeguards, we implement specific measures reduce the risk that the benchmark could be used in ways that cause reputational harm to individuals or communi- ties.
During dataset construction, we exclude content that contains defamatory statements, targeted ha- rassment, unverified allegations against identifiable persons, or highly sensitive personal topics (e.g., criminal accusations, medical disclosures, or politically charged attacks directed at specific individ- uals). Videos or associated annotations that center on exposing, shaming, or negatively profiling private individuals are removed based predefined exclusion criteria.
Furthermore, MIMIC-Bench not structured to support person-level profiling or tracking. The benchmark is organized around task-based evaluation of multimodal reasoning and language gener- ation, rather than aggregation of content linked to specific creators or individuals. We do not provide tools, indices, annotations that facilitate the identification systematic analysis of particular per- sons.
The dataset is released strictly for academic research and benchmarking purposes. It is not intended for surveillance, behavioral profiling, automated content moderation targeting specific individuals, any application that may adversely impact the rights, dignity, or reputation of individuals. Users of the benchmark are expected to adhere to applicable legal and ethical standards downstream use.
We recognize that no large-scale dataset can entirely eliminate misuse risks. Therefore, we commit monitoring community feedback and revising the benchmark if credible concerns about harmful impact arise. Where appropriate, we will update filtering rules or remove problematic entries in future releases maintain responsible stewardship of the dataset.
31


Reviews:
[{"review_id": "Ls3J2RN5Cs", "reviewer": "Reviewer_Y9c1", "summary": "This paper introduces MIMIC-Data, a large-scale dataset of over 150K user-shared videos with rich contextual metadata such as captions, tags, and comments. Building upon it, the authors present MIMIC-Bench, a benchmark of 4K videos designed to evaluate MLLMs’ ability to mimic human-like thinking and responses in realistic social media contexts. They further develop MIMIC-Chat, a model that integrates spatiotemporal understanding with fine-tuning for user-like reasoning and comment imitation. Experiments on 24 existing MLLMs show limited human-like behavior, while MIMIC-Chat demonstrates improved alignment with user-style cognition and response.", "questions": "Please refer to the Weaknesses section — the most critical point is to increase the difficulty of MIMIC-Bench to better match the rapidly improving capabilities of current VLMs.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper nicely fills a research gap by being the first to focus on human-like thinking and imitation of MLLMs in user-generated video contexts. The proposed MIMIC-Data is large-scale and well-curated, offering rich multimodal metadata that will benefit future research. The MIMIC-Bench benchmark is also valuable for assessing real-world user-aligned reasoning.\n\n2. The proposed MIMIC-Chat model is well-motivated — it integrates spatiotemporal features into an LLM and uses instruction tuning to balance structured reasoning with open-ended generation.\n\n3. The experiments are thorough and convincing, comparing 24 state-of-the-art MLLMs (both open and closed-source) and incorporating human annotations as an upper bound, which strengthens the reliability of the results.", "weaknesses": "1. My main concern is that the core contribution — MIMIC-Bench — feels too simple. In Tables 1 and 2, existing models already achieve very high scores, and the benchmark cases shown (e.g., in Fig. 1) lack strong distractors, making the evaluation less convincing.\n\n2. The paper does not report the video length distribution in MIMIC-Data. If short clips dominate, it may fail to assess long-video understanding. Also, the computational cost (e.g., GPU memory, inference speed) of MIMIC-Chat is not discussed, which limits understanding of its practical deployability.\n\n3. The paper does not address potential overfitting — whether MIMIC-Chat overfits MIMIC-Data or whether MIMIC-Data and MIMIC-Bench are too similar. Moreover, results on other benchmarks are missing, leaving generalization unverified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hYbt7B1yu1", "reviewer": "Reviewer_f8nN", "summary": "This paper addresses the gap where existing MLLM benchmarks primarily focus on basic visual reasoning and neglect the ability of models to emulate user-like thinking when interpreting videos and their associated social context on platforms like social media. The authors propose a new dataset, a benchmark, and a specialized model. Experiments reveal MLLMs' limited capabilities in human-aligned thinking and responses.", "questions": "Please refer to weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- This paper proposes a fundamentally new evaluation paradigm, shifting the focus from basic visual reasoning to user-like thinking and mimicking capabilities for User-Generated Content\n- This paper is well-written and easy to understand.", "weaknesses": "- The evaluation paradigm for the Mimicking Task (Comment Generation) suffers from poor scalability. Any new model introduced to the benchmark would necessitate re-evaluation, making comparative assessment difficult. Furthermore, the dimensions used for evaluation are insufficient.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1I9RI06hWm", "reviewer": "Reviewer_cT9q", "summary": "This paper presents **MIMIC-Bench**, a large-scale benchmark for human-aligned video understanding, together with **MIMIC-Data**, a supporting dataset, and **MIMIC-Chat**, a model fine-tuned for human-like reasoning and comment generation. The benchmark consists of two complementary tracks—User-like Thinking Tasks (structured reasoning based on metadata) and User-like Mimicking Tasks (comment imitation). The authors evaluate 24 multimodal large language models (MLLMs), including both open-source and proprietary systems, and provide human evaluations for reference.", "questions": "1. Could the authors elaborate on whether there is any potential overlap between MIMIC-Data (used for training) and MIMIC-Bench (used for evaluation)? If so, how is such overlap avoided or controlled to ensure fair assessment?\n\n2. It would be helpful if the authors could describe the human annotation process for comment imitation in greater detail—e.g., annotation interface, annotator expertise, and disagreement resolution.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The creation of MIMIC-Data and MIMIC-Bench fills a gap in evaluating human-aligned reasoning in video understanding, moving beyond visual recognition to social and cognitive aspects.\n2. The benchmark incorporates structured reasoning tasks and a creative comment imitation task, encouraging evaluation of both perceptual and cognitive understanding.\n3. The dual-branch spatiotemporal encoding and unified instruction format in MIMIC-Chat demonstrate careful design and solid engineering implementation.\n4. Extensive experiments on a large set of baselines and human participants give a comprehensive picture of current model capabilities.", "weaknesses": "1. Since MIMIC-Chat is trained on MIMIC-Data, which shares sources with the benchmark,  there is potential concern regarding content overlap or stylistic memorization.\n2. MIMIC-Bench focuses on highly engaging videos (top 2–5%), which might bias the content toward certain social or cultural styles. The work would benefit from analysis or experiments on more diverse or out-of-domain video data.\n3. While the Comment Imitation task benefits from human judgment, this can lead to variability in evaluation. Including more information on the annotation process, strategies for mitigating bias, and possible alternative metrics would help improve reproducibility and clarity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7zoRjg0Y3a", "reviewer": "Reviewer_wUW5", "summary": "To emulate user-like thinking and behaviour, the paper introduces MIMIC-Bench, a large-scale benchmark for evaluating human-like reasoning and social cognition in multimodal large language models (MLLMs). Built on MIMIC-Data—a dataset of over 150K user-generated videos with rich metadata such as titles, tags, and comments—MIMIC-Bench emphasizes how people think, feel, and react to videos, rather than only recognizing visual content.  It includes two main tasks: (1) User-like Thinking, covering creator intent, content attributes, and user interactions; and (2) User-like Mimicking, assessing models’ ability to generate human-like comments. The authors also develop MIMIC-Chat, a multimodal model trained on MIMIC-Data to jointly learn video semantics and human-style reasoning. Experiments on 24 state-of-the-art MLLMs show limited performance, while MIMIC-Chat demonstrates improved human-aligned understanding.", "questions": "None", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. The explanation of the problem and experimental protocol is clear, and the figures and tables are well-organized, effectively helping readers understand both the problem and the model architecture.\n2. The task is interesting and has clear real-world applications. Given a video, the model can generate human-like emotional responses and categorize the content into relevant tags. Such capabilities are essential for video understanding, predicting audience feedback on movies or online videos, and improving video recommendation systems.\n3. The paper presents comprehensive experiments comparing the performance of multiple models on the proposed dataset.\n4. For this task, the authors also propose MIMIC-Chat to estimate human responses. Based on the results, it serves as a strong and effective baseline.", "weaknesses": "1.  There are a few formatting issues—for example, in Lines 83–84, some words are incorrectly bolded, and in Lines 261–264, two metrics are presented but only one is highlighted in bold.\n2. The evaluation procedure for the User-like Mimicking task is unclear. Is the generated response from each of the 24 MLLMs individually compared against the real user comments?\n3. It would be helpful to include further insights and analysis based on the current models’ results—for example, whether the lower performance stems from limited perceptual ability or insufficient common-sense knowledge.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Conditions for acceptance have been satisfied.
Decision: Accept (Poster)