Title: ConvBench: A Multi-Turn Conversation Evaluation Benchmark with Hierarchical Ablation Capability for Large Vision-Language Models
Benchmark: ConvBench
Summary: This paper introduces the ConvBench to evaluate LVLMs across hierarchical capabilities such as perception, reasoning, and creativity. It enables fine-grained error attribution and includes an automatic evaluation framework.
Phenomenon type: General Capability (A broadly useful ability, which could be relevant to multiple applications)
Target phenomenon: multi-turn visual conversation
Phenomenon defined by authors: Yes
Phenomenon definition: Open-ended multi-turn visual conversations
Definition scope: Comprehensive
Definition integrity: Composite phenomenon
Definition integrity detail: Yes
Phenomenon contested: Widely-agreed
Task definition: Take an image, and provide answer during multi-turn conversation
Task item definition: <image> user question 1, model answer 1, user question 2, model answer 2
Task source: Expert-crafted task examples (e.g. hand-written examples), Modified from another benchmark (e.g. translation into another language), LLM-generated task examples (e.g. Filtered from responses to a prompt)
Task ecology: Partial real task (e.g. answering medical questions collected from real people), Representative task (e.g. answering medical licensing exam questions)
Dataset size: 577
Dataset metadata available: Yes
Dataset metadata detail:  hierarchy of multimodal capabilities
Sampling method: Specific criteria (items were taken from a larger set based on specified rules)
Split reviewed: Test
Response format: Free response (e.g. summary paragraph, executable code)
Metric definition: LLM-as-a-Judge (text quality, preferences, NOT extracting answers for other metrics)
Metric aggregation: Simple Mean, Weighted Mean
Metric statistics: simple mean
Metric subscores: No
Metric access: Outputs alone
Few-shot evaluation: No
Authorship: Academia
Benchmark availability: Yes
Task face validity: Yes
Metric face validity: Yes
Results interpreted by authors: Yes
Results compared: No
Results comparison explanation: No comparisons made
Results realism: No
Human baseline: Yes
Authors discuss validity: No
Taxonomy: User Interaction / 
New taxonomy: General Capability / User Interaction