EXPERT : An Explainable Image Captioning Evaluation Metric with Structured Explanations (2025)

Abstract: Recent advances in large language models and vision-language models have led to growing interest in explainable evaluation metrics for image captioning. However, these metrics generate explanations without standardized criteria, and the overall quality of the generated explanations remains unverified. In this paper, we propose EXPERT, a reference-free evaluation metric that provides structured explanations based on three fundamental criteria: fluency, relevance, and descriptiveness. By constructing large-scale datasets of high-quality structured explanations, we develop a two-stage evaluation template to effectively supervise a vision-language model for both scoring and explanation generation. EXPERT achieves state-of-the-art results on benchmark datasets while providing significantly higher-quality explanations than existing metrics, as validated through comprehensive human evaluation. Our code and datasets are available at https://github.com/hjkim811/EXPERT.

## introduction automatic evaluation of image captions is crucial for measuring and improving image captioning models without the significant cost and time required for human evaluation @xcite . along with the advancement of large language models (llms) @xcite @xcite @xcite and vision-language models (vlms) @xcite @xcite , explainable evaluation metrics have recently drawn increasing attention in image captioning @xcite . explainable metrics not only provide numerical scores but also textual explanations, enhancing the interpretability and transparency of the evaluation. however, existing research on explainable metrics † corresponding author. has two limitations. first, the explanations provided by existing metrics are not generated based on standardized criteria or guidelines, which may lead to inconsistency in content and structure. second, previous studies lack a thorough evaluation of the quality of generated explanations, leaving the overall quality unverified. to address these issues, we propose expert foot_0 , a vlm-based reference-free evaluation metric for image captioning that provides structured explanations. we first construct large-scale datasets containing over 42,000 structured explanations by extending existing human judgment datasets. each explanation is structured according to three fundamental criteria-fluency, relevance, and descriptiveness-and its quality is validated through human evaluation. we then train expert using a two-stage evaluation template that effectively guides both scoring and explanation generation. unlike existing explainable metrics, expert generates structured, criterion-specific explanations, as illustrated in figure 1 . experimental results on benchmark human judgment datasets show that expert establishes new state-of-the-art performance. through comprehensive human evaluation, we further demonstrate that expert generates significantly higher-quality explanations than existing explainable metrics. to the best of our knowledge, we are the first to systematically assess the explanation capabilities of explainable image captioning evaluation metrics. ## conclusion in this work, we propose expert, a state-of-theart reference-free evaluation metric that provides structured explanations based on standardized criteria. along with our novel explanation datasets and analysis of explanation quality, we hope this work offers a valuable contribution to advancing the explainability of image captioning evaluation.