Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation (2024)

Abstract: Assessing long-form responses generated by Vision-Language Models (VLMs) is challenging. It not only requires checking whether the VLM follows the given instruction but also verifying whether the text output is properly grounded on the given image. Inspired by the recent approach of evaluating LMs with LMs, in this work, we propose to evaluate VLMs with VLMs. For this purpose, we present a new feedback dataset called the Perception Collection, encompassing 15K customized score rubrics that users might care about during assessment. Using the Perception Collection, we train Prometheus-Vision, the first open-source VLM evaluator model that can understand the user-defined score criteria during evaluation. Prometheus-Vision shows the highest Pearson correlation with human evaluators and GPT-4V among open-source models, showing its effectiveness for transparent and accessible evaluation of VLMs. We open-source our code, dataset, and model.

## introduction while recently developed vision-language models (vlms) are capable of generating long-form text from a combination of an image and instruction, assessing the quality of the output remains a significant challenge @xcite @xcite @xcite . traditional metrics, which rely on text-based exact matches or edit distances, fall short in adhering to the granular evaluation criterion of interest and capturing the rich context within the outputs @xcite @xcite . for instance, as shown in figure 1 , conventional metrics fail to explain what is missing within the response compared to the answer. consequently, the role of high-quality human evaluations remains pivotal for a comprehensive assessment. however, human evaluators are prone to biases and scaling up is expensive in terms of time and cost @xcite . to address the need for flexible and automatic text evaluation, the 'lm-as-a-judge' paradigm proposes using language models (lms) as evaluators, where initial findings suggest its potential to emulate human judgement @xcite @xcite @xcite @xcite . however, lms cannot perceive visual contexts, which necessitates an additional model that could convert the image to text. as a result, such a multistage pipeline could potentially suffer from error propagation and also require multiple inference calls. this situation calls for the direct utilization of vlms, referred to as vlm-as-a-judge. however, despite gpt-4v's (openai, 2023) potential as an evaluator, its closed-source nature limits transparent evaluation @xcite . on the other hand, our initial tests indicate that open-source vlms are not appropriate to be used as evaluators to assess based on granular aspects, demonstrating a low score correlation with both human evaluators and gpt-4v. to address these challenges, we propose prometheus-vision, a 7b & 13b vlm evaluator that could assess based on fine-grained criteria. as shown in figure 1 , prometheus-vision could evaluate based on the given criteria, pinpointing the differences between the parody artwork and the original masterpiece. to develop prometheus-vision, we construct the perception collection, the first multimodal feedback dataset that includes 15k finegrained score rubrics, thus going beyond traditional coarse-grained criteria such as helpfulness, relevance, accuracy, and comprehensiveness. using language feedback helps to understand the reasoning behind the score decision. the image appears to be a humorous, digitally-altered parody of leonardo da vinci's famous painting "mona lisa." instead of the iconic face of the mona lisa, the painting features the head of a brown dachshund dog @xcite the surreal combination of a classic art masterpiece and a modern pet creates a playful and intriguing image that plays on the familiarity and stature of the original painting. the response accurately identifies the image as a parody of the mona lisa, with a detailed description of the dog's appearance and the blending of its fur with the original painting's clothing. it also captures the essence of the background, noting the atmospheric landscape and the lighting, which are reminiscent of the original artwork. @xcite this painting is a creative fusion of the famous mona lisa and a dog [...] the artist has skillfully combined the two subjects into one, giving the piece a whimsical and amusing quality. prometheus-vision 13b instruction evaluation criteria feedback score response to evaluate answer so the overall score is 5. perception collection, we fine-tune llava-1.5 to obtain prometheus-vision. surprisingly, prometheus-vision shows the highest correlation with both human evaluators and gpt-4v on 3 visual instruction tuning benchmarks, 3 visual question answering datasets, and 2 captioning benchmarks among the open-source models. this highlights prometheus-vision's potential to serve as an inexpensive yet effective open-source alternative to gpt-4v evaluation. for instance, prometheus-vision-13b exhibits a pearson correlation of 0.639 with human evaluators and 0.786 with gpt-4v on the llava-bench, one of the most widely used visual instruction tuning benchmark. moreover, prometheus-vision also exhibits high accuracy on human preference datasets, indicating that prometheus-vision could potentially function as a reward model in the reinforcement learning from human feedback (rlhf) pipeline. our contributions are summarized as follows: ## the perception collection in contrast to the language domain, to the best of our knowledge, there do not exist any available feedback, critique, or preference datasets applicable to train an evaluator vlm that could assess in a fine-grained manner. for this purpose, we first construct a comprehensive multimodal feedback dataset called the perception collection. as shown in figure 2 , each instance in the per-ception collection consists of five input components (image, instruction, response to evaluate, customized score rubric, reference answer) and two output components (language feedback and score decision). the number of each component in the perception collection is shown in specifically, the five input components are: moreover, the two output components are: does the response express aesthetic harmony and design elements such as color coordination, spatial layout, lighting, and decor synergies @xmath0 score 5 description: ~~~t he response does acknowledge the presence of fundamental components. [...] however, the it lacks a detailed understanding of color harmony, as it does not delve into how the colors within the room interact or complement each other effectively. so the overall score is 2. customized score rubric feedback score decision (1-5) analyze the pictured living space in terms of its aesthetic appeal and functionality. instruction analyze the pictured living space in terms of its aesthetic appeal and functionality. instruction image instruction this living space leverages a contemporary design, with a focus on minimalism and an abundance of natural light. the room features a neutral color palette, with white walls that enhance the brightness created by the large windows. [...] overall, from the choice and placement of the furniture to the commanding yet balanced role of art, the design of the room speaks to an elegance that is both modern and timeless. ## discussions and analysis due to space limitations, we include our discussion and analysis section in the appendix. this includes analysis of potential biases from vlm evaluators (appendix b), the potential of prometheus-vision as a reward model for visual instruction tuning (appendix c), and ablation experiments (appendix d). we find that using (1) an absolute grading scheme, (2) fine-grained score rubrics, and (3) language feedback distillation enables prometheus-vision mitigate length biases and also operate in a relative grading settings. ## conclusion in this paper, we expand the 'lm-as-a-judge' paradigm to the multimodal space and introduce 'vlm-as-a-judge'. we first propose a multimodal feedback dataset called the perception col-lection, which contains fine-grained score criteria for each instance, unlike existing multimodal datasets that do not sufficiently consider critical factors in each instance. using the perception collection, we train prometheus-vision, an open-source model specialized for evaluation purposes. the uniqueness of prometheus-vision is that it can adhere to user-defined criteria during evaluation. through experiments, we show that prometheus-vision is an effective open-source alternative for gpt-4v evaluation. we hope our work advances the development of accessible and transparent evaluators for diverse modalities.