MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal
  Large Language Models
Authors: name: Gio Paik, name: Geewook Kim, name: Jinbae Im

This paper introduces MMRefine, a MultiModal Refinement benchmark designed to
evaluate the error refinement capabilities of Multimodal Large Language Models
(MLLMs). As the emphasis shifts toward enhancing reasoning during inference,
MMRefine provides a framework that evaluates MLLMs' abilities to detect and
correct errors across six distinct scenarios beyond just comparing final
accuracy before and after refinement. Furthermore, the benchmark analyzes the
refinement performance by categorizing errors into six error types. Experiments
with various open and closed MLLMs reveal bottlenecks and factors impeding
refinement performance, highlighting areas for improvement in effective
reasoning enhancement. Our code and dataset are publicly available at
https://github.com/naver-ai/MMRefine.