Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Tasks (2025)

Abstract: While Vision-Language Models (VLMs) have shown remarkable abilities, they invariably generate flawed responses. Self-correction that instructs models to refine their outputs presents a promising solution to this issue. Previous studies have mainly concentrated on Large Language Models (LLMs), while the self-correction abilities of VLMs, particularly concerning both visual and linguistic information, remain largely unexamined. This study investigates the self-correction capabilities of VLMs during both inference and fine-tuning stages. We introduce a Self-Correction Learning (SCL) approach that enables VLMs to learn from their self-generated self-correction data through Direct Preference Optimization (DPO) without relying on external feedback, facilitating self-improvement. Experimental results demonstrate that although VLMs struggle to self-correct effectively during iterative inference without additional fine-tuning and external feedback, they can enhance their performance and avoid previous mistakes through preference fine-tuning when their generated self-correction data are categorized into preferred and disfavored samples. This study emphasizes that self-correction is not merely a refinement process; rather, it should enhance models’ reasoning ability through additional training, enabling them to generate high-quality responses directly without further refinement.

## introduction large language models (llms) have shown exceptional versatility across numerous natural language processing domains @xcite . building upon the foundational capabilities of llms, vision-language models (vlms) @xcite standing by combining pre-trained llms and vision models through instruction fine-tuning, leading to significant advancements in multimodal tasks @xcite . despite their robust vision-language understanding capabilities, vlms are still prone to generating inaccurate information @xcite @xcite . selfcorrection, an approach enabling models to identify and rectify mistakes in their outputs @xcite , is thus a promising method for enhancing the quality of vlm-generated responses. while previous studies have predominantly focused on self-correction in llms, the self-correction capabilities of vlms remain underexplored. given that vlms integrate visual and linguistic information during reasoning, self-correction in vlms presents additional challenges. this complexity arises from the inherent difficulty of accurately aligning and rectifying multimodal data, underscoring the critical need for a systematic investigation into vlm self-correction to advance their performance in vision-language reasoning tasks. existing self-correction strategies focus on the inference stage without parameter updates @xcite @xcite . these methods typically involve instructing models to revise their initially generated answers by leveraging self-correction prompts. while prior work has demonstrated the effectiveness of inference-time self-correction in improving llm performance on reasoning tasks @xcite , and such approaches offer advantages like no dedicated training cost and operational simplicity, recent studies have reported contradictory findings @xcite . this controversy highlights two main shortcomings of inference-time self-correction: (1) unreliable performance: the effectiveness of selfcorrection is highly sensitive to the formulation of self-correction prompts @xcite . (2) limitations of models' reasoning abilities: without additional training to enhance their intrinsic reasoning capabilities, models often struggle to effectively self-correct when confronted with identical challenging tasks @xcite . besides these two challenges, a crucial distinction between existing self-correction methods during the inference stage and the more natural human self-correction process lies in their correction goals. as illustrated in figure 1 , the former approach focuses on better refinement, specifically enabling the model to correct its initial response through additional revisions @xcite . conversely, the latter approach emphasizes better initial generation, aiming to provide the correct answer on the first attempt without subsequent revisions @xcite . this discrepancy indicates that current inference-time self-correction methods offer only a temporary solution for rectifying mistakes. while a model can correct mistakes in its generated content through iterative selfcorrection, its underlying reasoning ability remains unchanged. consequently, the model may continue to produce suboptimal answers when faced with identical questions in the future, leading to inefficient resource expenditure on repeated refinement. therefore, we emphasize the ultimate aim of selfcorrection: not merely to fix initial mistakes but to fundamentally improve the model's capability to generate correct answers directly. in this paper, we investigate the self-correction capabilities of vlms through two research questions (rqs): (1) inference-based self-correction mechanisms: can vlms perform self-correction during inference without external feedback? we evaluate the intrinsic self-correction abilities of vlms and the efficacy of scl across multiplechoice questions (mcq) benchmarks. experimental results demonstrate that while vlms initially struggle with intrinsic self-correction, they can benefit from learning from their self-correction samples. specifically, vlms fine-tuned with scl are better able to avoid previous errors and exhibit superior performance compared to existing preference optimization methods for vlms. our main contributions are as follows: firstly, we define the ultimate objective of self-correction as not only rectifying initial errors but fundamentally enhancing a model's ability to generate accurate responses directly. secondly, we systematically evaluate the self-correction abilities of vlms during inference by developing three visual selfcorrection prompts. we further analyze the reliability of inference-based self-correction mechanisms. thirdly, we introduce self-correction learning (scl), a novel approach that enables vlms to self-improve via dpo by leveraging self-generated preference data. our findings demonstrate the effectiveness of scl and underscore the advantages of training-based self-correction mechanisms. intrinsic self-correction in large language models. self-correction in large language models (llms) aims to guide these models in rectifying their flawed generated content, such as harmful outputs @xcite . intrinsic self-correction refers to a self-correction paradigm during inference where a model revises its output solely by leveraging its inherent capabilities and the input context, without external feedback @xcite @xcite . this iterative, multi-turn self-correction process is distinct from single-turn test-time inference of @xcite . while recent research has demonstrated the effectiveness of intrinsic selfcorrection @xcite @xcite , some studies indicate that llms face challenges with it. for instance, intrinsic self-correction may sometimes decrease output quality @xcite and potentially introduce bias @xcite . these conflicting results suggest that the self-correction ability of llms remains unreliable without external feedback. previous work has primarily explored the intrinsic self-correction abilities of llms on unimodal tasks like arithmetic reasoning. our study investigates the intrinsic self-correction abilities of vlms on visual and language reasoning tasks. improvement in language models and vision-language models. the enhancement of language models (lms) can be broadly classified into self-improvement and external improvement. self-improvement in lms relies on their ability to learn from the data they generate, whereas external improvement involves leveraging external models or tools to enable lms to learn from provided data. regarding self-improvement, @xcite al. (2023); wang et al. (2025b) show that llms can self-improve by learning from self-generated data selected via self-consistency @xcite . @xcite al. (2024b); wu et al. (2024a); huang et al. (2025b ) leverage the llm's self-evaluation capability to generate training signals, such as for mitigating factual inaccuracies and enhancing response personalization. @xcite introduce score, a multi-turn online reinforcement learning method that achieves self-correction improvements by training on self-generated data and utilizing a two-stage training process to prevent behavior collapse. additionally, @xcite propose self-improvement modality alignment (sima) which uses in-context self-critic to improve the modality alignment of vlms. in sima, the model generates two one-turn responses for each question using greedy decoding and temperature sampling. it is then prompted to critique these responses as preferred or disfavored, thereby constructing a preference dataset. distinguished from sima, our work constructs a preference dataset using pairs of initial and revised responses from vlms, generated during intrinsic self-correction. for external improvement, @xcite demonstrate that error data generated by strong llms can enhance the reasoning capabilities of weaker llms. @xcite show that small lms can enhance their self-correction capabilities through instruction fine-tuning. however, this method still requires the generation of selfmodification responses. our work emphasizes that the goal of self-correction is not only to correct mistakes iteratively, but rather to enhance the models' ability to produce correct answers directly. we explore whether vlms can improve the quality of their responses without further refinement steps by leveraging both their successful and erroneous intrinsic self-correction data for self-improvement. ## methodology the framework of self-correction learning (scl) is illustrated in figure 2 , comprising three stages: inference, dataset construction, and preference finetuning. in the inference stage, we propose three visual self-correction prompts and investigate the intrinsic self-correction capabilities of vlms to address rq1. subsequently, in the dataset construction and fine-tuning stage, we create selfcorset for each vlm based on its intrinsic self-correction and explore rq2 through dpo. ## conclusion this work investigates the intrinsic self-correction capabilities and limitations of vision-language models (vlms) during the inference stage, and their potential for self-improvement through finetuning, across various multiple-choice question (mcq) benchmarks. we propose self-correction learning (scl), a novel framework that employs direct preference optimization (dpo) to enable vlms to learn from their own self-correction responses. this approach aims to fundamentally enhance the models' ability to generate correct answers directly, rather than merely refining initial errors. our experiments reveal that while vlms face significant challenges and exhibit unreliable performance during intrinsic self-correction, they can effectively leverage their own self-correction samples (both successful and erroneous) to selfimprove without requiring external feedback.