Self-Improvement in Multimodal Large Language Models: A Survey (2025)

Abstract: Recent advancements in self-improvement for Large Language Models (LLMs) have efficiently enhanced model capabilities without significantly increasing costs, particularly in terms of human effort. While this area is still relatively young, its extension to the multimodal domain holds immense potential for leveraging diverse data sources and developing more general self-improving models. This survey is the first to provide a comprehensive overview of self-improvement in Multimodal LLMs (MLLMs). We provide a structured overview of the current literature and discuss methods from three perspectives: 1) data collection, 2) data organization, and 3) model optimization, to facilitate the further development of self-improvement in MLLMs. We also include commonly used evaluations and downstream applications. Finally, we conclude by outlining open challenges and future research directions.

## introduction self-improvement aims to enable models to collect and organize data required to build a better generation of themselves, which offers a path to overcome the costly scaling issues and potential performance ceilings of static training paradigms. in multi-modal large language models (mllms), self-improvement seeks to use mllms themselves to obtain their own training data, resulting in improved mllms. recent research @xcite @xcite show that this approach can significantly reduce hallucinations and improve performance on general tasks with relatively low cost. significant progress has been made in this direction. some current studies @xcite partially leverage self-improvement by combining it with external tools or peer models, while others @xcite explore approaches that rely solely on a single model to handle all processes, toward full self-improvement. although previous work @xcite has summarized the self-improvement in text-only llms and other surveys study the general scope of mllms @xcite or specific issues such as hallucinations @xcite , there is no comprehensive survey that focuses on these self-improvement methods for mllms. to fill this gap, we dedicate this paper to providing a comprehensive review of this area and identifying the challenges that need to be addressed. compared to self-improvement in llms @xcite , self-improvement in mllms faces unique challenges, such as the inclusion of multiple modalities. this can introduce modality alignment problems, which are known to cause issues like hallucination in mllms @xcite . additionally, mllms often cannot generate all the training data it needed by themselves, as most current models @xcite are unable to generate images directly. despite these challenges, there is growing in-terest in leveraging self-improvement in mllms to build models more effectively and efficiently. promising results have already been achieved in this area. this paper aims to summarize previous works, compare methods, and provide clearer guidance for future research directions in this field. in this survey, we follow the structure outlined below: first, we provide an overview of the field. next, we introduce the most commonly used seed models that serve as starting points for self-improvement. for the detailed methodology, we divide the discussion into three parts as shown in fig. 2 : data collection, data organization, and model optimization. we list current approaches and discuss their differences. we also collect evaluation methods commonly used to measure performance gains from self-improvement, compiling benchmark results for a comprehensive comparison. additionally, we discuss downstream applications, to highlight the real-world impact of this paradigm. finally, we identify the challenges in this field, which also represent potential future directions, and conclude the survey. with this work, we aim to establish a clearer pathway for developing the next generation of mllms with better self-improvement mechanisms, moving beyond random exploration with biases. we hope to attract more researchers to explore this promising direction. ## overview in this section, we first formally define selfimprovement in multimodal large language models (mllms) in the context of this paper, and then compare it to similar concepts that have been used in mllms research. afterwards, we summarize representative works in this domain to provide a general overview of the existing methods. ## seed models a seed model does not need to be exceptionally strong, but it must clear a small set of capability floors that the self-improvement loop relies on. if these floors are missing, the model tends to generate low-quality data and the loop either stalls or collapses. capability floors. some skills are costly to "retrofit" purely from self-improvement and therefore should be present in the seed: level who does the heavy-lifting? typical technique / example l0 -no self-improvement humans do all data collection and curation instructgpt-style sft (ouyang et al., 2022) l1 -human-guided improvement model generates responses, while humans choose preferred data rlhf-v, human-guided reward modelling (yu et al., 2024a) l2 -peer improvement external models (e.g. gpt-4-v) supply data; minimal direct human toil distillation (liu et al., 2024a), l3 -hybrid self-improvement model collects its own data, but queries external augmentations or verifiers hybrid approaches (zhou et al., 2024a) l4 -conditional self-improvement target model runs its own data loop except images come from existing datasets rlaif-v (yu et al., 2024b) with self-reward l5 -high self-improvement model generates and curates both images and text without external data sources unirl (mao et al., 2025) beyond these commonly used seed models, some works train their own seed models from scratch using a pretrained llm to maintain more control over the entire process and address specific needs. ## data collection effective data collection is crucial for enabling mllms to acquire and refine specific abilities. in conventional machine learning approaches, data collection typically relies on extensive human labor. this labor-intensive process, while effective, can be both time-consuming and costly, and is often limited by the availability and scalability of human resources. in the context of self-improvement for mllms, a shift towards autonomous data collection is both desirable and increasingly feasible, thereby reducing the dependency on human intervention. this approach not only enhances efficiency but also enables continuous and scalable learning. we compare advantages and disadvantages of these methods in tab. 2. ## data organization the data collected by mllms may not be directly suitable for feeding back into the models without further processing. to ensure the efficacy of selfimprovement, a thorough verification and processing step is essential before leveraging the newly obtained data. the quality of this organization process is paramount, as it directly determines the robustness and reliability of the self-improvement mechanism in mllms. we compare these methods in tab. 3. ## model optimization after obtaining the filtered dataset, the next step is to update the parameters of the seed model. several training methods have been employed in selfimprovement for mllms, including supervised fine-tuning, reinforcement learning, and direct preference optimization. as discussed in the paper deepseekmath @xcite , all these methods are actually connected. we compare advantages and disadvantages of these methods in tab. 4. ## dataset and evaluation there are some datasets released for improving mllms but no benchmarks specifically designed for self-improvement in mllms. most of the time, researchers use existing mllm benchmarks and report performance gains compared to the seed model and other sota models. some attempts, such as llm-evolve @xcite aim to build a new type of benchmark; however, this particular benchmark operates in a non-parametric setting. ## conclusion in this paper, we presented a comprehensive and structured survey of self-improvement in multimodal large language models (mllms). we defined the concept of self-improvement as used in this survey and clarified its differences from other related concepts. we discussed and compared representative works in this domain, highlighting their similarities and differences from three perspectives: 1) data collection, 2) data organization, and 3) model optimization. further, we summarized commonly used evaluations and applications. finally, we identified current challenges and potential opportunities for future research. we hope this survey serves as a valuable guide for researchers interested in exploring and developing new self-improvement methods for mllms.