MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal
  Understanding
Authors: name: Xin Jin, name: Siyuan Li, name: Siyong Jian, name: Kai Yu, name: Huan Wang
Organization: Westlake University

Vision-language alignment in multi-modal large language models (MLLMs)
typically relies on supervised fine-tuning (SFT) or reinforcement learning
(RL). SFT is stable and efficient but requires large-scale human annotations
and cannot capture subtle preferences, while RL brings in a reward signal for
training, but suffers from overhead and instability. These limitations
highlight a trade-off between scalability, robustness, and alignment quality.
To address this, we propose MergeMix, a training-time augmentation paradigm
that bridges SFT and RL. It first applies an attention-aware image mixing via
token merge with more cluster representation and spatial context, and then
presents a preference-driven training paradigm for MLLMs by building preference
pairs with mixed images and raw images, and optimizing via SimPO loss. As a
mixup augmentation, MergeMix enhances attention consistency and efficiency,
surpassing other heuristic-based methods in classification. Extensive
experiments demonstrate that MergeMix achieves competitive accuracy with
improved efficiency, providing a scalable approach to preference alignment in
classification and MLLMs.