ViSurf: Visual Supervised-and-Reinforcement Fine-Tuning for Large
  Vision-and-Language Models
Authors: name: Yuqi Liu, name: Liangyu Chen, name: Jiazhen Liu, name: Mingkang Zhu, name: Zhisheng Zhong, name: Bei Yu, name: Jiaya Jia

Typical post-training paradigms for Large Vision-and-Language Models (LVLMs)
include Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable
Rewards (RLVR). SFT leverages external guidance to inject new knowledge,
whereas RLVR utilizes internal reinforcement to enhance reasoning capabilities
and overall performance. However, our analysis reveals that SFT often leads to
sub-optimal performance, while RLVR struggles with tasks that exceed the
model's internal knowledge base. To address these limitations, we propose
ViSurf (Visual Supervised-and-Reinforcement
Fine-Tuning), a unified post-training paradigm that integrates the
strengths of both SFT and RLVR within a single stage. We analyze the derivation
of the SFT and RLVR objectives to establish the ViSurf objective, providing a
unified perspective on these two paradigms. The core of ViSurf involves
injecting ground-truth labels into the RLVR rollouts, thereby providing
simultaneous external supervision and internal reinforcement. Furthermore, we
introduce three novel reward control strategies to stabilize and optimize the
training process. Extensive experiments across several diverse benchmarks
demonstrate the effectiveness of ViSurf, outperforming both individual SFT,
RLVR, and two-stage SFT \textrightarrow RLVR. In-depth analysis corroborates
these findings, validating the derivation and design principles of ViSurf.