Directional Reasoning Injection for Fine-Tuning MLLMs
Authors: name: Chao Huang; name: Zeliang Zhang; name: Jiang Liu; name: Ximeng Sun; name: Jialian Wu; name: Xiaodong Yu; name: Ze Wang; name: Chenliang Xu; name: Emad Barsoum; name: Zicheng Liu
Published: 2025-10-16 18:06:46

Abstract: Multimodal large language models (MLLMs) are rapidly advancing, yet their
reasoning ability often lags behind that of strong text-only counterparts.
Existing methods to bridge this gap rely on supervised fine-tuning over
large-scale multimodal reasoning data or reinforcement learning, both of which
are resource-intensive. A promising alternative is model merging, which
interpolates parameters between reasoning-enhanced LLMs and multimodal
variants. However, our analysis shows that naive merging is not always a "free
lunch": its effectiveness varies drastically across model families, with some
(e.g., LLaVA, Idefics) benefiting while others (e.g., Qwen) suffer performance
degradation. To address this, we propose Directional Reasoning Injection for
Fine-Tuning (DRIFT) MLLMs, a lightweight method that transfers reasoning
knowledge in the gradient space, without destabilizing multimodal alignment.
DRIFT precomputes a reasoning prior as the parameter-space difference between
reasoning and multimodal variants, then uses it to bias gradients during
multimodal fine-tuning. This approach preserves the simplicity of standard
supervised fine-tuning pipelines while enabling efficient reasoning transfer.
Extensive experiments on multimodal reasoning benchmarks, including MathVista
and MathVerse, demonstrate that DRIFT consistently improves reasoning
performance over naive merging and supervised fine-tuning, while matching or
surpassing training-heavy methods at a fraction of the cost.