Reinforced Visual Perception with Tools
Authors: name: Zetong Zhou, name: Dongping Chen, name: Zixian Ma, name: Zhihan Hu, name: Mingyang Fu, name: Sinan Wang, name: Yao Wan, name: Zhou Zhao, name: Ranjay Krishna

Visual reasoning, a cornerstone of human intelligence, encompasses complex
perceptual and logical processes essential for solving diverse visual problems.
While advances in computer vision have produced powerful models for various
perceptual tasks, leveraging these for general visual reasoning remains
challenging. Prior work demonstrates that augmenting LLMs with vision models
via supervised finetuning improves performance, but faces key limitations such
as expensive data generation, reliance on careful data filtering, and poor
generalization. To address these issues, we propose ReVPT to enhance
multi-modal LLMs' abilities to reason about and use visual tools through
reinforcement learning. We introduce a novel RL algorithm based on GRPO,
designed to train models to reason with a suite of four visual tools. Through
extensive experiments, we show that our method achieves state-of-the-art
performance on several perception-heavy benchmarks, including SAT, CV-Bench,
BLINK and MMStar, significantly outperforming the supervised and text-based RL
finetuning baselines. Notably, Our ReVPT-3B and ReVPT-7B outperform the
instruct models by 9.03% and 9.44% on CV-Bench. Finally, we bring to the
community new insights on RL-based visual tool-usage through extensive
ablations. Our code is available at https://github.com/ls-kelvin/REVPT.