PixelThink: Towards Efficient Chain-of-Pixel Reasoning
Authors: name: Song Wang; name: Gongfan Fang; name: Lingdong Kong; name: Xiangtai Li; name: Jianyun Xu; name: Sheng Yang; name: Qiang Li; name: Jianke Zhu; name: Xinchao Wang
Published: 2025-05-29 17:55:49

Abstract: Existing reasoning segmentation approaches typically fine-tune multimodal
large language models (MLLMs) using image-text pairs and corresponding mask
labels. However, they exhibit limited generalization to out-of-distribution
scenarios without an explicit reasoning process. Although recent efforts
leverage reinforcement learning through group-relative policy optimization
(GRPO) to enhance reasoning ability, they often suffer from overthinking -
producing uniformly verbose reasoning chains irrespective of task complexity.
This results in elevated computational costs and limited control over reasoning
quality. To address this problem, we propose PixelThink, a simple yet effective
scheme that integrates externally estimated task difficulty and internally
measured model uncertainty to regulate reasoning generation within a
reinforcement learning paradigm. The model learns to compress reasoning length
in accordance with scene complexity and predictive confidence. To support
comprehensive evaluation, we introduce ReasonSeg-Diff, an extended benchmark
with annotated reasoning references and difficulty scores, along with a suite
of metrics designed to assess segmentation accuracy, reasoning quality, and
efficiency jointly. Experimental results demonstrate that the proposed approach
improves both reasoning efficiency and overall segmentation performance. Our
work contributes novel perspectives towards efficient and interpretable
multimodal understanding. The code and model will be publicly available.