Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive
  Reinforcement
Authors: name: Yuqi Liu; name: Bohao Peng; name: Zhisheng Zhong; name: Zihao Yue; name: Fanbin Lu; name: Bei Yu; name: Jiaya Jia
Published: 2025-03-09 08:48:51

Abstract: Traditional methods for reasoning segmentation rely on supervised fine-tuning
with categorical labels and simple descriptions, limiting its out-of-domain
generalization and lacking explicit reasoning processes. To address these
limitations, we propose Seg-Zero, a novel framework that demonstrates
remarkable generalizability and derives explicit chain-of-thought reasoning
through cognitive reinforcement. Seg-Zero introduces a decoupled architecture
consisting of a reasoning model and a segmentation model. The reasoning model
interprets user intentions, generates explicit reasoning chains, and produces
positional prompts, which are subsequently used by the segmentation model to
generate precious pixel-level masks. We design a sophisticated reward mechanism
that integrates both format and accuracy rewards to effectively guide
optimization directions. Trained exclusively via reinforcement learning with
GRPO and without explicit reasoning data, Seg-Zero achieves robust zero-shot
generalization and exhibits emergent test-time reasoning capabilities.
Experiments show that Seg-Zero-7B achieves a zero-shot performance of 57.5 on
the ReasonSeg benchmark, surpassing the prior LISA-7B by 18\%. This significant
improvement highlights Seg-Zero's ability to generalize across domains while
presenting an explicit reasoning process. Code is available at
https://github.com/dvlab-research/Seg-Zero.