Test-Time Scaling with Reflective Generative Model
Authors: name: Zixiao Wang, name: Yuxin Wang, name: Xiaorui Wang, name: Mengting Xing, name: Jie Gao, name: Jianjun Xu, name: Guangcan Liu, name: Chenhui Jin, name: Zhuo Wang, name: Shengzhuo Zhang, name: Hongtao Xie

We introduce our first reflective generative model MetaStone-S1, which
obtains OpenAI o3's performance via the self-supervised process reward model
(SPRM). Through sharing the backbone network and using task-specific heads for
next token prediction and process scoring respectively, SPRM successfully
integrates the policy model and process reward model(PRM) into a unified
interface without extra process annotation, reducing over 99% PRM parameters
for efficient reasoning. Equipped with SPRM, MetaStone-S1 is naturally suitable
for test time scaling (TTS), and we provide three reasoning effort modes (low,
medium, and high), based on the controllable thinking length. Moreover, we
empirically establish a scaling law that reveals the relationship between total
thinking computation and TTS performance. Experiments demonstrate that our
MetaStone-S1 achieves comparable performance to OpenAI-o3-mini's series with
only 32B parameter size. To support the research community, we have
open-sourced MetaStone-S1 at https://github.com/MetaStone-AI/MetaStone-S1.