SoftCoT++: Test-Time Scaling with Soft Chain-of-Thought Reasoning
Authors: name: Yige Xu, name: Xu Guo, name: Zhiwei Zeng, name: Chunyan Miao

Test-Time Scaling (TTS) refers to approaches that improve reasoning
performance by allocating extra computation during inference, without altering
the model's parameters. While existing TTS methods operate in a discrete token
space by generating more intermediate steps, recent studies in Coconut and
SoftCoT have demonstrated that thinking in the continuous latent space can
further enhance the reasoning performance. Such latent thoughts encode
informative thinking without the information loss associated with
autoregressive token generation, sparking increased interest in
continuous-space reasoning. Unlike discrete decoding, where repeated sampling
enables exploring diverse reasoning paths, latent representations in continuous
space are fixed for a given input, which limits diverse exploration, as all
decoded paths originate from the same latent thought. To overcome this
limitation, we introduce SoftCoT++ to extend SoftCoT to the Test-Time Scaling
paradigm by enabling diverse exploration of thinking paths. Specifically, we
perturb latent thoughts via multiple specialized initial tokens and apply
contrastive learning to promote diversity among soft thought representations.
Experiments across five reasoning benchmarks and two distinct LLM architectures
demonstrate that SoftCoT++ significantly boosts SoftCoT and also outperforms
SoftCoT with self-consistency scaling. Moreover, it shows strong compatibility
with conventional scaling techniques such as self-consistency. Source code is
available at https://github.com/xuyige/SoftCoT.