Title: LATTS: LAtent space Test Time Scaling for diffusion language models
Primary area: foundation or frontier models, including LLMs
Keywords: Diffusion Large Language Model, Test Time Scaling, Latent Reasoning

Abstract: Test-time scaling (TTS) improves the performance of autoregressive (AR) large language models by adding computation at inference.
While the prominent sequential TTS enhances accuracy by inducing models to generate longer chain-of-thought (CoT) reasoning, its computational overhead emerges as a drawback.
Meanwhile, diffusion large language models (DLLMs) have emerged as a promising alternative that offers parallel decoding and self-correction capabilities.
However, existing sequential TTS methods are incompatible with modern masked DLLMs.
This incompatibility arises from two fundamental constraints:
(1) DLLMs operate holistically on fixed-length sequences, preventing the dynamic token-level expansion required for CoT,
and (2) DLLMs' number of refinement (i.e., denoising) steps is capped by the sequence length, precluding further quality improvements via additional computation.
We introduce LATTS, a novel sequential TTS method for DLLMs that overcomes the above challenges by operating in the latent embedding space.
LATTS reframes CoT reasoning from a spatial process of extending sequence length to a temporal process that uses additional computation to extend the iterative self-refinement steps over the entire sequence's latent representation.
Our evaluation on the LLaDA-Instruct model shows that LATTS achieves notable improvements over SFT baselines on reasoning and code generation benchmarks
with gains of +4.1\% on GSM8K, +4.8\% on MATH, +3.2\% on MBPP, and an average of +4.6\% on commonsense reasoning tasks with minimal additional inference computation.
These results establish sequential TTS as a promising technique for optimizing DLLMs.

TL;DR: We introduce LATTS, a latent space test time scaling method to enhance the performance of masked diffusion language models.