Planned Diffusion
Authors: name: Daniel Israel, name: Tian Jin, name: Ellie Cheng, name: Guy Van den Broeck, name: Aditya Grover, name: Suvinay Subramanian, name: Michael Carbin

A central challenge in large language model inference is the trade-off
between generation speed and output quality. Autoregressive models produce
high-quality text but generate tokens sequentially. Diffusion models can
generate tokens in parallel but often need many iterations to match the same
quality. We propose planned diffusion, a hybrid method that combines the
strengths of both paradigms. Planned diffusion works in two stages: first, the
model creates a short autoregressive plan that breaks the output into smaller,
independent spans. Second, the model generates these spans simultaneously using
diffusion. This approach expands the speed-quality Pareto frontier and provides
a practical path to faster, high-quality text generation. On AlpacaEval, a
suite of 805 instruction-following prompts, planned diffusion achieves
Pareto-optimal trade-off between quality and latency, achieving 1.27x to 1.81x
speedup over autoregressive generation with only 0.87\% to 5.4\% drop in win
rate, respectively. Our sensitivity analysis shows that the planning mechanism
of planned diffusion is minimal and reliable, and simple runtime knobs exist to
provide flexible control of the quality-latency trade-off.