Title: Horizon Imagination: Efficient On-Policy Training in Diffusion World Models
Primary area: reinforcement learning
Keywords: world models, diffusion, model-basedreinforcement learning

Abstract: We study diffusion-based world models for reinforcement learning, which offer high generative fidelity but face critical efficiency challenges in control. 
Current methods either require heavyweight models at inference or rely on highly sequential imagination, both of which impose prohibitive computational costs. 
We propose Horizon Imagination, an on-policy imagination process for discrete stochastic policies that denoises multiple future observations in parallel, coupled with a stabilization mechanism and a novel sampling schedule that decouples denoising budget from decay horizon, i.e., the effective number of observations over which denoising is applied. 
Experiments on Atari 100K and Craftium show that our approach maintains control performance with half the denoising budget and achieves superior generation quality under varied schedules. 
Code is released open-source https://anonymous.4open.science/r/horizon-3EA1.