Title: Next-Scale Autoregressive Models are Zero-Shot Single-Image Object View Synthesizers
Primary area: generative models
Keywords: novel view synthesis, generative modeling, autoregressive modeling

Abstract: Learning to synthesize novel views without explicit 3D representations or hand-crafted 3D inductive bias has recently gained attention: it is simpler, more formally direct, and better aligned with the lesson that scalable learning paradigms with less assumptions built into architectural design (e.g. regarding geometry) often win. However, the current dominant solutions are diffusion-based, which typically inherit heavy 2D pretraining (e.g., from Stable Diffusion) and suffer from slow inference. We introduce ArchonView, the first next-scale autoregressive model for zero-shot single-image, object-centric novel view synthesis (NVS) that requires no 2D pretraining and achieves substantially faster inference. We design innovative methods of both global and local conditioning to suit characteristics of the NVS task. Crucially, a naïve application of next-scale autoregression fails; we identify two design choices that unlock performance: local conditioning pre-filling, and removing global AdaLN at the classifier head. ArchonView delivers state-of-the-art zero-shot results across six standard benchmarks (GSO, ABO, OmniObject3D, RTMV, NeRF-Synthetic, ShapeNet), while being several times faster than diffusion baselines (e.g., 0.22s v.s. 1.7–1.8s per view at matched parameter count). It consistently improves synthesis accuracy, and scales predictably with both model size (135M–2B) and data size, exhibiting clear scaling-law-like trends. Our findings suggest a paradigm shift: for object-centric NVS, next-scale autoregression can be faster, simpler, and more accurate than diffusion, notably without fine-tuning 2D-pretrained models. Our code is open-sourced at https://anonymous.4open.science/r/ArchonView/.