Equivariant Image Modeling
Authors: name: Ruixiao Dong; name: Mengde Xu; name: Zigang Geng; name: Li Li; name: Han Hu; name: Shuyang Gu
Published: 2025-03-24 17:59:57

Abstract: Current generative models, such as autoregressive and diffusion approaches,
decompose high-dimensional data distribution learning into a series of simpler
subtasks. However, inherent conflicts arise during the joint optimization of
these subtasks, and existing solutions fail to resolve such conflicts without
sacrificing efficiency or scalability. We propose a novel equivariant image
modeling framework that inherently aligns optimization targets across subtasks
by leveraging the translation invariance of natural visual signals. Our method
introduces (1) column-wise tokenization which enhances translational symmetry
along the horizontal axis, and (2) windowed causal attention which enforces
consistent contextual relationships across positions. Evaluated on
class-conditioned ImageNet generation at 256x256 resolution, our approach
achieves performance comparable to state-of-the-art AR models while using fewer
computational resources. Systematic analysis demonstrates that enhanced
equivariance reduces inter-task conflicts, significantly improving zero-shot
generalization and enabling ultra-long image synthesis. This work establishes
the first framework for task-aligned decomposition in generative modeling,
offering insights into efficient parameter sharing and conflict-free
optimization. The code and models are publicly available at
https://github.com/drx-code/EquivariantModeling.