Boosting Generative Image Modeling via Joint Image-Feature Synthesis
Authors: name: Theodoros Kouzelis; name: Efstathios Karypidis; name: Ioannis Kakogeorgiou; name: Spyros Gidaris; name: Nikos Komodakis
Published: 2025-04-22 17:41:42

Abstract: Latent diffusion models (LDMs) dominate high-quality image generation, yet
integrating representation learning with generative modeling remains a
challenge. We introduce a novel generative image modeling framework that
seamlessly bridges this gap by leveraging a diffusion model to jointly model
low-level image latents (from a variational autoencoder) and high-level
semantic features (from a pretrained self-supervised encoder like DINO). Our
latent-semantic diffusion approach learns to generate coherent image-feature
pairs from pure noise, significantly enhancing both generative quality and
training efficiency, all while requiring only minimal modifications to standard
Diffusion Transformer architectures. By eliminating the need for complex
distillation objectives, our unified design simplifies training and unlocks a
powerful new inference strategy: Representation Guidance, which leverages
learned semantics to steer and refine image generation. Evaluated in both
conditional and unconditional settings, our method delivers substantial
improvements in image quality and training convergence speed, establishing a
new direction for representation-aware generative modeling.