ICML 2025

GMAIL: Generative Modality Alignment for generated Image Learning
Shentong Mo 1 2 Sukmin Yun 3
Abstract
Generative models have made it possible to syn-
thesize highly realistic images, potentially provid-
ing an abundant data source for training machine
learning models. Despite the advantages of these
synthesizable data sources, the indiscriminate use
of generated images as real images for training
can even cause mode collapse due to modality
discrepancies between real and synthetic domains.
In this paper, we propose a novel framework for
discriminative use of generated images, coined
GMAIL, that explicitly treats generated images as
a separate modality from real images. Instead of
indiscriminately replacing real images with gener-
ated ones in the pixel space, our approach bridges
the two distinct modalities in the same latent space
through a multi-modal learning approach. To be
specific, we first fine-tune a model exclusively on
generated images using a cross-modality align-
ment loss and then employ this aligned model to
further train various vision-language models with
generated images. By aligning the two modalities,
our approach effectively leverages the benefits
of recent advances in generative models, thereby
boosting the effectiveness of generated image
learning across a range of vision-language tasks.
Our framework can be easily incorporated with
various vision-language models, and we demon-
strate its efficacy throughout extensive experi-
ments. For example, our framework significantly
improves performance on image captioning, zero-
shot image retrieval, zero-shot image classifica-
tion, and long caption retrieval tasks. It also shows
positive generated data scaling trends and notable
enhancements in the captioning performance of
the large multimodal model, LLaVA.
1Department of Machine Learning, CMU, USA 2Department
of Machine Learning, MBZUAI, UAE 3Department of Artificial
Intelligence, Hanyang University ERICA, South Korea. Corre-
spondence to: Sukmin ...