ECLIPSE: A Resource-Efficient Text-to-Image Prior for Image Generations

Overview

The rapid advancement of generative models facilitates the creation of hyper-realistic images from textual descriptions. However, this is achieved by increasing the training data and model parameters. This is not an efficient utilization of resources. Many of the recent methods focus on knowledge distillation while leaving the resource requirements of the main teacher model from the training stage as it is.

To address this challenge, we attempt to approach this from the unCLIP T2I family stack and show that Text-to-Image prior can be trained by merely 33M parameters using 5M image-text pairs. This ECLIPSE prior model can be trained within the 50 GPU hours.

We want to open-source the demo to promote more research on robustness and efficient T2I models. This is just the beginning, and combining ECLIPSE prior with other training and distillation strategies such as Pixart-Alpha and LCM can further compress the existing T2I models.

Project Details

• Project Page: https://eclipse-t2i.vercel.app/
• PDF Paper: Paper (ArXiv paper will be released with the next announcement)