MIRO: MultI-Reward cOnditioned pretraining improves T2I quality and
  efficiency
Authors: name: Nicolas Dufour, name: Lucas Degeorge, name: Arijit Ghosh, name: Vicky Kalogeiton, name: David Picard

Current text-to-image generative models are trained on large uncurated
datasets to enable diverse generation capabilities. However, this does not
align well with user preferences. Recently, reward models have been
specifically designed to perform post-hoc selection of generated images and
align them to a reward, typically user preference. This discarding of
informative data together with the optimizing for a single reward tend to harm
diversity, semantic fidelity and efficiency. Instead of this post-processing,
we propose to condition the model on multiple reward models during training to
let the model learn user preferences directly. We show that this not only
dramatically improves the visual quality of the generated images but it also
significantly speeds up the training. Our proposed method, called MIRO,
achieves state-of-the-art performances on the GenEval compositional benchmark
and user-preference scores (PickAScore, ImageReward, HPSv2).