ViPer: Visual Personalization of Generative Models via Individual Preference Learning
Tuning-free framework for personalized image generation
We introduce ViPer, a method that personalizes the output of generative models to align with different users’ visual preferences for the same prompt. This is done via a one-time capture of the user’s general preferences and conditioning the generative model on them without the need for engineering detailed prompts.
Installation
For install instructions, please see https://github.com/EPFL-VILAB/ViPer.
Usage
This model can be loaded from Hugging Face Hub as follows:
from transformers import AutoModelForVision2Seq
from peft import PeftModel
model = AutoModelForVision2Seq.from_pretrained("HuggingFaceM4/idefics2-8b")
model = PeftModel.from_pretrained(model, "EPFL-VILAB/VPE-ViPer")
Please see https://github.com/EPFL-VILAB/ViPer for more detailed instructions.
For more examples and interactive demos, please see our website and Hugging Face Space.
Citation
If you find this repository helpful, please consider citing our work:
@article{ViPer,
title={{ViPer}: Visual Personalization of Generative Models via Individual Preference Learning},
author={Sogand Salehi and Mahdi Shafiei and Teresa Yeo and Roman Bachmann and Amir Zamir},
journal={arXiv preprint arXiv:2407.17365},
year={2024},
}
License
Licensed under the Apache License, Version 2.0. See LICENSE for details.