metadata
license: other
license_name: sample-code-license
license_link: LICENSE
library_name: viper-vpe
ViPer: Visual Personalization of Generative Models via Individual Preference Learning
Tuning-free framework for personalized image generation
We introduce ViPer, a method that personalizes the output of generative models to align with different users’ visual preferences for the same prompt. This is done via a one-time capture of the user’s general preferences and conditioning the generative model on them without the need for engineering detailed prompts.
Installation
For install instructions, please see https://github.com/EPFL-VILAB/ViPer.
Usage
This model can be loaded from Hugging Face Hub as follows:
from transformers import AutoProcessor, BitsAndBytesConfig, AutoModelForVision2Seq
from peft import PeftModel
model = AutoModelForVision2Seq.from_pretrained("HuggingFaceM4/idefics2-8b")
model = PeftModel.from_pretrained(model, "EPFL-VILAB/Metric-ViPer")
Please see https://github.com/EPFL-VILAB/ViPer for more detailed instructions.
Citation
If you find this repository helpful, please consider citing our work:
@article{ViPer,
title={{ViPer}: Visual Personalization of Generative Models via Individual Preference Learning},
author={Sogand Salehi and Mahdi Shafiei and Teresa Yeo and Roman Bachmann and Amir Zamir},
journal={arXiv preprint arXiv:2407.17365},
year={2024},
}
License
Licensed under the Apache License, Version 2.0. See LICENSE for details.