|
--- |
|
license: other |
|
license_name: sample-code-license |
|
license_link: LICENSE |
|
library_name: viper-vpe |
|
--- |
|
|
|
# ViPer: Visual Personalization of Generative Models via Individual Preference Learning |
|
|
|
*Tuning-free framework for personalized image generation* |
|
|
|
[`Website`](https://viper.epfl.ch) | [`GitHub`](https://github.com/EPFL-VILAB/ViPer) | [`BibTeX`](#citation) |
|
|
|
|
|
We introduce **ViPer**, a method that personalizes the output of generative models to align with different users’ visual preferences for the same prompt. This is done via a one-time capture of the user’s general preferences and conditioning the generative model on them without the need for engineering detailed prompts. |
|
|
|
|
|
## Installation |
|
For install instructions, please see https://github.com/EPFL-VILAB/ViPer. |
|
|
|
|
|
## Usage |
|
|
|
This model can be loaded from Hugging Face Hub as follows: |
|
|
|
```python |
|
from transformers import AutoProcessor, BitsAndBytesConfig, AutoModelForVision2Seq |
|
from peft import PeftModel |
|
|
|
model = AutoModelForVision2Seq.from_pretrained("HuggingFaceM4/idefics2-8b") |
|
model = PeftModel.from_pretrained(model, "EPFL-VILAB/Metric-ViPer") |
|
``` |
|
|
|
Please see https://github.com/EPFL-VILAB/ViPer for more detailed instructions. |
|
|
|
## Citation |
|
|
|
If you find this repository helpful, please consider citing our work: |
|
``` |
|
@article{ViPer, |
|
title={{ViPer}: Visual Personalization of Generative Models via Individual Preference Learning}, |
|
author={Sogand Salehi and Mahdi Shafiei and Teresa Yeo and Roman Bachmann and Amir Zamir}, |
|
journal={arXiv preprint arXiv:2407.17365}, |
|
year={2024}, |
|
} |
|
``` |
|
|
|
## License |
|
|
|
Licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/sogandstorme/ViPer_Personalization/blob/main/LICENSE) for details. |
