SVDQuant
Collection
Models and datasets for "SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models"
•
6 items
•
Updated
SVDQuant seamlessly integrates with off-the-shelf LoRAs without requiring re-quantization. When applying LoRAs, it matches the image quality of the original 16-bit FLUX.1-dev.
Please follow the instructions in mit-han-lab/nunchaku to set up the environment. Then you can run the model with
import torch
from nunchaku.pipelines import flux as nunchaku_flux
pipeline = nunchaku_flux.from_pretrained(
"black-forest-labs/FLUX.1-dev",
torch_dtype=torch.bfloat16,
qmodel_path="mit-han-lab/svdq-int4-flux.1-dev", # download from Huggingface
).to("cuda")
pipeline.transformer.nunchaku_update_params(mit-han-lab/svdquant-models/svdq-flux.1-dev-lora-anime.safetensors)
pipeline.transformer.nunchaku_set_lora_scale(1)
image = pipeline("a dog wearing a wizard hat", num_inference_steps=28, guidance_scale=3.5).images[0]
image.save("example.png")
Work in progress.
If you find this model useful or relevant to your research, please cite
@article{
li2024svdquant,
title={SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models},
author={Li*, Muyang and Lin*, Yujun and Zhang*, Zhekai and Cai, Tianle and Li, Xiuyu and Guo, Junxian and Xie, Enze and Meng, Chenlin and Zhu, Jun-Yan and Han, Song},
journal={arXiv preprint arXiv:2411.05007},
year={2024}
}
Base model
black-forest-labs/FLUX.1-dev