馃獎SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing
Zeyinzi Jiang 路 Chaojie Mao 路 Yulin Pan 路 Zhen Han 路 Jingfeng Zhang
Alibaba Group
|
SCEdit is an efficient generative fine-tuning framework proposed by Alibaba TongYi Vision Intelligence Lab. This framework enhances the fine-tuning capabilities for text-to-image generation downstream tasks and enables quick adaptation to specific generative scenarios, saving 30%-50% of training memory costs compared to LoRA. Furthermore, it can be directly extended to controllable image generation tasks, requiring only 7.9% of the parameters that ControlNet needs for conditional generation and saving 30% of memory usage. It supports various conditional generation tasks including edge maps, depth maps, segmentation maps, poses, color maps, and image completion.
Use Models
pip install scepter
python -m scepter.tools.webui
BibTeX
@article{jiang2023scedit,
title = {SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing},
author = {Jiang, Zeyinzi and Mao, Chaojie and Pan, Yulin and Han, Zhen and Zhang, Jingfeng},
year = {2023},
journal = {arXiv preprint arXiv:2312.11392}
}