Memory-efficient Diffusion Transformers with Quanto and Diffusers
•
52
Sayak Paul
sayakpaul
AI & ML interests
Diffusion models, representation learning
Articles
Organizations
Posts
12
Post
1838
Did some little experimentation to resize pre-trained LoRAs on Flux. I explored two themes:
* Decrease the rank of a LoRA
* Increase the rank of a LoRA
The first one is helpful in reducing memory requirements if the LoRA is of a high rank, while the second one is merely an experiment. Another implication of this study is in the unification of LoRA ranks when you would like to
Check it out here:
sayakpaul/flux-lora-resizing
* Decrease the rank of a LoRA
* Increase the rank of a LoRA
The first one is helpful in reducing memory requirements if the LoRA is of a high rank, while the second one is merely an experiment. Another implication of this study is in the unification of LoRA ranks when you would like to
torch.compile() them. Check it out here:
sayakpaul/flux-lora-resizing
Post
2301
Here is a hackable and minimal implementation showing how to perform distributed text-to-image generation with Diffusers and Accelerate.
Full snippet is here: https://gist.github.com/sayakpaul/cfaebd221820d7b43fae638b4dfa01ba
With @JW17
Full snippet is here: https://gist.github.com/sayakpaul/cfaebd221820d7b43fae638b4dfa01ba
With @JW17
Collections
1
Provides a list of papers focusing on optimizing T2I diffusion models, targeting fewer timesteps, architecture optimization, and more.
-
Progressive Distillation for Fast Sampling of Diffusion Models
Paper • 2202.00512 • Published • 1 -
On Distillation of Guided Diffusion Models
Paper • 2210.03142 • Published -
InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation
Paper • 2309.06380 • Published • 32 -
Consistency Models
Paper • 2303.01469 • Published • 6
spaces
19
Running
15
📈
Demo Docker Gradio
Runtime error
🧨
Diffusers Docs QA Chatbot
Ask questions to the Diffusers documentation.
Build error
7
🧨
Convert Kerascv SD to Diffusers
Runtime error
🚀
Inpainting Tool
Build error
13
🐶
Generate Custom Pokemons with Stable Diffusion
Sleeping
8
⏰
Evaluate StableDiffusionPipeline with Different Schedulers
models
60
sayakpaul/tiny-sd-pipeline-with-lora
Text-to-Image
•
Updated
•
20
•
1
sayakpaul/tiny-flux-pipeline-with-lora
Text-to-Image
•
Updated
•
28
•
1
sayakpaul/flux-lora-resizing
Updated
•
76
•
14
sayakpaul/cog-t5-nf4
Updated
•
32
•
1
sayakpaul/flux.1-dev-int8-pkg
Updated
•
2
sayakpaul/flux.1-schell-int8wo-improved
Updated
•
228
•
2
sayakpaul/flux.1-dev-nf4-pkg
Updated
•
7
sayakpaul/flux.1-dev-nf4-with-bnb-integration
Updated
•
137
•
10
sayakpaul/fp8-dog-lora-flux
Text-to-Image
•
Updated
•
24
•
sayakpaul/flux.1-dev-nf4
Updated
•
882
•
14
datasets
27
sayakpaul/sample-datasets
Viewer
•
Updated
•
4
•
2
•
1
sayakpaul/poses-controlnet-dataset
Viewer
•
Updated
•
496
•
129
•
4
sayakpaul/torchao-diffusers
Updated
•
2
sayakpaul/pickapic_v2_webdataset
Viewer
•
Updated
•
8.7k
•
2
sayakpaul/generated-gemini-responses
Viewer
•
Updated
•
115
•
2
sayakpaul/no_robots_only_coding
Viewer
•
Updated
•
350
•
2
•
1
sayakpaul/diffusers-qa-chatbot-artifacts
Viewer
•
Updated
•
265k
•
2
•
1
sayakpaul/mgie-results
Viewer
•
Updated
•
8
•
2
sayakpaul/coco-30-val-2014
Viewer
•
Updated
•
30k
•
11.7k
•
4
sayakpaul/drawbench-sdxl-refiner
Viewer
•
Updated
•
200
•
7
•
1