leffff/diffusion-distillation-workshop-adapters

Diffusion Distillation Workshop: 4-step Sana-600M students

LoRA adapters that distill Sana-600M-512px from 40 NFE (20 steps with CFG 4.5) into 4-NFE students using four methods. They were trained with the workshop notebook on one H100. Data: leffff/diffusion-distillation-workshop.

file	method	sampler (4 NFE)	training	CLIP ↑	diversity	latent MSE to teacher
teacher	—	20 Euler steps + CFG (40 NFE)	—	32.29	0.075	0
teacher	—	4 Euler steps + CFG (8 NFE)	—	30.73	0.117	0.54
adapter_cm.pt	multistep (phased) consistency distillation	Euler	5 min, 933 it × 16	32.67	0.088	0.41
adapter_ladd.pt	LADD (adversarial, teacher-feature discriminator)	x0 + renoise	5 min, 853 it × 16	31.95	0.101	1.42
adapter_dmd.pt	DMD2 (reverse KL, fake score, backward simulation)	x0 + renoise	30 min, 3101 it × 16, LR 1e-5	31.25	0.091	1.01
adapter_pdd.pt	Parallel Decoding Distillation (N=32, L=8, midpoint)	4 block-Euler steps	5 min, 866 it × 16	32.88	0.093	0.42

Metrics: 6 eval prompts × 4 seeds, CLIP ViT-B/32. Trajectory methods (CM, PDD) keep the teacher's noise→image map (low MSE); distribution methods (LADD, DMD) do not.

Note on DMD: the same config with LR 5e-5 (3100 it × 16) reached CLIP 32.37. That adapter is not included here.

Format

Each .pt file is a dict produced by the notebook's save_adapter: a peft LoRA state dict (get_peft_model_state_dict; r=32 for CM/LADD/PDD, r=128 for DMD; targets to_q, to_k, to_v, to_out.0, conv_inverted, conv_point, proj_out.base). adapter_pdd.pt also contains grid.w / grid.b: PDD's 32 output heads.

Usage (in the workshop notebook)

from huggingface_hub import hf_hub_download
import shutil
for m in ["cm", "ladd", "dmd", "pdd"]:
    shutil.copy(hf_hub_download("leffff/diffusion-distillation-workshop-adapters", f"adapter_{m}.pt"), f"{CACHE_DIR}/adapter_{m}.pt")
    load_adapter(m)