Cuttle-fish-my/Rethinking-CFG-OPD-ckpts

Rethinking CFG in On-Policy Distillation — checkpoints

Paper: Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation
Project page: https://rethinking-cfg-opd.github.io
Code: https://github.com/rethinking-cfg-opd/Rethinking-CFG-OPD

LoRA adapters from our study of classifier-free guidance in on-policy distillation (OPD). One folder per experimental setting, named as the paper names it. Every adapter is a LoRA over a frozen base model, so a checkpoint is tens of MB rather than the full backbone.

Folder	Setting	Base model
dense-to-sparse-video/	Dense→sparse video control: a generator receiving pose/depth/scribble on every frame distilled into one that only gets 4 keyframes	Wan-AI/Wan2.1-VACE-1.3B
text-rendering/	Single-teacher OCR distillation on SD3.5-Medium — the shared negative-conditioning setting, where PDM and naive come out level	stabilityai/stable-diffusion-3.5-medium
reference-conditioned/	Style distillation on FLUX.2-klein-4B-base — the privileged negative-conditioning setting, where the teacher sees a reference image the student does not, and where NBA appears	black-forest-labs/FLUX.2-klein-4B-base

The objectives

Matching only the CFG-composed velocity constrains the branch errors e₊, e₋ solely through γe₊ + (1−γ)e₋ = 0. That is under-identified at the branch level: the two errors can compensate for each other instead of both going to zero. When the teacher's negative branch carries information the student cannot see, the compensation turns antagonistic — ‖e₊‖ falls while ‖e₋‖ grows. It stays hidden at the training guidance scale and surfaces as guidance-scale sensitivity at inference.

	objective
naive	‖γe₊ + (1−γ)e₋‖²	matches only the composed transition
PDM	‖e₊‖² + λ‖d_T − d_S‖², d = v⁺ − v⁻	supervises before composition — ours
IBM	γ²‖e₊‖² + (γ−1)²‖e₋‖²	branch-aware foil

e₊ = v_S⁺ − v_T⁺, e₋ = v_S⁻ − v_T⁻.

Naming

<setting>/<objective>_<variant>

e.g. dense-to-sparse-video/pdm_unicontrol.safetensors — the PDM objective, unified control (pose + depth + scribble jointly); text-rendering/pdm_lambda2.safetensors — PDM at λ=2.

reference-conditioned/pdm_lambda2/ is a directory, not a bare .safetensors: it is a PEFT dual-adapter checkpoint, so student/ and teacher/ each need their adapter_config.json alongside the weights for PeftModel.from_pretrained to load them. The student is the deliverable; the teacher is the EMA it was distilled from, kept because reproducing the branch-error diagnostic needs it.

Each folder carries its own README.md with the training configuration, the training step the checkpoint was selected at, and the numbers it reproduces, plus a MANIFEST.sha256.

Licence

Apache-2.0. Each base model carries its own terms.