Sample-Aligned Decoders for Latent Generative Models
Yu Lei1,2, Yi Li2, Hongcheng Gao3, Yipeng Wei2, Liang Li1, Li Su2, Qingming Huang2
1Institute of Computing Technology, Chinese Academy of Sciences
2University of Chinese Academy of Sciences 3Tsinghua University
Paper · Project page · Code
These are drop-in decoders for latent generative models. Each file holds only the decoder (decoder.*, post_quant_conv.*) of
the original autoencoder. It was fine-tuned with SAD on latents from one generator. The encoder, the latent space and the
generator stay unchanged, and so does the inference cost.
Usage
pip install git+https://github.com/torpedo2003/SAD
from diffusers import DiTPipeline
from sad import load_sad_decoder
pipe = DiTPipeline.from_pretrained("facebook/DiT-XL-2-256").to("cuda")
load_sad_decoder(pipe.vae, "dit-xl2-256/sad-gan") # swaps the decoder in place
images = pipe(class_labels=[279, 388], num_inference_steps=250, guidance_scale=1.5).images
You can also load the weights without our package:
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
pipe.vae.load_state_dict(load_file(hf_hub_download("torpedo2003/SAD", "dit-xl2-256/sad-gan.safetensors")), strict=False)
The VA-VAE and E2E-VAE decoders are not diffusers models. Load them into the autoencoder classes of the
LightningDiT and REPA-E code bases,
using the same load_sad_decoder call.
Files
Each decoder is trained on latents from its own generator and sampler, so use the one that matches your pipeline.
| File | Generator | Tokenizer | Sampler |
|---|---|---|---|
dit-xl2-256/sad-gan.safetensors |
DiT-XL/2 | SD-VAE (ft-EMA) | official 250-step DDPM, cfg 1.5 |
dit-xl2-256/sad-mmd.safetensors |
DiT-XL/2 | SD-VAE (ft-EMA) | official 250-step DDPM, cfg 1.5 |
dit-xl2-256-dpm50/sad-gan.safetensors |
DiT-XL/2 | SD-VAE (ft-EMA) | 50-step DPM-Solver++, cfg 1.5 |
dit-xl2-256-dpm50/sad-mmd.safetensors |
DiT-XL/2 | SD-VAE (ft-EMA) | 50-step DPM-Solver++, cfg 1.5 |
sit-xl2-repa-256/sad-gan.safetensors |
SiT-XL/2 + REPA | SD-VAE (ft-EMA) | SDE 250 steps |
lightningdit-xl-vavae-256/sad-gan.safetensors |
LightningDiT-XL | VA-VAE | official recipe |
repa-e-e2evae-256/sad-gan.safetensors |
REPA-E (SiT-XL/1) | E2E-VAE | SDE 250 steps |
pixart-alpha-256/sad-gan.safetensors |
PixArt-α 256px | SD-VAE (ft-EMA) | 20 steps, guidance 4.5 |
Every decoder was trained for 2,000 steps, about one GPU-hour. Training used the GAN discrepancy in DINOv2 space (λ = 0.05 on ImageNet, 0.01 for PixArt-α) or MMD (λ = 1), plus the L1 encoder anchor. To train a decoder for your own generator, see the training guide.
Citation
@article{lei2026sad,
title = {Sample-Aligned Decoders for Latent Generative Models},
author = {Lei, Yu and Li, Yi and Gao, Hongcheng and Wei, Yipeng and Li, Liang and Su, Li and Huang, Qingming},
journal = {arXiv preprint arXiv:2611.22458},
year = {2026}
}