ReSeed

A First-Step Bias Restores Lost Diversity in Diffusion Model Distillation

Paper · Project page · Code

ReSeed improves diversity in a frozen four-step student by fitting small residual biases to the teacher's first-interval velocity. The biases are enabled only in the first denoising step; the original weights and final three steps remain unchanged.

This repository provides four ReSeed adapters and two self-trained SD3 DMD students. The selected adapter runs and our two student runs use training seed 42. Generation seeds can be chosen independently.

Student models and their sources

SD3-Medium and SD3.5-Medium: trained by us

We trained these four-step students in our CRRA unified_distill codebase from the corresponding Stability AI models. Both released students are seed 42, checkpoint-2000. Their full transformer weights are included:

Student	Released weights	Teacher	Teacher sampling reference
SD3-Medium DMD	checkpoint-2000	SD3-Medium	28 steps, CFG 7
SD3.5-Medium DMD	checkpoint-2000	SD3.5-Medium	40 steps, CFG 4.5

These are our experimental students, not checkpoints from the official DMD2 release. They use four-step FlowMatch Euler inference at CFG 1. For the underlying methods, see DMD and DMD2.

The original training manifests are included beside each student. The SD3-Medium manifest retains the historical objective label tdm and records the implementation sd_distill_classic_v1; the SD3.5-Medium manifest records dmd. These records have not been relabelled. The release's exact student and adapter pairings are listed in release_manifest.json.

Wan2.1 DMD2: public checkpoint from the DFD authors

The video student is the Wan2.1-T2V-1.3B DMD2 checkpoint released by the Data-Forcing Distillation authors. We use dmd2_t2v_vipe/0001000, trained on the ViPE long set:

• Checkpoint files
• Teacher: Wan2.1-T2V-1.3B
• DFD / fastgen code and setup

We did not train this DMD2 student. We trained the ReSeed adapter on top of it and provide only that adapter here. The separately released DFD post-trained model (ours_wan_t2v_vipe/0000040) is a different baseline, not the student used by this adapter.

FLUX: public SenseFlow student

We use the four-step SenseFlow-FLUX transformer from domiso/SenseFlow, with shared components from FLUX.1-dev. This student was released by the SenseFlow authors, not trained by us. Download it separately; this repository contains its ReSeed adapter only.

Released adapters

Student	Adapter directory	Adapter updates	Bias parameters
SD3-Medium DMD	adapters/sd3m/seed_42/checkpoint-1900	1900	72,192
SD3.5-Medium DMD	adapters/sd35m/seed_42/checkpoint-2100	2100	72,192
SenseFlow-FLUX	adapters/flux/seed_42/checkpoint-2000	2000	233,472
Wan2.1 DMD2	adapters/wan21/seed_42/checkpoint-300	300	46,080

Each directory contains residual_biases.pt and manifest.json. Adapter updates count ReSeed fitting, not student distillation: both included SD3 students remain at update 2000. Use each adapter with its listed student. These are residual-bias adapters, not LoRA weights.

Download and install

Use Python 3.10+ and a CUDA/BF16-capable PyTorch installation. Our image runtime used PyTorch 2.6.0, Diffusers 0.34.0 and Transformers 4.45.2.

pip install huggingface_hub==0.36.2
hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights

git clone https://github.com/TWWinde/ReSeed.git
git -C ReSeed checkout eed9a03bd85a0af8a7dc36e8da5b48d2a31f58ca
pip install -r reseed-weights/requirements.txt
pip install -e ReSeed/image --no-deps

The full download is about 18.22 GB, mostly the two student transformers. To download only SD3-Medium, use:

hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights \
  --include README.md requirements.txt inference.py release_manifest.json \
  "adapters/sd3m/*" "students/sd3m/*"

Use sd35m instead of sd3m for SD3.5-Medium. For FLUX or Wan, only the corresponding adapter is needed from this repository. Upstream gated models require accepting their terms and logging in with hf auth login.

SD3 image generation

Download the public components for the model you want to run:

hf download stabilityai/stable-diffusion-3-medium-diffusers \
  --revision ea42f8cef0f178587cf766dc8129abd379c90671 --local-dir models/sd3m

hf download stabilityai/stable-diffusion-3.5-medium \
  --revision b940f670f0eda2d07fbb75229e779da1ad11eb80 --local-dir models/sd35m

Generate an image with the matching DMD student and ReSeed:

python reseed-weights/inference.py --weights reseed-weights --model sd3m \
  --base models/sd3m --prompt "a photo of a cat" --seed 0 \
  --output samples/sd3m_reseed.png

python reseed-weights/inference.py --weights reseed-weights --model sd35m \
  --base models/sd35m --prompt "a photo of a cat" --seed 0 \
  --output samples/sd35m_reseed.png

Add --without-reseed and choose a different output filename to generate the plain DMD baseline with the same prompt and initial noise.

The script loads our student transformer and the public model's text encoders, tokenizers, VAE and scheduler. It uses 1024×1024 BF16 inference, four Euler steps, CFG 1 and static shift 3. The pipeline receives unshifted sigmas [1, .75, .5, .25]; do not substitute the shifted boundaries from the training manifest. Initial noise is drawn in FP32 on CPU, then cast to BF16. The adapter is disabled after the first step.

FLUX image generation

hf download black-forest-labs/FLUX.1-dev \
  --revision 3de623fc3c33e44ffbe2bad470d0f45bccf2eb21 --local-dir models/flux-dev
hf download domiso/SenseFlow --revision 1bbd81546d590acf24470b39ab28f9cd918c3927 \
  --include "SenseFlow-FLUX/*" --local-dir models/senseflow

Create prompts.csv:

case_number,evaluation_seed,prompt,prompt_id,sample_id
0,0,a photo of a cat,0,0
1,1,a photo of a cat,0,1

Then run:

cd ReSeed/image
python tools/flux_generate.py --model senseflow \
  --dev-model ../../models/flux-dev \
  --senseflow-transformer ../../models/senseflow/SenseFlow-FLUX \
  --first-interval-residual-bias-checkpoint ../../reseed-weights/adapters/flux/seed_42/checkpoint-2000 \
  --prompts ../../prompts.csv --expected-count 2 --samples-per-prompt 2 \
  --evaluation-protocol reseed-example \
  --steps 4 --resolution 1024 --max-sequence-length 512 \
  --output ../../samples/flux_reseed
cd ../..

Omit the adapter argument for the plain SenseFlow baseline. This student uses its native four-step dynamically shifted Euler schedule with the guidance embedding disabled. Do not pass the teacher's guidance 3.5 to the student. The transformer is in SenseFlow-FLUX, not a nested transformer/ folder.

Wan video generation

First follow the ReSeed video setup to install the video code and apply the ReSeed inference patch to DFD. The video runtime has separate dependencies from the image package.

hf download csusupergear/wan_t2v_checkpoints_release \
  --include "dmd2_t2v_vipe/0001000*" --local-dir wan-checkpoints

Keep the .pth and .net_model.zip files under their original names. From the configured DFD checkout, use a text file with one prompt per line:

export DFD_CKPT_ROOT=/absolute/path/to/wan-checkpoints
export DFD_RESEED_ADAPTER=/absolute/path/to/reseed-weights/adapters/wan21/seed_42/checkpoint-300
export DFD_PROMPT_FILE=/absolute/path/to/video-prompts.txt
python evaluation/generate_multiseed.py --config evaluation/config.yaml \
  --models dmd2 dmd2_reseed --seeds 0 1 \
  --output-root evaluation/reseed-example

This generates matched DMD2 and DMD2 + ReSeed videos. The student uses four ODE steps with boundaries [.999, .937, .833, .624, 0], no student CFG, 81 frames at 832×480, 16 FPS and BF16. The teacher reference uses 50 UniPC steps, CFG 5 and shift 5. ReSeed is active only in the first student step.

Evaluation prompts and noise

The image diversity evaluation uses 100 COCO prompts × 10 noise seeds:

• 100 prompts
• 1,000 prompt/seed cases
• Noise generator and protocol

These inputs are shared by SD3-Medium, SD3.5-Medium and SenseFlow-FLUX at 1024×1024. Case i uses seed 2026081900 + i. Each noise tensor has shape [16, 128, 128], drawn on CPU in FP32 before the model's BF16 cast or FLUX packing. We publish the seeds and generator rather than 1 GB of tensor files. Use PyTorch 2.6.0 to reconstruct the reference inputs:

hf download Wenwu2000/ReSeed --revision main --local-dir reseed-weights \
  --include "evaluation/diversity_100x10/*"

# Export one case; omit --case to export all 1,000 tensors (about 1 GB).
python reseed-weights/evaluation/diversity_100x10/generate_noise.py \
  --case 0 --output evaluation-noise-case0

The export includes a SHA-256 hash of every generated FP32 tensor. This is the image diversity bank, not the COCO5K quality bank or Wan video inputs. The case records match the original evaluation list; only CRLF line endings have been changed to LF for this release, with both hashes in the protocol.

Files and licenses

The release contains inference weights, not optimizer state, fake-score models, public teacher weights or generated samples. Text encoders and VAEs come from the linked base models. File hashes, pinned source revisions and checkpoint pairings are recorded in release_manifest.json; its branch and visibility fields describe the original private release. The repository is now public, with current instructions on main and the original single-seed branch retained as an archive.

The ReSeed code is MIT-licensed. Model weights retain their upstream licenses and access conditions; see NOTICE.

Citation

@misc{tang2026reseed,
  title={A First-Step Bias Restores Lost Diversity in Diffusion Model Distillation},
  author={Wenwu Tang and Dong Wang and Yun Cheng and Olga Saukh},
  year={2026},
  eprint={2610.37110},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2610.37110}
}