dollyx/vdc-five-attributes-sd14

VDC: five one-shot visual conditions

Research reproduction artifacts for DeRain, DeHaze, DeBlur, DeNoise and SR. Each attribute completed 200 optimization updates from one paired visual example, with four paired augmentations per update. The Stable Diffusion 1.4 backbone was frozen. These are learned VDC steering conditions and their MLP training states, not standalone diffusion models or the original authors' released weights.

Based on Visual Diffusion Conditioning by Omar Elezabi, Eduard Zamfir, Zongwei Wu and Radu Timofte; original implementation.

Contents

Each lowercase attribute directory contains:

• learned_conds.npy: inference-ready per-step conditions, shape (10,77,768).
• training-state-step-0200.pt: ten MLP state dictionaries, mapping inputs, conditions, Adam, cosine scheduler, AMP scaler, RNG states and run identity.
• conditions-step-0200.npy: the checkpoint's condition export.
• inference.png: one held-out RGB 512x512 example output.
• experiment.json, progress.json: exact settings, provenance and history.
• artifacts.tar.gz, export-ready.json: original verified training export.

reproduction/code.tar.gz is the exact immutable training source snapshot, including the adapted runner. reproduction/attributes.json and reproduction/data-manifest.json record input locations, original author-release links, preprocessing and hashes. Base SD/CLIP weights and source dataset images are obtained separately; they are not included here. Original experiment paths refer to the Colab/Kaggle environments and must be relocated locally.

Download and inference

After authenticating to access this private repository:

import numpy as np
from huggingface_hub import hf_hub_download

conditions_path = hf_hub_download(
    repo_id="dollyx/vdc-five-attributes-sd14", filename="derain/learned_conds.npy",
    revision="v1.0.0",
)
conditions = np.load(conditions_path, allow_pickle=False)
assert conditions.shape == (10, 77, 768)

To run inference, extract the source snapshot into a separate workspace, set up its original environment, and obtain SD 1.4 and CLIP as described in vdc/README.md and vdc/colab/setup.sh. From its vdc directory, use the actual path returned above and an RGB 512x512 input image:

PYTHONPATH=. .venv-colab/bin/python scripts/CondMapping_Aug.py \
  --cond_path /absolute/path/to/derain/learned_conds.npy \
  --inference-imgs /absolute/path/to/input.png \
  --outdir outputs/derain-inference \
  --ckpt models/sd-v1-4.ckpt \
  --ddim_steps 100 --strength 0.1 --scale 7 \
  --seed 42 --precision autocast --opt_inversion false

Select the corresponding directory to use another attribute. The frozen runtime is Python 3.10.22, torch 2.4.0+cu121 and NumPy 1.26.4. SD checkpoint SHA-256: fe4efff1e174c627256e44ec2991ba279b3816e364b49f9be2abc0b3ff3f8556.

Protocol and checkpoint reuse

200 updates; batch 4; Adam; learning rate 0.005 to 0.001 with cosine scheduling; hidden width 128; Fourier features 256; ten timestep-specific MLPs; seed 42; 512x512 RGB; DDIM 100 steps with strength 0.1 (10 effective steps); scale 7; autocast. Training and saved inference use inversion correction disabled.

DeRain completed on Colab T4. DeHaze resumed from verified Colab step 120; the remaining attributes started on Kaggle T4 x2. Two independent workers processed up to 40 additional updates per category between saved versions. Resume restored the full optimizer, scheduler, scaler and RNG states; the total schedule remained 200. Training completed on 2026-11-05 UTC.

For recovery use the exact source snapshot, runtime, backbone and image hashes recorded in experiment.json, with colab/train_category.py --resume and the full .pt checkpoint. All included states are already at step 200: resuming with the original total performs no further optimization. The checkpoint does not contain SD/CLIP weights or source images. Load these trusted PyTorch checkpoints with torch.load(..., map_location="cpu", weights_only=False); inference only needs the NumPy conditions and the original backbone/code.

Validation and limitations

All five checkpoint and condition hashes were checked on the host, including optimizer/scaler/RNG fields, run identity, 200 history entries, expected tensor shape and finite values. Exported conditions exactly match the checkpoint conditions. All final archives were checked against standalone files. The verification/ records provide SHA-256, sizes and training completion evidence. SHA256SUMS covers the files in this distribution.

One held-out output per attribute is an execution example, not a quantitative benchmark. PSNR/SSIM and broad generalization have not been evaluated for this reproduction; original-paper benchmark numbers are not claimed here.

This repository is initially private for research. No new redistribution license is asserted for the combined release. Upstream VDC, SD 1.4, CLIP and image-data terms and attribution must be reviewed before a public release. Generated examples and unchanged source documentation are retained for provenance.