OracleZoom

Shubhashis Roy Dipta*, Sourajit Saha*, Shaswati Saha, Nobin Sarwar · University of Maryland, Baltimore County

*Equal contribution.

Privileged-Latent Distillation for faithful extreme super-resolution. OracleZoom drives Chain-of-Zoom's recursive 4x super-resolution out to 256x while staying faithful, adding real detail instead of hallucinating. This repo is self-contained: the merged model, the inference code, and the required checkpoints are all here. You only download two public base models (Stable Diffusion 3-medium, Qwen2.5-VL-3B) automatically.

Quickstart (one image, all scales)

Needs one NVIDIA GPU (~16 GB) and Python 3.10.

# 0. One-time: Stable Diffusion 3 is gated, so accept its license on HF, then log in
pip install -U "huggingface_hub[cli]"
hf auth login

# 1. Download this repo (merged model + code + checkpoints)
hf download dipta007/OracleZoom --local-dir OracleZoom
cd OracleZoom

# 2. Install dependencies
pip install -r requirements.txt

# 3. Super-resolve ONE image (4x -> 16x -> 64x -> 256x)
python inference.py --input /path/to/photo.jpg --output ./outputs

Results in ./outputs/: photo_1x.png (the 512x512 input crop), photo_4x.png, photo_16x.png, photo_64x.png, photo_256x.png.

Stable Diffusion 3-medium and Qwen2.5-VL-3B download automatically on first run.

Batching many images: inference.py exposes zoom_image(sr, model, proc, pvi, image_path, out_dir). Build the models once (build_sr(...), build_vlm(...)) and call zoom_image in a loop over your images.

Training data

The curated training set is released separately at dipta007/OracleZoom-4KLSDB-train. The released model uses its 1k config (1,000 curated 4K images).

What's in this repo

Path	What it is
merged_transformer.safetensors	The OracleZoom super-resolution transformer (SD3 + Chain-of-Zoom's SR module + our distilled adapter, merged), fp32, ~8.35 GB.
inference.py	Self-contained runner: one image in, all scales out (recursive zoom + VLM prompting).
coz/	Vendored Chain-of-Zoom inference code (the one-step SR wrapper + helpers).
ckpt/	Chain-of-Zoom's SR-VAE and VLM-prompt (Qwen LoRA) checkpoints needed by the pipeline.
requirements.txt	Python dependencies.

Method

Recursive SR (Chain-of-Zoom) reuses a 4x backbone step after step to reach 16x-256x. Each step is blind: it sees only a blurred crop of its own previous output and must invent the missing detail, so errors compound and the invention may be hallucinated.

Privileged-latent distillation. A privileged teacher is shown the ground-truth high-resolution patch at training time only and distills its real detail into the blind student, in decode space. Only a small adapter is trained; the backbone, VAE, and prompter stay frozen. A KL leash to the deployed backbone keeps a deep sharpness reward from drifting into a metric-gaming texture, so detail stays faithful. Trained: rank-16 adapter (7.1M params), 1,000 curated 4K images; beta_reward 0.4, beta_kl 8.0. The released weights have this adapter already merged in.

Results

Under Chain-of-Zoom's exact protocol on a curated 4K benchmark and seven test sets:

Axis	Metric	Ours	CoZ / best baseline
Sharpness (no-reference)	CLIPIQA @256x	0.706	0.579 (CoZ)
Fidelity @4x (ground truth exists)	LPIPS	0.199	0.215 (CoZ)
Deep faithfulness (MLLM judge, 64-256x)	preferred vs CoZ	68-78%	-
Deep faithfulness	hallucination rate vs CoZ	2-5x lower	-

Sharpness is the axis prior methods are built for; the decisive gap is faithfulness, verified by full-reference metrics at 4x and by two cross-family MLLM judges plus a blinded human study past 4x.

Intended Use

• In-scope: research on faithful extreme (recursive) super-resolution of natural photographs.
• Out-of-scope: forensic/evidentiary use (detail past 4x is generated, not recovered); real-camera-zoom claims (the benchmark uses synthetic center-crop zoom).

Acknowledgements & Licensing

The coz/ code and the checkpoints in ckpt/ are from Chain-of-Zoom and are redistributed here for convenience; please respect their original license and cite them. The pipeline uses Stable Diffusion 3-medium and Qwen2.5-VL-3B under their respective licenses. OracleZoom's own contribution (the distilled adapter, merged into merged_transformer.safetensors) is released for research, non-commercial use (CC-BY-NC-4.0).

Citation

@inproceedings{dipta2026oraclezoom,
  title={OracleZoom: Privileged-Latent Distillation for Faithful Extreme Super-Resolution},
  author={Roy Dipta, Shubhashis and Saha, Sourajit and Saha, Shaswati and Sarwar, Nobin},
  year={2026},
  note={In submission, WACV 2026}
}

Please also cite Chain-of-Zoom and OSEDiff.