StereoDiffGen

Single-image stereo generation with a two-view diffusion model. Given one (left) image and a stereo baseline in metres, StereoDiffGen generates the matching right view directly. It uses no depth estimate, no warping and no inpainting.

Anonymous release accompanying a CVPR 2027 submission.

Model

• Denoiser: two-view latent diffusion U-Net (SD-2.1 layout). Self-attention runs jointly over the tokens of both views, so the right view is generated in correspondence with the left.
• Camera conditioning:
  • Each view's latent is concatenated with a Plücker ray map and a mask that marks the given view.
  • A 16-d rig-pose vector derived from the baseline is added to the timestep embedding.
  • A CLIP image token of the input is used as cross-attention context.
• Stereo-aware decoder: the SD-2 VAE decoder with a light LoRA and a cross-view attention block, so both views decode consistently.
• Prediction: v-prediction; DPM-Solver++(2M), 25 steps uniform in log-SNR; cosine classifier-free guidance 5 → 1.

Frozen inference configuration

Resolution	768 × 768
Steps / sampler	25, DPM-Solver++(2M)
Guidance	cosine CFG 5 → 1
Field of view	fixed 54°
Precision	bf16 autocast, fp32 weights
Seed	image i of a benchmark uses seed i

Files

File	Size	SHA-256
stereodiffgen.safetensors	3.8 GB	40d36117e7ab9f5ef27818a94e4cf2ff37e0c703befe7ed6792ae6ced04847e5

The file holds inference weights only (unet.*, vae.* including vae.mixer.*, and schedule.*). The decoder LoRA is stored unmerged so outputs are bit-identical to the paper's pipeline. The CLIP ViT-H/14 image encoder (laion2b_s32b_b79k) is fetched from open_clip on first use.

Usage

from stereodiffgen.pipeline import StereoDiffGen, read_rgb
from PIL import Image

model = StereoDiffGen()                                                    # loads weights/stereodiffgen.safetensors
right = model.generate(read_rgb("left.png"), baseline_m=0.065, seed=0)    # uint8 768x768x3
Image.fromarray(right).save("right.png")

Download the weights:

wget https://huggingface.co/cvprsubmission/stereodiffgen/resolve/main/stereodiffgen.safetensors -P weights/

The inference code includes:

• a bit-exact reproduction check;
• generation scripts for all nine benchmarks;
• the common-protocol evaluation.

A link to the code will be added after review.

Results (common protocol, Tab. 1 of the paper)

FoundationStereo measures disparity on the generated pair in a shared frame (GT aspect ratio, short side 768). Against a pseudo-GT from the real pair (stereo benchmarks) or metric GT depth (depth benchmarks), it reports δ1, AbsRel and SAWA-H. LPIPS (vs. GT right), MEt3R and iSQoE are computed on the native pair. ↑ higher is better, ↓ lower is better.

Benchmark	δ1 ↑	AbsRel ↓	SAWA-H ↓	LPIPS ↓	MEt3R ↓	iSQoE ↓
Booster	0.9730	0.0389	0.8078	0.3011	0.0913	0.6306
LayeredFlow	0.8765	0.0767	1.0965	0.4136	0.1187	0.6486
Middlebury	0.9031	0.0835	0.8908	0.3253	0.0980	0.6679
Spring	0.7891	0.1174	1.3949	0.1412	0.0952	0.7838
KITTI	0.9663	0.0533	1.3284	–	0.0725	0.7208
Sintel	0.7077	0.2213	1.5943	–	0.1487	0.8210
ETH3D	0.9261	0.0841	1.1191	–	0.0669	0.5504
TUM	0.9190	0.0836	1.6404	–	0.0873	0.7803
NYUv2	0.9689	0.0552	1.4524	–	0.0944	0.7376

Stereo benchmarks are generated at each benchmark's median ground-truth baseline; depth benchmarks at 10 cm.

Limitations

• The field of view is fixed at 54° at test time. Images with very different intrinsics get a mismatched disparity scale.
• Output is 768 × 768.

License

CreativeML Open RAIL++-M, inherited from Stable Diffusion 2.1, on which the denoiser and VAE are based. Its use-based restrictions apply to this model and any derivatives.

Citation

@inproceedings{stereodiffgen2027,
  title     = {StereoDiffGen},
  author    = {Anonymous},
  booktitle = {Under review},
  year      = {2027}
}