DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models (ECCV 2026)

Zhengming Yu^1,2, Li Ma², Mingming He², Leo Isikdogan³, Yuancheng Xu^2,3, Dmitriy Smirnov³, Pablo Salamanca^2,3, Dao Mi³, Pablo Delgado³, Ning Yu^2,3, Julien Philip², Xin Li¹, Wenping Wang¹, Paul Debevec^2,3
1Texas A&M University, ²Eyeline Labs, ³Netflix

📝 Abstract

Most digital videos are stored in 8-bit low dynamic range (LDR) formats, where much of the original high dynamic range (HDR) scene radiance is lost due to saturation and quantization. This loss of highlight and shadow detail precludes mapping accurate luminance to HDR displays and limits meaningful re-exposure in post-production workflows. Although techniques have been proposed to convert LDR images to HDR through dynamic range expansion, they struggle to restore realistic detail in over- and underexposed regions. To address this, we present DiffHDR, a framework that formulates LDR-to-HDR conversion as a generative radiance inpainting task in the latent space of a video diffusion model. By operating in Log-Gamma color space, DiffHDR leverages spatio-temporal generative priors from a pretrained video diffusion model to synthesize plausible HDR radiance in over- and underexposed regions while recovering the continuous scene radiance. Our framework further enables controllable LDR-to-HDR video conversion guided by text prompts or reference images. To address the scarcity of paired HDR video data, we develop a pipeline that synthesizes high-quality HDR video training data from static HDRI maps. Extensive experiments demonstrate that DiffHDR significantly outperforms state-of-the-art approaches in radiance fidelity and temporal stability, producing realistic HDR videos with considerable latitude for re-exposure.

🛠️ Setup

conda create -n diffhdr python=3.10 -y
conda activate diffhdr

# Install PyTorch (CUDA 11.8)
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 \
    --index-url https://download.pytorch.org/whl/cu118

# Install DiffHDR
cd /path/to/DiffHDR_Code
pip install -e .
pip install -r requirements.txt

Base model

Download Wan2.1-VACE-14B (~75 GB). This single repo contains everything needed -- the 7 DiT shards, the T5 text encoder, the VAE, and the umt5-xxl tokenizer:

hf download Wan-AI/Wan2.1-VACE-14B --local-dir models/Wan-AI/Wan2.1-VACE-14B

Expected layout:

models/Wan-AI/Wan2.1-VACE-14B/
├── diffusion_pytorch_model-0000{1..7}-of-00007.safetensors
├── models_t5_umt5-xxl-enc-bf16.pth
├── Wan2.1_VAE.pth
└── google/umt5-xxl/

Set the MODEL_BASE environment variable to point the scripts at a different root instead of models/.

LoRA checkpoints

mkdir -p models
hf download ZhengmingYu/DiffHDR --local-dir models

This fetches both LoRA weights (58 MB each) into models/:

File	Use with
DiffHDR.safetensors	infer_video.py, infer_image.py, infer_long_video.py
DiffHDR_Pano.safetensors	infer_hdri.py (360 panoramas)

Optional: Flash Attention

Not required -- all inference paths fall back to PyTorch SDPA. Install it only if you want the speedup, and note that it compiles CUDA kernels from source (needs nvcc, takes tens of minutes):

pip install psutil ninja packaging wheel   # flash_attn's setup.py needs these
pip install flash_attn --no-build-isolation

🎥 Inference

Our paper results were produced with the default --num_inference_steps 50. In practice we found that 10 steps gives comparable quality on many cases, so the video demo commands below pass --num_inference_steps 10 to keep them fast. Drop that flag to reproduce the paper setting.

Video (MP4 or image folder)

# From MP4 file
python infer_video.py \
    --lora_path models/DiffHDR.safetensors \
    --input_path demo/room_window.mp4 \
    --output_dir results/video_mp4 \
    --prompt "" \
    --num_inference_steps 10 \
    --add_mask --use_under_exposure_mask --crop_and_resize --srgb_to_lg

Text-conditioned inference

Provide a descriptive prompt to guide HDR reconstruction:

python infer_video.py \
    --lora_path models/DiffHDR.safetensors \
    --input_path demo/wooden_house \
    --output_dir results/text_cond \
    --num_inference_steps 10 \
    --prompt "over-exposed: A bright ocean landscape visible through the skylight window, with a wide blue sea stretching to the horizon and soft clouds in the sky. Sunlight shines through the window and softly illuminates the wooden attic interior while keeping the indoor scene unchanged." \
    --seed 33 \
    --add_mask --crop_and_resize --srgb_to_lg

Image-conditioned inference

Provide a reference image to guide the style and tone of the HDR output:

python infer_video.py \
    --lora_path models/DiffHDR.safetensors \
    --input_path demo/wooden_house \
    --output_dir results/image_cond \
    --reference_image_path demo/ref_gemini_city.jpg \
    --prompt "" \
    --num_inference_steps 10 \
    --add_mask --crop_and_resize --srgb_to_lg

Single Image

python infer_image.py \
    --lora_path models/DiffHDR.safetensors \
    --input_path demo/sample_image.png \
    --output_dir results/image_output \
    --prompt ""

Long Video (sliding window)

For videos with more than 33 frames:

python infer_long_video.py \
    --lora_path models/DiffHDR.safetensors \
    --input_path demo/long_video_frames \
    --output_dir results/long_video_output \
    --prompt "" \
    --window_size 33 --window_stride 16 \
    --use_prev_window_reference \
    --add_mask --crop_and_resize --srgb_to_lg

How it works:

• Processes the video in overlapping windows of window_size frames
• Stride of window_stride frames between windows (overlap = window_size - window_stride)
• Linear temporal blending in overlap regions for smooth transitions
• --use_prev_window_reference: passes a reference frame from the previous window for temporal consistency

HDRI Panorama

For single LDR panorama images (We extend this work to HDRI):

python infer_hdri.py \
    --lora_path models/DiffHDR_Pano.safetensors \
    --input_path demo/sample_pano.png \
    --output_dir results/hdri_output

This uses overexposure mask detection (luma + channel clipping) and outputs a single HDR EXR panorama at 1024x2048 by default.

📊 Eval

Evaluate generated HDR EXR frames using eval/cal_sample.py:

# NR metrics only (MUSIQ, CLIPIQA, PU21-PIQE):
python eval/cal_sample.py \
    --gen_dir results/video_mp4 \
    --out_csv results/video_mp4_eval.csv

# With ground truth (adds FovVideoVDP):
python eval/cal_sample.py \
    --gen_dir results/video_mp4 \
    --gt_dir /path/to/gt_exr_frames \
    --out_csv results/video_mp4_eval.csv

# With DOVER video quality metric:
python eval/cal_sample.py \
    --gen_dir results/video_mp4 \
    --out_csv results/video_mp4_eval.csv \
    --dover_repo /path/to/DOVER

Metric	Type	Description
MUSIQ	NR	No-reference image quality (tonemapped)
CLIPIQA	NR	CLIP-based image quality (tonemapped)
PU21-PIQE	NR	Perceptual quality on PU21-encoded HDR luminance
FovVideoVDP	FR	Full-reference HDR visual difference (JOD)
DOVER	NR	No-reference video quality (tonemapped MP4)
FID	FR	Distribution distance on tonemapped patches

For HDR-VDP-3, we follow LEDiff to use the Matlab scripts, please refer the run_hdrvdp3_dir.m for the configuration details.

🏋️ Training

# Launch LoRA training
bash scripts/train.sh

The training script uses HuggingFace Accelerate for distributed training.

Training data format: EXR frames organized by the metadata CSV, with sRGB LDR and linear HDR pairs.

📚 Citation

@article{yu2026diffhdr,
  title={DiffHDR: Re-Exposing LDR Videos with Video Diffusion Models},
  author={Yu, Zhengming and Ma, Li and He, Mingming and Isikdogan, Leo and Xu, Yuancheng and Smirnov, Dmitriy and Salamanca, Pablo and Mi, Dao and Delgado, Pablo and Yu, Ning and others},
  journal={arXiv preprint arXiv:2604.06161},
  year={2026}
}

🙏 Acknowledgements

Our work is built upon many awesome prior works:

• DiffSynth-Studio -- the diffsynth/ package in this repository is a reduced, modified fork of it.
• Wan2.1-VACE-14B -- the base video diffusion model that our LoRA is trained on top of.

We thank these authors for their great works and open-source contribution.

📄 License

This project is released under the licence in LICENSE.

It bundles third-party code: diffsynth/ is derived from DiffSynth-Studio, licensed under Apache-2.0. Files in that directory have been modified from the originals. The upstream copyright and licence terms continue to apply to them.