Resfusion

Official model weights for Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise.

This repository hosts pretrained checkpoints intended for image restoration tasks, including:

• Shadow removal
• Low-light image enhancement
• Raindrop removal

If you use this model or repository in research, please cite the original paper.

---

Overview

Resfusion is a diffusion-based image restoration framework that incorporates prior residual noise patterns into the denoising diffusion process. Compared with standard diffusion-based restoration pipelines, Resfusion is designed to better exploit degradation-aware residual structure for higher-quality restoration.

This Hugging Face repository is intended to provide:

• Pretrained model weights
• Basic task descriptions
• Inference guidance
• Links to the original codebase and paper

---

Intended uses

This model is intended for research and experimental applications in:

• Image restoration benchmarking
• Shadow removal
• Low-light enhancement
• Raindrop removal
• Diffusion-based restoration studies

---

Training data

The original implementation supports the following datasets:

• ISTD for shadow removal
• LOL for low-light image enhancement
• Raindrop for raindrop removal

Please refer to the original project for exact data preparation, folder structure, and training configuration.

---

Usage

Option 1: Use with the official codebase

Please use the official Resfusion repository for inference and evaluation:

• Project repository: https://github.com/nkicsl/Resfusion
• arXiv link: https://arxiv.org/abs/2311.14900
• NeurIPS poster page: https://nips.cc/virtual/2024/poster/95696

Typical workflow:

1. Clone the official repository.
2. Create the environment from environment.yaml.
3. Download the checkpoint from this Hugging Face repository.
4. Place the checkpoint in your preferred checkpoint directory.
5. Run the corresponding test script for your task.

---

Citation

If you find this work useful for your research, please consider citing::

@inproceedings{NEURIPS2024_ebc62a3a,
 author = {Shi, Zhenning and zheng, haoshuai and Xu, Chen and Dong, Changsheng and Pan, Bin and xueshuo, Xie and He, Along and Li, Tao and Fu, Huazhu},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang},
 pages = {130664--130693},
 publisher = {Curran Associates, Inc.},
 title = {Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise},
 url = {https://proceedings.neurips.cc/paper_files/paper/2024/file/ebc62a3af9342eb4ebc728e5c5bc4cca-Paper-Conference.pdf},
 volume = {37},
 year = {2024}
}