dronefreak/clearview-derain-resnet50-unet

ClearView: Image Deraining with ResNet50-UNet (Mixed-Domain)

A U-Net [8] with a ResNet-50 [9] encoder (ImageNet-pretrained, fine-tuned end to end), the largest of ClearView's three ResNet-backbone variants, trained on a blended synthetic + real-world rain set, selecting checkpoints against a blended real-world validation metric rather than a single benchmark, for a model that holds up across domains instead of maxing out one dataset's quirks. Not a mirror of someone else's checkpoint, see Citation.

---

Quickstart

Requires the ClearView library:

pip install git+https://github.com/dronefreak/clearview.git

from clearview.api import DerainingModel
from huggingface_hub import hf_hub_download

weights = hf_hub_download(repo_id="dronefreak/clearview-derain-resnet50-unet", filename="clearview-derain-resnet50-unet.pth")
model = DerainingModel.from_pretrained("resnet50_unet", weights=weights)

clean = model.process("rainy_image.png", output_path="derained.png")

---

Training Data

5 sources combined via ClearView's --mix-config (recipe), oversampling the real-world sources 2x:

Source	Type	Weight	Pairs
Rain13K [1]	Synthetic	1.0	13,711
DDN-Data / Rain1400 [4]	Synthetic	1.0	12,600
SPA-Data [5]	Real-world	2.0	6,385
RealRain-1k-H [6]	Real-world	2.0	784
RealRain-1k-L [6]	Real-world	2.0	784

~62% synthetic / ~38% real by effective sampling weight (before oversampling: ~77%/23% by raw pair count).

Checkpoint selection uses a separate blended validation set (recipe): SPA-Data val (capped to 150 of 1,000 pairs so it can't dominate), RealRain-1k-H/L validation (112 each), and Rain100L (100) as a synthetic sanity anchor.

---

Detailed Test-Set Metrics

Full per-dataset breakdown across all 6 tracked metrics, computed on each source's own held-out test/eval split (not the blended validation set used for checkpoint selection during training).

Test Set	Domain	PSNR	SSIM	MAE	MSE	Rain Removal Rate	NIQE
Rain100L [2]	Synthetic	29.79	0.906	0.0218	0.00128	0.045	10.66
Rain100H [2]	Synthetic	25.37	0.794	0.0398	0.00350	0.684	12.46
Test100 [3]	Synthetic	26.16	0.839	0.0408	0.00340	0.417	10.12
Test1200 [3]	Synthetic	28.44	0.856	0.0306	0.00221	0.318	7.77
Test2800 [4]	Synthetic	28.67	0.883	0.0269	0.00156	0.251	790.05
DDN-Data [4]	Synthetic	28.72	0.886	0.0268	0.00156	0.226	1033.27
SPA-Data [5]	Real-world	37.07	0.973	0.0099	0.00040	-0.476	6.45
RealRain-1k-H [6]	Real-world	34.94	0.970	0.0135	0.00066	0.742	4.67
RealRain-1k-L [6]	Real-world	36.52	0.978	0.0106	0.00042	0.671	4.80
AllWeather (rain+fog) [7]	Cross-domain (stress)	13.61	0.555	0.1875	0.05468	0.103	217.52

One negative Rain Removal Rate value here (SPA-Data: -0.476) is worth calling out explicitly: PSNR/SSIM are still decent there, but the Sobel-residual metric says this checkpoint added slightly more high-frequency error than it removed on that set specifically, likely mild over-sharpening/artifacting from the ResNet encoder path rather than a training failure. Unlike ResNet18/34-UNet, Rain100L's Rain Removal Rate here is actually positive (0.045), the smallest positive value in the whole ClearView lineup, but positive nonetheless. See Use Cases below.

Metric definitions:

• PSNR (Peak Signal-to-Noise Ratio, dB): pixel-level fidelity between the restored output and ground truth. Higher is better. The standard image-restoration metric, but insensitive to structural/perceptual quality on its own.
• SSIM (Structural Similarity Index, 0-1): perceptual similarity based on luminance, contrast, and structure. Higher is better, tracks human-perceived quality more closely than PSNR.
• MAE (Mean Absolute Error, normalized [0,1] pixel space): average per-pixel absolute intensity difference. Lower is better, less sensitive to outlier pixels than MSE.
• MSE (Mean Squared Error, normalized [0,1] pixel space): average per-pixel squared intensity difference, the term PSNR is derived from. Lower is better, penalizes large errors more heavily than MAE.
• Rain Removal Rate: ClearView-specific metric. Compares the Sobel-gradient ("high-frequency") residual energy of the output vs. ground truth against that of the input vs. ground truth: 1 - (residual after) / (residual before). 1.0 means perfect rain removal, 0.0 means no change, negative means the model added more high-frequency error than it removed (e.g. hallucinated detail or over-sharpening). Higher is better.
• NIQE (Natural Image Quality Evaluator): no-reference perceptual quality score. Its pristine reference statistics are refit per test set from that set's own clean images, so NIQE values are only comparable within the same row, not across rows, a NIQE of 800 on one dataset and 10 on another does not mean one output is 80x worse, the reference scale itself differs per dataset. Lower is better within a given row.

---

ClearView Model Comparison

How this model compares to the rest of the ClearView model zoo (PSNR / SSIM), same test sets and evaluation protocol for every model. Columns sorted highest to lowest by average PSNR across the 9 rain-only test sets.

Test Set	Domain	Restormer	NAFNet (Large)	NAFNet (Mid)	Restormer-Small	UNet (Vanilla)	NAFNet (Small)	ResNet50-UNet	ResNet34-UNet	ResNet18-UNet	Histoformer
Rain100L [2]	Synthetic	36.04 / 0.969	34.59 / 0.961	34.14 / 0.957	31.57 / 0.933	30.96 / 0.932	30.20 / 0.922	29.79 / 0.906	29.00 / 0.894	28.87 / 0.893	25.83 / 0.836
Rain100H [2]	Synthetic	27.78 / 0.868	27.65 / 0.856	27.72 / 0.849	25.92 / 0.806	26.41 / 0.823	25.02 / 0.763	25.37 / 0.794	24.67 / 0.774	24.95 / 0.776	12.22 / 0.364
Test100 [3]	Synthetic	28.65 / 0.881	27.71 / 0.865	27.96 / 0.873	26.10 / 0.842	24.91 / 0.836	25.26 / 0.820	26.16 / 0.839	26.38 / 0.839	25.99 / 0.833	22.01 / 0.684
Test1200 [3]	Synthetic	31.91 / 0.906	31.37 / 0.898	31.28 / 0.898	30.66 / 0.882	29.08 / 0.868	30.43 / 0.874	28.44 / 0.856	28.19 / 0.852	28.03 / 0.845	24.20 / 0.727
Test2800 [4]	Synthetic	32.05 / 0.928	31.75 / 0.924	31.66 / 0.923	31.00 / 0.914	30.61 / 0.909	30.58 / 0.906	28.67 / 0.883	28.31 / 0.875	28.34 / 0.875	24.71 / 0.785
DDN-Data [4]	Synthetic	32.21 / 0.931	31.90 / 0.927	31.84 / 0.926	31.28 / 0.918	30.67 / 0.912	30.83 / 0.910	28.72 / 0.886	28.47 / 0.879	28.42 / 0.878	25.04 / 0.784
SPA-Data [5]	Real-world	44.67 / 0.989	41.99 / 0.986	41.77 / 0.986	38.46 / 0.978	39.01 / 0.980	37.13 / 0.973	37.07 / 0.973	36.97 / 0.971	36.55 / 0.970	32.18 / 0.929
RealRain-1k-H [6]	Real-world	40.28 / 0.985	39.34 / 0.982	38.68 / 0.980	35.29 / 0.969	35.98 / 0.971	34.33 / 0.957	34.94 / 0.970	35.21 / 0.969	34.64 / 0.966	21.86 / 0.761
RealRain-1k-L [6]	Real-world	42.35 / 0.989	41.17 / 0.987	40.64 / 0.986	37.56 / 0.978	38.04 / 0.980	36.59 / 0.970	36.52 / 0.978	36.88 / 0.977	36.25 / 0.975	25.47 / 0.867
AllWeather (rain+fog) [7]	Cross-domain (stress)	13.72 / 0.584	13.53 / 0.576	13.64 / 0.579	13.59 / 0.572	13.66 / 0.570	13.59 / 0.574	13.61 / 0.555	13.54 / 0.558	13.66 / 0.558	30.75 / 0.923

All ClearView models trained under the identical mixed-domain recipe, only batch size/accumulation steps vary per architecture size. Histoformer is included as an external, inference-only reference point (original authors' checkpoint, not trained under this recipe), its strong AllWeather (rain+fog) score and comparatively weak scores on the rain-only test sets reflect that it was trained on a rain+fog+snow mix, while ClearView's current mix is rain-only. ResNet50-UNet is the largest of the three ClearView ResNet-backbone variants at 73.3M params, but that extra capacity doesn't translate into a clean win, it edges out ResNet34/18-UNet only slightly (30.63 vs. 30.45 vs. 30.23 avg PSNR across the 9 rain-only sets) and still trails NAFNet (Small) at 1.1M params. Likely a ResNet-stem effect (the stock ResNet stem's aggressive early downsampling, a stride-2 7x7 conv followed by a stride-2 max-pool, can blur or alias away the thin rain streaks it's supposed to detect before any residual block runs) shared across all three ResNet variants regardless of depth, rather than a capacity issue, still a working hypothesis rather than a confirmed cause.

---

Use Cases

Good for: dashcam/surveillance footage, photo restoration, real-world rain (not just synthetic-style streaks); a reference point for how a plain ImageNet-pretrained ResNet encoder compares against ClearView's purpose-built architectures on this task, at the largest capacity point in that comparison.

Limitations: no temporal consistency for video (processes frames independently); AllWeather rain+fog is an explicit out-of-scope stress test, fog was not in the training mix and this model is not expected to handle it; trails every other ClearView model on rain-only test sets despite being the largest by params (73.3M); negative Rain Removal Rate on SPA-Data, PSNR/SSIM are still reasonable there but the residual high-frequency error metric suggests mild over-sharpening/artifacting rather than clean streak removal on that set; largest and slowest of the three ResNet-backbone variants, needs a GPU with meaningfully more VRAM than ResNet18/34-UNet for a modest quality gain.

---

Training

clearview-train --model resnet50_unet --mix-config configs/mix/rain_mixed_synthetic_real.yaml --mix-sampler \
  --val-mix-config configs/mix/rain_mixed_val.yaml --loss custom --loss-config '{"charbonnier": {"weight": 1.0}}' \
  --crop-size 256 --batch-size 32 --epochs 100 --mixed-precision --ema --compile \
  --output-dir ./runs/rain_mixed_resnet50_unet

---

Training Curves

---

Citation

@software{saksena2025clearview,
  author = {Saksena, Saumya Kumaar},
  title = {ClearView: Practical Image Deraining},
  year = {2025},
  url = {https://github.com/dronefreak/clearview}
}

Architecture and datasets this model builds on:

References

1. Fu et al. and others, Rain13K composite (Rain800/Rain100H/L/Rain14000/Rain12), standard MPRNet/Restormer training set.
2. Yang et al., Deep Joint Rain Detection and Removal from a Single Image, CVPR 2017 (Rain100H/L).
3. Zhang & Patel / Zhang, Sindagi & Patel (Test100 / Test1200).
4. Fu et al., Removing Rain from Single Images via a Deep Detail Network, CVPR 2017 (DDN-Data / Rain1400 / Test2800).
5. Wang et al., Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset, CVPR 2019 (SPA-Data).
6. Li et al., RealRain-1k, arXiv:2206.05514, 2022.
7. Li et al., Heavy Rain Image Restoration, CVPR 2019 (AllWeather rain+fog / Outdoor-Rain).
8. Ronneberger, Fischer & Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation, MICCAI 2015, arXiv:1505.04597 (decoder topology).
9. He, Zhang, Ren & Sun, Deep Residual Learning for Image Recognition, CVPR 2016, arXiv:1512.03385 (encoder backbone).

Full bibtex for each: main repo README.

---

Model Card Authors / Contact

Saumya Kumaar Saksena, GitHub Issues