uniuyuni/nagi_denoise

Nagi Denoise — NagiV2-L

A blind denoiser for real high-ISO photographs. Float32 linear-light RGB in, float32 linear-light RGB out. HDR-safe (values above 1.0 are preserved, never clipped), seam-free at any resolution including 40MP+, and fully deterministic — same bytes in, same bytes out.

• Architecture: NagiV2-L, 15.43M parameters (width 64, encoder (2,2,4), middle 6, decoder (1,2,2)).
• Code: https://github.com/uniuyuni/nagi_denoise (Apache-2.0)
• Weights: this repo (CC BY-NC 4.0 — see Licence)

Files in this repo

file	size	what it is
nagi_v2_l_ft2_final.pt	236 MB	the production PyTorch checkpoint
nagi_v2_l_ft2_t768_fp16.mlpackage/	30 MB	Core ML export, 768px tile, fp16 — the fast path
nagi_v2_l_ft2_t768_fp32.mlpackage/	59 MB	Core ML export, 768px tile, fp32

The .mlpackage assets are directories, stored here as folders of files.

Usage

import numpy as np
from nagi_denoise import denoise

# img: (H, W, 3) float32 numpy array, linear-light RGB. Values > 1.0 are fine.
out = denoise(img)                       # PyTorch reference path
out = denoise(img, backend="coreml")     # Core ML fast path on Apple Silicon

The weights are resolved locally first and downloaded from this repo only if they are not already present: an explicit weights= path wins, then $NAGI_DENOISE_WEIGHTS, then the in-repo runs/ copy, then an existing Hugging Face cache, and only then a download. allow_download=False (or NAGI_DENOISE_OFFLINE=1) forbids the network entirely.

To fetch deliberately, ahead of time:

from nagi_denoise.assets import resolve_weights, resolve_coreml_package
resolve_weights()             # -> Path to nagi_v2_l_ft2_final.pt
resolve_coreml_package()      # -> Path to the fp16 .mlpackage

⚠️ Core ML: never use compute_units="all"

ALL lets Core ML dispatch to the Apple Neural Engine, and the ANE computes this fp16 graph wrongly. On every one of the 23 validation tiles, peak output values run 1.25x–4.8x higher than the PyTorch reference — worst absolute per-pixel error 8.7 on a scene whose true peak is 5.3. Visually this is blown-out garbage, not a subtle numerical difference.

The same graph on cpu_and_gpu agrees with PyTorch to 0.018 max, which is just fp16 rounding. cpu_and_gpu is the default in this project and it must stay that way.

Training data

source	weight	licence
SIDD (Smartphone Image Denoising Dataset)	0.25	MIT
PolyU Real-World Noisy Images Dataset	0.35	non-commercial
synthetic Poisson-Gaussian degradations, generated by this project	0.40	this project's code (Apache-2.0)

The synthetic noise is spatially correlated, not white. Real demosaiced sensor noise is correlated; training on white noise alone teaches the model that "isolated pixel deviation = noise", so it preserves real correlated noise blobs as if they were fine structure. Measured lag-1 autocorrelation of the residual after a 3x3 median:

source	lag-1 autocorrelation
Fujifilm X-T5 (real)	+0.235
Pentax K-5 (real)	+0.139
PolyU (real)	+0.27 … +0.48
SIDD (real, but near-white)	+0.024
naive white synthetic noise	−0.146

The synthetic noise field is therefore drawn as a Gaussian-blurred, unit-std normal field, with the blur sigma sampled uniformly per image over [0.0, 1.0] pixels (chroma field: [0.8, 2.0]). The sweep used to calibrate it — sigma 0.5 / 0.7 / 0.9 → lag-1 0.005 / 0.217 / 0.365 — puts sigma ≈ 0.7 on the real X-T5 figure and sigma ≈ 0.9 in the PolyU range.

Measured results

benchmark	result
SIDD Validation (sRGB PSNR)	39.030 dB
— NAFNet-w64 teacher, for reference	40.21 dB, at 116M parameters (7.5x the size)

Structure-vs-noise selectivity, X-T5 Occi hair ROI — how much more high-frequency energy survives on structure than on flat noise:

pipeline	selectivity	retention on structure
NagiV2 (this model)	+11.2 pt	90.5%
legacy in-house v12 pipeline	+9.8 pt	82.1%

More detail retained and cleaner — not a trade.

HDR highlight retention ≥ 0.99 on every scene with true HDR content.

Speed, 39.8MP frame, end to end, on an M1 with 16GB:

path	time
Core ML fp16 / cpu_and_gpu	~83 s
pure PyTorch / MPS	~261 s

Known limitations

Stated plainly, because they are real:

• Isolated small specular highlights on low-dynamic-range files get treated as impulse noise. On Z7 fix / Z7 bird, top-1% luma retention is 0.62–0.92. The conditional highlight guard does not help here by design: it only arms when the image genuinely contains above-SDR content, because arming it on low-range scenes blends noisy input back in over large areas (measured 3.7x more flat-region noise) and reads as uneven denoising.
• The confidence-gated detail head is inert on this checkpoint. The gate is closed, so detail_strength is a no-op by design. It is wired up so it activates automatically if a future checkpoint opens the gate.
• It removes noise; it does not generate texture. This is a restoration model, not a generative one. It will not match the synthesised detail of DxO PhotoLab DeepPrime XD, and it is not trying to.

Licence

The weights in this repo are CC BY-NC 4.0 — non-commercial. The source code at https://github.com/uniuyuni/nagi_denoise is Apache-2.0. Full text: https://creativecommons.org/licenses/by-nc/4.0/legalcode

The weights are not more restrictive by choice. They are a derived work of their training data, and one of the sources carries a non-commercial restriction that flows through to anything trained on it:

PolyU Real-World Noisy Images Dataset Copyright (c) 2018, The Hong Kong Polytechnic University "Any redistribution, use, or modification is done solely for non-commercial purposes."

PolyU supplied 35% of the training mixture, and it was not an incidental ingredient: it was the only source of real spatially-correlated camera noise, which is exactly what made the model work on real photographs at all. The other sources are permissive — SIDD is MIT, and the synthetic degradations are generated by this project's own code.

You may: use the weights for any non-commercial purpose; redistribute them, modified or not; fine-tune, distil, quantise or convert them and redistribute the result. You must: give attribution, state that the weights are CC BY-NC 4.0, and indicate whether you modified them. Derivative weights inherit these terms.

If you need commercially usable weights

Retrain without PolyU. The code to do so is Apache-2.0 and complete: drop polyu from data.mixture in a training config, redistribute its weight between sidd and synthetic, and train per the repo's README. Expect a quality cost — removing the only real correlated-noise source is a material change and the result needs re-validating against the project's gates. Weights you produce that way are yours, and are not covered by this licence.

No warranty

The weights are provided "as is", without warranty of any kind, express or implied. See the CC BY-NC 4.0 text for the full disclaimer.

Citation

The datasets and reference models this work depends on:

@inproceedings{abdelhamed2018sidd,
  title     = {A High-Quality Denoising Dataset for Smartphone Cameras},
  author    = {Abdelhamed, Abdelrahman and Lin, Stephen and Brown, Michael S.},
  booktitle = {CVPR},
  year      = {2018}
}

@article{xu2018polyu,
  title   = {Real-world Noisy Image Denoising: A New Benchmark},
  author  = {Xu, Jun and Li, Hui and Liang, Zhetong and Zhang, David and Zhang, Lei},
  year    = {2018},
  note    = {PolyU Real-World Noisy Images Dataset. Non-commercial use only.}
}

@article{chen2022nafnet,
  title   = {Simple Baselines for Image Restoration},
  author  = {Chen, Liangyu and Chu, Xiaojie and Zhang, Xiangyu and Sun, Jian},
  journal = {arXiv:2204.04676},
  year    = {2022}
}

@article{zhang2022scunet,
  title   = {Practical Blind Image Denoising via Swin-Conv-UNet and Data Synthesis},
  author  = {Zhang, Kai and Li, Yawei and Liang, Jingyun and Cao, Jiezhang and
             Zhang, Yulun and Tang, Hao and Timofte, Radu and Van Gool, Luc},
  year    = {2022}
}

NAFNet (MIT) is vendored in the repo as the distillation teacher and benchmark reference; SCUNet (Apache-2.0) as a benchmark reference. Full third-party notices are in NOTICE in the source repo.