kohils/dlp-nppe3-denoise-sr

Joint Denoising + x4 Super-Resolution (DLP 26T2 NPPE3)

Takes a noisy low-light 160x256 RGB image and produces the 640x1024 luminance image. Trained directly for the competition's PSNR metric.

Results

	holdout (130 unseen train scenes)	public LB
bicubic upsample	31.46	31.75
NLM(h=6) + bicubic	32.71	33.93
NLM(h=10) + bicubic	33.69	35.80
this model	37.41	39.88

Reference point: an identical model trained on noise-free input reaches 40.68 on the same holdout. Essentially all remaining error is denoising, not super-resolution.

The task, as actually measured

The evaluation script converts predictions with PIL.Image.convert('L') and takes .flatten()[::8]. The image is 640x1024 and 1024 % 8 == 0, so that selects every 8th column, all rows — 81,920 values per image. Chroma is never scored, so the model predicts one luma channel rather than RGB, which puts the loss directly on the scored quantity and spends no capacity on colour.

Degradation model, recovered from the data

The provided low-resolution images are reproduced to within 0.25% residual MSE and 0.023 grey levels of mean by:

LR = clip(floor( g * Poisson(area_down4(GT)/g) + N(0, rv) ), 0, 255)
g  = 2.492     # gain, measured against GT: 2.4924 (train) / 2.4914 (val)
rv = 3.10      # read-noise variance

Three details each mattered:

• the downsampling kernel is area (4x4 box mean), not bicubic — lowest residual MSE and lowest correlation between residual and local gradient;
• quantisation is truncation (.astype(np.uint8)), not rounding — rounding leaves a +0.5 grey-level DC offset against the real data;
• the noise is pure shot noise: spatially white (lag-1 autocorrelation 0.003) and channel-independent (0.005), with variance linear in signal.

Because the degradation is known exactly, training pairs are synthesised on the GPU every step from clean ground truth, giving unlimited fresh noise realisations rather than 1,242 fixed ones.

Architecture

A NAFNet-style U-Net at low resolution with a x4 pixel-shuffle head, 4 downsampling levels, ~25.7M parameters. Two structural choices:

• a fixed bicubic-upsampled-luma skip with a zero-initialised final conv, so training begins exactly at the 31.46 dB bicubic solution and only has to learn the residual;
• generalized Anscombe transform input channels alongside the raw RGB: 2*sqrt(y/g + 3/8 + rv/g^2) has ~unit variance at every brightness, turning heteroscedastic Poisson noise homoscedastic. Measured: raw noise std runs 8.12 -> 16.17 across brightness bins, GAT-space 1.006 -> 1.016.

Training

• 2x NVIDIA T4, DDP, fp16 AMP
• AdamW, lr 2e-4, cosine schedule, 1000-step warmup, EMA 0.9995
• 64x64 LR patches, batch 32 per GPU
• Charbonnier loss, switching to MSE at 85% of progress (MSE is the direct PSNR surrogate; L1-type losses optimise the conditional median instead)
• 1,242 scenes (train + val), 80% GPU-synthesised / 20% provided pairs
• validation on 130 held-out train scenes, scored with the exact competition metric (PIL luma, every 8th column)

Ablations at matched wall-clock, delta vs control:

change	delta
Anscombe VST input	+0.044
4th downsampling level	+0.027 (and 40% faster per step)
drop real-pair mix	-0.007
128px patches	-0.014
MSE switch at 30% instead of 85%	-0.076

Inference

import torch, numpy as np
from PIL import Image
import models                       # from src/

sd  = torch.load("final_deep_vst.pt", map_location="cpu")
net = models.build_model(sd["model_name"], vst=sd.get("vst", False))
net.load_state_dict(sd["ema"])      # EMA weights, not the raw ones
net.eval().cuda()

lr = np.array(Image.open("test_00001.png").convert("RGB"))     # 160x256x3 uint8
x  = torch.from_numpy(lr.transpose(2, 0, 1))[None].float().cuda() / 255.
with torch.no_grad():
    y = net(x)                                                  # [1,1,640,1024]
out = np.clip(np.rint(y[0, 0].cpu().numpy() * 255), 0, 255).astype(np.uint8)

x8 geometric self-ensemble adds ~0.05 dB — average the members in float and round once at the end (src/infer.py: predict_float, blend).

Files

• final_deep_vst.pt — checkpoint; load the ema key for inference
• src/ — full training and inference code
• results.json — raw run metrics