upscale-factor-nano

46,899 parameters. 187 KB. Was this image upscaled, and by how much?

Given a 64×64 patch, predicts whether it was resampled up from a smaller original by 2×, 3×, or 4× — the signature of a "4K" stream that is really 720p wearing a bigger container.

Measured

Trained on real photographs (COCO val2017) with real resampling. Split by source image.

task	nano (held out)	best scalar (in-sample)	chance
upscale factor — 2× / 3× / 4×	0.980	0.489	0.344
upscaled at all, yes/no	0.977	0.847	0.514

The factor task is the one worth having. Energy loss alone cannot answer "by how much" — every tested scalar (mean, std, Laplacian variance, HF ratio, gradient, entropy) lands between 0.357 and 0.489, because the answer lives in the periodicity interpolation leaves behind, not in how much detail is missing. The model beats the best optimistically-fitted scalar by +0.491.

A prediction made before running this was wrong and is recorded here for honesty: the binary task was expected to fall to a scalar, since upscaling removes high frequencies much as blur does. It did not — 0.977 vs 0.847. Upscaling leaves structure a single statistic cannot capture even for the coarse question.

Scope

For: deciding whether to spend decode budget on a stream that has no real detail, media triage, dataset hygiene (finding upscaled images in a training corpus), and quality auditing on cheap hardware.

Not for:

• Not a deepfake or AI-generation detector. It detects resampling, nothing else.
• Not a general image-quality score. An upscaled image is not necessarily a bad one.
• Not provenance or authenticity. Resampling happens for many innocent reasons.
• Not a substitute for reading the container metadata when that is available and trustworthy.

Usage

import cv2, numpy as np, onnxruntime as ort
FACTORS = [2, 3, 4]
sess = ort.InferenceSession("upscale_factor.onnx", providers=["CPUExecutionProvider"])

img = cv2.imread("frame.png", cv2.IMREAD_GRAYSCALE)
p = img[y:y+64, x:x+64].astype(np.float32)        # a NATIVE 64x64 crop -- do not resize first
p = (p - p.mean()) / (p.std() + 1e-8)
print(FACTORS[int(sess.run(None, {"input": p[None,None]})[0][0].argmax())])

Take a native crop; never resize the image to 64×64 first. Resizing is itself a resampling operation and destroys the very signature the model reads. This is the single easiest way to get meaningless output.

Patches are cheap — run several across the frame and take the majority vote.

Known failure modes

1. Assumes the image WAS upscaled. The three-class head always answers 2×, 3×, or 4×. Gate it with the binary head or your own check first.
2. Trained on five interpolation kernels (nearest, linear, cubic, Lanczos4, area), randomised per example. A kernel outside that set — a neural upscaler, for instance — is untested and likely fails.
3. Grayscale, 64×64, single crop. Chroma upsampling artefacts are not used.
4. Heavy compression after upscaling will attenuate the periodic signature. Untested on strongly re-compressed material.
5. Flat regions carry no information. Sky, walls and blur have nothing to resample. Sample textured crops.

Training

• 2,600 COCO images, native 64×64 crops, split by source image (75/25)
• Downscale by the factor with INTER_AREA, restore with a randomly chosen kernel
• 4 conv layers (16→32→48→64), BatchNorm, global average pool
• Adam 3e-3, 22 epochs, batch 64

Verification

ONNX vs PyTorch, both CPU, 256 inputs: max relative logit difference 2.8e-07, 100% argmax agreement.

Provenance

COCO val2017, a public dataset. No personal data involved.