pipeline-forensics-nano

Three detectors, 46,769 parameters each, 187 KB each. What has this image been through?

• interlaced — interlaced field artefacts (comb teeth from displaced alternate rows)
• chroma_sub — luma resampled as if chroma-subsampled
• low_bitdepth — quantised to fewer than 8 bits

All three read the block-DCT of a 64×64 patch, not the pixels.

Measured on a second sensor

Trained on COCO, evaluated on real Logitech BRIO frames — a different sensor with its own image pipeline. The "transferred scalar" column is the honest engineering baseline: the best single threshold, fitted on COCO and applied unchanged, which is what a deployed threshold actually is.

detector	COCO	real frames	transferred scalar	margin
interlaced	0.968	1.000	0.504	+0.496
chroma_sub	0.999	0.980	0.521	+0.459
low_bitdepth	0.719	0.901	0.611	+0.290

Note two of the three score higher on real frames than on COCO. That is consistent with real frames coming from a single known 8-bit pipeline, while COCO JPEGs carry unknown prior processing that contaminates the negative class.

Why three models and not one

The obvious move is a single multi-head model. It was tried, with the prediction registered in advance that the three faults collide in high-frequency DCT structure and would therefore reinforce each other. The prediction failed. Every head degraded:

head	separate	joint
interlaced	+0.496	+0.263
chroma_sub	+0.463	+0.330
low_bitdepth	+0.289	+0.258

The reason is visible once stated: chroma subsampling is a low-pass. It erases the comb structure interlacing leaves and the quantisation steps bit-depth reduction leaves. One fault destroys the evidence of another.

That is a different thing from the overlap that helps a model. Elsewhere in this family — blur, sensor noise and hot pixels — faults confuse a single statistic while remaining separable to a network, and margins grow. Here they are mutually masking, and the model degrades along with the scalar. Ship these separately, and run chroma_sub first: if it fires, distrust the other two.

Scope

For: transcoding and archival pipelines, dataset hygiene, deciding whether content arrived as originally encoded.

Not for:

• Not manipulation or deepfake detection. These detect processing history. All three operations are routine and overwhelmingly innocent.
• Not evidence. A detection means processing happened, not that anything was concealed.
• Not a judgement about any person.

Known failure modes

1. low_bitdepth is the weak one — 0.719 on COCO. Quantisation is subtle at 6 bits and this is a binary detector over 4–6 bits.
2. Needs texture. Patches with standard deviation below ~6 carry no information; skip them.
3. Mutual masking, as above. A chroma-resampled image will hide interlacing from interlaced.
4. Operations the camera ISP already performs cannot be detected this way. Sharpening and denoising were screened out for exactly this reason — a webcam sharpens and denoises everything, so the negative class does not exist in real data. sharpened scored 0.375 on real frames against 0.333 chance.
5. Single 64×64 patch. Vote across patches for a frame-level answer.

Usage

import cv2, numpy as np, onnxruntime as ort

def dct_blocks(p):                       # p: float32 64x64 grayscale patch
    out = np.zeros_like(p)
    for by in range(0, 64, 8):
        for bx in range(0, 64, 8):
            out[by:by+8, bx:bx+8] = cv2.dct(p[by:by+8, bx:bx+8] - 128.0)
    return np.sign(out) * np.log1p(np.abs(out))

so = ort.SessionOptions()
so.intra_op_num_threads = 1
so.add_session_config_entry("session.intra_op.allow_spinning", "0")

img = cv2.imread("frame.png", cv2.IMREAD_GRAYSCALE)
p = img[y:y+64, x:x+64].astype(np.float32)        # native crop; do NOT resize
if p.std() < 6:
    raise SystemExit("flat patch, no information")
v = dct_blocks(p); v = (v - v.mean()) / (v.std() + 1e-8)

for name in ("chroma_sub", "interlaced", "low_bitdepth"):   # chroma first: it masks the others
    s = ort.InferenceSession(f"{name}.onnx", sess_options=so,
                             providers=["CPUExecutionProvider"])
    logit = s.run(None, {"dct": v[None, None]})[0][0][0]
    print(name, float(1 / (1 + np.exp(-logit))))

Take a native crop and do not resize — resizing is itself a resampling operation and destroys what these read.

Deployment note: cap the ONNX Runtime thread pool

ONNX Runtime sizes its intra-op pool to the core count and those workers spin-wait between inferences. Three sessions left this way will burn cores to serve 0.28 ms inferences. With intra_op_num_threads=1 and allow_spinning=0, measured idle CPU dropped from 192% to 16.5% of one core with throughput unchanged.

Training

~1,650 train / ~550 held-out COCO images, split by source image · block-DCT input, log-magnitude, per-patch standardised · 4 conv layers (16→32→48→64) · single sigmoid head · Adam 3e-3, 20 epochs.

Same architecture as resoajoe/blockgrid-nano, resoajoe/camera-motion-nano and resoajoe/alarm-nano, unchanged.

Verification

ONNX vs PyTorch, both CPU: max relative logit difference below 1e-6 for all three heads.

Prior art — processing-history forensics is an established field

Detecting resampling, requantisation and interlacing history is long-studied image forensics, with a mature literature on resampling detection (Popescu & Farid), JPEG requantisation and double- compression, and deinterlacing artefacts. These detectors are a small-scale re-implementation, not a new class of capability.

What is different here. Three small shipped detectors rather than analytical methods, plus three measurements the forensics literature does not usually report:

• Second-sensor evaluation rather than in-corpus only — and two of three score higher on real camera frames than on the training corpus.
• Transferred-threshold baselines instead of in-sample ones, which is what a deployed threshold actually is.
• A negative result on multi-head design: a joint model degrades every head, because chroma resampling low-passes away the evidence interlacing and bit-depth reduction leave. Hence three separate files and the instruction to run chroma_sub first.

Classical resampling and requantisation detectors remain the reference for accuracy and interpretability on full images. These are for cheap per-patch screening at the edge.

What "scalar baseline" means on this card

Every margin quoted here is against a stated baseline, because a margin without one is not a measurement. The baseline is the best single-threshold classifier over ten cheap statistics, fitted optimistically:

mean · std · lapvar · hf (high-frequency energy ratio) · grad (Sobel magnitude) · entropy · centre_edge · radial_slope · row_fft_peak · col_fft_peak

The last four are spatially aware, added after an earlier six-statistic baseline — all global aggregates — was found to systematically overstate model value on spatially structured tasks. A baseline that cannot see where anything is loses to a CNN by default. On one test task that flaw inflated an apparent margin from +0.060 to +0.261.

Two questions are asked with it, and they disagree:

• in-sample — threshold fitted on the data it is scored on. Deliberately generous. Answers is there structure beyond a low-order statistic?
• transferred — threshold fitted on the training corpus, applied unchanged to the target. Answers what should I ship? On one task the in-sample figure was 0.954 and the transferred figure 0.565.

Where this card quotes a single scalar figure without qualification, it is the in-sample one.

Provenance

COCO val2017 (public) plus frames from the author's own rig. No personal data.