siddhant20/task1

CyberAI Cup 2026 — Task 1: Packaging Material Difference Mining

Complete working state of the solution: trained weights, all source code, documentation, and the raw evidence behind every reported number. Private repository — this is a working handover, not a release.

Current result

Fold-0 out-of-fold global F1 0.9049 (precision 0.9225, recall 0.8881, score threshold 0.31), measured on 40 held-out validation pairs with the competition's own metric: TP/FP/FN accumulated across all images before computing precision and recall, boxes matched at IoU ≥ 0.5.

For scale, the classical image-difference baseline scores 0.029 on the same split.

Folds 1–4 are untrained and no submission has been generated. HANDOVER.md section 5 holds the prioritized queue for whoever continues.

What is in here

Path	Contents
checkpoints/stage1_fold0_convnext_tiny/best.pt	The model — 128 MB, epoch 19, the 0.9049 checkpoint
checkpoints/stage1_fold0_convnext_tiny/last.pt	384 MB, epoch 35, with optimizer and scheduler state for resuming
checkpoints/oof/	Out-of-fold candidate boxes and the threshold sweep
checkpoints/stage2_ab/	Verifier from the A/B test — reference only, its measured delta was +0.0009
src/pmdm/	14 modules, ~1500 lines: preprocessing, model, losses, decoding, training, inference, metric
modal_app.py	Every Modal function and entrypoint used to produce these results
scripts/	CPU smoke test, local training without Modal, dataset analysis, error analysis
runs/	Raw training logs, per-epoch history, and analysis output from every run
Task1/	The competition dataset: 200 training pairs with 1407 annotated boxes, 100 test pairs. Uploading separately — 2.5 GB of page-scale PNGs takes hours on a home uplink, so it lands after the code and weights (scripts/push_dataset_to_hf.py)
HANDOVER.md	Start here — architecture, results, error breakdown, mistakes, next steps
RUNBOOK.md	Every command with measured timings
IMPLEMENTATION_PLAN.md	Original design rationale

Architecture

A siamese CenterNet. Both images pass through a shared ConvNeXt-tiny encoder taking 4 channels (BGR plus a local-background-normalized "ink map" that makes the features invariant to the photo's shadows). Features fuse per scale as conv(concat(a, b, |a − b|)), decode through a U-Net upsampling path, and emit four heads at output stride 2 — a Gaussian focal heatmap for difference centers, width/height, a sub-pixel offset, and an auxiliary change-mask segmentation.

Stride 2 is the load-bearing choice: 83 of the ground-truth boxes are 8×8 pixels, which vanish entirely at the stride 4 a standard CenterNet uses.

Three things about the data drove the rest of the design:

1. Pairs are already sub-pixel aligned. Of 300 pairs, 284 need no registration at all and 16 need a pure translation. No warping is applied by default.
2. Blur, not geometry, is the dominant false-positive source. The photo is softer than the template, so the template is blur-matched to it with a σ grid search minimizing the Laplacian-variance gap before any comparison.
3. Nothing is ever deleted. Of 1404 analysed boxes, 842 are additions and 558 are modifications of existing content — and exactly 0 are deletions. That licenses a polarity filter discarding any candidate that is inked in the template and blank in the photo.

Inference tiles each page at 768 px with 25% overlap, then merges candidates with Weighted Boxes Fusion rather than NMS — with boxes this small, averaging the coordinates beats picking one.

Loading the model

import torch
from pmdm.model import SiamCenterNet

model = SiamCenterNet(backbone="convnext_tiny")
ck = torch.load("checkpoints/stage1_fold0_convnext_tiny/best.pt", map_location="cpu")
model.load_state_dict(ck["model"])   # all keys match
model.eval()

ck["metric"] carries the evaluation that selected this checkpoint, including the 0.31 threshold. Full tiled inference over a page is pmdm.infer; scoring is pmdm.metric.global_f1.

Reproducing

uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python -e . torch torchvision timm \
    opencv-python-headless "numpy<2" pandas

# CPU pre-flight over the whole pipeline, ~2 min
PMDM_DATA=Task1/PackagingMaterialDifferenceMiningDataset \
PMDM_WORK=/tmp/pmdm_work PMDM_CKPT=/tmp/pmdm_ckpt \
.venv/bin/python scripts/local_smoke.py

# every dataset claim above
PMDM_DATA=Task1/PackagingMaterialDifferenceMiningDataset \
.venv/bin/python scripts/analyze_dataset.py

# the error breakdown, from the archived out-of-fold predictions
.venv/bin/python scripts/error_analysis.py checkpoints/oof/fold0_convnext_tiny.npz

Training runs either on Modal (modal run --detach modal_app.py::train --fold 0 --epochs 40, ~45 min on an A100-40GB) or on any GPU box with no Modal account (python scripts/train_local.py --fold 0 --epochs 40). Same loop, same checkpoints.

Where the remaining errors are

Of 268 ground-truth boxes across the validation fold: 252 were proposed and matched, 5 were proposed but localized too loosely to clear IoU 0.5, and 11 were never proposed at all — a recall ceiling of 0.959 for the current proposal stage.

By box size, the picture is lopsided: boxes above 24 px match 97.5% of the time, 12–24 px match 97.9%, and boxes under 12 px match 82.8%. Ten of the eleven complete misses are sub-12px marks. Everything else is solved. That is why the next step is synthetic data weighted toward tiny marks (src/pmdm/synth.py, written and smoke-tested but never run at scale) rather than more box refinement or a stronger verifier.

License

The dataset in Task1/ belongs to the CyberAI Cup 2026 organizers and is included here only so this private repository is self-contained for the team. Do not redistribute it.