Nieuwlaar/resnet50_7.9um_scroll1_frags-repaired

resnet50_7.9um_scroll1_frags (repaired)

Code is also mirrored on GitHub: https://github.com/Nieuwlaar/resnet50-scroll1-checkpoint-repair

A drop-in repaired model.safetensors for scrollprize/resnet50_7.9um_scroll1_frags. The released checkpoint contains a well-trained 3D ResNet-50 backbone but a decoder head whose weights were never trained, so the full model outputs ~0.5 everywhere and finds no ink. I kept the released backbone, retrained the head on the public Grand Prize ink labels, and lightly fine-tuned. On two held-out Scroll 1 segments the checkpoint goes from chance level to strong ink detection:

held-out segment	stock checkpoint	repaired checkpoint
20230827161847 (never used for training or model selection)	0.5022	0.9369
20231210121321 (used only to pick the best epoch)	0.5122	0.9431

(ROC-AUC against the Grand Prize ink labels, full-segment tiled inference with the model card's own settings. 0.5 = random guessing, 1.0 = perfect.)

Held-out segment 20231210121321: scan layer, GP ink labels, stock checkpoint output (constant ~0.5), repaired checkpoint output. This segment was used for epoch selection, so the cleaner number is the one below.

Held-out segment 20230827161847: scan layer, Grand Prize ink labels, stock checkpoint output (constant ~0.5), repaired checkpoint output.

Author: Erwin Nieuwlaar (GitHub: Nieuwlaar)

Verify the defect yourself in 60 seconds

CPU only, no scroll data, no torch. Just numpy and safetensors:

pip install numpy safetensors huggingface_hub
huggingface-cli download scrollprize/resnet50_7.9um_scroll1_frags model.safetensors --local-dir stock
python verify_checkpoint.py stock/model.safetensors

Expected output on the original checkpoint (this run took 1.2 s):

file:   stock/model.safetensors
size:   342,867,728 bytes
sha256: fa00728db74d714769960dd77e9911a0e0938b3f272933c153632b5bed543c34

decoder conv weights vs PyTorch default init U(-b, +b), b = 1/sqrt(fan_in)
tensor                        absmax/b  mean|w|/(b/2)  std/(b/sqrt(3))
decoder.convs.0.0.weight        1.0000         1.0004           1.0002
decoder.convs.1.0.weight        1.0000         1.0001           1.0000
decoder.convs.2.0.weight        1.0000         0.9999           0.9999
decoder.logit.weight            0.9835         0.9990           0.9898
backbone.conv1.weight          48.2277   (trained contrast)

BatchNorm num_batches_tracked (training batches seen):
  decoder.convs.0.1.num_batches_tracked        0
  decoder.convs.1.1.num_batches_tracked        0
  decoder.convs.2.1.num_batches_tracked        0
  backbone (53 layers)                        2,234,791

VERDICT: decoder head is at PyTorch default init (UNTRAINED).
Full-model ink predictions from this file will be chance-level.

The same script on this repository's model.safetensors reports the decoder as trained (decoder conv ratios 6.6–146x their init bound, decoder BatchNorm counters 32,536) and exits 0.

The defect in the original checkpoint

PyTorch gives an untrained conv layer random weights drawn uniformly from a fixed range that depends only on the layer's shape (±1/√fan_in). In the released file (HF revision f3975de34d4734dd507360304f828808b7182139), three separate checks all say the same thing:

1. Weight statistics. Every decoder conv sits exactly on its init bound, with mean and spread matching the untrained uniform distribution to 3–4 significant digits (table above). The backbone's first conv, by contrast, is ~48x its bound. That is what trained weights normally look like.
2. Batch-norm counters. All three decoder batch-norms read num_batches_tracked = 0 with pristine default statistics (mean 0, variance 1); every backbone batch-norm reads 2,234,791. A supporting oddity: config.json declares n_classes=1139, and the 1139-way classification head from pretraining is still present in the file as backbone.fc, trained (absmax ~14x its init bound) but unused by the ink pipeline. That fits a checkpoint exported from a pretraining setup, with the segmentation decoder that replaced that head left at default init.
3. It doesn't detect ink. On a Scroll 1 surface crop that is 26% labeled ink, the model card's own inference settings give output confined to ~0.50 ± 0.03 and ROC-AUC 0.504. That is chance level. The same crop through the same code scores 0.89 with the 2023 Grand Prize TimeSformer checkpoint, so the data pipeline and labels are sound; the failure is specific to this checkpoint's decoder.

This may be an intentional backbone-only export. But nothing on the model card says so, the card's "get started" link points at the ink-detection pipeline, and config.json loads the full model. A user who runs it gets silent chance-level output that looks like "no ink found".

Corpus audit: a second broken release, and the rest verify clean

I ran the same integrity check across the public scrollprize corpus (41 HF model repos), covering the main Scroll 1 ink-detection lineages: both ResNet-50 releases, all 14 current-generation ink_9um checkpoints, and the three Scroll 1 TimeSformer releases. 19 files, anonymous downloads, CPU only:

• resnet50_3um_01122024/model.safetensors is broken in exactly the same way: a different, independently trained backbone (batch-norm counters 2,231,011 vs 2,234,791; different weights) with its own never-trained decoder: all decoder convs on the untrained-init bound, decoder batch-norm counters 0. The unmodified verify_checkpoint.py detects it and exits 1.
• The other 17 files verify clean, including all 14 ink_9um checkpoints (head-group tensors 3.7–5.5x off their init bounds, the ink task head itself 6.1x or more, and the head weights move smoothly between released training steps, which is what an actively trained head looks like) and the TimeSformer-family files (ink_detection_pipeline is byte-identical to timesformer_GP_scroll1).

Full table, per-file statistics, method notes, and the list of repos not covered are in audit/CORPUS_AUDIT.md with machine-readable results in audit/results.csv; audit/audit_checkpoint.py generalizes the verifier to the other architectures.

Using the repaired checkpoint

The weight file has the same key layout as the original (backbone.* + decoder.*), so it is a drop-in replacement wherever the original is loaded: download the original repository for the model code and config, then point the weights at this file.

import sys
from huggingface_hub import snapshot_download, hf_hub_download

code_dir = snapshot_download("scrollprize/resnet50_7.9um_scroll1_frags")  # model code + config
weights  = hf_hub_download("Nieuwlaar/resnet50_7.9um_scroll1_frags-repaired",
                           "model.safetensors")

sys.path.insert(0, "scripts")            # scripts/ from this repository
from common import load_resnet_model
model, cfg = load_resnet_model(code_dir, ckpt_path=weights)   # torch nn.Module + config dict

Input contract (from the model card and the villa ink-detection code it links to): tiles of 256×256 px × 18 surface-volume layers (the center 18 layers as scripts/common.py selects them, global z 24–41 of the standard 65-layer segment stacks), uint8 intensities normalized as clip(x, 0, 200) / 255, sigmoid over the output logits, output at 1/4 resolution. scripts/eval_panel.py implements full-segment tiled inference (window 256, stride 128, Gaussian blending) end to end:

python scripts/prep_data.py --seg-dir data/20230827161847 --seg-id 20230827161847
python scripts/eval_panel.py --model resnet --model-dir <code_dir> \
    --ckpt model.safetensors --seg-dir data/20230827161847 \
    --out-prefix eval/20230827161847_after

Segment layers and *_mask.png come from the public data server (https://dl.ash2txt.org/full-scrolls/Scroll1/PHercParis4.volpkg/paths/); the *_inklabels files come from the ScrollPrize villa repository at deprecated/ink-detection/all_labels (until 2026-08-17 at ink-detection/all_labels). 20231022170901's label is a .tiff there, which prep_data.py handles.

Training recipe

I kept the released backbone and retrained the rest, using only public Scroll 1 data: surface volumes (layers 17–42) and Grand Prize ink labels for 15 segments: 48,813 training tiles of 256×256 px × 18 layers. The labels are the official public Grand Prize ink labels, taken bit-identical from the ScrollPrize villa repository (deprecated/ink-detection/all_labels; until 2026-08-17 at ink-detection/all_labels). Two segments were held out and never trained on: 20231210121321 and 20230827161847.

Training segments: 20231022170901, 20231106155351, 20231005123336, 20230820203112, 20230826170124, 20230702185753, 20230522215721, 20230531193658, 20230903193206, 20230902141231, 20231007101615, 20230929220926, 20231016151000, 20231012184423, 20231031143850

Two phases, fixed seed 20260803, Dice+BCE loss (label smoothing 0.25), AdamW (weight decay 1e-6), cosine schedule, gradient clipping 1.0, mixed precision, batch 12, and standard augmentation (flips, 90° rotations, brightness/contrast jitter, coarse dropout, ±2 layer jitter):

1. Decoder only: decoder re-initialized (seeded), backbone frozen with its batch-norm statistics kept intact, 5 epochs at lr 1e-3. This alone reaches tile AUC ≈ 0.926 on the validation segment, confirming the backbone features were good all along and only the head was missing.
2. Light full fine-tune: whole model, 4 epochs at lr 1e-5. Best checkpoint: epoch 3, validation tile AUC 0.9314.

phase	epoch	val tile AUC
decoder only	1 → 5	0.9165 → 0.9256
full fine-tune	1	0.9272
full fine-tune	3 (released)	0.9314
full fine-tune	4	0.9306

Nearly all of the gain (0.50 → 0.926 of the 0.931) comes from the repaired head. Training took about two and a half hours (roughly four hours wall-clock including data download and prep) on one rented consumer GPU (RTX 4090). Full history in training/metrics.json; exact configuration is the defaults of scripts/train_decoder.py:

# one-time setup: the original repo provides model code, config, and the stock weights
huggingface-cli download scrollprize/resnet50_7.9um_scroll1_frags --local-dir models/resnet50_7.9um_scroll1_frags

python scripts/train_decoder.py --data-root data --out out/job1 --always-full-ft

Repository contents

• model.safetensors: repaired weights, drop-in replacement (same key layout as the original checkpoint); sha256 6133ef74905d32a40d0e053d9378bf76207c29614cb77210218f1df898b8d3cf
• verify_checkpoint.py: the 60-second CPU diagnosis script
• scripts/: train_decoder.py, prep_data.py, eval_panel.py, common.py, the exact training and evaluation code, segment lists, and seed
• eval/: before/after panels and per-segment AUC JSONs for both held-out segments
• training/: metrics.json (full training history), best_info.json (selected checkpoint, seed, segment lists)
• audit/: corpus audit of the public scrollprize checkpoints, with CORPUS_AUDIT.md (findings), results.csv (per-file statistics), audit_checkpoint.py (the verifier generalized to the other architectures)

Relation to the newer ~9 µm models

The 2026-08 ink_9um release targets ~9 µm isotropic inputs. This repaired model does a different job: it restores a working detector for the ~7.9 µm surface-volume stacks that most existing Scroll 1 segments ship in. It is not the only detector for those stacks (the 2023 Grand Prize TimeSformer also runs natively on them; it is the control in defect Evidence 3 above), but it is the only released 3D-ResNet one, useful as an independent-architecture second opinion and as an initialization for distillation or fine-tuning work.

Limitations

• Trained and evaluated only on Scroll 1 (PHerc. Paris 4) surface volumes at the native ~7.9 µm, 54 keV scan. Not validated on other scrolls, resolutions, or scan energies.
• The AUC is measured against the Grand Prize ink labels, which are themselves imperfect; it is agreement with those labels, not with a human reading. These are the GP-era labels shipped with the official training code; the team's newer curated ink-labels dataset (2026-07) refines labels on updated segmentations and was not used here.
• 20231210121321 was used to choose the best epoch, and the stock model card lists it among the backbone's pretraining fragments. Both are reasons 20230827161847 is the cleaner held-out number.
• The light fine-tune means the backbone is no longer bit-identical to the released one; if the team publishes the originally intended trained decoder, that should supersede this repair.

License

The scripts and the retrained weights in this repository are released under the MIT license. The backbone derives from the scrollprize/resnet50_7.9um_scroll1_frags release; see that repository for its terms. Scroll data and ink labels come from the Vesuvius Challenge data server (https://dl.ash2txt.org/LICENSE.txt).