Redstonexs/danbooru-tagger-round3-artifacts

Kagami-24k — training artifacts

English · 中文

Working artifacts for the round that produced Redstonexs/kagami-24k. This repo is the audit trail, not the release — if you want the model, go there.

Derived from wd-eva02-large-tagger-v3 (Apache-2.0).

Result

11,639 held-out posts with id > 8,600,750, scored on the intersection of all seven candidate vocabularies (7,779 tags; 3,101 with ≥10 positives).

model	macro-AP	fine AP
Kagami-24k	0.5086	0.6170
danbooru-tagger-v1	0.4672	0.5643
wd-eva02-large-v3	0.4668	0.5979

Paired bootstrap over images, 200 resamples, vs wd-eva02: macro-AP +0.0418 [+0.0393, +0.0448], fine AP +0.0190 [+0.0060, +0.0288]. Both lower bounds clear of zero. Against wd-vit-large the fine-AP interval grazes zero ([−0.0001, +0.0349]) and that comparison is not established.

What is here

• ck_last.pt — full state at step 196,000: weights, AdamW state, EMA shadow. The shipped model is the raw weights, materialised from this with scripts/92_materialize.py. EMA scored +0.0001 macro-AP / −0.0002 fine AP against raw on dev — noise — so the simpler artifact shipped.
• irfs/ — repeat factors, corpus size (677,355), mean repeat (2.303). Rebuilding these is a full corpus scan.
• scripts/ — teacher precompute, its three validation gates, the 2×2 ablation, teacher selection and verification, training, dev eval, test bootstrap.
• logs/ — every run log: precompute, all gates, training, both evaluations.
• reports/ — gate outputs and analyses as JSON, plus the 2×2 write-up.

The 19.7 GB precomputed teacher is not here. It is deterministic given 80_teacher_precompute.py and the corpus, and it passed three gates (reports/spotcheck.json, dev_reconstruct.json, and a manifest validation over all 261 shards) — regenerating is cheaper than hosting.

Method, briefly

The previous release fine-tuned WD's encoder on ground truth alone, won the F1 metrics and lost fine-grained AP by 0.0336. A 2×2 ablation (reports/dev_2x2_decomposition.md) recombined the two encoders with the two heads and found the drift is not attributable to either component: the mixed cells collapse to 0.1743 and 0.0613 macro-AP against 0.4230/0.4080 for the matched pairs, and an additive fit predicts the fourth cell at −0.1873 — an impossible value. So "freeze the early blocks" had no evidence behind it and was dropped.

Instead, a frozen teacher

teacher = 0.35 · previous + 0.65 · mean(wd_eva02, wd_vit_large, wd_swinv2)

was precomputed over all 677,355 corpus images on the 7,800 general tags shared with WD's vocabulary. The 0.35 weight came from out-of-fold cross-fitting on a held-out dev split (buckets 0900–0913); per-tag and frequency-banded weightings were tried and rejected (reports/teacher_choice.json). Training used every block under layer-wise LR decay 0.92, ASL on ground truth plus a teacher term decaying 1.0 → 0.3, EMA 0.9998, 196,000 steps at batch 12.

The test set was opened once, after the checkpoint and the raw-vs-EMA choice were both fixed on dev.

Reproducing

scripts/ runs in order: 80 precompute → 82/83/84 gates → 90 train → 91 dev eval → 92 materialise → 93 test bootstrap. Paths assume a corpus laid out as data/corpus/shards/*.tar with matching data/teacher/*.npz.