ToiTenBao/haloprobe-ours-coco-llava15-13b

HaLoProbe — LLaVA-1.5-13B, COCO

Object-hallucination probe trained on a LLaVA-1.5-13B COCO corpus, with three independent training seeds (0, 42, 1337). It is not interchangeable with the 7B release: features, geometry and thresholds are model-specific.

Base model	llava-hf/llava-1.5-13b-hf
Decoder layers / hidden / heads	40 / 5120 / 40
Vision patch grid	24 x 24
Ours probe layers	10,15,20,25,30
TruthPrInt layer	20
Seeds	0, 42, 1337 (independent; do not ensemble)

Corpus

10,000 COCO train2014 images split 80/20 by image: 8,000 train / 54,253 object spans and 2,000 holdout / 13,789 object spans. Captions are generated by 13B itself, so this corpus is not the 7B corpus.

Results (mean +/- SD over the three seeds)

Thresholds are selected by best F1 on the COCO validation split and then frozen. Nothing is tuned on the test holdout, AMBER, or POPE.

COCO test holdout — mention view, n=13,787, 16.19% positive

AUROC	AUPRC	ACC	F1
0.9315 +/- 0.0019	0.7381 +/- 0.0076	0.8935 +/- 0.0017	0.6929 +/- 0.0092

AMBER, strict zero-shot — n=2,367 rows / 977 images, 5.37% positive, COCO thresholds frozen

AUROC	AUPRC	ACC	F1
0.8526 +/- 0.0126	0.2829 +/- 0.0121	0.8527 +/- 0.0277	0.3290 +/- 0.0357

AMBER's object universe is the human-labelled (image, noun) set; nouns 13B mentions that carry no AMBER label are not scored (2,367 of 3,350 candidate pairs matched).

POPE-Adversarial, zero-shot — n=3,000, 18.9% positive

AUROC	AUPRC
0.6012 +/- 0.0434	0.2389 +/- 0.0313

Validation best-F1 threshold, seeds 0 / 42 / 1337: carried inside each checkpoint (selected on its own training-validation split)

Layout

checkpoints/              one per seed
cache/coco/{train,holdout}/   features over the 13B corpus
cache/amber/seed*/        zero-shot AMBER features
cache/pope_adversarial/   zero-shot POPE-Adversarial features
metadata/model_profile.json   model id, layers, width, heads, vision grid
metadata/thresholds.json      validation-selected thresholds per seed
metadata/manifest.json        corpus provenance and cache source

Before using these artifacts, compare metadata/model_profile.json with model_workflow.sh show-profile; a mismatch means the cache must be rebuilt.