Ours-Probe raw — LLaVA-1.5-7B
This release contains the image-conditioned (h_image) Ours-Probe trained on
COCO captions. It replaces the earlier contrastive (h_image - h_no_image)
release. The three checkpoints are independent; do not ensemble them.
Configuration
| Setting | Value |
|---|---|
| Backbone | llava-hf/llava-1.5-7b-hf |
| Probe input | Raw image-conditioned hidden states, no subtraction |
| Decoder layers used by the probe | 16,20,24,28 (zero-based) |
| Cache layers | 0,4,8,12,16,20,24,28,31 |
| Head | One MLP per selected layer; mean of sigmoid probabilities |
| MLP | 4096 → 256 → 256 → 1; GELU and LayerNorm after each hidden layer |
| Training seeds | 0,42,1337 |
| Optimizer | AdamW, learning rate 3e-4, weight decay 0.05 |
| Batch / maximum epochs | 256 / 20 |
| Loss | Positive-class-weighted BCE with 0.01 label smoothing |
| Checkpoint selection | Highest validation within-object AUROC |
The COCO corpus has 8,000 training-pool images and a separate 2,000-image
holdout. Of the training pool, 7,979 images yield 52,756 object mentions. An
image-disjoint 20% of those usable images is reserved for validation. The
holdout cache contains 13,396 object mentions. Captions use the prompt
Please describe this image in detail.
| Seed | Selected epoch | Validation-best-F1 threshold |
|---|---|---|
| 0 | 20 | 0.7389142513275146 |
| 42 | 17 | 0.7845199108123779 |
| 1337 | 20 | 0.8478354811668396 |
These thresholds are for detection. The caption-mitigation threshold must be selected separately from the COCO-validation truthfulness–recall sweep.
COCO holdout detection
All three seeds were scored on the same 13,396 mentions. Mean ± sample SD, in percent:
| AUROC | AUPRC | Accuracy | F1 |
|---|---|---|---|
| 96.11 ± 0.02 | 84.75 ± 0.13 | 91.23 ± 0.12 | 77.56 ± 0.25 |
Files
checkpoints/: the three raw, independent-mean checkpoints.cache/coco/train/: image-conditioned features for the COCO training pool.cache/coco/holdout/: image-conditioned features for the untouched holdout.predictions/coco_holdout/: one holdout prediction file per seed.metadata/release_manifest.json: exact source hashes, split counts, and checkpoint selection details.
The cache stores more decoder layers than the four used by this probe so that
future layer studies can reuse the same complete raw cache. The original
holdout cache manifest mistakenly said mode: train; this release corrects
that metadata to mode: holdout without changing the feature payload.
AMBER and POPE caches are not included. AMBER caches predating the seed-42 image split must be rebuilt.
Training and scoring code: hallucination.