datause-tier-clf

Fine-tune of answerdotai/ModernBERT-large classifying data-use mention spans (marked >>> mention <<< in context) into evidence tiers, v3.

Trained on 62k luna-judged spans (rafmacalaba/data-use-evidence-classifier-data; val+holdout dual-pass

• train-top single-pass verdicts), decontaminated against the 573 human-adjudicated gold controls used for evaluation below.

Labels

• tier1_evidential — data used as warrant/analytical consumption
• tier2_declaration — named/consulted without analytical use
• tier3_nonmention — promissory/logframe/bibliographic/caveat/concept
• junk — malformed extraction

Input format

prev[<=220 chars] >>> mention <<< next[<=280 chars]

Training

• base model: answerdotai/ModernBERT-large
• epochs: 3, lr: 2e-05, batch: 32 x1
• max_length: 512, precision: bf16

Benchmark (573 gold controls, human tiers)

• 4-class accuracy: 0.5079, macro-F1: 0.5662
• per-class F1: tier1_evidential=0.545, tier2_declaration=0.413, tier3_nonmention=0.580, junk=0.727
• binary T1-vs-rest @0.5: P=0.5641 R=0.5033 F1=0.5320
• best threshold 0.20 (F1=0.5743)

Silver consensus (n=3,999 clean dual-pass, held out of training)

• 4-class accuracy: 0.9002, macro-F1: 0.8355