gliner-probe

Fine-tune of urchade/gliner_large-v2.1 for data-use mention extraction (dataset / survey / census / registry mentions in economics research papers).

Labels

• NAMED_DATA — a proper name, title, or acronym of a specific data source
• DESCRIPTIVE_DATA — a source described in words but not named
• VAGUE_DATA — generic data wording with no identifiable source

Training

• base model: urchade/gliner_large-v2.1
• dataset: rafmacalaba/usage-sensitivity-probe (gliner config)
• corpus: all
• epochs: 3
• learning rate: 5e-06
• batch size: 16
• precision: bf16

Evaluation (holdout)

thr	tp	fp	fn	precision	recall	f0.5	f1
0.10	2153	3550	36	0.3775	0.9836	0.4306	0.5456
0.20	2132	2728	57	0.4387	0.9740	0.4929	0.6049
0.30	2086	2171	103	0.4900	0.9529	0.5427	0.6472
0.40	2009	1699	180	0.5418	0.9178	0.5902	0.6814
0.50	1862	1179	327	0.6123	0.8506	0.6486	0.7120
0.60	1544	695	645	0.6896	0.7053	0.6927	0.6974
0.70	1019	299	1170	0.7731	0.4655	0.6829	0.5811

Best F0.5: 0.6927 (thr=0.6) Best F1: 0.7120 (thr=0.5)

Evaluation breakdown (holdout)

group	examples	spans	thr	precision	recall	f0.5	f1
overall	2901	2195	0.60	0.6896	0.7053	0.6927	0.6974
prwp	1217	1160	0.60	0.7274	0.6765	0.7166	0.7010
fcv	1684	1035	0.60	0.6546	0.7377	0.6697	0.6937
general_prwp	1217	1160	0.60	0.7274	0.6765	0.7166	0.7010
fcv_pads_east_africa	1133	710	0.60	0.6606	0.7147	0.6707	0.6866
jdc_operational	47	20	0.70	0.5625	0.4500	0.5357	0.5000
refugee_pads	164	67	0.70	0.7209	0.4627	0.6485	0.5636
reliefweb	340	238	0.70	0.7459	0.5798	0.7055	0.6525

Per-label (overall)

label	examples	spans	thr	precision	recall	f0.5	f1
NAMED_DATA	2901	533	0.70	0.7395	0.4643	0.6611	0.5704
DESCRIPTIVE_DATA	2901	1118	0.60	0.6908	0.4874	0.6376	0.5716
VAGUE_DATA	2901	544	0.70	0.5598	0.5938	0.5663	0.5763

Origin breakdown (per-origin metrics)

origin	examples	spans	thr	precision	recall	f0.5	f1
fcv_pads_east_africa	1133	710	0.60	0.6606	0.7147	0.6707	0.6866
general_prwp	1217	1160	0.60	0.7274	0.6765	0.7166	0.7010
jdc_operational	47	20	0.70	0.5625	0.4500	0.5357	0.5000
refugee_pads	164	67	0.70	0.7209	0.4627	0.6485	0.5636
reliefweb	340	238	0.70	0.7459	0.5798	0.7055	0.6525

Per-label details

label	examples	spans	thr	precision	recall	f0.5	f1
NAMED_DATA	2901	533	0.70	0.7395	0.4643	0.6611	0.5704
DESCRIPTIVE_DATA	2901	1118	0.60	0.6908	0.4874	0.6376	0.5716
VAGUE_DATA	2901	544	0.70	0.5598	0.5938	0.5663	0.5763