lfm2.5-350M-datause

LoRA SFT of LiquidAI/LFM2.5-350M for data-mention extraction: emit the data-bearing phrases in a text as compact JSON (data_mentions with data_mention/specificity_type).

Training

• base model: LiquidAI/LFM2.5-350M
• dataset: rafmacalaba/data-use-mention-sft
• epochs: 3
• learning rate: 0.0002
• LoRA: r=16 alpha=32 dropout=0.05
• completion-only masking (loss on assistant JSON turn)

Evaluation (holdout, n=12531)

Jaccard entity-level matching: acronym-aware span clustering + Hungarian optimal bipartite match (match thr=0.5), F0.5 primary. Aligned with rafmacalaba/gliner_datause_extended. The holdout is rafmacalaba/data-use-mention-sft, so numbers are not directly comparable to the GLiNER model's data-use-mentions-extended holdout.

label	tp	fp	fn	precision	recall	f0.5	f1
overall	13176	3097	2998	0.8097	0.8146	0.8107	0.8122

Per-label

label	tp	fp	fn	precision	recall	f0.5	f1
descriptive	5522	2631	2078	0.6773	0.7266	0.6866	0.7011
named	5344	1538	1629	0.7765	0.7664	0.7745	0.7714
vague	670	604	934	0.5259	0.4177	0.5000	0.4656

Per-corpus

label	tp	fp	fn	precision	recall	f0.5	f1
fcv	3142	619	730	0.8354	0.8115	0.8305	0.8233
prwp	10034	2478	2268	0.8020	0.8156	0.8047	0.8087

Per-origin

label	tp	fp	fn	precision	recall	f0.5	f1
fcv_pads_east_asia	728	96	110	0.8835	0.8687	0.8805	0.8761
general_prwp	10034	2478	2268	0.8020	0.8156	0.8047	0.8087
jdc_operational	149	25	24	0.8563	0.8613	0.8573	0.8588
refugee_pads	704	147	152	0.8273	0.8224	0.8263	0.8248
reliefweb	1561	351	444	0.8164	0.7786	0.8086	0.7970

Sample predictions (holdout)

gold	predicted
{"data_mentions":[{"data_mention":"administrative data","specificity_type":"descriptive"}]}	{"data_mentions":[{"data_mention":"administrative data","specificity_type":"descriptive"}]}
{"data_mentions":[{"data_mention":"World Development Indicators","specificity_type":"named"},{"data_mention":"CPHS","spe	{"data_mentions":[{"data_mention":"World Development Indicators","specificity_type":"named"},{"data_mention":"CPHS","spe
{"data_mentions":[{"data_mention":"ES data","specificity_type":"named"}]}	{"data_mentions":[{"data_mention":"ES data","specificity_type":"named"}]}
{"data_mentions":[]}	{"data_mentions":[{"data_mention":"EUROMOD I4.0+","specificity_type":"named"}]}
{"data_mentions":[{"data_mention":"World Development Indicators","specificity_type":"named"},{"data_mention":"IMF World	{"data_mentions":[{"data_mention":"World Development Indicators","specificity_type":"named"},{"data_mention":"IMF World
{"data_mentions":[]}	{"data_mentions":[]}
{"data_mentions":[]}	{"data_mentions":[]}
{"data_mentions":[{"data_mention":"UNESCO-provided data","specificity_type":"descriptive"}]}	{"data_mentions":[{"data_mention":"PIRLS","specificity_type":"named"},{"data_mention":"UNESCO-provided data","specificit
{"data_mentions":[{"data_mention":"IHPS dataset","specificity_type":"named"},{"data_mention":"IHS4 data","specificity_ty	{"data_mentions":[{"data_mention":"IHS4","specificity_type":"named"},{"data_mention":"IHPS dataset","specificity_type":"
{"data_mentions":[{"data_mention":"HFSSS","specificity_type":"named"}]}	{"data_mentions":[{"data_mention":"HFSSS","specificity_type":"named"}]}
{"data_mentions":[{"data_mention":"HFCS data","specificity_type":"named"}]}	{"data_mentions":[{"data_mention":"HFCS data","specificity_type":"named"}]}
{"data_mentions":[{"data_mention":"data on patents and publications","specificity_type":"descriptive"}]}	{"data_mentions":[{"data_mention":"data on patents and publications","specificity_type":"descriptive"}]}