gliner-probe-smoke
Fine-tune of urchade/gliner_large-v2.1 for data-use mention extraction
(dataset / survey / census / registry mentions in economics research papers).
Labels
NAMED_DATA — a proper name, title, or acronym of a specific data sourceDESCRIPTIVE_DATA — a source described in words but not namedVAGUE_DATA — generic data wording with no identifiable source
Training
- base model:
urchade/gliner_large-v2.1
- dataset:
rafmacalaba/usage-sensitivity-probe (gliner config)
- corpus:
all
- epochs: 1
- learning rate: 5e-06
- batch size: 16
- precision: bf16
Evaluation (holdout)
| thr |
tp |
fp |
fn |
precision |
recall |
f0.5 |
f1 |
| 0.10 |
36 |
865 |
4 |
0.0400 |
0.9000 |
0.0494 |
0.0765 |
| 0.20 |
33 |
445 |
7 |
0.0690 |
0.8250 |
0.0845 |
0.1274 |
| 0.30 |
29 |
243 |
11 |
0.1066 |
0.7250 |
0.1285 |
0.1859 |
| 0.40 |
22 |
142 |
18 |
0.1341 |
0.5500 |
0.1580 |
0.2157 |
| 0.50 |
16 |
75 |
24 |
0.1758 |
0.4000 |
0.1980 |
0.2443 |
| 0.60 |
6 |
35 |
34 |
0.1463 |
0.1500 |
0.1471 |
0.1481 |
| 0.70 |
3 |
18 |
37 |
0.1429 |
0.0750 |
0.1210 |
0.0984 |
Best F0.5: 0.1980 (thr=0.5)
Best F1: 0.2443 (thr=0.5)
Evaluation breakdown (holdout)
| group |
examples |
spans |
thr |
precision |
recall |
f0.5 |
f1 |
| overall |
64 |
40 |
0.50 |
0.1758 |
0.4000 |
0.1980 |
0.2443 |
| prwp |
14 |
12 |
0.50 |
0.1212 |
0.3333 |
0.1389 |
0.1778 |
| fcv |
50 |
28 |
0.50 |
0.2069 |
0.4286 |
0.2308 |
0.2791 |
| general_prwp |
14 |
12 |
0.50 |
0.1212 |
0.3333 |
0.1389 |
0.1778 |
| fcv_pads_east_africa |
27 |
11 |
0.50 |
0.1481 |
0.3636 |
0.1681 |
0.2105 |
| jdc_operational |
6 |
5 |
0.70 |
1.0000 |
0.2000 |
0.5556 |
0.3333 |
| refugee_pads |
5 |
3 |
0.60 |
1.0000 |
0.3333 |
0.7143 |
0.5000 |
| reliefweb |
12 |
9 |
0.50 |
0.3077 |
0.4444 |
0.3279 |
0.3636 |
Per-label (overall)
| label |
examples |
spans |
thr |
precision |
recall |
f0.5 |
f1 |
| NAMED_DATA |
64 |
9 |
0.60 |
0.1200 |
0.3333 |
0.1376 |
0.1765 |
| DESCRIPTIVE_DATA |
64 |
10 |
0.30 |
0.0149 |
0.1000 |
0.0180 |
0.0260 |
| VAGUE_DATA |
64 |
21 |
0.50 |
0.1154 |
0.1429 |
0.1200 |
0.1277 |
Origin breakdown (per-origin metrics)
| origin |
examples |
spans |
thr |
precision |
recall |
f0.5 |
f1 |
| fcv_pads_east_africa |
27 |
11 |
0.50 |
0.1481 |
0.3636 |
0.1681 |
0.2105 |
| general_prwp |
14 |
12 |
0.50 |
0.1212 |
0.3333 |
0.1389 |
0.1778 |
| jdc_operational |
6 |
5 |
0.70 |
1.0000 |
0.2000 |
0.5556 |
0.3333 |
| refugee_pads |
5 |
3 |
0.60 |
1.0000 |
0.3333 |
0.7143 |
0.5000 |
| reliefweb |
12 |
9 |
0.50 |
0.3077 |
0.4444 |
0.3279 |
0.3636 |
Per-label details
| label |
examples |
spans |
thr |
precision |
recall |
f0.5 |
f1 |
| NAMED_DATA |
64 |
9 |
0.60 |
0.1200 |
0.3333 |
0.1376 |
0.1765 |
| DESCRIPTIVE_DATA |
64 |
10 |
0.30 |
0.0149 |
0.1000 |
0.0180 |
0.0260 |
| VAGUE_DATA |
64 |
21 |
0.50 |
0.1154 |
0.1429 |
0.1200 |
0.1277 |
