barb-1-ie-vi
Joint entity and relation extraction for Vietnamese. The strongest model we have built on every text condition we measure - clean prose, text with the diacritics stripped, and ALL CAPS - by a wide margin.
The earlier models are the goby line - goby-4, goby-7, goby-9, goby-12 - and they stay where they are, under those names. This model starts a new line, because the recipe behind it is different enough that the version numbering would have been misleading. The goby line remains useful, and goby-12 is still the one to reach for on job-title boundaries.
What changed
Two things, both of which turned out to matter more than any hyperparameter we tried:
- Fine-tuned directly from knowledgator/gliner-relex-large-v1.0. Earlier gobies were initialized from goby-1, which carried habits learned on an uncleaned dataset.
- Degraded copies ADDED to the training set, not substituted for the clean ones. 5,824 clean sentences kept in full, plus 3,784 degraded copies (30% diacritics stripped, 20% lowercased, 15% uppercased). Earlier versions replaced clean sentences with degraded ones and paid for robustness with clean-text accuracy. Adding instead of replacing removes that trade-off entirely.
8,910 steps.
Results
450 sentences, 1,328 spans. Each model at its own best threshold, with the real relation label list passed to inference. Job-title column is a separate held-out benchmark of 55 hand-annotated spans with correct boundaries.
| model | clean | diacritics stripped | ALL CAPS | job-title boundaries |
|---|---|---|---|---|
| goby-4 | 71.41 | 22.45 | 31.96 | 12.00 |
| goby-7 | 70.23 | 51.30 | 45.71 | 20.47 |
| goby-9 | 71.63 | 21.34 | 30.53 | 17.24 |
| goby-12 | 63.34 | 50.79 | 43.50 | 30.48 |
| barb-1 | 74.80 | 63.25 | 56.06 | 23.36 |
Paired bootstrap on clean text, barb-1 minus the previous best (goby-11, unreleased): +3.09 F1, 95% CI [+1.95, +4.25], 100% of resamples positive.
Threshold sweep on clean text: 0.40 -> 68.08 | 0.50 -> 72.12 | 0.60 -> 74.77 | 0.65 -> 74.80 | 0.70 -> 74.12. Use 0.65. For degraded input use 0.50-0.60.
Known limitations
- Job-title boundaries regressed against goby-12. goby-12 is the same recipe stopped at 2,970 steps and scores 30.48 on the boundary benchmark against 23.36 here. Training longer re-learns the truncation convention that the training data itself contains, so the initialization was only half the cause. If complete job titles are what you need, use goby-12 or the base model, and expect this to stay true until we have correctly-bounded job-title data at scale.
- The job-title benchmark is 55 spans - hand-annotated and held out, but small.
- The 450-sentence benchmark gold is still missing entities, so absolute F1 understates every model on it. Comparisons between models are unaffected.
- Trained on Vietnamese Wikipedia-style prose. Legal, medical and conversational text are out of distribution.
Relations: the operating point matters twice
A relation can only be found if BOTH of its endpoints are detected, so the entity threshold silently caps relation recall. The entity threshold that is best for NER is not the one that is best for relations - they have to be swept together.
Best relation F1 for this model, sweeping both thresholds on 450 sentences / 428 gold relations:
| strict F1 | partial F1 | precision | recall | entity threshold | relation_threshold |
|---|---|---|---|---|---|
| 63.41 | 75.48 | 64.72 | 62.15 | 0.5 | 0.5 |
Same benchmark, every model at its own best setting (strict F1): goby-4 59.20 | goby-7 56.97 | goby-9 60.50 | goby-12 47.03 | barb-1 63.41.
Earlier versions of this card reported relations measured at the NER-optimal entity threshold only, which understated them.
Usage
pip install gliner==0.2.29
import os, sys
from gliner import GLiNER
from huggingface_hub import hf_hub_download
model = GLiNER.from_pretrained("belumind/barb-1-ie-vi").to("cuda")
sys.path.insert(0, os.path.dirname(
hf_hub_download("belumind/barb-1-ie-vi", "barb_labels.py")))
from barb_labels import extract
ents, rels = extract(model, [text], threshold=0.65, json_keys=True)
Two things that silently cost you a lot of accuracy
1. Pass the real relation list, even when you only want entities. The relation head feeds back into entity detection. Calling inference with a placeholder relation label instead of the real list costs these models 2 to 6 F1. extract() passes the full list by default.
2. Write labels the way Vietnamese is written. GLiNER encodes the label STRING with the same text encoder as the input, so the label you pass is itself a piece of Vietnamese text. Measured on this model, threshold 0.65:
| labels passed to the model | NER F1 | cost |
|---|---|---|
| with diacritics, spaces | 74.8 | - |
| ASCII, spaces (to chuc) | 39.8 | -35.0 |
| snake_case ASCII (to_chuc) | 29.6 | -45.2 |
These are two separate penalties. Dropping the diacritics costs 35 F1; replacing the space with an underscore costs another 10 on top. Neither raises an error on its own, which is why barb_labels.check_labels() raises on the first and warns on the second. snake_case is perfectly fine as an OUTPUT key - barb_labels maps it for you, with to_model_labels(keys) on the way in and extract(..., json_keys=True) on the way out. It just must never be what you hand the model. extract(..., json_keys=True) calls the model correctly and hands back snake_case keys in the OUTPUT, which is free.
Weights ship as both model.safetensors (what gliner loads by preference) and pytorch_model.bin; the two are byte-identical, tensor for tensor.
Which model do I want?
| what you need | model |
|---|---|
| Anything general - clean or degraded Vietnamese text | barb-1 (this one) |
| Complete job titles / chuc vu boundaries | goby-12 |
Related
- goby-12-ie-vi - job-title boundaries
- goby-9-ie-vi
- goby-7-ie-vi
- goby-4-ie-vi
Citation
@misc{barb1ievi2026,
title = {barb-1-ie-vi: joint entity and relation extraction for Vietnamese},
author = {Belumind},
year = {2026},
howpublished = {https://huggingface.co/belumind/barb-1-ie-vi}
}
- Downloads last month
- -