goby-12-ie-vi

Joint entity and relation extraction for Vietnamese. The job-title specialist: the first model in the goby family whose chuc vu spans keep their full boundaries, and the first to beat the base model at it.

What is different about this one

It is fine-tuned directly from knowledgator/gliner-relex-large-v1.0. Every earlier goby was initialized from goby-1, which had learned a span-truncation convention from an uncleaned dataset. Cleaning the data later did not undo it, because the convention lived in the weights, not the labels. Taking goby-1 out of the chain removes it.

Training data: 5,824 clean Vietnamese Wikipedia-style sentences, plus 3,784 degraded copies ADDED alongside them (30% diacritics stripped, 20% lowercased, 15% uppercased). The clean sentences are kept, not replaced - that is what buys robustness without paying for it in clean-text accuracy. 2,970 steps.

Which goby do I want?

what you need	model
Full job titles, correct chuc vu boundaries	goby-12 (this one)
Degraded input: no diacritics, ALL CAPS, mixed case	goby-12 or goby-7
Best accuracy on clean well-formed prose	goby-4 or goby-9

Results

All numbers below are measured with the REAL relation label list passed to inference. Passing a placeholder relation label instead costs every goby several F1 - see the Usage section.

Job-title boundaries

Held-out instrument: 40 sentences, 55 hand-annotated chuc vu spans with correct boundaries. Strict exact match.

model	threshold	exact (of 55)	predictions	F1
gliner-relex-large (base)	0.6	22	121	25.14
goby-9	0.4	10	62	17.24
goby-7	0.4	13	73	20.47
goby-11 (unreleased)	0.4	12	46	24.00
goby-12	0.4	16	51	30.48
goby-12	0.3	20	79	30.08

Every earlier goby scored BELOW the base model on this benchmark. goby-12 is the first one above it. It gets there on precision: 16 correct out of 51 predictions, against the base model 22 out of 121.

Degraded text

Same 450-sentence benchmark, whole sentence degraded, each model at its own best threshold:

condition	goby-12	goby-7	goby-9	goby-4
diacritics stripped	50.79	51.30	21.34	22.45
ALL CAPS	43.50	45.71	30.53	31.96

goby-12 is in the same class as goby-7, the robustness model, and roughly 2x goby-4 and goby-9 on stripped text.

Typed NER on clean text - this is the cost

450 sentences, 1,328 spans, each model at its own best threshold:

model	best F1	threshold
goby-9	71.63	0.60
goby-4	71.41	0.70
goby-7	70.23	0.65
goby-12	63.34	0.50

goby-12 is 8.4 F1 behind on general clean-text extraction. Do not pick it for that job.

The honest reading: training loss ended at 5.94 for goby-12 against 3.69 for the same recipe initialized from goby-1. The model had not converged at this step budget, so this gap is at least partly the price of the step budget rather than the price of dropping goby-1. A longer run is testing exactly that, and if it holds the gap should close in a later release.

Relations: the operating point matters twice

A relation can only be found if BOTH of its endpoints are detected, so the entity threshold silently caps relation recall. The entity threshold that is best for NER is not the one that is best for relations - they have to be swept together.

Best relation F1 for this model, sweeping both thresholds on 450 sentences / 428 gold relations:

strict F1	partial F1	precision	recall	entity threshold	relation_threshold
47.03	61.94	52.6	42.52	0.4	0.6

Same benchmark, every model at its own best setting (strict F1): goby-4 59.20 | goby-7 56.97 | goby-9 60.50 | goby-12 47.03 | barb-1 63.41.

Earlier versions of this card reported relations measured at the NER-optimal entity threshold only, which understated them.

Usage

pip install gliner==0.2.29

import os, sys
from gliner import GLiNER
from huggingface_hub import hf_hub_download

model = GLiNER.from_pretrained("belumind/goby-12-ie-vi").to("cuda")

sys.path.insert(0, os.path.dirname(
    hf_hub_download("belumind/goby-12-ie-vi", "goby_labels.py")))
from goby_labels import extract

text = "Nam 1960, ong duoc bo nhiem Quan truong quan Duc Hoa thuoc tinh Long An."
ents, rels = extract(model, [text], threshold=0.4, json_keys=True)

Two things that silently cost you accuracy

1. Pass the real relation list, even if you only want entities. The relation head feeds back into entity detection. Measured on this benchmark, calling inference with a placeholder relation label instead of the real list costs every fine-tuned goby between 2 and 6 F1. extract() passes the full list by default.

2. Labels must keep their Vietnamese diacritics. GLiNER encodes the label STRING with the same text encoder as the input, so a de-accented label is a different vector. Measured on goby-7 at threshold 0.5:

labels passed to the model	NER F1	cost
with diacritics, spaces	74.8	-
ASCII, spaces (to chuc)	39.8	-35.0
snake_case ASCII (to_chuc)	29.6	-45.2

Re-measured on goby-13 at threshold 0.65 with real relation labels. These are TWO separate penalties: dropping the diacritics costs 35 F1, and replacing the space with an underscore costs another 10 on top of that. An earlier version of this table, measured with a placeholder relation label, reported -21.1 and -24.9 - it understated the damage by about half.

snake_case is perfectly fine as an OUTPUT key - goby_labels maps it for you, with to_model_labels(keys) on the way in and extract(..., json_keys=True) on the way out. It just must never be what you hand the model.

goby_labels.check_labels() raises on that mistake instead of letting it pass silently, and extract(..., json_keys=True) gives you snake_case keys in the OUTPUT for free.

Weights ship as both model.safetensors (what gliner loads by preference) and pytorch_model.bin; the two are byte-identical, tensor for tensor.

Known limitations

• Clean-text general extraction is 8.4 F1 behind goby-4 and goby-9. If your input is well-formed and you are not specifically after job titles, use one of those.
• Undertrained. Training loss 5.94 vs 3.69 for the goby-1-initialized twin at the same step count. This checkpoint is released because the boundary result is worth having now, not because it is finished.
• The job-title benchmark is 55 spans. It is hand-annotated and held out, but it is small.
• The main 450-sentence benchmark gold is still missing entities; absolute F1 understates every model on it.
• Trained on Vietnamese Wikipedia-style prose. Legal, medical and conversational text are out of distribution.

Related

• goby-4-ie-vi
• goby-7-ie-vi - robustness
• goby-9-ie-vi - best calibrated

Citation

@misc{goby12ievi2026,
  title   = {goby-12-ie-vi: job-title-boundary-corrected joint IE for Vietnamese},
  author  = {Belumind},
  year    = {2026},
  howpublished = {https://huggingface.co/belumind/goby-12-ie-vi}
}