Keithsel/Aster-Vietnamese-NER-MiniLM-GGUF

Aster Vietnamese NER MiniLM GGUF

F16 GGUF conversion of Aster Vietnamese NER MiniLM, a 15-label Vietnamese named-entity recognizer.

Files

• Aster-Vietnamese-NER-MiniLM-F16.gguf: 232 MiB MiniLM encoder with XLM-R SentencePiece tokenizer.
• classifier.json: the 16-by-384 token-classification matrix and ordered label names.

Runtime contract

Run the model in llama.cpp with --embedding --pooling none. llama.cpp returns one 384-dimensional embedding per token. Apply classifier.json to each embedding:

logits = embedding @ weight.T + bias
label = labels[argmax(logits)]

llama-server does not expose a token-classification endpoint, so the small final projection remains application-owned.

Parity

On 100 held-out VLSP 2021 test sentences, F16 GGUF output retained 99.86% exact entity-span F1 against the original FP32 checkpoint. Per-token label agreement was 99.9877%.

Performance

On an 8-thread AMD Ryzen 7 8840HS, local llama-server embedding requests measured 44.8 ms median and 82.2 ms p95 per sentence, including local HTTP overhead.

The underlying FP32 checkpoint scored 81.89% exact typed F1 on VLSP 2021 test, 86.33% on WikiANN Vietnamese test, and 92.06% on Vietnamese COVID NER validation.

This is a research artifact. Review the training datasets' terms before redistribution or production use.