DakeraAI/gliner-x-small-int8

gliner-x-small

Dakera's default multilingual entity extraction model · int8 ONNX by Dakera, the self-hosted memory engine for AI agents

Dakera's int8 build of knowledgator/gliner-x-small for fast CPU inference with ONNX Runtime: 189.2 MB of graphs, the same entities as the original. The model is by its original authors; all credit goes to them.

In Dakera

The default for every Dakera server with a multilingual embedding model (bge-m3, multilingual-minilm): about 100 languages.

docker run -p 3000:3000 -e DAKERA_ROOT_API_KEY=dk-mykey ghcr.io/dakera-ai/dakera:latest

Quality and speed

Measured through Dakera's own extraction pipeline (threshold 0.5; the public sets with the labels person, organization, location; ONNX Runtime 1.31 with weight prepacking, one CPU thread per pass on an AMD Ryzen 9 7950X):

Set	F1
Dakera chat sentences, English (20; 10 labels)	0.62
Dakera chat sentences, 7 languages (44; 10 labels)	0.59
WNUT-17 (social media, English)	0.37
CoNLL-2003 (news, English)	0.71

MultiCoNER v2 (150 sentences per language):

de	en	es	fr	hi	it	pt	zh
0.56	0.57	0.67	0.63	0.51	0.67	0.58	0.20

Median time per request on the chat set: 0.016 s.

What changed

Built from the original PyTorch checkpoint by build.py (reproducible: the same tools give the same bytes):

• The span head, factored exactly. GLiNER's span head runs two dense layers on every one of words × 12 spans; the first is split into a start part and an end part computed once per word, and the last is folded into the label side. The logits are unchanged (largest difference 5.7e-06); the cost moves from per span to per word.
• 8-bit weights. The encoder's and the heads' weight matrices are block-wise 8-bit (MatMulNBits, block 32, symmetric, int8 arithmetic) and the word-embedding table is int8 per row. Activations stay FP32: quantizing them is what makes the uniformly quantized GLiNER exports find nothing.
• Checked against the original on multilingual probe sentences: logits correlation at least 1.000, 99% of the entities identical.

Usage

This is a uni-encoder: the labels go in the prompt before the text, <<ENT>> label … <<SEP>> text words, tokenized as pre-split words.

Word splitting: runs of letters, marks and digits (joined by - or _), any other character alone, and each Chinese or Japanese character alone. Spans of up to 12 words; sigmoid over the logits, threshold 0.5. Keep ONNX Runtime's weight prepacking on (the default): without it the 8-bit MatMuls repack their weights on every run, 4–5× slower.

Files

File	Size
onnx/model.onnx	189.2 MB
tokenizer.json	16.3 MB
gliner_config.json	3.5 kB

SHA-256 checksums

2f6f0a848f9210d0fb7c30f1b35a952509afa198edf872aaab7f19527a33cede  onnx/model.onnx
fa84928bec4c38fdf40a153724857ba1ba86aa3789d9801251bed82c8eaa033d  tokenizer.json
2e2d806a3510d532be55e37fd6bcf7741bb00776cc6be79783f97c7acc25f3e0  gliner_config.json

Licence and credit

Apache-2.0, the same as the original model (see LICENSE). For what the model does, how it was trained and its evaluation, see the original model card.

@misc{zaratiana2023gliner,
  title={GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer},
  author={Urchade Zaratiana and Nadi Tomeh and Pierre Holat and Thierry Charnois},
  year={2023},
  eprint={2311.08526},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

About Dakera

Dakera is a self-hosted memory engine for AI agents. It gives agents long-term memory that survives sessions: hybrid vector and full-text retrieval, reranking, entity extraction and importance decay, in a single Rust binary that runs on your own servers, with an MCP server for Claude, Cursor and other agents.

docker run -p 3000:3000 -e DAKERA_ROOT_API_KEY=dk-mykey ghcr.io/dakera-ai/dakera:latest

🌐 Website	dakera.ai
📚 Documentation	dakera.ai/docs
💻 GitHub	github.com/Dakera-AI
🚀 Self-hosting (Docker, Kubernetes, Helm)	dakera-deploy
🔌 MCP server	dakera-mcp
🧰 SDKs	Python · TypeScript · Go · Rust
💼 LinkedIn	Dakera AI

Questions or feedback: amine@dakera.ai.