DakeraAI/gliner-bi-small-v2.0-int8

gliner-bi-small

Dakera's default entity extraction model (English) · int8 ONNX by Dakera, the self-hosted memory engine for AI agents

Dakera's int8 build of knowledgator/gliner-bi-small-v2.0 for fast CPU inference with ONNX Runtime: 117.6 MB of graphs, the same entities as the original. The model is by its original authors; all credit goes to them.

In Dakera

The default for every Dakera server with an English embedding model: it tags people, organisations, places and any label a namespace names on stored memories, in the background.

docker run -p 3000:3000 -e DAKERA_ROOT_API_KEY=dk-mykey ghcr.io/dakera-ai/dakera:latest

Quality and speed

Measured through Dakera's own extraction pipeline (threshold 0.5; the public sets with the labels person, organization, location; ONNX Runtime 1.31 with weight prepacking, one CPU thread per pass on an AMD Ryzen 9 7950X):

Set	F1
Dakera chat sentences, English (20; 10 labels)	0.88
Dakera chat sentences, 7 languages (44; 10 labels)	0.70
WNUT-17 (social media, English)	0.67
CoNLL-2003 (news, English)	0.69

MultiCoNER v2 (150 sentences per language):

de	en	es	fr	hi	it	pt	zh
0.43	0.65	0.36	0.50	0.01	0.54	0.41	0.01

Median time per request on the chat set: 0.013 s.

What changed

Built from the original PyTorch checkpoint by build.py (reproducible: the same tools give the same bytes):

• The span head, factored exactly. GLiNER's span head runs two dense layers on every one of words × 12 spans; the first is split into a start part and an end part computed once per word, and the last is folded into the label side. The logits are unchanged (largest difference 3.8e-06); the cost moves from per span to per word.
• 8-bit weights. The encoder's and the heads' weight matrices are block-wise 8-bit (MatMulNBits, block 32, symmetric, int8 arithmetic) and the word-embedding table is int8 per row. Activations stay FP32: quantizing them is what makes the uniformly quantized GLiNER exports find nothing.
• Checked against the original on multilingual probe sentences: logits correlation at least 0.999, 100% of the entities identical.

Usage

This is a bi-encoder: the text goes in without a label prompt, and the labels go in as vectors (labels_embeddings, one row per label) from onnx/labels_encoder.onnx (inputs input_ids, attention_mask from labels_tokenizer.json; output one pooled vector per label). Encode a label once and reuse its vector for every text.

Word splitting: runs of letters, marks and digits (joined by - or _), any other character alone, and each Chinese or Japanese character alone. Spans of up to 12 words; sigmoid over the logits, threshold 0.5. Keep ONNX Runtime's weight prepacking on (the default): without it the 8-bit MatMuls repack their weights on every run, 4–5× slower.

Files

File	Size
onnx/model.onnx	80.6 MB
tokenizer.json	3.6 MB
gliner_config.json	6.3 kB
onnx/labels_encoder.onnx	37.1 MB
labels_tokenizer.json	711.6 kB

SHA-256 checksums

1fa420c63c757863c3296a450e00d9ec155597dffb57771a15d77e416e2bc379  onnx/model.onnx
2315a8bea85452f3c4e8ce980f7853cac013820238e7776a4e48159037a5f164  tokenizer.json
5e3ad08c2c6a6e9fe07f9d91b9d8096fc28606e795336f2b4a06613e3df419ff  gliner_config.json
b3e1fb9a10335c72d96d320b0af0292840c3cbee7b4868e3f24c9a81794d078a  onnx/labels_encoder.onnx
91f1def9b9391fdabe028cd3f3fcc4efd34e5d1f08c3bf2de513ebb5911a1854  labels_tokenizer.json

Licence and credit

Apache-2.0, the same as the original model (see LICENSE). For what the model does, how it was trained and its evaluation, see the original model card.

@misc{zaratiana2023gliner,
  title={GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer},
  author={Urchade Zaratiana and Nadi Tomeh and Pierre Holat and Thierry Charnois},
  year={2023},
  eprint={2311.08526},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

@misc{stepanov2026millionlabelnerbreakingscale,
  title={The Million-Label NER: Breaking Scale Barriers with GLiNER bi-encoder},
  author={Ihor Stepanov and Mykhailo Shtopko and Dmytro Vodianytskyi and Oleksandr Lukashov},
  year={2026},
  eprint={2602.18487},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

About Dakera

Dakera is a self-hosted memory engine for AI agents. It gives agents long-term memory that survives sessions: hybrid vector and full-text retrieval, reranking, entity extraction and importance decay, in a single Rust binary that runs on your own servers, with an MCP server for Claude, Cursor and other agents.

docker run -p 3000:3000 -e DAKERA_ROOT_API_KEY=dk-mykey ghcr.io/dakera-ai/dakera:latest

🌐 Website	dakera.ai
📚 Documentation	dakera.ai/docs
💻 GitHub	github.com/Dakera-AI
🚀 Self-hosting (Docker, Kubernetes, Helm)	dakera-deploy
🔌 MCP server	dakera-mcp
🧰 SDKs	Python · TypeScript · Go · Rust
💼 LinkedIn	Dakera AI

Questions or feedback: amine@dakera.ai.