DakeraAI/gliner_medium-v2.1-int8

gliner-medium

Entity extraction for Dakera (English) · int8 ONNX by Dakera, the self-hosted memory engine for AI agents

Dakera's int8 build of urchade/gliner_medium-v2.1 for fast CPU inference with ONNX Runtime: 271.8 MB of graphs, the same entities as the original. The model is by its original authors; all credit goes to them.

In Dakera

Dakera's default before 0.12.4, kept for deployments that chose it: DAKERA_GLINER_MODEL=gliner-medium.

docker run -p 3000:3000 -e DAKERA_ROOT_API_KEY=dk-mykey ghcr.io/dakera-ai/dakera:latest

Quality and speed

Measured through Dakera's own extraction pipeline (threshold 0.5; the public sets with the labels person, organization, location; ONNX Runtime 1.31 with weight prepacking, one CPU thread per pass on an AMD Ryzen 9 7950X):

Set	F1
Dakera chat sentences, English (20; 10 labels)	0.81
Dakera chat sentences, 7 languages (44; 10 labels)	0.76
WNUT-17 (social media, English)	0.58
CoNLL-2003 (news, English)	0.69

MultiCoNER v2 (150 sentences per language):

de	en	es	fr	hi	it	pt	zh
0.45	0.58	0.49	0.50	0.16	0.55	0.42	0.07

Median time per request on the chat set: 0.049 s.

What changed

Built from the original PyTorch checkpoint by build.py (reproducible: the same tools give the same bytes):

• The span head, factored exactly. GLiNER's span head runs two dense layers on every one of words × 12 spans; the first is split into a start part and an end part computed once per word, and the last is folded into the label side. The logits are unchanged (largest difference 7.6e-06); the cost moves from per span to per word.
• 8-bit weights. The encoder's and the heads' weight matrices are block-wise 8-bit (MatMulNBits, block 32, symmetric, int8 arithmetic) and the word-embedding table is int8 per row. Activations stay FP32: quantizing them is what makes the uniformly quantized GLiNER exports find nothing.
• Checked against the original on multilingual probe sentences: logits correlation at least 1.000, 100% of the entities identical.

Usage

This is a uni-encoder: the labels go in the prompt before the text, <<ENT>> label … <<SEP>> text words, tokenized as pre-split words.

Word splitting: runs of letters, marks and digits (joined by - or _), any other character alone, and each Chinese or Japanese character alone. Spans of up to 12 words; sigmoid over the logits, threshold 0.5. Keep ONNX Runtime's weight prepacking on (the default): without it the 8-bit MatMuls repack their weights on every run, 4–5× slower.

Files

File	Size
onnx/model.onnx	271.8 MB
tokenizer.json	8.3 MB
gliner_config.json	2.7 kB

SHA-256 checksums

22d105be97dc466927edd34935778a5566cf7e095c89e811a3e5f8396674c48f  onnx/model.onnx
1731c431e5e851a4a73487b5bbfbef265b29ab64de190c402b4e6df9ce2cb0ab  tokenizer.json
771037f6b2ad65f907a9e8d86271f02ce6beede04ce01a62c2a446bf5917a6a2  gliner_config.json

Licence and credit

Apache-2.0, the same as the original model (see LICENSE). For what the model does, how it was trained and its evaluation, see the original model card.

@misc{zaratiana2023gliner,
  title={GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformer},
  author={Urchade Zaratiana and Nadi Tomeh and Pierre Holat and Thierry Charnois},
  year={2023},
  eprint={2311.08526},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

About Dakera

Dakera is a self-hosted memory engine for AI agents. It gives agents long-term memory that survives sessions: hybrid vector and full-text retrieval, reranking, entity extraction and importance decay, in a single Rust binary that runs on your own servers, with an MCP server for Claude, Cursor and other agents.

docker run -p 3000:3000 -e DAKERA_ROOT_API_KEY=dk-mykey ghcr.io/dakera-ai/dakera:latest

🌐 Website	dakera.ai
📚 Documentation	dakera.ai/docs
💻 GitHub	github.com/Dakera-AI
🚀 Self-hosting (Docker, Kubernetes, Helm)	dakera-deploy
🔌 MCP server	dakera-mcp
🧰 SDKs	Python · TypeScript · Go · Rust
💼 LinkedIn	Dakera AI

Questions or feedback: amine@dakera.ai.