onnx-community/GLiNER2.5-Decide-ONNX

GLiNER2.5-Decide, ONNX for the browser

ONNX export of the classification path of fastino/GLiNER2.5-Decide, packaged for Transformers.js and the open-jev typed-decision library. It runs on WebGPU (WASM fallback) with nothing leaving the tab.

GLiNER2.5-Decide is a DeBERTa-v3-large encoder with a trained label head, made by Fastino for typed operational decisions: routing, sentiment, yes/no gates, ordinal scores. Its extraction heads (span, boundary, record) are not part of this export.

Use with open-jev

import { OpenJev, choice, noul, score } from "open-jev";

const jev = await OpenJev.load({ model: "gliner2-decide" }); // or "onnx-community/GLiNER2.5-Decide-ONNX"

const [team, angry, severity] = await jev.decide(
  "The export button crashes in Safari but works in Chrome. Not blocking.",
  [
    choice("Which team should handle this?", ["billing", "technical support", "sales"], {
      billing: "charges, refunds, invoices",
      "technical support": "bugs, outages",
    }),
    noul("Is the customer angry?"),
    score("How severe is this bug?", ["cosmetic", "degraded but there is a workaround", "blocking"]),
  ],
);

Files

File	Size	Notes
onnx/model.onnx + _data	1.74 GB	fp32, reference
onnx/model_fp16.onnx + _data	872 MB	default on WebGPU with shader-f16
onnx/model_q4.onnx + _data	888 MB	4-bit MatMul weights, fp32 elsewhere (the 128k-token embedding table dominates)
onnx/model_q4f16.onnx + _data	523 MB	4-bit MatMul weights, fp16 elsewhere; smallest download

Graph

Inputs: input_ids [batch, sequence], attention_mask [batch, sequence], marker_positions [batch, markers] (int64, index of every [L] token). Output: logits [batch, markers], one logit per label from the model's 1024→2048→1 head. Softmax within each question's markers gives the distribution.

The sequence is the GLiNER2 processor's classification layout, which the caller must reproduce exactly (open-jev does):

( [P] prompt ( [L] label_1 [L] label_2 … ) ) [SEP_STRUCT] ( [P] … ) [SEP_TEXT] word word … .

• The prompt is the task text, with per-label descriptions appended as [DESCRIPTION] label: description.
• The prompt and the labels keep their case and are tokenized as whole strings.
• The state is lowercased, split with the processor's word regex, given a terminal . if it has none, and tokenized one word at a time with no special tokens.
• ( and ) are tokenized as standalone words. No [CLS]/[SEP].

Fidelity

Compared with the Python gliner2 library (v2.0.0) on 6 states and 13 questions:

Variant	Same argmax	Worst probability difference
fp32	13/13	5.8e-7
fp16	13/13	2.5e-4
q4	12/13	7.2e-2
q4f16	12/13	7.1e-2

The q4 flip is a three-way call at 0.50 / 0.42 probability. Scripts to reproduce the export and these numbers are in conversion/.

Limits

• 512-token context. Long states are cut at the end by open-jev.
• Trained on English operational text; not trained on public benchmarks (see the base model card).
• This export covers classification only. Entity, relation and structured extraction need the Python library.

Credits

Model by Fastino (Apache-2.0). Export and open-jev integration by Shreyas Karnik. Not affiliated with Fastino or TypeSafe.