beshkenadze/gliner2.5-multi-onnx

GLiNER2.5-multi, ONNX export for the browser

ONNX export of fastino/gliner2.5-multi-v1 (GLiNER2, boundary architecture, mDeBERTa-v3-base, Apache 2.0) split into the two graphs that form-pilot runs with onnxruntime-web to turn résumés and other documents into Label: value profiles entirely on-device.

File	Size	What
encoder_fp16.onnx	556 MB	mDeBERTa-v3-base encoder, weights in fp16, float32 output. Inputs input_ids, attention_mask (int64, [1, seq]), output last_hidden_state ([1, seq, 768]).
head.onnx	5 MB	Boundary head. Inputs text_states [1, W, 768], text_mask (bool [1, W]), query_states [1, Q, 768], query_mask (bool [1, Q]); outputs indices [1, Q, C, 2] (int64 word spans, end exclusive), valid_mask [1, Q, C], pair_logits [1, Q, C].
tokenizer.json, tokenizer_config.json	16 MB	Unchanged from the base model; the schema marker tokens ([P], [C], [SEP_TEXT], [DESCRIPTION], …) are already registered.

Usage

The pre- and post-processing (word splitting, schema prompt layout, routing indices, span decoding) is a port of gliner2's SchemaTransformer and lives in packages/extract of form-pilot. In short: encode ( [P] prompt ( [C] field … ) ) [SEP_TEXT] words… token by token, gather last_hidden_state at each text word's first subword and at each [C] marker, run the head, take the argmax over pair_logits where valid_mask is set.

Run the encoder on WebGPU (≈200 ms per 300-token chunk on an Apple M-series GPU) or WASM (≈2 s single-threaded); run the head on WASM, it returns wrong results on the WebGPU execution provider of onnxruntime-web 1.22–1.30. Keep chunks at ~120 words: the boundary model is not meant for whole documents at once.

How it was exported

• Encoder: torch.onnx.export (opset 17, legacy exporter) of the encoder converted with .half(), output cast to float32. Post-hoc fp16 converters broke DeBERTa's Cast/If nodes.
• Head: torch.onnx.export(dynamo=True) of BoundaryHead.forward(..., return_candidates=True) returning candidates.indices / valid_mask / pair_logits, with torch.sort and torch.argsort patched to drop stable=True (no ONNX function for the stable overload); external data embedded into the single file.
• Verified against PyTorch on a synthetic résumé: identical best spans, probabilities within 0.01 in fp16.

Not quantized on purpose: int8 dynamic quantization lowered confidences on low-margin fields.