mariojcr/laya-onnx

Laya · ONNX export

ONNX export of convaiinnovations/laya, the non-autoregressive decision model by Convai Innovations (ModernBERT-large encoder + typed decision head, 421M parameters, Apache-2.0). All weights are the original authors' work; this repository only changes the format: one ONNX graph with the encoder and the head together, so the whole model runs under ONNX Runtime without PyTorch.

It is the artifact Altherium serves through its inference gateway (POST /v1/decisions), with the model loaded in-process by each inference node.

Files

The source repo bundles three checkpoints; each is exported to its own folder with its own sidecars, and the graph file names differ on purpose (Altherium identifies a model by its file name without extension):

Folder	Graph	Encoder	Context	Use it for
english/	laya.onnx	ModernBERT-large (421M)	512	English
multilingual/	laya-multilingual.onnx	mmBERT-base (322M)	1024	100+ languages, ~2× faster
typed-decisions/	laya-typed-decisions.onnx	ModernBERT-large (421M)	1024	the typed-decisions workflows

Next to every graph:

File	Purpose
<name>.onnx	the graph (encoder + head) with the weights inline (F32, 1.3–1.7 GB), exported with torch.onnx.export(dynamo=True), opset 18
tokenizer.json	the checkpoint's tokenizer, unchanged (byte-level BPE for ModernBERT; Gemma's SentencePiece BPE for mmBERT)
tokenizer_config.json	which token is CLS/SEP/MASK/PAD ([CLS]… for ModernBERT, <bos>/<eos>/<mask>/<pad> for mmBERT)
rl_agent_config.json	SDK configuration (max_len, head_max_len, calibration temperatures), unchanged

export_onnx.py is the script that produced all of it.

Graph contract

Inputs (all with dynamic axes: n questions, L tokens, k options):

Name	Type	Shape
input_ids	int64	[n, L]
attention_mask	int64	[n, L]
marker_pos	int64	[n, k] — position of the [MASK] marker of each option
marker_mask	bool	[n, k] — which markers are real (padding → false)
qtype	int64	[n] — 0 = choice, 1 = score, 2 = noul

Outputs: logits float32 [n, k] (masked options at −1e4) and act_logits float32 [n, 2].

The sequence layout, option budgets, calibration temperatures and answer format are those of the Laya SDK (laya/common.py, laya/agent.py): [CLS] <type> question: <instructions> [SEP] [MASK] option… [SEP] <state> [SEP], up to 48 tokens per option and head_max_len for all of them, max_len = 512 tokens total. Apply the SDK's temperature_by_options (clamped to [0.5, 5]) before the softmax to get calibrated probabilities.

Parity

Verified against the PyTorch reference on the SDK's README example and a second batch shape: max |Δ logits| = 1.8e-6. ONNX Runtime 1.29, CPU execution provider: ~370 ms for a batch of 4 questions of 96 tokens on 20 threads.

Limits (inherited)

512-token context (state is truncated on the right); English checkpoint (the multilingual Laya variants are a different encoder and are not exported here); see the original model card for the honest limits on high-cardinality choice questions and zero-shot typed decisions.

Credits and license

Model, training and SDK: Convai Innovations, Apache-2.0. This export: Mario Campos, Apache-2.0.