MissingPackage/jev-d-0.4b-onnx

ONNX fp32 export of laya-typed-decisions

Format conversion. The weights are numerically unchanged: no retraining, fine-tuning or distillation. Apache 2.0, inherited. See NOTICE.

The tensors live in a sidecar .data. meta.json holds the budgets, the special tokens and the fitted temperatures.

Loading

import * as ort from "onnxruntime-web/webgpu";

const weights = new Uint8Array(await (await fetch("jev-d-0.4b.onnx.data")).arrayBuffer());

const session = await ort.InferenceSession.create("jev-d-0.4b.onnx", {
  executionProviders: ["webgpu"],
  graphOptimizationLevel: "basic",
  externalData: [{ path: "laya-typed-decisions.onnx.data", data: weights }],
});

Inputs are input_ids, attention_mask, marker_pos, marker_mask and qtype, all int64. The output is logits, one row per question. The sequence is [CLS] <type> question: instructions [SEP] [MASK] opt0 [MASK] opt1 ... [SEP] state [SEP].

Four things that break silently

• externalData.path is the location string stored inside the graph, not the served filename. Renaming a file does not rename it inside the graph.
• graphOptimizationLevel must be "basic". On "all" the optimiser fuses residual and LayerNorm into SkipLayerNormalization, which the onnxruntime-web WebGPU kernel rejects with «Beta must be 1D».
• The graph was traced with K >= 2 and requires at least two options per question.
• The state must be serialised with Python's json spacing (", " and ": "). One missing space is a different token sequence, a different answer, and no error anywhere.

The temperature needs clamping

The checkpoint's choice:11+ bucket is 0.1006. A temperature below 1 sharpens logits: that one multiplies them about tenfold and republishes a 0.24 top probability as 0.99. meta.json carries both the raw and the clamped values; the clamp is [0.5, 5.0].