andrzej/basal-1.5-mini-onnx-web

basal-1.5-mini ONNX for the browser (WebGPU)

An ONNX conversion of Remek/basal-1.5-mini (1.5B, fine-tuned from speakleash/Bielik-1.5B-v3.0-Instruct) for onnxruntime-web on WebGPU. basal is a typed-decision model by Remigiusz Kinas: it reads a state and answers a question by giving a probability for each allowed option, without generating text. See the original model card and the basal engine for what it does, its benchmarks and its limits.

This is an unofficial port. It is used by the SemIf browser lab to compare basal with Vega 0.8B and Qwen3 0.6B locally in the browser.

Files

file	what
model.onnx + model.onnx.data	decoder with KV cache (GroupQueryAttention), 1.50 GB
tokenizer.json, tokenizer_config.json, chat_template.jinja	the original tokenizer and chat template
CALIBRATION.json, basal.json	the original per-type temperatures, certified thresholds and readout description
genai_config.json	onnxruntime-genai config written by the builder

Inputs: input_ids, attention_mask (int64) and past_key_values.{0..31}.{key,value} (float32, [batch, 2, past, 128]). Outputs: logits (float32, [batch, seq, 32000]) and present.*.

Quantisation and precision

• Weights: asymmetric int4 (block 32, MatMulNBits), with int8 for the most quantisation-sensitive layers (first and last eighth of the layers, every third layer's qkv / v / down projections). The output head and the embeddings stay in fp32, because the decision is read from the output head's logits of the option letters.
• Activations and KV cache: fp32. The fp16 WebGPU graph gave wrong answers on an Intel Xe-LPG GPU (the same option for almost every prompt) while the fp32 graph matched the CPU result there, so this port keeps fp32 activations.

Built with onnxruntime-genai 0.17.1:

python -m onnxruntime_genai.models.builder -i Remek/basal-1.5-mini -o out -p int4 -e cpu \
  --extra_options block_size=32 nodes_to_exclude=/lm_head/MatMul accuracy_level=1 \
  is_symmetric=false matmul_mixed_precision=mixed_layers:int8

Agreement with the original

On the 24 example items of the basal repository (choice, noul, score), each in both option orders (48 prompts), against the original weights in fp32 PyTorch:

runtime	same top option per prompt	mean of max |Δp| per prompt
onnxruntime CPU	47 / 48	0.070
onnxruntime-web 1.30.0, WebGPU (Intel Xe-LPG)	47 / 48	0.069

Other variants measured the same way: plain symmetric int4 0.112 (45/48), k-quant 0.097 (45/48), asymmetric int4 0.080 (46/48), asymmetric int4 block 16 0.074 (47/48). This is a small check, not a benchmark; CALIBRATION.json holds the bf16 model's temperatures and thresholds, which were not refitted for this port.

Reading a decision

Follow basal/prompt.py and basal/server.py of the basal engine:

1. Render the chat template with the fixed system prompt (Polish if the state or question contains Polish letters, English otherwise), the state, the question and the options lettered A. to J., then append {"answer": ".
2. Run the prompt and take the logits of the last position at the token ids of the option letters; softmax over them.
3. Do it again with the options reversed, map both back to the original order and average.
4. Apply the temperature for the question type: p ← softmax(log p / T).

The page's basal-worker.js does exactly this, computes the shared prefix (system prompt and state) once per request, and checks three reference answers before use.

License

Apache-2.0, like the original model. See LICENSE and NOTICE.