RASMUS/Nanbeige4.2-3B-ONNX

Nanbeige4.2-3B, ONNX q4f16 for WebGPU

An ONNX export of Nanbeige/Nanbeige4.2-3B @ b82e54bd609793562a75cbf9337970a93369eab5 by the Nanbeige LLM Lab, for Transformers.js on WebGPU. One graph, int4 weights with fp16 activations (q4f16), 3.11 GB in total. This is not an official release of the Nanbeige LLM Lab.

How it was made

Nanbeige4.2-3B runs its 22 stored decoder layers twice for each token, with the final norm after each pass. The graph does the same: layers 0-21, the norm, the same weights again as layers 22-43, the norm, the LM head. So config.json describes a 44-layer Llama, and the graph has 44 layer KV-cache pairs (88 past tensors). The second pass refers to the first pass's weights, which are stored once.

Built with the onnxruntime-genai 0.15.2 model builder (int4 round-to-nearest, block 32, WebGPU target, LM head on the last position only) from the vendor's safetensors. No repository code of the vendor ran. The build scripts are not published with this repo.

Requirements and limits

• WebGPU with shader-f16. There is no q4 or fp32 variant.
• A GPU buffer limit (maxBufferSize) of at least 1,020,788,736 bytes: the fp16 token-embedding table is one tensor of that size.
• Context: 32,768 tokens. The vendor model declares 262,144; this export's RoPE tables stop at 32,768.
• Thinking is on by default, as in the vendor template.
• Tool calling was not tested with this export.

Measured

On Transformers.js 4.3.2, onnxruntime-web 1.30.0, Chrome 156 (Windows), NVIDIA RTX 4080:

• prefill: 1,512 and 1,542 ms for a 2,048-token prompt in two browser launches (each the best of 2 timed runs after a warm-up; generate with one new token)
• decode: 15.9 and 16.7 ms per generated token in the same two launches (each the median of 5 runs of 128 tokens after a 64-token prompt)
• accuracy: 163/200 on a subset: the first 100 ARC-Challenge test questions and 100 MMLU test questions sampled with seed 0. Thinking off; an item counts as right when the most likely next token over the whole vocabulary is the letter of the right option. These are subset results, not full benchmark scores. An fp32 PyTorch reimplementation on CPU scored 161/200 on the same items (exact McNemar p = 0.824): no difference was detected, and equivalence was not shown.

The reference is our own reimplementation. It was not compared with the vendor's modeling code.

Changes to the vendor's files (Apache-2.0, section 4)

manifest.json lists every other file of this repo with its size, SHA-256 and origin.

• onnx/model_q4f16.onnx, onnx/model_q4f16.onnx_data, onnx/model_q4f16.onnx_data_1: new files, derived from the vendor's weights (converted and quantized to int4). The graph's doc_string and metadata_props carry this notice for all three.
• config.json: modified from the vendor's file, with the notice in _modification_notice. Each change, as vendor value -> this repo's value:
  • architectures: ["NanbeigeForCausalLM"] -> ["LlamaForCausalLM"]: Transformers.js has no Nanbeige class, and the graph needs none
  • model_type: "nanbeige" -> "llama": the graph is a plain Llama decoder
  • num_hidden_layers: 22 -> 44: the graph runs the 22 stored layers twice, so it has 44 layer KV-cache pairs (88 past tensors)
  • max_position_embeddings: 262144 -> 32768: the export's RoPE tables hold 32,768 positions
  • transformers.js_config: absent -> {"kv_cache_dtype": {"q4f16": "float16"}, "use_external_data_format": {"model_q4f16.onnx": 2}}: the external data chunks and the fp16 KV cache
  • _nanbeige_note: absent -> "Nanbeige4.2-3B looped graph: 22 stored layers run twice; the graph exposes 44 KV caches, so Transformers.js sees a 44-layer Llama.": explains the 44 layers
  • _modification_notice: absent -> "Modified from the config.json of Nanbeige/Nanbeige4.2-3B at revision b82e54bd609793562a75cbf9337970a93369eab5 for the ONNX export RASMUS/Nanbeige4.2-3B-ONNX. README.md lists every change.": the notice that this file was modified
  • mlp_bias: absent -> false: a Llama config field
  • auto_map: {"AutoConfig": "configuration_nanbeige.NanbeigeConfig", "AutoModel": "modeling_nanbeige.NanbeigeModel", "AutoModelForCausalLM": "modeling_nanbeige.NanbeigeForCausalLM"} -> absent: no repository code is needed
  • num_loops: 2 -> absent: the loop is in the graph
  • skip_loop_final_norm: false -> absent: the loop is in the graph
  • loop_loss_weights: [] -> absent: training only
  • kv_channels: 128 -> absent: equal to head_dim
  • pretraining_tp: 1 -> absent: training only
  • transformers_version: "4.42.4" -> absent: not the version that wrote this file
  • max_length: null -> absent: null (unset) in the vendor's file; absent means the same
• chat_template.jinja: new file, a copy of the template in the vendor's tokenizer_config.json.
• tokenizer.json, tokenizer_config.json, special_tokens_map.json, generation_config.json: the vendor's files, unchanged.

Licence

Apache License 2.0 (LICENSE), the licence that the vendor's model card declares for Nanbeige/Nanbeige4.2-3B.

Citation

From the vendor's model card:

@article{lab2026nanbeige4,
  title={Nanbeige4. 2-3B: Unlocking Agentic Capabilities in a Compact Mode},
  author={Lab, Nanbeige and Yang, Chen and Huang, Chengrui and Lan, Fufeng and Chen, Hanhui and Zhou, Hao and Song, Huatong and Cao, Jiaqi and Zhu, Jiaying and Niu, Jinlin and others},
  journal={arXiv preprint arXiv:2607.22083},
  year={2026}
}