florianvoss/Qwen3.5-9B-Autoround-a16w4-Modalix

Qwen3.5-9B AutoRound A16W4 for Modalix

Runtime artifacts compiled for the LLiMa runtime on SiMa.ai Modalix.

• SmoothQuant alpha: 0.5.
• Decoder Linear weights, including DeltaNet QKV/Z/output projections: symmetric AutoRound INT4, group size 256.
• Output head: symmetric GPTQ INT4, group size 256.
• Vision and projector Linear weights: RTN INT8 per output channel.
• DeltaNet A/B projections, convolution parameters, normalization parameters, A_log, and dt_bias: BF16.
• Context capacity: 4096 tokens; prefill group size: 128.
• Vision input: 448 × 448; vision encoder packaged as per-layer ELFs.
• Filter sharing, quantized embeddings, and quantized KV cache enabled.

Updated chat template

devkit/chat_template.jinja includes a prefix-stability fix for multi-turn, non-thinking conversations: it omits an empty <think> wrapper when an assistant message has no reasoning content. This keeps completed history stable when another user turn is appended and avoids large prompt replays with LLiMa's checkpoint cache. The packaged original is preserved as devkit/chat_template_original.jinja.

Run

Download the complete repository while preserving its directory structure. On Modalix with a compatible LLiMa runtime installed:

llima run /path/to/Qwen3.5-9B-Autoround-a16w4-Modalix

devkit/ contains runtime configuration, tokenizer assets, and embeddings. elf_files/ contains 142 compiled MLA programs. This repository contains Modalix runtime artifacts rather than a Transformers checkpoint.

Compilation, archive validation, and local llima-deploy completed successfully. Modalix runtime evaluation has since covered text, vision, multi-turn, and native tool-call workflows.