Qwen3-ASR-0.6B for SpacemiT K1/K3

This repository contains a SpacemiT edge-deployment version of Qwen/Qwen3-ASR-0.6B. The speech encoder is split into dynamic-quantized ONNX frontend/backend models. This package uses the multi-batch frontend artifact under the standard filename and runs through the SpacemiT Execution Provider, while the Qwen text decoder is stored as a Q4_1 GGUF model and runs through the SpacemiT llama.cpp backend.

Model origin and acknowledgement

Qwen3-ASR was introduced by the Qwen Team in the Qwen3-ASR Technical Report. It is a multilingual automatic speech recognition family built for language identification and speech transcription. The original Qwen3-ASR family supports 30 languages and 22 Chinese dialects, and is designed for both offline and streaming inference.

This repository is a deployment conversion, not a new base model. We thank the Qwen Team for releasing the original architecture, model weights, technical report, and inference code:

• Qwen3-ASR paper
• Official Qwen3-ASR repository
• Official Qwen3-ASR-0.6B model

The current SpacemiT package exposes offline WAV transcription through the OpenAI-compatible llama-server chat completions API. Chinese transcription has been validated on both K1 and K3 with the bundled zh.wav sample.

Files

Qwen3-ASR-0.6B-SpacemiT/
├── Qwen3-ASR-0.6B-text-q41.gguf
├── Qwen3-ASR-0.6B-encoder-frontend.dynq.onnx
├── Qwen3-ASR-0.6B-encoder-backend.dynq.onnx
├── configs/
│   ├── K1/config.json
│   └── K3/config.json
├── zh.wav
└── README.md

Qwen3-ASR-0.6B-encoder-frontend.dynq.onnx is the multi-batch frontend converted from Qwen3-ASR-0.6B-encoder-frontend.dynq.batched.onnx. The filename is unchanged so the existing K1/K3 configurations remain compatible.

Supported platforms

Platform	Accelerated cores	SMT config	llama-server threads
SpacemiT K1 / X60	0,1,2,3	configs/K1	-t 4
SpacemiT K3 / A100	8,9,10,11,12,13,14,15	configs/K3	-t 8

The platform-specific config.json controls the ONNX encoder's SpaceMIT EP thread count and affinity. The -t argument controls the GGUF text decoder. Do not use the K3 config on K1: K1 has four accelerated cores at IDs 0-3, whereas K3 has eight accelerated cores at IDs 8-15.

The SpacemiT llama.cpp runtime automatically detects the accelerated CPU cores. Normal runs do not require manually setting SPACEMIT_PERFER_CORE_ARCH, SPACEMIT_PERFER_CORE_ID, or SPACEMIT_MEM_BACKEND.

Prerequisites

Two runtime components are required:

1. A SpacemiT ONNX Runtime package containing libonnxruntime and the SpaceMIT Execution Provider: spacemit-com/onnxruntime releases
2. An SMT-enabled SpacemiT llama.cpp build containing llama-server: spacemit-com/llama.cpp

The validation recorded below used SpacemiT ORT 2.0.6 and an SMT-enabled RISC-V llama-server from the SpacemiT fork. The official prebuilt spacemit-llama.cpp.riscv64.0.1.7 package was also separately validated on K3.

Option A: use the prebuilt packages

Download and unpack the current RISC-V glibc releases. The following versions were current when this model card was prepared:

wget https://github.com/spacemit-com/onnxruntime/releases/download/2.0.6/spacemit-ort.riscv64.2.0.6.tar.gz
tar -xf spacemit-ort.riscv64.2.0.6.tar.gz

wget https://github.com/spacemit-com/llama.cpp/releases/download/v0.1.7/spacemit-llama.cpp.riscv64.0.1.7.tar.gz
tar -xf spacemit-llama.cpp.riscv64.0.1.7.tar.gz

Check the release pages for newer mutually compatible packages when deploying this model in another software image.

Option B: build llama.cpp from source

Cross-compilation requires a SpacemiT RISC-V toolchain in addition to the unpacked ORT package.

git clone --recursive https://github.com/spacemit-com/llama.cpp.git
cd llama.cpp

export RISCV_ROOT_PATH=/path/to/spacemit-riscv-toolchain
export SPACEMIT_ORT_DIR=/path/to/spacemit-ort.riscv64.2.0.6

bash build_spacemit.sh glibc

The installed runtime is generated under build/installed/. The equivalent manual CMake build must enable at least GGML_CPU_RISCV64_SPACEMIT=ON and LLAMA_SERVER_SMT_MTMD=ON, and pass SPACEMIT_ORT_DIR to CMake.

Run on a K1 or K3 board

Copy the model folder, the unpacked ORT package, and the prebuilt or locally built llama.cpp installation to the board. Set the following paths to match their actual locations:

export MODEL_DIR=/path/to/Qwen3-ASR-0.6B-SpacemiT
export ORT_DIR=/path/to/spacemit-ort.riscv64.2.0.6
export LLAMA_DIR=/path/to/spacemit-llama.cpp.riscv64.0.1.7

export LD_LIBRARY_PATH="${LLAMA_DIR}/lib:${ORT_DIR}/lib:${LD_LIBRARY_PATH:-}"

K1

K1 uses four accelerated cores, 0-3:

"${LLAMA_DIR}/bin/llama-server" \
  -m "${MODEL_DIR}/Qwen3-ASR-0.6B-text-q41.gguf" \
  --media-backend smt \
  --smt-config-dir "${MODEL_DIR}/configs/K1" \
  -t 4 \
  --host 0.0.0.0 \
  --port 8080 \
  --warmup

The K1 config applies:

"ep_config": {
  "SPACEMIT_EP_INTRA_THREAD_NUM": "4",
  "SPACEMIT_EP_INTER_THREAD_NUM": "1",
  "SPACEMIT_EP_INTRA_THREAD_AFFINITY": "0;1;2;3"
}

K3

K3 uses eight accelerated cores, 8-15:

"${LLAMA_DIR}/bin/llama-server" \
  -m "${MODEL_DIR}/Qwen3-ASR-0.6B-text-q41.gguf" \
  --media-backend smt \
  --smt-config-dir "${MODEL_DIR}/configs/K3" \
  -t 8 \
  --host 0.0.0.0 \
  --port 8080 \
  --warmup

The K3 config applies:

"ep_config": {
  "SPACEMIT_EP_INTRA_THREAD_NUM": "8",
  "SPACEMIT_EP_INTER_THREAD_NUM": "1",
  "SPACEMIT_EP_INTRA_THREAD_AFFINITY": "8;9;10;11;12;13;14;15"
}

Wait until the server prints a line similar to:

llama_server: listening on http://0.0.0.0:8080

If the service is reachable outside a trusted local network, configure an API key and appropriate network access controls instead of exposing an unauthenticated 0.0.0.0 endpoint.

Send a transcription request

The request can be sent from the board itself or another machine that can reach the board. Set SERVER_URL=http://127.0.0.1:8080 when testing locally, or replace BOARD_IP when testing remotely.

export SERVER_URL=http://127.0.0.1:8080

base64 < "${MODEL_DIR}/zh.wav" | tr -d '\n' | jq -Rs '{
  model: "qwen3-asr",
  messages: [
    {
      role: "user",
      content: [
        {
          type: "input_audio",
          input_audio: {
            data: .,
            format: "wav"
          }
        },
        {
          type: "text",
          text: "language Chinese<asr_text>"
        }
      ]
    }
  ],
  max_tokens: 128,
  temperature: 0
}' | curl "${SERVER_URL}/v1/chat/completions" \
  -H "Content-Type: application/json" \
  --data-binary @-

The transcription is returned in choices[0].message.content. Replace zh.wav with another 16 kHz WAV file to transcribe your own audio. The language Chinese prefix can be changed to another supported language name.

Verified example

The bundled zh.wav file was tested on 2026-08-07 with --warmup enabled and the multi-batch frontend described above. Both boards returned HTTP 200 and the same content:

language Chinese<asr_text>开放时间：早上9点至下午5点。

Observed single-request wall time after server warmup:

Board	HTTP status	Wall time
K1 (0-3, -t 4)	200	3.76 s
K3 (8-15, -t 8)	200	0.85 s

These figures are functional smoke-test observations rather than a formal benchmark. Startup and first-request time can be longer while ONNX graphs are compiled and caches are populated.

The official prebuilt llama.cpp v0.1.7 package was additionally tested on K3 with ORT 2.0.6. It returned HTTP 200, the same transcription, and a 0.86 s single-request wall time after warmup.

Citation

Please cite the original Qwen3-ASR work when using this converted model:

@article{Qwen3-ASR,
  title={Qwen3-ASR Technical Report},
  author={Xian Shi and Xiong Wang and Zhifang Guo and Yongqi Wang and Pei Zhang and Xinyu Zhang and Zishan Guo and Hongkun Hao and Yu Xi and Baosong Yang and Jin Xu and Jingren Zhou and Junyang Lin},
  journal={arXiv preprint arXiv:2601.21337},
  year={2026}
}

License

The original Qwen3-ASR model is released under the Apache License 2.0. The SpacemiT llama.cpp and ONNX Runtime packages are separate dependencies and remain subject to their respective repository licenses.