Confucius4-R2T2 for AMD Ryzen™ AI NPUs (FastFlowLM)

Confucius4-R2T2 by NetEase Youdao ("Real Real-Time Transcription", fine-tuned from Qwen3-ASR-1.7B) converted for FastFlowLM (FLM) on the XDNA 2 NPU. 2B parameters, 30 languages (best at Chinese and English).

Any modifications made to the original model in this Derivative Work are not endorsed, warranted, or guaranteed by the original right-holder of the original model, and the original right-holder disclaims all liability related to this Derivative Work.

This is a converted copy of the original weights, subject to the NetEase Youdao Model Use License Agreement (MODEL_LICENSE, included). Please read it: among other terms it requires a separate license from NetEase Youdao for organizations with more than 100 million monthly active users or more than RMB 1 billion annual revenue (including their affiliates), and it forbids use of the model or its outputs to improve other AI models except non-commercial ones.

Run

Requires FastFlowLM 1.0.14 or newer.

flm run r2t2:1.7b
# then:  /input "audio.wav"            (wav, flac, mp3, ogg)
#        /input "audio.wav" hot words, names, topic        (text after the file is context for the model)

flm serve r2t2:1.7b
curl http://localhost:52625/v1/audio/transcriptions -F file=@audio.wav -F model=r2t2:1.7b

Whole-file transcription, or streaming with -F stream=true (append-only text while the audio is processed, chunk_size_ms 80-2000). See the model page and the benchmarks.

What is in this package

File Content
model.q4nx text decoder (Qwen3-1.7B shape): Q8_0 projections and output head, BF16 embedding (the CPU prompt prefill reads it)
npu_q41.q4nx Q4_1 copies of q/k/o/gate/up and the output head for the NPU decode (v and down stay Q8_0 from model.q4nx)
audio_weight.q4nx audio encoder (24 layers), BF16, in the NPU matmul tile layout
config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja from the original checkpoint
MODEL_LICENSE NetEase Youdao Model Use License Agreement (original)

The NPU decode reads mixed precision: 4-bit everywhere (Q4_0, Q4_1, Q4_K round-to-nearest) breaks this fine-tune (empty transcripts or Mandarin answered in English), but Q4_1 for q/k/o/gate/up and the output head with v_proj and down_proj at Q8_0 keeps its output - and decodes ~24 % faster than Q8_0 throughout. On 120 English (LibriSpeech test-clean) and 60 Mandarin (FLEURS) utterances, compared with the original BF16 model, FLM gives WER 2.81 % vs 2.89 % (English) and CER 6.37 % vs 8.96 % (Mandarin; the gap is mostly how numbers are written, digits vs Chinese numerals, which the scoring does not normalise).

Source

Converted from netease-youdao/Confucius4-R2T2 (BF16, revision 185ce639) with the FastFlowLM q4nx converter.

Downloads last month
11
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for FastFlowLM/Confucius4-R2T2-NPU2

Finetuned
(4)
this model