ModaLeap/fun-asr-nano-onnx

Fun-ASR-Nano-2512 ONNX for CUDA

This model is used by https://github.com/mercallureAI/local-multimodal-infra .

This repository provides an ONNX export of FunAudioLLM/Fun-ASR-Nano-2512 for ONNX Runtime with the CUDA execution provider, together with the FunASR pipeline files the runtime loads with it: FSMN-VAD and the CAM++ speaker model.

Original model:

https://huggingface.co/FunAudioLLM/Fun-ASR-Nano-2512 (exported from the ModelScope snapshot, which includes the trained CTC head)

Original project:

https://github.com/FunAudioLLM/Fun-ASR

This is not an official Fun-ASR release.

Description

Fun-ASR-Nano is SenseVoiceSmall's encoder feeding two heads: an adaptor whose output is placed in a Qwen3-0.6B prompt, so the LLM writes the transcript, and a CTC decoder. It takes hotwords and a language hint in its prompt, and handles mixed Chinese and English better than SenseVoice.

The package has two graphs:

• asr/audio_encoder.onnx: LFR features (speech [1, T, 560], speech_lengths) to the adaptor output audio_embeds [1, T, 1024], the greedy CTC token per frame ctc_ids (argmax on the GPU) and encoder_out_lens. Linear weights int8 (MatMulNBits, blocks of 32), activations float32; the CTC head stays float32. Weights at 256-byte aligned offsets, listed in audio_encoder.shared.json, for CUDA graph sessions that share them.
• asr/llm.onnx: one Qwen3 step over embeds [1, S, 1024] (token embeddings looked up by the runtime in asr/embed_tokens.f16, with the adaptor output at the audio positions): 28 layers built by the onnxruntime-genai builder (FP16, GroupQueryAttention over a shared past/present KV cache), int8 layer and head weights (blocks of 128), the tied LM head at the last position and its argmax next_id. Prefill and decode use the same graph; decode shapes are static, so the CUDA provider can capture and replay it.
• The residual stream runs scaled by a power of two: Qwen3's first-token activation in layer 2 (about 1.25e5) would otherwise overflow FP16. The stream is only read through RMSNorm, so the scaling changes nothing else.

The runtime runs the prompt upstream decodes with (<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\n语音转写：[audio]<|im_end|>\n<|im_start|>assistant\n), decodes greedily and can check the LLM's text against the CTC tokens to catch hallucinated output. Prompt ids, dimensions and special ids are in asr/manifest.json.

Built with scripts/local/fun_asr_nano_export.py of local-multimodal-infra.

Files

File	Role	License	Source
asr/audio_encoder.onnx + .onnx.data, asr/audio_encoder.shared.json	Encoder, adaptor and CTC head	Apache-2.0	Fun-ASR-Nano-2512, exported here
asr/llm.onnx + .onnx.data	Qwen3-0.6B decoder step	Apache-2.0	Fun-ASR-Nano-2512 (Qwen3-0.6B), exported here
asr/embed_tokens.f16	Token embeddings, row-major little-endian FP16 (151936 x 1024)	Apache-2.0	Fun-ASR-Nano-2512
asr/tokenizer.json	Qwen3 tokenizer	Apache-2.0	Fun-ASR-Nano-2512 (Qwen3-0.6B)
asr/ctc_tokens.json	CTC vocabulary, base64 bytes per id	Apache-2.0	Fun-ASR-Nano-2512
asr/config.yaml	Frontend configuration (80-bin FBANK, LFR 7/6)	Apache-2.0	Fun-ASR-Nano-2512, unchanged
asr/manifest.json	Package contract: prompt token ids, dimensions, special ids	Apache-2.0	Generated
vad/model_quant.onnx, vad/am.mvn, vad/config.yaml	FSMN-VAD	Apache-2.0	funasr/fsmn-vad-onnx
speaker/campplus_cn_en_common_200k.onnx	CAM++ speaker embeddings	Apache-2.0	3D-Speaker CAM++ via welcomyou/campplus-3dspeaker-200k-onnx
manifest.json	Sources, per-file licenses, size and SHA-256 of every file
config.json	Package summary; the Hub counts downloads by requests for this file. Not read at run time
LICENSE, NOTICE	Apache-2.0 text and attribution

Keep the layout: the runtime looks for asr/, vad/ and speaker/ in one model directory.

Use

With local-multimodal-infra's fun_asr_nano adapter:

hf download ModaLeap/fun-asr-nano-onnx --local-dir workdir/models/fun-asr-nano-onnx

Runtime notes:

• GPU only in practice: the LLM decoder is FP16 and too slow on the CPU for live use.
• About 2.2 GB of GPU memory as loaded (RTX 4090, ONNX Runtime 1.31).
• Audio: 16 kHz mono. Hotwords and a language (Chinese, English or Japanese) go into the prompt.

Testing

fun_asr_nano_export.py parity compares the adaptor output, the CTC tokens and the transcripts with the PyTorch model token by token. On an RTX 4090 a 1.5 s segment takes about 30-45 ms. It has been used for live captions on Chinese and English. No formal WER benchmark was run.

Source

Item	Revision
FunAudioLLM/Fun-ASR-Nano-2512 (ModelScope)	db823224e5c2510001b4a3d3ed97f430bd52ac88, model.pt SHA-256 81fec8616083c69377f3ceef36aba3655660ee0ca69a5d4a1e9810cd340ca499
funasr/fsmn-vad-onnx	f6e9fbb4cefa7397216c763f21307993f147f585
welcomyou/campplus-3dspeaker-200k-onnx	6265ff7af2a104d745b4389026ed9815c6c1c6ff

License

Apache-2.0, as the original model (LICENSE). The VAD and speaker models are Apache-2.0 too. See NOTICE for the sources and changes.

All rights to the original models belong to their authors (Alibaba Group Tongyi Lab, the Qwen team and the FunASR and 3D-Speaker teams).

Disclaimer

This is an unofficial ONNX export. It is not affiliated with, maintained by, or endorsed by the Fun-ASR authors.