OpenVoiceOS/cohere-transcribe-2b-onnx

Cohere Transcribe 2B — ONNX (nemo-conformer-aed)

ONNX export of CohereLabs/cohere-transcribe-03-2026 for onnx-asr.

Stock onnx-asr loads this model. No patches and no new model family are needed.

Despite the "2B ASR" framing, cohere_asr is not a speech-LLM. It is an attention encoder-decoder built on a NeMo FastConformer: CohereAsrConfig declares sub_configs = {"encoder_config": ParakeetEncoderConfig}, and its decoder prompt is token-for-token the NVIDIA Canary prompt. It therefore drops straight into the existing nemo-conformer-aed family that already serves Canary.

Usage

import onnx_asr

# straight from the Hub; the model type is read from config.json
model = onnx_asr.load_model("OpenVoiceOS/cohere-transcribe-2b-onnx", quantization="int8")

# or from a local copy
model = onnx_asr.load_model("nemo-conformer-aed", "path/to/this/repo")

print(model.recognize("audio_16khz.wav", language="en"))

Drop quantization="int8" for the fp32 graphs.

language accepts any of the 14 supported codes: ar de el en es fr it ja ko nl pl pt vi zh. pnc=True|False toggles punctuation and capitalisation.

Graph contract

Two graphs, drop-in compatible with istupakov/canary-1b-v2-onnx:

graph	inputs	outputs
encoder-model.onnx	audio_signal [B,128,T] f32, length [B] i64	encoder_embeddings [B,T/8,1024] f32, encoder_mask [B,T/8] i64
decoder-model.onnx	input_ids [B,C] i64, encoder_embeddings [B,E,1024] f32, encoder_mask [B,E] i64, decoder_mems [9,B,P,1024] f32	logits [B,C,16384] f32 (log-softmax), decoder_hidden_states [9,B,P+C,1024] f32

Two details make the zero-patch fit work:

• The 1280 -> 1024 decoder.proj linear is folded into the encoder output, so the decoder graph sees 1024-dim memories exactly as the contract requires.
• The HF CohereAsrDecoder uses a standard transformers KV cache. The export re-expresses it as NeMo-style decoder_mems (per-layer pre-layernorm hidden states, num_layers + 1 = 9 entries), which is what the onnx-asr AED decode loop drives.

No feature extractor is baked into the graph. CohereAsrFeatureExtractor turned out to be the standard NeMo log-mel front end — dither 1e-5, preemphasis 0.97, n_fft 512 / win 400 / hop 160, symmetric Hann, 128 slaney mels, log(x + 2**-24), per-feature normalisation — so onnx-asr's built-in nemo128 preprocessor already matches it, down to the frame-count formula.

Files

file	size
encoder-model.onnx + .data	7.59 GB
decoder-model.onnx	676 MB
encoder-model.int8.onnx + .data	1.91 GB
decoder-model.int8.onnx	170 MB

fp32 total 8.3 GB, int8 total 2.1 GB.

Accuracy

Four FLEURS clips (2 en, 2 pt), greedy decoding, against native transformers on the same clips.

build	token-identical to native
fp32	4 / 4
int8	2 / 4

fp32 is exact. int8 dynamic quantisation costs a little accuracy — observed drift is a spurious comma and a mis-spelled rare proper noun. Session-swapping shows both graphs contribute (encoder int8 alone: 3/4 drift; decoder int8 alone: 2/4 drift), so there is no single subgraph to exclude. Use fp32 when accuracy matters and int8 when size matters.

Note: onnx-asr's built-in detokeniser drops the space before an opening bracket or quote (sugar(especially). That is upstream onnx-asr behaviour shared with Canary, not an export defect — the token ids are identical.

Speed

AMD Ryzen 5 7600 (6 cores / 12 threads), CPU execution provider, OMP_NUM_THREADS=6, nice -n 10.

clip	duration	fp32 RTF	int8 RTF
en_1	6.5 s	0.178	0.186
en_2	16.4 s	0.355	0.186
pt_1	11.8 s	0.348	0.214
pt_2	14.6 s	0.301	0.186
mean		0.30	0.19

Scope

Single-clip transcription only. The source processor splits audio longer than 35 s at low-energy boundaries and stitches the pieces back together; that chunking is not part of this export. Feed clips under about 30 s, or segment them yourself.

Attribution

Source model and weights: Cohere Labs, CohereLabs/cohere-transcribe-03-2026, Apache-2.0. This repository contains only an ONNX conversion; the license is inherited unchanged.