BR-Voice-Reasoner

BR-Voice-Reasoner is a multimodal reasoning model for spoken interaction. It understands spoken requests and responds with knowledge, reasoning, and instruction-aware text. It operates directly on audio, allowing applications to reason over what was said without inserting a separate transcription model into the pipeline.

The model is designed for spoken question answering, knowledge-intensive voice queries, multi-step reasoning, instruction following, open-ended interaction, and spoken interaction involving safety and refusal behavior.

Model overview

Item	Value
Input	Audio, text, image, and video
Output	Text
Architecture	Multimodal Mixture-of-Experts
Thinker LM	30B total / 3B activated MoE
Primary evaluation language	English
Weights	BF16 Safetensors

Post-training primarily targets speech-conditioned interaction; image and video capabilities are inherited from the base model and were not comprehensively re-evaluated.

Training

BR-Voice-Reasoner is trained from Qwen3-Omni-30B-A3B-Thinking using cross-modal on-policy distillation. During training, the student processes spoken requests, while a frozen teacher uses their aligned text forms to provide token-level learning signals along student-generated trajectories.

Optimization is restricted to two audio-language projection modules. The language model, vision encoder, and remaining audio encoder parameters stay frozen, focusing post-training on the audio-language interface used to access the model's existing knowledge, reasoning, and instruction-following capabilities.

Training data

Training data cover spoken knowledge, reasoning, instruction following, open-ended interaction, and safety-oriented tasks using both real-world and synthesized speech.

Evaluation

BR-Voice-Reasoner is evaluated on nine VoiceBench subsets covering knowledge, reasoning, instruction following, safety, and open-ended spoken interaction. All scores are reported on a 0–100 scale; higher is better.

VoiceBench subset	Qwen3-Omni 30B-A3B-Thinking	Nemotron 3 Nano Omni	BR-Voice-Reasoner
IFEval	80.6	88.7	83.2
BBH	88.9	91.1	90.4
AdvBench	97.2	100.0	99.8
AlpacaEval	96.4	95.0	97.3
CommonEval	90.5	91.3	94.2
WildVoice	90.5	91.7	93.2
OpenBookQA	94.3	93.0	96.0
MMSU	83.0	82.3	85.7
SD-QA	78.1	71.4	74.3
VoiceBench Avg	88.8	89.4	90.5

Qwen3-Omni results are taken from the official Qwen3-Omni model card, while Nemotron 3 Nano Omni results are taken from its official model card. BR-Voice-Reasoner results were obtained using the evaluation protocol described below. BR-Voice-Reasoner values are rounded to the same one-decimal format. The external columns and BR-Voice-Reasoner were not produced by a single shared evaluation run; cross-column differences are therefore reported as references rather than as a strictly controlled comparison.

BR-Voice-Reasoner evaluation protocol

Item	Setting
Reasoning	Enabled
Generation	temperature 0.6, top-p 0.95, top-k 20
Model judge	GPT-4o-mini, three judgments
Overall	Mean of nine normalized subset scores

AlpacaEval, CommonEval, and WildVoice ratings are normalized to a 0–100 scale before aggregation. The VoiceBench average is the arithmetic mean of the nine normalized subset scores.

Full generation, seeding, evaluator, and retry settings are provided in evaluation_protocol.json.

Installation

pip install "transformers==5.12.1" "accelerate" "qwen-omni-utils==0.0.9"

ffmpeg must also be available on the system for media loading. FlashAttention 2 is optional; install it separately with pip install flash-attn --no-build-isolation on compatible hardware.

Usage

from transformers import (
    Qwen3OmniMoeForConditionalGeneration,
    Qwen3OmniMoeProcessor,
)
from qwen_omni_utils import process_mm_info

MODEL_ID = "brgroup/BR-Voice-Reasoner"

model = Qwen3OmniMoeForConditionalGeneration.from_pretrained(
    MODEL_ID,
    dtype="auto",
    device_map="auto",
    # attn_implementation="flash_attention_2",  # optional
)
processor = Qwen3OmniMoeProcessor.from_pretrained(MODEL_ID)

conversation = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": "example.wav"},
            {"type": "text", "text": "Answer the question in the audio."},
        ],
    }
]

prompt = processor.apply_chat_template(
    conversation,
    add_generation_prompt=True,
    tokenize=False,
)
audios, images, videos = process_mm_info(conversation)
inputs = processor(
    text=prompt,
    audio=audios,
    images=images,
    videos=videos,
    return_tensors="pt",
    padding=True,
)
inputs = inputs.to(model.device).to(model.dtype)

text_ids, _ = model.generate(
    **inputs,
    return_audio=False,
    thinker_return_dict_in_generate=True,
    max_new_tokens=2048,
)
response = processor.batch_decode(
    text_ids.sequences[:, inputs["input_ids"].shape[1]:],
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False,
)
print(response[0])

Limitations

• Primary training and evaluation are English-focused; multilingual, visual, and video capabilities inherited from the base model were not comprehensively re-evaluated.
• Performance may vary across accents, recording conditions, noise levels, domains, and prompting styles.
• The model may mishear speech, hallucinate, reason incorrectly, or produce unsafe or biased content.
• Model-based evaluation results may vary with evaluator versions and inference settings.

License and attribution

BR-Voice-Reasoner is released under the Apache License 2.0 and is derived from Qwen3-Omni-30B-A3B-Thinking, whose original model copyright is Copyright 2025 Alibaba Cloud. Modifications are Copyright 2026 Bairong Inc. See LICENSE and NOTICE for license terms and attribution.