Mirror of inclusionAI/Ming-3.0-audio-tiny. All credit to the original authors. Not an official release from them. License unchanged (mit).

Ming-3.0-audio-tiny

Hear clearly. Understand deeply. Act intelligently.

Ming-3.0-audio-tiny is a compact native audio-language model from inclusionAI. It accepts audio together with text instructions and supports speech recognition, spoken question answering, audio understanding, audio reasoning, and optional synthesized-speech responses.

GitHub

·

🤗 Flash

·

🤗 Tiny

·

💻 Examples

Model Summary

Property	Description
Developer	inclusionAI
Model type	Compact native audio-language Mixture-of-Experts model
Parameters	9.3B total parameters, 2.7B activated per inference step
Inputs	Audio and text instructions
Outputs	Text, or text with optional synthesized speech
Primary languages	Chinese and English, including varied accents and dialects
License	MIT

Highlights

• Efficient audio intelligence: Provides speech and audio understanding in a compact Mixture-of-Experts model.
• Complex speech recognition: Handles Chinese and English speech, dialects, accents, noisy environments, and multi-speaker recordings.
• Long-audio understanding: Comprehends and analyzes long-form recordings.
• Multidimensional perception: Understands speech, non-speech sounds, acoustic scenes, and music.
• Audio reasoning: Answers questions and performs reasoning over audio content.
• Audio interaction: Produces text responses and can optionally synthesize speech with built-in voices or a reference voice.

Architecture

Ming-3.0-audio-tiny combines an audio encoder and projector with a compact Mixture-of-Experts language backbone. An optional Talker generates speech responses from the model's output.

Quick Start with Transformers

The model uses custom Transformers code. Review the repository code before enabling trust_remote_code=True.

git clone https://github.com/inclusionAI/Ming-3.0-audio-flash.git
cd Ming-3.0-audio-flash

python -m venv .venv-transformers
source .venv-transformers/bin/activate
pip install -r requirements-transformers.txt

Audio Understanding

python examples/transformers_audio.py examples/audio/introduce_ant_group.wav \
  --model inclusionAI/Ming-3.0-audio-tiny \
  --prompt "Please answer the question in the audio."

Speech Recognition

python examples/transformers_audio.py examples/audio/introduce_ant_group.wav \
  --model inclusionAI/Ming-3.0-audio-tiny \
  --task asr \
  --max-new-tokens 256 \
  --no-repeat-ngram-size 10

The validated ASR prompt asks the model to identify the spoken language and place a tab between the language label and transcription.

Direct Python Usage

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_id = "inclusionAI/Ming-3.0-audio-tiny"
audio_path = "examples/audio/introduce_ant_group.wav"

processor = AutoProcessor.from_pretrained(
    model_id,
    trust_remote_code=True,
    use_fast=False,
)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
).eval()

messages = [
    {
        "role": "user",
        "content": [
            {"type": "audio", "audio": audio_path},
            {"type": "text", "text": "Please answer the question in the audio."},
        ],
    }
]

inputs = processor(messages).to(model.device)
inputs["audio_feats"] = inputs["audio_feats"].to(model.dtype)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        do_sample=False,
    )

generated_ids = output_ids[0, inputs["input_ids"].shape[1] :]
response = processor.decode(generated_ids, skip_special_tokens=True).strip()
print(response)

Speech Output

The Talker example supports built-in voices and reference-audio voice prompting:

python examples/transformers_thinker_talker.py \
  --model inclusionAI/Ming-3.0-audio-tiny \
  --audio examples/audio/introduce_ant_group.wav \
  --ip-name US_Stella \
  --output output.wav

See the voice guide for available voice names, invocation rules, and reference-audio usage.

Serving

vLLM-Omni

Text output or ASR uses the Tiny Thinker on one GPU:

export MODEL_PATH=inclusionAI/Ming-3.0-audio-tiny
MING_MODEL_VARIANT=tiny MING_OUTPUT_MODE=text \
  ./scripts/serve_vllm_omni.sh "$MODEL_PATH"

For synthesized-speech output, select the full-audio profile:

export MODEL_PATH=inclusionAI/Ming-3.0-audio-tiny
MING_MODEL_VARIANT=tiny MING_OUTPUT_MODE=audio \
  ./scripts/serve_vllm_omni.sh "$MODEL_PATH"

The reference full-audio profile places the Thinker on GPU 0 and the Talker on GPU 1. Select the deployment YAML that matches text-only, ASR, or Talker output. See the deployment guide for details.

For speech output, vLLM-Omni uses the built-in IP voice 灵小柔 by default. The model directory also provides 灵楷 and US_James as preset reference voices under voice_presets/voice_audio/, with their matching transcripts in voice_presets/voice_prompts.json. Both names are also available as built-in IP voices.

SGLang

export MODEL_PATH=inclusionAI/Ming-3.0-audio-tiny
MING_MODEL_VARIANT=tiny CUDA_VISIBLE_DEVICES=0 \
  ./scripts/serve_sglang.sh "$MODEL_PATH"

The validated Tiny SGLang profile uses one GPU and provides text output. See the SGLang example for client usage.

Evaluation

The shared benchmark figures below include results for both Ming-3.0-audio-tiny and Ming-3.0-audio-flash.

Speech Recognition

Audio Intelligence

Text Benchmarks

All benchmark values are percentages. Higher is better unless the metric is marked with a downward arrow. Results marked with an asterisk are self-evaluated. Refer to the GitHub README for the latest notes and evaluation details.

License

This model is released under the MIT License. See LEGAL.md for the legal disclaimer.