FlexiSLM/Qwen2_5-Omni-Audio_Encoder

Qwen2.5-Omni Audio Encoder

Standalone audio encoder extracted from Qwen/Qwen2.5-Omni-7B for FlexiSLM.

This checkpoint keeps only Qwen2_5OmniAudioEncoder (thinker.audio_tower.*: conv layers, 32 transformer layers, ln_post, and proj). The Omni thinker LLM, talker, vision tower, and tokenizer are omitted.

Property	Value
Parameters	640M
Size	~1.19 GiB (bfloat16)
Output dim	3584
Mel bins	128

• Paper: arXiv:2606.31247
• FlexiSLM code: AmphionTeam/FlexiSLM

Usage with FlexiSLM

Required for both 7B and 0.5B FlexiSLM inference and training. Shared by all Stage 1 / Stage 2 / Stage 3 recipes.

Download

MODEL_ROOT="$PWD/models"
hf download FlexiSLM/Qwen2_5-Omni-Audio_Encoder \
  --local-dir "$MODEL_ROOT/Qwen2_5-Omni-Audio_Encoder"

With FlexiSLM auto_download=True, this encoder is fetched automatically together with the Stage 2 SLM, SenseVoice, and FlexiCodec assets.

Inference config (Python)

from pathlib import Path
from src.inference_flexislm import FlexiSLMInferenceConfig

model_root = Path.cwd() / "models"
config = FlexiSLMInferenceConfig(
    checkpoint="stage2_7B",  # or "stage2_0.5B"
    model_path=str(model_root / "FlexiSLM-7B-Stage2"),
    qwen25o_encoder_path=str(model_root / "Qwen2_5-Omni-Audio_Encoder"),
    qwen25o_encoder_config_path=str(
        model_root / "Qwen2_5-Omni-Audio_Encoder/config.json"
    ),
    # ... FlexiCodec / SenseVoice / flow-matching paths ...
)

Training / YAML

use_qwen25omni_feature: true
qwen25omni_encoder_path: models/Qwen2_5-Omni-Audio_Encoder
qwen25omni_encoder_config_path: models/Qwen2_5-Omni-Audio_Encoder/config.json

Full TTS / ASR / audio QA / speech-to-speech examples: FlexiSLM Inference Guide.

Load with Transformers

Weights are loadable with transformers.models.qwen2_5_omni.modeling_qwen2_5_omni.Qwen2_5OmniAudioEncoder.

License

Apache 2.0, same as the source Qwen2.5-Omni-7B checkpoint. Copyright 2025 Alibaba Cloud.

Citation

If you use this encoder as part of FlexiSLM:

@misc{li2026flexislmdynamiccontrollableframe,
      title={FlexiSLM: A Dynamic and Controllable Frame Rate Spoken Language Model},
      author={Jiaqi Li and Chaoren Wang and Xiaohai Tian and Mingjie Chen and Xinyu Liang and Xu Li and Yufan Lin and Junwen Qiu and Jun Zhang and Lu Lu and Haizhou Li and Zhizheng Wu},
      year={2026},
      eprint={2606.31247},
      archivePrefix={arXiv},
      primaryClass={cs.SD},
      url={https://arxiv.org/abs/2606.31247},
}