Audio-R1 — Qwen2.5-Omni-7B (LoRA)

LoRA adapter for Qwen/Qwen2.5-Omni-7B trained with Audio-R1, an RL post-training framework that uses audio self-supervised tasks as a source of verifiable rewards.

Paper: Audio-R1: Audio Reasoning through Reinforcement Learning on Self-Supervised Tasks
Project page: https://ijihoon98.github.io/Audio-R1/

Training Details

Hyperparameter	Value
Method	GRPO, 1 epoch, LoRA
Reward	accuracy (1.0 if the chosen letter is correct) + format (0.2)
Format reward	`<think>` and `<answer>` tags present
Optimizer / LR schedule	AdamW, cosine
Learning rate / warmup ratio	5e-6 / 0.03
Rollouts per prompt	8
Sampling temperature / top-p	0.8 / 0.9
KL coefficient β	0.04
Clip ε	0.2
Per-device batch × grad. accumulation	8 × 4
GPUs	4
LoRA rank / α / dropout	8 / 32 / 0.05
LoRA targets	all linear layers; audio encoder frozen
Precision / seed	bf16 / 42

Usage

import torch
from peft import PeftModel
from transformers import Qwen2_5OmniForConditionalGeneration, Qwen2_5OmniProcessor
from qwen_omni_utils import process_mm_info

base = "Qwen/Qwen2.5-Omni-7B"
processor = Qwen2_5OmniProcessor.from_pretrained(base)
model = Qwen2_5OmniForConditionalGeneration.from_pretrained(base, torch_dtype=torch.float16, device_map="auto")
model = PeftModel.from_pretrained(model, "NearAndDear98/Audio-R1-Qwen2.5-Omni-7B").eval()

SYSTEM = (
    "Your response MUST contain two parts in this exact order: "
    "1. Your reasoning enclosed within <think> </think> tags. "
    "2. The final answer — a single letter for the correct option — enclosed "
    "within <answer> </answer> tags."
)
conversation = [
    {"role": "system", "content": [{"type": "text", "text": SYSTEM}]},
    {"role": "user", "content": [
        {"type": "audio", "audio": "example.wav"},
        {"type": "text", "text": "Question ...\n(A) ...\n(B) ...\n(C) ...\n(D) ..."},
    ]},
]
text = processor.apply_chat_template(conversation, add_generation_prompt=True, tokenize=False)
audios, images, videos = process_mm_info(conversation, use_audio_in_video=False)
inputs = processor(text=text, audio=audios, images=images, videos=videos,
                   return_tensors="pt", padding=True, use_audio_in_video=False).to(model.device).to(model.dtype)
out = model.generate(**inputs, max_new_tokens=1024, thinker_do_sample=False,
                     use_audio_in_video=False, return_audio=False)
print(processor.batch_decode(out, skip_special_tokens=True)[0])

License

The adapter is released under Apache 2.0, the license of the base model.

Citation

@article{jeong2026audior1,
  title   = {Audio-R1: Audio Reasoning through Reinforcement Learning on Self-Supervised Tasks},
  author  = {Jeong, Jihoon and Ravanelli, Mirco and Wang, Zhepei and Subakan, Cem},
  journal = {arXiv preprint arXiv:2610.34609},
  year    = {2026}
}