akakaiy/MOSS-Audio-8B-Thinking-ru

MOSS-Audio-8B-Thinking — Russian LoRA (r1) + GGUF

RU коротко: LoRA для MOSS-Audio-8B-Thinking, которая учит модель нормально слышать русский — речь в шуме, по телефону, издалека и пение поверх музыки — и при этом не ломает главное умение модели: описывать звук (музыку, тон голоса, фон). Плюс готовые GGUF Q6 для CrispASR: дообученная и базовая. p.s. увидел большой спрос, уже обучается второй заход с бОльшим датасетом. 28.10.26 выложу.

MOSS-Audio listens to the whole sound — music, tone of voice, background, several speakers — which plain ASR models do not do. Its Russian transcription, however, was weak. This LoRA fixes most of that while keeping the "ears" (audio / music / voice description) alive.

Files

file	size	what
lora-r1/	364 MB	PEFT LoRA (r=32, alpha=32) for OpenMOSS-Team/MOSS-Audio-8B-Thinking
moss-audio-8b-thinking-ru-r1-q6_k-encq8.gguf	7.86 GB	base + LoRA merged, Q6_K LLM, audio-encoder linears in Q8_0
moss-audio-8b-thinking-q6_k-encq8.gguf	7.86 GB	the untouched base in the same format (for A/B)
crispasr-moss-decode.patch	7 KB	patch for CrispASR's moss-audio decoder: token budget, think budget, loop breaker

The GGUFs run in CrispASR (--backend moss-audio); ~7.7 GB VRAM, fits a 10 GB RTX 3080.

Results

Measured on one RTX 3080, GGUF Q6 through CrispASR with the patch below (loop breaker on). CER, lower is better. None of the test sets was trained on.

set	Whisper large-v3 (faster-whisper, int8)	GigaAM v3 (e2e RNN-T)	MOSS base	MOSS r1 (this)
FLEURS-ru, 47 clips without digits in the reference	1.02 %	1.01 %	3.65 %	3.25 %
dev-300 (held-out test splits: noisy / phone / far-field / crowd)	10.08 %	4.93 %	13.18 %	7.14 %
Russian songs, 30 full-mix chunks of held-out artists	69.2 %	17.2 %	33.7 %	24.7 %
repetition loops ("а-а-а…") on songs	0	0	0	0

Description behaviour (45 prompts: music description, voice tone; 15 of them on unseen audio):

	base	epoch-1 LoRA	r1
answers with the reasoning block skipped	0	25	11
voice-tone answers skipped reasoning	0	10 / 17	0 / 17
clean speech wrongly described as "background: music"	0	10	0

Honest summary. On noisy speech and on singing over music this is much better than the base and than Whisper; on clean read speech a dedicated ASR (GigaAM) is still ~3× better. The intended use is a hybrid: a fast ASR writes the words, MOSS r1 listens to how it sounds (tone, music, background) — it can take the ASR text in the prompt.

Limitations

• Numbers are written as words ("тридцать пять миллиметров" instead of "35 мм") — learned from the speech corpora. On FLEURS clips whose reference has digits CER goes up because of this (whole FLEURS: 6.19 % vs base 4.53 %).
• With the "Describe this audio in detail…" prompt it still sometimes answers without the reasoning block (11/45).
• Russian only was targeted; other languages were not measured.

Training

• QLoRA on an RTX 3090: LLM in NF4, audio encoder / adapter / lm_head in bf16; LoRA on all LLM attention + MLP projections and on the audio encoder's q/k/v.
• Data: ~129 h of public Russian speech (SOVA RuDevices, Russian LibriSpeech, Sber Golos crowd & far-field, Taiga speech, Podlodka) + ~41 h of Russian song chunks (vocal-separated, aligned to lyrics or labelled where two ASR systems agreed; the model always hears the full mix) + self-distilled descriptions of the base model (its own answers to music / voice-tone prompts, with reasoning).
• Epoch 1: 106 812 rows, lr 1e-4, cosine, batch 1 × accum 8. Repair pass: 22 052 rows with ~9 % descriptions, lr 5e-5 — the epoch-1 adapter had started to skip reasoning on descriptions, this brought it back.
• The training audio itself is not published.

Usage (CrispASR server)

# build CrispASR with the patch (git apply crispasr-moss-decode.patch), then:
CRISPASR_MOSS_MAX_NEW=1024 CRISPASR_MOSS_THINK_BUDGET=640 CRISPASR_MOSS_LOOP_BREAK=6 \
crispasr --server --port 11443 --backend moss-audio -m moss-audio-8b-thinking-ru-r1-q6_k-encq8.gguf -n 1024
# transcription: CRISPASR_MOSS_MAX_NEW=768 CRISPASR_MOSS_THINK_BUDGET=256
curl -F file=@voice.ogg -F response_format=json \
     -F prompt="Transcribe the speech in this audio verbatim, in the original language. Output only the transcript." \
     http://127.0.0.1:11443/v1/audio/transcriptions

Patch env knobs: CRISPASR_MOSS_MAX_NEW (upstream hard-codes 512 and ignores -n), CRISPASR_MOSS_THINK_BUDGET (closes <think> after N tokens), CRISPASR_MOSS_LOOP_BREAK (stops on N repeats of the same n-gram), CRISPASR_MOSS_REP_PENALTY.

With transformers + PEFT: load the base with its own code (trust_remote_code), then PeftModel.from_pretrained(model, "<this repo>", subfolder="lora-r1").

License: follows the base model's license.