laion/moss-sports-commentator-lora

MOSS Voice-Acting — Sports-Commentator LoRAs

Three PEFT/LoRA adapters that push laion/moss-tts-local-transformer-4.55b-voice-acting-v2 toward energetic live sports commentary — the shouted, fast, rising-intensity register of a broadcaster calling a goal, a world record or a knockout as it happens.

These are the three best-scoring cells out of a 13-cell sweep (ranks 16/32/64 × epochs 1/2/3/8), selected by a blind listening evaluation with a real-audio ceiling control.

🎧 Listen to the full evaluation — all 13 cells, base, and real human commentary

---

⚠️ Read this before you use them: the base model is already good at this

We measured the honest thing, so here it is up front.

configuration	mean judge (0–2)	% rated 2	vs base	p (permutation)
r64_e1	1.969	97 %	+0.219	0.027
r64_e8	1.969	97 %	+0.219	0.027
r32_e1	1.938	94 %	+0.188	0.081
real human Mediathek commentary	1.775	83 %	—	—
base model, no LoRA	1.750	75 %	0.000	1.000

n = 32 clips per cell, 456 clips total, judged by gemini-3-flash on the same 0/1/2 rubric used to filter the training data. 0 unparsed.

Three things follow, and none of them should be glossed over:

1. The base model (1.750) is statistically indistinguishable from real human sports commentary (1.775). The metric's ceiling is ≈1.8 and the un-adapted model is already there. There is very little headroom for an adapter to occupy.
2. No cell survives multiple-comparison correction. Two cells clear p < 0.05 raw, but with 12 comparisons against base the Bonferroni threshold is p < 0.0042. Treat the ranking above as suggestive, not established.
3. A 0–2 absolute scale was the wrong instrument. Every cell lands between 1.75 and 1.97 and 75–97 % of clips get the top score — the scale is compressed against its ceiling. A follow-up should use pairwise A/B preference against base, which stays sensitive when everything already sounds like sports commentary.

Practical guidance: try the base model first. Reach for these adapters if you want the register pushed harder and more consistently (the % rated 2 column is where the difference is most visible: 97 % vs 75 %), not because the base model fails at the task.

Validation loss did not predict the listening result

The training log says epoch 1 is best and epoch 8 badly overfits — at rank 64, val loss goes 4.4237 → 5.4369, a 0.905 regression. Listeners rate r64_e1 and r64_e8 identically at 1.969. That is why both are published here: whatever epoch 8 lost in validation loss, it did not lose in how the audio sounds.

This is the third time on this model stack that val loss failed to rank checkpoints the way a listener does. Rank on generation-side metrics, not on loss.

---

Which adapter to pick

adapter	rank	alpha	size	val loss	mean judge	notes
r64_e1	64	128	525 MB	4.4237	1.969	default choice — best score at the lowest val loss
r64_e8	64	128	525 MB	5.4369	1.969	same score after 8 epochs; published as the counter-example to loss-based selection
r32_e1	32	64	263 MB	4.4135	1.938	half the size, within noise of the rank-64 cells

If you are stacking adapters or care about download size, r32_e1 is the sensible pick — the difference between it and rank 64 is one clip out of 32.

---

Where everything lives

🧩 Base model (required)	laion/moss-tts-local-transformer-4.55b-voice-acting-v2 — trained against v2; they will degrade on the earlier …-voice-acting checkpoint
📦 Model home, demos, prompting guide	github.com/LAION-AI/laion-moss-local-1.5-voice-acting-4.55b
📖 Prompting manual	projects.laion.ai/moss-voiceacting-manual
🔬 Pipeline, training & measured learnings	github.com/LAION-AI/Voice-Acting-Pipeline-WIP · docs/17
🎭 40 emotion adapters	TTS-AGI/moss-emotion-loras-v3
🎛️ 64 vocal-burst adapters	laion/vocal-burst-lora-adapters
🎧 This evaluation, with players	sports_commentator_lora.html

---

Quickstart

import torch, soundfile as sf
from transformers import AutoProcessor, AutoModel
from peft import PeftModel

BASE    = "laion/moss-tts-local-transformer-4.55b-voice-acting-v2"
CODEC   = "OpenMOSS-Team/MOSS-Audio-Tokenizer-v2"
ADAPTER = "r64_e1"                    # or "r64_e8" / "r32_e1"

# AutoModel, NOT AutoModelForCausalLM -- MossTTSLocalConfig is not registered for the
# CausalLM auto-class and from_pretrained raises "Unrecognized configuration class".
proc  = AutoProcessor.from_pretrained(BASE, trust_remote_code=True, codec_path=CODEC)
model = AutoModel.from_pretrained(
    BASE, trust_remote_code=True, dtype=torch.bfloat16,
    attn_implementation="sdpa",       # flash-attn 2.x is incompatible with this model
).cuda().eval()

pm = PeftModel.from_pretrained(
    model, "laion/moss-sports-commentator-lora",
    subfolder=ADAPTER, adapter_name=ADAPTER,
).eval()

# `instruction` is the whole director's note; `text` is ONLY the spoken words.
# Empty fields render as the literal string "None", so fill them deliberately.
GENERAL = ("A voice that is extremely energetic and aroused; extremely fast and rapid in tempo; "
           "extremely emphatic and projected; strongly ranting and worked up; adult; strongly "
           "masculine; wide-ranging in pitch; genuine and spontaneous rather than performed; "
           "volatile and unstable.")
CUE = ("Extremely energetic and aroused, extremely fast and rapid in tempo, adult, strongly "
       "masculine, extremely emphatic and projected, genuine and spontaneous")

sents = ["He's through, he's one on one — and he's buried it!",
         "In the last minute of the final!",
         "This entire stadium has lost its mind!"]
script = " ".join(f'({CUE}) "{s}"' if i != 1 else f'({CUE}) [pause 0.3s] "{s}"'
                  for i, s in enumerate(sents))
instruction = f"GENERAL: {GENERAL}\nSCRIPT:\n{script}"
text = " ".join(sents)

# The length model that was fitted for English on this stack: words dominate ~4:1,
# ~2.65 words/s, ~12.5 tokens/s.
tokens = int(round(len(text.split()) / 2.65 * 12.5))

conv  = [[proc.build_user_message(text=text, instruction=instruction,
                                  language="English", tokens=tokens)]]
batch = proc(conv, mode="generation")

with torch.no_grad():
    out = pm.generate(
        input_ids=batch["input_ids"].cuda(),
        attention_mask=batch["attention_mask"].cuda(),
        max_new_frames=300, do_sample=True,
        text_temperature=0.7, text_top_k=50, text_top_p=1.0,
        audio_temperature=1.0, audio_top_k=30, audio_top_p=0.95,
        audio_repetition_penalty=1.1,
    )

msg = proc.decode(out)[0]
w = msg.audio_codes_list[0].cpu().float().numpy()   # ALREADY a waveform -- see Traps
if w.ndim > 1:
    w = w.mean(0)
sf.write("commentary.wav", w, 48000)

eval_prompts.json in this repo holds all 16 held-out evaluation prompts in exactly this shape (8 scenes × male/female voice, 16 sports), so you can reproduce the table above.

Adapter strength (merge scale)

The delta is added scaled by alpha / r — 2.0 for every adapter here. Multiply by a dose λ:

from peft.tuners.lora import LoraLayer

# Capture the untouched scaling ONCE. Reading the current value and multiplying makes the
# scale compound on every change and silently drift.
base_scaling = {n: dict(m.scaling) for n, m in pm.named_modules() if isinstance(m, LoraLayer)}

def set_dose(adapter: str, lam: float):
    for n, m in pm.named_modules():
        if isinstance(m, LoraLayer) and adapter in m.scaling:
            m.scaling[adapter] = base_scaling[n][adapter] * lam

set_dose(ADAPTER, 0.5)

The evaluation above was run at λ = 1.0. On a related sweep, ECAPA speaker similarity to a reference clip fell 0.62 → 0.57 → 0.50 → −0.03 at λ = 0 / 0.5 / 1.0 / 1.5, against a 0.105 unrelated-speaker floor — so if you are voice-cloning from a reference, keep λ ≤ 0.5. For a generic commentator voice with no reference to preserve, λ = 1.0 is what was measured.

Stacking

Swapping the active adapter costs ~0.021 s across 268 modules, so combining is cheap:

pm.load_adapter("TTS-AGI/moss-emotion-loras-v3", subfolder="Anger", adapter_name="Anger")
pm.base_model.set_adapter([ADAPTER, "Anger"])
set_dose(ADAPTER, 0.8); set_dose("Anger", 0.3)

Untested combination — the evaluation here covers the sports adapter alone.

Traps

• audio_codes_list on the output side already holds a decoded waveform, not codes. Decoding it again yields exactly 0.16 s of pad per clip — a silent failure that still writes plausible-looking WAV files.
• Pass the sampling parameters explicitly. generate()'s continuation test runs on the text channel; a hot text_temperature collapses every take to ~0.16 s of pad.
• audio_lm_heads.* / text_lm_head.weight reported MISSING at load is benign — those heads are weight-tied. Do not "fix" it, and never call initialize_local_text_lm_head_from_text_lm_head().

---

Training

Data — 1,288 clips (4.08 h), source-balanced:

source	clips	hours	what
generated	820	2.51	English MOSS generations over 50 scripted moments × 40 sports × 5 prompt patterns, filtered by a listening judge
mined	468	1.57	real German sports commentary segments from the ARD/ZDF Mediathek corpus, 12 sports

The sampler draws each source with equal probability per epoch, so neither the synthetic English half nor the real German half can dominate the adapter. This is also why the adapters carry a German accent-flavoured energy that the pure-English base does not.

Hyperparameters: rank 16/32/64, alpha = 2 × rank, lora_dropout = 0.05, lr 2e-4 with linear decay, 8 epochs, bf16. Targets are the global q/k/v/o/gate/up/down projections, the local decoder's c_attn/c_proj/fc_in/fc_out, and all 12 audio_lm_heads — the audio heads matter; adapting attention alone moves the voice much less. All three ranks trained on identical batches against one shared frozen base, so the rank comparison is paired.

The data filter is the solid result from this line of work

Stronger than anything about the adapters themselves:

The literal WER × quality selection formula put 602 of 1,000 candidates at exactly score 0 (60 % had WER 0.00) and therefore selected the half with worse transcription. Replacing it with a listening filter kept 820 of 1,000 at the top rating, and a blind judge scored the corrected selection higher on 3 of 5 dimensions:

dimension	Δ	p
commentator-likeness	+0.43	0.024
euphoria	+0.63	0.003
broadcast-likeness	+0.53	0.009

That result is properly powered and it stands. Never divide quality by (1 + WER) — most candidates have a negative core score, so the division form increases the reward as transcription gets worse.

---

Contents

r64_e1/   adapter_config.json + adapter_model.safetensors   (rank 64, epoch 1)
r64_e8/   adapter_config.json + adapter_model.safetensors   (rank 64, epoch 8)
r32_e1/   adapter_config.json + adapter_model.safetensors   (rank 32, epoch 1)
samples/  12 MP3s: 3 held-out prompts × {base, r64_e1, r64_e8, r32_e1}
eval_prompts.json   the 16 held-out evaluation prompts, ready for build_user_message
eval_results.csv    the full 13-cell table with permutation p-values

Caveats

• Judged by a model (gemini-3-flash), not by human raters.
• The evaluation is English-only, 16 prompts × 2 takes per cell. Training was mixed English/German; German output was not separately evaluated.
• The mined half is real broadcast audio; the generated half is synthetic. Anything the adapters reproduce about crowd noise or broadcast character comes from the mined half.
• No claim is made that these adapters improve on the base model at p < 0.0042. See the top of this card.

Provenance

Trained by LAION as part of the MOSS voice-acting line. Full experimental record: LAION-AI/laion-moss-local-1.5-voice-acting-4.55b.