Rabe3/Saudi-tts-full-finetune-1

Saudi TTS — MOSS-TTS Local Transformer v1.5 (full finetune)

Full-parameter finetune of OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5 on Rabe3/saudi-tts-synthetic-200k for Saudi-dialect Arabic speech synthesis.

All 4.13B parameters were trained (no LoRA/adapters). Output is 48 kHz stereo.

Usage

Requires the MOSS-TTS runtime and the v2 audio tokenizer:

git clone https://github.com/OpenMOSS/MOSS-TTS.git && cd MOSS-TTS
pip install --extra-index-url https://download.pytorch.org/whl/cu128 -e ".[torch-runtime]"

import torch, torchaudio
from transformers import AutoModel, AutoProcessor

model_path = "Rabe3/Saudi-tts-full-finetune-1"
reference  = "reference.wav"   # a clip of the target voice

processor = AutoProcessor.from_pretrained(
    model_path, trust_remote_code=True,
    codec_weight_dtype="fp32", codec_compute_dtype="bf16",
)
processor.audio_tokenizer = processor.audio_tokenizer.to("cuda")

model = AutoModel.from_pretrained(
    model_path, trust_remote_code=True,
    dtype=torch.bfloat16, attn_implementation="sdpa",
).to("cuda").eval()

conversation = [[processor.build_user_message(
    text="أهلاً وسهلاً، كيف حالك اليوم؟",
    reference=[reference],
    language="Arabic",
)]]

batch = processor(conversation, mode="generation")
outputs = model.generate(
    input_ids=batch["input_ids"].to("cuda"),
    attention_mask=batch["attention_mask"].to("cuda"),
    max_new_tokens=4096,
    do_sample=True,
    audio_temperature=1.2,
    audio_top_p=1.0,
    audio_top_k=25,
)

audio = processor.decode(outputs)[0].audio_codes_list[0]
torchaudio.save("out.wav", audio, processor.model_config.sampling_rate)  # 48 kHz stereo

The voice is set by the reference clip at inference time, so you can steer the speaker without retraining.

Training

Base	OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5 (Qwen3-4B backbone, 12-layer RVQ)
Codec	OpenMOSS-Team/MOSS-Audio-Tokenizer-v2 (48 kHz, 12.5 Hz frames, 12 codebooks)
Data	199,473 clips / ~246 h, single speaker, Saudi Arabic (527 clips dropped as 0 s or > 20 s)
Hardware	2× H100 NVL, DDP
Precision	bf16
Epochs	1 (3,116 optimizer steps)
Global batch	64 (16 per device × 2 GPUs × 2 grad-accum)
Optimizer	AdamW, LR 2e-5, cosine schedule, 3% warmup, weight decay 0.1, adam_eps 1e-4
Loss	channelwise-loss-weight 1,32 (text head : total audio heads)
Other	Gradient checkpointing, max_grad_norm 1.0
Runtime	2h 04m

Training used precomputed audio codes; a single fixed reference clip was applied to every sample, matching the dataset's train_raw.jsonl recipe.

Results

Objective on identical data (the training loss above, re-scored on 256 samples):

Model	Loss
Base	4.913
This model	4.556

Notes

• audio_lm_heads.* / text_lm_head.weight report as MISSING on load. This is expected and harmless: those heads are weight-tied to audio_embeddings.* and transformer.embed_tokens, so they are intentionally not serialized and are re-tied at load time. The base model prints the same message.
• Training followed the upstream recipe, which keeps master weights and optimizer states in bf16. At LR 2e-5 a large share of updates fall below bf16 resolution, so the run is on the conservative side; fp32 master weights (via DeepSpeed ZeRO) would extract more from the same data.
• The training data is synthetic, single-speaker Saudi Arabic. Quality outside that dialect and domain is untested.

License

Apache-2.0, following the base model. Review the base model and dataset licenses before production use.