hamidfzm/MOSS-TTS-Realtime-Persian-lora

Persian LoRA for MOSS-TTS-Realtime

LoRA adapter that makes OpenMOSS-Team/MOSS-TTS-Realtime (1.7B, Apache 2.0, streaming-oriented) speak correct Persian. The base model lists fa in its language table but its Persian is not recognizably Persian; this adapter fixes pronunciation, ezafe voicing, and conversational register in the weights, so no grapheme-to-phoneme front-end is needed in the serving hot path.

Try it now, free: (demo.ipynb in this repo, runs on the free T4 tier, plays audio inline, lets you clone your own voice from a 10-30 s clip).

Browser demo Space: hamidfzm/persian-tts.

This checkpoint

• Training run: turn4-checkpoint-epoch-1
• LoRA rank 16, alpha 32, target modules: down_proj, gate_proj, up_proj
• Trained on an RTX 4070 Ti (12 GB), bf16, batch 1 with gradient accumulation

Training data

Dataset	License	Role
MahtaFetrat/Mana-TTS	CC0	read-speech backbone, ~64 h HIGH-quality, 24 kHz resampled
MahtaFetrat/GPTInformal-Persian	CC0	conversational register, oversampled
Thomcles/Persian-Farsi-Speech	CC BY 4.0	vocabulary breadth (runs 5+)

Records are reference-conditioned (same-speaker reference audio per utterance) to preserve the base model's zero-shot voice cloning.

Usage

The base model's inference code lives in the MOSS-TTS repository; the adapter loads on top with peft:

import sys
sys.path.insert(0, "MOSS-TTS/moss_tts_realtime")  # git clone https://github.com/OpenMOSS/MOSS-TTS

import torch
from peft import PeftModel
from transformers import AutoModel, AutoTokenizer
from inferencer import MossTTSRealtimeInference
from mossttsrealtime.modeling_mossttsrealtime import MossTTSRealtime

base = "OpenMOSS-Team/MOSS-TTS-Realtime"
model = MossTTSRealtime.from_pretrained(base, torch_dtype=torch.bfloat16, attn_implementation="sdpa")
model = PeftModel.from_pretrained(model, "hamidfzm/MOSS-TTS-Realtime-Persian-lora").merge_and_unload().to("cuda")
tokenizer = AutoTokenizer.from_pretrained(base)
codec = AutoModel.from_pretrained("OpenMOSS-Team/MOSS-Audio-Tokenizer", trust_remote_code=True).eval().to("cuda")
inferencer = MossTTSRealtimeInference(model, tokenizer, max_length=1200, codec=codec,
                                      codec_sample_rate=24000, codec_encode_kwargs={"chunk_duration": 8})
result = inferencer.generate(text=["، سلام، حال شما چطور است؟"],
                             reference_audio_path=["persian_reference.wav"],
                             temperature=0.8, top_p=0.6, top_k=30,
                             repetition_penalty=1.1, repetition_window=50, device="cuda")
tokens = torch.tensor(result[0]).to("cuda")
wav = codec.decode(tokens.permute(1, 0), chunk_duration=8)["audio"][0]

Serving notes learned by ear (all implemented in the demo Space, see its app.py):

• Always pass a Persian reference clip. The model is a zero-shot cloner; the output voice is the reference. A non-Persian reference degrades pronunciation.
• Prepend "، " to each utterance. The model clips the first word of a cold start; a leading comma-pause restores it. Long sentences are unaffected either way.
• Normalize digits to Persian number words before synthesis and chunk text to roughly 12 second utterances.
• Cap generation around 1200 tokens (~90 s) to stop runaway generations.

Known limits

• Homograph disambiguation is out of scope for the weights (مرد as mard vs mord needs sentence semantics); handle it upstream by injecting diacritics (مُرد).
• Code-switched English is intelligible but accented.
• English-only input shows no adapter drift versus the base model.