Saidakmal/ltx-2.5-uzbek-speech-lora

LTX-2.5 Uzbek speech LoRA

A LoRA for LTX-2.5 (22B dev, audio + video) so that generated people speak natural, fluent Uzbek with a native accent and matching lip movement.

• uzbek_speech_lora.safetensors — the LoRA weights
• scripts/ — inference pipelines (voice only, voice continuation, LTX voice + MiniMax-H3 picture, MCP server)
• samples/ — before/after and pipeline outputs (see below)

Two ways to use it

	Video → audio	Audio → video
Order	picture first (MiniMax-H3 or any video), then this LoRA makes the Uzbek speech to the video	this LoRA makes the Uzbek speech first, then MiniMax-H3 draws the person to the speech
How	LTX retake over the clip with --keep-video (scripts/ltx_voice_continue.py)	LTX text-to-audio, then H3 Studio reference mode with the speech as an audio guide at 0 s (scripts/uz_speak.py, scripts/uz_dialogue.py)
Good for	real scenes: movement, several people in one frame, camera motion, dubbing a ready video	presenters, talking heads, interviews, long dialogues where each line is checked
Lip sync	speech is timed to the mouths already in the video	H3 animates lips to the given speech (envelope corr ~0.99, 0 ms shift)
Voices	distinct per speaker; one voice per person across scenes via a context clip of that person	one voice per speaker across the whole script (each line continues the speaker's previous one)
Samples	samples/video_to_audio/	samples/audio_to_video/
Measured	office scene: words exact, 2 speakers (H3's own audio: 1 voice for both)	3-line dialogue: all lines take 1, CER 0, diarization 2 speakers, reply on the right voice

Videos

Turn the sound on. (The repo is private: the players work while you are logged in to Hugging Face.)

Video → audio: speech made to a ready video

The same MiniMax-H3 office scene. Left: H3's own sound (one voice for both women). Right: this LoRA made the speech from the video (two distinct voices, words exact).

H3's own audio

this LoRA: audio from the video

Audio → video: speech first, H3 draws the person to it

A 3-line dialogue, each person keeps one voice:

LTX-2.5 before / after the LoRA (same prompt and seed)

base LTX-2.5

with the Uzbek LoRA

Use

Needs the LTX-2 repo (packages/ltx-pipelines) and the LTX-2.5 split weights: transformer ltx-2.5-22b-dev-transformer-bf16, text encoder gemma4-12b-with-proj-ltx-2.5-bf16, ltx-2.5-video-vae-bf16, ltx-2.5-audio-vae-bf16. Prompts work best in this style: A person ... speaks Uzbek in a natural, fluent voice with clear lip movements, saying: "<line>" The audio is clean Uzbek speech in a native accent. — the Uzbek line in quotes, Latin script, numbers written as words.

1. Uzbek voice only (text → audio, ~1 min per line)

python -m ltx_pipelines.t2a_one_stage \
  --transformer-path ltx-2.5-22b-dev-transformer-bf16.safetensors \
  --text-encoder-path gemma4-12b-with-proj-ltx-2.5-bf16.safetensors \
  --audio-vae-path ltx-2.5-audio-vae-bf16.safetensors \
  --lora uzbek_speech_lora.safetensors 1.0 \
  --num-frames 121 --frame-rate 24 --seed 42 \
  --audio-cfg-guidance-scale 7 --audio-stg-guidance-scale 1 \
  --prompt 'A young Uzbek woman ... speaks Uzbek in a natural, fluent voice with clear lip movements, saying: "Assalomu alaykum! Bugun sizlarga yangi loyihamiz haqida gapirib beraman." The audio is clean Uzbek speech in a native accent.' \
  --output-path line.wav

2. Audio for an existing video / continue a voice (scripts/ltx_voice_continue.py)

LTX retake with this LoRA: everything before --start (picture + sound) is kept as context, the window is regenerated. --keep-video freezes the picture and makes only the speech, timed to the lips in the video.

python scripts/ltx_voice_continue.py --video src.mp4 --start 5.0417 --end 10.0417 --keep-video \
  --prompt '... continues talking in the same voice as before, saying: "..." ...' --out out.mp4

Findings: the source must be 8k+1 frames, sides divisible by 32 (we use 768×448 / 448×768). It needs torch.inference_mode() (set in the script). Retake continues the last voice heard before the window: for a dialogue give each speaker their own previous line as context, and the context must end with a full sentence (otherwise LTX finishes that sentence and drops the start of the new line).

3. Audio → video: LTX voice + MiniMax-H3 picture (scripts/uz_speak.py, scripts/uz_dialogue.py)

H3 draws better faces and teeth; this LoRA gives the better Uzbek voice. The LTX speech goes into H3 Studio's reference mode as an audio guide at 0 s, with the speaker's keyframe as <Picture 1> and an image guide: H3 keeps the speech (loudness envelope correlation ~0.99, 0 ms shift) and animates the lips to it. uz_dialogue.py builds a whole script (speakers + ordered lines): per-speaker voice continuation, ASR check with re-takes, H3 per line with a hard-cut check, captions from ASR word timings, 16:9 or 9:16, calm / natural expression. mcp_uz_video.py exposes it as an MCP server (uz_make_video, uz_job_status, uz_redo, …).

4. Video → audio: picture first, speech made to it

For scenes with movement and several people in frame: H3 image-to-video with the dialogue in the prompt, then ltx_voice_continue.py --keep-video --start 0 over the whole clip makes the Uzbek speech to the picture. H3's own audio used one voice for both speakers; the LTX pass gave two distinct voices, words exact. See scripts/examples/social_protection_make.py and scripts/examples/court_make.py.

Samples

Folder	What
samples/ltx_lora/	LTX-2.5 text-to-video, same prompts and seed: base_prompt* (no LoRA) vs lora_prompt*
samples/video_to_audio/	the same H3 office scene: office_h3_own_audio (H3's sound, one voice for two people) vs office_ltx_audio_from_video (this LoRA, speech made to the video)
samples/audio_to_video/	MiniMax-H3 picture driven by this LoRA's speech (audio guide): prompt1-3, long_10s = one 10 s line
samples/voice_continuation/	retake continuing a voice: monologue with the video frozen / regenerated; a dialogue built per speaker
samples/videos/	full pipeline: a 3-line dialogue (16:9); a social-protection skit, talking heads (9:16)

ASR check (our recognizer) on the pipeline outputs: all lines word-exact or within CER 0.04.

Limits

• Men of similar age tend to get similar voices; describe voices distinctly or continue from a reference.
• Retake drops words when the context stops mid-sentence (see above); check every line with ASR on the whole text (a substring-fit CER hides a missing beginning).
• MiniMax-H3 clips longer than ~10 s may cut to another shot; keep lines short or anchor first and last frame.

Licences

• LoRA and outputs: derived from LTX-2.5 — LTX-2 Community License applies.
• Samples that use MiniMax-H3: MiniMax H3 Community License (display "MiniMax H3"; EU, UK, South Korea and USA excluded).
• Sample keyframes were made with Qwen-Image-2.1 (Qwen research licence, non-commercial); remake them with an Apache-2.0 image model before any commercial or official use.