Lightricks

/

LTX-2.5

like 1.94k

Follow

LTX.io 4.82k

I’ve got custom audio lip-sync working in LTX 2.5 / ComfyUI. The fix turned out to be simpler than expected.

The important part is: don’t use the custom audio as LipDub/reference conditioning. Use it as a frozen audio latent.

This is the setup that worked for me:

1. Load your custom WAV/MP3

Use:

Load Audio

Then trim it so it matches the video duration:

Load Audio → Trim Audio Duration

2. Encode the actual audio with the LTX 2.5 Audio VAE

Use:

Trim Audio Duration → LTXV Audio VAE Encode

with:

ltx-2.5-audio-vae-bf16.safetensors

3. Freeze the audio latent

Feed the encoded audio into:

LTXVSetAudioRefTokens

BUT — this is important — I am NOT using the positive/negative reference-conditioning outputs from this node.

I am only using its:

frozen_audio

output.

That output has a zero noise mask, so the sampler is not allowed to rewrite or regenerate the supplied speech.

So effectively:

Custom Audio → Audio VAE Encode → LTXVSetAudioRefTokens → frozen_audio

4. Feed the SAME frozen audio into BOTH video-generation stages

The frozen audio goes into the audio input of LTXVConcatAVLatent.

For stage 1:

Video Latent + frozen_audio → LTXVConcatAVLatent → Sampler

Then use the same frozen_audio again when the upscaled video latent goes into stage 2:

Upscaled Video Latent + frozen_audio → LTXVConcatAVLatent → Stage 2 Sampler

This seems to be one of the critical parts.

The audio is never generated again between stages.

5. Use plain Euler — NOT Euler Ancestral

Both samplers are:

euler

I originally had:

euler_ancestral

That appears to introduce too much randomness for accurate mouth shapes and timing.

So:

Stage 1 = euler

Stage 2 = euler

6. Turn OFF Modality Guidance

This surprised me.

I originally added LTXVModalityGuidance at around 3.0 because it sounded ideal for lip-sync.

It actually made things worse.

In the working version it is completely bypassed.

So there is no extra A/V modality-guidance pass.

7. Do NOT use the reference-audio conditioning

Another mistake I made was feeding the outputs of LTXVSetAudioRefTokens back into the positive/negative conditioning.

That node is useful for things like audio reference / speaker identity, but that is not what we want here.

For custom dialogue I keep the normal LTX positive/negative conditioning.

I use LTXVSetAudioRefTokens ONLY because its frozen_audio output gives me the properly masked audio latent.

8. Disable LoRAs while testing

This was also important.

I disabled all:

• character LoRAs
• voice LoRAs
• audio LoRAs
• A2V/V2A LoRA influence

Especially old LTX 2.3 LoRAs running on LTX 2.5.

Once basic custom-audio lip-sync works, LoRAs can be introduced again carefully.

For diagnosing it, run completely clean first.

9. Keep Dual CFG simple

My working test uses:

LTXV Dual CFG Guider

with:

video CFG = 1

audio CFG = 1

No aggressive audio CFG or weird guidance values.

10. Put the ORIGINAL audio back onto the final video

Do not use the VAE-decoded sampled audio as your final soundtrack.

The VAE-encoded version exists purely to tell LTX what the mouth should be following.

For the final Create Video node I use the original trimmed WAV:

Trim Audio Duration → Create Video audio

So the complete working logic is basically:

WAV
↓
Trim to video duration
↓
LTX 2.5 Audio VAE Encode
↓
Freeze audio latent
↓
Concat with video latent
↓
Euler Stage 1
↓
Upscale video
↓
Concat SAME frozen audio again
↓
Euler Stage 2
↓
Decode video

Then:

Original trimmed WAV → final Create Video

The things I REMOVED/BYPASSED were just as important:

❌ Euler Ancestral
❌ Modality Guidance 3.0
❌ reference-audio positive/negative conditioning
❌ audio/voice LoRAs
❌ old LTX 2.3 character LoRAs during testing
❌ letting the sampler regenerate the custom audio

The thing that finally worked was:

Keep the supplied audio completely frozen and let LTX 2.5 generate the VIDEO around the audio rather than trying to generate or “improve” the audio along with the video.

That is the first LTX 2.5 custom-audio setup I’ve tested where the lip-sync actually starts behaving properly.

gracias por la explicacion, pero me pierdo. podrias dejar el workflow directamente? gracias!

We're working on adapting the dubbing pipeline to work with LTX 2.5 — it's currently still optimized for LTX 2.3. Support should be available very soon.

LTX 2.5 does lip-sync!

After multi-day testing, I can share what worked and what the failure modes were. I was using RuneXX's workflows with LTX 2.5 distilled at cfg 1, First/Middle/Last pinned images, custom audio input, no modality guidance. The audio provided was music containing a lead vocal and instruments - and I got the character to sing every word.
I did not need stem separation or mel-band isolation.

BUT it was very material-specific and LTX 2.5 seems to be much stricter than LTX 2.3 at deciding what to say/sing.
The same config that worked with one song was completely refused on another (the character just stood there and moved a little, as if listening to the music).
I think the model decided it was background music, not something it should be singing.
I also found that where there was a lot of wet reverb on the vocals, it was much more likely to be interpreted as backing singers - and only the dry lead vocals were mouthed.
Prompt coaching and explicit sync instructions in the prompt made no difference on refused material - the audio interpretation takes precedence.

However, the seed made a huge difference.
One seed gave near-perfect sync, while 2 other seeds gave partial sync with word drops.

Things I ruled out or convicted along the way:

Modality guidance was unusable at every strength I tested. It deformed the mouth and face and didn't help at any value I tried.
Sample rate of the audio didn't seem to make a difference (44.1k/48k Hz)

What worked: distilled model, cfg 1, having a forgiving middle image guide (0.3), finding a good seed.

hey, please help a guy out. can you email me the comfyui workflow. since you solved it. thank you!
coccihardy@gmail.com
or you can send it to this telegram
https://t.me/Tenfigur