A2Vid two-stage (LTX-2.5): stage 2 distilled refine corrupts non-keyframe latents
#59
by bgyngell - opened
Setup
- Official A2VidPipelineTwoStage from LTX-2 (400fd31054597515f47125691032c04b1c3ee24e)
- Hub pack Lightricks/LTX-2.5: ltx-2.5-22b-dev-transformer-bf16 + Gemma TE + video/audio VAE + ltx-2.5-22b-distilled-lora-450-bf16 + spatial x2 upsampler
- Distilled LoRA on stage 2 only, LTXV_LORA_COMFY_RENAMING_MAP, strength 1.0
- 1536×1024, 24 fps, 30 steps, seed 10, detect_params video guider, DEFAULT_NEGATIVE_PROMPT
- Stereo speech WAV as A2Vid audio; original waveform muxed at the end (not VAE-decoded audio)
- Image conditioning: avatar still at frame 0, strength 1.0
- A100 80GB, offload_mode=cpu (LoRA fuse OOMs at ~80GB with offload_mode=none)
What we see
- Full two-stage: frame 0 is the pinned still. Later frames decode as a grid of coloured squares (not a face).
Latent dumps: - Stage 1 (1, 128, T, 16, 24): person visible on frame 0 and later frames
- After x2 upsample (1, 128, T, 32, 48): structure still looks like a person
- After stage 2: same spatial shape, but stats jump (~std 0.73 → 2.44, range ~±6 → ±14). Frame 0 still looks like a person; frame 1+ is salt-and-pepper. VAE decode of those frames is the colour mosaic.
- Mux/encode is not the cause: encoded MP4 stills match the bad decode.
- Skip stage 2 (decode the upsampled stage-1 latent with the same VAE): coherent talking-head clip, no lip motion. Read on this Stage 1 + upsample + decode look usable. Stage 2’s 3-step distilled Euler refine (STAGE_2_DISTILLED_SIGMAS + SimpleDenoiser + distilled LoRA) is what blows up every latent except the strength-1.0 keyframe.
The model files were downloaded directly from HF. I checked that the deployed files match the size of the files on HF.
Has anyone else had this issue? Can anyone help me figure out what is going wrong?