Stage-2 refine removes subtle transient motion (e.g. eye blinks)
Summary:
In the 3-step Regenerate refine, subtle, low-amplitude motion that is present in the draft disappears or is strongly dampened in the output. Examples: eye blinks removed, a cat's yawn not performed, water movement reduced to almost nothing, and other small motions similarly lost.
Setup (main reproduction recipe):
- Base: minimax_h3_ref2va_pruned_int8_convrot (stock pruned ref2va)
- Pack: rdh9616/ComfyUI-Open-H3-Regenerate (also reproduced with the Songssx ComfyUI-OpenH3-Regenerate2K pack)
- LoRA: open_h3_regenerate_2k_lora.safetensors (official merged, rank 576) + sharp_curve_branches.safetensors
- Refine: 3 Euler steps, sigmas 0.909375 → 0.857 → 0.667 → 0, CFG 1, LoRA strength 1.0, curve strength 1.0
- Draft: 56-frame clip, 2.33 s, latent upscaled to 1440x2560 (9:16) with the LBH 3D latent upscaler
- ComfyUI 0.43.2, Windows, RTX 3090 Ti (24 GB)
Observations:
- Full recipe (merged LoRA + curve branches) → subtle motion lost: blinks removed, a cat's yawn not performed, water movement reduced to almost nothing, and other low-amplitude motions similarly dampened.
- Same draft, latent upscale + decode WITHOUT the Regenerate LoRA → motion preserved (correct).
- Tried several component combinations (refiner-only via the Kijai rank-384 extraction, curves-only, full recipe minus the LightX2V Turbo) → the issue persists.
Expected:
The refine should preserve the draft's motion, including fast transient and low-amplitude motion such as blinks, yawns, and water movement, while adding 2K detail.
Actual:
Subtle motion is smoothed / redrawn away; the refined output favors the dominant static pose (eyes open, no yawn, near-still water).
Hypothesis (not verified):
At start_sigma 0.909 with only 3 steps, the refiner reconstructs from its prior and favors dominant static structure over few-frame and low-amplitude transients.
Question:
Is this a known limitation of the refine, and is there a recommended setting (lower start_sigma, different sigma schedule, or another refine mode) for clips with fast transient or low-amplitude motion? I can provide before/after frames of the draft and the refined output on request.
Update 2 — full official recipe on Singularity V2 int8 + Flex FL2VA: subtle motion is largely preserved, but not completely.
With these settings, the subtle motions that were being removed (blinks, a cat's yawn, water movement) are fixed to a large extent, but not completely:
Base: Minimax-h3_Singularity_ref2va_v2_int8 (full, non-pruned)
Upstream stack: pika_h3_flex_fl2va_v1_bf16 at 1.0
rdh9616 node:
- open_h3_regenerate_2k_lora (official merged) at 1.0
- sharp_curve_branches at 1.0, curve table = minimax_h3_ref2va_pruned_int8_convrot
This revises my earlier update: the full official recipe (merged 1.0 + curves 1.0) preserves motion largely on this base, whereas on the stock pruned ref2va base without Flex it removed subtle motion. The common factor in both working combinations is Flex at 1.0 + curves at 1.0, while the failing combination had no Flex — which suggests Flex may be the protective factor, though the base also changed. Single-variable tests are still needed to separate the Flex effect from the base effect.
Following from the above, my suggestion is: instead of the LightX2V Turbo baked into the refine LoRA, use Flex fl2va and merge it for more precise results. In my tests, Flex ref2va preserves subtle motion more than Flex fl2va, at the cost of slightly lower quality, so a merged refine LoRA with Flex fl2va instead of LightX2V Turbo would be the better choice when precise subtle motion matters most.
Screenshot of the new working stack attached.