infosave

/

MiniMax-H3-Turbo-cmf

like 10

Two ways, and the first one shipped today in 0.5.92.

1. At runtime, no repack. cortiq animate now takes --lora, the way ltx-video already did:

cortiq animate mmh3-turbo-clipproj4b-fl2va-v2-q4tp.cmf \
  --prompt "<your trigger word> a woman in a red raincoat, close-up" \
  --lora your-likeness.safetensors --lora-strength 0.8 \
  --first-frame face.ppm --out take.avi

It reads the file as it ships — diffusion_model.… (ComfyUI single-file), base_model.model.dit.… (PEFT) or the bare module path; F32/F16/BF16; lora_A/lora_B or lora_down/lora_up. It binds attn.qkv_proj, attn.out_proj, mlp.fc1 and mlp.fc2 on all fifty blocks and both token-refiner blocks, and prints what it bound. On fal's Realism adapter:

lora: rank 32, 104/104 branches bound

Anything it cannot bind it names rather than dropping in silence. There is one real gap, worth knowing before you train: adaln_proj.linear. This container stores the modulation as a rank-24 curve over the timestep — that weight is [96768, 2688] per block in the release, 40% of the model, and collapsing it is most of why the file is 14 GB instead of 60. Folding an adaLN update into the curve is exact arithmetic, but it needs the time embedding, which only the packer has. So a LoRA that trains attention and MLP (the usual likeness recipe, and what the fal adapter does) applies fully at runtime; one that also trains adaLN applies partially and says so.

2. Baked in, for a permanent file — this is how the Turbo LoRA itself got into these containers:

cortiq animate-pack --dit minimax_h3_fl2va_pruned_bf16.safetensors \
  --lora your-likeness.safetensors --lora-scale 1.0 \
  --time-embedder h3_silu_temb_grid.safetensors \
  --te … --video-vae … --audio-vae … --tokenizer … --out mine.cmf

The merge happens while the source is still bf16, which is why baking needs the original checkpoint: the packed weights are q4tp, and a rank-32 update cannot be folded into a four-bit ladder without dequantizing the whole DiT.

What it costs, measured on an M4 (24 GB), 512×288, 22 frames: 33.7 s a step with the adapter against 29.8 without — and the branch's own arithmetic is half a per cent of the projection. The cost is the attention fusion standing down, because the fused kernel keeps exactly the panels a branch has to read. --lora-strength 0 reproduces the base render byte for byte, which is the gate on that path.

Samples from the run are in the repo — without and with fal's Realism adapter, same seed, same prompt.

For training the adapter itself, follow the ComfyUI recipe from that video — the output is a plain .safetensors and both paths above read it as it is.

Follow-up with a cleaner measurement, because the first one was taken on a machine that was swapping.

With memory freed and the two runs back to back — 512×288, 22 frames, four steps, M4 24 GB:

1.06× on the denoise, and the VAE moved 6.5% between the same two runs — so on this machine an adapter costs at or below the run-to-run noise. My earlier 1.13× and 1.37× came from the base render itself drifting from 22.2 to 32.5 s a step under 6 GB of swap.

Where weights stream off disk, the adapter's extra work hides inside the streaming. Where the DiT is fully resident, the attention fusion it stands down should show up — that number is still owed, and I will post it when a card with headroom runs it.

The latent upscaler is ported — 0.5.93, and it runs rather than being described:

cortiq animate model.cmf --prompt "…" --width 512 --height 288 \
  --upscale minimax_h3_latent_upscaler_3d_fp16.safetensors --upscale-by 2.0

Render small, the net resizes the latent, the VAE decodes once at the larger size — no decode → resize → encode round trip through the 5 B VAE. Parity against the node's own torch module on the same weights: worst 6.7e-6, relative rms 3.9e-7.

--lora from 0.5.92 is unchanged and still the way to attach a likeness adapter.