akatz-ai/MiniMax-H3-Character-Swap-LoRA

MiniMax H3 Character Swap LoRA v1

An experimental character-replacement LoRA trained by Akatz Labs for 1,000 updates. It aims to replace a selected character using an image reference while preserving the source scene. In our local comparisons it often preserved the background more closely than the base model, but motion timing, facial expressions, and hard cuts remain unreliable.

Download: final 1,000-step LoRA. Only the final 1,000-step checkpoint is published. Start with the final checkpoint at strength 1.0. This is an adapter, not a standalone model or a Turbo distillation LoRA.

Dataset: H3 Character Swap v1.

Use

1. Use an H3 Ref2VA-capable runtime and obtain the base model and VAEs separately.
2. Place the final .safetensors file in ComfyUI/models/loras/ (or your configured shared LoRA directory), and apply it to the H3 model at strength 1.0 using a compatible model-only LoRA loader.
3. Supply the source video as <Video 1> and replacement image or character sheet as <Picture 1>.
4. Specify the target person in the prompt. No additional trigger word was trained.

Example:

Replace only the man in the purple shirt in <Video 1> with the character in <Picture 1>. Keep the replacement character's identity, outfit, and art style from <Picture 1>. Preserve the source video's camera, background, lighting, objects, and all other people. Match the target person's position, scale, pose, and movement. Do not show the reference sheet or its background.

The training captions were shorter, for example Swap the man in the purple shirt in <Video 1> with the character in <Picture 1>. Preservation instructions helped some local evaluations, but stronger expression instructions sometimes suppressed the swap entirely. Prompt wording is not a guarantee of strict source alignment.

Short, continuous shots of roughly 4–5 seconds were more promising than our full 14-second tests. A precise maximum duration has not been established. Use 24 fps and your runtime's supported H3 frame grid. The character-swap LoRA does not require a Turbo LoRA, Spectrum, or Sol attention.

Base model and tested configurations

Training used minimax_h3_ref2va_pruned_int8_convrot.safetensors from Comfy-Org/MiniMax-H3, plus the frozen Ostris Ref2VA training assistant. The assistant and base weights are not merged into this adapter and are not distributed here. Model revisions and hashes are in training/base-model-files.json.

We evaluated on the Ref2VA base and a local FL2VA/Ref2VA hybrid (blocks 25–49, INT8). A later experimental configuration combined this character-swap adapter with a separate 768p Turbo 8-step LoRA, res_multistep / simple, and native Sol attention. Those are evaluation choices, not the training base or universal compatibility claims. The local hybrid is not bundled. The text-to-video-only FastH3 experiment did not demonstrate a useful replacement workflow.

Training record

Setting	Recorded value
Updates / saves	1,000 / every 250 updates
Hardware	RunPod RTX PRO 4500 Blackwell, 32 GB
LoRA rank / alpha	16 / 16, excluding adaln_proj
Optimizer / learning rate	AdamW8bit / 5e-5
Batch / accumulation	1 / 1
Precision	BF16, convrot8 transformer, NVFP4 text encoder
Edit target resolution	Area budget 1024; 1344×768 buckets
Video regularization resolution	Reduced area budget 384
Regularization duration	73 frames at 24 fps, approximately 3.04 seconds
Memory measures	Gradient checkpointing, layer offload, cached latents/text, chunked MLP
Sampling during training	Disabled

The dataset has 94 synthetic image-edit triplets and 40 unchanged video/audio examples. Optimization used 76 edits and 32 regularization clips; 18 edits and 8 clips were held out. The edit targets are single still images, with five-frame static source-video controls. This is not training on long moving character-swap targets. AI Toolkit interleaves regularization, so file-count ratios are not update ratios. The data include cross-style swaps and varied character sheets, but only one-character replacement targets.

The author estimates the overnight rental at around $11; this is not a measured cost benchmark. training/train-1000-noeval.json records the run configuration, with inactive local evaluation paths removed. The launcher and package snapshot are supplied for reference. The copied AI Toolkit source was not a Git checkout, so the embedded version label is not an exact source revision. See the dataset's runtime-contract fingerprints.

Findings and limitations

• Local side-by-side reviews suggest improved scene/background preservation relative to the base model. These are qualitative observations, not a benchmark score.
• Long windows can drift in framing, placement, or source timing. Hard cuts can become zooms or gradual repositioning.
• Close-up facial expressions may not match the original performance; added expression prompting was not consistently helpful.
• Two-character inference was tested, but multi-character replacements were not supervised in the training targets.
• Short-window continuation improved some joins but did not guarantee camera-cut timing or full source adherence.
• Generated audio was closer in some early LoRA comparisons but skipped or drifted in the continuation test. The later review used original source audio copied onto the generated video. That audio preservation is postprocessing, not proof of the LoRA's audio fidelity; remuxing does not repair lip-sync drift.

Next experiments include aligned moving swap targets, explicit hard-cut examples, expression supervision, and longer sequences. More VRAM alone is not a demonstrated quality fix.

License and attribution

This adapter is distributed under the MiniMax H3 Community License Agreement, including its acceptable-use, distribution, commercial, and territorial provisions. It is not Apache-2.0. The agreement excludes the US, EU, UK, and Republic of Korea from its standard territorial grant and describes separate authorization. Read the complete upstream terms; this repository does not expand them. See NOTICE for attribution and the modification notice.

The separately published dataset uses Apache-2.0 for Akatz Labs contributions and preserves the upstream VidGen license/provenance. Its license does not replace this model's terms.