stdstu123/LynnReal-Onmi-beta-0.1

LynnReal-Omni · Standard four-step + Flash three-step 🎬

One model for text-to-video, image-to-video, body- and hand-pose control, omni-reference generation, video editing, frame-by-frame repair and streaming long video — generated with four denoiser forwards per clip.

⚡🔥 The Flash three-step variant now runs in ComfyUI too: a five-second 1344×768 clip with native stereo audio in ~8.4 s on a single H100 — see ComfyUI!

Code · Paper · YouTube demo · Bilibili demo

🆕 Latest updates

Newest first — the full history is in CHANGELOG.md.

• 2026-09-19 · Standard Lite — exact four-step BF16 and INT8 checkpoints. The optional BF16 checkpoint is 37.6 GiB instead of 61.7 GiB, and the INT8 checkpoint is 20.4 GiB instead of 44.5 GiB. With the current node pack and the five *_4step_lite.json workflows, every sampler state and denoiser output from all four steps is bit-identical to the original checkpoint for t2v, i2v, r2v, pose2v and v2v (max |Δ| = 0, seed 970000). Keep Standard Lite at 4 steps; other step counts are not validated.
• 2026-09-19 · Flash Lite — the same three-step model in 16.7 GiB instead of 37.0 GiB. lynnreal_omni_flash_int8_lite.safetensors is an optional Flash checkpoint whose adaLN step table stores the exact modulation vectors the three-step schedule visits: −55% on disk, −20.4 GiB of VRAM, same speed, and the same frames as the original Flash at the same seed. Open one of the new *_lite.json workflows — no flags. Keep Flash Lite at 3 steps; other step counts are not validated.
• 2026-09-18 · ComfyUI node pack fix — Flash no longer crashes on DynamicVRAM machines. RuntimeError: aimdo memory compile error (torch ≥ 2.8 with a cu13x build, e.g. an RTX 5090) is fixed. Update comfyui/custom_nodes/ComfyUI-LynnReal and re-run — no launcher flags needed.
• 2026-09-17 · Flash three-step in the ComfyUI bundle. Three Flash workflows (text → video, first frame → video, references → video), the Light VAE loader and the INT8 switch node: 5 s at 1344×768 warm in ≈8.4 s on one H100 80 GB.
• 2026-09-16 · Standard four-step ComfyUI port. Five workflows (t2v, i2v, r2v, pose2v, v2v) with their demo inputs: 5 s at 1344×768 warm in ≈50 s on one H100 80 GB.
• 2026-09-15 · Initial beta. Diffusers-format weights, the ComfyUI bundle, demo videos and the paper link.

Demo

Compressed preview (≈10 MB, 960×540, audio included). Full-quality demo: YouTube · Bilibili · paper.

ComfyUI 🧩 — start here

The ComfyUI port runs these checkpoints out of the box, with no launcher flags — Standard four-step and Flash three-step. Everything it needs is in this repository: the workflows, the small custom node pack and the model files, all in ComfyUI format under comfyui/models/.

1. Copy comfyui/custom_nodes/ComfyUI-LynnReal into ComfyUI/custom_nodes/.
2. Copy the workflows below into ComfyUI/user/default/workflows/.
3. Copy the model files under comfyui/models/ into the matching ComfyUI/models/ folders (see the table further down for what each file is).
4. Open a workflow and press Run. On one H100 at 1344×768, a warm five-second video takes ≈50 s with the Standard 4-step model and ≈8.4 s with Flash.

Task	Workflow	Node pack	Demo input
Text → video + audio	t2v_lynnreal_4step.json	—	—
First frame → video	i2v_lynnreal_4step.json	—	transparent_rgb_gaming_mouse.png
Reference images → video	r2v_lynnreal_4step.json	—	red_superboy_on_city_roof.png, mecha_dragon_lightning.png
Pose control clip → video	pose2v_lynnreal_4step.json	required	pose_boxing_first.png, pose_boxing_control.mp4
Video continuation	v2v_lynnreal_4step.json	—	snowboard.mp4
⚡ Flash text → video + audio	t2v_lynnreal_flash_3_step.json	required	—
⚡ Flash first frame → video	ti2v_lynnreal_flash_3_step.json	required	beauty_first_frame.png
⚡ Flash references → video	ref2v_lynnreal_flash_3_step.json	required	beauty_reference_a.png, beauty_reference_b.png

The five Standard tasks also have matching *_4step_lite.json workflows. They require the current node pack and preserve the same BF16/INT8 switch while selecting the two Standard Lite checkpoints.

The demo inputs sit next to the workflows under comfyui/input/. comfyui/README.md documents every file each task loads, the optional INT8 switch and the VRAM behaviour; the workflows are also in the GitHub repository.

⚡🔥 Flash three-step is live, and it is fast!

The three Flash workflows run the W8A8 Flash DiT at its trained three steps with the Light A lighter variant of the same checkpoint, lynnreal_omni_flash_int8_lite.safetensors, cuts the weight file to 16.7 GiB from 37.0 GiB (−55%, −20.4 GiB of VRAM) at the same speed and the same frames at the same seed; load it with the *_lite.json workflows.

VAE! On one H100 80 GB at 1344×768, warm — model already loaded, the way a session runs — with three measured runs per cell:

Task	5 s · generate	5 s · click-to-video	10 s · generate	10 s · click-to-video
Text → video	8.4 s	12.2 s	22.0 s	29.1 s
First frame → video	8.9 s	13.0 s	23.1 s	30.1 s
References → video	9.5 s	13.2 s	24.2 s	31.1 s

A five-second 1344×768 clip with native stereo audio in about eight and a half seconds — three denoiser steps and both decoders included — on a single GPU. generate is the Generate-wall convention used throughout this project (first denoiser forward to decoded frames); click-to-video is what you actually wait for, prompt encoding and muxing included. Repeat runs agree to ±0.02 s!

Weights: comfyui/models/diffusion_models/lynnreal_omni_flash_int8.safetensors and comfyui/models/vae/lynnreal_omni_light_vae_fp16.safetensors.

Videos longer than 11 seconds are not usable in the ComfyUI Flash path yet — the accelerated path for long clips is still being fixed.

Nothing is assumed about the card it lands on: the node pack picks the fastest attention it can find and verifies it numerically (FlashAttention-3 → FA2 → cuDNN SDPA → native), falls back to ComfyUI's own block math when a fused kernel is unavailable, and tunes the INT8 GEMMs for the GPU it actually runs on!

The ComfyUI port is experimental and under active construction 🚧 — it runs the same checkpoints and schedules, but the pipeline around them is ComfyUI's. Same-seed output is not comparable between the two engines: compare quality, not pixel identity. The Flash three-step checkpoint and its Light VAE are already here, with the three workflows above; videos longer than 11 seconds are not usable in that path yet.

What the model does

LynnReal-Omni connects controllable visual workflows through one shared multimodal diffusion transformer (32B, following the MiniMax H3 architecture). A native packing layout distinguishes appearance references, frame-aligned controls and causal history, so a single checkpoint covers:

Generate	Control	Edit & repair
Text-to-video and first-frame conditioning	Single- and multiple-subject references	Instruction-guided image editing
Video continuation	Body- and hand-pose control	Video appearance editing
Speech and video generation	Game- and mesh-video rendering	General frame-by-frame video repair

Long video is generated in chunks that condition on the preceding context and emit 17 new frames per step, with a fixed head-overlap interface that bounds continuation memory. This release also contains the Standard DiT in ComfyUI format (comfyui/models/), including an optional W8A8 INT8 checkpoint built from the same weights.

On a single H100, warm generation and decoding of a 22-frame 540p clip takes 843 ms with the Standard model and 377 ms with Flash, using the original scripts; Flash, a 27B three-step variant with a lightweight VAE decoder, is the path toward real-time streaming.

Files in this repository

File	Size	Where it goes (ComfyUI)
comfyui/models/diffusion_models/lynnreal_omni_standard_bf16.safetensors	61.7 GiB	models/diffusion_models/
comfyui/models/diffusion_models/lynnreal_omni_standard_bf16_lite.safetensors	37.6 GiB	models/diffusion_models/ (optional Standard Lite)
comfyui/models/diffusion_models/lynnreal_omni_standard_int8.safetensors	44.5 GiB	models/diffusion_models/ (optional, INT8 switch)
comfyui/models/diffusion_models/lynnreal_omni_standard_int8_lite.safetensors	20.4 GiB	models/diffusion_models/ (optional Standard Lite)
comfyui/models/diffusion_models/lynnreal_omni_flash_int8.safetensors	37.0 GiB	models/diffusion_models/ (Flash workflows)
comfyui/models/diffusion_models/lynnreal_omni_flash_int8_lite.safetensors	16.7 GiB	models/diffusion_models/ (Flash workflows, optional — *_lite.json)
comfyui/models/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors	14.6 GiB	models/text_encoders/
comfyui/models/vae/minimax_h3_video_vae_fp16.safetensors	4.9 GiB	models/vae/
comfyui/models/vae/lynnreal_omni_light_vae_fp16.safetensors	3.6 GiB	models/vae/ (Flash workflows)
comfyui/models/vae/minimax_h3_audio_vae_fp32.safetensors	577 MiB	models/vae/
comfyui/models/embeddings/minimaxh3_art_is_explosion.safetensors	500 KiB	models/embeddings/
assets/lynnreal-demo-preview.mp4	9.4 MiB	— (demo preview)

The diffusers-format bundle (transformer shards, text encoder, VAE and schedulers) stays at the repository root and is what the script/sample/ launchers in the GitHub repository load.

More

• Setup, tasks and launchers (reference implementation): GitHub README
• ComfyUI bundle details: comfyui/README.md · end-to-end verification record
• Flash (three-step) weights: stdstu123/LynnReal-Onmi-flash-beta-0.1 · in ComfyUI format under comfyui/models/
• Lightweight VAE: stdstu123/LynnReal-Onmi-light-vae

This is an early beta: bugs, compatibility issues and inconsistent generation quality may remain. We plan to release the training code, part of the training data and a more efficient DiT. Issues and pull requests are welcome. 🙌

Acknowledgments

We thank the MiniMax H3 team for the model that forms the foundation of LynnReal-Omni, and Qwen3-VL for the multimodal encoder, tokenizer and processor implementation used for conditioning. Model licenses and upstream attribution are retained in the bundle; see LICENSE.