WaveCut/Vimoo-1.0-OrbitQuant-W4A8

Vimoo 1.0 — OrbitQuant W4A8

OrbitQuant 4-bit weights for Vimoo-1.0, the highest-quality Vimoo video model: two ≈11.7B transformers, one for text to video (T2V) and one for image to video (I2V), at 720p and 24 fps. Both transformers and the Qwen2.5-VL text encoder run 8-bit activations over 4-bit weights. The 4-bit codes were chosen with GPTQ rounding against calibration prompts that are not in the evaluation set. The runtime is the official Vimoo modular pipeline of the Vimoo-AI Diffusers fork with fused OrbitQuant kernels, and the ComfyUI-OrbitQuant nodes. The download is 24.9 GB. The original needs 64.3 GB (its two transformers, glyph encoder and the Qwen2.5-VL-7B checkpoint).

One RTX 5090 makes a 5-second 1280×720 video (121 frames, 50 steps) in about 50 min (31 min with INT8 attention) with 25.2 GiB of device memory at the peak. The BF16 original does not fit in the card's 32 GB, with or without model offload. On an RTX PRO 6000 the same video takes about 34 min instead of 94 min (2.8× faster). The Performance section has the details.

Examples

BF16 original on the left, this release on the right: the same prompt, seed and sampler, 848×480, 49 frames (2 s at 24 fps), 50 steps. The evaluation uses a smaller size than the 1280×720×121 default because the BF16 original takes about 94 minutes per default video on an RTX PRO 6000. None of these cases was used for calibration. The image-to-video first frames come from the Diffusers documentation images (huggingface/documentation-images).

Text to video, seed 201. A lighthouse on a rocky cliff during a storm at night. Huge waves crash against the rocks and spray rises into the wind. The lighthouse beam sweeps across the dark sea and rain. Lightning briefly lights the clouds. Slow push-in from the sea, dramatic cinematic lighting.

Text to video, seed 203. Close-up of a hand-painted wooden sign outside a cozy cafe that reads "FRESH COFFEE" in white letters. A barista opens the door and steam from a cup drifts past the sign. Morning light, shallow depth of field, gentle handheld movement.

Text to video, seed 204. A red fox trots through deep fresh snow in a quiet pine forest, stops, listens, and pounces headfirst into the snow. Snowflakes fall slowly. Telephoto tracking shot, soft overcast light, nature documentary style.

Image to video, first frame cat.png, seed 213. The black and white cat slowly lifts its head, its ears twitch, and it looks to the left and then back at the camera, blinking calmly. Its whiskers move slightly. Static camera, soft natural light.

Image to video, first frame astronaut.jpg, seed 211. The small astronaut hops forward across the lunar surface, kicking up fine grey dust that drifts slowly in low gravity. The astronaut turns its helmet toward the camera. The camera slowly tracks alongside.

Usage

Use Linux, Python 3.12 and an NVIDIA GPU. The tested stack is Torch 2.12.1 with CUDA 13.0 on an RTX 5090 and an RTX PRO 6000 (Blackwell).

python3.12 -m venv .venv && . .venv/bin/activate
pip install torch==2.12.1 torchvision==0.27.1 --index-url https://download.pytorch.org/whl/cu130
pip install huggingface_hub
hf download WaveCut/Vimoo-1.0-OrbitQuant-W4A8 --local-dir vimoo-orbitquant
cd vimoo-orbitquant
pip install -r requirements.txt

requirements.txt installs orbitquant 0.17.1 and the Vimoo-AI Diffusers fork at a pinned commit. On the first run the pipeline downloads the Qwen2.5-VL tokenizer, the Wan 2.2 autoencoder and, for image to video, the SigLIP image encoder from their upstream repositories, as the original does. It does not download the Qwen2.5-VL-7B weights: the quantized text encoder is in this repository.

Text to video:

python run_vimoo.py --prompt 'A lighthouse on a rocky cliff during a storm at dusk. Huge waves crash against the rocks and throw white spray into the air, the beam of the lighthouse sweeps through the rain, dark clouds roll across the sky. Wide shot, slow push-in.' --output lighthouse.mp4

Image to video (the image is the first frame):

python run_vimoo.py --image cat.png --prompt 'The cat slowly turns its head toward the camera and blinks. The camera remains fixed throughout.' --output cat.mp4

Python, inside vimoo-orbitquant:

import torch
from vimoo_orbitquant import load_pipeline

pipe = load_pipeline(".", task="t2v")  # task="i2v" for image to video, with reference_image=PIL image
video = pipe(prompt="A prompt ...", **pipe.vimoo_defaults, generator=torch.Generator("cpu").manual_seed(1234),
             output_type="np", output="videos")[0]

load_pipeline returns the official Vimoo ModularPipeline with the W4A8 transformer and text encoder; pipe.vimoo_defaults holds the settings of the official examples. Options of run_vimoo.py (load_pipeline takes the same settings as arguments):

• --offload: keep the models in host memory; each moves to the GPU for its own stage.
• --attention int8: INT8 Q·K attention. It is faster (text to video on an RTX 5090: 31 min instead of 50 min) and changes the video slightly; the distance to BF16 stays about the same (LPIPS 0.375 with INT8 attention, 0.377 without; see Quality). It needs orbitquant 0.17.1 or later.
• --vae-tiling auto|on|off: decode the video in tiles. auto (the default) tiles on GPUs with less than 60 GiB: the Wan 2.2 decode of a 1280×720×121 video does not fit next to the models in 32 GB.
• --width, --height, --frames (4k+1), --fps, --steps, --seed, --negative-prompt.

Sampling defaults follow the official Vimoo-1.0 examples: 1280×720, 121 frames at 24 fps, 50 steps; text to video with flow shift 12 and CFG 4.5 (rescale 1.2), image to video with flow shift 6 and CFG 3 (rescale 0.5). Image to video loads the separate transformer_i2v. The original prompting guide applies: detailed English prompts work best.

ComfyUI

ComfyUI-OrbitQuant 0.10.0 or later lists this release as Vimoo-1.0. Connect OrbitQuant Model Loader to OrbitQuant Generate Model Video and save the result with the standard Save Video node. Connect one image to reference_images for image to video. The generator runs in the Python environment that python_executable selects. Install the packages of the Usage section there.

• profile: default, model-offload or int8-attention.
• keep_loaded (off by default for video): the worker keeps the model for the next request.

Checked with ComfyUI v0.44.0 on one RTX 5090 (Torch 2.12.1, CUDA 13.0, orbitquant 0.17.1) through the queue and the standard Save Video node, at 848×480 with 49 frames and 50 steps (the default 1280×720×121 takes about 50 minutes per video on this GPU, see Performance). Device memory is the NVML total and includes the 1.0 GiB that the idle ComfyUI server holds. Times are per request and include loading when the request starts a worker or changes the kept model.

Request	Profile, worker	Device memory peak	Time
Text to video, first request	default, kept	20.3 GiB	150 s
Image to video, model change	default, kept	21.0 GiB	154 s

The image-to-video request replaced the kept text-to-video transformer with transformer_i2v in the same worker. The MP4 of the first request was byte-identical to the output of run_vimoo.py for the same prompt and seed.

Quality

Evaluation: 5 text-to-video prompts and 3 image-to-video cases that are not in the calibration set, at 848×480 with 49 frames and 50 steps, with the same prompt, seed and sampler as the BF16 original. At the default 1280×720×121 the BF16 original takes about 94 minutes per video on an RTX PRO 6000. LPIPS (AlexNet) and PSNR are computed on every 4th frame against the BF16 video.

Variant	Text to video: LPIPS / PSNR	Image to video: LPIPS / PSNR
OrbitQuant W4A8 (this release)	0.377 / 17.5 dB	0.094 / 34.9 dB
OrbitQuant W4A8, INT8 attention	0.375 / 17.5 dB	—

Text to video drifts more than image to video. Over 50 steps a small difference changes the path of the motion: the W4A8 videos keep the scene, the subjects, the sign text and the camera move of the BF16 videos, but the motion takes another path, and LPIPS counts that as a difference. In image to video the first frame anchors the video, and the W4A8 videos stay close to BF16 (35 dB). Frame sheets (start, 1/3, 2/3 and end of every video): text to video, image to video. On the cafe sign BF16 writes "COFEE" and W4A8 writes "COFFEE", as the prompt asks; with INT8 attention it is "CO FEE". The text encoder is the same 4-bit Qwen2.5-VL text tower as in Vimoo-1.0-Lite-OrbitQuant-W4A8, where it was compared with the original BF16 text encoder: the distance to BF16 was the same or smaller with the 4-bit one.

Performance

One video per configuration: 1280×720, 121 frames, the official sampler settings; text to video with the lighthouse prompt of the examples, image to video with the astronaut.jpg case. Each configuration runs in its own process: it loads the pipeline, makes a two-step warm-up video of the same shape and then the timed video. The time covers prompt encoding, sampling and VAE decoding, not loading. Device peak is the NVML total (CUDA context included) sampled every 5 ms during the timed video. Host RAM peak is the maximum resident set of the process, loading included. Torch 2.12.1, CUDA 13.0, orbitquant 0.17.1; the W4A8 rows load the release through load_pipeline.

The Vimoo-1.0 configurations ran 6 timed steps after the warm-up instead of 50; a 50-step BF16 video takes about 94 minutes on the RTX PRO 6000. The tables extrapolate to 50 steps: the measured time plus 44 times the measured time per step (prompt encoding and VAE decoding do not depend on the step count). Time per step on the RTX PRO 6000: BF16 112.4 s, W4A8 40.5 s, W4A8 with INT8 attention 33.4 s; on the RTX 5090: W4A8 59.1 s, with INT8 attention 36.7 s. At 1280×720×121 the transformer attends over about 112k video tokens, so attention takes most of a step. 4-bit weights do not shorten attention; INT8 attention does. In a separate measurement on the RTX 5090 one attention call over 112k tokens took 0.67 s with cuDNN and 0.40 s with OrbitQuant INT8 attention.

RTX PRO 6000 Blackwell (96 GB)

Configuration	Text to video	Device peak	Image to video	Device peak	Host RAM peak
BF16 (original)	93.9 min	69.8 GiB	93.9 min	70.5 GiB	41.2 GiB
BF16 (original), model offload	95.5 min	53.2 GiB	94.6 min	53.1 GiB	51.9 GiB
OrbitQuant W4A8	34.0 min	40.9 GiB	34.4 min	40.6 GiB	13.4 GiB
OrbitQuant W4A8, INT8 attention	28.1 min	40.9 GiB	—	—	11.8 GiB
OrbitQuant W4A8, model offload	34.5 min	26.0 GiB	34.7 min	27.2 GiB	26.1 GiB

RTX 5090 (32 GB)

Configuration	Text to video	Device peak	Image to video	Device peak	Host RAM peak
BF16 (original)	out of memory	—	out of memory	—	—
BF16 (original), model offload	out of memory	—	out of memory	—	—
OrbitQuant W4A8	49.7 min	25.2 GiB	50.3 min	25.7 GiB	13.4 GiB
OrbitQuant W4A8, INT8 attention	31.1 min	26.4 GiB	—	—	11.8 GiB
OrbitQuant W4A8, model offload	50.2 min	17.8 GiB	50.6 min	17.7 GiB	29.2 GiB

The RTX 5090 decodes the video in tiles (--vae-tiling auto); the RTX PRO 6000 decodes it whole, which raises its device peak. The BF16 original runs out of memory on the RTX 5090, resident and with the official model offload. INT8 attention was measured for text to video only.

Components

Component	Precision	Size
Text-to-video transformer, ≈11.7B parameters	OrbitQuant 4-bit weights with GPTQ rounding and 8-bit activations for the 480 attention and feed-forward projections of the 40 dual-stream blocks. The patch and condition embedders, time and low-rank modulation projections, token refiner and output head keep the original precision (BF16, and FP32 where the original stores FP32).	9.82 GB
Image-to-video transformer, ≈11.7B parameters	The same layout, quantized with its own image-to-video calibration.	9.82 GB
Text encoder, Qwen2.5-VL-7B text tower	OrbitQuant 4-bit weights with GPTQ rounding and 8-bit activations for the projections of layers 0–25. The pipeline reads the output of layer 25 (hidden_states[-3]), so the loader skips layers 26–27 and the final norm. BF16 token embeddings and norms.	4.36 GB
Glyph encoder (ByT5)	FP32, unchanged; runs on the CPU as in the original	0.88 GB
Autoencoder (Wan 2.2), SigLIP image encoder, tokenizers	Unchanged, downloaded from the upstream repositories	—

GPTQ calibration of each transformer: its inputs at 12 of the 50 steps of BF16 sampling at 848×480 with 33 frames, for 6 text-to-video prompts (transformer) and 9 image-to-video cases (transformer_i2v: 3 images, and the first frames of 6 calibration videos that Vimoo-1.0-Lite made from the text-to-video prompts, with those prompts). Text encoder: 600 prompts (the calibration prompts, the negative prompt and PartiPrompts), encoded as the pipeline encodes them. None of them is an evaluation case. With nearest rounding the mean relative output error of the 320 projection groups is 0.0091 for each transformer; GPTQ lowers it to 0.00063 (text to video) and 0.00064 (image to video) with the same codebook, bit width and file format (text encoder: 0.0059 → 0.00010). gptq-info.json in each component folder lists the error of each group. In the image-to-video transformer the feed-forward of the first block's video stream never fires: every GELU input is below −5.7, so the GELU output is exactly 0 in BF16 and the block adds only the output bias. GPTQ sets the 4-bit weights of that projection to 0, its report lists an error of 0, and the block output stays the same.

Files

• transformer/ (text to video) and transformer_i2v/ (image to video): the packed transformers (model.safetensors), their OrbitQuant configurations and manifests, the original config.json files and gptq-info.json (the error of each projection group before and after GPTQ rounding).
• text_encoder/: the quantized Qwen2.5-VL text tower and its gptq-info.json. text_encoder_2/, tokenizer_2/, scheduler/, scheduler_i2v/, guider/, guider_i2v/: the files of the original model.
• vimoo_orbitquant.py: the loader. run_vimoo.py: the command-line entry point. requirements.txt: the runtime pins. The fused transformer blocks come from orbitquant 0.17.
• modular_model_index.json, inference_defaults.json, glyph_source.json: the original pipeline description.
• manifest.json: the components, the source revision and the runtime pins. SHA256SUMS: the checksums of all files.
• assets/: the chart and videos of this card.

Limits and license

• The fused blocks run one sample at a time; the pipeline's classifier-free guidance batch runs as consecutive samples.
• Tested on Blackwell GPUs (RTX 5090, RTX PRO 6000) with Torch 2.12.1 and CUDA 13.0, at 1280×720×121 and 848×480×49. Other GPUs and sizes were not tested.
• Use orbitquant 0.17.1 or later (requirements.txt pins it). With 0.17.0 the INT8 attention overflowed FP16 on Vimoo-1.0-Lite and returned a black video.
• Text to video does not reproduce the BF16 motion frame by frame (see Quality).

This release is a quantized version of Vimoo-1.0 by Vimoo-AI under the Apache License 2.0 (see LICENSE and THIRD_PARTY_NOTICES.md). The text encoder is quantized from Qwen2.5-VL-7B-Instruct (Apache License 2.0). Vimoo-AI and the Qwen team did not create, endorse or validate this release.