WaveCut/MOSS-Image-SFT-OrbitQuant-W4A8

MOSS-Image-SFT OrbitQuant W4A8

OrbitQuant 4-bit weights for MOSS-Image-SFT, the OpenMOSS text-to-image model that renders text from prompts in many languages. The transformer and the Qwen3.5 text encoder both run 8-bit activations over 4-bit weights. The 4-bit codes were chosen with GPTQ rounding against calibration prompts that are not in the evaluation set. The runtime uses the model's own pipeline (pipeline_moss_image.py, 50 steps, CFG 3) and the ComfyUI-OrbitQuant nodes. The bundle is 8.8 GB. The original is 27.2 GB.

On one RTX 5090 a 1024×1024 image takes 13.7 s with 12.3 GiB of device memory at the peak. With --offload the peak is 9.4 GiB and the image takes 16.4 s. The original in BF16 with model offload takes 55.8 s. The Performance section has the details.

Usage

Use Linux, Python 3.12 and an NVIDIA GPU with compute capability 8.0 or newer. The tested stack is Torch 2.12.1 with CUDA 13.0 on an RTX 5090 (driver 580.178.04).

python3.12 -m venv .venv && . .venv/bin/activate
pip install torch==2.12.1 torchvision==0.27.1 --index-url https://download.pytorch.org/whl/cu130
pip install huggingface_hub
hf download WaveCut/MOSS-Image-SFT-OrbitQuant-W4A8 --local-dir moss-orbitquant
cd moss-orbitquant
pip install -r requirements.txt

Text to image:

python run_moss.py --prompt 'A wide view of a refrigerator beside a kitchen counter with a fruit bowl. A small sticky note on the refrigerator door is handwritten in Chinese: "记得买牛奶和鸡蛋！"' --seed 42 --out fridge.png

Python, inside moss-orbitquant:

import torch
from moss_orbitquant import load_pipeline

pipe = load_pipeline(".")
image = pipe("A prompt ...", height=1024, width=1024, num_inference_steps=50, guidance_scale=3.0,
             uncond_skip_layers=[22], generator=torch.Generator("cpu").manual_seed(42)).images[0]
image.save("image.png")

load_pipeline returns the original MOSSImagePipeline; call it with the original arguments. Options of run_moss.py (load_pipeline takes the same settings as arguments):

• --offload: keep the transformer, the text encoder and the autoencoder in host memory. Each moves to the GPU for its own stage.
• --attention int8: INT8 Q·K attention. An image takes 12.2 s instead of 13.7 s, and it changes slightly.
• --width, --height (multiples of 16), --steps, --cfg, --seed, --negative-prompt.

Sampling defaults follow the original model: 50 Euler steps, CFG 3 with layer 22 skipped in the negative branch, shift 5. The user guide of the original has prompt and resolution advice.

ComfyUI

ComfyUI-OrbitQuant 0.8.0 lists this release as WaveCut/MOSS-Image-SFT-OrbitQuant-W4A8. Connect OrbitQuant Model Loader to OrbitQuant Generate Image and save the result with a standard save node. The generator runs in the Python environment that python_executable selects. Install the packages of the Usage section there.

• profile: default, model-offload or int8-attention.
• keep_loaded (on by default): the worker keeps the model for the next request. On the RTX 5090 the first request took 32.4 s with loading and the next request 15.1 s.

Through ComfyUI v0.41.2 the device peak was 12.8 GiB with default and 9.9 GiB with model-offload (38.4 s in a new worker), including the 1.0 GiB that the idle ComfyUI server holds. The images were pixel-identical to the output of run_moss.py.

Quality

All numbers compare against the original BF16 model with the same prompt, seed and sampler, at 1024×1024.

Measure	Result
Mean LPIPS to BF16, 33 images (lower is closer)	0.190
Same, 16 images of seed 20261009 (no calibration prompt at this seed)	0.205
Two BF16 attention backends against each other, 8 images	0.055
Blind checklist, 26 images with 3–7 checks each: OrbitQuant W4A8 / BF16	92 / 118 and 93 / 118 checks
Blind preference, 6 portraits and the refrigerator photo: W4A8 / tie / BF16	3 / 3 / 1

The 33 prompts are 14 tasks of the OpenRouter Image Benchmarks and two extra tasks (a long Russian text and attribute binding), each at two seeds, plus the refrigerator prompt of the original README. The checklist was judged blind: the two images of each prompt were shown in random order without labels. Most differences are details and layout. The largest loss is in the four-script sign: BF16 rendered the Korean and Arabic lines correctly at both seeds, and W4A8 broke the Arabic line at both seeds and the Korean line at one. Long English and Russian text came out as good as or better than BF16 in this set.

Performance

One RTX 5090 (575 W, driver 580.178.04), Torch 2.12.1 with CUDA 13.0. One process per configuration, batch size 1, 1024×1024, 50 steps, CFG 3. Time is the median of 3 runs after 1 warm-up run; for BF16 it is the mean of 2 runs after 1 warm-up run. It covers prompt encoding, sampling and VAE decoding, with CUDA synchronized at both ends. Device peak is the total from NVML at 5 ms intervals, including the CUDA context. Tensor peak is the PyTorch allocator peak. Host RAM is the peak resident set of the process, including loading.

Configuration	Time	Device peak	Tensor peak	Host RAM peak	Checkpoint
OrbitQuant W4A8	13.7 s	12.3 GiB	11.2 GiB	6.5 GiB	8.8 GB
OrbitQuant W4A8, INT8 attention	12.2 s	12.3 GiB	11.2 GiB	6.5 GiB	8.8 GB
OrbitQuant W4A8, offload	16.4 s	9.4 GiB	8.4 GiB	14.3 GiB	8.8 GB
BF16 (original), offload	55.8 s	16.8 GiB	15.7 GiB	31.0 GiB	27.2 GB

Sampling alone, with the prompt embeddings computed in advance: 13.5 s for W4A8 and 31.4 s for BF16, with device peaks of 9.7 and 16.9 GiB. The loader keeps the 2 GiB token-embedding table of the text encoder in host memory and enables expandable allocator segments when PYTORCH_CUDA_ALLOC_CONF is empty. Transformers prints that causal_conv1d and flash-linear-attention are missing; the text encoder then uses their reference PyTorch paths, which these measurements include. Other GPUs are untested.

Components

Component	Precision	Size
Transformer, 5.1B parameters	OrbitQuant 4-bit weights with GPTQ rounding and 8-bit activations for the 252 projections of the 42 joint blocks. BF16 input embedders, text refiner, time embedding, shared modulation and output head.	3.03 GB
Text encoder, Qwen3.5 language model	OrbitQuant 4-bit weights with GPTQ rounding and 8-bit activations for 200 projections. BF16 token embeddings, norms and the small gate projections in_proj_a and in_proj_b. The vision tower is not included: the pipeline encodes text only.	5.51 GB
Autoencoder (FLUX.2 small decoder)	FP32, unchanged	0.25 GB

With nearest rounding (plain OrbitQuant, no calibration) the 4-bit transformer gave a mean LPIPS of 0.371 on 8 prompts; GPTQ rounding lowered it to 0.209 with the same codebook, bit width and file format. GPTQ calibration: 10 prompts (8 OpenRouter tasks at seed 20261010 and 2 extra text prompts) at 12 of the 50 sampling steps for the transformer; 600 prompts (DrawBench, PartiPrompts, DiffusionDB and 40 multilingual prompts) for the text encoder. transformer/gptq-info.json and text_encoder/gptq-info.json list the error of each projection group before and after GPTQ rounding.

Files

• transformer/: the packed transformer (model.safetensors), its codebooks, rotations, OrbitQuant configuration and manifest, and the original config.json.
• text_encoder/: the quantized Qwen3.5 language model. tokenizer/, scheduler/, vae/: the files of the original model.
• pipeline_moss_image.py: the original pipeline. moss_orbitquant.py: the loader. run_moss.py: the command-line entry point. The fused transformer blocks come from orbitquant 0.14.
• manifest.json: the components and the runtime pins. SHA256SUMS: the checksums of all files.

Limits and license

• The fused blocks run one sample at a time, as the original transformer does.
• Text in rarer scripts (Arabic and Korean in the evaluation) fails more often than in BF16. Small text, counts and negation can fail, as in the original model.

This release is a quantized version of MOSS-Image-SFT by OpenMOSS under the Apache License 2.0 (see LICENSE). OpenMOSS did not create, endorse or validate this release.