Concyclics/Cosmos3-Edge-NVFP4-AWQ

Cosmos3-Edge NVFP4 AWQ-lite (W4A16)

This is the frozen generation-side NVFP4 AWQ-lite weight overlay for NVIDIA Cosmos3-Edge, released by Concyclics as part of the ThunderWorld experiments. It contains real packed FP4 weights, E4M3 group scales, FP32 tensor scales, and BF16 activation prescales. It is not an NVIDIA release.

This repository contains the quantized portion, not a standalone full model. The included loader downloads the remaining components from the pinned official base checkpoint. Do not pass this repository directly to a standard DiffusionPipeline.from_pretrained or use an INT4 AWQ loader.

Quantization and scope

Item	Definition
Base	nvidia/Cosmos3-Edge
Base revision	a9d944e2c6a1bf9f48b92ad16348e70c5f1836ba
Calibration	Official ModelOpt NVFP4_AWQ_LITE_CFG, input/output quantizers disabled
Quantized modules	168 generation-side linear operators across all 28 layers
Per-layer scope	mlp_moe_gen.{up_proj,down_proj} and self_attn.{add_q_proj,add_k_proj,add_v_proj,to_add_out}
Weight representation	Packed E2M1 FP4; E4M3 scales per K16 group; FP32 global scale
Activations	BF16, multiplied by the calibrated channel prescale on the current input
Attention	BF16 native Torch flash attention
Execution	vLLM's native Marlin W4A16 kernel, FP32 reduction
Other parameters	Retained from the pinned base, including the AR tower and VAE
Residual repair	None; this is the AWQ comparison model, not ThunderWorld W4A4 repair

Calibration uses the existing Edge I2V development teacher panels (16 full-M panels per operator, spanning four denoising stages and both CFG branches). Weights were calibrated offline; inference performs no recalibration. The export retains the original quantizer scales and exactly recovers the frozen FP4 codes, including tie-boundary choices. It does not requantize a decoded BF16 checkpoint.

The packed tensor payload is 793,813,664 bytes (~0.794 GB). This is the quantized generation-side payload, not total model storage or total VRAM. Scales and prescales are included. The base model must also be downloaded and is initially loaded in BF16 before its 168 target modules are replaced, so peak loading memory exceeds the final packed model's resident memory.

Download and run

Use Linux, Python 3.12, and a CUDA-enabled PyTorch/vLLM installation compatible with your GPU. The release environment uses Torch 2.11.0 and vLLM 0.25.1; requirements.txt pins the Cosmos3-capable Diffusers commit. A generic stable Diffusers release may not yet expose Cosmos3OmniPipeline.

hf download Concyclics/Cosmos3-Edge-NVFP4-AWQ --local-dir Cosmos3-Edge-NVFP4-AWQ
cd Cosmos3-Edge-NVFP4-AWQ
python -m pip install -r requirements.txt
python infer.py --image /path/to/input.jpg \
  --prompt "A robot carefully picks up the object and places it on the table." \
  --output output.mp4

The first run downloads the pinned base components automatically. To use an existing local copy of that exact base, add --base-path /path/to/Cosmos3-Edge. The example uses 480x832, 121 frames, 20 denoising steps, CFG 6, flow shift 12, 24 FPS, and seed 0. These sampling settings follow the Edge model-card example; they are not a claim of reproducing any benchmark leaderboard. The prompt is used verbatim; no prompt expansion service is called.

Programmatic use after downloading the repository:

from awq_loader import load_pipeline
pipe = load_pipeline(".", device="cuda:0")
# Call pipe with Cosmos3OmniPipeline arguments as in infer.py.

Install the overlay before CUDA Graph capture. The portable example uses eager Diffusers inference, not the research runner's graph/AR-cache optimizations. Use the final GPU from the start. Device offloading, pipeline parallelism, training, and re-saving the mutated pipeline via save_pretrained are not supported by this loader. There is no automatic BF16 fallback if Marlin fails. The example disables the optional safety checker, matching the research path; see the upstream safety documentation before deployment.

Validation and limitations

• validation/export.json: all 168 Safetensors files retain the original tensor bytes. Independently decoded BF16 weights match the frozen AWQ reference in numerical value and prescales match exactly. Signed-zero differences between the decoder and reference are reported separately; they are not described as bitwise equality of decoded weights.
• validation/portable_loader.json: 72 operator checks on RTX PRO 6000 Blackwell, covering six operator types, M129/M416, with/without bias, changed/zero inputs, and eager/Graph equality. Maximum decoded-reference output NMSE was 1.334e-5 (screen threshold: 1e-4).
• validation/pipeline_smoke.json: complete base loading, replacement of all 168 modules, and a two-step 61-frame 192x320 generation through this portable Diffusers loader passed. This checks integration, not full-length quality.
• The source research adapter completed Edge I2V smoke generation on two scenes and two seeds per scene on RTX PRO 6000 Blackwell. This is limited smoke evidence, not a full benchmark qualification or a claim of matching BF16/FP8 quality.
• The loader uses fused FP4 weight dequantization with BF16 GEMM, not native FP4 activation Tensor Core computation. Neither 4x speedup nor superiority to BF16/FP8 is claimed. RTX 5090 performance is not established by this release.
• Calibration targets Edge I2V. Policy, reasoning, and other modalities have not been qualified for this quantized release.

The base license and provenance are retained in LICENSE, NOTICE, and upstream/. No benchmark images, calibration inputs, API credentials, videos, or private filesystem manifests are included.