netprojects/Cosmos3-Edge-Omni-mlx-bf16

Cosmos3-Edge video generator for Apple Silicon (MLX, bf16)

An MLX port of the video generator in nvidia/Cosmos3-Edge (revision 344d602). It turns a text prompt, optionally with a first frame, into a video (or a single image). This build keeps NVIDIA's bf16 weights unchanged (the timestep embedder in fp32), for the closest match to NVIDIA's output. Size: 7.6 GB.

Published by NetProjects (Internet Projects Ltd). Not affiliated with NVIDIA.

For text answers about images and video (the reasoner): Cosmos3-Edge-mlx-8bit · Cosmos3-Edge-mlx-6bit.

Build	Size	Latent cosine vs NVIDIA's pipeline (i2v / t2v)	Video PSNR (i2v / t2v)
netprojects/Cosmos3-Edge-Omni-mlx-8bit	5.0 GB	0.9960 / 0.9959	29.8 / 26.0 dB
netprojects/Cosmos3-Edge-Omni-mlx-bf16	7.6 GB	0.9975 / 0.9975	30.7 / 27.1 dB
fp32 (same code, for reference)	—	0.9998 / 0.9995	41.9 / 35.2 dB

Measured on a MacBook Pro M5 Max (128 GB), 7 November 2026, against NVIDIA's Cosmos3OmniPipeline in diffusers 0.42.1 (fp32), from the same initial noise: NVIDIA's image-to-video example image and prompts, 448×256, 17 frames, 20 steps, guidance 6, flow shift 12. The fp32 row shows the port follows NVIDIA's pipeline; the bf16 and 8-bit rows add the usual low-precision drift that builds up over 20 sampling steps. On small random-weight models the port matches diffusers to about 1e-5 for text-to-video, image-to-video and text-to-image.

With the 8-bit build, NVIDIA's full image-to-video example (832×480, 121 frames, 24 fps, 20 steps) takes 113 s end to end, with a peak of 18.8 GB.

Use

git clone https://github.com/internet-projects/cosmos3-edge-mlx && cd cosmos3-edge-mlx
uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -r requirements.txt
hf download netprojects/Cosmos3-Edge-Omni-mlx-bf16 --local-dir cosmos3-edge-omni-mlx-bf16

# image to video, NVIDIA's settings (832x480, 121 frames, 24 fps, 20 steps, guidance 6, flow shift 12)
.venv/bin/python -m cosmos3_edge_mlx.gen.cli --model cosmos3-edge-omni-mlx-bf16 --image first_frame.jpg \
    --prompt "A car drives along a coastal road ..." --out out.mp4

# text to video, and text to image (one frame, written as .png)
.venv/bin/python -m cosmos3_edge_mlx.gen.cli --model cosmos3-edge-omni-mlx-bf16 --prompt "..." --frames 61 --out clip.mp4
.venv/bin/python -m cosmos3_edge_mlx.gen.cli --model cosmos3-edge-omni-mlx-bf16 --prompt "..." --frames 1 --out image.png

--prompt-file and --negative-file take NVIDIA's JSON prompts (such as example_i2v_prompt.json and negative_prompt.json from NVIDIA's repo) or plain text. Frame counts are 4n+1; height and width are multiples of 16. --steps, --guidance, --shift, --seed and --fps override the defaults. From Python:

from cosmos3_edge_mlx.gen.pipeline import Cosmos3EdgeGenerator, save_video
gen = Cosmos3EdgeGenerator.from_pretrained("cosmos3-edge-omni-mlx-bf16")
res = gen("A robot arm stacks two cubes.", image="first_frame.jpg", num_frames=121, height=480, width=832)
save_video(res.video, "out.mp4", fps=24)

Memory: the command line caps the MLX buffer cache (COSMOS_CACHE_LIMIT_GB, default 2), clears it every step, and COSMOS_MEMORY_LIMIT_GB (default 32) sets a memory limit. NVIDIA's model card asks users to add their own safety guardrails for their use case.

How it works

• Transformer: NVIDIA's 28-layer Cosmos3 Omni transformer (hidden 2048, 16 query / 8 key-value heads, interleaved 3D M-RoPE). Each layer has a text stream (the reasoner's weights) and a generation stream with its own attention and MLP; video tokens attend to all text tokens and to each other. The text keys and values are computed once per prompt and reused at every step.
• VAE: the Wan 2.2 video VAE shipped in NVIDIA's repo (16× spatial, 4× temporal, 48 latent channels), in bf16, decoding one latent frame at a time.
• Sampler: UniPC (flow prediction, order 2) with NVIDIA's native flow schedule and classifier-free guidance.
• Image to video: the first frame is encoded by the VAE and held fixed as the first latent frame.
• bf16: the weights are NVIDIA's own, in MLX's file layout; the reasoner-only lm_head is left out.

Build and check it yourself

hf download nvidia/Cosmos3-Edge --revision 344d602b128d1bbdacb43b08d0a3626f46343e29 --local-dir cosmos3-edge-hf \
  --include "*.json" --include "*.md" --include "LICENSE*" --include "NOTICE*" --include "transformer/*" \
  --include "vae/*" --include "scheduler/*" --include "text_tokenizer/*" --include "assets/*"
.venv/bin/python convert_gen.py --src cosmos3-edge-hf --dst cosmos3-edge-omni-mlx-bf16
.venv/bin/python -m pytest -q tests/test_gen_*.py      # tiny random models vs diffusers, no real weights
.venv/bin/python scripts/gen_parity.py ref --snapshot cosmos3-edge-hf --out gen-ref --dtype float32   # diffusers, torch
.venv/bin/python scripts/gen_parity.py mlx --ref gen-ref --snapshot cosmos3-edge-hf --model cosmos3-edge-omni-mlx-bf16 --out parity.json

License

Cosmos3-Edge is released by NVIDIA under the OpenMDW-1.1 license, and this conversion keeps it: LICENSE is the OpenMDW-1.1 text and NOTICE describes the conversion. NVIDIA's model documents (BIAS.md, EXPLAINABILITY.md, PRIVACY.md, SAFETY.md) are included unchanged.

Credits: NVIDIA (Cosmos3-Edge), Wan team (Wan 2.2 VAE), Hugging Face diffusers (reference pipeline), Apple MLX. Port by Paris Michalas, Partner at NetProjects. Not affiliated with NVIDIA.