NVIDIA-Nemotron-3-Nano-30B-A3B

BaseRT .base builds of nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 for fast local inference on Apple Silicon (Metal).

A 31.6B-parameter hybrid Mamba-2 / attention mixture-of-experts model (~3.5B active parameters per token: 128 routed experts, 6 active, plus one always-on shared expert) with a 256K context window. Every build here is converted directly from the bf16 checkpoint — never from another quantization.

Files

File	Precision	Size
NVIDIA-Nemotron-3-Nano-30B-A3B-BF16.base	bf16, as published	63.2 GB
NVIDIA-Nemotron-3-Nano-30B-A3B-Q8.base	8-bit weights (RTN from bf16), f16 norms	33.0 GB
NVIDIA-Nemotron-3-Nano-30B-A3B-Q4.base	4-bit weights (RTN from bf16), f16 norms	17.8 GB

BF16 is the checkpoint as published and is the reference build. Q8 and Q4 are round-to-nearest quantizations of the bf16 weights (base-q8: group size 128, 64 on the routed down-projections whose width is not a multiple of 128; base-q4: group size 64; f16 scales). The Mamba-2 state-space parameters, norms and router stay in full precision in every build. A bare basert pull fetches Q4; ask for the others by width.

Usage

curl -LsSf https://basecompute.co/install.sh | sh
basert pull basecompute/NVIDIA-Nemotron-3-Nano-30B-A3B          # Q4
basert pull basecompute/NVIDIA-Nemotron-3-Nano-30B-A3B:q8       # Q8
basert pull basecompute/NVIDIA-Nemotron-3-Nano-30B-A3B:bf16     # bf16, as published
basert chat basecompute/NVIDIA-Nemotron-3-Nano-30B-A3B

Released under the NVIDIA Nemotron Open Model License, inherited from the base model.