engram-ae/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-GGUF

At a Glance

Total parameters	31B (Mamba2-Transformer hybrid MoE)
Active parameters	~3B per token
Modalities (in)	Video, Audio, Image, Text; video with its soundtrack in one pass
Modality (out)	Text
Reasoning mode	On by default
Quantization	Q4_K_M (GGUF); projector F16 (audio tower F16/BF16)
Files	Nine quantizations (Q3_K_M, IQ3_M, IQ4_XS, MXFP4_MOE, Q4_K_M, Q5_K_M, Q6_K, Q8_0, BF16) plus the unified projector mmproj-omni-vision-audio-F16.gguf (2.8 GB); single-tower projectors kept for compatibility. Every quantization is verified on image, audio, and video with soundtrack: see TEST-MATRIX.md
Runtime	llama.cpp: audio and video require the llama.cpp-omni fork (feat/nemotron-video); image and text run on stock
Minimum hardware	Any system llama.cpp supports with ~28 GB free memory; verified on NVIDIA DGX Spark (GB10, unified memory)

Model Parameters

Mode	temperature	top_p	max_tokens
Thinking mode	0.6	0.95	20480
Transcription / deterministic	0.0	:	2048

Model Overview

Description:

NVIDIA Nemotron 3 Nano Omni is a multimodal large language model that unifies video, audio, image, and text understanding to support enterprise-grade Q&A, summarization, transcription, and document intelligence workflows. It extends the Nemotron Nano family with integrated video+speech comprehension, Graphical User Interface (GUI), Optical Character Recognition (OCR), and speech transcription capabilities. NVIDIA Nemotron 3 Nano Omni was developed by NVIDIA as part of the Nemotron model family.

This repository provides a GGUF release of the model for llama.cpp, quantized to Q4_K_M, together with a unified multimodal projector: a single mmproj GGUF carrying both the C-RADIO v4-H vision encoder (image and video frames) and the Parakeet speech encoder (audio inputs). Both towers are packaged in the mixed-modality GGUF layout llama.cpp resolves per modality (clip.vision.projector_type = nemotron_v2_vl, clip.audio.projector_type = parakeet), so one server process accepts video, audio, image, and text without swapping projector files. A video that carries an audio track is consumed in one pass: the frames go to the vision tower and the soundtrack is demuxed and fed to the audio tower in the same request, so the model answers about what it sees and what it hears from a single file.

This model is available for commercial use.

License/Terms of Use

Governing Terms: Use of this model is governed by the NVIDIA Open Model Agreement

Deployment Geography:

Global

Use Case:

This release is intended for users who require the Nemotron 3 Nano Omni multimodal capabilities: video and speech analysis, document intelligence (OCR, charts, long documents), speech transcription: on hardware served by llama.cpp, including single-GPU and unified-memory systems where the PyTorch serving stacks are not practical.

Release Date:

Hugging Face 08/22/2026

Model Architecture:

Architecture Type: Mamba2-Transformer Hybrid Mixture of Experts (MoE)

Network Architecture:

• Nemotron 3 Nano LLM (30B A3B): 31B-parameter Mamba2-Transformer hybrid MoE backbone with ~3B active parameters per token, quantized here to Q4_K_M.
• CRADIO v4-H: vision encoder for image and video frames, F16, in the unified projector.
• Parakeet: speech encoder for audio inputs, F16/BF16, in the unified projector.

Changes from the source checkpoint:

• GGUF conversion of the language model from Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16, quantized to Q4_K_M.
• Both encoder towers converted from the same checkpoint and merged into one mixed-modality projector file; encoder weights preserved bit-exact from the per-tower conversions.
• The audio (Parakeet FastConformer) compute graph and the video temporal-patch path are additions to llama.cpp carried in the llama.cpp-omni fork, validated against a PyTorch reference (video graph to 0.0019%, audio to 0.03%). An upstream contribution is planned.

Input(s):

Input Type(s): Video, Audio, Image, Text

Input Format(s):

• Video: mp4
• Audio: wav, mp3 (16 kHz mono recommended for transcription)
• Image: Red, Green, Blue (RGB) (jpeg, png)
• Text: String
  

Output(s)

Output Type(s): Text

Other Properties Related to Output:

• Supports reasoning output with chain-of-thought (on by default)
  

Software Integration:

Runtime Engine(s):

• llama.cpp (image, text: stock; audio, video: llama.cpp-omni, branch feat/nemotron-video)
  

Supported Hardware Microarchitecture Compatibility:

• Any platform llama.cpp supports (CUDA, Metal, CPU); verified on NVIDIA Blackwell (DGX Spark / GB10)
  

Preferred/Supported Operating System(s):

• Linux, macOS
  

Quick Start

git clone https://github.com/VincentKaufmann/llama.cpp-omni
cd llama.cpp-omni && git checkout feat/nemotron-video
cmake -B build -DGGML_CUDA=ON && cmake --build build -j --target llama-mtmd-cli llama-server

# image
./build/bin/llama-mtmd-cli -m Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Q4_K_M.gguf \
  --mmproj mmproj-omni-vision-audio-F16.gguf --image photo.jpg -p "Describe this image."

# audio
./build/bin/llama-mtmd-cli -m Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Q4_K_M.gguf \
  --mmproj mmproj-omni-vision-audio-F16.gguf --audio speech.wav -p "Transcribe this audio."

# video: a soundtrack, when present, is heard in the same pass
./build/bin/llama-mtmd-cli -m Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Q4_K_M.gguf \
  --mmproj mmproj-omni-vision-audio-F16.gguf --video clip.mp4 \
  -p "What is shown, and what does the narrator say?"

# server: one process, all modalities on /v1/chat/completions
./build/bin/llama-server -m Nemotron-3-Nano-Omni-30B-A3B-Reasoning-Q4_K_M.gguf \
  --mmproj mmproj-omni-vision-audio-F16.gguf

Roadmap

The next release rebases this work onto current llama.cpp master and adopts the upstream audio projector layout. After that, image, text and audio run on stock llama.cpp with these files; the fork remains required only for video and one-pass audio+video.

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility. Please refer to the source model's model card for the training data, evaluation, bias, and safety documentation that applies to these weights. This repository changes the storage format and serving runtime only; it does not alter model behavior beyond the effects of Q4_K_M quantization.