Hugging Face model: weathermanj/NVIDIA-Nemotron-Nano-9B-v2-gguf
Author: weathermanj
Created: 2025-08-28 19:28:08
Last modified: 2025-08-29 00:20:12
Pipeline: text-generation
Library: llama.cpp
Likes: 1 | Downloads: 244
Tags: llama.cpp, gguf, nemotron_h, text-generation, quantized, nvidia, nemotron, mamba2, transformer, en, base_model:nvidia/NVIDIA-Nemotron-Nano-9B-v2, base_model:quantized:nvidia/NVIDIA-Nemotron-Nano-9B-v2, license:other, endpoints_compatible, region:us
Base model relation: quantized
Base models: id: nvidia/NVIDIA-Nemotron-Nano-9B-v2
Card metadata:
tags: gguf, llama.cpp, text-generation, quantized, nvidia, nemotron, mamba2, transformer
language: en
license: other
license_name: nvidia-open-model-license
license_link: https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
base_model: nvidia/NVIDIA-Nemotron-Nano-9B-v2
library_name: llama.cpp
pipeline_tag: text-generation
model_type: nemotron_h
quantized: yes
quantization_type: gguf
quantization_config: quantized: yes
format: gguf
variants: filename: NVIDIA-Nemotron-Nano-9B-v2-gguf-Q2_K.gguf
size: 4.7GB
bits_per_weight: ~2.0
description: 2-bit K-quantization, maximum compression, filename: NVIDIA-Nemotron-Nano-9B-v2-gguf-Q8_0.gguf
size: 8.9GB
bits_per_weight: ~8.0
description: Near-lossless, reference quality, filename: NVIDIA-Nemotron-Nano-9B-v2-gguf-Q6_K.gguf
size: 8.6GB
bits_per_weight: ~6.0
description: High quality, recommended, filename: NVIDIA-Nemotron-Nano-9B-v2-gguf-Q5_K_M.gguf
size: 6.6GB
bits_per_weight: ~5.0
description: Good quality, balanced, filename: NVIDIA-Nemotron-Nano-9B-v2-gguf-Q4_K_M.gguf
size: 6.1GB
bits_per_weight: ~4.0
description: Standard choice, good compression, filename: NVIDIA-Nemotron-Nano-9B-v2-gguf-Q4_1.gguf
size: 5.5GB
bits_per_weight: ~4.0
description: Legacy 4-bit (Q4_1), slightly better quality than Q4_0, filename: NVIDIA-Nemotron-Nano-9B-v2-gguf-Q4_0.gguf
size: 5.0GB
bits_per_weight: ~4.0
description: Legacy 4-bit (Q4_0), smaller, lower quality, filename: NVIDIA-Nemotron-Nano-9B-v2-gguf-Q4_K_S.gguf
size: ~5.8GB
bits_per_weight: ~4.0
description: 4-bit K (small), smaller than Q4_K_M, filename: NVIDIA-Nemotron-Nano-9B-v2-gguf-IQ4_XS.gguf
size: 5.0GB
bits_per_weight: 4.25
description: Integer quantization, excellent compression, filename: NVIDIA-Nemotron-Nano-9B-v2-gguf-IQ3_M.gguf
size: 4.9GB
bits_per_weight: 3.66
description: Ultra-small, mobile/edge