Green-Lab/Gemma-4-31B-IT-NVFP4

Model Overview

Description:

Gemma 4 31B IT is an open multimodal model built by Google DeepMind that handles text and image inputs, can process video as sequences of frames, and generates text output. It is designed to deliver frontier-level performance for reasoning, agentic workflows, coding, and multimodal understanding on consumer GPUs and workstations, with a 256K-token context window and support for over 140 languages. The model uses a hybrid attention mechanism that interleaves local sliding-window and full global attention, with unified Keys and Values in global layers and Proportional RoPE (p-RoPE) to support long-context performance. The GreenNode Gemma 4 31B IT NVFP4 model is quantized with NVIDIA Model Optimizer using a mixed-precision recipe: attention in FP8 block-scaled, MLP in NVFP4 with the 10 most sensitive layers in FP8 block-scaled, and FP8 KV cache.

This model is ready for commercial/non-commercial use.

Third-Party Community Consideration

This model is not owned or developed by GreenNode. This model is a quantized derivative of Google's Gemma 4 31B IT; see link to the original Gemma 4 31B IT Model Card

License and Terms of Use:

Apache License 2.0 | Gemma | Google AI for Developers

Deployment Geography:

Global

Use Case:

Use Case: Designed for text generation, chatbots and conversational AI, text summarization, image data extraction, reasoning, coding, multimodal understanding, function calling, and research or educational use.

Release Date:

Hugging Face [10/05/2026] via link

Model Architecture:

Architecture Type: Transformer
Network Architecture: Gemma 4
Number of model parameters: 30.7B Vocabulary Size: 262,144

Input:

Input Type(s): Text, Image, Video
Input Format(s): String, Red, Green, Blue (RGB), Video (MP4/WebM)
Input Parameters: One-Dimensional (1D), Two-Dimensional (2D), Three-Dimensional (3D)
Other Properties Related to Input: Supports variable image aspect ratios and resolutions, configurable visual token budgets of 70, 140, 280, 560, and 1120, and video inputs up to 60 seconds at one frame per second. The vision tower is kept in BF16; the evaluation below is text-only.
Input Context Length (ISL): 256K

Output:

Output Type(s): Text
Output Format: String
Output Parameters: 1D (One Dimensional): Sequences
Other Properties Related to Output: Generates text responses for chat, reasoning, coding, multimodal understanding, and function-calling workflows.

This model is optimized to run on NVIDIA Blackwell GPUs: NVFP4 and FP8 block-scaled GEMMs run on native tensor-core kernels, giving faster inference and a smaller memory footprint than the BF16 model.

Software Integration:

Supported Runtime Engine(s):

• vLLM
  

Supported Hardware Microarchitecture Compatibility:
NVIDIA Blackwell

Preferred Operating System(s):

• Linux
  

Model Version(s):

The model version is v1.0, quantized with nvidia-modelopt v0.47.0rc0 (source build, commit 333ace1bc)

Training, Testing, and Evaluation Datasets:

We calibrated the model using the dataset noted below, and performed evaluation using the benchmarks noted under Evaluation Datasets. We did not perform training or testing for this Model Optimizer release. The methods noted under Training and Testing Datasets below represent the data collection and labeling methods used by the third-party to train and test the underlying Gemma 4 31B IT model.

Calibration Dataset:

Link: Internal (not released)

Evaluation Dataset:

Data Collection Method by dataset: Hybrid: Human, Automated
Labeling Method by dataset: Hybrid: Human, Automated
Properties: We evaluated the model on GPQA Diamond, AIME 2025, AIME 2026, MMLU-Pro, LiveCodeBench v6, IFEval, GSM8K-Platinum, MATH-500, Global-MMLU (Vietnamese) and XQuAD (Vietnamese).

Inference:

Engine: vLLM 0.30.0
Test Hardware: 1x NVIDIA RTX PRO 6000 Blackwell (96 GB)

Post Training Quantization

This model was obtained by quantizing the weights and activations of Gemma-4-31B-IT with a per-layer mixed-precision recipe (ModelOpt MIXED_PRECISION, max calibration), ready for inference with vLLM:

Component	Format
self_attn q/k/v/o_proj	FP8 W8A8, weight block 128×128, dynamic per-128 activation (fp8_pb_wo)
MLP gate/up/down_proj	NVFP4 W4A4, group size 16
MLP of layers 1, 3, 15, 25, 27, 28, 40, 47, 58, 59	FP8 block 128×128 (fp8_pb_wo)
KV cache	FP8 with calibrated k_scale / v_scale
lm_head, vision tower	BF16 (not quantized)

The 10 FP8 MLP layers were selected by leave-one-out KL sensitivity: quantize all MLPs to NVFP4, then restore one layer at a time and measure the full-vocabulary KL divergence against BF16. The checkpoint averages 5.93 bits per weight over the language-model Linear layers; total checkpoint size is 24 GB, down from 59 GB in BF16.

Usage

To serve this checkpoint with vLLM (tested with 0.30.0), run the sample command below. vLLM auto-detects the ModelOpt mixed-precision config (modelopt_mixed), and the KV cache runs in FP8 as declared by the checkpoint:

vllm serve Green-Lab/Gemma-4-31B-IT-NVFP4 \
  --reasoning-parser gemma4 \
  --enable-auto-tool-choice --tool-call-parser gemma4

Evaluation Results:

Benchmark	Baseline (BF16)	RedHatAI FP8-block	Google QAT W4A16	NVIDIA NVFP4	GreenNode NVFP4 (ours)
GPQA Diamond	86.1%	86.6%	86.6%	84.6%	85.9%
AIME 2025	89.2%	87.5%	81.7%	86.7%	87.5%
AIME 2026	90.0%	90.0%	85.8%	88.3%	90.0%
MMLU Pro	84.3%	84.6%	84.5%	83.5%	83.5%
LiveCodeBench v6 (pass@1)	74.9%	79.4%	77.1%	75.4%	78.3%
IFEval (prompt-level strict)	91.1%	91.3%	91.3%	89.5%	90.8%
GSM8K-Platinum	98.3%	98.4%	98.7%	98.3%	98.4%
MATH-500	98.2%	98.2%	97.6%	98.6%	98.6%
Global-MMLU (vi)	87.2%	86.4%	86.6%	86.7%	86.0%
XQuAD (vi) F1	82.1	82.0	82.7	82.6	82.4
Average recovery vs BF16	100%	100.4%	99.1%	99.2%	100.1%

Efficiency & fidelity	Baseline (BF16)	RedHatAI FP8-block	Google QAT W4A16	NVIDIA NVFP4	GreenNode NVFP4 (ours)
Checkpoint size	59 GB	32 GB	22 GB	31 GB	24 GB
KV cache	BF16	BF16	BF16	FP8	FP8
Output throughput (tok/s, conc 64)	354 (×1.00)	656 (×1.86)	609 (×1.72)	964 (×2.73)	1070 (×3.03)
TPOT median (ms, conc 64)	75.9	81.7	89.4	59.7	54.0
KL divergence vs BF16 (mean / p99)	0	0.0032 / 0.050	0.0193 / 0.374	0.0211 / 0.395	0.0194 / 0.359
Top-1 token agreement vs BF16	100%	99.07%	97.61%	97.47%	97.62%

Baseline: google/gemma-4-31B-it. Compared models: RedHatAI FP8-block, Google QAT W4A16 (converted to compressed-tensors INT4 g32), NVIDIA NVFP4.
All models were benchmarked on the same engine and hardware (vLLM 0.30.0, 1x RTX PRO 6000 Blackwell, TP=1, max-model-len 65536) with the BF16 chat template.
Thinking ON for GPQA (×2 repeats), AIME (×4 repeats) and LiveCodeBench (2025-01 → 2025-04, 175 problems), max 60K new tokens; thinking OFF for MMLU-Pro (1,400 questions, 5-shot CoT), GSM8K-Platinum (8-shot CoT), IFEval and MATH-500. Sampling: temperature=1.0, top_p=0.95, top_k=64; Vietnamese tasks use greedy decoding.
Average recovery = mean of per-benchmark (quantized / BF16) over the 10 benchmarks above. Throughput: vllm bench serve, random 1024 input / 512 output tokens, 256 requests at concurrency 64. KL / top-1: teacher-forced on 291 held-out production prompts (75,977 response tokens), top-20 logprobs.

Model Limitations:

The base model was trained on data that contains toxic language and societal biases originally crawled from the internet. Therefore, the model may amplify those biases and return toxic responses especially when prompted with toxic prompts. The model may generate answers that may be inaccurate, omit key information, or include irrelevant or redundant text producing socially unacceptable or undesirable text, even if the prompt itself does not include anything explicitly offensive.

Ethical Considerations

GreenNode believes Trustworthy AI is a shared responsibility. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please make sure you have proper rights and permissions for all input image and video content; if image or video includes people, personal health information, or intellectual property, the image or video generated will not blur or maintain proportions of image subjects included.