LeDissolution/Gemma-4-31B-it-Gewell_G0

Gemma 4 31B IT — Gewell G0

G0 is the mixed-precision Gemma 4 31B IT text bundle used by Gewell, a single-GPU inference engine specialized for NVIDIA Blackwell (sm_120/sm_120a). This repository also includes the extracted BF16 vision tower and multimodal projector. The text bundle is in Gewell's native schema-6 format; it is not a Transformers, vLLM, or llama.cpp checkpoint.

Bundle contents

• weights.gwt — 25,954,750,464 bytes (24.17 GiB), mixed FP8/NVFP4 weights.
• manifest.json — artifact identity, tensor inventory, layout, source provenance, and checksums.
• tokenizer.json — tokenizer required by the Gewell runtime.
• vision.safetensors — 1,151,536,616 bytes (1.07 GiB), BF16 vision tower and multimodal projector.
• g0.mask — the complete precision recipe used for G0.

The weight file SHA-256 is 595b5b9e7e0dc019d6494a73a15a5db4524facc1046ab15253c576f57eb5e499. The vision file SHA-256 is 01121d84f3261586f7da72166293183f5b44b0c704ef334ea13de4f263e1d6a9. The manifest also records the payload, header, entry-table, serving-asset, and per-tensor checksums.

Precision layout

G0 contains 266 FP8 W8A8 projections and 144 NVFP4 W4A4 projections. Q/K/V projections use FP8 throughout. O and MLP projections use FP8 in layers 0–5, 10–11, 16–17, 22–23, 28–29, 34–35, 40–41, 46–47, 52–53, and 58–59, and NVFP4 elsewhere. Embeddings, norms, and other scalar tensors remain BF16. Layer indices are zero-based; g0.mask is the authoritative recipe.

Download and verify

Follow the Gewell build instructions, then download this repository and the pinned official MTP assistant. The assistant is loaded directly in its original Google safetensors format and is not redistributed here.

MODEL_DIR=/absolute/path/to/Gemma-4-31B-it-Gewell_G0
ASSISTANT_DIR=/absolute/path/to/gemma-4-31B-it-assistant
ASSISTANT_REV=4735700dca7bd22fad5dc348c228b50ec6cbac6d

hf download LeDissolution/Gemma-4-31B-it-Gewell_G0 \
  --local-dir "$MODEL_DIR"

hf download google/gemma-4-31B-it-assistant model.safetensors \
  --revision "$ASSISTANT_REV" \
  --local-dir "$ASSISTANT_DIR"

build/gewell verify "$MODEL_DIR/weights.gwt"

Use hf auth login first if your Hugging Face setup requires authentication for the upstream Google repository.

The pinned assistant file is 939,042,560 bytes and has SHA-256 9f80df6099fa1fd7db71220ec9ee864d5ecff769878697dd5763e4285b15a1da.

Launch with vision and MTP

The following launches the OpenAI-compatible server with image input and three assistant proposals per speculative cycle:

build/gewell \
  --vision "$MODEL_DIR/vision.safetensors" \
  --assistant "$ASSISTANT_DIR/model.safetensors" \
  --mtp-depth 3 \
  serve-http \
  --model-dir "$MODEL_DIR" \
  --max-batch 1 \
  --kv-cache-gpu-mib 8192 \
  --model gemma-4-31b

Omit --vision to disable image input. Set --mtp-depth 0 to disable MTP; the assistant is loaded only for a positive depth. Vision and assistant weights add approximately 1.15 GB and 0.94 GB of GPU allocation respectively, before vision scratch and KV cache. Adjust the cache budget and batch limit for your GPU and workload.

The engine was developed and tested on an RTX PRO 6000 Blackwell. RTX 5090 is an intended target but has not been validated by the project author. See the launch guide and HTTP API documentation for current hardware requirements, image request format, and serving options.

Scope and provenance

This repository contains the G0 text model and its compatible BF16 vision component. The speculative-decoding assistant remains a separate optional component downloaded from the pinned official Google revision above.

The text and vision components were derived from google/gemma-4-31B-it at revision fcf2302760ae9c6e528a8dbba9dd636e56848237. Conversion uses the G0 precision mask and Gewell's recorded reference activation scales. Quantization can change model quality; evaluate the model on your own workloads.

Use is subject to the upstream Gemma 4 license.