mrysh/gemma-4-12b-it-prune_width_ffn30

gemma-4-12b-it - prune_width_ffn30

Pruned version of google/gemma-4-12B-it made for a pruning benchmark (4 models x 8 pruning variants). The benchmark compares the variants with each other and with quantization on BFCL, hiring-bias and MedQA, run with vLLM.

• method: width, level 0.3
• library: nvidia-modelopt 0.46.0rc1
• size on disk: 18.751 GB
• reduction of the language model: 21.4% of its parameters (of the whole checkpoint: 21.4%)

Width pruning with NVIDIA ModelOpt Minitron: the FFN size of every layer 15360 -> 10752 (30% requested), the number of layers is unchanged. Importance of the neurons was measured on a calibration set of 512 mixed samples.

The checkpoint is a text-only model (Gemma4ForCausalLM): the vision / audio blocks of google/gemma-4-12B-it are not included.

No distillation or fine-tuning was done after pruning, so expect a quality drop against the base model; the benchmark measures that drop. In the benchmark it is evaluated as a text model (vLLM, thinking off). Same license as the base model; see the base model card.