gemma-4-e4b-it - prune_width_ffn15
Pruned version of google/gemma-4-E4B-it made for a pruning benchmark (4 models x 8 pruning variants). The benchmark compares the variants with each other and with quantization on BFCL, hiring-bias and MedQA, run with vLLM.
- method: width, level 0.15
- library: nvidia-modelopt 0.46.0rc1
- size on disk: 14.924 GB
- reduction of the language model: 7.3% of its parameters (of the whole checkpoint: 6.9%)
Width pruning with NVIDIA ModelOpt Minitron: the FFN size of every layer 10240 -> 8704 (15% requested), the number of layers is unchanged. Importance of the neurons was measured on a calibration set of 512 mixed samples.
The vision / audio blocks of the base model, if it has them, are kept unchanged and only the language part is pruned. The multi-token-prediction head of Qwen3.5 is dropped by the pruning tool.
No distillation or fine-tuning was done after pruning, so expect a quality drop against the base model; the benchmark measures that drop. In the benchmark it is evaluated as a text model (vLLM, thinking off). Same license as the base model; see the base model card.
- Downloads last month
- -