gemma-4-e4b-it - prune_24
Pruned version of google/gemma-4-E4B-it made for a pruning benchmark (4 models x 8 pruning variants). The benchmark compares the variants with each other and with quantization on BFCL, hiring-bias and MedQA, run with vLLM.
- method: semi_structured_24
- library: nvidia-modelopt 0.46.0rc1
- size on disk: 15.915 GB
2:4 semi-structured sparsity: 2 zeros in every 4 consecutive weights of the decoder linear layers (50.0% zeros), made with NVIDIA ModelOpt SparseGPT on a calibration set of 512 mixed samples. The file is dense (zeros are stored as zeros), so there is no size gain, and the speed gain needs a sparse runtime that was not used here.
No distillation or fine-tuning was done after pruning, so expect a quality drop against the base model; the benchmark measures that drop. In the benchmark it is evaluated as a text model (vLLM, thinking off). Same license as the base model; see the base model card.
- Downloads last month
- 20