mrysh/qwen3.5-4b-prune_width_ffn50

qwen3.5-4b - prune_width_ffn50

Pruned version of Qwen/Qwen3.5-4B made for a pruning benchmark (4 models x 8 pruning variants). The benchmark compares the variants with each other and with quantization on BFCL, hiring-bias and MedQA, run with vLLM.

• method: width, level 0.5
• library: nvidia-modelopt 0.46.0rc1
• size on disk: 6.167 GB
• reduction of the language model: 26.9% of its parameters (of the whole checkpoint: 26.9%)

Width pruning with NVIDIA ModelOpt Minitron: the FFN size of every layer 9216 -> 4608 (50% requested), the number of layers is unchanged. Importance of the neurons was measured on a calibration set of 512 mixed samples.

Only the language part is pruned. The vision tower and the multi-token-prediction head of Qwen3.5 are removed, so this is a text-only model (like the quantized Qwen checkpoints of this benchmark).

No distillation or fine-tuning was done after pruning, so expect a quality drop against the base model; the benchmark measures that drop. In the benchmark it is evaluated as a text model (vLLM, thinking off). Same license as the base model; see the base model card.