hbfreed/Laguna-S-2.1-winnow-keep50

Laguna-S-2.1 · Winnow keep-50

poolside/Laguna-S-2.1 (118B-A8B) with 50% of routed-expert channels removed by winnow variable-width expert pruning — 60.8B parameters, no healing/retraining.

Instead of deleting whole experts (REAP-style), winnow shrinks each expert's SwiGLU intermediate width according to a per-channel importance score (router_weight × |post-SwiGLU activation| × down-projection column L2), allocated by a per-layer global rank. 80% of experts survive at reduced, ragged widths; the shared expert, attention, and embeddings are untouched.

Scoring, pruning, and evaluation all ran layer-streamed on 3×RTX 3090 (the source model never fit in memory): one decoder layer resident per GPU, residual streams ping-ponged through disk, calibration data-parallel across GPUs. Calibration: 2.1M tokens of bigcode/the-stack-smol.

Results (unhealed)

model	params	held-out code ppl	HumanEval pass@1
Laguna-S-2.1 (base)	118B	2.947	—
keep-75 (here)	89.2B	2.980 (+1.1%)	—
keep-50 (this repo)	60.8B	3.399 (+15.3%)	89.6%

• Perplexity: 64 held-out the-stack-smol sequences (2048 tokens each), identical ruler for all three rows, computed layer-streamed.
• HumanEval: greedy, chat-extract protocol, thinking disabled, max 1280 new tokens, INT8-expert serving. Not comparable to raw-completion HumanEval numbers or to poolside's published agentic benchmarks (SWE-bench, Terminal-Bench 2.1), which use a different harness and far larger budgets.
• Qualitative agentic check: with a 16k serving context the model autonomously solved several classic Terminal-Bench 1.0 core tasks end-to-end on the same 3×3090 host; most failures were context exhaustion, not task competence.

Serving

At INT8 W8A16 expert weights this model serves on 3×24GB GPUs via the glean vLLM plugin (ragged experts are not representable in stock FusedMoE):

vllm serve hbfreed/Laguna-S-2.1-winnow-keep50 \
  --enforce-eager --pipeline-parallel-size 3 --quantization experts_int8 \
  --max-model-len 16384

Measured on 3×3090: 34 tok/s single-stream, ~150 tok/s at batch 8. Tensor parallelism is unsupported (packed ragged weights don't shard); pipeline parallelism is the intended mode.

Reference (unfused) loading works with plain transformers ≥ 5.13:

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
    "hbfreed/Laguna-S-2.1-winnow-keep50", trust_remote_code=True)

winnow.json records the full pruning plan, calibration provenance, and per-expert channel indices.

Pruned with winnow's stream-prune; see the winnow repository for the method writeup. Base model by poolside, released under OpenMDW-1.1; this derivative keeps that license.