Shadow-Siren-14B-v2-Pruned

Shadow-Siren-14B-v2-Pruned is a structural optimization of the Shadow-Siren mixture-of-experts model, tailored for expressive roleplay, conversational depth, and creative writing.

---

Why v2?

The v1 release encountered an architectural mismatch due to Gemma-4's batched 3D expert tensor configuration ([num_experts, hidden_dim, intermediate_dim]):

• Tensor Slicing Correction: Conventional 2D MoE weight slicing failed to properly alter Gemma-4's batched expert dimensions, leaving the model parsed at ~25B parameters on Hugging Face.
• True 14.5B Footprint: In v2, all batched 3D expert blocks (.experts. and routing matrices) were sliced along dim=0 from 128 down to 64 experts.
• Metadata Alignment: Both root and nested text_config attributes now reflect num_experts: 64, accurately reporting the intended ~14.5B total parameter count.
• Cleaned Tokenizer Config: Fixed potential parser warnings regarding legacy special token formatting.

---

Architecture Specifications

• Base Architecture: Gemma-4 MoE
• Original Experts: 128
• Active Pruned Experts: 64
• Parameter Count: ~14.5B
• Precision: BF16
• Routing: Top-K dynamic gating with per-expert scaling intact

---

GGUF Quantizations

Optimized GGUF quantization weights (including IQ4_XS) are available at Goopr/Shadow-Siren-14B-v2-GGUF.