Darwin-V9-Chimera-4B-SFT (Generation 2 · instruction-tuned)

VIDRAFT attention + Qwen3-4B / Gemma4-E4B FFN crossbreed + SFT. A Qwen3-4B × Gemma4-E4B hybrid — NOT from-scratch. Private research checkpoint.

Lineage

Gen Model Composition
Gen1 Darwin-Chimera-4B-Gen1 Qwen3-4B attention-healing adapter
Gen2 (healed) Darwin-V9-Chimera-4B + Gemma4-E4B FFN crossbreed → re-healing
Gen2 SFT (this) Darwin-V9-Chimera-4B-SFT + B4-Hybrid SFT (think toggle + ko/en CoT)

SFT

  • data: B4_merged_hybrid, 17,202 ex (think/no_think toggle, Korean/English CoT)
  • method: ChatML + prompt masking (assistant-only loss) + full fine-tune, lr 2e-5, 3 epochs

Evaluation (KMMLU, ChatML)

measure score
max_new=8 (short answer) 25.6% (measurement artifact)
max_new=768 (think) 52.2% (Math 70%, CS 67%)

→ With sufficient thinking budget, 52.2% — ~2× the healed checkpoint (27.8%). Instruction following is complete (think/no_think toggle, markdown, code). The short-answer score is an artifact: this model reasons before answering.

Weight fingerprint (honest)

  • vs Qwen3-4B: FFN 16% rel-change, attention 3%, embed 0.78%. FFN cosine 0.998 → Qwen-dominant.
  • vs Gemma4-E4B FFN: cosine 0.09 (15% blend, weak at weight level; Gemma knowledge partially visible in generation).
  • At the weight level this is clearly a Qwen3-4B derivative. We make no from-scratch / independence claim.

NEG (Native Entropy Gating)

  • A NEG-Head was trained (entropy-prediction pearson 0.92) and neg_modules.safetensors is included.
  • However, NEG showed no effect on KMMLU (threshold untuned / mid-difficulty). Potential effect on harder reasoning (GPQA) is unverified.

License

Gemma Terms of Use (Gemma4-E4B weights blended in) + Apache 2.0 (Qwen3-4B). Built on Qwen/Qwen3-4B and google/gemma-4-E4B.

Downloads last month
3
Safetensors
Model size
4B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for FINAL-Bench/Darwin-V9-Chimera-4B-SFT

Finetuned
Qwen/Qwen3-4B
Finetuned
(1018)
this model
Quantizations
2 models