Ling-3.0-flash is inclusionAI's next-generation native hybrid reasoning model: 124B total / 5.1B active (~12.4% of their previous 1T-class flagship). It uses a native hybrid-linear stack from pretraining — 5:1 Kimi Delta Attention (KDA) + gated MLA, 1/64 sparse MoE, 512 routed experts (top-8) + 1 shared expert, 2 dense layers, and a trained MTP head. Official context schedule is 8K → 32K → 256K. Thinking is on by default in the official card.
This community package is not an official inclusionAI release. It applies two extra steps on top of the official weights:
Refusal edit (mergeable recipe) via abliterixv6 trial 21 LoRA, stream-merged back to BF16. Write path: MPOA weight_normalization=full, LoRA rank 3, o_proj + down_proj only. This is the static merge, not the later runtime-only gate (v36).
GGUF: BF16 convert + APEX mixed-precision tensor-type files (no imatrix on these tiers). Architecture in the GGUF is bailingmoe3.
License: MIT (same family as the official model). Follow inclusionAI terms and local law.
⚠️ Abliterated notice
After merging abliterix v6 trial 21, this model shows a much lower refusal rate and can differ from official Ling-3.0-flash. Evaluate compliance and safety for your use case; control access and audit as needed.
Refusals (search eval)
17 / 98 (17.3%)
KL divergence
0.0933 nats/token
Selected trial
abliterix v6 trial 21 (mergeable). Not v36 (runtime gate).
Compact smoke (thinking off)
Capability ~9/10; benign refuse 0/10; harmful refuse ~2–3/40. Ablation holds. Thinking-on eval can eat the token budget — use --reasoning off or a larger max_tokens.
Implementation sketch: LoRA merge W += (B @ A) * (alpha / r) with MPOA full-norm rank 3 on o_proj + down_proj. Adapter path: Ling-3.0-flash-LoRA-Trial21-Refusals17-KL0.093.
Coding / agent ability is largely retained on Compact smoke; refusal and alignment behavior are changed. Official SWE / Tau / MCP tables were not re-run for this derivative.
💡 What is APEX?
These files use APEX-style MoE-aware mixed precision: precision follows tensor role + layer position (higher on edges, more aggressive in the middle).
This package is not I-tier (no imatrix). Common settings: 43 GGUF blocks (42 backbone + MTP blk.42); first 2 layers dense; routers ffn_gate_inp=Q8_0; MTP + nextn.eh_proj forced Q8_0; KDA/MLA tensor names as in llama.cpp bailingmoe3.
Recommended default — steadier than Compact (no IQ mid)
*-APEX-Compact.gguf
56.25 GiB
edge Q4_K / mid Q3_K; shared Q6_K; attn Q4_K
Fits ~64–96 GB unified / VRAM with context room
*-BF16.gguf
237.57 GiB
native BF16 (norms / router / KDA state F32)
Requantize / archive. Too large for a 128 GB box.
How to choose: start with Balanced if you have the RAM. Use Compact on 128 GB UMA (this is what we smoke-tested). Quality if you want IQ mid-experts and a smaller file than Balanced. Skip BF16 unless you are converting again.
🚀 Usage (llama.cpp)
These GGUFs are general.architecture = bailingmoe3. Stock llama.cpp does not load them yet. You need PR #26608 (or a build that merged it). Do not mix with unofficial bailing-hybrid forks — the tensor names do not match.
./llama-server -m ./models/Ling-3.0-flash-abliterated-APEX-Compact.gguf --port 8080 -sm none --flash-attn on --fit on --ctx-size 32768 --reasoning off --jinja --host 0.0.0.0
Official sampling: temperature=0.6, top_p=0.95, top_k=20. Official HF/SGLang/vLLM path enables thinking by default; for llama.cpp eval we used --reasoning off so thinking does not eat the token budget.
MTP is present as blk.42. This package was served with --spec-type none; speculative decode needs a build that understands Ling MTP.
Text-only GGUF; no mmproj.
🎛️ Recommended sampling
Official default
temperature 0.6, top_p 0.95, top_k 20
Thinking
Official: on by default. llama.cpp: --reasoning on|off as needed.
Community derivative (behavior edit + quantization). Not an official inclusionAI release. Use at your own risk; follow local law and the upstream MIT license.
