vcruz305/Solar-Open2-250B-GGUF

Solar-Open2-250B GGUF

First GGUF quantizations of Upstage Solar-Open2-250B.

Requires a llama.cpp build with solar_open2 support

Stock llama.cpp does not yet support this architecture. Build from the fork that adds it:

Fork: https://github.com/vcruz305/llama.cpp (branch solar-open2-support)

git clone -b solar-open2-support https://github.com/vcruz305/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j

Architecture

Solar-Open2 is a 48-layer hybrid: 36 Kimi Delta Attention (KDA) linear-attn layers + 12 gated-GQA (NoPE) layers, with a 320-expert DeepSeek-V3-style MoE (shared expert + sigmoid router + e_score correction bias). ~250B total, ~15B active per token.

Validation

• Logit parity: 8/8 greedy top-1 match vs Upstage transformers reference.
• Coherence: fluent generation on the full 250B model.
• Perplexity (Q2_K, wikitext-2, c=512, 80 chunks): PPL = 5.9322 +/- 0.10022.

Running on a single DGX Spark (GB10, 128GB unified)

Use --no-mmap for the large quants - default mmap double-allocates host+device and can exceed unified memory:

llama-cli -m Solar-Open2-250B-Q2_K.gguf --no-mmap -ngl 999 \
  -p "The capital of France is"

Larger quants (Q4_K_M and up) exceed a single 128GB box; run them across two GB10s with llama.cpp RPC (--rpc host:port --no-mmap -ts 0.5,0.5) or on a larger-memory machine.

Quants

Rung	Notes
Q8_0	near-lossless reference
Q4_K_M	recommended quality (multi-box / big-RAM)
Q2_K	fits a single GB10; PPL 5.93
IQ4_XS / IQ3_XXS / IQ2_M / IQ1_M	imatrix low-bit rungs

(Rungs upload as they finish quantizing.)

Credit

Base model (c) Upstage. GGUF arch support built on ggml-org/llama.cpp (MIT); KDA adapted from kimi-linear.cpp, MoE from deepseek-v3, gating from qwen3next.cpp.