Solar-Open2-250B-W4A8

Weight-INT4 / activation-FP8 (W4A8) quantization of upstage/Solar-Open2-250B, self-baked and served in production on H200 GPUs. Only the routed MoE expert weights are quantized to INT4 (group size 128); the shared expert, dense MLP, attention projections, and embeddings stay bf16. KV cache is served in FP8.

• Recipe: OMSE (optimal mean-squared-error) calibration for the INT4 routed-expert weights — dominated a plain RTN (round-to-nearest) bake on the same accuracy gate, so OMSE is the shipped recipe.
• Footprint: 132 GiB total (vs. ~467 GiB bf16) — fits within TP4 on H200×4 minimum, TP8 recommended for headroom.
• Serving stack: vLLM fork (v0.26.0-solar-open2) with 3 upstream 0.22→0.26 porting-regression fixes applied to the hybrid-attention (KDA linear-attention) decode path — see Serving below.

Accuracy

Self-measured, same-question paired A/B against the bf16 original (frozen question sets, identical harness, temperature=0.6, max_tokens=40960):

Benchmark	n	This model (W4A8)	bf16 anchor	Δ
GPQA-Diamond	50	0.780 (39/50)	0.780 (39/50)	exact parity
MMLU-Pro	100	0.830	0.840	−1.0pp (within 1-question sampling noise, n=100 SE ≈ ±3.8pp)

Larger-sample backstop (same harness, un-paired):

Benchmark	n	raw	conclusion-only (excludes token-budget truncations)
GPQA-Diamond	198	0.763	0.853
MMLU-Pro	300	0.820	0.840

Additional quality probes, both PASS: end-of-sequence behavior on 3 long-form generation prompts (finish_reason=stop 3/3, no runaway repetition), tool-calling correctness on 2 single/parallel-call probes (2/2 valid parsed calls). "raw" scores are pulled down mainly by max_tokens truncation on the hardest reasoning items (finish_reason=length) — a harness property shared with the bf16 anchor, not a quantization-specific effect; the conclusion-only column removes those unanswered items.

⚠️ These benchmarks run at a few-thousand-token input context. Long-input-context accuracy (production p50 context is far longer) has not been separately measured — quantization error can be context-length dependent, so treat this as a partial accuracy cover, not a full one.

Performance

In-house serving measurements on a fixed TP4×2 (8×H200) topology with an fp8-KV, prefix-caching engine, at an offered load near the goodput knee for this checkpoint (open-loop replay of production-shaped traffic, with a DSpark speculative-decoding draft attached — see Solar-Open2-250B-DSpark):

Metric	Value	Gate
TTFT p50	0.433s	≤ 8s
TPOT p50 (per output token)	15.1ms	≤ 25ms
Cache hit rate	0.87	—
Draft accept length	3.33	—

Absolute throughput (TPM) and node/GPU counts are withheld per internal reporting policy; the latency figures above are the release gate.

Serving

vLLM ≥ 0.26.0 with the Solar-Open2 fork. The hybrid-attention (12 softmax-attention layers interleaved with 36 KDA linear-attention layers) decode path needs three fixes ported from the 0.22.0 lineage — without them the model either corrupts state, mis-tracks speculative-decoding accept counts, or crashes on CUDA-graph capture. All three are baked into the served image; see the repo's serving notes if building your own image.

vllm serve vessl/Solar-Open2-250B-W4A8 \
  --served-model-name solar-open2-250b \
  --tensor-parallel-size 4 --enable-expert-parallel \
  --max-num-batched-tokens 65536 \
  --moe-backend triton \
  --enable-prefix-caching \
  --kv-cache-dtype fp8 \
  --stream-interval 18 \
  --max-num-seqs 64 \
  --attention-config '{"flash_attn_max_num_splits_for_cuda_graph": 8}' \
  --compilation-config '{"cudagraph_mode":"FULL_AND_PIECEWISE"}' \
  --default-chat-template-kwargs '{"think_render_option":"preserved"}' \
  --reasoning-parser solar_open2 --tool-call-parser solar_open2 \
  --enable-auto-tool-choice \
  --port 8000

Required environment on multi-GPU-per-node hosts: VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm. Some hosts auto-select the mnnvl all-reduce backend, which initializes without error but produces far worse latency — check the boot log line Auto-selected flashinfer allreduce backend: ... and force trtllm if it isn't already selected.

--enable-expert-parallel is required regardless of tensor-parallel degree: the INT4 grouped-GEMM MoE kernel needs intermediate_size divisible into 256-wide partitions, which plain tensor parallelism can violate at some TP degrees; expert parallelism sidesteps the slicing entirely.

To attach the DSpark speculative-decoding draft:

--speculative-config '{"method":"dspark","model":"vessl/Solar-Open2-250B-DSpark","num_speculative_tokens":4}'

CUTLASS W4A8 MoE requires intermediate_size to be divisible by 256 per TP partition — this checkpoint is validated at TP4/EP4; a TP8 variant would need re-baking (or re-verification of the same divisibility constraint) before use.

License

This is a derivative of Solar Open 2 and is distributed under the same Upstage Solar License as the base model. Per the license's requirements for derivative AI models: this model's name is prefixed with "Solar," and this card carries a "Built with Solar" attribution. See the base model's license for full terms.

---

Original model card: upstage/Solar-Open2-250B

Solar Open 2

Solar Open 2 is Upstage's 250B-A15B open-weight large language model, built for agentic use cases such as office productivity, document-intensive work, and coding. Its Hybrid-Attention Mixture-of-Experts (MoE) architecture with linear attention delivers highly efficient inference even in long-context settings.

Technical Report | Blog | Upstage Website

Highlights

• Agentic Specialist: Purpose-built for agentic workflows — tool calling, multi-step reasoning, and end-to-end task execution. Competitive with the strongest open-weight models on agent benchmarks.

• Minimal Inference Cost: A 250B-parameter MoE that activates only 15B per token, built on a hybrid attention stack that interleaves three linear-attention layers with one softmax-attention layer.

• 1M-Token Context: The linear-attention layers encode token order intrinsically in their recurrent state, so positional encoding is removed entirely (NoPE), lifting the RoPE extrapolation limit.

• Efficiently Trained at Low Cost: Initialized by selective weight transfer from Solar Open 1 (102B) — only 2.3% of weights survive the architectural change.

• Multilingual: English, Korean, and Japanese.

---

Model Overview

Field	Value
Model Name	Solar Open 2 (250B-A15B)
Architecture	Hybrid-Attention Mixture-of-Experts (MoE)
Total Parameters	250B (250,287,794,944)
Active Parameters	15B (per token)
Layers	48
Hidden Size	4096
Attention	Hybrid — Softmax + Linear Attention, pattern [Softmax, Linear×3] × 12
Position Encoding	NoPE (no rotary positional encoding)
Number of Attention Heads (GQA)	(Softmax) 64 query / 8 KV, (Linear) 64 query
Number of Experts	321 (320 routed + 1 shared)
Number of Activated Experts	8 routed (top-8) + 1 shared
Vocabulary	196,608
Context Length	1M
Pre-training Tokens	~12 Trillion
Supported Languages	English, Korean, Japanese
Training Hardware	NVIDIA B200 GPUs
Training GPU Time	2M GPU Hours
License	Upstage Solar License
Hardware Requirements	Minimum: H200 × 4 / Recommended: H200 × 8

---

Performance

English Benchmarks

Benchmark	Solar Open 2 250B-A15B	Solar Open 100B 102B-A12B	Command A+ 218B-A25B	Mistral Medium 3.5 128B	MiMo-V2.5 310B-A15B	DeepSeek-V4-Flash 284B-A13B
MMLU-Pro	86.2	80.4	79.0	81.2	84.6	85.9
GPQA-Diamond	86.3	66.2	75.6	77.5	83.0	88.9
HLE (w/o tools)	28.8	11.5	11.4	12.8	24.3	32.3
LiveCodeBench (v6)	92.4	56.5	86.1	84.9	89.1	92.3
ArtifactsBench	55.9	43.4	42.8	49.8	59.3	61.0
HMMT2602	93.9	68.9	73.5	62.9	61.4	94.7
AIME2026	95.7	87.7	96.0	89.0	92.3	97.0
Multi-Challenge	61.0	40.5	45.8	49.8	39.0	62.0
IFBench	80.0	57.7	73.9	69.0	67.1	80.3
AA-LCR	62.3	36.0	46.0	61.0	62.7	63.7
SWE-Bench Verified	70.4	15.4	14.4	69.6	73.0	73.8
Terminal Bench Hard	28.3	2.3	25.0	33.3	41.7	34.1
APEX-Agents	16.6	2.4	1.6	6.1	13.4	13.2
MCP-Atlas	58.2	34.4	27.2	30.7	63.9	58.2
τ³ (banking)	19.6	7.4	5.8	5.8	8.7	22.3
GDPval-AA v2 (ELO)	1128	–	712	929	1145	1187

Korean Benchmarks

Benchmark	Solar Open 2 250B-A15B	Solar Open 100B 102B-A12B	MiMo-V2.5 310B-A15B	DeepSeek-V4-Flash 284B-A13B	Claude Haiku 4.5	GPT-5.4 mini
KMMLU-Pro	78.4	64.0	69.1	78.9	67.9	78.1
CLIcK	90.7	78.9	78.4	89.2	53.5	89.6
HAE-RAE v1.1	73.8	73.3	61.7	73.1	38.5	69.4
Ko-AIME'25	97.7	80.0	88.0	98.0	81.7	90.7
HRM8K	92.2	87.6	90.7	93.4	90.6	91.3
KBank-MMLU	80.8	65.5	71.0	79.5	68.9	79.0
KBL	75.5	65.5	69.8	72.8	69.9	75.3
KorMedMCQA	93.0	84.4	87.7	94.1	87.0	94.2
Ko-GDPval	86.8	3.4	81.0	85.0	68.3	59.4

---

Quickstart

Transformers

Install dependencies:

python -m pip install -U \
  "git+https://github.com/upstageAI/transformers.git@v5.14.1-solar-open2" \
  "fla-core[cuda]>=0.5.1" \
  accelerate einops

Run the model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "upstage/Solar-Open2-250B"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=False,
)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    dtype=torch.bfloat16,
    trust_remote_code=False,
)
model.eval()

messages = [
    {"role": "user", "content": "What is Upstage?"},
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    reasoning_effort="high",
    think_render_option="preserved",
)
input_device = model.get_input_embeddings().weight.device
model_inputs = tokenizer(prompt, return_tensors="pt").to(input_device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=32768,
    do_sample=True,
    temperature=1.0,
    top_p=1.0,
)

new_token_ids = generated_ids[0, model_inputs.input_ids.shape[-1] :].tolist()
think_end_id = tokenizer.convert_tokens_to_ids("<|think:end|>")

if think_end_id in new_token_ids:
    answer_start = len(new_token_ids) - new_token_ids[::-1].index(think_end_id)
else:
    answer_start = len(new_token_ids)

reasoning = tokenizer.decode(
    new_token_ids[:answer_start],
    skip_special_tokens=True,
).strip()
answer = tokenizer.decode(
    new_token_ids[answer_start:],
    skip_special_tokens=True,
).strip()

print("[reasoning]", reasoning)
print("[answer]", answer)

Serving with vLLM (Recommended)

Option 1: Docker

docker run --rm --gpus all --ipc=host \
  -p 8000:8000 \
  -v "${HF_HOME:-$HOME/.cache/huggingface}:/root/.cache/huggingface" \
  upstage/vllm-solar-open2 \
  upstage/Solar-Open2-250B \
  --served-model-name solar-open2-250b \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --moe-backend triton \
  --default-chat-template-kwargs '{"think_render_option":"preserved"}' \
  --reasoning-parser solar_open2 \
  --tool-call-parser solar_open2 \
  --enable-auto-tool-choice \
  --logits-processors vllm.v1.sample.logits_processor.solar_open2:SolarOpen2TemplateLogitsProcessor

Option 2: Install from source

pip install -U uv

VLLM_PRECOMPILED_WHEEL_LOCATION="https://github.com/vllm-project/vllm/releases/download/v0.22.0/vllm-0.22.0%2Bcu129-cp38-abi3-manylinux_2_28_x86_64.whl" \
VLLM_USE_PRECOMPILED=1 \
uv pip install --reinstall-package vllm --torch-backend=cu129 \
  "git+https://github.com/UpstageAI/vllm.git@v0.22.0-solar-open2"

Start the server:

vllm serve upstage/Solar-Open2-250B \
  --served-model-name solar-open2-250b \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --moe-backend triton \
  --default-chat-template-kwargs '{"think_render_option":"preserved"}' \
  --reasoning-parser solar_open2 \
  --tool-call-parser solar_open2 \
  --enable-auto-tool-choice \
  --logits-processors vllm.v1.sample.logits_processor.solar_open2:SolarOpen2TemplateLogitsProcessor

Send a chat completion request:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "solar-open2-250b",
    "messages": [
      {"role": "user", "content": "What is Upstage?"}
    ],
    "max_tokens": 131584,
    "temperature": 1.0,
    "top_p": 1.0,
    "reasoning_effort": "high"
  }'

Quantized Versions

Official quantized models by NotaAI are available:

• Solar-Open2-250B-Nota-INT4-GlobalPruned
• Solar-Open2-250B-Nota-NVFP4
• Solar-Open2-250B-Nota-INT4

---

Capabilities

Reasoning

Use reasoning_effort="high" for reasoning and reasoning_effort="none" for a direct response. The recommended vLLM configuration limits a reasoning block to 131,072 tokens.

Effort	Behavior
none	Direct response
high	Reasoning, capped at 131,072 tokens

Example:

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")

response = client.chat.completions.create(
    model="solar-open2-250b",
    messages=[
        {
            "role": "user",
            "content": "Prove that the square root of 2 is irrational.",
        },
    ],
    reasoning_effort="high",
    temperature=1.0,
    top_p=1.0,
    max_tokens=131584,
)

print(response.choices[0].message.reasoning)
print(response.choices[0].message.content)

Tool Calling

Tool calls follow the standard OpenAI function-calling interface. Start the server with --tool-call-parser solar_open2 and --enable-auto-tool-choice.

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"},
                },
                "required": ["location"],
            },
        },
    },
]

response = client.chat.completions.create(
    model="solar-open2-250b",
    messages=[
        {
            "role": "user",
            "content": "What's the weather in Seoul?",
        },
    ],
    tools=tools,
)

print(response.choices[0].message.tool_calls)

---

Agentic Use

Both Anthropic's Claude Code and Nous Research's Hermes Agent can run on Solar Open 2 served locally with vLLM. A single vLLM server exposes both interfaces.

Claude Code

vLLM exposes an Anthropic-compatible /v1/messages endpoint:

export ANTHROPIC_BASE_URL=http://localhost:8000
export ANTHROPIC_AUTH_TOKEN=dummy
export ANTHROPIC_MODEL=solar-open2-250b
export ANTHROPIC_SMALL_FAST_MODEL=solar-open2-250b
claude

Hermes Agent

Register the local vLLM server in ~/.hermes/config.yaml:

model:
  provider: custom
  default: solar-open2-250b
  base_url: http://localhost:8000/v1
  api_key: dummy

---

Best Practices

Recommended client-side generation settings

Solar Open 2 is a reasoning-capable model. Use reasoning_effort="high" for complex or agentic tasks.

Parameter	Recommended	Notes
reasoning_effort	high	Recommended for complex reasoning and agentic tasks
temperature	1.0
top_p	1.0
max_tokens	up to 256K	Covers reasoning + output budget

Recommended settings by reasoning mode

Mode	temperature	top_p	max_tokens
reasoning_effort="none"	1.0	1.0	up to 128K
reasoning_effort="high"	1.0	1.0	up to 256K

Key recommendations:

• Set max_tokens high enough (up to 256K) — reasoning traces can be long and may otherwise truncate the answer.
• The reasoning trace is preserved by default (think_render_option=preserved).
• Multi-turn: Keep prior reasoning traces in the conversation history. The default think_render_option=preserved handles this automatically.
• Parsing: the OpenAI-compatible server returns reasoning in a separate message.reasoning field; with local transformers, split the raw output on the reasoning markers.

---

License

Solar Open 2 is distributed under the Upstage Solar License.

Key requirements for Derivative AI Models:

• Naming: prefix your model name with "Solar" (e.g., Solar-MyModel-v1).
• Attribution: prominently display "Built with Solar" in related public-facing materials.
• Notice: include a copy of the Upstage Solar License with your derivative model.

---

Citation

@misc{solaropen2-2026,
    title={Solar Open 2 Technical Report},
    author={Upstage AI},
    year={2026},
    eprint={2607.20062},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2607.20062}
}