IFM/K2-Horizon-32B-GGUF

K2-Horizon-32B-Stage1-GGUF

This repository contains GGUF versions of the IFM/K2-Horizon-32B for use with llama.cpp.

The model tensors are stored in their original BF16 precision. The GGUF files include the tokenizer metadata and a llama.cpp-compatible chat template.

Compatibility: These models require a version of llama.cpp containing K2 Horizon architecture support. PR to llama.cpp is in progress. MBZUAI-IFM fork of llama.cpp is in https://github.com/MBZUAI-IFM/llama.cpp/tree/model/K2Horizon

K2-Horizon-32B-Stage1 is the large dense member of the K2-Horizon family: a 32B decoder-only model with a 512K context window. Note: final checkpoint to be released.

K2-Horizon-32B-Stage1 Highlights

• Strong dense baseline. A 32B dense model evaluated on the same agentic, coding, and reasoning benchmarks as the rest of the family (see Benchmark Results). Results are for stage 1 of the final model training; results for stage 2 will be out soon.
• 512K context. Native 524,288-token context from the midtraining stages onward.
• Intermediate checkpoints. Intermediate checkpoints will be released so capability changes can be studied across training rather than at a single checkpoint.
• Fully open. Training data/recipe and the training code will be made public.

Benchmark Results

		Open-weight dense models
	K2-Horizon-32B-Stage1	Qwen3.8-27B	Muse Glimmer-30B	IBM Granite 4.2 30B
# Params	32B	27B	30B	30B
# Activated params	32B	27B	30B	30B
Architecture	Dense	Dense	Dense	Dense
Agents
tau3-Banking Agentic tool use	22.5	48.0	23.5	14.4
Coding
Terminal-Bench 2.1 Agentic terminal use	36.6	79.8	51.7	26.6
SciCode Scientific coding	30.2	44.7	43.6	36.6
Scientific Reasoning
Humanity's Last Exam (without tools) Expert-level reasoning	22.8	33.9	22.0	11.2
GPQA Diamond Graduate-level science QA	82.3	90.5	83.5	64.4
CritPt Frontier physics reasoning	1.4	5.4	2.6	0.3
General
AA-LCR Long-context reasoning	65.3	77.3	80.0	46.7
AA-Omniscience Accuracy Factual accuracy	16.8	15.6	27.0	10.1
AA-Omniscience Non-Hallucination Non-hallucination rate	58.3	69.7	18.1	74.4

Scores in %. Bold marks the best score in each row. Sections follow the Artificial Analysis Intelligence Index categories. Baseline scores are from Artificial Analysis; Muse Glimmer-30B at high reasoning effort, other open models in their reasoning mode.

Quickstart

Serving

vLLM, recipe at recipes.vllm.ai/IFM:

vllm serve IFM/K2-Horizon-32B \
  --revision main \
  --model-impl vllm \
  --tensor-parallel-size 2 \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --reasoning-parser k2_horizon \
  --enable-auto-tool-choice \
  --tool-call-parser k2_horizon

SGLang recipe validated on 2× H200 in the SGLang K2 Horizon cookbook:

python3 -m sglang.launch_server \
  --model-path IFM/K2-Horizon-32B \
  --revision main \
  --tp 2 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --reasoning-parser k2_horizon \
  --tool-call-parser k2_horizon \
  --host 0.0.0.0 --port 30000

API Usage

Recommended settings: reasoning_effort="high", temperature=1.0, top_p=0.95. Reasoning depth is selected per request through chat_template_kwargs. Thinking is returned in reasoning_content and the answer in content.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="IFM/K2-Horizon-32B",
    messages=[{"role": "user", "content": "Explain the result step by step."}],
    temperature=1.0,
    top_p=0.95,
    max_tokens=32768,
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high", "tool_call_format": "xml"}},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)

Our model supports multiple tool calls formats, which can be changed with chat_template_kwargs. The supported values are json, xml, and xml_typed . The default is xml. Keep --tool-call-parser k2_horizon enabled to parse the selected format.

Transformers

Validated with Transformers 5.15.0, PyTorch 2.13.0, Safetensors 0.8.0.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "IFM/K2-Horizon-32B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
)

inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device)
inputs.pop("token_type_ids", None)
outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Best Practices

1. Reasoning effort: always high. All reported results use high reasoning effort. Pass {"chat_template_kwargs": {"reasoning_effort": "high"}} on every request.
2. Sampling parameters. temperature=1.0, top_p=0.95.
3. Serving. Use the validated SGLang recipe above: BF16, TP=2, FlashAttention-3. Full recipes for every K2-Horizon size, with measured H200 latency and throughput, are in the SGLang cookbook and the vLLM recipe.
4. Parsers. Enable the k2_horizon reasoning parser for chat, and add the k2_horizon tool-call parser for agent use. Leave both off for plain completion-style generation.

Citation

@misc{k2horizon2026,
  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
  author = {{IFM Team}},
  year   = {2026},
  url    = {https://ifm.ai/blog/k2/},
}