Groot-280B-A16B-261023

Groot is a family of open-weight reasoning models for complex real-world tasks. The models are compact and easy to deploy, strong on reasoning and tool-using workloads, and efficient with token budgets.

Groot-280B-A16B-261023 has 280B total parameters and activates about 16B per token. It starts from DeepSeek-V4.1-Flash-1017 and is tuned for reasoning and tool-using workloads. It has the highest 16-benchmark average of any model here below 552B, ahead of the larger GLM-5.3-Flash (320B) and DeepSeek-V4-Flash-0731 (284B).

Where it runs. We release FP8 (with FP4 experts) and GGUF (UD-Q4_K_XL) weights. The device fits below are estimated from memory capacity. It runs the full 1M context entirely on GPU on 4x 96GB cards (H20, RTX PRO 6000 Blackwell), 4x H200, 8x H100 or 2x B300. Compared with DeepSeek-V4.1-Flash-1017: on 4x 96GB cards Groot runs 1M fully on GPU, while DeepSeek does not fit and needs at least 256GB of host memory for offloading.

Model size vs. capability

The chart compares total model parameters with the unweighted average across all 16 benchmarks. Smaller models appear farther left and higher average scores appear higher. Groot-280B-A16B uses 12.5% fewer total parameters than GLM-5.3-Flash and 1.4% fewer than DeepSeek-V4-Flash-0731. It averages 58.85, compared with 58.78 for GLM-5.3-Flash and 55.25 for DeepSeek-V4-Flash-0731, the highest 16-benchmark average among the models shown below 552B.

Benchmarks

Accuracy. Groot-280B-A16B-261023 beats both GLM-5.3-Flash and DeepSeek-V4-Flash-0731 on 8 of the 16 benchmarks, with the largest gaps on NL2Repo-Bench (56.71 vs 48.57 / 39.20), SWE-bench Pro (63.47 vs 57.59 / 59.37) and AutomationBench (50.61 vs 46.44 / 40.33).

Cost. On DeepSWE 1.1 it scores 64.2% pass@1 at about $0.13 per task, versus 63% at $0.24 for GLM-5.3-Flash and 53% at $0.46 for DeepSeek-V4-Flash: higher accuracy at roughly half the cost or less, because it is a smaller model.

* Each benchmark was evaluated in thinking mode at the maximum reasoning effort (reasoning_effort=100). Unless otherwise stated, sampling uses temp=1.0, top_p=0.95, and a 1M-token context window.
* Terminal-Bench 2.1: Evaluated with the Claude Code agent; max_output_tokens=128K, max_turns=500, 12h timeout; temp=1.0, top_p=0.95.
* ALE Agent: Evaluated with ale_claude on top of Claude Code 2.1.170; max_output_tokens=128K, 4h timeout (extended up to 8h for specific long-running tasks); temp=1.0, top_p=0.95.
* Toolathlon Verified: Reported as pass@1, averaged over 3 independent runs; temp=1.0, top_p=0.95.
* AutomationBench: Evaluated on AutomationBench v1.0.6; max_turns=50; temp=1.0, top_p=0.95.
* JobBench: max_tokens=200K, 600s timeout; temp=1.0, top_p=0.95.
* DeepSWE 1.1: Evaluated with the mini-SWE-agent scaffold; 11h timeout; temp=1.0, top_p=0.95.
* SWE-bench Pro: Evaluated with the SWE-agent scaffold; temp=1.0, top_p=0.95.
* NL2Repo-Bench: Evaluated with the Claude Code agent; max_turns=1000, 80K context window; temp=1.0, top_p=0.95.
* HLE (with tools): Evaluated with gpt-5.5 as the judge model; max_iterations=1000, 6h timeout, 256K context window; temp=1.0, top_p=0.95.

---

Model Overview

• Type: Natively multimodal Mixture-of-Experts causal language model
• Total Parameters: ~280B backbone, plus 197B Engram conditional memory and 14B of MTP and DSpark draft layers
• Activated Parameters: ~8B per token in prefill, ~16B in decode (causal encoder and decoder, as in DeepSeek-V4.1-Flash)
• Layers: 40 (20-layer causal encoder + 20-layer decoder)
• Hidden Size: 5120
• Experts: 192 routed + 1 shared, 6 routed experts activated per token
• Context Length: 1M tokens
• Precision: FP8 (32x32 block scales, UE8M0) with FP4 expert weights; a GGUF release is also available
• Multi-Token Prediction: 3 layers, plus the DSpark speculative-decoding module

The Engram memory (197B parameters, about 197 GB in FP8) is a token lookup table. It adds no activated compute and is included in the release size below.

Available editions:

Edition	Precision	Size on disk
Groot-280B-A16B-261023	FP8 + FP4 experts	365.8 GB
Groot-280B-A16B-261023-GGUF	UD-Q4_K_XL	356.2 GB

---

Quickstart

Groot-280B-A16B-261023 is a mixture-of-experts model with ~280B total parameters and ~16B activated parameters per token, supporting contexts of up to one million tokens.

The model operates in thinking mode by default and supports a continuously controllable reasoning effort (integer 1 to 100) that trades inference cost for accuracy. All reported results use the maximum effort setting (reasoning_effort=100).

Recommended sampling parameters:

Parameter	Value
temperature	1.0
top_p	0.95 or 1.0
context_window	1M tokens
max_tokens	≥ 256K

For local deployment, we recommend temperature=1.0, top_p=1.0.

Chat Template Options

The chat template accepts the following extra kwargs (see chat_template_jinja in config.json):

Kwarg	Values	Default
thinking_mode	"thinking" | "chat" (enable_thinking is accepted as an alias)	"thinking"
reasoning_effort	integer in [1, 100], or "low" | "medium" | "high" | "max"	"high"
drop_thinking	true | false	true

The string efforts map to budgets low=50, medium=62, high=75, max=100. Assistant reasoning is read from reasoning_content (or thinking as an alias).

---

Serving Groot-280B-A16B-261023

vLLM

vllm serve Groot/Groot-280B-A16B-261023 \
    --served-model-name Groot-280B-A16B-261023 \
    --tensor-parallel-size 8 \
    --max-model-len 1048576 \
    --enable-auto-tool-choice \
    --tool-call-parser deepseek_v41 \
    --reasoning-parser deepseek_v41

SGLang

python -m sglang.launch_server \
    --model Groot/Groot-280B-A16B-261023 \
    --tp 8 \
    --context-length 1048576 \
    --reasoning-parser deepseek-v41 \
    --tool-call-parser deepseekv41

For large-scale deployments, data-parallel attention can be enabled as in the DeepSeek recipes:

python -m sglang.launch_server \
    --model Groot/Groot-280B-A16B-261023 \
    --tp 8 --dp 8 --enable-dp-attention

---

Basic Usage

The Chat Completions API can be used with most inference frameworks. Before starting, make sure the SDK is installed and the API key and the API base URL are configured:

pip install -U openai

# Set the following accordingly
export OPENAI_BASE_URL="http://localhost:8000/v1"
export OPENAI_API_KEY="EMPTY"

from openai import OpenAI

# Configured by environment variables
client = OpenAI()

messages = [
    {"role": "user", "content": "Write a Python function to merge two sorted linked lists."},
]

chat_response = client.chat.completions.create(
    model="Groot-280B-A16B-261023",
    messages=messages,
    temperature=1.0,
    top_p=0.95,
    max_tokens=262144,
    extra_body={
        "chat_template_kwargs": {
            "enable_thinking": True,   # thinking mode; set False for non-thinking
            "reasoning_effort": 100,   # integer budget in [1, 100]; "low"/"medium"/"high"/"max" also accepted
            "drop_thinking": True,
        },
    },
)

message = chat_response.choices[0].message
# reasoning_content holds the thinking trace; content holds the final answer.
print("reasoning:", getattr(message, "reasoning_content", None))
print("answer:", message.content)

Reasoning effort can also be passed as low, high, xhigh, max, or a float in [0, 0.99], depending on the serving framework; integer budgets in [1, 100] go through chat_template_kwargs.

---

Chat Template

The chat template ships with the checkpoint (chat_template.jinja, also embedded in config.json as chat_template_jinja), so messages can be encoded directly with Transformers, with no separate encoding script:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "Groot/Groot-280B-A16B-261023"
tokenizer = AutoTokenizer.from_pretrained(model_name)

messages = [
    {"role": "user", "content": "Write a Python function to merge two sorted linked lists."},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    chat_template_kwargs={
        "enable_thinking": True,   # thinking mode; set False for non-thinking
        "reasoning_effort": 100,   # integer budget in [1, 100], or "low"/"medium"/"high"/"max"
    },
)

model = AutoModelForCausalLM.from_pretrained(model_name, dtype="auto", device_map="auto")
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=8192, temperature=1.0, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))

Assistant reasoning is emitted inside the generated text; serving frameworks with a reasoning parser (see --reasoning-parser above) surface it in the separate reasoning_content field instead.

---

Citation

If you use this model in your work, please cite it as:

@misc{groot2026,
  title  = {Scaling Down LLMs for Broader Deployment},
  author = {The Groot Team},
  year   = {2026},
}