abacusai/Smaug-Flash

---

1. Model Introduction

Smaug-Flash is an agentic-coding finetune of DeepSeek-V4-Flash-0731, finetuned by Abacus.AI. It improves end-to-end agentic performance by a wide margin (+14.3 on LiveBench agentic-coding, +10.1 on Terminal Bench 2.1 under the DeepSeek harness, +13.7 on AutomationBench, +19 on NL2Repo-Bench) while improving general capability (+3.2 overall LiveBench). The weights load exactly like the official release: same layout, same quantization formats, same context length (1M), same reasoning interface.

This card describes the training approach and the evaluation results. Dataset contents are not disclosed; training data consists of curated agentic loop traces from various sources - public repositories, traces synthesized from real world agentic product use cases, augmented with synthetic data variations.

2. Model Summary

Architecture	Mixture-of-Experts (MoE)
Number of Layers	43
Attention Mechanism	MLA (low-rank q/o factors) with sparse token indexer
Attention Hidden Dimension	4096
Number of Attention Heads	64
Number of Routed Experts	256
Selected Experts per Token	6
Number of Shared Experts	1
MoE Hidden Dimension (per Expert)	2048
Vocabulary Size	~129K
Context Length	1,048,576
Speculative Decoding	DSpark multi-token module (inherited, fully functional)
Quantization	Block-FP8 attention (e4m3, 128×128) / packed-FP4 experts
Modality	Text
Base Model	deepseek-ai/DeepSeek-V4-Flash-0731
Adaptation	SFT + preference LoRA adapters, merged as full deltas (attention-only)

3. Evaluation

Measured on fresh 2026 task generations, self-served, with agentic benchmarks run via their public harnesses and the base model run through identical harnesses for every paired comparison.

Smaug-Flash vs its base

Higher is better. Unstarred rows are paired runs under the same harness; in starred rows the base score is the vendor-reported number for the identical public task set.

LiveBench category profile

Scores 0–100; overall = mean of the seven category averages. Same harness, same questions (LiveBench 2026-06-25 release), paired against the base model.

4. Training Approach

Smaug-Flash was trained to make long context agentic loops faster and less prone to spins and stalls in max reasoning mode. We curated trajectories from real world use cases along with generated synthetic variants to address various issues around agentic tool use.

The model is the composition of three LoRA adapters trained in sequence and merged as full deltas - 2 SFT steps and a KTO step. Only the MLA attention factor matrices (129 in total) are adapted; experts, router, embeddings, and the speculative-decoding module remain byte-identical to the official release, packed FP4 included. Adapter deltas are applied in full (no rank truncation) and touched matrices are requantized to the checkpoint-native block-FP8 format. Because only attention changes, any serving stack that runs the official release runs Smaug-Flash unmodified — FP4 expert kernels and speculative decoding paths included. One caution: serve in the checkpoint-native format; requantizing merged weights into a different scheme at load time measurably degrades constrained instruction-following.

5. Known Behaviors and Limitations

Smaug-Flash is more decisive than its base: it finishes tasks in fewer, denser turns and submits sooner. Under tight wall-clock budgets this is an asset (it wins several timeout-bound tasks the base loses); under hidden-test grading it can submit at ~98% spec coverage without spending remaining budget on self-verification. If your use case rewards exhaustive verification over decisiveness, prompt for explicit self-testing before completion.

6. Deployment

• SGLang ≥ 0.5.15 on Blackwell: tp=4/8 with --enable-dp-attention, --moe-runner-backend flashinfer_mxfp4 (required for the packed-FP4 experts), fp8 KV cache, full 1M context.
• Reasoning effort max; sampling temperature 1.0, top-p 0.95.
• For agentic harnesses, note that wall-clock task budgets are effectively inference-compute budgets: serving throughput changes measured scores on timeout-bounded benchmarks. Benchmark on serving comparable to what you deploy.

Chat template and encoding

Like the base release, this repository ships no Jinja chat template. The encoding folder contains Python scripts and test cases for encoding OpenAI-format messages into model input strings and parsing model output. The reasoning_effort parameter supports low, high, and max.

from encoding_dsv4 import encode_messages, parse_message_from_completion_text

messages = [
    {"role": "user", "content": "hello"},
    {"role": "assistant", "content": "Hello!", "reasoning_content": "thinking..."},
    {"role": "user", "content": "1+1=?"}
]

prompt = encode_messages(messages, thinking_mode="thinking", reasoning_effort="max")

import transformers
tokenizer = transformers.AutoTokenizer.from_pretrained("abacusai/Smaug-Flash")
tokens = tokenizer.encode(prompt)

Running with vLLM

Because only attention matrices differ from the official release (§4), any stack that serves DeepSeek-V4-Flash-0731 serves Smaug-Flash unmodified, DSpark speculative decoding included. Example, single 4×GB300 node (see the base model's vLLM recipe for other hardware):

vllm serve abacusai/Smaug-Flash \
  --trust-remote-code --kv-cache-dtype fp8 --block-size 256 \
  --data-parallel-size 4 --enable-expert-parallel \
  --moe-backend deep_gemm_mega_moe \
  --attention-config '{"use_fp4_indexer_cache": true}' \
  --speculative-config '{"method":"dspark","num_speculative_tokens":7,"draft_sample_method":"greedy"}'

Running with SGLang

sglang serve \
  --trust-remote-code \
  --model-path abacusai/Smaug-Flash \
  --tp 4 \
  --enable-dp-attention \
  --moe-runner-backend flashinfer_mxfp4 \
  --speculative-algorithm DSPARK \
  --mem-fraction-static 0.90 \
  --chunked-prefill-size 4096 \
  --swa-full-tokens-ratio 0.1

For the high and max reasoning effort levels we recommend a maximum output length of 384K tokens. Local weight conversion and interactive demos: see the inference folder.

7. License

This repository and the model weights are licensed under the MIT License, matching the base release.

8. Citation

@misc{abacusai2026smaugflash,
  title  = {Smaug-Flash},
  author = {Abacus.AI},
  year   = {2026},
  note   = {Agentic fine-tune of deepseek-ai/DeepSeek-V4-Flash-0731},
  url    = {https://huggingface.co/abacusai/Smaug-Flash}
}

Smaug-Flash builds on DeepSeek-V4-Flash-0731 by DeepSeek-AI:

@misc{deepseekai2026deepseekv4,
      title={DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence},
      author={DeepSeek-AI},
      year={2026},
}

9. Contact Us

If you have any questions, please reach out at Abacus.AI.