Model Card

Model Details

Model Description

Qwen3-4B-EAGLE3 is a lightweight draft model for speculative decoding with Qwen/Qwen3-4B. The model follows EAGLE-3 and adopts a fixed 32K draft vocabulary. It was trained as part of the SpecVocab paper and is used as the autoregressive drafter for Qwen3-4B in WhiFlash.

This model is released for research purposes only.

Trained by: Samsung AI Center, Cambridge
Model type: EAGLE-3 speculative decoding draft model
Target model: Qwen/Qwen3-4B
Inference framework: SGLang
License: Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License

Model Sources

Repository: SamsungLabs/SpecVocab
Paper: Speculative Decoding with a Speculative Vocabulary
WhiFlash: SamsungLabs/WhiFlash

Uses

Direct Use

The model is intended for EAGLE-3 speculative decoding with Qwen/Qwen3-4B. Draft tokens are verified by the target model, preserving the target output distribution under lossless speculative decoding.

Out-of-Scope Use

The model is not intended for standalone generation or use with incompatible target models or tokenizers.

Bias, Risks, and Limitations

When used for lossless speculative decoding, this draft model does not alter the output distribution of the target model and therefore does not introduce additional risks arising from the model output distribution beyond those of Qwen/Qwen3-4B. Users should refer to the Qwen3-4B model card and Qwen3 Technical Report for further information on the target model.

The fixed 32K draft vocabulary reduces projection cost but cannot directly propose target tokens outside this vocabulary, which can reduce acceptance length. End-to-end acceleration depends on the workload, hardware, and speculative decoding configuration.

Training Details

Training Data

Training used approximately 464K UltraChat examples, with assistant responses generated by Qwen/Qwen3-4B.

Training Procedure

The model was trained following the EAGLE-3 training protocol as closely as possible.

Training Hyperparameters

Training steps: 800K
Batch size: 8
Learning rate: 5e-5
Warmup: 1.5% of training steps
Adam beta_1 / beta_2: 0.9 / 0.95
Weight decay: 0
TTT length: 7

Evaluation

Testing Data, Factors & Metrics

Testing Data

We evaluated on Spec-Bench (Xia et al., 2024) in its entirety. Spec-Bench comprises 480 examples across multi-turn conversation, machine translation, summarization, question answering, mathematical reasoning, and retrieval-augmented generation.

Factors

We used 8 decoding steps, top-k 10, and 60 draft tokens. Inference experiments used a single NVIDIA H100 GPU.

Metrics

We report acceptance length (AL) and throughput. Results are averaged over five seeds.

Results

Results on Spec-Bench are reported below.

Category	Acceptance Length	Throughput (tokens/s)
Conversation	5.02	357.0
Machine Translation	3.37	200.2
Summarization	4.31	295.9
Question Answering	4.94	353.9
Mathematical Reasoning	5.79	405.8
Retrieval-Augmented Generation	4.94	344.7
Mean	4.73	326.2 (2.08x)

Technical Specifications

Model Architecture and Objective

Architecture: EAGLE-3 (LlamaForCausalLMEagle3)
Decoder layers: 1
Hidden size: 2560
Attention heads: 32
KV heads: 8
Target vocabulary: 151,936
Draft vocabulary: 32,000
Maximum position embeddings: 40,960

Compute Infrastructure

Hardware

Training used 4 x NVIDIA H100 80GB GPUs.

Citation

SpecVocab

If you use this draft model or report results obtained with this checkpoint, please cite Speculative Decoding with a Speculative Vocabulary.

@inproceedings{williams-etal-2026-speculative,
    title = "Speculative Decoding with a Speculative Vocabulary",
    author = "Williams, Miles and Kwon, Young D. and Li, Rui and Kouris, Alexandros and Venieris, Stylianos I.",
    editor = "Liakata, Maria and Moreira, Viviane P. and Zhang, Jiajun and Jurgens, David",
    booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {ACL} 2026",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.findings-acl.2000/",
    doi = "10.18653/v1/2026.findings-acl.2000",
    pages = "40240--40254",
    ISBN = "979-8-89176-395-1"
}

WhiFlash

If you use this draft model as the autoregressive drafter in WhiFlash, please additionally cite WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routing.

@misc{kwon-etal-2026-whiflash,
    title = {WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routing},
    author = {Young D. Kwon and Miles Williams and Rui Li and Alexandros Kouris and Stylianos I. Venieris},
    year = {2026},
    eprint = {2606.07710},
    archivePrefix = {arXiv},
    primaryClass = {cs.LG},
    url = {https://arxiv.org/abs/2606.07710}
}