Kolibri-1 DSpark

DSpark draft model for speculative decoding of Aleph-Alpha/Kolibri-1.


Target	Aleph-Alpha/Kolibri-1 (78B MoE, FP8)
Draft layers	5 (hidden 2560, 20 heads / 4 KV heads, sliding window 2048)
Block size	8 (anchor + 7 draft tokens)
Target layers tapped	4, 14, 24, 34, 44
Vocabulary	128,000
Size	~826 MB, bf16

Benchmarks

4x H200, tensor-parallel 4, FP8 weights and KV cache, 7 draft tokens, chat template with thinking on. Datasets: MT-Bench, GSM8K, HumanEval, MATH-500, MBPP, TAT-QA, FinanceReasoning and FinQA. Speedup is DSpark output tok/s over the no-speculation baseline on the same engine.

Benchmark	Baseline tok/s	DSpark tok/s	Speedup	Accept length
MT-Bench	188.9	346.1	1.83x	2.98
GSM8K	196.3	509.1	2.59x	3.94
HumanEval	196.5	446.0	2.27x	3.55
MATH-500	197.0	526.8	2.67x	4.25
MBPP	184.2	454.3	2.47x	3.54
TAT-QA	181.8	472.4	2.60x	3.78
FinanceReasoning	183.2	490.3	2.68x	4.12
FinQA	183.3	480.0	2.62x	3.98
Mean			2.47x	3.77

Accept length is tokens produced per target forward pass, including the bonus token.

Usage

Requires vLLM 0.29 with the aleph-alpha-inference plugin. vLLM loads this checkpoint natively.

pip install 'aleph-alpha-inference>=1'

vllm serve Aleph-Alpha/Kolibri-1 \
  --trust-remote-code \
  --tensor-parallel-size 4 \
  --kv-cache-dtype fp8 \
  --max-model-len 32768 \
  --enable-prefix-caching \
  --speculative-config '{"method": "dspark", "model": "intuit/Kolibri-1-dspark", "num_speculative_tokens": 7}'

Offline:

from vllm import LLM, SamplingParams

llm = LLM(
    model="Aleph-Alpha/Kolibri-1",
    trust_remote_code=True,
    tensor_parallel_size=4,
    kv_cache_dtype="fp8",
    max_model_len=32768,
    enable_prefix_caching=True,
    speculative_config={"method": "dspark", "model": "intuit/Kolibri-1-dspark", "num_speculative_tokens": 7},
)
out = llm.chat([{"role": "user", "content": "What is 17 * 23?"}], SamplingParams(temperature=0, max_tokens=512))

Training

We trained this DSpark draft model using speculators 0.8.0 on mlabonne/open-perfectblend plus finance datasets. All responses were regenerated by Kolibri-1, so the draft learns from the target model's own outputs.