freakyskittle/kimi-k2.5-dflash.gguf

Kimi K2.5 DFlash (GGUF)

This repository contains a GGUF format conversion of the Kimi K2.5 DFlash speculative decoding draft model, including quantized versions.

Model Details

Property	Value
Base Model	z-lab/Kimi-K2.5-DFlash
Architecture	DFlashDraftModel
Format	GGUF v3 (float32 + quantized)
Original Dtype	bfloat16 (converted to float32 for GGUF compatibility)
Hidden Size	7,168
Layers	6
Attention Heads	64 (Q) / 8 (KV)
Intermediate Size	18,432
Vocab Size	163,840
Max Position Embeddings	262,144
RoPE Type	YaRN (factor=64, theta=50k)
Tensor Count	69

Files

File	Size	Quantization	Description
Kimi-K2.5-DFlash.gguf	~13.9 GB	F32 (float32)	Full precision base model
Kimi-K2.5-DFlash-Q8_0.gguf	~3.5 GB	Q8_0	4x smaller, high quality
Kimi-K2.5-DFlash-Q5_0.gguf	~2.3 GB	Q5_0	6x smaller, good quality
Kimi-K2.5-DFlash-Q4_0.gguf	~1.9 GB	Q4_0	7x smaller, fastest inference

What is DFlash?

DFlash (DeepFlash) is a speculative decoding technique that uses a lightweight draft model to predict tokens, which are then verified by the full target model (Kimi K2.6 / K2.5). This speeds up inference by 2-3x on average.

Usage

With oxidize (Rust inference engine)

The GGUF files are compatible with the oxidize inference engine (PR #1+), which has native DFlash speculative decoding support.

oxidize --model /path/to/Kimi-K2.5-DFlash-Q4_0.gguf --speculative-config ...

With llama.cpp / other GGUF tools

This model uses the custom dflash-draft architecture. Standard llama.cpp does not natively support DFlashDraftModel inference. Use oxidize or wait for upstream support.

Conversion & Quantization Details

• Source: z-lab/Kimi-K2.5-DFlash (SafeTensors, bfloat16)
• Converter: Custom Python script using gguf library + safetensors.torch
• Quantizer: Python script using gguf.quants.quantize() with GGMLQuantizationType
• Weight Mapping: HuggingFace names (layers.N.self_attn.q_proj.weight) → oxidize GGUF names (blk.N.attn_q.weight)
• Metadata: Includes general.architecture=dflash-draft and all model hyperparameters
• 1D tensors (norm weights, biases) are kept at F32 for accuracy

Limitations

• No lm_head or embed_tokens weights: DFlash draft models reuse the target model's token embeddings
• This is a draft model only — it requires a full target model (e.g. moonshotai/Kimi-K2.6) for speculative decoding

License

Apache 2.0 — same as the original z-lab/Kimi-K2.5-DFlash model.