davidnichols-ops/qwen3-0.6b-devin-draft

Qwen3-0.6B Devin Draft (Speculative Decoder)

Draft model for speculative decoding, distilled from the fine-tuned Qwen3-4B Devin SFT target model. The 0.6B model proposes tokens; the 4B target model verifies them, giving 2-3x inference speedup.

Training Details

• Base model: Qwen/Qwen3-0.6B
• Distillation source: davidnichols-ops/qwen3-4b-devin-sft (target model)
• Method: Generated 1,000 greedy-decoding outputs from the target model, then SFT the draft model on those outputs
• Epochs: 3
• Batch size: 32
• Learning rate: 5e-5, cosine schedule
• Precision: BF16
• Hardware: NVIDIA GB10 (128GB VRAM)

Usage with Speculative Decoding

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Target (verifier) model
target = AutoModelForCausalLM.from_pretrained(
    "davidnichols-ops/qwen3-4b-devin-sft",
    dtype=torch.bfloat16, device_map="auto",
)
# Draft (proposer) model
draft = AutoModelForCausalLM.from_pretrained(
    "davidnichols-ops/qwen3-0.6b-devin-draft",
    dtype=torch.bfloat16, device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("davidnichols-ops/qwen3-4b-devin-sft")

# Use with vLLM or transformers speculative decoding
# vLLM: --speculative-model davidnichols-ops/qwen3-0.6b-devin-draft --num-speculative-tokens 5

Pair With

• Target model: davidnichols-ops/qwen3-4b-devin-sft