Instructions to use alphakek/Qwen3.8-27B-heretic-ara-DFlash2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use alphakek/Qwen3.8-27B-heretic-ara-DFlash2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="alphakek/Qwen3.8-27B-heretic-ara-DFlash2")

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("alphakek/Qwen3.8-27B-heretic-ara-DFlash2")
model = AutoModel.from_pretrained("alphakek/Qwen3.8-27B-heretic-ara-DFlash2", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use alphakek/Qwen3.8-27B-heretic-ara-DFlash2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "alphakek/Qwen3.8-27B-heretic-ara-DFlash2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "alphakek/Qwen3.8-27B-heretic-ara-DFlash2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/alphakek/Qwen3.8-27B-heretic-ara-DFlash2

SGLang

How to use alphakek/Qwen3.8-27B-heretic-ara-DFlash2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "alphakek/Qwen3.8-27B-heretic-ara-DFlash2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "alphakek/Qwen3.8-27B-heretic-ara-DFlash2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "alphakek/Qwen3.8-27B-heretic-ara-DFlash2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "alphakek/Qwen3.8-27B-heretic-ara-DFlash2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use alphakek/Qwen3.8-27B-heretic-ara-DFlash2 with Docker Model Runner:
```
docker model run hf.co/alphakek/Qwen3.8-27B-heretic-ara-DFlash2
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.8-27B-heretic-ara-DFlash2

SpecForge-tuned DFlash 2 drafter for heretic-org/Qwen3.8-27B-heretic-ara.

Target-specific — reads the target's hidden states, so use this only with heretic-ara. For base Qwen/Qwen3.8-27B, use z-lab/Qwen3.8-27B-DFlash2.

Trained on

Target: heretic-org/Qwen3.8-27B-heretic-ara
Method: SpecForge, warm-started from z-lab/Qwen3.8-27B-DFlash2
Data: synthetic multi-turn corpus matching production traffic (reasoning_effort=low, temp 1.0 / top_p 0.95 / top_k 20, tool-call episodes)
Steps: 400 (selected from an 800-step run; step 400 wins on real vLLM bench)

vs stock `z-lab/Qwen3.8-27B-DFlash2`

Controlled back-to-back A/B on Helga (4×3090 TP4, vLLM DFlash2 fork, n6, thinking-low, greedy, 3 prompt types × 2 reps):

checkpoint	tok/s	accept len	Δ tok/s	Δ accept
this drafter (step 400)	123.2	4.24	+5.4%	+5.8%
stock z-lab	116.9	4.01	—	—

Spec-off baseline on the same setup: 38.1 tok/s.

Quick start (vLLM)

vllm serve heretic-org/Qwen3.8-27B-heretic-ara \
  --tensor-parallel-size 4 \
  --reasoning-parser qwen3 \
  --speculative-config '{"model": "alphakek/Qwen3.8-27B-heretic-ara-DFlash2", "method": "dflash", "num_speculative_tokens": 6}'

Requires vLLM with DFlash2 support (PR #52816).

License

Apache-2.0

Downloads last month: 422

Safetensors

Model size

2B params

Tensor type

BF16

Model tree for alphakek/Qwen3.8-27B-heretic-ara-DFlash2

Base model

Qwen/Qwen3.8-27B

Finetuned

heretic-org/Qwen3.8-27B-heretic-ara

Finetuned

(1)

this model

Quantizations