Instructions to use RadixArk/Qwen3.8-27B-DSpark with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RadixArk/Qwen3.8-27B-DSpark with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RadixArk/Qwen3.8-27B-DSpark", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("RadixArk/Qwen3.8-27B-DSpark", trust_remote_code=True)
model = AutoModel.from_pretrained("RadixArk/Qwen3.8-27B-DSpark", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use RadixArk/Qwen3.8-27B-DSpark with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RadixArk/Qwen3.8-27B-DSpark"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RadixArk/Qwen3.8-27B-DSpark",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/RadixArk/Qwen3.8-27B-DSpark

SGLang

How to use RadixArk/Qwen3.8-27B-DSpark with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RadixArk/Qwen3.8-27B-DSpark" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RadixArk/Qwen3.8-27B-DSpark",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RadixArk/Qwen3.8-27B-DSpark" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RadixArk/Qwen3.8-27B-DSpark",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use RadixArk/Qwen3.8-27B-DSpark with Docker Model Runner:
```
docker model run hf.co/RadixArk/Qwen3.8-27B-DSpark
```

spec_generate in dflash.py reads only 6 of 7 noise rows

by zxytim - opened 6 days ago

Discussion

zxytim

6 days ago

Hi, I think there's a small but costly bug in the reference spec_generate in dflash.py.

The draft logits are sliced as:

draft_logits = target.lm_head(
    self(... )[:, -block_size + 1 :, :]
)

so only rows 1..6 of the 7-row noise block produce proposals (6 draft tokens, verify width 7+bonus). But the model card says "block size 7 draft tokens (verify width 8, including the target bonus token)" — i.e. the anchor row (block position 0) should also predict the next token, giving 7 draft tokens per iteration. All 7 noise rows should be read (row j predicts position start+j+1).

With the 6-of-7 readout, proposals come out shifted by one position and measured acceptance drops far below the card's numbers (roughly ~1.7 vs ~4.5 on GSM8K-style prompts in my tests). Switching to all-7-rows readout brings acceptance right into the card's range.

cm00cm

RadixArk org 6 days ago

@Jiminator

yisun0618

6 days ago

Thank you for catching this. The checkpoint and current SGLang runtime use block_size=gamma=7 proposed draft tokens and an 8-token verify window.

I opened PR #2 to align the reference spec_generate with that contract. The patch:

keeps the 7-row noise block;
reads all 7 draft logits;
verifies anchor + 7 drafts;
reserves the complete target-bonus slot at the output-buffer boundary.

A targeted H100 contract probe observed 7 noise -> 6 logits -> verify width 7 before the patch and 7 noise -> 7 logits -> verify width 8 after it. A live SGLang load of this checkpoint independently resolved gamma=7 and verify_num_draft_tokens=8.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment