RSRS64/qwen3-8b-eagle3-rust-mixed

Rust EAGLE-3 — mixed400

Rust/prose trade-off: 15.05% lower fresh-Rust latency vs upstream; directly 1.44% slower Rust and 4.02% faster prose than scale400. JSON recovery was inconclusive.

This is a BF16 EAGLE-3 draft for frozen Qwen3-8B, not a standalone language model. It is not quantized and does not improve the target's coding capability. One transformer block, width 4,096; 399,523,840 stored parameters excluding vocabulary buffers. The upstream draft vocabulary mapping is retained.

Training

Pool: 1,536 Rust + 512 general. Optimizer updates: 400. Seed: 20260918. Batch 1 × accumulation 8; peak LR 1e-5; cosine horizon 400; gradient clip 0.5; training length 7. Checkpoint selected by separate Rust validation simulated acceptance length, not test timing. Target weights remain frozen.

Target: Qwen/Qwen3-8B at b968826d9c46dd6066d109eabc6255188de91218. Warm start: Tengyunw/qwen3_8b_eagle3 at 2a1059d51f622b8cad7d7d72840153ffea5488a0. SpecForge: ed64d275bac8a48e126adb2827368603566a3029; observed SGLang 0.5.18.

Rust prompts: Fortytwo-Network/Strandset-Rust-v1 at 0a8d223302712a2b34a6ad4ce1fd679031894b3d (Apache-2.0). Answers regenerated by Qwen; crate-disjoint splits. Mixed training additionally uses Databricks Dolly 15k at bdd27f4d94b9c1f951818a7da7fd7aeea5dbff1a (CC-BY-SA-3.0), with Qwen-regenerated answers. See included source cards and NOTICE for attribution.

Intended use and limits

Research serving experiments on H100 80GB with the pinned target and compatible SGLang. Tested: BF16, three speculative steps, top-k one, four draft slots, temperature zero, thinking off, prefix cache off, 1,024 output-token cap, concurrency one. See reproduction instructions for the full environment and benchmark workflow.

Results are conditional on small synthetic prompt cohorts, not all Rust programs. Rust-only 400-update candidates slowed a small JSON diagnostic. The mixed candidate did not establish universal non-regression. Pilot batch-concurrency output differences remain unresolved. No functional code benchmark or stochastic-distribution test was performed. Do not treat an acceptance proxy as measured latency or a frozen target as sufficient proof of output parity.

Download and serve

Full experiment and code · Readable write-up · Other main draft

Use a compatible GPU environment with the observed SGLang 0.5.18 stack. The measured hardware was one H100 80GB. This repository is a draft component, not a standalone Transformers text-generation pipeline. The original cached environment was tested; a clean GPU rebuild was not revalidated for this upload.

Download both the frozen target and this draft, verify the original weight checksums, and start the server:

export QWEN_TARGET_PATH="$(hf download Qwen/Qwen3-8B --revision b968826d9c46dd6066d109eabc6255188de91218)"
# This immutable revision contains the released weights.
export RUST_DRAFT_PATH="$(hf download RSRS64/qwen3-8b-eagle3-rust-mixed --revision 49a936f8ee9161811960b0cc88dd8e89e26914d4)"
python - <<'CHECK'
import hashlib, json, os
from pathlib import Path
p = Path(os.environ["RUST_DRAFT_PATH"])
for name, expected in json.loads((p / "SHA256.json").read_text()).items():
    with (p / name).open("rb") as f:
        assert hashlib.file_digest(f, "sha256").hexdigest() == expected, name
print("Draft checksums verified")
CHECK
python -m sglang.launch_server \
  --model-path "$QWEN_TARGET_PATH" --served-model-name rust-target \
  --host 127.0.0.1 --port 30000 --dtype bfloat16 \
  --context-length 8192 --mem-fraction-static 0.7 --random-seed 20260918 \
  --max-running-requests 16 --disable-radix-cache --enable-metrics \
  --speculative-algorithm EAGLE3 \
  --speculative-draft-model-path "$RUST_DRAFT_PATH" \
  --speculative-num-steps 3 --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4

For the measured setting, send requests at concurrency one, temperature zero, with chat_template_kwargs={"enable_thinking": false} and a 1,024-token output cap. The complete benchmark client and fixed prompts are in the experiment repository. Serving requires the target weights and tokenizer downloaded above; no target model files are included here.

Files and terms

model.safetensors: 799,457,128 bytes. SHA-256: c0222c70bfc4471aaaa5c7b7f61dfb867b2191291c2e3a4b1c139a6be0a5084c. config.json and SHA256.json accompany the weights. The checkpoint does not include target weights, tokenizer, optimizer state, or feature tensors.

Derived from the upstream MIT-declared draft; its exact model card and the target's Apache license are included. Original adaptation contributions use the MIT license. Dataset source material retains its own terms; NOTICE.md and licenses/ preserve attribution. No blanket relicensing of third-party material is intended.