d0rj/GigaChat3-10B-A1.8B.dspark-base-dev

GigaChat3 10B-A1.8B DSpark Base Dev

Experimental development checkpoint. This model is not production-ready. On the tested RTX 5070 Ti setup, its DSpark path does not consistently accelerate the verifier. The checkpoint is published to make the experiment reproducible and to support further adapter and training work, not as a production speedup recommendation.

This repository contains a three-layer DSpark speculative draft model trained for the frozen ai-sage/GigaChat3-10B-A1.8B verifier. It is a speculative head, not a standalone language model.

The companion Large Dev checkpoint has five draft transformer layers. It was initialized from this checkpoint by adding two identity-initialized residual layers and then fine-tuned for one epoch on the same deterministic data snapshot.

The repository includes inference weights, complete optimizer and scheduler state, validation metrics, continuation metadata, exact launchers, TensorBoard events, and raw BS1/BS4 benchmark artifacts.

Model details

Item	Value
Draft architecture	DSparkDraftModel
Verifier	ai-sage/GigaChat3-10B-A1.8B
Draft transformer layers	3
Draft hidden / intermediate size	1,536 / 8,960
Attention	32 query heads, 32 KV heads, head dimension 64
Draft vocabulary	32,000 tokens mapped from the verifier vocabulary
Speculative block	8 positions: one anchor and up to 7 proposed tokens
Runtime proposal range	1-7 tokens per step
Verifier auxiliary states	Layers 2, 7, 13, 19, and 24, plus the normalized final state
Markov head	Vanilla first-order bias, rank 256
Confidence head	Enabled and conditioned on the Markov embedding
Precision	BF16 draft weights; FP8 verifier checkpoint
Released checkpoint	Epoch 0, global step 55,165
Status	Experimental / development only

Usage with vLLM

Install a vLLM 0.23-compatible environment and the GigaChat3 vLLM plugin:

pip install "git+https://github.com/d0rj/gigachat3-vllm-plugin.git"

The plugin is discovered automatically through the vllm.general_plugins entry point; a shell-level VLLM_PLUGINS export is not required.

The explicit verifier-plus-draft form makes the number of proposed tokens visible and configurable:

vllm serve ai-sage/GigaChat3-10B-A1.8B \
  --host 127.0.0.1 \
  --port 8000 \
  --served-model-name gigachat3-dspark-base-dev-k3 \
  --trust-remote-code \
  --linear-backend triton \
  --max-model-len 2048 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --compilation-config '{"cudagraph_mode":"PIECEWISE"}' \
  --speculative-config '{"model":"d0rj/GigaChat3-10B-A1.8B.dspark-base-dev","method":"dflash","num_speculative_tokens":3}'

Set num_speculative_tokens to an integer from 1 through 7. K=3 and K=7 are benchmarked below. PIECEWISE CUDA Graph mode was used for the published measurements; full CUDA Graph capture is not compatible with the tested GigaChat3 MLA path. The Triton linear backend avoids the block-FP8 CUTLASS failure observed on the tested RTX 5070 Ti.

The vLLM method name remains dflash because DSpark uses the DFlash-family scheduler and proposer interface; the plugin dispatches to DSpark from DSparkDraftModel in the checkpoint configuration.

Request example:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gigachat3-dspark-base-dev-k3",
    "messages": [{"role": "user", "content": "Prove the fixed-point theorem."}],
    "max_tokens": 128,
    "temperature": 0
  }'

This checkpoint is not intended to be loaded as a standalone Transformers causal language model. The verifier remains responsible for exact output distribution and speculative verification.

Training

Training used online hidden-state extraction: the frozen verifier generated auxiliary states on demand and the complete hidden-state corpus was not persisted. The verifier and trainable draft shared one RTX 5070 Ti 16 GB GPU under WSL2. Layerwise verifier offload was used during training; it is not part of the benchmark serving configuration.

Data

The training snapshot contains exactly 200,000 deterministically selected rows from t-tech/T-Wix, pinned to revision 0059b0967915624069b5b4dc224ebe7ef7c041a6 and shuffled with seed 42. T-Wix is a Russian/English SFT mixture spanning general instructions, reasoning, mathematics, science, code, dialogue, and long-context tasks.

The prepared snapshot used a 90%/10% train/validation split. Conversations were rendered with the GigaChat3 chat template and limited to 8,192 tokens. The template contributes approximately 992 tokens even for a minimal exchange.

Prepared-data statistic	Value
Rows	200,000
Total tokens	428,427,745
Mean sequence length	2,142.14
Median sequence length	1,924
p95 sequence length	3,536
p99 / maximum	8,192 / 8,192

Exact preprocessing metadata is in training/data_config.json. T-Wix is ODC-BY-1.0 and its card notes that included third-party model outputs may have separate terms.

Hyperparameters

Parameter	Value
Initialization	From scratch
Draft layers	3
Released training duration	1 completed epoch
Wrapper default	2 epochs; the released dev checkpoint is the first completed epoch
Optimizer	AdamW
Learning rate / weight decay	2e-4 / 0.01
Scheduler	Cosine; automatic 1% warmup
Sequence budget	8,192 tokens
Maximum anchors / block size	128 / 8
Draft vocabulary / Markov rank	32,000 / 256
Main loss	Adaptive lk_hybrid objective
Confidence loss weight	0.25
Draft attention implementation	SDPA
Verifier	Frozen
Seed	42
Training revision	4bd884b204230bbbec44dbb6538eb82593c4016f
Software	Speculators 0.7.0.dev72; Transformers 5.12.1; PyTorch 2.12.0

training/train.sh is the three-layer wrapper, while training/common_train.sh contains the complete online extraction and training command. train_command.txt records the resolved command and software versions from the run.

TensorBoard

The Base Dev TensorBoard event stream is included under tensorboard/base_dev:

tensorboard --logdir tensorboard --port 6006

Validation metrics

These are teacher-forced metrics from the released epoch-0 validation pass. Runtime speculative acceptance is lower because every later proposal depends on all preceding draft tokens being accepted.

Metric	Value
Validation loss	1.4354
Teacher-forced acceptance rate	24.83%
Teacher-forced acceptance length	1.995
Full-block token accuracy	34.80%
Position 1 accuracy	62.37%
Position 2 accuracy	43.97%
Position 3 accuracy	34.75%
Position 4 accuracy	29.77%
Position 5 accuracy	26.41%
Position 6 accuracy	24.06%
Position 7 accuracy	22.05%
Confidence loss	0.2306
Confidence absolute error	0.1218

Unrounded values are available in val_metrics.json.

vLLM performance

The benchmark compares the raw verifier with this checkpoint at K=3 and K=7. It uses the same chat-template-rendered prompt snapshots for all runs: 100 prompts each from MATH-500, GSM8K, HumanEval, MBPP, MT-Bench, and UltraChat. Each request uses greedy decoding, seed 0, a 2,048-token model limit, up to 128 output tokens, 4,096 maximum batched tokens, 0.85 GPU memory utilization, chunked prefill, the Triton linear backend, Inductor compilation, and PIECEWISE CUDA Graph mode. vLLM prefix caching remains enabled for every run. Eight warm-up requests are excluded per dataset.

BS1 uses concurrency/max-num-seqs 1; BS4 uses 4. Latency, TTFT, and ITL are reported as mean/p95 milliseconds. Acceptance rate is accepted draft tokens divided by all proposals; p1 is first-position acceptance; acceptance length is 1 + accepted_tokens / draft_steps.

Batch size 1

Dataset	K	Verifier tok/s	DSpark tok/s	Speedup	Latency mean/p95 ms	TTFT mean/p95 ms	ITL mean/p95 ms	Accept rate	p1	Accept len
math500	3	90.95	65.99	0.726x	1932.82/2776.39	102.58/173.02	14.45/21.33	31.00%	52.70%	1.930
math500	7	90.95	68.23	0.750x	1861.43/2443.76	117.49/168.46	13.83/18.88	14.31%	52.83%	2.001
gsm8k	3	90.49	63.80	0.705x	2006.14/2898.44	107.20/170.24	14.95/21.91	28.12%	50.51%	1.843
gsm8k	7	90.49	63.04	0.697x	2011.36/2701.44	106.86/169.17	15.14/20.39	13.30%	50.05%	1.931
humaneval	3	89.12	61.08	0.685x	2046.92/2988.33	86.23/160.50	15.81/23.00	22.24%	42.60%	1.667
humaneval	7	89.12	60.42	0.678x	2058.93/2758.95	104.15/165.66	15.84/20.87	12.12%	44.91%	1.849
mbpp	3	90.61	58.87	0.650x	2169.37/3063.66	101.38/172.03	16.32/23.52	21.65%	42.82%	1.650
mbpp	7	90.61	62.17	0.686x	2019.67/2650.94	116.26/169.23	15.31/20.05	11.19%	44.12%	1.783
mt_bench	3	90.62	56.86	0.627x	1778.19/3102.34	92.20/164.12	17.30/24.22	20.83%	39.47%	1.625
mt_bench	7	90.62	64.19	0.708x	1518.63/2565.23	110.75/169.51	15.36/23.56	11.99%	42.94%	1.839
ultrachat	3	84.34	47.61	0.564x	2292.17/3328.51	115.22/160.20	20.51/28.95	12.83%	28.37%	1.385
ultrachat	7	84.34	50.57	0.600x	2076.80/3346.33	104.64/160.94	19.64/27.69	9.18%	34.77%	1.642

Batch size 4

Dataset	K	Verifier tok/s	DSpark tok/s	Speedup	Latency mean/p95 ms	TTFT mean/p95 ms	ITL mean/p95 ms	Accept rate	p1	Accept len
math500	3	311.77	174.82	0.561x	2900.10/3610.71	169.42/217.69	21.51/26.37	30.14%	52.05%	1.904
math500	7	311.77	173.31	0.556x	2887.25/3454.64	151.32/220.66	21.76/26.17	14.16%	52.34%	1.991
gsm8k	3	296.77	174.22	0.587x	2922.43/3307.05	142.30/199.52	21.90/24.61	28.34%	50.74%	1.850
gsm8k	7	296.77	171.32	0.577x	2939.83/3500.97	153.17/207.46	22.06/26.60	13.66%	51.22%	1.957
humaneval	3	298.25	152.75	0.512x	3261.30/3879.07	142.69/196.08	24.86/29.45	22.11%	42.16%	1.663
humaneval	7	298.25	157.77	0.529x	3109.93/3853.25	150.92/200.86	24.13/29.82	12.04%	43.63%	1.843
mbpp	3	288.52	157.75	0.547x	3210.96/3803.81	141.92/194.40	24.16/29.02	21.58%	42.26%	1.647
mbpp	7	288.52	159.02	0.551x	3149.70/3736.16	149.32/206.61	23.86/28.22	11.22%	45.39%	1.785
mt_bench	3	287.49	148.57	0.517x	2755.77/4079.02	143.24/196.09	25.86/32.55	19.53%	38.19%	1.586
mt_bench	7	287.49	159.61	0.555x	2342.21/3829.04	141.72/199.56	24.51/32.65	11.52%	42.07%	1.807
ultrachat	3	286.59	102.39	0.357x	4176.10/5536.63	177.80/202.66	37.67/44.08	13.25%	28.74%	1.397
ultrachat	7	286.59	109.99	0.384x	3850.86/5520.51	181.99/209.71	35.50/44.89	8.19%	31.61%	1.573

No Base Dev configuration in this matrix exceeded the verifier baseline. The best observed speedup was 0.750x at BS1 and 0.587x at BS4. K=3 produced materially higher proposal acceptance than K=7, but its reduced draft work was still insufficient to offset the external draft and verification overhead.

Training-held-out T-Wix subset

An additional short run uses 50 examples from indices [180000, 200000) of the prepared 200,000-row T-Wix Arrow dataset. The trainer used [0, 180000) for gradient updates, so these prompts were not training examples. They are, however, part of the validation split used for checkpoint selection and must not be interpreted as a fully untouched test set.

For each selected row, the benchmark removes the final supervised assistant answer and retains its chat-formatted prompt prefix. Only prefixes fitting the 1,920-token input budget without truncation are admitted. BS1 and BS4 use the same 50 source indices and prompt SHA-256 202dcb06e266cc65767bb0da94caf6adb3d919526f817336fda90c3381961483; all 250 measured requests in each campaign completed without errors. All other runtime settings match the main benchmark above.

Batch	K	Verifier tok/s	DSpark tok/s	Speedup	Latency mean/p95 ms	TTFT mean/p95 ms	ITL mean/p95 ms	Accept rate	p1	Accept len
1	3	85.23	55.50	0.651x	2227.45/2948.03	116.56/174.74	17.29/22.39	20.81%	40.90%	1.624
1	7	85.23	65.38	0.767x	1876.09/2934.36	94.20/162.12	14.57/22.06	11.58%	43.51%	1.810
4	3	274.53	142.40	0.519x	3441.11/4137.98	152.16/203.73	26.41/31.47	20.27%	39.79%	1.608
4	7	274.53	152.11	0.554x	3190.46/4431.21	156.57/198.09	24.74/33.83	11.95%	45.78%	1.837

The best T-Wix result is 0.767x at BS1/K=7. The in-domain held-out prompts improve neither runtime acceptance nor throughput enough to overcome DSpark overhead.

Full reports, raw request samples, all recorded quantiles, prompt hashes, Prometheus snapshots, run configuration, and server logs are included under benchmarks.

Repository contents

• model.safetensors: selected three-layer DSpark weights.
• config.json, config.py: architecture and verifier linkage.
• optimizer_state_dict.pt: complete AdamW state.
• scheduler_state_dict.pt: scheduler state.
• training_state.json: epoch/global-step continuation metadata.
• train_command.txt: exact resolved training command and package versions.
• val_metrics.json: selected-checkpoint validation metrics.
• training/: dataset metadata and exact launchers.
• tensorboard/: TensorBoard event stream.
• benchmarks/: main BS1/BS4 campaigns plus the 50-prompt training-held-out T-Wix campaigns for the verifier, Base Dev, and Large Dev.

For optimizer-level continuation, restore the root checkpoint files as an epoch directory in a compatible Speculators checkpoint tree.

Limitations

• This is an experimental research artifact and is not production-ready.
• The tested DSpark configuration may be slower than the raw verifier. High training-time teacher-forced accuracy does not guarantee runtime acceleration.
• Performance depends strongly on hardware, workload, batch size, vLLM version, compiler cache state, and proposal count.
• The benchmark retains vLLM prefix caching. GigaChat3 prompts share a large chat-template prefix, so TTFT and aggregate throughput include prefix-cache benefits equally for baseline and DSpark runs.
• The published benchmark covers one RTX 5070 Ti 16 GB GPU under WSL2. Tensor parallelism, pipeline parallelism, multi-GPU serving, and other accelerators were not validated.
• The adapter targets the vLLM 0.23 development API used for these runs.
• GigaChat3 already exposes a native MTP speculative path. Native MTP was not included in this external-DSpark benchmark and should be evaluated for production use.
• K=7 matches the trained seven-proposal block; K=3 reduces draft work but changes the runtime block length.
• Generated outputs inherit the behavior, risks, and limitations of the verifier and T-Wix data.

License and attribution

The verifier is published under the MIT license. This checkpoint is distributed under MIT subject to the verifier's terms and all applicable dataset and third-party-data conditions. T-Wix is licensed under ODC-BY-1.0; consult its dataset card and source-specific terms before downstream use.

Training used the vLLM Speculators project with the GigaChat3 DSpark, online-extraction, confidence/Markov, and vLLM adapter extensions from the accompanying development branch.