Kimi-K3-DSpark-for-8L-dummy

The Inferact/Kimi-K3-DSpark draft config, retargeted at riverclouds/Kimi-K3-8L-dummy for testing. It contains no weights.

The only change is target_layer_ids: [2, 23, 47, 71, 89] of the 93-layer Kimi-K3 becomes [0, 1, 4, 6, 7] of the 8-layer dummy. The draft keeps its 5 layers.

Usage

vllm serve riverclouds/Kimi-K3-8L-dummy \
  --trust-remote-code \
  --load-format dummy \
  --speculative-config '{"method": "dspark", "model": "riverclouds/Kimi-K3-DSpark-for-8L-dummy", "num_speculative_tokens": 7}'

The draft is loaded with the target's --load-format dummy.

License

Distributed under the original Kimi K3 License.