Instructions to use DaoCloud/Kimi-K3.1-Pro-pruned75 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DaoCloud/Kimi-K3.1-Pro-pruned75 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="DaoCloud/Kimi-K3.1-Pro-pruned75", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForSeq2SeqLM
model = AutoModelForSeq2SeqLM.from_pretrained("DaoCloud/Kimi-K3.1-Pro-pruned75", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DaoCloud/Kimi-K3.1-Pro-pruned75 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DaoCloud/Kimi-K3.1-Pro-pruned75"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaoCloud/Kimi-K3.1-Pro-pruned75",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/DaoCloud/Kimi-K3.1-Pro-pruned75

SGLang

How to use DaoCloud/Kimi-K3.1-Pro-pruned75 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DaoCloud/Kimi-K3.1-Pro-pruned75" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaoCloud/Kimi-K3.1-Pro-pruned75",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DaoCloud/Kimi-K3.1-Pro-pruned75" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DaoCloud/Kimi-K3.1-Pro-pruned75",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use DaoCloud/Kimi-K3.1-Pro-pruned75 with Docker Model Runner:
```
docker model run hf.co/DaoCloud/Kimi-K3.1-Pro-pruned75
```

Kimi-K3.1-Pro-pruned75

This is a calibration-free, safetensors-only expert-pruned checkpoint derived from moonshotai/Kimi-K3.1-Pro. It removes 75% of the routed experts, following the same method as mgoin/Kimi-K3-pruned75. It is intended for vLLM serving and evaluation smoke tests. Accurate calibration-based pruning, such as REAP in llm-compressor, is a separate line of work.

This is an experimental prototype, not a REAP-pruned checkpoint. Expect a large quality regression relative to the source model.

Pruning

Source experts per MoE layer: 896
Retained experts per MoE layer: 224 (25% retained, 75% removed)
Ranking: descending signed sum(router.weight[i]), independently per layer
Removed data: pruned expert tensors and their corresponding router rows
Router and expert indices were compacted, and num_experts was updated to 224
Calibration data: none
MXFP4 expert weights were copied with their scale/packing intact (not dequantized)
On-disk size: about 443G across 96 safetensors shards plus the weight index

Only routed experts were changed. Dense first-layer MLP, shared experts, attention / DSA / indexer, latent up-down projections, vision weights, and the tokenizer were left untouched.

Validated vLLM run

The checkpoint loaded and served with the default vLLM loader on one node with eight NVIDIA H200 GPUs:

Hardware: 8x NVIDIA H200
Parallelism: TP=8 on one node
Served model name: kimi-k3.1-pro-pruned75
Steady GPU memory: about 127,797 MiB per GPU

vllm serve DaoCloud/Kimi-K3.1-Pro-pruned75 \
  --served-model-name kimi-k3.1-pro-pruned75 \
  --tensor-parallel-size 8 \
  --trust-remote-code \
  --host 0.0.0.0 \
  --port 8000

GET /v1/models returned HTTP 200 after load.

GSM8K

Evaluation used a local OpenAI-compatible /v1/completions server and the full GSM8K test split, aligned with the mgoin harness settings:

1,319 questions, 5-shot, max_tokens=256, temperature=0, seed=42

Checkpoint	Accuracy	Correct	Invalid	Latency	Questions/s
Kimi-K3.1-Pro-pruned75	35.3298%	466/1319	0/1319	541.028 s	2.438

For reference, mgoin/Kimi-K3-pruned75 scored 34.4200% (454/1319) with the same shot/token/temperature/seed settings on the original Kimi-K3 pruned75 checkpoint (different base model and hardware).

License

This derivative is distributed under the upstream Kimi K3.1 Pro License.

Downloads last month: 507

Safetensors

Model size

738B params

Tensor type

F32

BF16

Model tree for DaoCloud/Kimi-K3.1-Pro-pruned75

Base model

moonshotai/Kimi-K3.1-Pro

Finetuned

(15)

this model

Evaluation results

Accuracy on GSM8K
test set self-reported

35.330