Instructions to use kueizen/Marco-Mini-Instruct-REAP20 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kueizen/Marco-Mini-Instruct-REAP20 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kueizen/Marco-Mini-Instruct-REAP20")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kueizen/Marco-Mini-Instruct-REAP20")
model = AutoModelForCausalLM.from_pretrained("kueizen/Marco-Mini-Instruct-REAP20", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kueizen/Marco-Mini-Instruct-REAP20 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kueizen/Marco-Mini-Instruct-REAP20"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kueizen/Marco-Mini-Instruct-REAP20",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kueizen/Marco-Mini-Instruct-REAP20

SGLang

How to use kueizen/Marco-Mini-Instruct-REAP20 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kueizen/Marco-Mini-Instruct-REAP20" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kueizen/Marco-Mini-Instruct-REAP20",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kueizen/Marco-Mini-Instruct-REAP20" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kueizen/Marco-Mini-Instruct-REAP20",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use kueizen/Marco-Mini-Instruct-REAP20 with Docker Model Runner:
```
docker model run hf.co/kueizen/Marco-Mini-Instruct-REAP20
```

Marco-Mini-Instruct-REAP20

Marco-Mini-Instruct (17.3B total, ~0.86B active, 256 experts, 8 active per token) with 20% of its experts removed by REAP: 205 of 256 experts kept. These are the full-precision (bf16) safetensors weights, for fine-tuning, re-quantising or running with transformers.

For ready-to-run quantised files at every pruning ratio, see kueizen/Marco-Mini-Instruct-REAP-GGUF.

Load it

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "kueizen/Marco-Mini-Instruct-REAP20"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")

How this was made

Pruning: REAP (Router-weighted Expert Activation Pruning, Cerebras Research), using the reference implementation. REAP scores each expert by its router weight times the size of its output over a calibration set, then removes the lowest-scoring experts whole. It is one-shot: no retraining. Seed 42.
Calibration set: theblackcat102/evol-codealpaca-v1 (train split, shuffled, seed 42), 64 samples per category, batch size 1, max sequence length 2048 tokens.
Experts removed: 20% (205 of 256 kept).

We have not run task benchmarks on these checkpoints (MMLU, coding, multilingual). Test them on your own workload before relying on them.

License and credits

Derived from ATH-MaaS/Marco-Mini-Instruct and released under the same Apache 2.0 licence. What we changed: removed experts with REAP. Nothing else was modified or retrained. REAP is by Cerebras Research: paper, code. All credit for the base model goes to its authors.

Published by Kueizen.

Downloads last month: -

Safetensors

Model size

14B params

Tensor type

BF16

Model tree for kueizen/Marco-Mini-Instruct-REAP20

Base model

ATH-MaaS/Marco-Mini-Instruct

Finetuned

(7)

this model

Paper for kueizen/Marco-Mini-Instruct-REAP20

REAP the Experts: Why Pruning Prevails for One-Shot MoE compression

Paper • 2510.13999 • Published Oct 15, 2025 • 23