Instructions to use 0ppxnhximxr/Motif-3-Beta-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 0ppxnhximxr/Motif-3-Beta-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="0ppxnhximxr/Motif-3-Beta-FP8", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("0ppxnhximxr/Motif-3-Beta-FP8", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 0ppxnhximxr/Motif-3-Beta-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "0ppxnhximxr/Motif-3-Beta-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0ppxnhximxr/Motif-3-Beta-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/0ppxnhximxr/Motif-3-Beta-FP8

SGLang

How to use 0ppxnhximxr/Motif-3-Beta-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "0ppxnhximxr/Motif-3-Beta-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0ppxnhximxr/Motif-3-Beta-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "0ppxnhximxr/Motif-3-Beta-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0ppxnhximxr/Motif-3-Beta-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 0ppxnhximxr/Motif-3-Beta-FP8 with Docker Model Runner:
```
docker model run hf.co/0ppxnhximxr/Motif-3-Beta-FP8
```

Motif-3-Beta-FP8

Warning: the upstream HF inference path is broken (acknowledged by the vendor on 2026-07-22: "HF.generate currently has some issues"). The original checkpoint produces incoherent output before any quantization is applied; the released modeling code and weights most likely belong to different revisions. This repository is a reference implementation of expert-only block-FP8 quantization for research purposes, not a usable text-generation model until upstream ships matching code.

Expert-only FP8 (E4M3, 128x128 block) quantization of Motif-Technologies/Motif-3-Beta (commit 32b0305): 630 GB BF16 -> 322 GB, with 97.84% of parameters quantized and routing decisions bit-identical to the original.

Model Overview


Base model	`Motif-Technologies/Motif-3-Beta` (preview checkpoint)
Architecture	MoE causal LM, ~314B total / ~13B active per token
Layers	53 (51 MoE + 2 dense), 384 routed experts (top-8) + 1 shared
Quantization	weight-only FP8 E4M3, 128x128 block scales (W8A16)
Schema	DeepSeek-V3 compatible (`quant_method: fp8`, FP32 `<param>_scale_inv`)
Size	322 GB, 155 safetensors shards

Quantized: moe.experts.gate_up_proj and moe.experts.down_proj of all 51 MoE layers (308.0B parameters). Preserved in BF16: attention (GDLA), mHC, per-expert PolyNorm coefficients, router, shared experts, dense MLP, MTP, embeddings, lm_head.

Fidelity

All comparisons are FP8 (dequantized) vs. the original BF16 checkpoint.

Level	Method	Result
Weights	full round-trip, all 155 shards	rel RMSE 2.65%, cosine 0.99965, zero NaN/Inf
Expert forward	PolyNorm included	rel error 4.5-5.6%, no amplification
MoE layer	all 51 layers	rel error 3.6% mean; routing bit-identical 51/51

The error is uniform across layers -- no fragile layers or experts.

End-to-end: cannot be certified -- the BF16 baseline itself is broken upstream. Under an identical serving pipeline (2x H100), original BF16 scores NLL 20.06 and this FP8 scores 20.95, failing identically. The breakage is therefore attributable to the upstream checkpoint, not to quantization.

Note: e2e figures were measured under Transformers' default grouped_mm experts backend, which is affected by the dispatch bug below (expert 0's PolyNorm coefficients applied to all experts) -- treat them as indicative only. The weight / expert / layer results above are unaffected: the layer test instantiated the MoE module standalone, which falls back to the eager path (correct per-expert coefficients, confirmed via the dispatch-fallback warning in our logs).

Known Upstream Issues

Found during verification (transformers 5.14.1, snapshot 32b0305):

Rotary embedding bug crashes the stock HF forward path on any input (yarn init ignores rope_head_dim; 192-dim cos/sin vs. 64-dim q_pe).
sliding_window silently ignored by the sdpa/eager attention paths.
Transformers' default grouped_mm/batched_mm MoE backends call _apply_gate() without an expert index, so every expert silently uses expert 0's per-expert PolyNorm coefficients (found in base repo discussion #6; workaround: experts_implementation="eager").
Config keys such as polynorm_output_scale and hidden_clamp are consumed by the vendor's native vLLM implementation but absent from the released HF modeling code -- direct evidence of the revision mismatch.
Suspected embedding-scale mismatch between checkpoint and code.
Non-standard MoE config key names break automatic recognition by serving frameworks.

The bundled modeling_motif.py patches the crash and adds vLLM compatibility; the config includes alias keys and a corrected auto_map.

Usage

Stock from_pretrained with the FP8 config fails: the Transformers FP8 loader replaces expert modules with a generic implementation that does not support this model's per-expert Grouped PolyNorm activation. Serving requires custom expert handling (dequantize routed experts on demand) -- see tools/stream_forward.py.

The vendor's official vLLM image (ghcr.io/motiftechnologies/vllm:v0.20.2-motif3.rc2) uses the same recipe (expert-only 128x128 E4M3, --quantization modelopt_blockfp8) but quantizes BF16 weights online at load time and does not accept pre-quantized checkpoints like this one.

# Weight-level dequantization
import torch
from safetensors import safe_open

with safe_open("model-00105-of-00155.safetensors", framework="pt") as f:
    w = f.get_tensor("model.layers.2.moe.experts.gate_up_proj")           # fp8 e4m3
    s = f.get_tensor("model.layers.2.moe.experts.gate_up_proj_scale_inv") # fp32

E, O, I = w.shape
deq = (w.float().reshape(E, O // 128, 128, I // 128, 128)
        * s[:, :, None, :, None]).reshape(E, O, I).to(torch.bfloat16)

Verified on 2x NVIDIA H100 80GB + 442 GB RAM: the full checkpoint fits in VRAM + system RAM with tiered expert residency (activated experts fetched and dequantized per token).

Repository Contents

Path	Description
`tools/quantize_motif_fp8.py`	BF16 -> FP8 conversion pipeline (restartable)
`tools/stream_forward.py`	Layer-streaming forward driver for verification
`quantization_recipe.json`	Recipe and measured fidelity summary

License

Inherits the original model license: personal, educational, and non-commercial research use only. Commercial use requires prior written permission from Motif Technologies.

Downloads last month: -

Safetensors

Model size

315B params

Tensor type

BF16

F32

F8_E4M3

Model tree for 0ppxnhximxr/Motif-3-Beta-FP8

Base model

Motif-Technologies/Motif-3-Beta

Quantized

(4)

this model