📝 Claude-OSS (Fable 5)

Hugging Face Metadata: The automated metadata tag at the top of this page miscalculates this model as a 12B BF16/U8 model because the Hub parser cannot natively calculate the 32-expert MoE layout of Claude-OSS. The actual model size is 21B parameters running natively on MXFP4 microquantization.

Prompt

-

Model description

Open-Source OpenAI model powered by Claude Fable 5 Agent

Highlights

• Claude Fable 5: Fully Train model to Calude Fable 5 specific use case through parameter fine-tuning.
• Agentic capabilities: Use the models’ native capabilities for function calling, web browsing, Python code execution, and Structured Outputs.
• MXFP4 quantization: The models were post-trained with MXFP4 quantization of the MoE weights, making model run within 16GB of memory. All evals were performed with the same MXFP4 quantization.
• Configurable reasoning effort: Easily adjust the reasoning effort (low, medium, high) based on your specific use case and latency needs.
• Full chain-of-thought: Gain complete access to the model’s reasoning process, facilitating easier debugging and increased trust in outputs. It’s not intended to be shown to end users.

---

Inference examples

Transformers

You can use Claude-OSS with Transformers. If you use the Transformers chat template, it will automatically apply the harmony response format. If you use model.generate directly, you need to apply the harmony format manually using the chat template or use our openai-harmony package.

To get started, install the necessary dependencies to setup your environment:

pip install -U transformers kernels torch 

Once, setup you can proceed to run the model by running the snippet below:

from transformers import pipeline
import torch

model_id = "Tesleum/Claude-OSS"

pipe = pipeline(
    "text-generation",
    model=model_id,
    torch_dtype="auto",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "Explain quantum mechanics clearly and concisely."},
]

outputs = pipe(
    messages,
    max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])

Alternatively, you can run the model via Transformers Serve to spin up a OpenAI-compatible webserver:

transformers serve
transformers chat localhost:8000 --model-name-or-path Tesleum/Claude-OSS

vLLM

vLLM recommends using uv for Python dependency management. You can use vLLM to spin up an OpenAI-compatible webserver. The following command will automatically download the model and start the server.

curl -X POST "http://localhost:8000/v1/chat/completions" \
    -H "Content-Type: application/json" \
    --data '{
        "model": "Tesleum/Claude-OSS",
        "messages": [
            {
                "role": "user",
                "content": "What is the capital of France?"
            }
        ]
    }'

vllm serve "Tesleum/Claude-OSS"

Docker

docker model run hf.co/Tesleum/Claude-OSS

Ollama

To achieve better performance and quality, use vLLM instead

LM Studio

To achieve better performance and quality, use vLLM instead

---

Reasoning levels

You can adjust the reasoning level that suits your task across three levels:

• Low: Fast responses for general dialogue.
• Medium: Balanced speed and detail.
• High: Deep and detailed analysis.

The reasoning level can be set in the system prompts, e.g., "Reasoning: high".

Tool use

The claude-oss models are excellent for:

• Agentic operations like browser tasks
• Function calling with defined schemas
• Web browsing (using built-in browsing tools)