Instructions to use exalandru/GPT-OSS-Coder-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use exalandru/GPT-OSS-Coder-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf exalandru/GPT-OSS-Coder-GGUF
# Run inference directly in the terminal:
llama cli -hf exalandru/GPT-OSS-Coder-GGUF

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf exalandru/GPT-OSS-Coder-GGUF
# Run inference directly in the terminal:
llama cli -hf exalandru/GPT-OSS-Coder-GGUF

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf exalandru/GPT-OSS-Coder-GGUF
# Run inference directly in the terminal:
./llama-cli -hf exalandru/GPT-OSS-Coder-GGUF

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf exalandru/GPT-OSS-Coder-GGUF
# Run inference directly in the terminal:
./build/bin/llama-cli -hf exalandru/GPT-OSS-Coder-GGUF

Use Docker

docker model run hf.co/exalandru/GPT-OSS-Coder-GGUF

LM Studio
Jan

vLLM

How to use exalandru/GPT-OSS-Coder-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "exalandru/GPT-OSS-Coder-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "exalandru/GPT-OSS-Coder-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/exalandru/GPT-OSS-Coder-GGUF

Ollama
How to use exalandru/GPT-OSS-Coder-GGUF with Ollama:
```
ollama run hf.co/exalandru/GPT-OSS-Coder-GGUF
```

Unsloth Studio

How to use exalandru/GPT-OSS-Coder-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for exalandru/GPT-OSS-Coder-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for exalandru/GPT-OSS-Coder-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for exalandru/GPT-OSS-Coder-GGUF to start chatting

How to use exalandru/GPT-OSS-Coder-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf exalandru/GPT-OSS-Coder-GGUF

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "exalandru/GPT-OSS-Coder-GGUF"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

OpenClaw new

How to use exalandru/GPT-OSS-Coder-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf exalandru/GPT-OSS-Coder-GGUF

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "exalandru/GPT-OSS-Coder-GGUF" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use exalandru/GPT-OSS-Coder-GGUF with Docker Model Runner:
```
docker model run hf.co/exalandru/GPT-OSS-Coder-GGUF
```

Lemonade

How to use exalandru/GPT-OSS-Coder-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull exalandru/GPT-OSS-Coder-GGUF

Run and chat with the model

lemonade run user.GPT-OSS-Coder-GGUF-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use exalandru/GPT-OSS-Coder-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf exalandru/GPT-OSS-Coder-GGUF

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default exalandru/GPT-OSS-Coder-GGUF

Run Hermes

hermes

Atomic Chat

GPT-OSS Coder

A gpt-oss-120b version focused on improving practical coding-agent behavior in repository-level software engineering tasks.

Also available in MLX for Apple silicon →

How it works

It digs deeper into the repository, follows evidence to the root cause, and keeps iterating until the fix holds under real tests instead of stopping at a plausible-looking patch.

Fixes the bug, not the symptom : traces the actual defect, not the first thing that looks broken
Inspects more before editing, and re-runs tests after : more reads, more checks, fewer false successes
Emits tool calls the harness can actually run : dramatically fewer rejected calls
Revisits files when new evidence appears
Reasons about state and invariants across components
Continues iterating when the first implementation is incomplete
Ends its turns with a real report of what was done — no empty summaries, no truncated turns

The fine-tune also significantly reduced malformed JSON arguments.

Works even better with my Adversarial Agent Engineering pack of skills and rules

Runtime

Trained and optimized for Codex.

llama.cpp

llama-server -m GPT-OSS-Coder.gguf --jinja -c 32768

--jinja matters: it enables the bundled Harmony chat template, which is how tool declarations reach the model. LM Studio and Ollama consume this GGUF directly.

Format: GGUF, MXFP4 MoE experts + bf16 attention, ~63 GB. Runs on ~64 GB of RAM/VRAM plus context.

The expert weights in this file are bit-identical to the MLX release they were fine-tuned and benchmarked as — the conversion re-packs the same MXFP4 blocks, it does not requantize.

Codex GPT-OSS Server

It provides native Codex integration for GPT-OSS on MLX. Rather than exposing GPT-OSS only through a generic OpenAI-compatible compatibility layer, it is designed so that Codex can use GPT-OSS as a native local model while preserving the GPT-OSS/Codex protocol details.

This includes the native Codex Responses protocol, GPT-OSS Harmony handling, reasoning continuity across tool turns, and Codex-specific routing and metadata.

Agent loops work smoothly without the stalls and rejections you get with generic OpenAI-compatible endpoints.

https://github.com/exalandru/Codex-GPT-OSS-Server

(Currently for Apple Silicon / MLX only. Let me now if you'd like a Linux/Windows version)

How it was trained

Supervised fine-tuning on ~10 000 steps carefully selected from real coding-agent sessions to isolate the targeted behavior : some of my personal sessions with Opus/Fable 5 and GPT 5.6 Sol, public SWE-agent, OpenHands, SWE-smith and Fable trajectories, keeping only runs that actually resolved their issue. A run that gave up, or ran out of context and submitted anyway, teaches exactly the habit this model is meant to shed, so those were filtered out.

Each training example is a real repository state plus the next action the successful agent took, so what is learned is the loop itself: look, edit, run, read the result, correct.

The fine-tune itself is deliberately small, a low-rank update on the last layers only, then consolidated back into the weights. The goal was to shift behaviour, not to overwrite what the base model already knows.

Custom small in-house benchmarks were used to validate the training. Models such as Qwen3.6, DeepSeek v4 Flash and other distilled gpt-oss variants all failed these benchmarks. Opus 5 and GPT 5.6 Sol served as references proving the tasks were solvable.

Built by exalandru. If you use it in a real agent loop, the failure reports are more useful than the success ones.

Downloads last month: 294

GGUF

Model size

117B params

Architecture

gpt-oss

Hardware compatibility

We're not able to determine the quantization variants.

View all variants

Model tree for exalandru/GPT-OSS-Coder-GGUF

Base model

openai/gpt-oss-120b

Finetuned

(111)

this model