sizzlebop/OxCoder-9B-GGUF

OxCoder-9B GGUF

This repository provides GGUF quantizations for OxCoder-9B, a 9-billion parameter coding model optimized for long-horizon agentic software engineering and terminal tasks.

OxCoder-9B is built on the Qwen 3.5 9B foundation and trained on agentic trajectories from frontier coding agents (including Fable-5.1 and GLM-5.3 traces across Claude Code, OpenCode, and Codex). It incorporates read-before-write inspection, tool call error recovery, and minimal edit diff generation.

All GGUF files were converted from the original safetensors weights using llama.cpp at native BF16 precision, then quantized into standard k-quant variants.

---

Available Files and Quantizations

File	Quant Type	Size	Description / Recommendation
OxCoder-9B-BF16.gguf	BF16	16.69 GB	Full precision base conversion. Highest fidelity reference weights.
OxCoder-9B-Q8_0.gguf	Q8_0	8.87 GB	Near-lossless 8-bit quantization. Recommended for production coding when RAM permits.
OxCoder-9B-Q6_K.gguf	Q6_K	6.85 GB	High quality retention with minimal degradation. Excellent accuracy-to-size balance.
OxCoder-9B-Q5_K_M.gguf	Q5_K_M	6.02 GB	Balanced quantization. Strong coding reasoning with moderate memory usage.
OxCoder-9B-Q4_K_M.gguf	Q4_K_M	5.24 GB	Recommended default. Fast, responsive, and fits easily into typical consumer GPUs.
OxCoder-9B-Q3_K_M.gguf	Q3_K_M	4.31 GB	Compact footprint when memory headroom is strictly limited.
OxCoder-9B-Q2_K.gguf	Q2_K	3.56 GB	Maximum compression. Noticeable quality loss, intended for memory-constrained testing.

---

Architecture Requirements

OxCoder-9B uses the Qwen 3.5 architecture (Qwen3_5ForConditionalGeneration / qwen35 in llama.cpp), which incorporates hybrid linear attention layers interspersed with full self-attention and interleaved MRoPE rotary position embeddings.

Ensure your llama.cpp build or downstream runtime (such as Ollama, LM Studio, or Jan) includes Qwen 3.5 support.

---

Quickstart

1. Terminal Inference with llama.cpp

Run the model interactively using llama-cli:

llama-cli -m ./OxCoder-9B-Q4_K_M.gguf \
    -p "<|im_start|>user\nWrite a Python script to parse git commit history and summarize author stats.<|im_end|>\n<|im_start|>assistant\n" \
    -n 1024 \
    -c 8192 \
    --temp 0.6

To run a local OpenAI-compatible API server:

llama-server -m ./OxCoder-9B-Q4_K_M.gguf \
    --host 127.0.0.1 \
    --port 8080 \
    -c 16384

2. Ollama

Create a file named Modelfile in the same directory:

FROM ./OxCoder-9B-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.6

Then register and start the model:

ollama create oxcoder-9b -f Modelfile
ollama run oxcoder-9b

3. LM Studio

1. Place any .gguf file from this folder into your LM Studio models directory.
2. Search for OxCoder-9B in your local models tab.
3. Select ChatML as the prompt format preset.

---

Benchmark Highlights

Benchmark	OxCoder-9B	Qwen3.5-9B	SWE-bench Verified / Task
Terminal-Bench 2.1 (Claude Code)	50.8	18.9	Agentic terminal coding
Terminal-Bench 2.1 (Terminus-2)	49.6	21.3	Agentic terminal coding
SWE-bench Verified	73.5	53.2	Agentic issue resolution
SWE-bench Pro	49.1	31.3	Real-world software engineering
NL2Repo	36.2	16.2	Repo-level code generation
GPQA Diamond	86.9	82.5	Scientific and logic reasoning

---

Model Details

• Base model: Qwen/Qwen3.5-9B
• Architecture: Qwen 3.5 (Qwen3_5ForConditionalGeneration)
• Parameters: ~9 Billion
• Native context length: 262,144 tokens
• Vocabulary size: 248,320 tokens
• License: Apache 2.0

---

Attribution & Credits

Base model architecture by Qwen. Training trajectories distilled from frontier coding agent traces.

GGUF conversions produced by Pink Pixel.

Made with 💖 by Pink Pixel