Instructions to use Omega-Factory/Qwen3-Coder-Next-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Use Docker

docker model run hf.co/Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Omega-Factory/Qwen3-Coder-Next-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Omega-Factory/Qwen3-Coder-Next-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Ollama
How to use Omega-Factory/Qwen3-Coder-Next-GGUF with Ollama:
```
ollama run hf.co/Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
```
Unsloth Desktop

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Omega-Factory/Qwen3-Coder-Next-GGUF with Docker Model Runner:
```
docker model run hf.co/Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M
```

Lemonade

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Qwen3-Coder-Next-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Omega-Factory/Qwen3-Coder-Next-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Omega-Factory/Qwen3-Coder-Next-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen 3-Coder-Next MoE GGUF Precision Ladder

Official Release & Multi-Quant Benchmarks:
View full precision comparison tables (Q8_0, Q4_K_M, Q2_K vs. BF16 GGUF Base), item-level McNemar discordance flips, and verified cryptographic receipts on the Omega Factory Qwen 3-Coder-Next MoE GGUF Precision Ladder Model Page.

Measured MoE GGUF precision quantization ladder. Hybrid architecture combining Gated DeltaNet linear attention and fine-grained mixture-of-experts routing.

Measured quant ladder

Every rung below is reported on its own exact bytes (sha256-bound). A verdict is never transferred between formats or rungs, and a rung that did not clear its bands is published with its real verdict rather than omitted.

⚠️ Quality Warning: Quantization degradation detected for: Q2_K (DEGRADED). Review capability retention before deploying.

Rung	Verdict	Seeds	Size	SHA256	Receipt
`Q8_0`	SCREEN	1	84.81 GB	`05dc5947082d`	`receipts/cert_Q8_0.json`
`Q4_K_M`	SCREEN	1	48.49 GB	`c6a4c85791cf`	`receipts/cert_Q4_K_M.json`
`Q2_K`	DEGRADED	1	29.27 GB	`df7a317496f3`	`receipts/cert_Q2_K.json`

SCREEN means the result did not use the full seed set and therefore is not a certification.

Delivered modality: text-only GGUF. Base-model vision support does not establish multimodal delivery.

BF16 GGUF Baseline vs. Quantization Precision Comparison

To evaluate quantization precision loss and capability retention, quant variants were benchmarked against the unquantized base model (Qwen/Qwen3-Coder-Next) under controlled deterministic evaluation settings.

Precision & Accuracy Matrix vs. BF16 GGUF Baseline

Variant / Format	Size (Disk/VRAM)	ARC-MC Accuracy	GSM8K Accuracy	MATH-500 Accuracy	Babble Rate	Status
BF16 GGUF Baseline	159.46 GB	96.5% (193/200)	95.5% (191/200)	61.5% (123/200)	0.0033	Baseline Ground Truth
`Q8_0`	84.81 GB	96.5% (Δ +0.0 pp)	95.5% (Δ +0.0 pp)	62.5% (Δ +1.0 pp)	0.0033	SCREEN
`Q4_K_M`	48.49 GB	97.0% (Δ +0.5 pp)	95.5% (Δ +0.0 pp)	61.0% (Δ -0.5 pp)	0.0033	SCREEN
`Q2_K`	29.27 GB	96.5% (Δ +0.0 pp)	91.5% (Δ -4.0 pp)	48.0% (Δ -13.5 pp)	0.0017	DEGRADED

Detailed Item-Level Contingency & Flips vs. BF16 GGUF

Per-item McNemar discordance analysis measuring questions that changed answer status relative to the unquantized BF16 GGUF baseline:

Quantization Variant	Category	Quant Correct	BF16 GGUF Base Correct	Flips Broke (BF16 GGUF ✓ → Quant ✗)	Flips Fixed (BF16 GGUF ✗ → Quant ✓)	Net Delta (pp)
`Q8_0`	GSM8K	191 / 200	191 / 200	3	3	+0.0 pp
`Q8_0`	MATH-500	125 / 200	123 / 200	10	12	+1.0 pp
`Q8_0`	ARC-MC	193 / 200	193 / 200	0	0	+0.0 pp
`Q4_K_M`	GSM8K	191 / 200	191 / 200	2	2	+0.0 pp
`Q4_K_M`	MATH-500	122 / 200	123 / 200	15	14	-0.5 pp
`Q4_K_M`	ARC-MC	194 / 200	193 / 200	0	1	+0.5 pp
`Q2_K`	GSM8K	183 / 200	191 / 200	11	3	-4.0 pp
`Q2_K`	MATH-500	96 / 200	123 / 200	36	9	-13.5 pp
`Q2_K`	ARC-MC	193 / 200	193 / 200	1	1	+0.0 pp

Hardware & VRAM Allocation Recommendations

Artifact size is disk usage, not a measured VRAM requirement. Runtime memory and the KV cache depend on architecture, backend, batch settings, and context length. Validate actual deployment with peak-memory profiling.

Reference capacity bands only:

Artifact size (disk)	Reference capacity band
Up to 8 GB	6–8 GB
Over 8 GB through 16 GB	12–16 GB
Over 16 GB through 24 GB	20–24 GB
Over 24 GB through 48 GB	32–48 GB
Over 48 GB	Multi-GPU or CPU offload; no single listed tier

Quant Variant	Artifact Size (disk)	Deployment sizing notes
`Q8_0`	84.81 GB	Multi-GPU or CPU offload; no single listed tier. SCREEN evaluation status; deployment is not recommended.
`Q4_K_M`	48.49 GB	Multi-GPU or CPU offload; no single listed tier. SCREEN evaluation status; deployment is not recommended.
`Q2_K`	29.27 GB	32–48 GB reference capacity. Candidate only, not a measured fit guarantee.

Recommended Sampling Parameters

Measured greedy mode:
- temperature = 0.0
- top_k = 1
- seed = 42

Model Architecture Specifications

Parameters: 79.6B (3.14B active)
Layers: 48
Experts: 512 (top-10 routed) + 1 shared
Context Length: 262144
Attention: Hybrid Gated DeltaNet Linear Attention and Full Attention

Chat template

{% macro render_extra_keys(json_dict, handled_keys) %}
    {%- if json_dict is mapping %}
        {%- for json_key in json_dict if json_key not in handled_keys %}
            {%- if json_dict[json_key] is string %}
                {{-'\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | string) ~ '</' ~ json_key ~ '>' }}
            {%- else %}
                {{- '\n<' ~ json_key ~ '>' ~ (json_dict[json_key] | tojson | safe) ~ '</' ~ json_key ~ '>' }}
            {%- endif %}
        {%- endfor %}
    {%- endif %}
{%- endmacro %}

{%- if messages[0]["role"] == "system" %}
    {%- set system_message = messages[0]["content"] %}
    {%- set loop_messages = messages[1:] %}
{%- else %}
    {%- set loop_messages = messages %}
{%- endif %}

{%- if not tools is defined %}
    {%- set tools = [] %}
{%- endif %}

{%- if system_message is defined %}
    {{- "<|im_start|>system\n" + system_message }}
{%- else %}
    {%- if tools is iterable and tools | length > 0 %}
        {{- "<|im_start|>system\nYou are Qwen, a helpful AI assistant that can interact with a computer to solve tasks." }}
    {%- endif %}
{%- endif %}
{%- if tools is iterable and tools | length > 0 %}
    {{- "\n\n# Tools\n\nYou have access to the following functions:\n\n" }}
    {{- "<tools>" }}
    {%- for tool in tools %}
        {%- if tool.function is defined %}
            {%- set tool = tool.function %}
        {%- endif %}
        {{- "\n<function>\n<name>" ~ tool.name ~ "</name>" }}
        {%- if tool.description is defined %}
            {{- '\n<description>' ~ (tool.description | trim) ~ '</description>' }}
        {%- endif %}
        {{- '\n<parameters>' }}
        {%- if tool.parameters is defined and tool.parameters is mapping and tool.parameters.properties is defined and tool.parameters.properties is mapping %}
            {%- for param_name, param_fields in tool.parameters.properties|items %}
                {{- '\n<parameter>' }}
                {{- '\n<name>' ~ param_name ~ '</name>' }}
                {%- if param_fields.type is defined %}
                    {{- '\n<type>' ~ (param_fields.type | string) ~ '</type>' }}
                {%- endif %}
                {%- if param_fields.description is defined %}
                    {{- '\n<description>' ~ (param_fields.description | trim) ~ '</description>' }}
                {%- endif %}
                {%- set handled_keys = ['name', 'type', 'description'] %}
                {{- render_extra_keys(param_fields, handled_keys) }}
                {{- '\n</parameter>' }}
            {%- endfor %}
        {%- endif %}
        {%- set handled_keys = ['type', 'properties'] %}
        {{- render_extra_keys(tool.parameters, handled_keys) }}
        {{- '\n</parameters>' }}
        {%- set handled_keys = ['type', 'name', 'description', 'parameters'] %}
        {{- render_extra_keys(tool, handled_keys) }}
        {{- '\n</function>' }}
    {%- endfor %}
    {{- "\n</tools>" }}
    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
{%- endif %}
{%- if system_message is defined %}
    {{- '<|im_end|>\n' }}
{%- else %}
    {%- if tools is iterable and tools | length > 0 %}
        {{- '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- for message in loop_messages %}
    {%- if message.role == "assistant" and message.tool_calls is defined and message.tool_calls is iterable and message.tool_calls | length > 0 %}
        {{- '<|im_start|>' + message.role }}
        {%- if message.content is defined and message.content is string and message.content | trim | length > 0 %}
            {{- '\n' + message.content | trim + '\n' }}
        {%- endif %}
        {%- for tool_call in message.tool_calls %}
            {%- if tool_call.function is defined %}
                {%- set tool_call = tool_call.function %}
            {%- endif %}
            {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
            {%- if tool_call.arguments is defined %}
                {%- for args_name, args_value in tool_call.arguments|items %}
                    {{- '<parameter=' + args_name + '>\n' }}
                    {%- set args_value = args_value if args_value is string else args_value | tojson | safe %}
                    {{- args_value }}
                    {{- '\n</parameter>\n' }}
                {%- endfor %}
            {%- endif %}
            {{- '</function>\n</tool_call>' }}
        {%- endfor %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "user" or message.role == "system" or message.role == "assistant" %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
    {%- elif message.role == "tool" %}
        {%- if loop.previtem and loop.previtem.role != "tool" %}
            {{- '<|im_start|>user' }}
        {%- endif %}
        {{- '\n<tool_response>\n' }}
        {{- message.content }}
        {{- '\n</tool_response>' }}
        {%- if not loop.last and loop.nextitem.role != "tool" %}
            {{- '<|im_end|>\n' }}
        {%- elif loop.last %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- else %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
{%- endif %}

License and attribution

License: Apache-2.0 (inherited from the base model).
Base model: Qwen/Qwen3-Coder-Next -- all credit for the underlying model belongs to its authors.
This repository publishes a compressed derivative: experts were pruned and the result re-quantised. It is not a new base model.
Commercial use is permitted under the terms of the license above.

Quantized GGUF preparation and evaluation card: Omega Factory. Base weights remain attributed to the upstream authors.

Contact / reporting an issue

Found a problem with these weights or this card? Open a discussion on this repository, or report a security concern to the maintainers via the organisation page at https://huggingface.co/Omega-Factory. We publish corrections and, where warranted, retract a release rather than leave a wrong card standing.

Official Model Page: https://omega-factory.ai/models/qwen3-coder-next-gguf-v1
Organization Website: https://omega-factory.ai

Downloads last month: 374

GGUF

Model size

80B params

Architecture

qwen3next

Hardware compatibility

2-bit

4-bit

8-bit

Model tree for Omega-Factory/Qwen3-Coder-Next-GGUF

Base model

Qwen/Qwen3-Coder-Next

Quantized

(118)

this model