Instructions to use MrFuzzihead/Nex-N2.5-mini-APEX-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use MrFuzzihead/Nex-N2.5-mini-APEX-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16
# Run inference directly in the terminal:
llama cli -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16
# Run inference directly in the terminal:
./llama-cli -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

Use Docker

docker model run hf.co/MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

LM Studio
Jan

vLLM

How to use MrFuzzihead/Nex-N2.5-mini-APEX-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "MrFuzzihead/Nex-N2.5-mini-APEX-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "MrFuzzihead/Nex-N2.5-mini-APEX-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

Ollama
How to use MrFuzzihead/Nex-N2.5-mini-APEX-GGUF with Ollama:
```
ollama run hf.co/MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16
```
Unsloth Desktop

How to use MrFuzzihead/Nex-N2.5-mini-APEX-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use MrFuzzihead/Nex-N2.5-mini-APEX-GGUF with Docker Model Runner:
```
docker model run hf.co/MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16
```

Lemonade

How to use MrFuzzihead/Nex-N2.5-mini-APEX-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

Run and chat with the model

lemonade run user.Nex-N2.5-mini-APEX-GGUF-BF16

List all available models

lemonade list

Hermes Agent

How to use MrFuzzihead/Nex-N2.5-mini-APEX-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use MrFuzzihead/Nex-N2.5-mini-APEX-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "MrFuzzihead/Nex-N2.5-mini-APEX-GGUF:BF16" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Nex-N2.5-mini APEX GGUF

APEX (Adaptive Precision for EXpert models) quantizations of nex-agi/Nex-N2.5-mini, covering every deployment scenario from maximum quality down to ~10 GB.

Quantized by: MrFuzzihead
Method: APEX — a MoE-aware mixed-precision quantization technique for llama.cpp, by the LocalAI team
More APEX quants: MrFuzzihead's APEX collection

APEX assigns quantization precision per tensor role (routed expert / shared expert / attention / SSM) and per layer position (edge layers high, middle layers compressed), exploiting the sparsity of Mixture-of-Experts models. On the APEX reference model (Qwen3.5-35B-A3B), APEX tiers match or beat Q8_0 quality at ~60% of the size — see the APEX technical report for details.

Note on the hardware-compatibility widget: Hugging Face detects quantization variants from standard quant names (Q4_K_M etc.) in the filename. APEX tiers are mixed-precision and use APEX names, so the widget cannot classify them (this also affects other APEX repos). Use the table below for sizes; mmproj files are vision projectors and never appear in the widget by design.

Files

File	Size	Best for
`Nex-N2.5-mini-APEX-Quality.gguf`	21.25 GB	lowest-perplexity tier, 24 GB GPUs
`Nex-N2.5-mini-APEX-I-Quality.gguf`	21.25 GB	best accuracy tier (recommended)
`Nex-N2.5-mini-APEX-Balanced.gguf`	23.60 GB	general purpose, serving
`Nex-N2.5-mini-APEX-I-Balanced.gguf`	23.60 GB	general purpose, lower KL divergence
`Nex-N2.5-mini-APEX-Compact.gguf`	15.40 GB	16–24 GB GPUs
`Nex-N2.5-mini-APEX-I-Compact.gguf`	15.40 GB	best accuracy at this size
`Nex-N2.5-mini-APEX-Mini.gguf`	12.54 GB	12–16 GB VRAM
`Nex-N2.5-mini-APEX-I-Micro.gguf`	10.15 GB	smallest footprint (IQ1_M experts)
`Nex-N2.5-mini-APEX-I-Nano.gguf`	10.97 GB	low tier with higher-fidelity experts
`mmproj-Nex-N2.5-mini-BF16.gguf`	0.84 GB	vision projector, full precision
`mmproj-Nex-N2.5-mini-Q8_0.gguf`	0.57 GB	vision projector, half the size

Sizes are in GB (2³⁰ bytes), as displayed by the Hub. Full-GPU offload needs the file size plus the mmproj (if you use vision) plus KV cache and overhead.

I- files are quantized with an importance matrix built from diverse calibration data (chat, code, reasoning, tool-calling, agentic traces — no Wikipedia). In APEX evaluations, I-variants trade a negligible perplexity increase for better downstream accuracy and lower KL divergence. The Mini, Micro and Nano tiers use the diverse imatrix by design (per the APEX method), which is why they ship as I--named files (I-Micro, I-Nano) with no separate non-imatrix version.

The model

Nex-N2.5-mini is Nex-AGI's agentic multimodal model, the smallest member of the Nex-N2.5 family (mini / Pro / Max). It is built for long-horizon tasks in real-world environments — computer use, web browsing, and visually grounded agentic workflows — where vision is the interface through which the agent perceives its environment, verifies outcomes, and self-corrects.

~35 B parameters, Mixture-of-Experts: 256 routed experts, 8 active per token, plus a shared expert.
40 layers, hybrid attention: three linear-attention (SSM) layers interleaved per full-attention layer (qwen3_5_moe architecture, same family as Qwen3.5). Only 10 of 40 layers use full attention, so KV-cache overhead is minimal.
Multimodal: vision tower with video support (temporal patches); 262,144-token context.
Reasoning: adaptive thinking via reasoning_effort (none / medium / high), emitted in thinking… response blocks.
Tool calling: Qwen3-Coder style <tool_call> / <function=…> XML calls.
License: Apache-2.0.

How APEX quantizes Nex-N2.5-mini

Routed experts are ~94% of the weights here, but only 8 of 256 fire for any given token, so they tolerate lower precision than the parts every token passes through. Attention + SSM are only ~~2.7% of the weights but dense, and the shared expert (~~0.4%) is always active and heavy-tailed. APEX classifies each tensor by role and applies a layer-wise precision gradient — edge layers keep higher precision, middle layers compress harder:

Tier	Routed experts	Shared expert	Attention + SSM	Size
Quality / I-Quality	Q6_K edges (L0–4, L35–39) → Q5_K near-edge (L5–9, L30–34) → IQ4_XS middle (L10–29)	Q8_0	Q6_K	21.25 GB
Balanced / I-Balanced	Q6_K edges → Q5_K (L5–34)	Q8_0	Q6_K	23.60 GB
Compact / I-Compact	Q4_K edges → Q3_K (L5–34)	Q6_K	Q4_K	15.40 GB
Mini	Q3_K edges (L0–9, L30–39) → IQ2_S middle (L10–29)	Q5_K edges / Q4_K middle	Q4_K near edges, Q3_K elsewhere	12.54 GB
I-Micro	Q3_K edges → IQ2_XS near-edge → IQ1_M middle	Q5_K edges / Q4_K middle	Q4_K near edges, Q3_K elsewhere	10.15 GB
I-Nano	Q3_K edges → IQ2_S near-edge → IQ2_XXS middle	Q5_K edges / Q4_K middle	Q4_K near edges, Q3_K elsewhere	10.97 GB

Embeddings and the LM head (~3% of weights) are kept at Q6_K (Q3_K on Mini) to preserve output-logit fidelity. Routers and norms stay F32. Because routing decisions use full-precision gates, quantization noise in inactive experts never affects the output.

For the tier configs and reproduction scripts, see the APEX repository.

Usage

Needs a recent llama.cpp build with qwen35moe (Qwen3.5 / Nex-N2.5) support. The low tiers additionally use IQ1_M / IQ2_XS / IQ2_S / IQ2_XXS expert formats, and the BF16 mmproj needs current BF16 support — when in doubt, use the latest release.

# text
llama-cli -m Nex-N2.5-mini-APEX-I-Quality.gguf -p "Your prompt" -ngl 99

# OpenAI-compatible server (tool calls + reasoning via --jinja)
llama-server -m Nex-N2.5-mini-APEX-I-Quality.gguf \
  --mmproj mmproj-Nex-N2.5-mini-BF16.gguf \
  --jinja --host 0.0.0.0 --port 8080 -ngl 99

# vision, standalone
llama-mtmd-cli -m Nex-N2.5-mini-APEX-I-Quality.gguf \
  --mmproj mmproj-Nex-N2.5-mini-Q8_0.gguf \
  --image image.png -p "What is in this image?" -ngl 99

The GGUF embeds the official chat template, including reasoning_effort thinking modes and the <tool_call> function-calling format. With llama-server --jinja, you can force a thinking mode with --chat-template-kwargs '{"reasoning_effort":"high"}' (omit for the default adaptive medium).

Recommended sampling parameters (from the Nex-AGI model card):

temperature: 0.7
top_p: 0.95
top_k: 40

Choosing an mmproj

mmproj-Nex-N2.5-mini-BF16.gguf (0.84 GB) is the full-precision projector; mmproj-Nex-N2.5-mini-Q8_0.gguf (0.57 GB) is the quantized one. Either pairs with any APEX tier above. If you are not using images or video, skip the mmproj entirely.

Notes

No MTP head: the base repo's config declares an MTP layer (mtp_num_hidden_layers: 1), but no MTP tensors are shipped in nex-agi/Nex-N2.5-mini, so these files do not bundle a speculative draft head — --spec-type draft-mtp does not apply.
Benchmarks: no perplexity or accuracy benchmarks were run on these specific files. The published APEX numbers are from the reference Qwen3.5-35B-A3B evaluation in the APEX repository; see the base model card for Nex-N2.5-mini's own evaluation results.
License: Apache-2.0, same as the base model.