NandoG-AI/Nex-N2.5-mini-GGUF

🧠 Nex-N2.5-mini — GGUF

GGUF quantizations of nex-agi/Nex-N2.5-mini for use with llama.cpp and compatible runtimes (LM Studio, Open WebUI, KoboldCpp, etc.).

Multimodal (vision) support is included via a separate mmproj projector file — see the Multimodal Support section.

---

📖 About Nex-N2.5

Nex-N2.5 is a next-generation family of agentic models designed for long-horizon reasoning and real-world task execution.

Building on the multimodal foundations of Nex-N2, the Nex-N2.5 family introduces significant improvements in:

• Software engineering
• Tool calling
• Browser use
• Computer use
• Agentic workflows
• Reasoning
• Multimodal understanding
• Long-context processing

Unlike traditional chat models, Nex-N2.5 is designed to continuously interact with environments, verify outcomes, self-correct, and execute complex multi-step workflows.

The family consists of:

• Nex-N2.5-mini (this repository)
• Nex-N2.5-Pro
• Nex-N2.5-Max

The original model supports a native context length of 262,144 tokens (256K).

---

📊 Model Information

Base model	nex-agi/Nex-N2.5-mini
Architecture	Qwen3.5 MoE
Parameters	35B (MoE)
Context length	262,144 tokens
License	Apache 2.0
Format	GGUF
Runtime	llama.cpp
Modality	Text + Vision

Primary use cases: coding, software development, autonomous agents, web automation, computer use, research, knowledge work, and multimodal tasks.

---

📥 Available Quantizations

⭐ Recommended

Quant	Use case
Q4_K_M	Best overall balance (start here)
Q5_K_M	High-quality daily driver
Q6_K	Near-lossless quality

⚡ Performance / Low Memory

Quant	Use case
Q2_K	Lowest memory usage
Q3_K_M	Budget systems
Q4_K_M	Fastest recommended balance

🎯 Maximum Quality

Quant	Use case
Q8_0	Highest practical quality

---

📂 Files

File	Type
Nex-N2.5-mini-Q2_K.gguf	Weights
Nex-N2.5-mini-Q3_K_M.gguf	Weights
Nex-N2.5-mini-Q4_K_M.gguf	Weights
Nex-N2.5-mini-Q5_K_M.gguf	Weights
Nex-N2.5-mini-Q6_K.gguf	Weights
Nex-N2.5-mini-Q8_0.gguf	Weights
mmproj-Nex-N2.5-mini-F16.gguf	Vision projector (optional)

Download a single quant

hf download <your-username>/Nex-N2.5-mini-GGUF \
  Nex-N2.5-mini-Q4_K_M.gguf \
  --local-dir ./Nex-N2.5-mini-GGUF

For vision, also download mmproj-Nex-N2.5-mini-F16.gguf.

---

🖼️ Multimodal Support

This repository includes a vision projector for image-enabled inference:

mmproj-Nex-N2.5-mini-F16.gguf

Pass it alongside your chosen quant with --mmproj:

llama-server \
  -m Nex-N2.5-mini-Q4_K_M.gguf \
  --mmproj mmproj-Nex-N2.5-mini-F16.gguf \
  --jinja \
  -c 8192

Text-only usage does not require the projector.

---

⚙️ Recommended Sampling Parameters

Original Nex-AGI recommendation:

temperature = 0.7
top_p       = 0.95
top_k       = 40

For deterministic benchmarking (identical input → identical output):

temperature = 0.0
top_p       = 1.0
top_k       = 0

---

💭 Thinking Modes

Nex-N2.5 supports multiple reasoning modes, usually exposed through your frontend's reasoning settings:

Mode	Description
none	Direct response generation
medium	Adaptive reasoning
high	Extended reasoning

---

🖥️ llama.cpp Usage

Chat (CLI)

llama-cli \
  -m Nex-N2.5-mini-Q4_K_M.gguf \
  -c 8192 \
  --jinja

OpenAI-compatible server

llama-server \
  -m Nex-N2.5-mini-Q4_K_M.gguf \
  --jinja \
  -c 8192 \
  --host 0.0.0.0 \
  --port 8080

Increase -c (context size) as memory allows — the model supports up to:

262144 tokens

---

✅ Features

• 262K context support
• Tool calling
• Reasoning / thinking modes
• Multimodal vision support
• Agentic workflows
• Coding-optimized
• OpenAI-compatible serving
• Compatible with llama.cpp, LM Studio, Open WebUI, and KoboldCpp

---

🔗 Links

• Original model: nex-agi/Nex-N2.5-mini
• Nex-AGI: nex-agi.com
• OpenRouter: openrouter.ai/nex-agi/nex-n2.5-mini

---

🙏 Credits

• Original model: Nex-AGI
• GGUF conversion & quantization: Nando Gaarenstroom
• Inference runtime: ggml-org/llama.cpp

Special thanks to the local AI community for testing and feedback.

---

⭐ Support

If these GGUF quantizations are useful to you, please consider:

• ❤️ Liking the repository
• 📊 Sharing benchmark results
• 🐛 Reporting issues
• 🧪 Contributing testing feedback

Happy local inferencing 🚀