---

🤗 Model   |    🔀 OpenRouter   |    💻 Github   |    🧭 ModelScope   |    🚀 Nex-AGI

Nex-N2-Mini-NVFP4

NVFP4 (4-bit) quantized build of Nex-N2-mini — an agentic model with Agentic Thinking.

This repository contains a post-training-quantized checkpoint of nex-agi/Nex-N2-mini, produced with llm-compressor and stored in the compressed-tensors format. The MoE expert stack is quantized to NVFP4 (W4A4) and the attention / output projections to FP8 (W8A8), shrinking the checkpoint from 70.2 GB to 24.8 GB (~2.8×) so the model fits comfortably on a single GPU.

	Base model	This repository
Weights	BF16	NVFP4 + FP8 (mixed precision)
Checkpoint size	70.2 GB	24.8 GB
Minimum GPUs	2× H100	1× (see Hardware)
Architecture	Qwen3_5MoeForConditionalGeneration	unchanged

---

Quantization Details

The model is quantized with compressed-tensors (quant_method: compressed-tensors, format: mixed-precision). The full recipe ships in recipe.yaml.

NVFP4 — W4A4 (nvfp4-pack-quantized)

Applied to the MoE feed-forward stack in layers 0–31 — this is the bulk of the parameters:

• model.language_model.layers.{0..31}.mlp.experts.{0..255}.{gate,up,down}_proj
• model.language_model.layers.{0..31}.mlp.shared_expert.{gate,up,down}_proj

	Weights	Activations
Format	FP4 (E2M1), packed 2-per-byte	FP4 (E2M1)
Strategy	tensor_group, group size 16	tensor_group, group size 16
Block scale dtype	float8_e4m3	float8_e4m3
Calibration	static, memoryless_minmax	dynamic (local), static_minmax global scale

FP8 — W8A8 (float-quantized)

Applied to attention, the linear-attention projections, the LM head, and the MoE stack of the last 8 layers (32–39), which are the most quantization-sensitive:

• self_attn.{q,k,v,o}_proj
• linear_attn.{in_proj_qkv,in_proj_z,out_proj}
• lm_head
• layers.{32..39}.mlp.experts.*.{gate,up,down}_proj and layers.{32..39}.mlp.shared_expert.{gate,up,down}_proj

	Weights	Activations
Format	FP8 (E4M3)	FP8 (E4M3)
Strategy	per-channel, symmetric	per-token, symmetric
Calibration	static, memoryless_minmax	dynamic

Kept in BF16

• The full vision tower (model.visual.*, 333 tensors)
• MoE routers — mlp.gate, mlp.shared_expert_gate
• Linear-attention gating / state projections — linear_attn.in_proj_a, linear_attn.in_proj_b, linear_attn.conv1d, A_log, dt_bias, and all linear_attn.norm*
• Token embeddings and all RMSNorm weights

The KV cache is not quantized (kv_cache_scheme: null).

---

Hardware

NVFP4 reaches its full speedup on NVIDIA Blackwell GPUs (SM100 / SM120 — B200, GB200, RTX PRO 6000 Blackwell), which have native FP4 tensor cores. vLLM can still load and run this checkpoint on pre-Blackwell hardware through its NVFP4 dequantization/emulation path, in which case you get the memory savings but not the throughput gain.

At 24.8 GB of weights the model serves on a single 80 GB card with room for a long-context KV cache; the launch command below is tuned for that shape.

---

Usage

Deployment with vLLM

Note: Qwen3.5-MoE support, the NVFP4 compressed-tensors kernels, and the --language-model-only flag all require a recent vLLM build. Use a current release or nightly.

vllm serve /models/Nex-N2-Mini-NVFP4 \
  --served-model-name nex-n2 \
  --host 0.0.0.0 --port 8000 \
  --trust-remote-code --language-model-only \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --max-num-seqs 32 \
  --gpu-memory-utilization 0.60 \
  --max-num-batched-tokens 8192 \
  --enable-chunked-prefill

Flag notes:

• --language-model-only skips the (unquantized, BF16) vision tower and serves the model as a pure text LLM. Drop it if you want the multimodal path.
• --max-model-len 131072 is a conservative cap; the checkpoint's max_position_embeddings is 262144.
• --gpu-memory-utilization 0.60 and --max-num-seqs 32 leave generous headroom — raise them if you have spare VRAM and want more concurrency.
• --tensor-parallel-size 1 is sufficient at this precision. Increase it for extra KV-cache capacity, not for weight capacity.

Query it through the OpenAI-compatible API:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nex-n2",
    "messages": [{"role": "user", "content": "Write a Python function that merges overlapping intervals."}],
    "temperature": 0.7,
    "top_p": 0.95,
    "top_k": 40
  }'

Function Calling and Reasoning Parser

Nex-series models emit explicit reasoning traces and support robust function calling. To have vLLM split the reasoning trace from the final answer and parse tool calls into the OpenAI tool_calls schema, add:

  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Without these flags the reasoning trace and the <tool_call> blocks are returned inline in content, and you parse them yourself.

Recommended Sampling Parameters

• temperature: 0.7
• top_p: 0.95
• top_k: 40

Deployment with SGLang

The base model is best served with the customized sglang fork described in the Nex-N2-mini card. That fork does not currently consume compressed-tensors NVFP4 checkpoints — use the BF16 base model there, and this repository with vLLM.

---

Performance

The scores below are the published results for the BF16 base models and are reproduced here for reference. They were not re-measured on this NVFP4 checkpoint; expect a small degradation typical of 4-bit post-training quantization. If you benchmark it, we would welcome the numbers in the Community tab.

Benchmark	Nex-N2-mini	Nex-N2-Pro	GPT-5.5	Opus 4.7	Kimi-K2.6	GLM-5.1	MiniMax M3	DeepSeek-V4-Pro
Agent
BrowseComp	74.1	83.7	84.4	79.8	83.2	79.3	83.5	83.4
GDPval	1402	1585	1769	1753	1481	1535	-	1554
Toolathlon	33.3	51.9	55.6	52.8	50.0	40.7	-	51.8
WildClawBench	47.7	53.5	58.2	62.2	-	48.2	-	43.7
WideSearch	62.0	75.6	-	-	80.8	-	-	-
TAU3	65.9	71.1	-	-	-	70.6	-	-
Coding & SWE
SWE-Bench Pro	50.2	58.8	58.6	64.3	58.6	58.4	59.0	55.4
Terminal-Bench 2.1	60.7	75.3	83.4	69.7	-	58.7	66.0	72.0
DeepSWE	8.0	33.6	70	54	24	18	-	8
SWE-Bench Verified	74.4	80.8	82.9	87.6	80.2	-	80.5	80.6
SWE Atlas QnA	31.5	37.9	45.4	45.2	-	-	37.9	-
SWE Atlas RF	30.0	32.9	44.8	48.6	-	-	-	-
SWE Atlas TW	23.3	40.0	42.6	38.2	-	-	30.8	-
General & Reasoning
GPQA Diamond	82.6	90.7	93.6	94.2	90.5	86.2	-	90.1
IFEval	89.1	94.0	-	-	94.5	94.5	-	91.9
Apex	9.4	36.5	-	-	24.0	11.5	-	38.3

---

About Nex-N2

Nex-N2 is an agent model built for real-world productivity scenarios. With first-tier coding and agentic capabilities, it keeps driving complex, long-horizon tasks forward in real environments to deliver stable, end-to-end results.

Rather than treating reasoning, tool use, and environment execution as separate capabilities, Nex-N2 unifies them through an Agentic Thinking framework that connects requirement understanding, task planning, code implementation, environmental feedback, evaluation and debugging, and continuous iteration into a single closed loop. The framework has two parts:

• Adaptive Thinking lets the model decide on its own when to think and how deeply — executing simple actions quickly while reasoning thoroughly on critical decisions.
• Coherent Thinking carries one consistent reasoning paradigm across general reasoning and diverse agentic tasks, staying consistent across tasks and modalities to enable stable capability transfer.

Nex-N2 ships in two variants, both post-trained on the Qwen3.5 series: Nex-N2-Pro (built on Qwen3.5-397B-A17B) and Nex-N2-mini (built on Qwen3.5-35B-A3B-Base).

Model Architecture

Architecture	Qwen3_5MoeForConditionalGeneration (hybrid linear + full attention MoE)
Layers	40 (full attention every 4th layer, linear attention elsewhere)
Hidden size	2048
Experts	256, 8 active per token, plus 1 shared expert
Context length	262,144
Vocabulary	248,320

Related Models

• Nex-N2-Pro: Hugging Face | ModelScope
• Nex-N2-mini (BF16 base): Hugging Face | ModelScope

License

Released under the Apache 2.0 license, matching the base model.