kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF

Nex-N2.5-mini — ROCmFP4 for AMD Strix Halo (gfx1151)

ROCmFP4 / ROCmFPX quantizations of nex-agi/Nex-N2.5-mini — 35,107,181,936 parameters (BF16), 40-layer Qwen3.5 MoE (30 Gated DeltaNet linear-attention + 10 full-attention layers), 256 routed experts / 8 active, 262,144-token context, text + image — built and measured on an AMD Ryzen AI Max+ 395 (Radeon 8060S, gfx1151). Upstream publishes no GGUF.

• Another public ROCmFP4 build of this model exists — julianmb/Nex-N2.5-mini-ROCmFP4-GGUF: its Nex-N2.5-mini-ROCmFP4-STRIX_LEAN.gguf stores output.weight as Q4_0_ROCMFP4_FAST and carries no imatrix metadata. Every tier here keeps output.weight at Q6_K, and the imatrix builds are a separate repo.
• Vision projector included.
• No MTP head. mtp_num_hidden_layers: 1 is declared in config.json, but the checkpoint ships no mtp.* weights (the converted BF16 GGUF reads back 0 nextn tensors). There is no speculative decoding on these files.
• Importance-matrix builds of the same three 4-bit tiers: kingjones777/Nex-N2.5-mini-ROCmFP4-imatrix-GGUF.

Which file should I use?

Ryzen AI Max+ 395 (MAX-1), ROCm 7.2.4, unpatched llama-server at d3ca537 (see Quick start), -c 65536, one request at a time (--parallel 1), greedy (temp 0, top_k 1), ignore_eos so every arm generates exactly 256 tokens after a code prompt of 7,094–7,102 tokens (the first 30,000 characters of convert_hf_to_gguf.py plus an instruction), a unique nonce per request and cache_prompt: false (cache_n = 0 asserted on every timed request), 1 warm-up then the median of 3. Decode numbers are the server's own predicted_per_second. Box iced: no other model loaded.

File	ftype	Size	BPW⁴	KLD vs BF16 ↓²	Same top-1 ↑	PPL (× BF16)	TG ROCm0	TG Vulkan0	PP ROCm0
Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf	106	17.46 GiB	4.32	0.1044 ± 0.0014	86.66 %	6.4740 ± 0.0798 (×1.0393)	63.94	68.16	1158
Nex-N2.5-mini-Q4_0-ROCmFP4-COHERENT.gguf	102	18.48 GiB	4.58	0.0971 ± 0.0013	87.29 %	6.5617 ± 0.0812 (×1.0534)	61.74	68.00	1187
Nex-N2.5-mini-Q4_0-ROCmFP4-FAST.gguf	103	17.37 GiB	4.30	0.1088 ± 0.0014	86.41 %	6.5498 ± 0.0809 (×1.0515)	63.31	68.62	1157
BF16 reference	32	64.61 GiB³	16.01	0	100 %	6.2290 ± 0.0753¹	—	—	—

TG = decode tokens/s after the 7,094–7,102-token code prompt, no draft head. PP = prefill tokens/s on ROCm0. ¹ The BF16 PPL shown is the paired base every "×" ratio is computed against (averaged over the same scored tokens in the KL-divergence runs). The standalone BF16 run's own summary line reads 6.2303 ± 0.0754. ² Quality columns: see Where the quality numbers come from below. ³ BF16 conversion of the checkpoint; not published. ⁴ BPW as printed by llama-quantize: bits per weight over the 34,660,610,688 weights in each GGUF. The 35,107,181,936-parameter count above also includes the 446,571,248-weight vision tower, which ships in the projector file.

Start with STRIX_LEAN; take COHERENT if quality matters more than speed. STRIX_LEAN decodes 3.6 % faster than COHERENT on ROCm0 (63.94 vs 61.74 tok/s; per-run ranges 63.56–63.96 and 61.74–61.80) and within 0.2 % of it on Vulkan0 (68.16 vs 68.00), and prefills within 2.4 % of it on ROCm0 (1158 vs 1187 tok/s). COHERENT's KLD is 7.1 % lower (3.8σ) for 1051 MiB more.

FAST does not buy a clear speed gain here: FAST decodes within 1.0 % of STRIX_LEAN on ROCm0 (63.31 vs 63.94 tok/s; per-run ranges 63.14–63.38 and 63.56–63.96) and within 0.7 % of it on Vulkan0 (68.62 vs 68.16), and prefills within 0.1 % of it on ROCm0 (1157 vs 1158 tok/s); its KLD is higher than STRIX_LEAN's (0.1088 vs 0.1044, +4.2 %, 2.2σ).

KLD order (lower is closer to BF16): COHERENT 0.0971, STRIX_LEAN 0.1044 (+7.6 % vs COHERENT), FAST 0.1088 (+12.1 % vs COHERENT). STRIX_LEAN is 1051 MiB smaller than COHERENT; FAST is 91 MiB smaller than STRIX_LEAN. Speed gaps below 3.0 % (decode) and 3.0 % (prefill) are called a tie: the larger of 3 % and the widest gap measured between files that do identical work per token (each imatrix file and its standard twin: decode 1.0 %, prefill 1.1 %).

The imatrix build is measurably closer to BF16 at the same size: STRIX_LEAN 0.1044 → 0.0852 (-18.5 %, 10.0σ); COHERENT 0.0971 → 0.0769 (-20.7 %, 11.3σ); FAST 0.1088 → 0.0890 (-18.2 %, 10.2σ).

Quality is graded against the BF16 GGUF (reference logits computed on the CPU) on a held-out corpus (wikitext-2 test, -c 2048, 40 chunks × 1,023 scored tokens each — the second half of every window, less its first token — = 40,920), never on the imatrix calibration text. KLD is the per-token KL divergence of each quant's next-token distribution from BF16's on the same tokens — far more sensitive than perplexity.

Where the quality numbers come from. Measured directly on these files, against BF16 logits computed on the CPU in the same session (first-window perplexity 5.6964; Vulkan0 gave 5.6953 for the same window). Every file was graded on ROCm0 (the columns above) and again on Vulkan0. The STRIX_LEAN grade was run twice on each backend and every per-chunk row matched: 40 of 40 on ROCm0, 40 of 40 on Vulkan0. Why not the GPU for the reference: see Known issues.

Same files, same reference, graded on each backend — KLD ROCm0 / Vulkan0: STRIX_LEAN 0.1044 / 0.1044 (0.0 %, 0.0σ); COHERENT 0.0971 / 0.0972 (+0.1 %, 0.0σ); FAST 0.1088 / 0.1088 (0.0 %, 0.0σ); imatrix STRIX_LEAN 0.0852 / 0.0836 (-1.9 %, 0.9σ); imatrix COHERENT 0.0769 / 0.0768 (-0.1 %, 0.1σ); imatrix FAST 0.0890 / 0.0891 (+0.2 %, 0.1σ). The two backends agree within noise on every file.

Quick start

1. Download

hf download kingjones777/Nex-N2.5-mini-ROCmFP4-GGUF --local-dir ~/models/nex

2. Build llama-server — ROCmFPX at the measured commit (ROCm and Vulkan prerequisites: the project's build guide). No patch.

git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git checkout d3ca537
HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DGPU_TARGETS=gfx1151 \
  -DGGML_HIP_GRAPHS=ON -DGGML_HIP_NO_VMM=ON -DLLAMA_CURL=OFF
cmake --build build --target llama-server -j

(The CMake options of the measured build are listed in Reproduction.)

d3ca537 is also in the history of the official ROCmFPX/ROCmFPX repository.

3. Serve

env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
build/bin/llama-server \
  -m ~/models/nex/Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf \
  --mmproj ~/models/nex/mmproj-Nex-N2.5-mini-BF16.gguf \
  --chat-template-file ~/models/nex/chat_template_enable_thinking.jinja --reasoning off \
  -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \
  -c 65536 --host 127.0.0.1 --port 8080

(LD_LIBRARY_PATH avoids a soname clash on machines that also have a Vulkan-only llama.cpp build.) The exact measured argv is in Reproduction.

4. Call — upstream sampling. Thinking is off unless the request sets "enable_thinking": true (as here; drop that line for a direct answer):

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "Hello"}],
    "temperature": 0.7,
    "top_p": 0.95,
    "top_k": 40,
    "chat_template_kwargs": {"enable_thinking": true}
  }'

Flag	Why
--chat-template-file …/chat_template_enable_thinking.jinja	The model's own template plus one line (see Reasoning controls). Without it llama-server leaves the reasoning in content and thinking-on tool calls fail (measured).
--reasoning off	Thinking stays off unless a request passes "enable_thinking": true.
--jinja	Already on by default in this build; keep it on — the reasoning controls (chat_template_kwargs) and tool calling rely on the Jinja chat template.
-fit off	Autofit reads MemAvailable on integrated GPUs and can silently shrink context or push tensors to CPU.
-cram <MiB>	Not set above (default 8 GiB of host RAM for saved prompts). Set it on a shared box — see Known issues.
--mmproj	Loads the 27-layer vision tower. Drop the flag for text-only.

Requires a llama.cpp build with ROCmFP4 / ROCmFPX tensor-type support; stock llama.cpp rejects these tensor types.

Reasoning controls

The model's own chat template switches thinking with chat_template_kwargs.reasoning_effort and ignores enable_thinking:

reasoning_effort	What the stock template emits
"none"	empty <think>\n\n</think> (no thinking)
"high"	opens <think>\n (always think)
"medium", unset, or anything else	opens <think> and lets the model decide (adaptive; upstream default is "medium")

llama-server decides how to split reasoning from the answer by rendering the template with enable_thinking on and off. With this template both renders are the same, so it never extracts the reasoning (measured). chat_template_enable_thinking.jinja in this repo is the model's chat_template.jinja (sha256 f1753536417ee87cded4bc5017354eb2123c079d20c634403a3d808b2ba3cc5b) with one line added at the top (file sha256 9183c7ba8510fb9628edd2265a4cad8c02b3aecec3cbdf65620f47494e75836d):

{%- if reasoning_effort is not defined and enable_thinking is defined %}{%- set reasoning_effort = 'high' if enable_thinking else 'none' %}{%- endif %}

Serve it with --chat-template-file and --reasoning off.

Measured on the standard STRIX_LEAN file with the included template file and --reasoning off (greedy probes: a direct question, a one-word instruction and a tool request, each with the tool schema attached; tool suite: 3 passes):

request	thinking	reasoning ends up in	measured
no chat_template_kwargs	off (the server default with --reasoning off)	—	0 of 3 replies with think tags in content
"enable_thinking": true	on	reasoning_content	tool suite with thinking on: 19/21 (a pass requires no think tags in content)
"enable_thinking": false	off	—	0 of 3 replies with think tags in content
"reasoning_effort": "none"	off	—	0 of 3 replies with think tags in content
"reasoning_effort": "high"	on	content — do not use	3 of 3 replies with think tags in content
"reasoning_effort": "medium"	adaptive	content — do not use	3 of 3 replies with think tags in content

So: switch thinking with enable_thinking only. Thinking-off tool checks: 21/21.

Upstream serving (SGLang) uses --reasoning-parser qwen3 --tool-call-parser qwen3_coder. Recommended sampling: temperature 0.7, top_p 0.95, top_k 40.

Earlier assistant turns are re-rendered with their reasoning (contexts grow faster than with templates that drop it). With thinking on and a small max_tokens, the whole budget can go to reasoning and content comes back empty — raise max_tokens before concluding the model is broken.

Tool calls use the XML-style <tool_call><function=…><parameter=…> format, which llama.cpp parses natively through the Jinja chat template (on by default).

Speed

File	Backend	Workload	Decode tok/s (min–max)	Prefill tok/s
Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf	ROCm0	code	63.94 (63.56–63.96)	1158
Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf	Vulkan0	code	68.16 (68.10–68.21)	1006
Nex-N2.5-mini-Q4_0-ROCmFP4-COHERENT.gguf	ROCm0	code	61.74 (61.74–61.80)	1187
Nex-N2.5-mini-Q4_0-ROCmFP4-COHERENT.gguf	Vulkan0	code	68.00 (67.88–68.09)	1001
Nex-N2.5-mini-Q4_0-ROCmFP4-FAST.gguf	ROCm0	code	63.31 (63.14–63.38)	1157
Nex-N2.5-mini-Q4_0-ROCmFP4-FAST.gguf	Vulkan0	code	68.62 (68.60–68.67)	999
Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf	ROCm0	code	63.29 (63.19–63.41)	1145
Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf	Vulkan0	code	67.77 (67.74–67.86)	995
Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-COHERENT.gguf	ROCm0	code	61.56 (61.53–61.63)	1184
Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-COHERENT.gguf	Vulkan0	code	67.81 (67.77–67.85)	999
Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-FAST.gguf	ROCm0	code	63.03 (62.55–63.11)	1154
Nex-N2.5-mini-imatrix-Q4_0-ROCmFP4-FAST.gguf	Vulkan0	code	68.44 (68.42–68.56)	994
Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf	ROCm0	prose	63.06 (62.94–63.09)	1139
Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf	Vulkan0	prose	67.34 (67.31–67.35)	978

STRIX_LEAN: ROCm0 decodes 6.2 % slower than Vulkan0 (63.94 vs 68.16 tok/s; ranges 63.56–63.96 / 68.10–68.21) and prefills 15.1 % faster than it (1158 vs 1006 tok/s). COHERENT: ROCm0 decodes 9.2 % slower than Vulkan0 (61.74 vs 68.00 tok/s; ranges 61.74–61.80 / 67.88–68.09) and prefills 18.5 % faster than it (1187 vs 1001 tok/s). FAST: ROCm0 decodes 7.7 % slower than Vulkan0 (63.31 vs 68.62 tok/s; ranges 63.14–63.38 / 68.60–68.67) and prefills 15.8 % faster than it (1157 vs 999 tok/s). STRIX_LEAN workload range on ROCm0: code 63.94 tok/s vs prose 63.06 (prose decodes within 1.4 % of code); Vulkan0 code 68.16 vs prose 67.34 (prose decodes within 1.2 % of code). Prompt lengths: code 7,094–7,102 tokens (the first 30,000 characters of convert_hf_to_gguf.py plus an instruction), prose 7,453–7,457 tokens (the first 34,000 characters of wikitext-2 train plus a writing instruction). Speed gaps below 3.0 % (decode) and 3.0 % (prefill) are called a tie: the larger of 3 % and the widest gap measured between files that do identical work per token (each imatrix file and its standard twin: decode 1.0 %, prefill 1.1 %).

Prompt caching

Measured: pairs of requests that share a long code prefix and differ only in the closing instruction. The second request of each pair runs warm (cache_prompt: true, resuming from what the first one left) and then cold (cache_prompt: false), and the two replies are compared byte for byte. Every prompt is padded to one token length so warm and cold see identical chunking.

server	second-request prompt tokens reused	processed	warm reply = cold reply
d3ca537, unpatched	6,011 of 7,039 (all 5 pairs)	1,028	5/5

In 5 request pairs sharing a long prefix at one fixed prompt length of 7,039 tokens, every second request resumed from the checkpoint the first one left 1,028 tokens before its end — 6,011 tokens reused (85 %), 1,028 processed — median prefill 1.0 s instead of 6.2 s cold (6.1× faster). Each warm reply was byte-identical to a cold run of the same prompt in 5/5 exchanges.

llama-server processes the last n_ubatch + 4 tokens of every prompt as two batches so it can checkpoint there (1,024 + 4 = 1,028 tokens with the -ub 1024 used in these measurements, where the server default is -ub 512; upstream PR #20288). A turn that resumes from a checkpoint left by a prompt of a different length therefore splits its tail differently from a cold run, and float rounding can flip a greedy token.

Tool calling

The template emits the XML-style <tool_call><function=…><parameter=…> format, which llama.cpp parses natively through the Jinja chat template (on by default). Suite run through llama-server, at the checkpoint's recommended sampling (temperature 0.7, top-p 0.95, top-k 40):

40/42 over three passes with the quick-start configuration, 6/14 with the stock template, run on Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf. Quick start = the included template file + --reasoning off, thinking switched with enable_thinking; stock = the model's own template, thinking switched with reasoning_effort (high / none). A check passes only with a native tool_calls entry carrying the right arguments and no raw XML or think tags left in content.

check	quick start, thinking ON	quick start, thinking OFF	stock template, thinking ON	stock template, thinking OFF
multi-arg	3/3	3/3	❌	✅
nested-object	2/3	3/3	❌	❌
enum	3/3	3/3	❌	✅
correct-decline	3/3	3/3	❌	✅
multi-turn	3/3	3/3	❌	✅
streaming	3/3	3/3	❌	✅
parallel	2/3	3/3	❌	✅

Stock chat template: 6/14. Every thinking-on check failed. Re-run with the raw replies kept, 7 of 7 thinking-on replies carried the reasoning and a </think> in content, and 0 had any reasoning_content. llama-server builds its reasoning parser by rendering the template with enable_thinking on and off; this template ignores enable_thinking (it switches on reasoning_effort), so the parser finds no reasoning markers and extracts nothing. Server switches did not help (--reasoning-format deepseek: 3 of 3 replies still had reasoning in content; --chat-template-kwargs with reasoning_effort: 3 of 3 replies still had reasoning in content; --reasoning on: 3 of 3 replies still had reasoning in content). With thinking off, nested-object failed on an HTTP 500 — see Known issues.

With the included chat_template_enable_thinking.jinja and --reasoning off (the quick start): 13/14, 13/14, 14/14 over three passes of the same suite (40/42) — thinking off 21/21, thinking on 19/21; the misses were nested-object with thinking on (pass 1), parallel with thinking on (pass 2). A pass requires a native tool_calls entry with the right arguments and no think tags in content. Each check is a single sample at the recommended temperature 0.7.

Vision

mmproj-Nex-N2.5-mini-BF16.gguf is the 27-layer vision tower (width 1152), loaded with --mmproj. Its attention follows the server's -fa setting, so both settings were checked.

Probe: a synthetic 448×448 image with a red circle and a blue square (a model that ignores the image cannot name both), sent to Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf with --mmproj, temperature 0. Pass = the reply names every expected term (red, blue, circle, square).

	-fa on	-fa off
STRIX_LEAN + projector	✅ 4/4 terms	✅ 4/4 terms

✅ Images work with -fa on and -fa off. The -fa on image test was repeated with the included template file and --reasoning off (the quick start): ✅ passed.

Reply (-fa on):

The image shows two shapes: a red circle on the left and a blue square on the right.

Memory

Measured footprint (drop in MemAvailable) of STRIX_LEAN with the vision projector, q8_0 KV cache, -cram 512, one slot, no draft head:

context	after load	after one request (30,000-character code prompt)
65,536	21.11 GiB	21.29 GiB
262,144	24.36 GiB	24.52 GiB

No row was decode-benchmarked beyond that one request. Nothing beyond these rows was measured.

Quantization methodology

# 1. convert: text model and the vision projector (the checkpoint has no mtp.* tensors)
python convert_hf_to_gguf.py hf --outtype bf16 --model-name Nex-N2.5-mini --outfile Nex-N2.5-mini-BF16.gguf
python convert_hf_to_gguf.py hf --outtype bf16 --mmproj --model-name Nex-N2.5-mini --outfile mmproj-Nex-N2.5-mini-BF16.gguf

# 2. quantize from BF16 only; the LM head is forced up on every tier and read back by exact tensor name
llama-quantize --output-tensor-type q6_K                            Nex-N2.5-mini-BF16.gguf OUT Q4_0_ROCMFP4_STRIX_LEAN 16
llama-quantize --output-tensor-type q6_K --token-embedding-type q6_K Nex-N2.5-mini-BF16.gguf OUT Q4_0_ROCMFP4_COHERENT 16
llama-quantize --output-tensor-type q6_K                            Nex-N2.5-mini-BF16.gguf OUT Q4_0_ROCMFP4_FAST 16

# 3. BF16 reference logits on the CPU only (this build's ROCm0 path computes the BF16 MoE wrong — Known issues)
llama-perplexity -m Nex-N2.5-mini-BF16.gguf -f wikitext-2-raw/wiki.test.raw -c 2048 -b 2048 --chunks 40 --kl-divergence-base bf16.kld \
  -dev none -ngl 0 --no-op-offload -t 16
# 4. grade each shipped file against those logits, on each GPU backend
llama-perplexity -m OUT --kl-divergence-base bf16.kld --kl-divergence -c 2048 -b 2048 -ngl 999 -fa on -dio -dev ROCm0
llama-perplexity -m OUT --kl-divergence-base bf16.kld --kl-divergence -c 2048 -b 2048 -ngl 999 -fa on -dio -dev Vulkan0

Receipts (the built file is the receipt — exact tensor names, never a substring match; recipe/logs/):

File	output.weight	token_embd.weight	tensors
Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf	Q6_K	Q5_K	733
Nex-N2.5-mini-Q4_0-ROCmFP4-COHERENT.gguf	Q6_K	Q6_K	733
Nex-N2.5-mini-Q4_0-ROCmFP4-FAST.gguf	Q6_K	Q4_0_ROCMFP4_FAST	733

tie_word_embeddings is false, so the output head is a real tensor and --output-tensor-type q6_K does real work. All three tiers pin output.weight to q6_K; COHERENT also pins token_embd.weight to q6_K, while STRIX_LEAN and FAST keep their tier's own embedding type (shown in the receipts).

Reproduction

server   : https://github.com/charlie12345/ROCmFPX @ d3ca537
           unpatched; build dir /opt/llama-rocm/rocmfpx-724/build-hipvk, Release, Unix Makefiles, GGML_HIP=ON GGML_VULKAN=ON
           GGML_HIP_GRAPHS=ON GGML_HIP_NO_VMM=ON GGML_NATIVE=ON AMDGPU_TARGETS=gfx1151 LLAMA_CURL=OFF
           CMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang
           sha256 llama-quantize   1559189968a30781f837e4d25ca3cdf26c8eb0351023675406535eddcbe0db13
           sha256 llama-imatrix    bfdce5c21a01bdc8043d69335b74a7b68b8335ae989991a1e10e21d5faeaeb8e
           sha256 llama-perplexity c2175925bb0c61df89ff787845d8e03a35403f684be52719ac2987b8ca636a53
           sha256 llama-server     d077b98f3f04df4b482a5826c8922e39cda3aa8d2e2b2fbe1870f36d245cebe7
source   : nex-agi/Nex-N2.5-mini  revision 87420286149d9cce9bd46cd335ef9bda33c37c1b
model    : Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf (the argv below; every file was measured the same way)
argv     : /opt/llama-rocm/rocmfpx-724/build-hipvk/bin/llama-server -m /mnt/models/nex-n2.5-mini/out/Nex-N2.5-mini-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui
template : the quick-start tool-suite and image rows add --chat-template-file chat_template_enable_thinking.jinja
           --reasoning off to this argv (recipe/pipeline/run_tools_roff.sh -> nex_tools_tpl.py; their server logs
           read "chat template, thinking = 0"); the speed rows use the stock template
env      : LD_LIBRARY_PATH=<build>/bin:/opt/rocm-7.2.4/lib
           HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1
box      : aimax — AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151), 124 GiB, GTT 131072 MiB,
           kernel 6.17.6-061706-generic, ROCm 7.2.4
protocol : 7,094–7,102-token code prompt, 256 generated tokens, temp 0 / top_k 1, ignore_eos, cache_prompt false,
           1 warm-up + median of 3, no co-resident models (box iced)
measured : 2026-09-16 to 2026-09-17, by the pipeline in recipe/ (every raw number in recipe/results_summary.json and recipe/raw/)

Files

File	Size	sha256
Nex-N2.5-mini-Q4_0-ROCmFP4-STRIX_LEAN.gguf	17.46 GiB	4437c3d8871acf04e4be8a9b773cf69697e5fc4b7b41fae41261f246ee5add9e
Nex-N2.5-mini-Q4_0-ROCmFP4-COHERENT.gguf	18.48 GiB	43673ce7b05f6d3924f6268e7e3102bab8fc4f91771505ef5734beaddacaa2f1
Nex-N2.5-mini-Q4_0-ROCmFP4-FAST.gguf	17.37 GiB	156b516ecfc0f8e400b165b7d42d959a0bd902814bb6949ca040c991494886d1
mmproj-Nex-N2.5-mini-BF16.gguf	0.84 GiB	a80a82977e4787a838a612316c79a6533c1b098fcac19372cc78929120162a4e
chat_template_enable_thinking.jinja	7.7 KiB	9183c7ba8510fb9628edd2265a4cad8c02b3aecec3cbdf65620f47494e75836d

SHA256SUMS covers every model file and the chat template file. recipe/ holds the measurement pipeline (recipe/pipeline/), raw per-run results (recipe/raw/), build and receipt logs (recipe/logs/), and results_summary.json with every measured value on this card. Architecture facts (layer counts, vocabulary, vision depth) come from the checkpoint's config.json at revision 87420286149d9cce9bd46cd335ef9bda33c37c1b.

Known issues and limits

• Do not run the BF16 GGUF of this model on ROCm0 with this build. d3ca537 computes it wrong on that path: wikitext-2 perplexity 105.9 on ROCm0 vs 6.23 on the CPU (first window 139.1 vs 5.70; also wrong with -fa off). The 4-bit files are not affected — their ROCm0 grades are in the table — and the BF16 file is not published here; the quality reference was computed on the CPU instead.
• llama-server rejects a tool call whose required arguments are not in the order the schema lists them — HTTP 500, The model produced output that does not match the expected peg-native format: this build's parser for the XML tool-call format expects required arguments in definition order. On the standard STRIX_LEAN file with the stock template and thinking off, the nested-object request (three required arguments) hit it in the tool suite and in 2 of 4 repeats; the model had written a well-formed call with the arguments reordered. Be ready to retry on this error.
• With the included template, do not set reasoning_effort to high or medium — the reasoning goes back into content (6 of 6 probe replies). enable_thinking: true is the way to turn thinking on.
• With thinking on, a very short answer can stay inside the think block. Served as in the quick start, Reply with the single word: ready (no tools, enable_thinking: true, temperature 0) came back with the word in reasoning_content and an empty content on 2 of 2 FAST files tested. With thinking off the same request returned ready in content.
• No MTP head. The checkpoint's config.json declares mtp_num_hidden_layers: 1, but the weights contain no mtp.* tensors (1,026 tensors total). There is no multi-token-prediction head and no speculative decoding on these files — do not pass --spec-type draft-mtp.
• The stock chat template ignores enable_thinking, and llama-server cannot separate its reasoning (see Tool calling). Serve with the included chat_template_enable_thinking.jinja and --reasoning off, and switch thinking per request with enable_thinking — see Reasoning controls.
• Earlier assistant turns are re-rendered with their reasoning, so multi-turn contexts grow faster than with templates that drop it.
• llama-server's host-RAM prompt cache defaults to 8 GiB (-cram 8192). On a shared box, set -cram explicitly.
• Measured on Linux only (Ryzen AI Max+ 395, ROCm 7.2.4, unpatched d3ca537).
• Not measured: decode beyond a 7,457-token prompt (a 262,144-token context was loaded in the memory test, not benchmarked), long-context quality, video input, concurrency above 1, and task-level accuracy. Perplexity/KLD measure next-token fidelity to BF16 on prose, not reasoning or code correctness.

License and attribution

Apache-2.0, inherited from the base model. Weights and architecture: Nex-AGI (nex-agi/Nex-N2.5-mini). ROCmFP4 / ROCmFPX quantization format and runtime: the ROCmFPX project. Quantization and measurements: kingjones777.