mattjoyce/antares-1b-GGUF

Antares-1B GGUF — the measured ladder

Nine quantizations of fdtn-ai/antares-1b (Cisco Foundation AI's security SLM — a CWE localizer that explores a repository through a terminal and submits a ranked list of suspect files; a fine-tune of ibm-granite/granite-4.0-1b).

Every file here was benchmarked before publishing on the model's own vulnerability-localization benchmark — the full 499-task set, detection (Phase A) and clean-tree false-alarm rate (Phase B), with three-run error bars on six of the nine rungs — and every file's sha256 is published. As of 2026-08-02, none of the dozen-plus community GGUF repos of this model we surveyed publish per-file hashes or measurements, and we found no imatrix builds among them.

Which file do I want?

You want	Take	Why
Best quality per GB	antares-1b-Q5_K_M.gguf	0.1723 ± 0.0014 vs the BF16 ladder cell (×3): Δ −0.002 ± 0.004, at 36% of the size. This recommendation is robust to every noise story on this card
Smallest safe file	antares-1b-Q4_K_M-imat.gguf	Recovers half of static Q4's quality gap at identical file size, and is the best retriever measured (three-run bars both ways) — also robust
Tightest VRAM, eyes open	antares-1b-IQ4_XS.gguf	Good mean, but the widest run-to-run spread measured (span 0.0154, n=3) — not a robust recommendation; see the table note
Reference / re-quantizing	antares-1b-BF16.gguf	The conversion baseline every claim is measured against
❌ Not these	Q3_K_M, Q2_K	Included to document the cliff: Q3 loses 4× the F1 with 51% abstention; Q2 cannot emit one valid tool call. They are data, not deployables

Retriever view — if you wire Antares as a cheap filter ahead of a stronger judge, the deciding metric is hit@all, not F1: Q4_K_M-imat is the best measured (0.350 ± 0.017, n=3, vs static Q4's 0.331); IQ4_XS is the worst healthy rung (0.287 ± 0.016) because it submits half as many files.

The measured ladder

File F1 on the 499-task benchmark (Phase A), abstain rate, and Phase B true-negative rate on clean trees. Rungs 0–6 measured 2026-07-30/31; rungs 7–8 measured 2026-08-01; BF16 and Q8_0 re-measured ×3 on 2026-08-02.

#	File	imatrix	Size	% of BF16	File F1	abstain	TNR
0	BF16	—	3.68 GB	100%	0.1740 ± 0.0074	0.9%	0.006
1	Q8_0	—	1.96 GB	53%	0.1741 ± 0.0035	1.6%	0.014
2	Q6_K	—	1.51 GB	41%	0.1622	1.4%	0.030
3	Q5_K_M	—	1.32 GB	36%	0.1723 ± 0.0014	1.2%	0.020
4	Q4_K_M	—	1.14 GB	31%	0.1610 ± 0.0010	3.7%	0.022
7	Q4_K_M-imat	✓	1.14 GB	31%	0.1667 ± 0.0011	2.7%	0.036
8	IQ4_XS	✓	1.05 GB	28%	0.1648 ± 0.0083	2.7%	0.034
5	Q3_K_M	—	0.95 GB	26%	0.0402	51.3%	0.539*
6	Q2_K	—	0.77 GB	21%	0.0000	100%	1.000*

Table notes:

• ± values are 1 sd over three full benchmark runs; every triple is published here: BF16 0.1735/0.1669/0.1817 · Q8_0 0.1721/0.1720/0.1781 · Q5_K_M 0.1720/0.1738/0.1711 · Q4_K_M 0.1602/0.1608/0.1621 · Q4_K_M-imat 0.1679/0.1659/0.1663 · IQ4_XS 0.1742/0.1588/0.1613. (Each sd carries 2 degrees of freedom — read spans, not third decimals.)

• Run-to-run noise is rung-dependent. K-quant triples span ≤ 0.003; BF16's own three runs span 0.0148 and IQ4_XS's 0.0154. Cells without ± (Q6_K, Q3, Q2, and all single-run TNR cells) carry doubt at whatever their rung's scale turns out to be — rungs are not rank-ordered within noise. Q6_K's dip below both neighbors is a single-run cell inside that noise, not a measured effect.

• A correction we're proud of: this table's first draft showed single-run Q8_0 (0.1798) above single-run BF16 (0.1670). Re-running both ×3 dissolved the gap entirely (both means 0.174, Δ = 0.000 ± 0.005) — a high draw had met a low draw. We re-measured our most interesting number before believing it; treat anyone's single-run cells (ours included) with the same skepticism.

• TNR cells are single runs of a binary outcome over 499 clean trees: binomial se is roughly ±0.004–0.008 at these rates (Q5's Phase-B triple: 0.020/0.006/0.014). Don't rank healthy rungs on TNR — most differences sit within a couple of se.

• *Q3/Q2's "good" TNR is a Goodhart artifact — a model that abstains on everything never false-alarms. Pair TNR with the detection column, always.

• IQ4_XS: the widest spread on the board at n=3 — suggestive, not proven, but treat any single-run IQ4_XS benchmark (ours or anyone's) as uninformative.

• None of these rungs is a stand-alone merge gate. Healthy rungs flag something on 96–99% of clean trees (Phase B). Treat output as leads for a human or a stronger model — which matches the upstream guidance ("a lead to verify, not proof").

• Token-level quant-damage metrics (KLD / top-token agreement vs BF16) were not measured — known future work; these numbers are task-level.

We measure ≈0.174 — and the serving stack is not the reason we miss 0.209

Cisco's model card reports 0.209 (three-run mean, their internal pipeline). Across seven full-corpus BF16 runs — two serving engines (llama.cpp and vLLM agree within 0.004 on the same weights), three sampler configurations, and a fresh ×3 repeat on 2026-08-02 — we measure 0.1735 ± 0.0057, and no run of the seven exceeded 0.1817: the published figure is not reachable by run-to-run luck. (The ladder table's BF16 cell is the ×3 subset run under the exact ladder protocol; this pooled family is the anchor.) We did not reproduce the published figure, and the serving stack is exonerated as the cause. The residual is consistent with either a reference-pipeline difference (the model card's usage snippet renders prompts differently than the published runner) or a checkpoint-labeling difference (the leaderboard's SFT row is 0.188) — neither verifiable from published artifacts. All rung claims in this card are relative to this repo's own BF16 anchor, which makes them internally consistent regardless of the absolute-scale question.

Serving — the traps that silently ruin results

Measurement configuration (what these numbers were produced with):

• llama.cpp llama-server, 4 slots × 32k ctx, raw /v1/completions — the agent harness formats its own prompts using the model's granite chat markers; the GGUF's embedded chat template is not applied during measurement.
• Server-side sampler pins: --top-k 0 --top-p 1.0 --min-p 0.0; the harness supplies temperature 0.3, top_p 1.0 per the published protocol. Pin the server: llama.cpp otherwise defaults top_k=40, top_p=0.95, min_p=0.05 for fields a client omits — a different model than the one measured.

Traps:

1. Ollama's /v1/completions is not raw. It wraps prompts in the GGUF's chat template — no error, degraded results. If you must use Ollama, set an identity template (TEMPLATE {{ .Prompt }}) and verify with a token-count sentinel (tokenize your prompt separately; compare prompt_tokens).
2. Double-BOS. The tokenizer adds BOS; if your client also prepends one, quality drops silently. The sentinel check above catches this too.

imatrix provenance

antares-1b.imatrix (567 chunks, final PPL 12.52) was computed from calibration.txt (included, 968 KB): 5 generic CWE-class prompts in agent framing, 40 agent transcripts from an out-of-corpus repository run, and 68 multi-language source files (C/C++, Python). Decontamination method, stated plainly: the corpus was constructed from sources that exclude the benchmark corpus and spot-checked by string search (0/25 sampled eval-task CWE descriptions found); no automated n-gram decontamination pass was run. Both imatrix rungs were quantized from the hash-verified BF16 below. One honest caveat: this corpus is a single point in a large space of reasonable calibration choices, no second corpus was tried, and the imatrix rungs' scores are conditional on it.

Provenance & reproducibility

• Upstream: fdtn-ai/antares-1b @ revision 10417eb35641b32e7141157db19c76eb545193b6.
• Conversion: llama.cpp f5b9bd3, convert_hf_to_gguf.py → BF16. The granitemoehybrid class converts to plain granite: the checkpoint is dense (363 tensors, zero experts, zero state-space tensors — verified by counting, not by trusting the label).
• Quantization: llama-quantize (same commit); imatrix rungs via llama-imatrix -m BF16 -f calibration.txt -ngl 99 then llama-quantize --imatrix antares-1b.imatrix <BF16> <out> {Q4_K_M|IQ4_XS}.
• Changes vs upstream: format conversion and quantization only — no fine-tuning, no re-training, no merges.
• IQ4_XS requires a llama.cpp new enough for IQ quants (any 2024+ build; the measured server was f5b9bd3).
• K-quants are not byte-reproducible across CPU architectures — the same command on ARM and x86 produces files differing in ~0.016% of bytes (weights sitting on a quantization-level boundary land on different sides depending on accumulation order). "Q4_K_M" is a recipe, not a file. These are the measured bytes:

e8b3c75677beb1044e64cdefe05cf151ab22c6e0a4e119d338876ee2b580bbac  antares-1b-BF16.gguf
e9f27ab7c232536f74721f79917beefdd0fbfcdc0baf313b86ea82701c40d51c  antares-1b-Q8_0.gguf
057b5523f25f428d2f78e59c5760638f93b54fee871dea8651b0fc20cba0e6a9  antares-1b-Q6_K.gguf
675106237244c22e5b69d90fc756a18ac81e09547579f67607373cb59948f4f1  antares-1b-Q5_K_M.gguf
9fe52cdd2036e165a9d355a2ea488f248fd0fd4839cdfff0a5b41ebf5f790a85  antares-1b-Q4_K_M.gguf
e51582e2c4b1f796aff3a8318b372bd46e2efab0fb4b31a6d01f53f05dd71687  antares-1b-Q4_K_M-imat.gguf
f1eb03cb07e0d0a80a3d09037acb3dcd5b85032f01525b71a88cadff92132612  antares-1b-IQ4_XS.gguf
269eaf3b34efcce471277e7634c2dca8c12665d78657c00c14036812208dccbf  antares-1b-Q3_K_M.gguf
04570f582bcdd1b2310d5edae67433c89ff9d46bd7629590be14fcfc3736698b  antares-1b-Q2_K.gguf
8eb4e617ce8a7921171fc7e40a6ab991c03d036058ea8cec69dd3a970780f196  antares-1b.imatrix

• Benchmark: the model's own vulnerability-localization benchmark, full task set. One task references a deleted GitHub repository, so the denominator is 499, not 500 — held identical on every rung (a fixed client timeout was raised so the three largest repos couldn't time out on slow rungs only).
• Measurement hygiene, every session: corpus raw-verified from disk before any GPU time; a prompt-fidelity sentinel (tokenize count == prompt_tokens) after every server start; orphaned benchmark containers swept between runs; model bytes hash-gated against the sums below before every measurement.
• Hardware: RTX 5070 Ti 16 GB, Ubuntu 24.04, CUDA sm_120 build, one runtime across every rung.
• Full methodology, raw per-run aggregates, and the harness scripts: (links land here when the companion repo and writeups go public).

Intended use & limitations

This is a 1.8B-parameter localizer, not an authority. It ranks files that deserve a look for a stated weakness class; it does not explain, patch, or prove anything. On its own benchmark the best run finds a relevant file about one time in three, and it flags something on nearly every clean tree — so wire it as a triage assistant feeding a human or a stronger reviewer, never as an autonomous blocker. Measured on its own benchmark corpus only; behaviour elsewhere is uncharacterized beyond one field test.

License

Apache-2.0, inherited from fdtn-ai/antares-1b (a full copy is in LICENSE; upstream ships no NOTICE file). The upstream repository is gated — if you want the original safetensors, the CLI tooling, or the benchmark, go through their gate; this repo exists to publish measured quantizations, not to route around upstream. If you are the upstream team and want anything here changed, open a discussion.