turnercore/needle2-automaticity-v9

Needle 2 — Automaticity V9 (negative result)

This is an experimental Cactus Needle 2 archive fine-tuned for Automaticity V9 function calling. Do not deploy it as a replacement for Needle Point / ln-point-v9. It is published for reproducibility of a completed negative result.

The artifact is a Cactus-engine .cact archive, not a Transformers or PyTorch checkpoint. It requires the pinned official Needle 2 engine family. It deliberately does not include the pickle LoRA adapter, training rows, benchmark rows, predictions, or environment files; those are retained in the private evidence repo.

Identity

Item	Value
Upstream Needle source	cactus-compute/needle@571fcd68f48b6649c91f353eb91d0a0c9b9135ce (v2.0.9)
Official engine	2.0.3
Base model	Cactus-Compute/needle2@98fbd955b0347e78059be0c253cc1ffa09b87bc7
Training	4,900 V9-native rows; 1 epoch; rank-16/alpha-32 LoRA; LR 1e-4; seed 0
Export	405 tensors; 13,737,807 bytes
needle2-automaticity-v9.cact SHA-256	4f19e7fb27e8c11780df8f2497b26dd96aef68b112c1a158a5e2fec68f4d7c70
Canonical implementation	turnercore/ai-gateway@1313c64c7a04232af7f1b5534a902d7f63ffaf50

Sealed Automaticity V9 benchmark v1.1

The unchanged private benchmark had 1,050 rows and SHA-256 c24db4269886018a400996d4d5e7c6f5ad9fad31ead5c6aab2a90217f691c9bb. Needle 2 and the retained Needle Point reference used the same rows, scorer fields, and semantic tool catalogues. Benchmark rows and labels were never used for training, prompt selection, decoding selection, or conversion tuning.

Metric	Needle Point v1	Needle 2 FT
End-to-end exact	96.10%	70.38%
Routing / contract match	98.95%	70.76%
Capability accuracy	100%	100%
Argument accuracy	96.19%	74.95%
Action exact	86.86%	0.32%
Schema-valid output	100%	100%
No-tool precision / recall	99.46% / 100%	70.69% / 100%
Wrong-tool / wrong-argument	11 / 30	307 / 4

Needle 2 preserved structural validity but mostly returned no tool where an action was required. The candidate retrieval stage retained the gold tool for all action rows, so this is not a retrieval-recall issue.

Runtime caveat

Needle Point used a warm persistent PyTorch ROCm runtime (324.81 ms average); Needle 2 used the official CPU engine with per-row agent binding (875.30 ms average). These are runtime-specific operational measurements, not a pure model-speed comparison. The tuned Needle 2 archive reports confidence: null by upstream design because the confidence head is not fine-tuned.

Files

• needle2-automaticity-v9.cact: final runnable archive.
• ab-comparison.json: aggregate metrics, failure counts, identities, and runtime confounders; SHA-256 c9ae921d5310722df0d48528715f41115c4b4a3463a628e98d2c443f4acc8905.
• provenance.json: safe, machine-readable artifact and source identities.
• artifact-runtime-smoke.json: sanitized representative official-engine smoke.

The native dataset, selection audits, benchmark rows, predictions, LoRA adapter, and detailed execution evidence live in the private dataset repo turnercore/automaticity-v9-needle2-experiment.