Latent tool-memory: trained probes and surfacing triggers (Qwen3-8B)

Trained readout probes and memory-surfacing triggers from the five-experiment latent tool-memory line (MATS). All artifacts read Qwen/Qwen3-8B residual-stream hidden states. Code and usage live in Sid-MB/mats-gf-latent-tool-memory; the training/eval data is in siddharthmb/mats-gf-latent-tool-memory-data.

Hidden-state convention throughout: hidden_states index = layer, with 0 = embeddings; features are the residual at the stated layer, L2-normalized unless a config says otherwise.

`exp2/probes/` — pre-generation tool-selection readout (experiment 2)

Trained on the 840-prompt BFCL v3 manifest (exp2/ in the data repo), features = unit-normalized last-prompt-token residuals.

File	What it is
`probe_a_best.joblib`	tool-needed logistic head, layer 28 (held-out AUROC 1.0)
`probe_b_slot_best.joblib`	which-slot logistic head, layer 29 (0.717 vs 0.417 TF-IDF)
`probe_b_retrieval_best.joblib`	ridge map to centered tool embeddings (unseen-tool retrieval; bounded at its shuffled floor)
`steering_directions_layer29.pt`	slot-pair difference-of-means directions, keys `"i->j"`

Load the .joblib heads with joblib.load; they are scikit-learn estimators.

`exp4/` — surfacing triggers, sensor-fleet task (experiment 4)

Decode-time triggers deciding when to surface a memory-store lookup. Features: L2-normalized layer-20 residual at each decode step.

triggers/ — need-labeled probe (probe.npz: w [4096], b, layer), FLARE logprob threshold (flare.json), adaptive classifier (adaptive.npz), P′ identity readout (pprime.npz; reads at chance, a recorded negative), chosen operating points (chosen.json).
triggers_p2/ — the retrieval-readiness probe (same file layout; chosen.json has τ=0.75): fires when the generated text can name the needed store entry. Beats FLARE on every slice (0.741 vs 0.598 overall).
triggers_km/ — text-side key-match trigger: km_config.json (embedder, window, θ) + km_clf.npz (logistic on [top1, margin, delta3, wlen]).

import numpy as np
d = np.load("triggers_p2/probe.npz")          # w, b, layer (=20)
score = 1 / (1 + np.exp(-(h20_norm @ d["w"] + d["b"])))  # fire if > tau

`exp5/triggers/` — in-domain retrained triggers, BFCL v3 multi-turn (experiment 5)

The frozen experiment-4 probe transfers at chance (AUROC 0.4996); these are the cheap in-domain retrains. Same layer-20 feature convention.

probe.npz + probe_chosen.json — retrained readiness probe (τ=0.998, dev event-F1 0.634; test P 0.66 / R 0.58; trained on 359 episodes).
probe_frac10.npz / probe_frac25.npz (+ _chosen.json) — label-efficiency variants trained on 10% / 25% of episodes.
km_clf.npz + km_chosen.json — retrained key-match classifier.
flare.json, retau.json — FLARE threshold and the dev-recalibrated frozen-probe threshold (both recorded baselines).
labels.parquet, summary.json, chosen.json — training labels and selection records.

Not included: the 49 GB of training-time step-table caches and floor-rollout intermediates behind these files, and all raw activation captures — not published; regenerable from the episodes + model with the repo code, or available on request.