pmarquees/blot-mini

Blot Mini

Built with Mill, the agentic research environment from Succinct Labs. Blot Mini finds API keys and developer-secret spans for downstream redaction. Agents wrote its neural weights directly, with development feedback and fork-and-flush iterations. No gradients, optimizer training, pretrained tensors or teacher distillation were used to construct this detector.

The default download now contains the 9.17 MB precision winner, checkpoint precision-a-2026-11-04, published November 5, 2026. Its source, NumPy weights and Core ML package are byte-identical to the candidate frozen before the final evaluation. It is a research release; publication does not establish production readiness or update the macOS test app.

Paper

Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, MIT CSAIL. Published November 2, 2026.

This research article inspired Blot Mini's direct weight authoring: agents construct and edit neural coefficients without gradient descent. Blot Mini applies that idea to secret detection with a fixed lexical frontend and frozen evaluation. It is our adaptation, not an official implementation or reproduction of every condition in the article.

From a 1 MB first release to the 9 MB winner

Our first published model, r3a, packed 257,751 authored parameters into 1.045 MB of inference assets. On the full pinned CredData corpus (11,030 files), it scored 49.64% labeled-location F1, versus 59.57% for stock Gitleaks 8.30.1: a gap of 9.93 percentage points. This is about 16.7% lower in relative terms. r3a had already gone through three fork-and-flush rounds; it was not a one-shot result. CredData has known provenance overlap with its development data, so this is not an independent generalization claim.

We continued authoring the weights, and selected the 9.17 MB precision winner now downloadable here. On a separate frozen sample of 2,000 labeled GitHub issue candidates, it scored 63.95% F1 versus Gitleaks' 52.49%, a gain of 11.46 points. Its precision was 96.18%, versus Gitleaks' 98.35%; its recall was 47.90%, versus 35.80%. It caught more labeled secrets, while Gitleaks remained more precise.

These are different evaluations, not a same-benchmark before-and-after curve. The new checkpoint has not been run on the full CredData corpus, and its larger size alone is not established as the cause of improvement. The issue sample measures localized labeled-candidate detection, not exhaustive document precision or universal scanner replacement. See the historical CredData report and the current evaluation below.

Current download	Value
Checkpoint	precision-a-2026-11-04
Parameters	2,283,666
Network	Authored two-layer ReLU scorer; 5,153 features, 443 hidden units, scalar readout
Inference assets	9,166,928 bytes, approximately 9.17 MB
Dense FP32 coefficients	9,134,664 bytes
Format	NumPy weights and Core ML FP32; no INT4/INT8 export claimed
Core ML target	iOS 18 / macOS 15 operations
Input pipeline	Required Python lexical candidate and feature frontend, followed by authored neural scoring
Output	SECRET spans with Unicode code-point offsets and exclusive ends
Status	Private research release

Asset size includes model.py, weights.npz and the Core ML package. It excludes runtime dependencies, documentation, historical artifacts, compiled caches and app packaging. It is not RAM usage or a complete application's size. A smaller ZIP or download reflects compression, not a smaller deployed network.

Evaluation of the current downloadable checkpoint

Mill's research workflow continued direct weight authoring using two independent author branches with fresh conversations, adapting Fork-and-Flush. Only the weight builder and its permitted prefix specification could change; candidate extraction, features, scoring threshold and decoder were frozen. No gradients, optimizer, pretrained tensors, distillation or automatic parameter sweep were used. Development feedback still guided the weight edits.

The selected candidate added general weight-based inhibitors for ordinary web addresses, source paths, filename references and password placeholders, while preserving supporting evidence for credential-bearing formats. A competing branch exceeded 99% precision on author-development examples but lost too much CredData selection recall, so it was rejected before the final test. This is an adaptation of the research method, not evidence that branching itself outperforms a matched-budget single-agent control.

Frozen issue-candidate comparison

The held-out sample contains 2,000 labeled candidates from 2,000 distinct GitHub repositories, with 1,000 positive and 1,000 negative upstream labels, drawn from the public issue-report research dataset. Its source repository is Secret-Breach-Prevention-In-Software-Issue-Reports. This is a custom balanced subset, not a full CredData run or an industry-standard leaderboard.

Frozen system	TP	FP	FN	TN	Precision	Recall	F1
Immediate research parent, 6.98 MB; not published r3a	480	40	520	960	92.31%	48.00%	63.16%
Blot Mini precision A, 9.17 MB; current download	479	19	521	981	96.18%	47.90%	63.95%
Stock Gitleaks 8.30.1	358	6	642	994	98.35%	35.80%	52.49%

Compared with the immediate research parent, the candidate reduced false positives on the labeled candidates from 40 to 19 (52.5%), while positive detections changed from 480 to 479. Precision increased 3.88 percentage points, with a paired 95% bootstrap interval [+2.31, +5.56]. Recall changed −0.10 points, interval [−0.31, 0.00]; F1 changed +0.79 points, interval [+0.38, +1.22]. The predeclared improvement gate passed: precision's lower interval endpoint >0, recall's lower endpoint ≥−2 points, and no F1 point-estimate regression. Intervals use 10,000 paired repository resamples, seed 20261104.

Against Gitleaks, the candidate had higher primary F1 and recall, but lower precision. The F1 difference was +11.46 points, 95% interval [+8.63, +14.31]. These figures measure localized labeled-candidate detection: a match requires overlap covering at least 50% of both the labeled target and predicted span. They do not estimate exhaustive document precision, production prevalence, exact-span accuracy or universal scanner replacement. Trained Blot, detect-secrets and TruffleHog were not rerun in this round.

Mandatory alternative views are included in the complete results. Exact-boundary F1 was 24.78% for both parent and candidate and 22.91% for Gitleaks. Full-target coverage recall was 41.60% for the candidate versus 35.80% for Gitleaks. If every nonmatching output, including unannotated findings, is conservatively counted as false, candidate precision/F1 becomes 27.85% / 35.22%, versus Gitleaks' 34.29% / 35.03%. That sensitivity view is not complete ground truth and its small F1 difference does not establish superiority.

Independence, integrity and correction

The data and evaluator were frozen before candidate selection, and the selected model was frozen before one final inference pass per tool. All tools received the same complete normalized issue bodies. Models used 8,192-character windows with 512 overlap. Gitleaks used stock rules, default decoding and inline-allow behavior; all findings were mapped without unresolved or ambiguous locations. No active credential verification occurred. Raw texts, values and per-case predictions are not published.

The overlap audit rechecked 11,030 CredData files and 35 JSONL corpora and excluded all repositories from the preceding test. An earlier audit had missed a Prowl span representation: 83 of the previous 1,000 cases contained previously seen labeled values. The earlier clean independent-test interpretation was withdrawn. The new sample excludes the corrected exact overlaps and all previous test repositories. This does not prove that the old overlaps caused particular predictions. Procedural separation kept new test contents and feedback away from authors until candidate freeze; there was no OS sandbox, and unknown forks, paraphrases and author-model pretraining overlap cannot be completely certified.

The frozen candidate's Core ML and NumPy outputs matched on 353 invented control documents and logits matched on 256 arbitrary feature vectors. The full new holdout ran through the exact sparse NumPy matrices; it was not a separate full Core ML or physical-device evaluation. The 9.17 MB figure is uncompressed inference assets, excludes runtime dependencies, and remains below the 30 MB research cap.

Candidate identity	SHA-256
model.py	a881b7aed53b012c7cac1b2d532c26cf628a7748cdba81f148b023874de689bc
weights.npz	b0b4b1ce704651f3f4429cc83f1138537b4ae339df20dda0dad1a2e7a1b0b3ba
New holdout	7b2ea9d01ee6c4160954caad55011b1db6e78309ae18e5cd5642d758ecf58022

See the predeclared protocol, aggregate metrics and intervals, candidate freeze, native validation, integrity receipt, and old-test correction. This holdout is now consumed; future optimization requires a new test for fresh generalization evidence.

Run locally

Use Python 3.12. For native Core ML inference, use macOS 15 or newer. Access to this private Hugging Face repository is required to download it; subsequent inference is local and does not verify whether credentials are active.

hf auth login
hf download pmarquees/blot-mini --local-dir blot-mini
cd blot-mini
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python model.py --coreml < examples/input.jsonl

Omit --coreml to run the exact sparse NumPy matrices. Each input line is JSON with a text field; each output line is a list of {start, end, label} objects. Offsets count Python Unicode code points, not UTF-8 bytes, UTF-16 code units or grapheme clusters. Scores are not calibrated probabilities. The invented examples have expected outputs, checked in both runtimes for this publication.

The raw Core ML graph consumes extracted feature arrays, not strings. Keep model.py, weights.npz and bundle/ together. This is not a Transformers AutoModel checkpoint or a native iOS SDK; deploying in an Apple app requires a frontend/decoder port and device validation.

The CLI processes each supplied text directly. To reproduce the benchmark on long documents, use the evaluator's fixed 8,192-character windows with 512 overlap, map outputs to original offsets and deduplicate identical spans. Direct full-document inference can differ at window boundaries; do not assume it reproduces the tiled benchmark.

python build.py reconstructs the authored arrays and exports Core ML. Run it in a separate copy to retain the frozen release bytes: serialization metadata can change even when numerical weights agree. frozen.json identifies the benchmarked inference files; SHA256SUMS covers the published files except itself.

How Mill made it

This experiment adapts Sparks of Weight-Space Programming by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. Agents wrote code that materializes the network's weight arrays. Prefix observations and contextual evidence become ReLU conjunctions, while negative readout coefficients inhibit placeholders, references and ordinary paths. Decisions use the saved matrices.

The detector includes a deterministic lexical candidate/feature frontend. It is a hybrid system, not an end-to-end neural text encoder. The work does not establish that representing these conditions as weights beats an equivalent symbolic implementation.

Mill's workflow adapted Fork-and-Flush: start independent authors in fresh conversations from a frozen parent and concise lessons, evaluate proposals on selection data, keep the eligible winner and discard unsuccessful branches. The original r3a release used three rounds with two authors per round. Later rounds improved recall and precision; the final precision round used two fresh authors with only the weight builder and permitted prefix specification editable. A separate evaluator held the final sample out until candidate freeze.

Development labels and feedback informed the edits. No gradient training does not mean no examples, no model selection or no pretrained knowledge in the authoring LLMs. There was no matched-budget single-author control, so this is not causal evidence for branching itself.

Release history and verification

The previous 1.045 MB r3a checkpoint remains available at its immutable revision:

hf download pmarquees/blot-mini --revision 371716aaae546de275aa98bb58a3c22394b14f88 --local-dir blot-mini-r3a

Its historical ProwlBench detection F1 was 86.19%, versus the previously recorded trained Blot v6a's 82.94%, on the same reused 24,603-snippet development corpus. These are r3a measurements, not results for the current 9.17 MB checkpoint. ProwlBench, CredData and the issue sample use different units and scoring rules; their numbers are not interchangeable. The historical model card and r3a evaluation files preserve that evidence.

Native Core ML and NumPy matched on 353 invented controls and 256 arbitrary binary feature vectors during candidate validation. Zeroing the readout removed every control output. The full issue holdout used NumPy, not a second full Core ML run. Publication additionally checks frozen hashes, regenerated numerical arrays, native/reference example parity and downloaded artifact integrity. See publication validation. No physical-device test is claimed.

Limitations and provenance

This release can miss credentials and flag benign text. It does not scan Git history, implement all scanner allowlists, verify live credentials or guarantee safe redaction. Exact-boundary results are much lower than the primary overlap metric, and full-target coverage is reported separately. Production traffic and complete annotation are needed before interpreting these results as operational precision.

The supplied pattern workbook and Blot synthetic generators informed format hypotheses. References included Gitleaks, detect-secrets and SecretBench. Neither scanner is the inference engine or a teacher. Trained Blot, detect-secrets and TruffleHog were not rerun in the precision comparison.

Development and evaluation used ProwlBench revision 06f6d2cdf6a64c6d77ffb95a8dfb7abf7913b885 (CC BY-NC 4.0), CredData, and the issue-report replication dataset (source receipt: CC BY 4.0). Dataset terms remain separate from this private model publication. Raw benchmark texts, secret values, per-case predictions and author conversations are not redistributed.

research/precision-2026-11-04/ contains the current checkpoint's frozen aggregate evidence and overlap correction. history/r3a/ contains superseded evaluation records. Root inference files, examples and frozen.json refer to the current precision winner.