Safka532/matched-data-small-gaps-specialists

Specialist classifiers and cached features for "Matched data, small gaps"

Fine-tuned Ettin 150M encoders (the paper's specialist) and the cached features its analyses read, for Banking77, CLINC150 and MASSIVE. The code, frozen protocols, item lists, saved model replies and analyses are in the paper's release repository (https://github.com/SafwanKamal/matched-data-small-gaps); this repository holds the files too large for git. Paper (preprint): https://doi.org/10.5281/zenodo.23448302. Every file here is listed with its SHA-256 in SHA256SUMS (and in the release's release/SHA256SUMS).

Layout

Paths mirror the release repository, so the files can be copied into a clone as they are.

Path	What it holds
artifacts/confirmatory/specialists/<variant>/<set>/seed-<seed>/	The confirmatory specialists: encoder/ (weights and tokenizer), head.pt (classification head), metrics.json, test_scores.npz
artifacts/confirmatory/features/	Cached logits and detector inputs for every confirmatory turn and development item, per set and seed
artifacts/massive-robustness/specialists/, features/	The same for the three further MASSIVE draws
artifacts/revision/features/	Features for the earlier exploratory analyses (Banking77, CLINC150, MASSIVE)
artifacts/robustness/<set>/seed-<seed>/	Exploratory seed runs (Banking77, CLINC150, MASSIVE)

Training

• Base model jhu-clsp/ettin-encoder-150m (revision pinned in primary_protocol.json), with a mean-pooled linear classification head, fine-tuned on the lower-cased, punctuation-stripped training rows of each set.
• Learning rate and epochs: Banking77 1e-4 for 6 epochs; CLINC150 5e-5 for 3; MASSIVE 2e-4 for 6 (chosen on an earlier 60-label task, not re-swept).
• Seeds 20261007, 20261015 and 20261016. Temperature is fitted on development in-scope items; the scope decision uses a development-selected detector (maximum softmax probability, energy, Mahalanobis or kNN) with a threshold weighted by a 20% out-of-scope prior.
• The full rules are in confirmatory_protocol.json and massive_robustness_protocol.json in the release repository.

Not included

Run D's specialists and features and the exploratory BANKING77-OOS and MultiDoGO runs are not published here: BANKING77-OOS states no licence for its split, and the MultiDoGO runs (CDLA-Permissive-1.0) are exploratory. Their hashes are in the release's release/SHA256SUMS, and the code rebuilds them from the frozen item lists.

Licences

The weights are released under MIT, like the base model. The training data carry their own licences: Banking77 CC BY 4.0, CLINC150 CC BY 3.0, MASSIVE CC BY 4.0. Attribute them when you use these files.

Citation

@misc{kamal2026matched,
  title     = {Matched data, small gaps: a pre-registered comparison of fine-tuned classifiers, retrieval-prompted language models and a typed-decision model for intent and out-of-scope decisions},
  author    = {Kamal, Safwan},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.23448302},
  url       = {https://doi.org/10.5281/zenodo.23448302},
  note      = {Preprint}
}

The DOI above always resolves to the latest version; version 1 is https://doi.org/10.5281/zenodo.23448303.