ruvnet/ruvector-typesafe-banking77

ruvector-typesafe — Banking77 bank

A trained example bank for @ruvector/typesafe: typed decisions over text, locally, with no API bill and no network in the decision path. 77 fine-grained banking intents — the standard hard test for intent classification.

npm install @ruvector/typesafe
curl -LO https://huggingface.co/ruvnet/ruvector-typesafe-banking77/resolve/main/bank.json
curl -LO https://huggingface.co/ruvnet/ruvector-typesafe-banking77/resolve/main/questions.json

echo "my card was declined at an atm" \
  | npx typesafe decide --questions questions.json --bank bank.json --embedder onnx \
      --engine-options '{"probeIterations":4000,"probeClassBalanced":true,"head":"probe"}'

The --engine-options are not optional. A bank stores examples and their frozen splits — never hyperparameters — and the head is refit from it on load. Omit them and you refit with the library defaults (400 iterations, head: auto), which is a different and materially worse model than the one measured below. These are the exact options this bank was trained under.

What this artifact is

Labelled examples with frozen split assignments — not weights. The heads (nearest-prototype, or a multinomial probe once a class has enough examples) and the temperature calibration are refit from the bank when the engine loads it.

Two consequences worth knowing:

• The bank is encoder-independent — it holds text and content-hashed split tags, nothing encoder-derived. For this dataset that is measured, not assumed: both bundled encoders exported byte-identical banks. Only the accuracy below is encoder-specific.
• The first decision after loading is slow. importBankJson just admits the examples; the head is fitted lazily on the first decide, and at probeIterations: 4000 over 9,993 examples and 77 classes that fit is minutes, not milliseconds. Every later call is the steady-state latency in the table. Import once, warm it with one throwaway decision, and keep the engine alive — do not load a bank per request.

A reloaded bank reproduces the trained engine's answers exactly — that round-trip is asserted by test/bank-roundtrip.test.mjs, not assumed.

Accuracy

Held-out test split, 3,077 utterances, 77 classes.

encoder	accuracy	p50 latency	p95 latency
all-MiniLM-L6-v2	76.7%	15 ms	16 ms
bge-small-en-v1.5	87.0%	6 ms	10 ms

Trained on 9,993 labelled examples (splitsHash: f51a48af2895c08d…).

Training

node scripts/typesafe-banks/build-bank.mjs --dataset banking77 --encoder all-MiniLM-L6-v2

The probe head is full-batch gradient descent with a fixed iteration count, and that count is the thing to tune when you add data: the default 400 iterations fits ~1k examples well and underfits ~10k badly. Raise it through EngineOptions:

createTypesafe({ embedder: …, engine: { probeIterations: 4000 } })

Limitations

• English only; both bundled encoders are English sentence encoders.
• The label set is closed. New intents need new examples and a refit.
• Accuracy is reported on this dataset's own test split — it is not a claim about your traffic.

Credit

Casanueva et al., Efficient Intent Detection with Dual Sentence Encoders (2020), arXiv:2003.04807. Dataset licence: CC-BY-4.0; this bank redistributes the utterance text under that licence. The @ruvector/typesafe code is MIT.