ruvnet/ruvector-typesafe-hwu64

ruvector-typesafe — HWU64 bank

A trained example bank for @ruvector/typesafe: typed decisions over text, locally, with no API bill and no network in the decision path. 64 home-assistant intents across 21 domains.

npm install @ruvector/typesafe
curl -LO https://huggingface.co/ruvnet/ruvector-typesafe-hwu64/resolve/main/bank.json
curl -LO https://huggingface.co/ruvnet/ruvector-typesafe-hwu64/resolve/main/questions.json

echo "set an alarm for seven in the morning" \
  | npx typesafe decide --questions questions.json --bank bank.json --embedder onnx \
      --engine-options '{"probeIterations":4000,"probeClassBalanced":true,"head":"probe"}'

The --engine-options are not optional. A bank stores examples and their frozen splits — never hyperparameters — and the head is refit from it on load. Omit them and you refit with the library defaults (400 iterations, head: auto), which is a different and materially worse model than the one measured below. These are the exact options this bank was trained under.

What this artifact is

Labelled examples with frozen split assignments — not weights. The heads (nearest-prototype, or a multinomial probe once a class has enough examples) and the temperature calibration are refit from the bank when the engine loads it.

Two consequences worth knowing:

• The bank is encoder-independent — it holds text and content-hashed split tags, nothing encoder-derived. Verified directly on the banking77 bank, where both bundled encoders exported byte-identical banks; here only one encoder was run. Only the accuracy below is encoder-specific.
• The first decision after loading is slow. importBankJson just admits the examples; the head is fitted lazily on the first decide, and at probeIterations: 4000 over 19,552 examples and 68 classes that fit is minutes, not milliseconds. Every later call is the steady-state latency in the table. Import once, warm it with one throwaway decision, and keep the engine alive — do not load a bank per request.

A reloaded bank reproduces the trained engine's answers exactly — that round-trip is asserted by test/bank-roundtrip.test.mjs, not assumed.

Accuracy

Held-out test split, 6,133 utterances, 68 classes.

encoder	accuracy	p50 latency	p95 latency
bge-small-en-v1.5	79.9%	5 ms	7 ms

Only bge-small-en-v1.5 was measured for this bank. The bank itself is encoder-independent — the other bundled encoder will load it, it just has no number here.

Trained on 19,552 labelled examples (splitsHash: 068e232f6f38b0c9…).

Training

node scripts/typesafe-banks/build-bank.mjs --dataset hwu64 --encoder bge-small-en-v1.5

The probe head is full-batch gradient descent with a fixed iteration count, and that count is the thing to tune when you add data: the default 400 iterations fits ~1k examples well and underfits ~10k badly. Raise it through EngineOptions:

createTypesafe({ embedder: …, engine: { probeIterations: 4000 } })

Limitations

• English only; both bundled encoders are English sentence encoders.
• The label set is closed. New intents need new examples and a refit.
• Accuracy is reported on this dataset's own test split — it is not a claim about your traffic.

Credit

Liu et al., Benchmarking Natural Language Understanding Services (2019). Dataset licence: CC-BY-4.0; this bank redistributes the utterance text under that licence. The @ruvector/typesafe code is MIT.