debuffjp/decide-small

Decide Small

Decide reads a short piece of text and picks one of your labels, on the user's device, with a confidence you can act on: answers at 0.9 are right about 90% of the time. It's the default model in decidekit.

Docs · Try it in your browser · GitHub · npm

Use it

npm install decidekit

import { createDecider } from "decidekit";

const d = await createDecider(); // loads this model
const { choice, confidence } = await d.decide("my card still hasn't arrived", ["card_arrival", "lost_card", "refund"]);

The files download from this repo on first use, then stay in the browser's cache. To serve them yourself, copy this repo's files to your own host and pass modelHost to createDecider().

Numbers

BANKING77, 77 labels, all training data	93.1%
CLINC150, 150 labels, all training data	96.3%
BANKING77 with 5 examples per label	83.7%
A decision in Chrome (worker, median)	7.7 ms
Download at 8-bit	34 MB

Ryzen 7 260 laptop, Chrome 156. Results and scripts: eval/results-h14.json.

Limits

• English. Use Decide Multilingual for other languages.
• It reads what a text is about better than how it feels: 61% on tweet sentiment.
• About a paragraph at a time. Split longer text.

What's in this repo

This repo holds the encoder, the part that turns text into vectors. decidekit adds the trained head and the calibrated confidence.

The weights are an unmodified copy of Xenova/bge-small-en-v1.5 at revision ea104da, the ONNX conversion of BAAI/bge-small-en-v1.5. We did not train them. They are here so decidekit has a stable, pinned source. MIT licensed, like the original.