decide v1
A small, single-job decision model for Indian-language customer support, built by togi labs.
Give it a message and a fixed set of options. In one forward pass it returns a probability for every option. No prompt, no sampling, no generated text. It was trained for three questions, intent, urgency and reply language, and answers any other fixed-option question in the same format.
It is built for the way people actually type: lowercase, no punctuation, chat spellings, Hindi and Tamil in Roman letters, and script switching in the middle of a line.
What it answers
| question | type | options |
|---|---|---|
| intent | choice | order_status, payment_issue, refund, delivery, account_login, complaint, product_question, other |
| urgency | score, 1 to 5 | can wait days, low, medium, high (needs today), critical (needs now) |
| reply language | choice | hi, hinglish, ta, te, en |
Example, with the model's own output:
message refund eppo varum 10 days aachu
intent refund 0.998
urgency 3
reply ta
Results
Every number below is from the togi support benchmark report of 25 October 2026. Full tables, charts and caveats are at togilabs.ai/research.
The togi support benchmark
753 customer-support messages in Hindi (Devanagari), Hindi (Roman script), Tamil and Telugu. Eight intents, a one-to-five urgency, and the language the reply should be in. Twelve models answer the same messages with the same options.
| model | intent | intent, 314 leak-free | tamil intent, leak-free | urgency, exact | reply language | hard 24 solved | confident and wrong |
|---|---|---|---|---|---|---|---|
| decide v1 | 0.98 | 0.97 | 0.98 | 0.80 | 0.98 | 20 | 0.01 |
| decide v0 | 0.96 | 0.94 | 0.94 | 0.77 | 0.98 | 10 | 0.03 |
| clef, 8-bit (27B) | 0.88 | 0.84 | 0.81 | 0.34 | 0.57 | 0 | 0.04 |
| pplx-decider-flash | 0.86 | 0.86 | 0.86 | 0.28 | 0.76 | 0 | 0.03 |
| yama | 0.78 | 0.76 | 0.84 | 0.21 | 0.37 | 6 | 0.06 |
The 314 leak-free rows are the messages with no near-duplicate in the training data. The hard 24 are the messages every released model got wrong.
Out of domain: MASSIVE Indic
500 human-localised voice-assistant requests per language from the MASSIVE test split, one 18-way scenario question. yama was trained on this data. pplx-decider-flash and decide v0 never saw it. decide v1 did.
| model | hindi | tamil | telugu | all |
|---|---|---|---|---|
| decide v1 | 0.936 | 0.930 | 0.932 | 0.933 |
| yama (trained on MASSIVE) | 0.898 | 0.852 | 0.882 | 0.877 |
| pplx-decider-flash (unseen) | 0.864 | 0.606 | 0.830 | 0.767 |
| decide v0 (unseen) | 0.822 | 0.638 | 0.780 | 0.747 |
Data no model trained on
IndicCOPA (2,949 cause-and-effect questions), a Hinglish retail intent set with 13 intents no model had seen (2,629 messages), and the benchmark re-scored with the option order shuffled.
| model | copa, all | copa, tamil | copa, hindi | copa calibration error | retail accuracy | retail calibration error | answers unchanged after shuffle |
|---|---|---|---|---|---|---|---|
| decide v1 | 0.848 | 0.842 | 0.849 | 0.211 | 0.940 | 0.023 | 98.8% |
| decide v0 | 0.642 | 0.540 | 0.717 | 0.207 | 0.930 | 0.042 | 99.2% |
| pplx-decider-flash | 0.737 | 0.622 | 0.849 | 0.116 | 0.955 | 0.079 | 91.2% |
| yama | 0.635 | 0.644 | 0.619 | 0.186 | 0.900 | 0.142 | 91.4% |
The test we lose. pplx-decider-flash is 1.5 points ahead on the unseen Hinglish retail schema. On cause and effect, decide v1 is 11 points ahead overall and 22 ahead on Tamil, but its probabilities there are over-confident. We publish both.
How it was trained
| torso | google/gemma-4-E4B-it |
| adapter | LoRA, rank 16, alpha 32, on q, k, v, o, gate, up and down projections. 36.2M trainable parameters |
| head | pointer head, 24 slots, scores each option in one forward pass |
| data | 272,577 training examples, one pass: open Indic sets with gold labels, 13.9k hand-reviewed drafts at weight 4, MASSIVE train and dev for seven Indic locales, SIB-200, IndicSentiment, Hinglish-TOP |
| compute | 8,518 steps, 10.3 hours on one NVIDIA L40S |
| validation | final validation accuracy 0.9725 |
| why gemma | 2.1 tokens per Tamil word, against 3.8 for Qwen3.5 |
Files
| file | what it is |
|---|---|
lora/ |
the PEFT adapter (adapter_model.safetensors, adapter_config.json) |
slot_head.pt |
the pointer head |
strands_decider_config.json |
head, adapter and calibration temperatures |
tokenizer.json, tokenizer_config.json, chat_template.jinja |
the gemma-4 tokenizer and template |
The model runs on the strands-decider engine. A request is a state (the message) and a dict of questions, each with a type (choice or score), an instructions string and criteria (the options with one-line descriptions). The response carries a probability for every option of every question.
Caveats
- The benchmark is in-distribution for decide: the same intents, writing style and urgency scale it was trained on. On the 314 messages with no near-duplicate in training, intent accuracy is 0.97.
- Urgency labels follow our own one-to-five scale. There is no public standard to compare against.
- The MASSIVE and SIB-200 training sets are translations, not native text. The retail test set was written by a language model.
- On cause-and-effect questions the model is over-confident (calibration error 0.21 against 0.12 for pplx-decider-flash). Calibrating on an unseen task is the first item for v2.
- None of this yet measures real support tickets or text written by native speakers in the wild.
About
togi labs builds small, single-job decision models for Indian languages. togi means to sharpen. We publish the test we lose next to the ones we win.
- Downloads last month
- -