Naluri-308M-encoder

Instinctive, multilingual decisions in one pass.

Naluri (Indonesian for instinct; Neural Answering via Label Understanding and Rapid Inference) is a small System One decision model. This is the 308M (middle) member of the family, built on mmBERT-base. You give it a text or JSON state and a typed question with its options written out. It returns calibrated probabilities over those options in a single forward pass, in about 9 ms per question on a laptop GPU. It never generates text.

Model	Backbone	Parameters	System One Bench	Latency (GPU, p50)
Naluri-279M-encoder	XLM-RoBERTa base	278.6M	50.4 (main, M6-pairs); 58.0 (m9 branch)	4.9 ms
Naluri-308M-encoder (this model)	mmBERT-base	307.5M	61.0	9.2 ms
Naluri-561M-encoder	XLM-RoBERTa large	560.9M	64.9	13.8 ms

It supports the same three question types as Jev and Laya:

Type	Options	Returns
noul	yes / no (optional descriptions)	p(true)
choice	2+ named options (optional descriptions)	one probability per option
score	ordered levels	one probability per level; expected level = score

Code, training pipeline and full report: github.com/jerichosiahaya/naluri.

Load with the bundled naluri.py (see Usage). The standard transformers pipeline and the hosted inference widget can't run Naluri: its decision head is custom code.

Usage

pip install torch "transformers>=4.48" safetensors huggingface_hub

from huggingface_hub import hf_hub_download
import importlib.util

# naluri.py (input format + model + loader) ships with this repo
spec = importlib.util.spec_from_file_location("naluri", hf_hub_download("jerichosiahaya/Naluri-308M-encoder", "naluri.py"))
naluri = importlib.util.module_from_spec(spec); spec.loader.exec_module(naluri)

tok, model = naluri.from_pretrained("jerichosiahaya/Naluri-308M-encoder", device="cpu")   # or device="cuda"

state = {"premise": "Pesanan saya sudah seminggu belum sampai dan CS tidak membalas.",
         "hypothesis": "The customer is satisfied with the service."}
questions = {"nli": {"type": "choice", "instructions": "What is the relation of `hypothesis` to `premise`?",
                     "criteria": {"entailment": "", "neutral": "", "contradiction": ""}}}
print(naluri.predict(tok, model, state, questions, device="cpu"))
# {'nli': {'entailment': 0.01, 'neutral': 0.13, 'contradiction': 0.86}}

for msg in ["Pesanan saya sudah seminggu belum sampai dan CS tidak membalas.", "Terima kasih, paketnya sudah sampai dengan baik!"]:
    print(naluri.predict(tok, model, {"message": msg},
                         {"complaint": {"type": "noul", "instructions": "Is `message` a complaint?"}}, device="cpu"))
# {'complaint': {'false': 0.23, 'true': 0.77}}
# {'complaint': {'false': 0.90, 'true': 0.10}}

• Named fields work. Put the inputs in named fields and point at them with backticks in the question.
• Option descriptions matter. Naluri reads them, so clear, distinct descriptions give better answers.
• Limits: the question plus options must fit in about 192 tokens, and the full input is capped at 512 tokens (the same as the 279M model, so results are comparable). mmBERT itself accepts 8,192 tokens, but Naluri was not trained on longer inputs.
• Uncertainty is meaningful: probabilities near 0.5 (or several options close together) are the cases to send to a human or an LLM.

Results: System One Bench

System One Bench: 16 suites, 3,250 cases. Index = mean suite accuracy. Both models were run on identical cases on the same laptop GPU (RTX 5050). The 279M column is M9, the best XLM-R checkpoint; it is the 279M repo's m9 branch (its main is M6-pairs, index 50.4).

	Naluri-279M (M9)	Naluri-308M
Index	58.0	61.0
Calibration error (ECE)	0.027	0.057
Accuracy on its 50% most confident answers	67.8%	70.6%
Latency per question (p50)	4.9 ms	9.2 ms

Suite	279M (M9)	308M
XNLI en / fr / ar	62.7 / 62.7 / 54.7	76.0 / 68.0 / 56.7
Emotion	70.0	84.5
MASSIVE intent (ar)	52.5	65.5
MASSIVE scenario (ar)	59.5	63.5
typed-decisions	46.9	54.6
MIRACL pairwise (ar)	69.0	76.0
Arabic offensive	75.5	79.5
AG News	80.0	79.5
Arabic sentiment	60.0	59.0
MIRACL rerank (ar)	34.0	34.0
Banking77	29.0	27.0
BoolQ	60.0	55.0
SST-2	74.5	68.5
SST-5	36.5	28.0

• Stronger at understanding: NLI, emotion, intent and typed business decisions improve the most.
• Weaker at sentiment and BoolQ than the 279M model, which had three earlier rounds of training that this model did not get. For sentiment, use the 279M model.
• Many options are still weak (Banking77, MIRACL rerank). The cause is the 192-token budget for options, not the backbone. Narrow the options first (for example, with an embedding model), then let Naluri decide among the top 12 or fewer.

Training

• Input format (the same layout as Laya): <s> {type} question: {instructions} </s> <mask> option 1 <mask> option 2 … </s> {state} </s>. Each option is scored from the encoder vector at its own <mask> by a small MLP head, and a softmax over a question's options gives the answer distribution.
• Objective: cross-entropy against one-hot labels or a teacher's soft probabilities, followed by one calibration temperature per question type (choice 1.83, score 1.68, noul 2.95).
• Recipe: one run from the mmBERT-base checkpoint on the M9 data (125k typed questions): up to 5 epochs with early stopping (patience 1), encoder learning rate 5e-5, head 5e-4, task-balanced sampling (p ∝ √size), synthetic decisions weighted 2× and minimal pairs 3×. Epoch 4 was kept, picked on the mean of per-task dev accuracies (SIB-200 and XCOPA validation splits plus synthetic dev sets). About 4 hours on an 8 GB RTX 5050 laptop GPU.
• Field layout: 50–70% of the public-task rows are shown as named fields (premise, hypothesis, passage, question…), so the model reads /v1/systemone-style states.
• Augmentation: options shuffled, labels renamed, instructions paraphrased, varying option counts, and noul, choice and score mixed.
• Weights: stored in fp16 (the encoder) and loaded in fp32. The largest probability difference against the fp32 training checkpoint was 0.0013 (65 benchmark cases, fp32 inference).

Training data

About 125k typed questions. The licenses below are as listed on each dataset's card; check them yourself before any use.

Dataset	Use	License (per dataset card)
XNLI	entailment (noul / choice)	CC BY-NC 4.0
MASSIVE (mteb/amazon_massive_intent)	intent (choice / noul)	CC BY 4.0 (original) / Apache 2.0 (mirror)
Amazon Reviews Multi (mirror)	review stars (score) + polarity	Amazon's own terms (non-commercial)
TextDetox multilingual toxicity	toxicity	OpenRAIL++
papluca/language-identification	language ID	derived from Amazon Reviews, XNLI and STSb
dair-ai/emotion	emotion	other
RACE	reading comprehension, plus yes/no option checks	non-commercial research
SQuAD 2.0	is the question answerable from the passage? (noul)	CC BY-SA 4.0
MultiRC (SuperGLUE)	is this answer correct? (noul)	see the dataset card
UCI SMS Spam	spam	unknown on the Hub
MasakhaNEWS, Yahoo Answers Topics, DBpedia-14	topic classification	see each dataset card
Synthetic decisions, ops decisions and minimal pairs (20k)	business decisions in 17 languages	written by DeepSeek V4 Flash, labeled with probabilities by DeepSeek V4 Pro

The SIB-200 and XCOPA validation splits were used only to pick the epoch. No test split of any eval set was trained on.

License

These weights are released under CC BY-NC 4.0 (non-commercial), because several training datasets restrict commercial use. The base model (mmBERT) is MIT. The code on GitHub is Apache 2.0. For commercial use, retrain on data you are licensed to use; the pipeline is in the GitHub repo.

Limitations

• Accuracy is below LLM-based decision models. The System One Bench leaders (decoders with 4B–27B parameters) score 76–83. Use Naluri as a fast first pass and send low-confidence cases to a larger model.
• Sentiment (SST-2, SST-5) and BoolQ are weaker than in the 279M model.
• Many options (more than about 12) are weak.
• Calibration is good but not as tight as the 279M model (ECE 0.057 vs 0.027).
• Unrelated fields can sway yes/no answers. Example: "is the writer of premise upset?" about an angry message scores 0.63 alone, but 0.28 when the state also has a hypothesis field that says the customer is satisfied. Put only the fields a question needs into the state.
• Seeds: this is a single run.

Acknowledgements

Built on mmBERT by JHU CLSP. The input format and fine-tuning recipe follow Laya.