liev-700m

Liev = Liquid + Jev: a LFM2-700M-based Jev-style decision model.

A Jev/Clef-style decision model: LiquidAI/LFM2-700M plus a joint schema head (Unsloth FastDecisionModel, Clef head), LoRA-tuned and merged. It reads a state and a set of typed questions (choice, noul yes/no, score) and returns calibrated probabilities for every option in one forward pass, without generating text.

Results (held-out test sets)

Test	Accuracy	ECE
banking77	76.4%	0.0762
clinc150	70.4%	0.104
synthetic	67.2%	0.1345
typed_decisions	74.9%	0.1242
holdout (10% of training mix)	82.2%	0.0597

Pooled typed-decisions + BANKING77 + CLINC150 accuracy: 74.4%.

Training

• Data: ~40.8k rows: 36k from Unsloth's public decision mixture (banking77, clinc150, mnli, snli, wanli, boolq, ag_news, sst5, mmlu aux-train, commonsense_qa, arc, prompt_injections; xlam skipped, gated), LocalLLaMA/typed-decisions train (1,030 rows after removing rows whose 13-grams mostly appear in the test split; 4 augmented copies), and a small hand-written synthetic set (131 rows, 5 copies). Public rows decontaminated against all test sets.
• LoRA r=64, lr 0.0001, head lr 0.0003, effective batch 16, 1 epoch (2535 steps, 112.9 min on a Kaggle T4), max length 4096, float16 training. Temperatures fitted on the holdout.
• Caveat: the typed-decisions decontamination is looser than Unsloth's reference script (which drops any 13-gram overlap).

Use

from unsloth import FastDecisionModel
model, tok = FastDecisionModel.from_pretrained("Manitchahar/LFM2-700M-decision", max_seq_length = 4096)
FastDecisionModel.for_inference(model)
FastDecisionModel.predict(model, tok, "I was charged twice, refund me", {
    "team": {"type": "choice", "instructions": "Which team?", "criteria": {"billing": "payments", "tech": "bugs"}},
    "refund": {"type": "noul", "instructions": "Asks for a refund?"}})

Benchmarks (re-measured from the public weights)

JevBench public split (231 decisions; the official board also uses held-out items, so these are not leaderboard numbers):

Tier	Accuracy	Brier	ECE
original (72)	77.8%	0.312	0.066
easy (48)	93.8%	0.102	0.052
hard (111)	36.9%	0.729	0.168
all public	61.5%

CPU latency (Kaggle CPU, 4 threads, fp32, batch 1, JevBench original): p50 2445 ms, p95 3156 ms.

Full comparison against other open sub-1B decision models: https://huggingface.co/datasets/Manitchahar/decision-bakeoff