liev-350m

Liev = Liquid + Jev: a LFM2.5-350M-based Jev-style decision model.

A Jev/Clef-style decision model: LiquidAI/LFM2.5-350M-Base plus a joint schema head (Unsloth FastDecisionModel, Clef head), LoRA-tuned and merged. It reads a state and a set of typed questions (choice, noul yes/no, score) and returns calibrated probabilities for every option in one forward pass, without generating text.

Results (held-out test sets)

Test	Accuracy	ECE
banking77	77.4%	0.0646
clinc150	66.6%	0.1245
synthetic	53.4%	0.1756
typed_decisions	73.9%	0.099
holdout (10% of training mix)	78.8%	0.0386

Pooled typed-decisions + BANKING77 + CLINC150 accuracy: 73.3%.

Training

• Data: ~40.8k rows: 36k from Unsloth's public decision mixture (banking77, clinc150, mnli, snli, wanli, boolq, ag_news, sst5, mmlu aux-train, commonsense_qa, arc, prompt_injections; xlam skipped, gated), LocalLLaMA/typed-decisions train (1,030 rows after removing rows whose 13-grams mostly appear in the test split; 4 augmented copies), and a small hand-written synthetic set (131 rows, 5 copies). Public rows decontaminated against all test sets.
• LoRA r=64, lr 0.0001, head lr 0.0003, effective batch 16, 1 epoch (2535 steps, 224.5 min on a Kaggle T4), max length 4096, float32 training. Temperatures fitted on the holdout.
• Caveat: the typed-decisions decontamination is looser than Unsloth's reference script (which drops any 13-gram overlap).

Use

from unsloth import FastDecisionModel
model, tok = FastDecisionModel.from_pretrained("Manitchahar/LFM2.5-350M-decision", max_seq_length = 4096)
FastDecisionModel.for_inference(model)
FastDecisionModel.predict(model, tok, "I was charged twice, refund me", {
    "team": {"type": "choice", "instructions": "Which team?", "criteria": {"billing": "payments", "tech": "bugs"}},
    "refund": {"type": "noul", "instructions": "Asks for a refund?"}})

Benchmarks (re-measured from the public weights)

JevBench public split (231 decisions; the official board also uses held-out items, so these are not leaderboard numbers):

Tier	Accuracy	Brier	ECE
original (72)	66.7%	0.478	0.141
easy (48)	89.6%	0.112	0.107
hard (111)	35.1%	0.716	0.171
all public	56.3%

CPU latency (Kaggle CPU, 4 threads, fp32, batch 1, JevBench original): p50 1213 ms, p95 1535 ms.

Full comparison against other open sub-1B decision models: https://huggingface.co/datasets/Manitchahar/decision-bakeoff