Leptos-ai/Aurelis-2B

Aurelis 2B

Aurelis is a small decision model: it reads a state and a set of typed questions (multiple choice choice, or yes/no noul) and returns a probability for every option, one forward pass per question. It speaks the same /v1/systemone interface as the other models on the Decision Index.

It is MiniCPM5-2B (2.5B parameters) with a merged LoRA adapter (rank 16).

Results

Decision Index 0.3, public suite (all 37 benchmarks, 140,620 requests, scored by the kit), one run of the published Ollama build; run files in Leptos-ai/aurelis-decision-index. The other models' numbers are from the board's data of 2026-10-16. Scores are the kit's chance-corrected skill: 0 is guessing, 100 is every answer right.

	Aurelis 2B	LiquidAI d1-3B	vLLM-SR d3-nano	CoCo-Decision-4B	Perplexity Decider Flash v1.2	Cloudflare clef-flash	Cloudflare clef
parameters	2.5B	3.1B	2.2B	4.2B	4.5B	9.7B	27.8B
public index	51.29	48.99	42.22	55.38	60.06	56.15	61.71
Knowledge & Reasoning	32.2	24.3	25.3	34.8	41.4	53.0	52.8
Language Understanding	54.1	56.5	42.2	63.2	65.1	47.2	61.3
Retrieval & Classification	54.2	52.8	58.8	59.9	65.2	52.2	62.8
Tools & Automation	79.5	74.5	58.9	76.8	81.0	81.9	80.7
Arts & Human Taste	35.9	38.9	22.4	40.2	46.6	48.4	48.9

On the board of 2026-10-16 (174 models) 51.29 is place 65, and the highest public index of any model under 4B parameters. The three models of 4.5B or less above it are CoCo-Decision-4B, vLLM-SR d3-mini (54.90) and Perplexity Decider Flash v1.2. Against Cloudflare's clef-flash, almost four times its size, Aurelis is 4.9 points behind and ahead on 11 of the 37 benchmarks, among them When2Call (87.0 against 53.8), CLINC150, VAST, PhishNChips and BBH; it is furthest behind where knowledge or multi-step reasoning decides (MuSR, WinoGrande, CRUXEval, CLadder).

All 37 benchmarks (bold: best of the seven)

benchmark	area	Aurelis 2B	LiquidAI d1-3B	vLLM-SR d3-nano	CoCo-Decision-4B	Perplexity Decider Flash v1.2	Cloudflare clef-flash	Cloudflare clef
GPQA Diamond	Knowledge	21.8	4.8	9.5	28.6	25.9	35.4	31.3
WinoGrande	Knowledge	51.7	55.5	59.7	67.2	77.3	95.1	87.1
GSM8K	Knowledge	27.9	19.0	42.2	25.9	61.2	36.3	52.1
ChessBench	Knowledge	9.0	8.3	0.6	10.2	14.1	16.2	17.8
MuSR	Knowledge	30.6	31.3	35.7	37.6	36.8	78.0	74.2
SATA-Bench	Knowledge	38.6	30.4	28.8	33.1	10.5	35.8	32.6
CRUXEval	Knowledge	9.0	0.0	7.6	11.5	40.7	78.6	78.6
CLadder	Knowledge	71.0	13.2	20.7	67.6	70.8	95.3	88.2
HLE	Knowledge	0.0	0.0	0.0	0.0	0.0	0.0	0.0
MMLU-Pro	Knowledge	31.8	38.0	36.5	49.2	54.0	60.9	61.7
BBH	Knowledge	62.1	60.0	33.9	47.7	61.5	54.9	61.8
ContractNLI	Language	74.8	71.9	55.0	77.6	75.3	77.6	73.0
ANLI	Language	46.2	43.6	34.0	54.1	54.1	38.8	55.0
HellaSwag	Language	79.6	85.6	87.9	92.1	91.8	98.1	97.6
ACOS	Language	36.1	35.7	10.2	34.1	33.4	23.6	31.0
FinEntity	Language	72.9	82.4	81.7	80.2	86.0	95.7	94.2
iSarcasmEval	Language	19.4	17.6	22.9	39.2	47.8	0.0	31.0
VAST	Language	59.4	65.5	35.5	66.7	65.3	24.4	39.0
NLI4CT	Language	45.9	49.6	44.7	61.5	65.6	58.3	66.9
RAGTruth	Language	49.4	53.5	0.0	59.4	63.7	0.0	57.6
BANKING77	Retrieval	86.3	87.8	91.2	89.6	90.0	90.7	94.0
CLINC150	Retrieval	89.6	91.1	95.1	92.1	93.3	66.7	97.4
BRIGHT	Retrieval	19.1	32.8	31.0	32.9	38.5	30.1	40.6
Amazon ESCI	Retrieval	41.3	38.8	33.7	46.0	47.0	46.6	46.5
PhishNChips	Retrieval	64.2	7.7	34.0	18.0	45.1	50.1	58.7
HoVer	Retrieval	18.1	48.2	59.8	74.0	72.4	22.4	31.2
BFCL	Tools	92.3	85.7	89.4	94.5	94.3	98.3	97.9
ToolRet	Tools	56.6	60.4	55.2	44.0	59.7	61.0	64.3
API-Bank	Tools	85.0	77.7	55.5	69.9	86.2	93.0	91.8
Home appliances	Tools	72.7	63.6	12.5	87.5	75.0	97.7	80.7
When2Call	Tools	87.0	82.1	76.4	86.0	86.2	53.8	63.3
BPoMP	Arts	88.5	64.3	14.3	87.0	83.2	90.8	94.1
Humicroedit	Arts	25.8	27.3	21.1	31.3	23.1	50.1	33.1
POP909	Arts	8.4	23.3	9.5	7.2	25.5	0.7	14.8
cfcolor	Arts	14.1	23.4	18.3	28.7	36.0	31.9	34.0
Habermas	Arts	43.1	40.2	8.1	22.4	44.3	59.0	54.5
New Yorker	Arts	35.8	55.0	63.3	64.7	67.6	57.9	62.8

Parts of the Artificial Analysis Decision Index, on the exact items its methodology lists: 3,423 When2Call tasks (options shuffled per task, as AA does) and 513 HLE multiple-choice questions. Aurelis's numbers are measured here, the others are AA's of 2026-10-17. Aurelis is not on that board, its AA-Omniscience part is private and WildGuardTest was not run, so there is no AA index for it. Aurelis was trained on When2Call's train split, never its test.

decision model	maker	When2Call	HLE-MultiChoice	AA index
Aurelis 2B (measured here)	Leptos	92.6	12.1	—
d1-ultra	Liquid AI	92.6	15.0	65.2
Celeris-1 Decision	Celeris	83.7	13.3	52.3
GLiDE	Fastino	83.7	10.5	53.9
Microsoft-Decision-1	Microsoft	81.9	10.9	53.8
Jev 1.14	TypeSafe	80.8	18.9	57.1
Jev 1.13	TypeSafe	80.7	20.9	57.5
Mercury Decide	Inception	79.3	10.1	52.7
d1	Liquid AI	77.4	11.5	50.3
Decider v1.1 27B	Perplexity	76.5	15.4	53.1
GPT-6 Luna (Decisions)	OpenAI	74.1	11.7	48.5
Clef	Cloudflare	71.0	11.1	48.6
Clef-Flash	Cloudflare	64.4	10.3	45.8
Clef-Omni	Cloudflare	62.1	9.0	44.3

Use

Ollama (0.40.5 or newer):

ollama pull mrscratchcat22/aurelis:2b
curl -s localhost:11434/v1/systemone -d '{
  "model": "mrscratchcat22/aurelis:2b",
  "state": "Customer: my card was charged twice for one order and I need the money back before Friday.",
  "questions": {
    "route":  {"type": "choice", "instructions": "Which team handles this?",
               "criteria": {"billing": "Billing", "technical": "Technical support", "account": "Account"}},
    "urgent": {"type": "noul", "instructions": "Is this urgent?"}
  }
}'

The GGUF here (aurelis-2b-q8_0.gguf, with Modelfile) is that same build: decision type pplx-decider and the trained prompt as tokenizer.chat_template.systemone, for llama.cpp's /v1/systemone (b11669 or newer) and Ollama. Other sizes, from Q4_K_M (1.56 GB) to F16, each measured against this one, are in Leptos-ai/Aurelis-2B-GGUF (ollama pull hf.co/Leptos-ai/Aurelis-2B-GGUF:Q4_K_M).

Transformers, with aurelis.py from this repo (same prompt and probabilities as Ollama; on 23 test questions the same answers, largest probability difference 0.009, which is the 8-bit rounding of the GGUF):

from aurelis import Aurelis
decide = Aurelis("Leptos-ai/Aurelis-2B")
decide("Customer: my card was charged twice for one order and I need the money back before Friday.",
       {"route": {"type": "choice", "instructions": "Which team handles this?",
                  "criteria": {"billing": "Billing", "technical": "Technical support", "account": "Account"}},
        "urgent": {"type": "noul", "instructions": "Is this urgent?"}})
# {'route': {'type': 'choice', 'choice': 'billing', 'probabilities': {'billing': 0.84, 'technical': 0.09, 'account': 0.07}},
#  'urgent': {'type': 'noul', 'noul': 0.91}}

Prompt format, if you write your own runner: one user message per question holding the JSON {"state": ..., "question": ..., "options": [{"label": "A", "key": ..., "description": ...}, ...]} (Python json.dumps spacing, ensure_ascii=False; an object state is compact JSON text; a yes/no question is false/No then true/Yes), then the assistant turn <think>\n\n</think>\n\n, and the answer is the softmax over the option letters. No <s> in front.

Limits

• At most 24 options per question (more need two rounds, as the Decision Index engine does).
• Trained on prompts of up to 4,096 tokens; Ollama serves it with a 16,384-token window, where long states still work but were seen less in training.
• English. Small model: knowledge-heavy questions (GPQA, HLE, MMLU-Pro) stay near what its size allows.

Training

61,901 examples, 3,869 steps, LoRA rank 16 (alpha 32), learning rate 5e-5, on one AMD RX 9070 XT. The output layer carries a calibration temperature of 1.55, fitted on the held-out split and built into the weights. Data:

• training splits of public datasets (NLI, claim verification, tool calling and tool selection, classification, multiple choice, ...) laid out as decision requests, with an overlap check that drops every row whose text also occurs in the Decision Index suite; no test set of any benchmark was trained on;
• tasksource/tasksource-jev-typed-decisions (5,924 rows);
• a few dozen rows from small game simulations and hand-written decisions.

License

Apache-2.0, as the base model.