MLukauskas/amber-4b

Amber 4B

Amber 4B is a typed decision engine: given a state and typed questions (multiple-choice choice or yes/no noul, each option described by its criteria), it returns a probability for every supplied option. It is Qwen3.5-4B with a LoRA adapter merged into its weights at load and a learned code readout. Each question is one chat prompt with the state, the question and every option under a short code (A..Z, then two-letter codes). The readout at the prompt's last token gives one logit per option, and the answer is their softmax.

Served parameters: 4.21B (Qwen3.5-4B's text model 4.21B and the code readout 0.65M). Unlike Amber 0.4B, a ModernBERT encoder, it is a decoder, and none of Amber 0.4B's weights are in it. The same decoder is half of Amber Hybrid 4B, which averages it with Amber 0.4B.

🇱🇹 Built in Lithuania. Amber is named after Baltic amber, gintaras in Lithuanian, which the sea washes up on the Lithuanian coast.

Decision Index

Public index 52.12 (raw 63.92) on the public 0.3 suite: rank 98 of 229 by public index on the board as of 2026-10-22, and rank 16 of 115 among models up to 5B parameters (served, as the board counts them); the first is StartLux-Decision v1.1 4B at 67.81. Since Decision Index 0.4 the board ranks on the Full score, which the maintainers compute from private tests only (tests of the same skills, 60%, and decision tasks from new domains, 40%) on submitted models; the public run validates a submission and is shown next to it. This model's Full score is not out yet.

Complete run from this repository's files alone, 140,178 scored requests, 0 unsupported, nothing truncated. Latency per request on one NVIDIA GeForce RTX 3080 Ti (bf16, torch 2.14.0+cu126): median 124.9 ms, mean 307.8 ms, p80 258.2 ms, p95 907.4 ms.

Model	Public index	Full score	Median ms
JPT-35B-A3B	52.68	54.09	250.2
Xor 1.2	52.28	53.62	149.9
Amber 4B	52.12	-	124.9
Decider chat · Qwen3.6-27B	51.57	54.67	83.6
frontier-infra Jebadiah 9B v2.1 (t1e)	51.53	50.90	125.1

Area	Skill	Raw
Knowledge & Reasoning	34.7	49.5
Language Understanding	58.3	70.7
Retrieval & Classification	54.6	67.6
Tools & Automation	71.9	73.8
Arts & Human Taste	40.1	58.3

It is not the best on that board on any benchmark.

Every benchmark in the index

Area	Benchmark	Metric	Raw	Skill
Knowledge & Reasoning	GPQA Diamond	accuracy	0.383	0.177
Knowledge & Reasoning	GSM8K	accuracy	0.612	0.535
Knowledge & Reasoning	ChessBench	accuracy	0.093	0.012
Knowledge & Reasoning	MuSR	accuracy	0.580	0.332
Knowledge & Reasoning	SATA-Bench	case exact accuracy	0.353	0.345
Knowledge & Reasoning	CRUXEval	accuracy	0.537	0.265
Knowledge & Reasoning	CLadder	accuracy	0.676	0.352
Knowledge & Reasoning	HLE	accuracy	0.104	0.000
Knowledge & Reasoning	MMLU-Pro	accuracy	0.513	0.453
Knowledge & Reasoning	BBH	accuracy	0.705	0.572
Knowledge & Reasoning	WinoGrande	accuracy	0.866	0.732
Language Understanding	ContractNLI	macro-F1	0.754	0.645
Language Understanding	ANLI	macro-F1	0.698	0.548
Language Understanding	HellaSwag	accuracy	0.944	0.925
Language Understanding	ACOS	per-review F1	0.330	0.309
Language Understanding	FinEntity	macro-F1	0.934	0.903
Language Understanding	iSarcasmEval	Sarcasm F1 · track A, English	0.421	0.255
Language Understanding	VAST	macro-F1	0.729	0.593
Language Understanding	NLI4CT	macro-F1	0.778	0.569
Language Understanding	RAGTruth	F1 on hallucinated class	0.729	0.438
Retrieval & Classification	BANKING77	macro-F1	0.819	0.816
Retrieval & Classification	CLINC150	macro-F1	0.884	0.884
Retrieval & Classification	BRIGHT	nDCG@10	0.451	0.379
Retrieval & Classification	Amazon ESCI	macro-F1	0.519	0.397
Retrieval & Classification	PhishNChips	accuracy	0.681	0.362
Retrieval & Classification	HoVer	accuracy	0.675	0.351
Tools & Automation	BFCL	case exact accuracy	0.972	0.962
Tools & Automation	ToolRet	nDCG@10	0.665	0.613
Tools & Automation	API-Bank	accuracy	0.638	0.631
Tools & Automation	Home appliances	case exact accuracy	0.477	0.477
Tools & Automation	When2Call	accuracy	0.912	0.882
Arts & Human Taste	BPoMP	accuracy	0.900	0.799
Arts & Human Taste	Humicroedit	accuracy	0.617	0.234
Arts & Human Taste	POP909	accuracy	0.018	0.010
Arts & Human Taste	cfcolor	accuracy	0.660	0.320
Arts & Human Taste	Habermas	accuracy	0.563	0.365
Arts & Human Taste	New Yorker	accuracy	0.742	0.678

Policy, the same for every benchmark: One fixed chat prompt per question: the state, the question and every option under a short code; softmax over the option codes at the prompt's last token (noul: P(true)), the choice its argmax. No truncation, no option filtering, no abstention.

Use

With the decision-index kit installed (pip install -e decision-index, commit f5a6e4fd85fed05e77878dfc4966d0568bfb45ec) and this repository as the working directory:

pip install -r requirements.txt
PYTHONPATH=. python -m decision_index run --engine decoder_engine:DecoderEngine \
    --option decoder_weights=decoder.safetensors --suite-dir <suite-0.3> --out runs/amber-4b --compact

The engine loads Qwen/Qwen3.5-4B@851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a from the Hub at that revision and merges the LoRA into its weights.

• token_budget (8192), max_batch (16): padded tokens and prompts per forward. A prompt over the budget is read in chunk_tokens (4096) chunks through the model's cache, so nothing is cut. The scored run used about 10 GB of GPU memory.
• flash-linear-attention gives Qwen3.5's Gated DeltaNet layers their fast kernels.

Files:

• decoder_engine.py: the engine, one file. sha256 f1fb7b913f360ed701160b3a3e27a4efec7f32233be7fb6e998252bcbb23bc5e.
• decoder.safetensors: the LoRA factors (rank 64, 248 modules, fp32) and code readout, with the base model's id and revision as metadata; the same file as in Amber Hybrid 4B. sha256 64add526b966051aaa7f3eaa002b9f01e852d037532bddb1b9f80c9a6c3dee37.

Training

The model was not trained as one run. Two LoRA fine-tunes of Qwen3.5-4B were trained from the same initialization, and the model is the weighted mean of their weight updates: 0.4 of the first run's and 0.6 of the second's, held exactly as one LoRA of rank 64 (the two factor pairs side by side), with the two readouts averaged the same way. Nothing was trained after the average. The 0.4 / 0.6 split was chosen on held-out tasksource task families (below), not on Decision Index rows.

Each run: the backbone frozen in bf16, LoRA r=32 and alpha 64 on every attention, MLP and Gated DeltaNet projection, and a readout of one row per answer code initialised from the code tokens' embeddings. One epoch, learning rate 2e-4 (readout 5e-5) with 200 warmup steps, gradient accumulation 4, batches of similar-length prompts up to 4,096 padded tokens. The loss is the cross-entropy of the softmax over a question's codes against its target. Run 1: 222,005 rows, 64.3M tokens, 5,016 updates. Run 2: 272,694 rows, 73.2M tokens, 5,739 updates.

Data

Data	Run 1	Run 2	Upstream, part used	Licence
tasksource	136,848	136,848	tasksource/tasksource-jev-typed-decisions @6f92aad train split, at most 400 rows from each of 547 sources. The task families of a held-out evaluation split are left out	per-source terms (license: other)
Amber's training mix	53,338	72,654	Subsets of the Amber 0.4B mix (its card has the table), most weight on WinoGrande, HellaSwag, ANLI, NLI4CT, New Yorker, When2Call and BBH. Run 2 shuffles every question's options and, in sources with two possible answers, thins the commoner one until the rarer is 30%	as on the Amber card
GSM8K and MetaMathQA	23,819	29,242	openai/gsm8k @740312a train, as the index's 4- and 10-option questions with distractors from the kit's own option builder; meta-math/MetaMathQA @aa4f34d, its GSM_Rephrased, GSM_SV and GSM_FOBAR problems	MIT
Generated functions	8,000	12,000	CRUXEval-style Python functions generated in this repo (no CRUXEval text)	generated
Multiple-choice knowledge	-	21,950	Train and validation splits of ARC, OpenBookQA, SciQ, QASC and CommonsenseQA, MMLU dev and validation, MedMCQA validation, in the MMLU-Pro and GPQA request shapes	dataset terms

No Decision Index row is in the training data. Every training row, its whole prompt (state, question and options), was checked against every row of the 0.3 suite (selected, added and GSM8K rows), and dropped if it equals a suite text or shares a 13-word span with one (lower-cased, word characters only). On the suite side the texts are the string leaves of the state and the question instructions. For BFCL, ToolRet, API-Bank and BRIGHT only the user's or query text is used, since their tool catalogs and corpora are shared by every row. A text or span found in more than 20 suite rows is a template and is not matched. Rows of WinoDict, SuperGLUE MultiRC and AxG, MC-TACO and PRM800K are dropped whole, since they are built from scored splits. Dropped:

• tasksource: 1,156 rows for overlap, 2,396 from those sources.
• Amber's mix: 5,441 in run 1 and 5,788 in run 2. These are shared corpora, such as the same clinical trial reports in NLI4CT train and test and Wikipedia sentences, and templates that copy an item's first words.
• GSM8K and MetaMathQA: 3,127 and 3,704. About 15% of GSM8K train problems appear in the suite's RouterBench rows, some MetaMathQA problems in BRIGHT, and 10 MetaMathQA problems are GSM8K test problems.
• Multiple-choice knowledge: 357.

Limitations

• Knowledge & Reasoning is its weakest area: multi-step arithmetic (GSM8K), expert questions (GPQA, MMLU-Pro) and code execution (CRUXEval) stay well below the largest models.
• About 10 GB of GPU memory in bf16.
• Mostly English.
• The probabilities are calibrated on the training mixes, not per deployment; re-check thresholds on your own data.