MLukauskas/amber-hybrid-4b

Amber Hybrid 4B

Amber Hybrid 4B is a typed decision engine: given a state and typed questions (multiple-choice choice or yes/no noul, each option described by its criteria), it returns a probability for every supplied option. Two models answer every question, and the answer is the plain mean of their two distributions:

• Amber 0.4B, unchanged: ModernBERT-large (395M, frozen) with a LoRA adapter (r=8) and a learned option scorer. The context and each option are encoded separately.
• A decoder: Qwen3.5-4B with a LoRA adapter merged into its weights at load and a learned code readout. Each question is one chat prompt with the state, the question and every option under a short code (A..Z, then two-letter codes). The readout at the prompt's last token gives one logit per option.

Served parameters: 4.61B (Qwen3.5-4B's text model 4.21B, ModernBERT-large 0.40B, Amber's adapter and scorer 5.2M, the code readout 0.65M).

🇱🇹 Built in Lithuania. Amber is named after Baltic amber, gintaras in Lithuanian, which the sea washes up on the Lithuanian coast.

Decision Index

Public index 59.87 (raw 70.05) on the public 0.3 suite: rank 49 of 229 by public index on the board as of 2026-10-22, and rank 7 of 115 among models up to 5B parameters (served, as the board counts them); the first is StartLux-Decision v1.1 4B at 67.81. Since Decision Index 0.4 the board ranks on the Full score, which the maintainers compute from private tests only (tests of the same skills, 60%, and decision tasks from new domains, 40%) on submitted models; the public run validates a submission and is shown next to it. This model's Full score is not out yet. Amber 0.4B alone scores 49.74 public and 26.00 Full on that board (same skills 35.73, new domains 7.86).

Complete run from this repository's files alone, 140,178 scored requests, 0 unsupported, nothing truncated. Latency per request on one NVIDIA GeForce RTX 3080 Ti (bf16, torch 2.14.0+cu126): median 237.8 ms, mean 419.0 ms, p80 336.5 ms, p95 1065.5 ms. Part of the run shared the GPU with other work. While the machine was otherwise idle, in the run's first six hours, each request took 1.00 to 1.16 times as long as in an earlier complete run of the same engine and base models (median 131.6 ms, mean 337.5 ms, p80 294.9 ms).

Model	Public index	Full score	Median ms
frontier-infra Jebadiah 27B v2.1 (t1e)	60.46	64.40	303.2
Perplexity Decider Flash v1.2 (4B)	60.06	54.29	74.8
Amber Hybrid 4B	59.87	-	237.8
AnyJev L0 · Gemma-4-31B	59.79	63.65	97.9
JEV-35B	59.73	57.61	50.5

Area	Skill	Raw
Knowledge & Reasoning	36.7	50.7
Language Understanding	60.1	72.2
Retrieval & Classification	68.9	75.9
Tools & Automation	84.9	86.7
Arts & Human Taste	55.3	72.5

It is not the best on that board on any benchmark.

Every benchmark in the index

Area	Benchmark	Metric	Raw	Skill
Knowledge & Reasoning	GPQA Diamond	accuracy	0.362	0.150
Knowledge & Reasoning	GSM8K	accuracy	0.597	0.518
Knowledge & Reasoning	ChessBench	accuracy	0.092	0.011
Knowledge & Reasoning	MuSR	accuracy	0.579	0.330
Knowledge & Reasoning	SATA-Bench	case exact accuracy	0.424	0.416
Knowledge & Reasoning	CRUXEval	accuracy	0.539	0.268
Knowledge & Reasoning	CLadder	accuracy	0.828	0.657
Knowledge & Reasoning	HLE	accuracy	0.114	0.000
Knowledge & Reasoning	MMLU-Pro	accuracy	0.495	0.433
Knowledge & Reasoning	BBH	accuracy	0.717	0.589
Knowledge & Reasoning	WinoGrande	accuracy	0.832	0.664
Language Understanding	ContractNLI	macro-F1	0.782	0.685
Language Understanding	ANLI	macro-F1	0.614	0.422
Language Understanding	HellaSwag	accuracy	0.939	0.919
Language Understanding	ACOS	per-review F1	0.452	0.434
Language Understanding	FinEntity	macro-F1	0.946	0.920
Language Understanding	iSarcasmEval	Sarcasm F1 · track A, English	0.424	0.259
Language Understanding	VAST	macro-F1	0.784	0.676
Language Understanding	NLI4CT	macro-F1	0.757	0.527
Language Understanding	RAGTruth	F1 on hallucinated class	0.778	0.541
Retrieval & Classification	BANKING77	macro-F1	0.902	0.901
Retrieval & Classification	CLINC150	macro-F1	0.950	0.949
Retrieval & Classification	BRIGHT	nDCG@10	0.406	0.329
Retrieval & Classification	Amazon ESCI	macro-F1	0.551	0.437
Retrieval & Classification	PhishNChips	accuracy	0.882	0.765
Retrieval & Classification	HoVer	accuracy	0.865	0.730
Tools & Automation	BFCL	case exact accuracy	0.973	0.964
Tools & Automation	ToolRet	nDCG@10	0.711	0.666
Tools & Automation	API-Bank	accuracy	0.805	0.801
Tools & Automation	Home appliances	case exact accuracy	0.943	0.943
Tools & Automation	When2Call	accuracy	0.892	0.856
Arts & Human Taste	BPoMP	accuracy	0.939	0.877
Arts & Human Taste	Humicroedit	accuracy	0.638	0.276
Arts & Human Taste	POP909	accuracy	0.816	0.815
Arts & Human Taste	cfcolor	accuracy	0.648	0.297
Arts & Human Taste	Habermas	accuracy	0.600	0.419
Arts & Human Taste	New Yorker	accuracy	0.708	0.635

Policy, the same for every benchmark: Both models answer every question; the answer is the fixed-weight mean of their distributions (choice: per option, noul: P(true)) and the choice its argmax. A request one model does not support is answered by the other alone. Decoder: one fixed chat prompt per question, softmax over the option codes at its last token. No truncation, no option filtering, no abstention.

Amber's part: One fixed rendering for every benchmark: string state as is, object state as key: value lines, option = its description or key. Softmax over the option scorer's logits; noul = P(true) over its true / false criteria. No abstention, no truncation, no option filtering. A context over the limit is read in consecutive windows of its longest field (state or question), each with the other field, and scored on the token-weighted mean of the window vectors; an over-length option is unsupported.

Use

With the decision-index kit installed (pip install -e decision-index, commit f5a6e4fd85fed05e77878dfc4966d0568bfb45ec) and this repository as the working directory:

pip install -r requirements.txt
PYTHONPATH=. python -m decision_index run --engine hybrid_engine:HybridEngine \
    --option amber_weights=amber.safetensors --option decoder_weights=decoder.safetensors \
    --option amber_weight=0.5 --suite-dir <suite-0.3> --out runs/amber-hybrid-4b --compact

The engine loads answerdotai/ModernBERT-large@45bb4654a4d5aaff24dd11d4781fa46d39bf8c13 and Qwen/Qwen3.5-4B@851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a from the Hub at those revisions, applies Amber's adapter and scorer, and merges the decoder's LoRA into Qwen3.5-4B's weights.

• amber_weight (default 0.5) is Amber's weight in the mean; the decoder's is 1 minus it.
• token_budget (8192), max_batch (16): padded tokens and prompts per decoder forward. A prompt over the budget is read in chunk_tokens (4096) chunks through the model's cache, so nothing is cut. The scored run used about 10 GB of GPU memory.
• A request that one model does not support is answered by the other alone; in the scored run there were none.
• flash-linear-attention gives Qwen3.5's Gated DeltaNet layers their fast kernels.

Files:

• hybrid_engine.py: the engine and the decoder, one file. sha256 3394c0308b97c33712360d3941254e7ef4c27fc3f0bcc144518e17d5c3630e29.
• amber_engine.py: Amber's engine, as in the Amber 0.4B repository. sha256 ffaade8f6ce39520785588c6817a6e60fb2ad4ff2b54e650f05ce64168a3a2c5.
• amber.safetensors: Amber's LoRA adapter and option scorer, as in the Amber 0.4B repository. sha256 c02f5a4ef32da2931413e59b6a92aa01d1d4602912dc83027b4397137fe24346.
• decoder.safetensors: the decoder's LoRA factors (rank 64, 248 modules, fp32) and code readout, with the base model's id and revision as metadata. sha256 64add526b966051aaa7f3eaa002b9f01e852d037532bddb1b9f80c9a6c3dee37.

Training

Amber's weights are those of Amber 0.4B; its card describes their training and data.

The decoder was not trained as one run. Two LoRA fine-tunes of Qwen3.5-4B were trained from the same initialization, and the decoder is the weighted mean of their weight updates: 0.4 of the first run's and 0.6 of the second's, held exactly as one LoRA of rank 64 (the two factor pairs side by side), with the two readouts averaged the same way. Nothing was trained after the average. The 0.4 / 0.6 split was chosen on held-out tasksource task families (below), not on Decision Index rows.

Each run: the backbone frozen in bf16, LoRA r=32 and alpha 64 on every attention, MLP and Gated DeltaNet projection, and a readout of one row per answer code initialised from the code tokens' embeddings. One epoch, learning rate 2e-4 (readout 5e-5) with 200 warmup steps, gradient accumulation 4, batches of similar-length prompts up to 4,096 padded tokens. The loss is the cross-entropy of the softmax over a question's codes against its target. Run 1: 222,005 rows, 64.3M tokens, 5,016 updates. Run 2: 272,694 rows, 73.2M tokens, 5,739 updates.

Data

Data	Run 1	Run 2	Upstream, part used	Licence
tasksource	136,848	136,848	tasksource/tasksource-jev-typed-decisions @6f92aad train split, at most 400 rows from each of 547 sources. The task families of a held-out evaluation split are left out	per-source terms (license: other)
Amber's training mix	53,338	72,654	Subsets of the Amber 0.4B mix (its card has the table), most weight on WinoGrande, HellaSwag, ANLI, NLI4CT, New Yorker, When2Call and BBH. Run 2 shuffles every question's options and, in sources with two possible answers, thins the commoner one until the rarer is 30%	as on the Amber card
GSM8K and MetaMathQA	23,819	29,242	openai/gsm8k @740312a train, as the index's 4- and 10-option questions with distractors from the kit's own option builder; meta-math/MetaMathQA @aa4f34d, its GSM_Rephrased, GSM_SV and GSM_FOBAR problems	MIT
Generated functions	8,000	12,000	CRUXEval-style Python functions generated in this repo (no CRUXEval text)	generated
Multiple-choice knowledge	-	21,950	Train and validation splits of ARC, OpenBookQA, SciQ, QASC and CommonsenseQA, MMLU dev and validation, MedMCQA validation, in the MMLU-Pro and GPQA request shapes	dataset terms

No Decision Index row is in the decoder's data. Every training row, its whole prompt (state, question and options), was checked against every row of the 0.3 suite (selected, added and GSM8K rows), and dropped if it equals a suite text or shares a 13-word span with one (lower-cased, word characters only). On the suite side the texts are the string leaves of the state and the question instructions. For BFCL, ToolRet, API-Bank and BRIGHT only the user's or query text is used, since their tool catalogs and corpora are shared by every row. A text or span found in more than 20 suite rows is a template and is not matched. Rows of WinoDict, SuperGLUE MultiRC and AxG, MC-TACO and PRM800K are dropped whole, since they are built from scored splits. Dropped:

• tasksource: 1,156 rows for overlap, 2,396 from those sources.
• Amber's mix: 5,441 in run 1 and 5,788 in run 2. These are shared corpora, such as the same clinical trial reports in NLI4CT train and test and Wikipedia sentences, and templates that copy an item's first words.
• GSM8K and MetaMathQA: 3,127 and 3,704. About 15% of GSM8K train problems appear in the suite's RouterBench rows, some MetaMathQA problems in BRIGHT, and 10 MetaMathQA problems are GSM8K test problems.
• Multiple-choice knowledge: 357.

Limitations

• Knowledge & Reasoning is still its weakest area: multi-step arithmetic (GSM8K), expert questions (GPQA, MMLU-Pro) and code execution (CRUXEval) stay well below the largest models.
• Two models run per request, about 10 GB of GPU memory in bf16.
• Mostly English.
• The probabilities are calibrated on the training mixes, not per deployment; re-check thresholds on your own data.