DeliVali/AIDAM_CODING_1.5B_V0.1.0

AIDAM coder expert — Qwen2.5-Coder-1.5B + RLVR (GRPO)

The code expert of AIDAM's cartridge pool, by Jeffrey Romero Del Val. Trained with RLVR — reinforcement learning on a verifiable reward, no imitation data: each attempt is scored by the SAME deterministic arbiter AIDAM uses in production (0.7·fraction of tests passing in a bubblewrap-confined sandbox + 0.2·format + 0.1·compiles), and GRPO pushes toward the better attempts of each group. 300 steps, QLoRA 4-bit + LoRA r=16 on attention, on a single RTX 5070 (12 GB). The LoRA is merged into the base; a Q4_K_M GGUF ships for llama.cpp runtimes.

Benchmark (honest, measured; details in the repo ROADMAP)

Harness	Base	+ RLVR	Δ
MBPP-sanitized test, n=200 (seed 7), greedy, ALL asserts must pass in the confined sandbox, tests shown in prompt	61.0% (122/200)	63.0% (126/200)	+2.0pp

Read this plainly: the margin is +4 problems out of 200 — real under a pre-registered gate (adapter must beat base on the same fixed subset, same judge as training), but modest. Our harness is stricter than the usual MBPP setup (greedy decoding, Spanish system prompt, full test-list pass required, sandboxed execution), which is why the base reads 61.0% here vs ~69% on its own card.

Promotion discipline: the round-1 adapter of this same expert was held back because the arena was flawed (the prompt omitted the function signature, so the harness measured name-guessing — base read 4.5%). The arena was fixed first, the sanity check re-passed, and only then was round 2 trained and gated. Both rounds are documented in docs/ROADMAP.md.

Training data

Prompts only — RLVR imitates nothing. MBPP-sanitized train split (120 problems, task + its asserts in the prompt); the gold key (the asserts) is used exclusively by the reward arbiter. Measured on the test split: no contamination.

Use in AIDAM

Drops into the cartridge pool at models/qwen2.5-coder-1.5b/ — the router sends programming tasks to it, one expert resident at a time, and every answer still passes AIDAM's verification core. It is an expert, not an oracle: the sandbox judges its code at runtime too.

The AIDAM model family — one verifier per register

Model	Register it judges	Backbone	Headline number
AIDAM_VNEWS_308M_V0.1.0	news summaries, grounding	mmBERT 308M	LLM-AggreFact 71.0
AIDAM_VWIKI_279M_V0.1.0	encyclopedic claims	mDeBERTa-v3 279M	FEVER 86.0
AIDAM_VSCI_279M_V0.1.0	scientific writing	mDeBERTa-v3 279M	SciFact 66.3
AIDAM_VREAL_279M_V0.1.0	real-world and viral claims	mDeBERTa-v3 279M	AVeriTeC-500 62.6
AIDAM_CODING_1.5B_V0.1.0	(not a verifier — code expert)	Qwen2.5-Coder 1.5B	MBPP-test 63.0%

Why five verifiers and not one. A register is the dialect a text is written in: a news summary, a scientific abstract, a Wikipedia article and a viral post state facts in structurally different ways. Training one small verifier on a new register measurably costs it capacity in the others — 30k encyclopedic-register pairs pulled the model toward that style and its score on viral claims fell from 62.6 to 60.2. At 279M capacity the AVeriTeC-optimal and the grounding-optimal models had genuinely diverged into different checkpoints.

So each model above is the champion of one register and is beaten on the others by its own siblings. That is not a hedge; it is the measurement. In AIDAM they are selected per claim and several are consulted at once, because betting on a single routing choice would produce a confident wrong verdict.

A specialist's score counts only on the benchmark it was built for. Quoting the SciFact number as if it described general capability would be the same self-deception as moving a gate after seeing the result.

Versioning. AIDAM_<SPECIALISATION>_<PARAMETERS>_<VERSION>, and the parameter count is the real one. These models are V0.1.0: the research era, the line that produced the measurements above. V1.0.0 is reserved for the models the project is about to train, which do not exist yet — so a name never designates two artefacts at once. Earlier names still redirect here.