countdown-rl-gemma4-e4b-v1-s0

LoRA adapter (r = 16, alpha = 32) for google/gemma-4-E4B (text decoder), trained with GRPO on the Countdown numbers game with a Python tool in the loop: recipe v1, from-scratch replay, seed 0. The policy writes <think>, may run <python> (its stdout comes back in <output>), and answers in <answer>. Training code, configs, logs and the write-up: the accompanying repository (configs/experiments/gemma4-e4b-v1-s0/, REPORT.md, EXPERIMENTS.md).

holdout3 (1,000 problems generated after all model decisions, greedy, 768 tokens, two code rounds): 30.9% pass@1 (easy 58.2%, medium 19.0%, hard 6.3%).

Trained with the fewshot_multi_retry prompt; evaluate with the same prompt, since the policies are prompt-specific. Bucket run ids: g4e4b-a..d.

The adapter targets the text decoder of Gemma-4-E4B, extracted from the multimodal checkpoint. Create it first (logits match the full model exactly with eager attention), and use --attn eager:

uv run python scripts/prepare_gemma4_text.py --model google/gemma-4-E4B --out models/gemma-4-E4B-text

Usage

hf download prada5023/countdown-rl-gemma4-e4b-v1-s0 --local-dir adapters/gemma4-e4b-v1-s0
uv run countdown-eval --model models/gemma-4-E4B-text --adapter adapters/gemma4-e4b-v1-s0 --eval-file data/holdout3.jsonl \
    --prompt-style fewshot_multi_retry --max-new-tokens 768 --max-rounds 2 --attn eager