Llama-3.2-3B GNU Prolog LoRA — AutoScientist Challenge (Math & Code)
LoRA adapter trained with Adaption Labs' AutoScientist on ~1,100
execution-verified GNU Prolog tasks (job 234f2cfb-00ef-46b7-8b61-f4870cf1b919).
Executable pass@1 on a frozen 175-task held-out slice (greedy, fp16):
| arm | pass@1 |
|---|---|
| base Llama-3.2-3B-Instruct | 24/175 (13.7%) |
| base + full rulebook in prompt | 23/175 (13.1%) |
| this adapter | 133/175 (76.0%) |
Platform judge head-to-head: 108 W / 0 L / 1 T of 109. Prompting the same rules moves the base model nothing; training on examples 5.5x's it. The deficit is distributional (Prolog is rare in pretraining), which is the one failure shape where adaptation beats any prompt we could write.
Archive sha256 be6ba99cbe5417b407f8bbe21cc988b0adcf0d1993d045aab46665eeb2c8f469. Part of a 19-experiment pre-registered
investigation; ledger and receipts at
https://github.com/Aryagarg23/adaption-findings
- Downloads last month
- -
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for AryaGarg23/llama-3.2-3b-gnu-prolog-lora
Base model
meta-llama/Llama-3.2-3B-Instruct