lamamkh/PhysAgentGym-Qwen2.5-Coder-7B

PhysAgentGym adapter for Qwen2.5-Coder-7B-Instruct

A QLoRA adapter from the NeurIPS 2026 paper PhysAgentGym: Free Physical-Law Verifiers for Training Small Code-Reasoning Agents. The model reads a physics scenario, writes Python that simulates it, and refines the code using feedback from physical-law checks.

• Project page: https://lamawmouk.github.io/PhysAgentGym/
• Code: https://github.com/lamawmouk/PhysAgentGym

Training

Supervised fine-tuning on 491 agent rollouts from Claude Sonnet 4.5 that pass every physics check and score at least 0.85 (data/sft/all_dense.jsonl in the code repository), with seed 42. The base model is loaded in 4-bit NF4, and the adapter uses LoRA rank 16, alpha 32 and dropout 0.05 on all attention and MLP projections, trained for 3 epochs at learning rate 2e-4 with scripts/train_sft.py.

This checkpoint was retrained with the same code, data and seed as the paper's main 7B run, on an NVIDIA A100. GPU training is not bit-for-bit reproducible, so its scores differ slightly from Table 2 of the paper, where the aggregate is 0.942.

Results

Mean verifier score on the 120 test tasks, 30 per subcategory, with failed episodes scored 0 and a 95% bootstrap interval on the aggregate.

PE-KE	Linear	Circular	Damping	Aggregate [95% CI]
0.941	0.971	0.911	0.920	0.936 [0.919, 0.950]

PE-KE and Linear are in the training distribution. Circular is near out of distribution and Damping is far out of distribution, since neither appears in training.

Usage

With the PhysAgentGym code:

hf download lamamkh/PhysAgentGym-Qwen2.5-Coder-7B --local-dir checkpoints/qwen-coder-7b-physagent
python scripts/rollout.py --task damping --provider hf \
    --model Qwen/Qwen2.5-Coder-7B-Instruct \
    --adapter_path checkpoints/qwen-coder-7b-physagent \
    --mode test --n_tasks 30 --seed 42

With PEFT directly:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct", device_map="auto")
model = PeftModel.from_pretrained(base, "lamamkh/PhysAgentGym-Qwen2.5-Coder-7B")
tokenizer = AutoTokenizer.from_pretrained("lamamkh/PhysAgentGym-Qwen2.5-Coder-7B")

The adapter is trained for the PhysAgentGym agent loop and its action format, on single-body classical mechanics.

License

Apache 2.0, the same as the base model.

Citation

@inproceedings{moukheiber2026physagentgym,
  title     = {PhysAgentGym: Free Physical-Law Verifiers for Training
               Small Code-Reasoning Agents},
  author    = {Moukheiber, Lama},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2026}
}