Qwen2.5-3B · Interphyre · GRPO · Two-Body-Problem · Skilled

Fine-tuned from Qwen/Qwen2.5-3B-Instruct using GRPO (Group Relative Policy Optimization) on the Interphyre physics-puzzle environment, level two_body_problem. A static skill bank is prepended to the system prompt at data-generation time and remains fixed during GRPO training.

Training Configuration

Field	Value
Base model	Qwen/Qwen2.5-3B-Instruct
Fine-tuning method	GRPO
Level	`two_body_problem`
Variant	Skilled
Training seeds	1–50
Val seeds	51–100
Checkpoint	global_step_50 (epoch 1)
Batch size	1 seed / step
Rollouts per step	n=4
Learning rate	1 × 10⁻⁶
Max prompt length	3072 tokens
Max turns per episode	25
Hardware	1 × A100 80 GB
Reward	Binary {0, 1} — environment success predicate
Observation masking	Yes (`mask_observations=True`)

Evaluation Results

Val success rate (greedy @1, 50 val seeds, two_body_problem) at epoch 1 (50 seeds seen):

Checkpoint	Val success rate
global_step_50 (epoch 1)	0%

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("vgandhi13/Qwen2.5-3B-Interphyre-GRPO-Skilled-TwoBodyProblem")
model = AutoModelForCausalLM.from_pretrained("vgandhi13/Qwen2.5-3B-Interphyre-GRPO-Skilled-TwoBodyProblem", device_map="auto")

This model operates inside the Interphyre ReAct tool loop. It expects a system prompt describing the physics level and available tools, then iteratively calls simulator tools to place a ball that satisfies the level goal.