HeXA
Collection
6 items • Updated
Qwen2.5-3B · Interphyre · GRPO (Evolving Skills) · Two-Body-Problem
Fine-tuned from Qwen/Qwen2.5-3B-Instruct using
GRPO with an evolving skill bank on the
Interphyre physics-puzzle environment,
level two_body_problem.
The skill bank is updated during training and injected into the model's context, combining gradient-based policy optimization with in-context skill accumulation.
Training Configuration
| Field | Value |
|---|---|
| Base model | Qwen/Qwen2.5-3B-Instruct |
| Fine-tuning method | GRPO + evolving skill bank |
| Level | two_body_problem |
| Training seeds | 1–50 |
| Val seeds | 51–100 |
| Checkpoint | global_step_50 (epoch 1) |
| Batch size | 1 seed / step |
| Rollouts per step | n=4 |
| Learning rate | 1 × 10⁻⁶ |
| Max turns per episode | 25 |
| Hardware | 1 × A100 80 GB |
| Reward | Binary {0, 1} — environment success predicate |
| Observation masking | Yes (mask_observations=True) |
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("vgandhi13/Qwen2.5-3B-Interphyre-Evolving-TwoBodyProblem")
model = AutoModelForCausalLM.from_pretrained("vgandhi13/Qwen2.5-3B-Interphyre-Evolving-TwoBodyProblem", device_map="auto")
This model operates inside the Interphyre ReAct tool loop. At inference time, the evolved skill bank should be prepended to the system prompt.
- Downloads last month
- 6