GameAgent Qwen2.5-Coder-14B (merged, full precision)

Fine-tuned via LoRA (rank 16) on Qwen2.5-Coder-14B-Instruct to act as a smolagents.CodeAgent-compatible assistant for writing Pygame games through an iterative Thought/Code/Observation loop. Trained via Hugging Face AutoTrain on real agentic trajectories (see theoracle/gameagent-pygame-trajectories) covering simple 2D games (Snake, Catcher, Pong, platformers, etc.).

This repo contains the merged full-precision (bf16) weights — the LoRA adapter (theoracle/autotrain-3tx0q-8ouy5) already baked into the base model, so it loads directly with transformers and no peft dependency.

Training params (from AutoTrain): lora_r: 16, lora_alpha: 32, lora_dropout: 0.05, target_modules: all-linear, quantization: int4 at training time, block_size: 1024, model_max_length: 8192, 3 epochs.

Usage

Works best inside the smolagents.CodeAgent framework (code_block_tags="markdown"), given a task that describes the sandbox's constraints (no global/nonlocal, don't invoke the real game loop mid-step, avoid double-escaping the final newline join, etc.) — the model was trained specifically on that agentic format, not general chat.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("theoracle/gameagent-qwen2.5-coder-14b-merged")
model = AutoModelForCausalLM.from_pretrained(
    "theoracle/gameagent-qwen2.5-coder-14b-merged",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

Known limitations

Trained with an 8192-token context cap — quality degrades once a multi-step agentic conversation grows much longer than that.
Occasionally reproduces a specific self-inflicted bug: escaping the final-answer join as '\\n'.join(lines) instead of '\n'.join(lines), which produces a syntactically broken (but usually logically correct) file. Worth adding an automatic un-escape check downstream of final_answer().
Informal spot-checks (not a rigorous benchmark): clean multi-step successes on Snake, Catcher, and Pong; a scoped-down single-screen platformer succeeded but hit the escaping bug above. More open-ended/complex tasks (e.g. full scrolling platformers) can push the conversation past the training context length and degrade in quality.
A Q4_K_M GGUF quantization of this same merge was also tested and showed real, measurable quality loss compared to this full-precision version — prefer this bf16 version where hardware allows.